Sounds like you've ruled out all the obvious things. Imprinting pad material onto the rotor would go away when you replaced the pads & rotors. Unless your driving habits put them right back into that condition...

How about loose ball joints (including the little ones at the tie-rod ends)?