I'd guess it's the rocker arms, or maybe even worn spots on the cam lobes; they may have some irregular wear in them from the valves being improperly adjusted by the previous owner. By the way, are you adjusting the valves with the motor warm or cold?