Are all your tires the same make/model/size? I've experienced a similar problem in someone else's car that had mis-matched tires. If not, then it sounds to me like something is loose in your suspension or steering as Bakeoff suggested. Toe problems can cause what you described, and a loose steering rack or bad tie rod would cause that. It won't be drivetrain or engine related since it doens't vary by engine RPMs.