It seems to me that if you messed up your frame your car would turn to the left the same amount no matter what (assuming you're keeping the wheel perfectly straight the whole time). Accelerating would only make you turn faster, not sharper.
Also not sure about the caliper being the issue. I'm sure if it was hanging up it could create some pull but not to the effect it is doing with you. And if it was dragging bad, wouldn't there possibly be some smell or smoke from the continuous friction?
I'm just shooting ideas out my butt although the only thing I can suggest is to take care of the known problems so you're left with nothing but that one.