You can avoid using it to:

1) Waste more gas.
2) Kill your engine faster.

Imagine driving without overdrive being like driving around in second gear, just not quite as bad. There is no benifit to staying in the lower gears unless you are at the track.