That shouldn't cause start failure... especially with a decent battery.

Make sure the amp itself is turning off when you turn the car off?
That happened to my dad's amp, it wasn't shutting off, so it was draining the battery.