if the subs are SVC, one voice coil, you are giving the amp too big of a load. thats why is it getting hot.
you really need a bigger amp, or just run 1 sub right now and bridge the amp. this is if the sub is indeed SVC.
and the subs really has to be in the right box, or they will sound like crap. MAYBE this is the reason to why they are sounding so bad. how much bigger than specs are you talking about?
__________________
.