OK well im not an expert but i have some suggestions. First is to run 4 guage to a fused block then run the two 8 guages out of the fuse block into the amps. You need to have the fuses to protect the 8 guage wire coming out of the fuse block because if there is a spike the 4 guage may handle it fine but a fuse will protect the 8 guage from problems. This will also give you the ability to go to bigger amps. Second is that it really seems like the sub is underpowered. It may be when your turning it up the sub amp is begining to hit its max and will no longer get louder.

Just to clarify when you turn the vol up on the deck do both the components and sub come up at the right volume? Or are you saying when you turn the gains up on the amps then you are having the problem? Have you checked to make sure your grounds are tight and on bare metal?