If the pipe becomes too big, you begin to lose velocity and it loses the vacuum effect of pulling gases from the manifold. This would be beneficial in higher RPM however as now you have the diameter piping to flow more gases. Likewise, you want some backpressure to keep from the exhaust gases from being expelled too quickly out of the engine, which reduces overlap essentially and you end up losing bottom end torque.