A turbo is exhaust-driven. The turbine relys on these exhaust gases to 'spool' up.'

In my opinion, a 3" exhaust will still see gains with a little turbo/low boost. It sees much more power at higher boost/larger turbo applications due to reduced backpressure; thus, enabling more exhaust to flow per unit of time.