The manifold gasket couldn't be the problem. If it were leaking badly, you would have a bad idle at best and a massive vacuum leak causing the rpms to climb to redline when the car is started at worst.

Look at the wastegate and the turbo. It sounds like either of those could be the problem.