The theory behind that is that there needs to be pull on the engine to build boost (hence why you can't build boost in neutral). With less inertia there is less pull on the engine, resulting in difficulty to build boost.