Some dumb ideas...

Clutch throw-out bearing. When its unloaded its making a loose rattling sound. When you press the clutch pedal you're pushing against the T-O bearing & it immediately quiets down. When you're driving (without your foot on the clutch) the noise is covered up by all the other noises...

Transmission bearing(s) - input shaft or ?? When the input shaft is spinning one of its bearings makes noise. Or maybe a synchronizer? When you disengage the clutch the input shaft takes a few seconds to coast to a stop, so the noise takes a second or 2 to quit.

Cross your fingers & hope its the T-O bearing cause that's easier. (cheaper) Maybe you can put a piece of hose up to your ear & narrow down the location of the noise?

PS... As soon as I posted this I saw DelSolSIinMD's note. I've heard T-O bearings go either way. Sometimes they get loose & rattle when they're NOT loaded. Sometimes they make grinding noises when they are loaded. The only one I had myself was loose & rattled when it was not loaded. The key is that the noise stopped IMMEDIATELY when I disengaged the clutch. Input shaft bearings should take a couple seconds to spin down.