What it is happening is that either the heatshield on the exhaust is rattling, or the A-pipe in the header is rattling... Neither one will kill your car, and both are fixable...
If it's the heatshield... Then go ahead either bend it so it isn't vibrating or remove it entirely... it wont hurt unless you plan to periodically park your car in tall grasses...
If it's the header (What happened to my car) then you can either replace it, or have someone wield it back together on the inside... I chose to just replace it with an aftermarket header... worked like a charm!