Sometimes, around 3.5K, some GS-Rs develop a buzzing noise that comes from faulty welding in the A-pipe of the exhaust manifold. It only appears for about a 400RPM band and after that, disappears as if it never existed. Sounds a lot like you have a broken heat shield, but a bit louder. Might wanna check that out.

It happened to my car

If it's that, there's no real way to fix it besides replacement... Well, I'm sure there's someone out there who could do it, but an aftermarket DC header wont cost you much, and give you _some_ power gains and a nice little weight reduction aswell.