your problem sounds like the EGR is not closing once it has opened. Be more specific about you lifting up the EGR diaphram. When you lift it up to open the EGR valve, does it idle rough or nothing happens? Heres what will happen. At idle th EGR is not suppose to open. It is suppose to open at part throttle. You state that when you lift it up and let it go the idle returns to normal and then idle resumes to fluctuation after you touch the throttle. I would have to say it's your EGR sytem or the EGR valve itself is being held open by carbon build up. Heres something you can do, let the car warm up at idle andfeel the metal piece of the EGR valve that is bolted to you intake manifold. When your idle is fluctuating, see if it's hot there or not. At idle it should not be hot, if it is youre EGR is stuck open.