bubblez... I'm assuming the oil was above the threads on your spark plug? Tube seals are the thing. Leaking valve guides, stem seals, or rings would get oil on the firing tip of the spark plug. If the oil is outside, then after a couple weeks simply pull off the plug wire & look down into the hole. No need to remove the plug, just look for oil.

jmg... Make sure you got the right part. You SHOULD feel the plunger rattling on it's spring. Actually, if the old one moves around properly, put it back in. Is it possible they gave you a simple hose barb instead of a PCV valve?