It's very difficult if you don't. You need to heat the entire housing equally in order to get the lense off. You might be able to use a heat gun, but I'd be afraid of melting or warping something...