I'm similar to you, I had bad cat allergies when I was younger but they seem to have largely faded as I got older. I took a chance and lived with 2 cats (my old roomate's) for 2.5 years, and it went really well. If I rubbed my face in the cat's fur my eyes would bug me, but mostly it went fine.

Some ideas:
1)pick up some Claritin/Loratidine. This anti-histamine helped me out a lot the few times my allergies bother me, and it's available for cheap over the counter
2)rinse your hands after handling kitty. You don't even have to use soap, I found a quick rinse protected me if rubbed my eyes later.
3)keep kitty out of your bedroom until you're sure you won't react.
4)consider "renting" a friends cat for a week or two to see if your allergies flare up. There's a big difference between handling a cat for a few hours and having one stay in your place, where their hair, dander and salive will accumulate all over the place.