Alright, a fresh nights sleep did wonders. I was able to use some vice grips and a small dowell the size of the bushing to push it back out. It still will not easily slide into the mount however. I've shaved down a little brass on the bearing, but this hasn't helped. It's in the freezer now. Hopefully this will be the extra hundredth of an inch I need to make it fit.