There's actualy a metal sleeve in that bushing. The bolt seizes inside the sleeve as Andy mentioned. If you heat it up, the rubber may melt, and the sleeve can come loose in the bushing. That still won't help you because the sleeve will still be rusted to the bolt on the other side of the hole.

Don't beat yourself up over it. It's actually a really common problem on higher mileage cars or older cars.