I relentlessly beat the shit out of my Si because it's too slow, I can't imagine an HF.
A swap isn't going to hurt gas mileage much, a B-series would still get 25-30 mpg which isn't bad at all.