It should eventually turn off after a minute or so, it takes longer after "spirited" driving. If it doesn't, then you probably have a problem with you thermoswitch. You could unplug your fan from everything, then hook it directly to a switch, that way you could turn the fan off and on whenever you want. I've seen plenty of people do that, especially those with excessive underhood temps.