If you disconnect the battery and it is still there, removing the fuse won't do anything. Fix the problem and disconnect the battery and it should go away. Disconnecting the battery is instantanous whereas the fuse takes a few seconds.