Is it rusted on (which is a bad thing given the location of tail lamps) or is it just really tight?

Eitherway spray some brake cleaner on it.