And expanding on the controlled environment aspect...

When you street race, there are too many other variables besides just driver's ability to control their car. Drugs, alcohol, unsafe cars, are just some of the additional elements. Violence is also a possibility (I mean these aren't exactly the most reputable people out there). Tracks have a vested interest in keeping drivers and spectators safe (ie insurance premiums and lawsuits), so for your own safety it's probably best to go to a track.

So the answer to your question is that it absolutely should be illegal. It's not so much as racing is unsafe as the ramifications of street racing are much more far reaching than racing at a track.