I get this a lot as well. Happens to me a few times a month. I just ignore them and keep driving. It's not worth any kind of risk, especially with my wife and kids at home. You did the right thing as to not race. I haven't street raced or been to a street race in over 4 years now