Ok, if you have not done it yet.the BEST thing to use is lighter fluid. it will also help remove pin stripes but after ward you need to was the spot with car wash soap realy well, EXPECIALY if your goign to paint it. The lighter fluid wont hurt the original paint so if you dont wash it well, it will still be ok.
Also for futuer referance. I dont know how you removed the badge, but a good strong fishing line or nylonbe thread works well to cut the badge from the car, then you remove the glue/tape whatever.
Good luck.