if you are putting in new speakers do you really want the old grills????? and yes you would have to take off the panels as the speakers are screwed into the door at the back of the panel.