I've replaced my rear speakers by going through the speaker grill only. The only thing you have to worry about is trying not to drop the screws.
If you can do the same for the front, I'd advise it since I've always had a bit of a hard time with re-installing the door panels.