Happens to my dad's car all the time. I fucking hate it.

Like you said, putting the panels back on the door is the worst part. I usually remove the speaker and try putting my hand through there. Sames me some time.