I pulled the solenoid out of the starter, cleaned it up and so far haven't had the problem either.

I mainly just sanded down the little bronze/gold/copper piece on the solenoid and sprayed it with WD-40. Dried it off and popped it back in there. Starts with no problem now... or so it seems since the problem didn't happen all the time.