My '92 civic has a similar problem and I've traced it to the HVAC unit itself.

I can actually press hard on the face of the unit right below the recirculate/fresh air buttons (or on one of them) and it will work without fail- sometimes it will then stay on while others you have to keep pressing on that spot.

I thought it was a relay issue at first and then the a/c button itself but nope. When it's real hot outside is when you normally have to keep pressure on it but it's random, sometimes in the cold it won't work either.

I believe it's either loose connectors or, more likely, that the solder joints have cracks in them. Similar to what causes the main/fuel relay to act up on our cars. I haven't gotten around to taking it apart yet but I'll buy a soldering iron and do the work myself before I pay $300 for a new unit.

The "press-location" most likely wouldn't apply to your accord but I wouldn't be surprised if it was the same issue - just throwing it out there.