It sounds like either a leak in your evaporator, or the connection to it. It could be a blown o-ring, but I bet there are other reasons besides outside temp.
Contracting o-rings during the winter? That's new one on me. O-rings are in compression when installed, so you'd need to have crummy ones on there for them to leak.
BTW, has your car been converted to R134a from R12?