The fact that you shouldn't hold the key and let your car crank and crank aside for a minute, perhaps your alternator isn't making the power like it should. I haven't purchased one in a while but I guess you can still get them remanufactured. Those are notorious for not lasting long. It takes a lot of amps to crank the engine in a car and if your starter is good, your batt sounds like its not up to the job, which could be the batt or the alternator that keeps it charged.
Also inspect the large cables that run off of your battery terminals and down to the starter, plus the large negative ground cable that mounts to the battery and car. Those cables wear out over time and can cause some weird issues with starting. Prior to trying to start the car does your radio, wipers, blower, etc. work fine?
Avoid wearing out your starter cranking and cranking. If no sign of combustion when you turn it over a few times then there could be other issues, too. If it is a bitter cold weather issue, I would suspect the Ignitor, sometimes called Ignition Module. I had a cold weather starting problem for two Winters until I finally figured out what was going on. Replaced distributor (cheaper than replacing ignitor) and she began starting up like new.