You say either the engine or the starter kept going when you took the key out. If you can't tell which by the sound, then you better ask for help.

If the starter solenoid was stuck, that would do it. Get another starter.

Or it might be a flaky ignition switch?? You'll want to do some tests with a voltmeter to find out which.