The starter takes A LOT of power, the harness wiring or switches can not handle that much. There's a high power relay usually called the "solenoid switch" that actually switches the starter motor power: pushing the starter button powers the coil of this relay.
The relay contacts get crudded up, corroded, and burnt with use. They can eventually stick closed, keeping the starter running. The kill switch doesn't affect the starter, but switching the key off just disconnects power to the stuff that can power the coil - your power contacts are stuck together and don't care about the coil. Just a good whack to the relay will probably solve it in a pinch but it will do it again soon if not serviced or replaced.
The solenoid switch relay can usually be disassembled, cleaned, lubed, reassembled and work for many more years. Only if it's severely burnt will it need to be replaced. A good non-OEM (ie cheaper) one can be purchased at any lawn tractor or quad dealer, you just have to mount it.
