I dont think its the starter though since its only doing what its suppose to do when there is 12volts going to it. The problem is going to be a relay/ignition problem I believe.
EDIT: sorry misunderstood I think, rereading it you mention the starter is not retracting. Now that I read it paying attention this time, if the starter is just free spinning and not retracting then yes its definatey the starter when could probably be cleaned and lubed and be just fine. However if its not disengaging because its getting 12v still then my other suggestions stand.
"You can live your life in a thousand ways,
but it all comes down to that single day,
when you realize what you regret,
but you cant relaim and you cant forget."
----Trans Siberian Orchestra