Try using a booster pack or jumper cables to bypass the starting circuit (make sure its out of gear). That would rule out problems with the starter. I"m betting the problem is with one of the connections and not the starter. It starts after you removed the starter because you also moved the cables/connections in the process.
My 450 did that for a while, I learned I could wiggle the cables and it would take off. My fix was new cables and new crimp on connectors.