Looks like I might be able to replace that shaft with the engine in the bike. If I hold the cam follower with some needle nose pliers, remove the speedo housing cover and slide the shaft out the side, replace and reassemble. I don't see anything on the shaft I would loose other than needing to guide the shaft back into the follower.
Thoughts?