I had a very similar set of symptoms with an '83 goldwing. After rebuilding the starter (twice) and using a better battery, the problem was traced to a poor ground contact between the starter body and the engine block. Cleaned the ccntact point with sandpaper and the problem went away. It seems when the parts heated up and expanded the contact wasn't good enough, the starter would turn very slowly if at all.
I suggest you pull the starter and clean the bottom boss where it bolts to the engine block, then check/ clean the ground strap at the frame contact point too.