Make sure you have the "nut" in correctly too. That's the one in his hand in the prior pic. I put mine on with the flats on the outside once, and it didn't sit in the forks correctly.
Camelman
yes be sure the hex goes to the inside. Also be sure you engage the speedo transmission into the speedo drive plate correctly. that won't cause the problem you have, but just be mindful. Two dogs in the transmission mesh with the two notches in the drive plate.
Other than that there is only one spacer, so its likely the nut. The axle butt goes on the speedo drive side, the spacer and nut on the opposite side.