put a finger in there to feel the edge of the spacer is centered on the inner race.
If youre sure the un-threaded portion of the axle fits into each bearing, then sure, id reach for a wood block and tap a bit till it runs through.
Just check that the threaded portion is centered best it can be, before tapping.