Yes, below the bearing should be a dust cover, then one of the 2 spacers/washers. The bearing sets usually ship with 2 to choose from, one being thinner than the other.
You'll need to remove the bottom bearing then install that spacer, then the dust cover, then bearing. This also adds enough length overall that when tightening the stem, the bearings get compressed sufficiently to seat them and hold the stem.