Don't reccommend pounding them on, the shoes need to pivot on this pin. Wipe the grease off and take some fine steel wool or fine emery cloth to the stubborn pivot pin. Shouldn't take much, the fit is close so any corrosion would likely be the culprit.
Corrosion is common here due to dissimilar metals, aluminum on steel.