Residue from cleaning solutions and the oils from hand prints, while virtually invisible, are enough to retard adhesion of your paint. Sand it back down, clean and degrease it properly, handle the piece with clean gloves, and then do several light coats, allowing just the flash time (from the can instructions) between coats, and then let dry for a couple of days.