I just did this chore this past Saturday (for the first time). When I put the put the piston back in, it was a tight fit. I had to blow into the brake line hole to get it to pop out a little, then wrestled it out with my fingers (I would recommend compressed air, brake parts cleaner tastes like sh!t). The piston definitely would not fall out just with the force of gravity. Also, I lubed the O-ring with anti-sieze lubricant.
My caliper also required lightly sanding some rust off of where the disc fit into the caliper. When I was done with this, the disk would slowly fall out with just the force of gravity when I turned it over. I also put anti-sieze here too.
Post repair report: So far, it's fully functioning with no leaks, squeaks or binding. I rode it around for a little while yesterday. Before the repair, when put the brake on, the brake would continue to drag after I let off the lever.