There is a lot of pressure between seal and piston that squeezes most of the lube out during assembly. On the fluid side, some of it is dissolved by the brake fluid itself. At any rate the seal must provide a sliding surface if it is to compensate for brake pad wear. So over time, the assembly "lube" slowly marches out as the piston does. Even with the lube, there is still purchase by the seal onto the piston for retraction. I've done several, it works fine. No dragging brakes. Far more important is the piston surface condition, the caliper groove condition, and having a compliant seal. Compare the resiliency of a new seal with one that's 25 years old. If it feels the same, then reuse the old one. If it feels hardened, it probably won't distort when the piston moves and will slide instead, that will certainly effect retraction.
Are you trying so hard to talk yourself out of using assembly lube because it is difficult to find? Have you seen the lube? It has only slightly thicker viscosity than Dot3. It isn't THAT much more slippery than Dot3. It's main benefit is that it doesn't harden, evaporate or phase change with age, and is not hygroscopic, providing a barrier to water and oxygen. The bottle I have says it is polyalkylene glycol and inhibitor, if you are keen to look up chemical properties. By the way, DON'T use this with silicone brake fluids.
You're right though, we all make our own choices. I've got brake parts assembly lube, and I use it on my caliper seals. Good luck with your choice!