Back to my original question:
the guides are out of spec, yes, but how much of an issue are exhaust valves with a little too much deflection? What's the downside/risk?
They all seem evenly worn regardless of the valves' condition, so I think it's safe to conclude this is typical wear and tear over time from being ridden, rather than acute damage from the cam sprocket coming loose.
If, worst case, it's going to steal a tiny bit of compression at high RPMs, I'm not going to invest more time/money to get the valve guides perfect. I take pride in my work and I enjoy riding fast bikes, but I'm not building a racing machine.
If it's risking further component damage or will make the engine unreliable, then I'd be concerned enough to invest the extra time and money, even if I have to wait to finish it.
1975 CB750F
1986 CN250
2005 KLR650