When you can't afford to buy new jeans after the oil ruins the ones you have.
Seriously, you can ride it as long as you have compression. Ride it until the off season or until it stops running, whichever comes first. Buy a compression tester, if you want, to monitor the situation.