As Ted said on the UK Forum, just put a cable tie through, do it up and use it as a pulling strap. No need to remove it, it's not in the way and isn't visible when the seat is on. I've never had the strength in my fingers to pull that rubber bit back, so it's always been the cable tie method for me.