I just drilled out that piece.
You could probably do it by hand, but I have a drill press with tilt table and I cobbled something up with a drill press vise and C clamps to get the angle right.
Shortening the banjo would certainly be easier.
You won't ever be able to go back to a hard line without the flare insert, for sure.