Oh that is the top, isn't it...
I didn't grind any races off; of either the neck or the tube. I used a long drift on the opposite end of the tube. I may have used a pry bar first to work the race up a bit on the top part of the tube. I honestly can't remember that part and I didn't document it :/.
A dremel and chisel would totally work, but obviously there's a risk of damaging the race surfaces. If you accept those risks, then go for it. Although, I used the lower race on the tube to help press the new race on, so might not want to destroy it.
Do you have the bearings to put back in to see if this is the race or not? It really looks like it is.