If you would like your petcock to actually stop the flow of gas in the future, definitely take it off. Must for Rust will eat it up in no time, ask me how I know... There's a ton of threads on cleaning out tanks, Phos. Acid (M4R), Molasses, etc. Do a search for some ideas.
The stronger the soultion, the quicker the flash rust after it's cleaned is what I've found. Have some Acetone handy to wring out and latent water after the final flush. Then either fill it with gas ASAP or swish some diesel in it to protect.