The 400F muffler is prone to rusting out badly from the inside. It could be that yours is partially clogged. Any small muffler (check that the ID on the new muffler matches the size of the pipe on the end of the header) shouold work OK. You might have to rejet the carbs if you run a less restrictive muffler. In fact, that may be why the bike seems to run better with the muffler off. Is the jetting in the carbs stock now, or are the main jets bigger? Stock is 75 mains if I remember correctly.
Don't have any idea if the GS550 header will fit. The header pipes on the 400F come straight out of the head, as opposed to the 750 that comes out at different angles. So if the distance between the header pipes matches up, you might get lucky.
Let the rest of us 400F owners know if the GS pipes fit.
Tracy