Mufflers have backpressure to gas flow. If you want all the cylinders to work in concert, they must be presented with the same gas flow conditions. Back pressure effects how the cylinder scavenges during the exhaust stroke, and thus, the mixture present for the subsequent firing stroke, and how much of it is burned and how hot the exhaust gasses are. (assumes your backfire is in the exhaust).
If you have a 4 horse team and two of them are lame, would you expect the team to travel smoothly?
Anyway, you haven't said where the backfire is occurring, the intake or the exhaust?
O rings for the main jet should fit tightly. I usually have to use grease or silicone lube to wedge them into the recess in the towers.
Try to dig your thumbnail into the rubber, it should be soft. If it isn't, they aren't any good.