Without that pilot air jet, the pilot circuit will run way lean, and may not allow the cylinder to fire at all. You should be able to tell that the head pipe for that cylinder was a lot cooler. A 4 cylinder bike running on only 3 cylinders, doesn't run well. A warmed up bike can probably stand a little leaner mixtures and still fire. As you open the throttle other carb circuits dominate fuel mixture. So, the pilot circuit has less effect at larger throttle openings. The way the engine uses fuel is different cold from hot, too.
I can't say it is THE reason. But, it wouldn't surprise me if that were at least part of the reason. Lot's of things can make the bike run poorly.
Cheers,