You could burn a valve if running too lean but like was said above, you would need to read your plugs to see.
Here's my meat head take on exhaust tuning, a few select drunken flashbacks from college...
You've got atmospheric pressure outside the engine and a bunch more than atmospheric coming out your exhaust valve. Those gases want to get out to lower pressure, so removing the baffle will give them an easier path. You'll hear about a scavenging effect in 2 strokes, which kind of applies here. Those pulses move thru the header and leave behind a pocket of lower pressure. Some dude got all worked up about entropy in the 2nd law of thermodynamics, and this is an example.
The gases won't stay away from that lower pressure, so more exhaust moves in to fill that lower pressure pocket. You get too many big pulses sucking on the intake and that will confuse the #$%* out of your carbs. Having a baffle in there will lessen the severity of those pulses by introducing a restriction. I think the net result of removing the restriction would be that your peak torque and horsepower would move up the rpm range. That's all considering you're jetted to maintain a 14.7 (aka stoichiometric) air/fuel ratio. It's times like this when closed-loop fuel injection would be handy.
Bonus points for "entropy" and "stoichiometric" in the same post?