So after some internet investigation it looks like the all the cells in the battery are dead. So I need a new one.
Could this be my issue?
The low voltage can certainly be a run issue, as that is where power to spark the plugs originates. Not yet sure if your battery is bad. None of them have infinite capacity, so simply being drained is a possibility. These bikes won't charge the battery at idle. In fact, they will deplete at idle, and faster with the lights on. To have the bike fully charge a depleted battery will take about 10 hours at 4000-5000 RPM.
Best to have charger separate from the bike and follow the directions that come with the charger. The standard battery for a CB550 shouldn't really be charged faster than a 1.2 Amp rate (for 10-12 Hours). But, they can usually tolerate 2 amps if the temperatures are favorable.
If you notice lots of bubbling (outgasing) or the voltage go higher than 14.5 V discontinued charging.
Once you have a known good, fully charged battery in the bike, then do the RPM and voltage test again to determine if your charging system is healthy. The fact that revving the engine increased battery voltage some, is a good sign, both for battery and the charging system. Simply not enough data to assign fault yet.
You can turn off the headlight by removing the headlight fuse, at least during the troubleshooting phase. That should save 4-5 amps of battery drain and help restore the battery charge state.
Cheers,