Yep. Sounds like the battery voltage is dropping below 10.0 volts while idling. This makes the coil's spark weak, which makes the symptoms you describe.
The alternators only make about 200 watts while idling, which is barely enough to run the ignition. If lights are used, the voltage falls while idling. It's supposed to "fill it back up" while above 2000 RPM.
Also, check your battery cables where they connect to the battery: they could be dirty with a white corrosion. This will resist the current flow into the battery from the alternator, slowing the charging process.