Yeah man, I would have definitely said something to the lead rider when that happened. It's the leader's responsibility to set a comfortable pace for the entire group, and to clearly signal the group's direction in upcoming intersections or stops.
Looks like he failed both of those duties. I would have said F it and rode home after some choice words with the lead rider.