Here are the specifics:
1976 CB750 K
~45,000 mi
bronze bushing ID .847"
collar OD .844"
Well, umm...having a clearance of .003" between the collar and bushing is the "worn out" point for the pair.

Here's why: you must add the clearances of BOTH sides' bearings, then multiply by the length to the rear axle, to get the side-to-side play the swingarm will see at the rear tire. In this case, that is:
(0.003" + 0.003") * 18" = 0.108" (not quite 1/8")
That much 'slop' will cause the bike to do things like: when coming up from a corner, the bike will feel like it slid sideways toward the outward side of the turn, each time you turn. If you are in a right turn and add throttle, it won't affect much: if you are in a left turn and apply throttle, it will try to stand the bike back up a little by shortening the turn radius. In a striaght line, it makes the bike feel like it has a mind of its own, not holding a solid line.
Many people over the years have 'blamed' this (in the Hondas, Suzys, and others of the era) on "poor frame design", but in my experience it has always been the combination of solid bushing clearances and tapered steering head bearings that made the bike handle, or not. (After that, there's other things like suspension, but that's another topic.)

If you measure that worn end of the collar, you will find it egg-shaped or oval-shaped. They typically wear on the front or back of the collar, based on how it was located in the frame. Use a mic if possible, as calipers have too much "wiggle" to get a good measure on round stuff like these. A small bore gage will get that ID, too.
Here's the 'real' specs the arm should see:
Most 550 arms need 1.0435"-1.0438" OD, 0.8444"-0.8446" ID (bronze bushings).
The collar, new, is 0.8440"-0.8442" (from Honda). This means that a freshly-assembled arm should have less than 0.0012" worst-case clearance (both sides added) for solid performance.
I suspect the best you can hope for is a new collar?
I just ran out of the last batch I had made in January, have to find someone to make some more for me. Honda quit them (again) a couple of years ago, but maybe Yamiya still has one? The 750K/550K collar is identical and the 750F/550F collar likewise. The 2 are somewhat different in sizes (OD), though.