It could sound stupid to ask this, but does it happen in any road? If a wheel is unbalanced, at low speed is not noticeable, but the higher the speed, the higher the effect.
In your case, the effect dissapears when speed increases. Some roads have slight bumps, and at low speed the bike cath them all. When the speed increases, the bike cross all of them in the crests, so you don't notice so many bumps.