You have it reversed: tall gear == slower spinning, harder, flat to downhill. Small gear == faster spinning, easier, hills, upwind.
The tall vs. small refers to the length traveled on a single crank revolution: big ring small cog this becomes tall, small ring big cog this becomes small.
Source: thousands of miles ridden and years of racing road, MTB and cyclo-cross.