There is a spanish proverb that goes in the line of "as long as I'm warm, I don't care if the people laugh"
That being said, I would not wear chaps if there are other options available, but only because I don't like them, not because everybody else doesn't like them. What I do in winter is to wear long cotton underwear. It really makes a difference and you don't go around looking like a rough rider.