Cognitive dissonance is much more surprising than just plain weirdness.

For example, a businessman wearing fuzzy slippers at a board meeting is far more disconcerting than a guy in a clown suit walking down the street. Little things that are out of place or disconnected can distract or confuse us much more than the utterly fanciful or elaborate.