It is often difficult to distinguish strangers’ permanent facial shapes from their transient facial expressions, for example, whether they are scowling or have narrow-set eyes. Overinterpretation of ambiguous cues may contribute to the rapid character judgments we make about others. Someone with narrow eyes might be judged untrustworthy, because of strong associations between facial anger and threat. To test this hypothesis, we investigated the trait judgments made by individuals with severe alexithymia, associated with impaired recognition of facial emotion. Consistent with the hypothesis, alexithymic participants demonstrated reduced interrater consistency when judging the character traits of unfamiliar faces, and the presence of subtle emotions. Nevertheless, where alexithymics perceived, or misperceived, emotion cues, the character traits inferred thereafter were broadly typical. The finding that individuals with developmental deficits of emotion recognition exhibit atypical attribution of character traits, confirms the hypothesis that emotion-recognition mechanisms play a causal role in character judgments.