Word Embeddings (Also) Encode Human Personality Stereotypes

Oshin Agarwal, Funda Durupınar, Norman I. Badler, Ani Nenkova


Abstract
Word representations trained on text reproduce human implicit bias related to gender, race and age. Methods have been developed to remove such bias. Here, we present results that show that human stereotypes exist even for much more nuanced judgments such as personality, for a variety of person identities beyond the typically legally protected attributes and that these are similarly captured in word representations. Specifically, we collected human judgments about a person’s Big Five personality traits formed solely from information about the occupation, nationality or a common noun description of a hypothetical person. Analysis of the data reveals a large number of statistically significant stereotypes in people. We then demonstrate the bias captured in lexical representations is statistically significantly correlated with the documented human bias. Our results, showing bias for a large set of person descriptors for such nuanced traits put in doubt the feasibility of broadly and fairly applying debiasing methods and call for the development of new methods for auditing language technology systems and resources.
Anthology ID:
S19-1023
Volume:
Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Rada Mihalcea, Ekaterina Shutova, Lun-Wei Ku, Kilian Evang, Soujanya Poria
Venue:
*SEM
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
205–211
Language:
URL:
https://aclanthology.org/S19-1023
DOI:
10.18653/v1/S19-1023
Bibkey:
Cite (ACL):
Oshin Agarwal, Funda Durupınar, Norman I. Badler, and Ani Nenkova. 2019. Word Embeddings (Also) Encode Human Personality Stereotypes. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 205–211, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Word Embeddings (Also) Encode Human Personality Stereotypes (Agarwal et al., *SEM 2019)
Copy Citation:
PDF:
https://aclanthology.org/S19-1023.pdf
Code
 oagarwal/personality-bias