There’s fresh evidence that artificial intelligence software absorbs human biases about race and gender, and it may be due to the very structure of languages.
Scientists came to that conclusion after creating a statistical system for scoring the positive and negative connotations associated with words in AI-analyzed texts.
A similar system, known as the Implicit Association Test or IAT, has suggested that humans harbor biases about the comparative status of different races, as well as men and women, even though they don’t explicitly acknowledge them.
Princeton University’s Aylin Caliskan and her colleagues adapted the IAT for a textual analysis tool they call the Word-Embedding Association Test, or WEAT. They describe the method, and its application, in research published today by the journal Science.
WEAT looks at associations between a given word and the words surrounding them, then judges whether the embedded word has a pleasant or unpleasant connotation. For example, the analysis found that flowers are more pleasant than insects, and that musical instruments are more pleasant than weapons.
Then the researchers did a statistical analysis of names regarded as European-American vs. characteristic African-American names, using a body of text harvested from the internet that amounted to roughly 2.2 million words.
Without consulting a human, the WEAT analysis found that the European-American names were more likely to be associated with pleasant words, such as “happy” or “gift.”
On the gender front, researchers used a slightly different statistical tool called the Word-Embedding Factual Association Test, or WEFAT. They looked at the links between different types of words and male vs. female words (for example, “man” vs. “woman,” “he” vs. “she,” “boy” vs. “girl”).
The analysis showed that female words, and traditional names for women, were more likely to be associated with family-associated words such as “parents” and “wedding.” Male words and names had stronger associations with career-associated words such as “professional” and “salary.”
The researchers also drilled into different fields of work and study, and found that female words and names were more closely associated with the arts, while male-associated words were more closely associated with math and science.
One illustration involved Google Translate, as applied to automatic translation of Turkish phrases into English. The Turkish language has a gender-neutral, third-person pronoun (“o”). But when gender-neutral sentences were translated from Turkish (for example, “O bir doktor” and “O bir hemşire”), the English equivalents came out as “He is a doctor” and “She is a nurse.”
Outside experts on AI and bias said the research pointed to a new frontier for study.
“This paper reiterates the important point that machine learning methods are not ‘objective’ or ‘unbiased’ just because they rely on mathematics and algorithms,” Hanna Wallach, a senior researcher at Microsoft Research in New York City, said in a news release. “Rather, as long as they are trained using data from society and as long as society exhibits biases, these methods will likely reproduce these biases.”
University of Washington psychologist Anthony Greenwald, who helped develop the Implicit Association Test almost two decades ago, said in a Science commentary that the new findings suggest “language might be a source of the implicit biases that the IAT reveals.”
He said there could be alternate explanations – for example, the bias measured by both analytical tools may be due to social or cultural factors other than language. But in any case, Greenwald said the findings add to the challenge of “debiasing” the way humans, and artificial intelligence programs, process language:
“Computational debiasing necessarily entails some loss of meaning, and gender is just one dimension on which AI text analyses might be debiased. How much useful meaning may disappear in the process of debiasing simultaneously for the legally protected classes of race, skin color, religion, national origin, age, gender, pregnancy, family status and disability status? Hopefully, the task of debiasing AI judgments will be more tractable than the as-yet-unsolved task of debiasing human judgments.”
In addition to Caliskan, the authors of “Semantics Derived Automatically From Language Corpora Contain Human-like Biases” include Joanna Bryson and Arvind Narayanan.