transparentkingdom - Tumblr blog

transparentkingdom · 3 years

Text

IAT

My friend and I got into a discussion about six months ago regarding the implicit bias tests put out by Harvard and their validity. He questioned their validity and had some articles to bolster his argument, and I disagreed with him. I thought folks might be interested in a few of my points.

In response to this article - https://www.thecut.com/2017/01/psychologys-racism-measuring-tool-isnt-up-to-the-job.html

and this one - https://digest.bps.org.uk/2018/12/05/psychologys-favourite-tool-for-measuring-implicit-bias-is-still-mired-in-controversy/

Here is what I said:

“what's up man,

so i read the two iat articles you sent me and found them interesting - so cool to be in grad school and be exchanging ideas on all sorts of things. i did want to get back to you and say that i read those pieces and looked at several scientific articles too (mostly by john jost and collaborators who developed the iat, but also investigators not affiliated with them). i maintain my position from yesterday that the iat is rigorous and that its structural framework can get at implicit biases. further, i would argue that there is a lack of sound logical integrity as well as generally flawed reasoning in the critiques of the iat you sent me. i'd love to share some of these thoughts as well as some studies and meta-analyses (and brief thoughts on these too) that look at associations between implicit bias and behavioral outcomes. sorry about this long email and inconsistent punctuation haha, but here are some of my personal opinions.

addressing the article from the cut first: i admit that it looks like the developers of the iat exaggerated the predictive powers of the iat if they said that it can shed light on "unconscious endorsements" people make of certain groups. this article goes on to flesh out this position and discuss how it is familiarity with certain stereotypes rather than actual endorsements of these stereotypes that can cause, for example, activists to score as high on these tests as non-activists. here are some quotes i've bolded:

experimenters were able to easily induce what the IAT would interpret as “implicit bias” against Noffians simply by forming an association between them and downtroddenness in general."

and also "Andreychik and Gill found that for those students who endorsed external explanations for the plight of African-Americans or a novel group, or who were induced to do so, high IAT scores correlated with

greater

degrees of explicitly reported more compassion and empathy for those groups. For those who rejected, or were induced to reject, external explanations, the correlation was exactly reversed: High IAT scores predicted lower empathy and compassion. In other words, the IAT appeared to indicate very different things for people who did or didn’t accept external explanations for black people’s lower standing in society. This suggests that sometimes high IAT scores indicate that someone feels high degrees of empathy and compassion toward African-Americans, and believes that the group hasn’t been treated fairly. Now, it could be that such people

also

have high amounts of implicit bias, but it’s striking how easily IAT scores can be manipulated with interventions that don’t really have anything to do with implicit bias." "So the question of whether the IAT measures something that can be fairly called

animus

, in the sense of being a preference (in this case, an unconscious one) for one group over another, rather than familiarity with stereotypes, is

anything but

“ill-posed”. "

Blanton said that he has never seen a psychological instrument in which less statistical noise predictably biases the results upward or downward. “What should happen is that as you remove random noise, you just get a better estimate of [the thing being measured],” he explained. Blanton provided a surprising example of how this plays out in test sessions, according to his team’s math: If a race IAT test-taker is exactly 1 millisecond faster on each and every white/good as compared to black/bad trial, they “will get the most extreme label,” he said. That is, the test will tell them they are extremely implicitly biased despite their having exhibited almost zero bias in their actual performance. That’s an extreme example, of course, but Blanton says he’s confident this algorithmic quirk is “affecting real-world results,” and in the Assessment paper he and his colleagues published the results of a bunch of simulated IAT sessions which demonstrated as such."

"To be sure, there’s no perfect psychological instrument. They all have their flaws and shortcomings — sometimes maddening ones. But there may not be any instrument as popular and frequently used as the race IAT that is as riddled with uncertainty about what, exactly, it’s measuring, and with the sorts of methodological issues that in any other situations would cause an epidemic of arched eyebrows. “What I’ve been convinced of is it’s very difficult to break down the origins of these associations,” said Elizabeth Paluck, a prejudice and intergroup relations researcher at Princeton and a co-author on the “Noffians” study. “They can’t be all attributed to personal preference, they certainly come from cultural associations and conditioning.” As for the authors of the internal/external explanations paper, they note in it that “our analysis is perfectly compatible with the possibility that, perhaps for the majority of people, implicit negativity is likely to be prejudice-based.” But even if you accept that, it means for a substantial minority of people, the implicit negativity revealed by the IAT isn’t connected to prejudice — which is one reasonable way to interpret those underwhelming meta-analyses."

My contention with this part of the article is semantic in nature, because implicit bias IS familiarity and association between two things rather than any type of endorsement (e.g. if you grow up in the united states, even in the third millennium, you are likely to associate black people with violence and women with domestic life), which explains why openly hateful people and activists who spend a lot of time thinking about these associations might converge on the iat tests. It does not matter if your conscious or explicit biases are positive or how hard you work to fight your implicit biases (e.g. in the case of activists.) This article confuses explicit and implicit bias (probably in large part because the iat creators overestimated the predictive powers of the test as i mentioned and even made this semantic error themselves), but in reality, it is those implicit biases that predict how quickly a police officer will pull a trigger when startled by a black civilian who thrusts their hand in their pocket. explicit biases predict how well white people will get along with black people in intergroup settings because in those situations, you have time to reflect on your own prejudices (which the cut article even addresses and calls "overcompensating"). for more examples of quick reaction times in the context of implicit racial bias, i think blink by malcolm gladwell has a few good examples (though i'm guessing you've read it lol, and not that i am a huge lover of this book, because i'm not), as well as some of the articles i link in a few sentences. anecdotally (for what it's worth), i noticed in myself that after the BLM movement resurgence this summer, i was more likely to lunge in fear when addressed unsuspectingly by black homeless individuals in chicago (because i was implicitly associating black people with violence because of those two stimuli being juxtaposed on the news despite the fact that clearly the police officers were at fault and their black victims were totally innocent). also, i do not understand the article's hypothetical argument about how if a speedy test-taker is one millisecond faster on the white/good associations than on the black/bad ones, then they will get a score suggesting extremely high implicit bias against black people. if a freakshow statistical anomaly took place where the test-taker happened to be consistently but slightly slower on the black/bad bias responses but did not have that bias, then great, cool, but in all likelihood, the test would be measuring exactly what it purports to which is an unconscious negative feeling towards black people. yhis also relates to the article's discussion regarding how important explicit vs implicit bias is as a target of intervention and that the police situation at legal level in Ferguson is reflective of bias. Again, this has nothing to do with the validity of IAT - a rigorous study would look at correlations between implicit bias and implicit behavior, not explicit biases that can occur within the context of legal proceedings. The question that needs to be asked is whether the association between implicit bias and implicit behavior are rigorous and significant. Over and over again, we see that they are (links:

https://psyarxiv.com/582gh/

https://journals.sagepub.com/doi/10.1177/1368430215596075

https://psycnet.apa.org/record/2004-21198-003

https://journals.sagepub.com/doi/full/10.1177/0963721418797309

https://onlinelibrary.wiley.com/doi/abs/10.1111/pops.12401

https://journals.sagepub.com/doi/10.1177/0956797617694866

). my favorite example of my point is from Horwitz and Davidio 2015 - in this article, the investigators found that implicit biases in a population sample in favor of rich folks predicts that this sample will grant more amnesty to rich folks than poor folks when the rich ones cause a car accident. what creators purported to measure with it e.g. positive vs negative feelings toward certain groups is the mistake - does not mean the test is not a rigorous metric of implicit bias.

the other main argument the cut piece (and for that matter the research digest piece) makes regards the reliability and repeatability of the iat tests, showing low ~.4 relatedness coefficients. however, the article does not define the parameters used to assess reliability/measurement error in this context. For example, are we seeing totally random variance between test trials (e.g. is a test-taker gets extreme bias towards black people one week and extreme bias against black people the next? or is it more like slight bias one week and moderate bias the next? within the scheme of multiple trials across many individuals of course, and the average amount of shifting in scores averaged or statistically corrected for across many tests). in the latter case, low levels of reliability could reflect examinee's fear of being perceived as a racist upon second taking of the test leading to overthinking and anxiety, consciousness of possible biases that damn them towards unwanted prejudices, or "doctoring" how they take the test ie doing so in bad faith, for example moving more slowly on the white + good associations. Also, the iat test has been shown to be extremely reliable compared to other tests that measure the same type of thing (see Jost 2018, which is one of the articles linked above), e.g. blood pressure, a trait that is multifactorial (can be caused by anxiety, mood, diet, sleep) despite being stable over time (in the case of blood pressure, chronic cardiovascular health). Also, in studies that have truly found low correlation between implicit bias and implicit behaviors mentioned in the cut article, jost 2018 points out that this has to do with low methodological correspondence and the fact that these studies have rarely adjusted for measurement error.

The final part of the article talks about the harm of a potentially uninformative test like the iat making people feel unnecessarily bad about themselves and harming intergroup relations - both irrelevant to the validity of the iat by the way - though interestingly, the article points out the iat does have the power to do what it aims to (inform people of their unconscious associations - i find it rich that the article concedes this when it has sought to debunk it up to this point). some quotes: "

So there is nothing wrong with implicit-bias training that covers this sort of research. Nor is there anything wrong with IAT-based trainings which merely explain to people that they may well be carrying around certain associations in their head they are unaware of, and that researchers have uncovered patterns about who is more likely to demonstrate which response-time differences. In situations where one group holds historic or current-day power over the other, for example, members of the in-group do tend to score higher on the IAT than the out-group. Some of these between-group differences appear to be pretty robust, and they deserve further study. These are all worthwhile subjects to discuss, as long as it is made clear to test-takers that their scores do not predict their behavior." "

So it’s an open question, at least: The scientific truth is that we don’t know exactly how big a role implicit bias plays in reinforcing the racial hierarchy, relative to countless other factors. We do know that after almost 20 years and millions of dollars’ worth of IAT research, the test has a markedly unimpressive track record relative to the attention and acclaim it has garnered. Leading IAT researchers haven’t produced interventions that can reduce racism or blunt its impact. They haven’t told a clear, credible story of how implicit bias, as measured by the IAT, affects the real world. They have flip-flopped on important, baseline questions about what their test is or isn’t measuring. And because the IAT and the study of implicit bias have become so tightly coupled, the test’s weaknesses have caused collateral damage to public and academic understanding of the broader concept itself. As Mitchell and Tetlock argue in their book chapter, it is “difficult to find a psychological construct that is so popular yet so misunderstood and lacking in theoretical and practical payoff” as implicit bias. They make a strong case that this is in large part due to problems with the IAT.

Unless and until new research is published that can effectively address the countless issues with the implicit association test, it might be time for social psychologists interested in redressing racial inequality to reexamine their decision to devote so much time and energy to this one instrument. In the meantime, the field will continue to be hampered in its ability to provide meaningful answers to basic questions about how implicit bias impacts society, because answering those questions requires accurate tools. So, contra Banaji, scrutinizing the IAT and holding it to the same standards as any other psychological instrument isn’t a sign that someone doesn’t take racism seriously: It’s exactly the opposite." In this case, it is hard to know what these "standards" are. At this point, it seems like the author's main contention is that the IAT creators almost misinterpreted the mandate of their test, which again, I agree is true (they confused explicit and implicit bias and overstated the power of IAT results to predict explicit-bias based behavior). However, this article hardly discusses specific standards in light of which the IAT needs to be revamped or interpreted and to which any rigorous psychological testing battery should be subject.”

Here is an extra correction I made - “oh my point at the end of the second paragraph "what creators purported to measure with it e.g. positive vs negative feelings toward certain groups is the mistake - does not mean the test is not a rigorous metric of implicit bias" refers to the iat itself, not to the horwitz and davidio article.”

#amplifyblackvoices #implicit bias #equity #justice

0 notes