1. Meta-analysis of methods to change implicit bias
Patrick S. Forscher

2. What interventions are available?
Implicit bias has been argued to contribute to a variety of social problems
Many people have issued calls for effective interventions to change implicit bias
What interventions are available?
How well do they work?

3. Study inclusion criteria
Must include measure of implicit bias
Also coded correspondent explicit, behavioral measures
The implicit measure must be a pre-existing tendency
positive scores = pre-existing tendency
Reported in English
Must have sufficient information to calculate effect sizes
Must be randomized
The study must have at least two unique conditions according to the coding scheme
Point 1: Mention that prevailing assumption is that if you change implicit, you’ll change behavior
Point 6: Mention that this means we can’t handle studies that compare two different methods of perspective taking

4. Searched PsychINFO, Web of Science, and ProQuest
(implicit measure title or acronym) AND
(malleab* OR chang* OR influenc* OR moderat*
OR reduc* OR increas* OR shift* OR alter*) AND
(1995 TO 2013)

5. 341 articles, 489 studies, 552 samples
Searched PsychINFO, Web of Science, and ProQuest
Completed manual searches through Google Scholar
Requested papers directly through direct and SPSP listserv
3607 articles
Completed more detailed check against inclusion criteria
341 articles, 489 studies, 552 samples
Requested additional data for 301 incomplete samples
273 articles (223 published, 50 unpublished)
395 studies, 442 samples
44,946 participants

6. Group coding categories
Pair incongruent/Pair congruent
Famous actor who played the lead role in “Remember the Titans”
Boxer who bit an opponent’s ears during a fight
Mention that coding scheme is attempting to classify procedures rather than the goals of the researchers
Serial killer who cannibalized his victims
Famous actor who played the lead role in “Apollo 13”

7. Group coding categories
Pair incongruent/Pair congruent
Pair incongruent (indirect)/Pair congruent (indirect)
Perspective taking

8. Group coding categories Direct instructions to exhibit
Pair incongruent/Pair congruent
Pair incongruent (indirect)/Pair congruent (indirect)
Incongruent goal/Congruent goal
Direct instructions to exhibit
a certain response

9. Group coding categories
Pair incongruent/Pair congruent
Pair incongruent (indirect)/Pair congruent (indirect)
Incongruent goal/Congruent goal
Threat

10. Group coding categories Considering one’s values
Pair incongruent/Pair congruent
Pair incongruent (indirect)/Pair congruent (indirect)
Incongruent goal/Congruent goal
Threat
Affirmation
Considering one’s values

11. Group coding categories Completing a cognitively taxing task
Pair incongruent/Pair congruent
Pair incongruent (indirect)/Pair congruent (indirect)
Incongruent goal/Congruent goal
Threat
Affirmation
Positive mood/Negative mood
Depletion
Completing a cognitively taxing task

12. Participant characteristics
88% student, 12% non-student
49% female, 27% male, 25% not reported
39% White, 9% non-White, 51% not reported
Measure characteristics
59% attitudes, 20% cognition, 8% self-esteem, 12% self-concept
67% IAT, 11% semantic priming, 5% LDT, 17% other
5% longitudinal
Demographic characteristics of intro psych students
Study characteristics
56% US, 27% Europe, 17% other
62% prejudice/stereotyping, 24% health/clinical, 15% self, 9% other
49% two groups, 21% three groups, 30% four+

13. Participant characteristics
88% student, 12% non-student
49% female, 27% male, 25% not reported
39% White, 9% non-White, 51% not reported
Measure characteristics
59% attitudes, 20% cognition, 8% self-esteem, 12% self-concept
67% IAT, 11% semantic priming, 5% LDT, 17% other
5% longitudinal
Participant characteristics reflect demographics of introductory psychology students. Wide variety of measures, but most studies use the IAT. Very few longitudinal studies. Multi-group studies are important because they mean that conventional methods of meta-analysis are not possible
Study characteristics
56% US, 27% Europe, 17% other
62% prejudice/stereotyping, 24% health/clinical, 15% self, 9% other
49% two groups, 21% three groups, 30% four+

14. Network meta-analysis
d = -.56
𝑑= 𝑥1 −𝑥2 𝑠𝑝

15. Network meta-analysis
d = -.56
Network meta-analysis handles multi-group studies, outcome is standardized mean difference, must choose a common reference group. There are a few more subtleties for those who are interested
d = .42
𝑑= 𝑥1 −𝑥2 𝑠𝑝

16. Assessing the network of interventions

18. Results: Implicit data

20. Indirect pairings are not symmetric, and neither are the goals manipulations.

21. Results: Moderators

23. Most of the IAT effect sizes are larger than the non-IAT effect sizes

25. Domain seems to matter for pair incongruent (indirect) and threat

26. Results: Behavioral data
I could add the explicit analyses here too, but I thought there might not be enough time

27. Here I point out that this network looks very different from the implicit network. In particular, most interventions are only connected to the neutral control group and their conceptual opposite.

28. Meta-analytic implicit-behavioral correlation:
r = .086, [.047, .12], k = 46

29. Meta-analytic implicit-behavioral correlation:
r = .086, [.047, .12], k = 46

30. Provisional conclusions
Implicit bias is malleable
Methodological caveats:
Most of our data come from students
We do not know if the effects endure
How effective a manipulation appears to be depends on the implicit measure used
Conceptual caveats:
The effectiveness of a manipulation depends on the content domain
Effects on implicit bias do not necessarily generalize to behavior

31. Collaborators Trish Devine Calvin Lai Michelle Herman Jordan Axt
Feel free to swap out any of these pictures for different ones if desired
Michelle Herman
Jordan Axt
Brian Nosek
Charlie Ebersole