The Experimenter Series: Diversity Training with Professor Edward Chang
The experimenter series is a collection of interviews with people pioneering experiments in organisations. This time, we speak to Professor Edward Chang from Harvard Business School about diversity training.
Edward Chang is an assistant professor of Business Administration at Harvard Business School and focuses his research on improving diversity, equity, and inclusion in the workplace. He is also co-author of a famous experiment and HBR article on the ambiguous effects of diversity training. Edward is part of the MoreThanNow Inclusion Lab: an academic partnership in which five brilliant researchers help us conduct D&I research in organisations.
“The lesson is not to throw away diversity training. Instead, companies should remember that diversity training by itself is not going to solve all of your problems.”
- Edward Chang, Harvard Business School.
Gus: Hi Edward, thanks for speaking to me! I’m going to dive straight in with a massive question: what do we know about the effects of diversity training, and what do we still need to discover?
Edward: We know that training can be effective at changing people’s explicit attitudes towards topics relating to diversity, equity and inclusion. We’ve seen evidence of this both in the experiment I ran with my collaborators, as well as in meta-analyses of the diversity training literature. But in spite of these benefits, there are important caveats to mention.
First, these measures are typically administered shortly after training, so it's unclear whether the effects will last. In other words, we don’t know how durable attitude change is.
Second, as with almost all diversity research, there's always the concern that people are responding in the ways that they do because of social desirability or demand effects. For example, if an employee sits through an hour of diversity training and they report feeling more positively towards diversity afterward, is it because they truly changed their mind, or is it because they believe that that’s what the company expects of them?
Given the limits of attitudinal measures, it’s thus a much more open question about the behavioural consequences of diversity training. Do these sorts of training really lead to meaningful changes in consequential workplace behaviors?
There’s also a philosophical question we should ask ourselves. Diversity training can come in many different forms, but a lot of training in recent years has focused on unconscious bias. Psychologically, we think of these sorts of bias as accumulating over a lifetime from existing in a society or culture. And so it’s an interesting question to think whether an hour or day of training can somehow overcome a lifetime of accumulated experiences.
G: The only way we'll learn more about diversity training is if organisations commit to testing it properly with experiments. You pioneered the largest trial ever with an organisational partner, can you tell us about what you learned?
E: Diversity training is one of the most implemented interventions that organisations use to try and address issues of DEI. And yet, there hadn’t really been any large-scale experiments conducted in non-educational organisational settings that measured people’s behaviors in response to training. Luckily, my collaborators and I were able to find an organisational partner that was interested in contributing to the science of D&I. They were willing to let us randomly assign thousands of employees to different online training programmes, and to measure both attitudes and behaviours in unobtrusive ways over several months.
In designing the training, we incorporated the best science we knew about prejudice reduction and how to teach people skills and strategies. We worked with our organisational partner to figure out ways to measure what we would consider meaningful behaviours. For behavioural measures, we measured whether there were gender gaps in people's willingness to mentor other people, people's willingness to recognize their peers for excellence, and their willingness to help someone they didn’t know
Consistent with past literature, we generally found evidence of attitude change, but attitudes were measured immediately after the training. We also did find some evidence of behavioural change as a result of the training, but it was not the sort of change that we expected. Rather than finding evidence of men or White people changing their behaviors towards women and racial minorities, the largest behavioural effect we documented was that our training seemed to cause women in the U.S. to lean in and be more proactive about seeking out mentorship in the workplace.
G: What would you say should organisations learn from your research?
E: Training can be one component of a multi-pronged or more holistic approach toward D&I. Our results suggest that diversity training alone is not going to have large behavioural consequences for most employees in organisations. Thus, it should not be viewed as a silver bullet to solving a company’s diversity-related issues.
If I were to rerun the experiment, I would want to do more to measure the social norm effects of training. One of the most potentially valuable consequences of diversity training is what training signals to employees about what the organisation values, and I believe this is a fruitful area for future research. In fact, in a current experiment with MoreThanNow and Ericsson, we’re trying to harness norm signalling by having a senior leader in the training talk about why diversity is so important to Ericsson. This could help make training even more potent.
G: Let’s talk more about that trial. We have amazing support from the leadership team, but at the heart of the study is a new idea called Focused Diversity Training. Why is this approach different and why are you hopeful it will lead to good results?
E: In general, I think people want to do the right thing when it comes to diversity, but they might not always know what the right thing to do is. I also think it’s easy to forget about diversity in our increasingly hectic work lives. Our focused diversity training tries to tackle both of these problems.
“It’s easy to forget about diversity in our increasingly hectic work lives. Our focused diversity training tries to tackle both of these problems.”
First, the training is focused on a specific workplace decision that people make. In that way, we can provide concrete advice and strategies that can be directly applied to the decision. Second, we’re making the training timelier by administering training right before the decision takes place. This helps address forgetfulness. We’re hoping that these two changes will make training more effective at changing behaviors.
G: And why do you run experiments in large organisations?
E: From a scientific perspective, field experiments are the gold standard for empirical evidence in that they combine the internal validity of experiments and the external validity of field research. There’s been so much great research done online or in lab settings, but I’ve always been concerned how these findings translate into real world behavior. Ultimately, because I care about changing people’s behaviour in the real world, I work with organisations to run these experiments to improve the science of DEI and create generalizable knowledge. This is only possible by collaborating with organisations who want to see this change too.
Thanks to Edward for sharing his incredible expertise. We’re already working on multiple experiments together - including our study at Ericsson - and are always looking for new organisations who want to take a more robust approach to D&I. If you’re interested, you can read more about The Inclusion Lab below.