Work Science Center Network member, Deborah Rupp, teamed up with other scientists to probe the causes of group-level differences in the way people were rated during the Assessment Center Method. The Assessment Center has been long viewed as an objective means of measuring performance. It involves standardized evaluation of behavior based on job-related simulations, interviews, and psychological tests. The job simulations measure candidates’ critical competencies for jobs.
In an age where we recognize internalized bias’ pernicious effects, the Assessment Center has long been considered immune. Some research has recently questioned this assertion, noting that demographic differences can impact ratings. Rupp and her team wanted to know why. In the present study, they examined how the assessors’ own demographic characteristics affected the ratings they gave. In particularly, they looked for leniency bias and similar-to-me bias, assuming that assessors would rate people of their same race more favorably, and that men would rate women more favorably overall.
Surprisingly, they found virtually no individual-level effects, and the group-level impact they found was quite small. In sum, based on their sample of professionals from a large metropolitan police department, the Assessment Center method seemed to work. They did not find systematic differences in the way assessors rated candidates based on their own characteristics.
Rupp and the team wondered why. In their considerations for future directions and limitations of their research, they suggest that their study paired assessors and assessees who didn’t know each other. This mutual novelty excluded personal history, knowledge of personality or values, and perhaps enabled greater objectivity. Similarly, in the study setting, the assessors received extensive training and resources to perform the Assessment Center. The researchers verified the process’ integrity from start to finish. Does the same caliber of Assessment Center exist universally? Certainly not! They implore organizations using the Assessment Center Method to safeguard the instrument to preserve its predictive power and overall fairness. But that’s just a hunch. In order to really find out what’s causing group-level discrepancies, scientists need to conduct more studies to examine what’s going on.
Thornton, G.C., Rupp, D.E., Gibbons, A.M., Vanhove, A.J. (2019). Same-gender and same-race bias in assessment center ratings: A rating error approach to understanding subgroup differences. International Journal of Selection and Assessment 27: 54-71. doi: 10.1111/ijsa.12229