Topic: Assessment, Staffing
Publication: International Journal of Selection and Assessment (SEP 2010)
Article: Do assessors have too much on their plates? The effects of simultaneously rating multiple assessment center candidates on rating quality
Authors: K.G. Melchers, M. Kleinmann, and M.A. Prinz
Reviewed By: Benjamin Granger
Assessment centers (ACs) usually consist of several job-related exercises that tap competencies necessary for the job. ACs are most often used by organizations to select, promote and develop their employees. Like many employee selection and assessment methods (e.g., interviews), ACs require a scorer or assessor to provide an evaluation of candidates’ performance. But here’s where it gets tricky.
In two studies, Melchers et al. found that assessor ratings of candidates decrease in accuracy as the number of candidates they assess simultaneously increases. While this finding may seem like a “no-brainer”, meta-analytic research (Woehr & Authur, 2003) has demonstrated that ACs with higher candidate-to-assessor ratios (i.e., fewer assessors evaluating more candidates) tend to be more valid than ACs with lower candidate-to-assessor ratios. However, as Melchers et al. point out, the candidate-to-assessor ratio in an AC is not the issue here, its how many candidates an assessor must observe and evaluate simultaneously that apparently decreases rating accuracy.
Unfortunately, Melchers et al. do not provide specific recommendations for the ideal number of candidates assessors should be assigned to assess simultaneously in an AC exercise.
As they note, experienced assessors may be able to assess two or three candidates on several competencies simultaneously while less experienced assessors may not. The key here is that AC designers should be aware that too high a work load can negatively influence the accuracy of assessor ratings. Thus, when possible, the number of candidates an assessor is assigned to observe and evaluate simultaneously should be limited.
By now you may be thinking, “that’s great and all, but more assessors cost more money!” And you would be correct. But ultimately, the initial costs associated with decreasing assessor workloads during group discussion exercises are likely much less than the potential costs of hiring or promoting the “wrong” candidate(s).