Multiple, speeded assessments are a relatively new personnel selection approach in which applicants participate in a number of short sessions that simulate a variety of job-relevant situations. This allows applicants to be rated comprehensively based on a diverse range of situations they may encounter on the job, while maintaining efficiency and keeping costs low. However, it remains unclear how effective multiple, speeded assessments may be in hiring the right candidates for the job.
MULTIPLE, SPEEDED ASSESSMENTS AND FUTURE PERFORMANCE
The researchers (Herde & Lievens, 2022) conducted two studies in which business students were rated on 18 different role-plays that each lasted 3 minutes. Seven months later, participants were rated on different aspects of their job performance, including performance on job tasks, social contributions to the workplace, teamwork, and communication.
Together, the results of these studies showed that overall scores on multiple, speeded simulations predicted future job performance better than a traditional multiple choice situational judgement test (a tool that asks applicants how they would act in different scenarios). However, when looking at ratings from individual role-plays, rather than aggregated scores from all of the role-plays, variability was too high.
Interestingly, there was not much difference in the predictive ability of assessors who were thoroughly trained and evaluated the full 3-minute sessions, versus assessors who received no training and only saw the first minute of each session. There were also similar outcomes when comparing assessors who administered the role-plays and evaluated participants in-person, and those who evaluated a recording of the role-plays.
PRACTICAL APPLICATIONS FOR ORGANIZATIONS
In addition to concluding that speeded simulations are useful in predicting job performance, the researchers say that organizations should keep three things in mind when using these tests. First, the studies suggest that quality is more important than quantity. That is, ensuring that role-play scenarios cover a wider variety of situations is more beneficial than observing behavior for a longer period in each situation.
The researchers also state that it is more important to aggregate ratings across all simulations and assessors than it is to train assessors extensively in one or two role-play scenarios. Finally, the authors suggest that multiple, speeded simulations can be assessed by the same individual who administered the role-play without sacrificing validity.
Herde, C. N., & Lievens, F. (2022). Multiple, speeded assessments under scrutiny: Underlying theory, design considerations, reliability, and validity. Journal of Applied Psychology. Advance online publication.
Photo credit: istockphoto/anyaberkut