Topic: Assessment
Publication: Journal of Applied Psychology
Article: Further Evidence for the Validity of Assessment Center Dimensions: A Meta-Analysis of the Incremental Criterion-Related Validity of Dimension Ratings
Blogger: Rob Stilson
Here I go again with a psychometrically heavy article, but I encourage you to stick it out with me as I guide you through the statistical minefield because there are some applicable findings at the end. First, a little bit of history. Assessment Centers (ACs) are often used to select or promote personnel at the managerial level or higher. An AC is collection of standardized exercises, such as a leaderless group discussion, inbox task, role play, etc., where you are rated on your ability in these different exercises and then given an overall assessment center rating (OAR) to determine if you get the position or the promotion. Each exercise is supposed to get at one or more dimensions, which roughly break down into the following seven (Arthur, Day, McNelly, & Edens, 2003):
· Problem solving
· Stress tolerance
· Influencing others
· Consideration/awareness of others
· Communication
· Organizing and planning
· Drive
ACs, seem to work very well at determining who will be best for the position, and they are also used for development – assessing strengths and development areas in order to tailor development plans. Due to their comprehensive nature, and what it takes to create “a day in the life of” simulation, they are very expensive and time consuming. Ok, now that you are caught up, on to the current study.
Meriac, Hoffman, Woehr, and Fleisher (2008) investigated whether ACs tell us anything we could not get from significantly less expensive paper-and-pencil personality and cognitive ability tests.
To answer this questions, the Meriac, et. al gathered studies that met the criteria of including correlations between AC dimensions, and either general mental ability (GMA) or personality variables amongst other criteria mentioned in the article. After various statistical corrections and some heavy number crunching, the answer is yes! An AC does offer different information than what you get from personality and cognitive ability tests.
The variance accounted for in job performance jumps from 20% to 30% when an AC is used in addition to personality and cognitive ability tests, which rates as frickin’ huge on the practical significance scale. Also, the seven AC dimensions mentioned previously only moderately correlate with GMA and the Big Five personality dimensions indicating that you are getting at new information by using the AC.
Practical implications of these findings are that even though an AC is expensive and time consuming, it is worth it because of the jump in variance accounted for in job performance. An additional bonus is that an AC typically does not have adverse impact on any subgroups the way a cognitive ability test might.