Integrity Tests May Have Lower Performance Validity (IO Psychology)

Topic(s): selection

Topic: Selection, Measurement
Publication: Journal of Applied Psychology
Article: The Criterion-Related Validity of Integrity Tests: An Updated Meta-Analysis
Authors: Van Iddekinge, C.H., Roth, P.L., Raymark, P.H., & Odle-Dusseau, H.N.
Reviewer: Neil Morelli

According to a recent meta-analysis by Van Iddekinge and colleagues, integrity tests may not be as predictive of job performance as once thought. Integrity tests have become popular with organizations and practitioners due to their high correlations with job performance and few differences between groups (based on race, gender, etc.). But, Van Iddekinge et al. were concerned that past meta-analytic results drew too heavily on unpublished studies authored by test publishers. In fact, only 10% of one meta-analysis’ sample was made up of studies published in peer-reviewed journals (pro-tip: we like things that are peer reviewed).

The authors used 104 studies (42 were published and 62 were unpublished) to investigate if including more “neutral” primary studies whose methodology has been more rigorously vetted would change the test’s validity. They reported that the overall job performance validity is .13 to .16 depending on whether it’s corrected for unreliability. In other words, this validity coefficient is much lower than originally reported and indicates that the integrity test is not as predictive of job performance as once thought. Although the test has a higher validity coefficient for counterproductive work behaviors (.26 to .32), this is still lower than originally reported in previous meta-analyses.

So what does this all mean? On the one hand integrity tests are still viable options for
predicting counterproductive work behaviors while maintaining low sub-group differences. On the other hand, integrity tests’ predictive validity is weaker than previously thought and practitioners may not be able to rely on meta-analytic results in lieu of a local validation study. The authors pointed out that one could argue test publisher data is overly optimistic while data from independent researchers is overly pessimistic. Regardless of your position the authors suggest that practitioners should consider the source when reporting integrity test validity and researchers may need to develop more primary studies on the standard integrity test’s true capability to predict future job performance.

Van Iddekinge, C.H., Roth, P.L., Raymark, P.H., & Odle-Dusseau, H.N. (2012). The
criterion-related validity of integrity tests: An updated meta-analysis. Journal of Applied
Psychology, 97, 499-530.

human resource management, organizational industrial psychology, organizational management



source for picture: