According to a recent meta-analysis, or statistical combination of many past studies, integrity tests may not be as predictive of job performance as once thought. Integrity tests have become popular with organizations and practitioners due to their high correlations with job performance and few differences between groups (based on race, gender, etc.). But researchers (Van Iddekinge et al. 2012) were concerned that past meta-analytic results drew too heavily on unpublished studies authored by test publishers. In fact, only 10% of one meta-analysis’ sample was made up of studies published in peer-reviewed journals.
THE RESEARCH STUDY
The authors used 104 studies (42 were published and 62 were unpublished) to investigate if including more “neutral” primary studies whose methodology has been more rigorously vetted would change the test’s validity. They reported that the overall job performance validity is .13 to .16 depending on whether it’s corrected for unreliability. In other words, this validity coefficient is much lower than originally reported and indicates that the integrity test is not as predictive of job performance as once thought. Although the test has a higher validity coefficient for counterproductive work behaviors (.26 to .32), this is still lower than originally reported in previous meta-analyses.
THE BOTTOM LINE
So what does this all mean? On the one hand integrity tests are still viable options for predicting counterproductive work behaviors while maintaining low sub-group differences. On the other hand, integrity tests’ predictive validity is weaker than previously thought and practitioners may not be able to rely on meta-analytic results in lieu of a local validation study. The authors pointed out that one could argue test publisher data is overly optimistic while data from independent researchers is overly pessimistic. Regardless of your position, the authors suggest that practitioners should consider the source when reporting integrity test validity and researchers may need to develop more primary studies on the standard integrity test’s true capability to predict future job performance.
Van Iddekinge, C.H., Roth, P.L., Raymark, P.H., & Odle-Dusseau, H.N. (2012). The criterion-related validity of integrity tests: An updated meta-analysis. Journal of Applied Psychology, 97, 499-530.