Adverse impact occurs when neutral-appearing employment practices have an unintentional, discriminatory effect on a protected group. The Equal Employment Opportunity Commission is charged with enforcing all federal legislation related to employment discrimination and adheres to the 1978 Uniform Guidelines on Employee Selection Procedures for “rules of thumb” on inferring whether adverse impact is present.
However, it’s tricky for a plaintiff to present conclusive evidence that adverse impact is present in an organization’s practices. Statistical evidence is needed to demonstrate whether employment practices are truly discriminating against a protected group. The authors of this study (Jacobs, Murphy, & Silva, 2013) investigate a common statistical method, known as “significance testing,” which is often used in courts to demonstrate evidence of adverse impact. Significance testing compares the difference between the proportion of majority candidates selected and the proportion of protected class candidates selected in an employment decision. If the test finds the difference between these proportions to be “statistically significant,” courts generally interpret this to mean that adverse impact is present.
PROBLEMS WITH SIGNIFICANCE TESTING FOR ADVERSE IMPACT
This method seems to make sense at face value, but problems arise when looking under the surface. The outcome of significance testing is greatly influenced by the number of people who are included in the analysis. Specifically, the more people included in a significance test, the greater the likelihood of finding a statistically significant difference between groups. So, in an organization with many people included in the analysis, there is a greater likelihood of yielding a significant difference between majority and protected groups than there would be in a smaller organization that has fewer people in the analysis. This is the primary argument of the authors; why do courts use significance testing to demonstrate adverse impact when, by nature of the test, the results would almost always find that big organizations are discriminating and smaller ones are not?
The authors conducted a series of studies to determine sources of differences in adverse impact significance testing. They found that the number of people included in the analysis was the strongest predictor in whether or not a statistically significant difference was found between groups. Size accounted for 49% of the final outcome of the analysis, which was almost five times greater than what any other factor accounted for (e.g., score differences on assessment in question, proportion in each group selected). They also discovered an interesting threshold: When an adverse impact significance test is conducted with 500 or more people in the analysis, very small differences between the groups’ selection proportions will be statistically significant, yet below 500, these same comparisons would not be significantly different.
PRACTICAL IMPLICATIONS FOR SELECTION
These findings support the powerful impact of sample size on determinations of adverse impact via significance testing, but they do not tell us if members of majority and protected class groups are really experiencing systemic, differential outcomes in employment practices. Unless a statistical method can accurately assess the latter issue, it is meaningless. This oversimplification leads us to believe that virtually all larger organizations are guilty of discrimination and virtually all smaller organizations are not. This common practice in courts only serves to make small organizations feel impervious and invincible and leave large organizations running in fear.
The authors close by asserting that regulatory standards should always reflect current scientific knowledge, yet the Uniform Guidelines on Employee Selection Procedures still reflects the science of decades past. The authors advocate for alternative methods to more appropriately measure adverse impact, and also for a more dynamic definition of adverse impact; one that considers multiple, interactive factors before a determination can be made. Current practice is supporting the message that to be big is to be bad and to be small is to be nice, which goes directly against the spirit of anti-discrimination legislation.