Size Matters in Court? Determinations of Adverse Impact Based on Organization Size (IO Psychology)

Topic: Assessment, Discrimination, HR Policy, Statistics

Publication: Journal of Business Psychology (JUN 2012)

Article: Unintended consequences of EEO enforcement policies: Being big is worse than being bad

Authors: R. Jacobs, K. Murphy, and J. Silva

Reviewed By: Megan Leasher


Adverse impact occurs when neutral-appearing employment practices have an unintentional, discriminatory effect on a protected group. The Equal Employment Opportunity Commission is charged with enforcing all federal legislation related to employment discrimination and adheres to the 1978 Uniform Guidelines on Employee Selection Procedures for “rules of thumb” on inferring whether adverse impact is present.

However, it’s tricky for a plaintiff to present conclusive evidence that adverse impact is present in an organization’s practices.  Statistical evidence is needed to demonstrate whether employment practices are truly discriminating against a protected group.  Jacobs and colleagues investigate a common statistical method, known as “significance testing,” which is often used in courts to demonstrate evidence of adverse impact.  Significance testing compares the difference between the proportion of majority candidates selected and the proportion of protected class candidates selected in an employment decision.  If the test finds the difference between these proportions to be “statistically significant,” courts generally interpret this to mean that adverse impact is present.

This method seems to make sense from a high level, but problems arise when you look under the surface.  The outcome of significance testing is greatly influenced by the number of people who are included in the analysis.  Specifically, the more people you include in a significance test, the greater the likelihood of finding a statistically significant difference between groups. So, if you have an organization with many people included in the analysis, you are much more likely to yield a significant difference between majority and protected groups than you would with a smaller organization with fewer people to include in the exact same analysis.  This is the primary argument of the authors; why do courts use significance testing to demonstrate adverse impact when, by nature of the test, the results would almost always find that big organizations are discriminating and smaller ones are not?

The authors conducted a series of studies to determine sources of differences in adverse impact significance testing.  They found that the number of people included in the analysis was the strongest predictor in whether or not a statistically significant difference was found between groups.  Size  accounted for 49% of the final outcome of the analysis, which was almost five times greater than what any other factor (e.g., score differences on assessment in question, proportion in each group selected, etc.) accounted for.  They also discovered an interesting threshold:  When an adverse impact significance test is conducted with 500 or more people in the analysis, very small differences between the groups’ selection proportions will be statistically significant (yet below 500 these same comparisons would not be significantly different).

These findings support the powerful impact of sample size on determinations of adverse impact via significance testing, but they do not tell us if members of majority and protected class groups are really experiencing systemic, differential outcomes in employment practices.  Unless a statistical method can accurately assess the latter issue, it is meaningless.  This oversimplification leads us to believe that virtually all larger organizations are guilty of discrimination and virtually all smaller organizations are not.  This common practice in courts only serves to make small organizations feel impervious and invincible and leave large organizations running in fear.

The authors close by asserting that regulatory standards should always reflect current scientific knowledge, yet the Uniform Guidelines on Employee Selection Procedures still reflect the science of decades past.  They advocate for not only alternative methods to more appropriately measure adverse impact, but also for a more dynamic definition of adverse impact; one that considers multiple, interactive factors before a determination can be made.  Current practice is supporting the message that to be big is to be bad and to be small is to be nice, which goes directly against the spirit of anti-discrimination legislation.

Jacobs, R., Murphy, K., & Silva, J. (2012, June).  Unintended consequences of EEO enforcement policies: Being big is worse than being badJournal of Business and Psychology

human resource management, organizational industrial psychology, organizational management




source for picture: