How to Evaluate Bias on Selection Tests

When practitioners use pre-employment tests for employee selection decisions, they must consider the potential biases that may result from the assessment. Using biased tests can lead to poor or unfair hiring decisions. Not only can this impact a company’s reputation and bottom line, but legal issues can arise if selection procedures are not free from bias.

WHAT DOES TEST BIAS REALLY TELL US?

The authors (Meade & Tonidandel, 2010) argue that despite the importance of assessing for test bias in the field of I-O psychology, there are pervasive misunderstandings regarding what evaluations of test bias really tell us. They believe that the standard technique for evaluating whether a test is biased (based on a statistical technique called regression) fails to tell us what we really need to know about a test. The authors explain that this older approach focuses on differences in regression lines across groups, which is known as differential prediction. This method examines the extent to which predicted performance matches observed performance in a specific selection context. The authors’ main concern with is that when differential prediction is found, many people assume there is a problem with the test itself. However, there are many other causes of differential prediction, for example, the way performance is measured or omitted variables.

PRACTICAL IMPLICATIONS

Based on their argument, the authors provide some recommendations to anyone responsible for selection tests:

(1)   Practitioners should work with clients to attempt to identify the source of the differential prediction and eliminate it if possible.

(2)   The entire selection system (all tests used to predict performance) should be assessed for adverse impact, which refers to unequal hiring outcomes for protected groups (e.g., based on race). If the differential prediction occurs for only part of the selection system, look for a predictor that can reduce the overall bias. For example, if a cognitive ability test leads to differential prediction for one group of applicants, practitioners should look for another, non-biased predictor (personality, motivation, cultural fit, etc.) to add to the selection system that will account for the differences in performance.

(3)   Practitioners should not assume a test is unusable in cases where differential prediction is encountered. Depending on the organization’s goals and priorities, differential prediction does not necessarily make a test unusable for selection. As long as a predictor accurately predicts performance and does not lead to adverse impact, suitability of these tests may depend on opinions of fairness within the organization and society at large.

 

Meade, A.W. and Tonidandel, S. (2010). Not Seeing Clearly With Cleary: What Test Bias Analyses Do and Do Not Tell Us. Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 192–205.

Image credit: Unsplash+