Many people are familiar with the mathematical concept of order of operations. For example, in an equation, multiplication is conducted prior to addition. The reason for working through math problems in this way is to lead us to the correct answer; if we solve the problem in any other order, we run the risk of coming up with an incorrect answer. It now appears that this same principle might apply to test administration. Previous studies have assessed how the order of items in a single test might impact individuals’ responses to the items. Now, a new study (Khorramdel & Frebort, 2011) suggests that the order in which multiple tests in a testing battery are administered may impact individuals’ responses on the tests.
THE RESEARCH STUDY
The current study examined these test order effects using two broad classes of tests: objective personality and cognitive ability. More specifically, the researchers hypothesized that the administration of either test type as the first test would impact participants’ performance on the second test (i.e. working through the objective personality tests would have an impact on performance on a subsequently-administered cognitive ability test, and vice versa).
Using a sample of 66 members of upper management from an Austrian automotive supplier, the authors found support for their hypothesis that taking cognitive ability tests first impacted later performance on objective personality tests. However, they did not find support for their other hypothesis: that is, taking objective personality tests first did not significantly impact performance on cognitive ability tests.
THE KEY TAKEAWAY
While the somewhat small and specific nature of the sample used in this study might limit the generalizability of the study’s findings, the study still demonstrates that it is possible for test order to impact test performance. This knowledge leads to a somewhat obvious conclusion: the fact that test order can have an impact on test performance should be considered when implementing a testing battery. It may be the case that, if the impact of test order in a personnel selection or development scenario is unknown, it would be most prudent to follow the advice of the order of operations, and administer the battery in the same order to all participants, so that the order of the tests would not affect certain applicants differently than others. Or, the organization might take a more active approach and utilize an I-O psychologist to investigate possible test order effects.
Khorramdel, L. & Frebort, M. (2011). Context effects on test performance: What about test order? European Journal of Psychological Assessment, 27, 103-100.
Image credit: istockphoto/champpixs