Topic: Assessment
Publication: European Journal of Psychological Assessment (SPRING 2011)
Article: Context Effects on Test Performance: What about Test Order?
Authors: L. Khorramdel & M. Frebort
Reviewed By: Thaddeus Rada
Many people are familiar with the mathematical concept of order of operations. Sometimes shortened to the acronym PEMDAS, order of operations instructs us to solve in parentheses (P) before we deal with exponents (E), to multiply (M) before we divide (D), etc. The reason for working through math problems in this way is to lead us to the correct answer; if we solve the problem in any other order, we run the risk of coming up with an incorrect answer. It now appears that this same principle might apply to test administration. Previous studies have assessed how the order of items in a single test might impact individuals’ responses to the items. Now, a new study by Lale Khorramdel and Martina Frebort suggests that the order in which multiple tests in a testing battery are administered may impact individuals’ responses on the tests.
The current study examined these test order effects using two broad classes of tests: objective personality and cognitive ability. More specifically, Khorramdel & Frebort hypothesized that the administration of either test type as the first test would impact participants’ performance on the second test (e.g. working through the objective personality tests would have an impact on performance on a subsequently-administered cognitive ability test, and vice versa).
Using a sample of 66 members of upper management from an Austrian automotive supplier, the authors found support for their hypothesis that taking cognitive ability tests first impacted later performance on objective personality tests. However, they did not find support for their other hypothesis: that is, taking objective personality tests first did not significantly impact performance on cognitive ability tests.
While the somewhat small and specific nature of the sample used in this study might limit the generalizability of the study’s findings, the study still demonstrates that it is possible for test order to impact test performance. This knowledge leads to a somewhat obvious conclusion: the fact that test order can have an impact on test performance should be considered when implementing a testing battery. It may be the case that, if the impact of test order in a personnel selection or development scenario is unknown, it would be most prudent to follow the advice of the order of operations, and administer the battery in the same order to all participants, so that the order of the tests would not affect certain applicants differently than others. Or, the organization might take a more active approach and utilize an I/O psychologist to investigate possible test order effects!