
When it comes to deciding how many response options should be given on a multiple choice test, many might argue that three versus four or five options is splitting hairs. But, researchers (Edwards, Arthur, & Bruce, 2012) would argue this issue is a perfect example of the gap between science and practice in I-O psychology.
THE RESEARCH STUDY
According to their review of the response option literature, most of the previous studies on this issue have discovered that three-response options is preferable for a number of practical and empirical reasons.
For example, tests with three response options take less time to complete, are less susceptible to “testwise” test takers, take less time and resources to develop, and have equal psychometric characteristics like reliability, discriminability, and difficulty. Similar measurement characteristics and lower time requirements mean more bang for the buck when administering these tests. In other words, test takers can be tested on a wider body of knowledge when using three options.
Given that three-option tests measure knowledge or traits just like their four and five-option cousins with less hassle, the authors questioned why practitioners have yet to broadly accept three-option tests. Guessing that practitioners still have concerns about test-taker reactions, criterion-related validity, and subgroup differences, the authors tested the assumption that there are no differences among the response-option types on these variables.
THREE VERSUS FIVE OPTIONS
In this study, the researchers gave groups of students either three-option or five-option items from the ACT college-entry test and compared students’ perceptions, race/age differences, and the test score relationship with GPA. Not only were the three-option equivalent to the five-option on their measurement characteristics, three-option tests were perceived to be just as fair, had similar correlations with GPA, and did not differ on their magnitude of race or gender-based differences. Perhaps these findings can help put practitioner fears to rest when considering the practical, financial, and administrative benefits of three-option multiple choice tests.
Edwards, B. D. Arthur, W. Jr., & Bruce, L. L. (2012). The three option format for knowledge and ability multiple-choice tests: A case for why it should be more commonly used in personnel testing. International Journal of Selection and Assessment, 20(1), 65- 81.
Image credit: Unsplash+