Category Archives: *International Journal of Selection and Assessment
Random Thoughts While Taking a Personality Test (IO Psychology)
Have you ever taken a personality test as part of a hiring process and found your mind wandering? Where did it go? Did you wonder how to best answer the questions to secure the job? Or did you ponder why puppies are just so damn cute? Or why the Flock of Seagulls hairstyle never took off?
In this study, researchers interviewed test takers to learn what they were thinking while taking a personality test. I was excited about this, because in industrial psychology we tend to fixate on how well personality tests succeed in identifying rockstar candidates, but we should also think about how candidates react when completing them. Here are a few of the findings:
- Some test-takers had a difficult time answering the questions, as no context was provided. For example, if they had to rate how likely they are to “start a conversation with a stranger” (with no other context), they felt as though their answer would differ if asked in a work setting versus their personal life. I can see this one.
- A few didn’t feel as though they knew themselves well enough to answer the questions. This one really threw me; I don’t know about you, but I’m not sure I would want to hire someone who lacks enough self-awareness to answer questions on a basic personality test.
- A number of individuals answered the questions in a way to make them look more favorable, to help secure the job in question. Others tried to answer as honestly as possible, thinking the test would catch them and figure out their devious plan. I see both reactions here; some assessment tests can, in fact, detect if someone is trying to cast themselves in a more favorable light; other tests do not. This is one of the great mysteries when taking an assessment test…Will the stats police uncover my lies?
- Many test-takers tried to figure out how their answers would be analyzed and interpreted. We all have this natural curiosity, but voodoo-magician test publishers keep this secret close. (Cueing “something D-O-O economics…”)
- Most individuals noticed questions that seemed to be similar and/or repeating, but assumed this was a way to detect how consistently they answered the questions. This is true; it helps test creators measure the consistency of your answers, see if you are paying attention whatsoever, or reveal if you are just having your cat select the answers for you (we all know how much cats love to sprawl out on keyboards).
So the moral of the story is: our reactions to personality tests are as different as our personalities themselves, which might mean that I am the only person out there that wants the Flock of Seagulls hairstyle to make a comeback.
Another Perk of Volunteering: It Does Look Good on a Résumé!
Topic: Recruiting
Publication: International Journal of Selection and Assessment (SEP 2012)
Article: Do I Look Like Someone Who Cares? Recruiters’ Ratings of Applicants’ Paid and Volunteer Experience
Authors: Christa Wilkin and Catherine Connelly
Reviewed By: Thaddeus Rada
In the world of human resource management, recruiters often have a limited number of sources from which to gather information about job applicants. One common source of information that recruiters typically do have access to are applicants’ résumés. Although résumés may contain a wide variety of information about applicants, one topic that often receives attention on a résumé is volunteer experience. It’s possible that recruiters believe that such experience may be indicative of an individuals’ generosity, dedication to a cause, or willingness to work for few external motivators; however, there is some debate as to how volunteer experience is really weighted by recruiters. Do they value it over paid experience? Does the relevance of the volunteer experience to the job being applied for make a difference? In a recent paper, Wilkin and Connelly address these questions.
Using a sample of human resources professionals and recruiters, Wilkin and Connelly found that paid experience is not inherently more desirable than volunteer experience. Perhaps not surprisingly, the authors also found that past experience that was job-relevant (whether or not it was paid or volunteer experience) was viewed by the recruiters more favorably than past experience that was not job-relevant. Finally, résumés with both paid and volunteer experience were viewed more favorably than résumés that only contained one type of experience.
This study’s findings are especially relevant to practitioners who may be coaching applicants on résumé writing. The results of this study suggest that relevant work experience should be emphasized, as opposed to any and all work experience, related or unrelated. It also suggests that, particularly for entry-level employees, any volunteer experience, in conjunction with at least a small amount of relevant paid experience, is worthy of inclusion in a résumé.
human resource management, organizational industrial psychology, organizational management
What are those emerging markets thinking? (IO Psychology)
Topic: Selection, Assessment
Publication: International Journal of Selection and Assessment (JUN 2012)
Article: Cross-cultural Examination of Applicant Reactions to Selection Methods: United States and Vietnam
Authors: Hoang, T.G., Truxillo, D.M., Erdogan, B., and Bauer, T.N.
Reviewer: Neil Morelli
Over the past several years I-O psychologists have become more interested in understanding applicant reactions to selection tools. Of course we still care about the reliability and validity of the selection tool, but we know that how an applicant reacts to the process could influence how fair or trusting the company is perceived to be, how well the newly hired employee adjusts to the job, or how vulnerable the selection tools are to legal challenge. And, as organizations become more global and enter emerging markets, it’s important to understand what the reactions of people in these new, unexamined candidate pools may be.
A good example of an emerging market is Vietnam—economic reports show that companies are moving operations to Vietnam in greater numbers and Vietnam’s economy is expected to become the 17th largest by 2025. To understand the potential reactions to selection methods in a country with very different cultural, legal, and socioeconomic conditions, the authors compared the reactions of 225 Vietnamese business students to 151 American business students. Examples of the selection methods included interviews, work samples, resumes, biodata, and graphology, while the specific reactions were process favorability and fairness (i.e.,
face validity, personal privacy, and legality).
Hoang et al. discovered that perceptions in either country were fairly similar to each other: interviews and work samples were viewed more positively than biodata, cognitive ability tests, and personality tests, which were viewed more positively than honesty tests and graphology. However, the finding that Americans viewed personal contacts as more positive than Vietnamese, while Vietnamese found written ability tests more favorable than Americans reveals that cultural differences do exist. Another important finding was that the perceived face validity and legality of the selection method were the strongest fairness predictors of process favorability—apparently no matter where you go the test should still look and feel appropriate to the job.
Overall, organizations should be mindful of the culture they’re moving to, but using more favorable methods while making sure they are as face valid as possible is a best practice for any organization looking to select candidates overseas.
Hoang, T.G., Truxillo, D.M., Erdogan, B., & Bauer, T.N.. (2012). Cross-cultural
examination of applicant reactions to selection methods: United States and Vietnam.
International Journal of Selection and Assessment, 20(2), 209-219.
human resource management, organizational industrial psychology, organizational management
Unproctored Testing: Increasing the Applicant Pool to Combat Concerns with Cheating (IO Psychology)
Topic: Selection, Evidence Based Management, Assessment
Publication: International Journal of Selection and Assessment (JUN 2012)
Article: Offsetting Performance Losses Due to Cheating in Unproctored Internet-Based Testing by Increasing the Applicant Pool
Authors: Richard N. Landers & Paul R. Sackett
Reviewed By: Thaddeus Rada
Unproctored Internet testing (UIT) has been a hot topic in IO psychology over the past several years. In a nutshell, UIT allows organizations to post some of their selection tests online, allowing applicants to access them from virtually anywhere, so they can complete them on their own time. Some of the research on UIT has confirmed its strengths, such as its accessibility and efficiency, but other research has highlighted some of its limitations; in particular, there remains widespread concern about cheating in UIT. Because UIT is unproctored (it’s right there in the name), applicants are not under any supervision when they take such tests, so it’s possible that cheating could occur in a wide variety of ways.
While many authors have debated the extent to which cheating in UIT is a problem (e.g., how prevalent it is, how to cope with it, etc.), Landers and Sackett argue that if UIT increases the size of the applicant pool, even if some of these individuals do cheat, the organization may still derive benefit from a UIT program. Their logic goes something like this: if an organization is only looking to hire a set number of people (e.g., 50), then increasing the size of the applicant pool allows the organization to increase the cut score that an individual needs to exceed in order to be hired, thus making it harder for cheaters to get job offers. To test their ideas, Landers and Sackett conducted a computer simulation, which confirmed their belief in the power of a large applicant pool to compensate for some of UIT’s limitations.
While it does not remove all the concerns that exist about UIT, Landers and Sackett’s study does demonstrate that some of UIT’s benefits outweigh its limitations. This study’s findings hinge, to a large extent, on the capacity of a UIT intervention to increase the size of the applicant pool. As such, the authors suggest that practitioners carefully assess whether or not they anticipate that this will be the case before recommending that an organization adopt UIT.
human resource management, organizational industrial psychology, organizational management
A Sequel to the Ring of Fire: How Internal and External Candidates React to Employment Testing (I/O Psychology)
Topic: Assessment, Personality Assessment, Selection
Publication: International Journal of Selection and Assessment (JUN 2012)
Article: Don’t you know me well enough yet? Comparing reactions of internal and external candidates to employment testing
Authors: G. W. Giumetti and E. F. Sinar
Reviewed By: Megan Leasher
Employment testing is gaining in popularity at all levels within organizations, leading internal candidates to complete assessment tests to be considered for promotion or lateral moves. When you test employees you already hired, you might expect some pushback!
When competing for the same job, do internal and external candidates react differently to employment tests? Gary Giumetti and Evan Sinar explored this question with over 2300 candidates across 12 organizations and found several key differences. As compared to external candidates, internal candidates reported lower perceptions regarding information they received about what to expect in the tests, but held more positive views on both the job- relatedness of the tests and their overall recommendation of the organization to others.
This study focused on candidates’ perceptions of the tests themselves immediately after completing the tests. But what if you asked them later on? As a practitioner, the article led me to a spiraling of even more questions; all of which could impact applicant reactions at various points in the hiring process. How were the scores on the tests used? Was there a hard cutoff that required a “passing” score (on one or more of the tests) to advance in the hiring process? If so, were exceptions ever made because of something else amazing in a candidate’s background? What were candidates told about the hiring process and how their test scores were used? Did any of the internal candidates take the same or similar tests when they vied for
their current job? What had internal candidates heard, if anything, about the ease or difficulty of the tests from their coworkers? Were interviewers made aware of their test scores?
When it comes to any hiring process, consistency in both process and communication is paramount, no matter who comprises the applicant pool. An organization can choose to be transparent, opaque, or somewhere in the middle in what they share about a hiring process to candidates. But they need to be consistent to ensure that all applicant pools are treated justly.
human resource management, organizational industrial psychology, organizational management
Job-Relevant Prediction: Cognitive Ability Tests with High Criterion and Content Validity
Topic: Selection
Publication: International Journal of Selection and Assessment (MAR 2012)
Article: Cognitive Tests Used in Selection Can Have Content Validity as Well as Criterion Validity: A Broader Research Review and Implications for Practice
Authors: Frank L. Schmidt
Reviewed By: Thaddeus Rada
It is a well-known fact in IO psychology that cognitive ability is the one of the single best predictors of job performance, in a vast array of occupations. As such, cognitive ability tests are commonly created and used as a personnel selection tool for organizations. Such tests are typically validated using a criterion-related validity strategy, meaning that the usefulness of the test for predicting subsequent job performance is assessed. However, content validity, another important “type” of validity that refers to the extent to which a test can be considered to adequately sample the domain of interest, is often ignored in the validation process. In a recent article, Frank Schmidt argues that both types of validity can (and should) be assessed when creating a new cognitive ability measure.
Schmidt suggests that, in addition to taking advantage of the existing literature concerning the predictive power of various selection methods, practitioners should conduct well-designed content validation studies when creating a cognitive ability test for an organization. Such a validation would typically consist of a job analysis to identify both the narrower cognitive skills and broader learning capabilities that employees need to be successful. A test can then be designed that will assess cognitive ability using questions that are related to the actual mental qualities that an employee will be using on the job if they are hired.
In summary, Schmidt argues that a content validity model can be applied to cognitive ability tests in addition to a criterion-related validity model. As Schmidt notes, the strengths of this framework are considerable, in that a cognitive ability test validated in this way is more legally defensible and professionally satisfying. In addition, applicant reactions to a content-valid cognitive ability test are likely to be higher than they would be a more generic cognitive ability test that is not built with an eye towards the job in question.
human resource management, organizational industrial psychology, organizational management
Are Five Choices Better Than Three? (IO Psychology)
Topic: Selection, Testing
Publication: International Journal of Selection and Assessment
Article: The three option format for knowledge and ability multiple-choice tests: A case
for why it should be more commonly used in personnel testing
Authors: Edwards, B. D. Arthur, W. Jr., and Bruce, L. L.
Reviewer: Neil Morelli
When it comes to deciding how many response options should be given on a multiple choice test, many might argue that three versus four or five options is splitting hairs. But, Edwards, Arthur, and Bruce would argue this issue is a perfect example of the gap between science and practice in I-O psychology.
According to their review of the response option literature, most of the previous studies on this issue have discovered that three-response options is preferable for a number of practical and empirical reasons. For example, tests with three response options take less time to complete, are less susceptible to “testwise” test takers, take less time and resources to develop, and have equal psychometric characteristics like reliability, discriminability, and difficulty. Similar measurement characteristics and lower time requirements mean more bang for the buck when administering these tests. In other words, test takers can be tested on a wider body of knowledge when using three options.
Given that three-option tests measure knowledge or traits just like their four and five-option
cousins with less hassle, the authors questioned why practitioners have yet to broadly accept three-option tests. Guessing that practitioners still have concerns about test-taker reactions, criterion-related validity, and subgroup differences, the authors tested the assumption that there are no differences among the response-option types on these variables.
Edwards et al. gave groups of students either three-option or five-option items from the ACT college-entry test and compared students’ perceptions, race and age differences, and the test score relationship with GPA. Not only were the three-option equivalent to the five-option on their measurement characteristics, three-option tests were perceived to be just as fair, had similar correlations with GPA, and did not differ on their magnitude of race or gender-based differences. Perhaps these findings can help put practitioner fears to rest when considering the practical, financial, and administrative benefits of three-option multiple choice tests.
human resource management, organizational industrial psychology, organizational management
Do You IQ? The Impact of Individual Differences on Perceptions of Cognitive Ability Testing (IO Psychology)
Topic: Personality, Selection
Publication: International Journal of Selection and Assessment (MAR 2012)
Article: The Role of Person Characteristics in Perceptions of the Validity of Cognitive Ability Testing
Authors: Reeder, M. C., Powers, C. L., Ryan, A. M., & Gibby, R. E.
Reviewed By: Thaddeus Rada
It will come as no surprise to many practitioners in IO psychology that cognitive ability tests remain a common selection test, used by many organizations when evaluating applicants. This usage is due in no small part to the effectiveness of these tests in predicting future job performance; however, cognitive ability tests can also have some undesirable outcomes, including the fact some applicants have negative perceptions of these tests. Although the fact that different applicants can have very different reactions to cognitive ability tests is well-recognized, we know relatively little about how individual differences impact these perceptions. A recent study by Matthew Reeder and colleagues begins to address this gap in the literature.
Reeder and his colleagues examined the influence of several individual differences on perceptions of two different cognitive ability tests (one mechanical reasoning test and one numerical reasoning test). Using a hybrid sample of both students and unemployed adults, the authors hypothesized that variables such as prior test experience, prior job experience, and prior selection system success (among other variables) would be positively related to perceptions of cognitive ability measures. The authors also hypothesized interaction effects, including an effect of locus of control on the relationship between self-assessed performance and test-taker perceptions (with lowest perceptions belonging to individuals with low self-assessed performance and an external locus of control).
Overall, the authors found support for the idea that these individual difference variables can impact individuals’ perceptions of cognitive ability tests, though some variables, such as prior selection system success, were only predictive of reactions to one of the tests. The authors point out that continued research should be conducted on the impact of individual differences and individuals’ implicit theories on their perceptions of selection systems. While research in this area continues, it is worth remembering that different people can have very different perceptions of cognitive ability tests, and appreciating all of these individuals’ perceptions is important.
human resource management, organizational industrial psychology, organizational management
The Power of Hope: Relevant to IO Psychology?
Topic: Selection
Publication: International Journal of Selection and Assessment (MAR 2012)
Article: Hope in Personnel Selection
Authors: Zysberg, L.
Reviewed By: Thaddeus Rada
Although practitioners in IO psychology commonly measure and examine a wide range of constructs in their work, “hope” has not typically been among them. However, a new paper by Leehu Zysberg aims to better understand the relationship of this personality trait (which has its roots in the positive psychology movement) to organizational psychology, specifically the personnel selection process.
As defined by Zysberg, hope can be conceptualized as a motivational factor, consisting of both emotional and cognitive components. In connecting hope to personnel selection, Zysberg hypothesized that hope would influence individuals’ coping styles; to the extent that an individual’s coping style was problem-solving-oriented, Zysberg hypothesized that performance on several selection assessments (e.g. a cognitive ability test, personality assessment, interview, etc.) would be enhanced. Said another way, Zysberg’s main hypothesis was that higher levels of hope would promote problem-oriented coping, and that this coping style would enhance performance on a variety of selection assessments, thus leading to a higher likelihood of a hiring recommendation.
Overall, structural equation modeling provided partial support for the author’s hypotheses; specifically, hope was significantly related to some, but not all, of the selection assessments, while coping was also found to have a direct relationship with hiring recommendation (i.e. without the mediating role of performance on the selection assessments).
Certainly, a great deal more research will be needed to identify what place, if any, hope has in evidence-based human resource management. Zysberg provides a first step in understanding hope and raising awareness for this construct; additional research will help us to understand if hope is relevant for prediction in IO psychology.
human resource management, organizational industrial psychology, organizational management
Do You Test? Factors Impacting the Use of Specific Selection Methods in Hiring Procedures (IO Psychology)
Topic: Selection
Publication: International Journal of Selection and Assessment (DEC 2011)
Article: Selection Practices in Canadian Firms: An Empirical Investigation
Authors: Mann, S. L., & Chowhan, J.
Reviewed By: Thaddeus Rada
A persistent paradox in IO psychology is the frequent use, by organizations, of selection methods that research has not shown to be effective for successfully hiring employees (e.g. unstructured interviews), in conjunction with the frequent underuse of methods that research has shown to be effective (e.g. various paper-and-pencil tests). Although this trend is well-established, relatively little is known about the factors that contribute to this science-practice gap. However, a new study by Sara Mann and James Chowhan takes some steps towards increasing our understanding in this area.
Using data from a large number of employees in a wide array of Canadian organizations, the authors investigated the predictive role that a number of organizational and situational characteristics might have in determining which selection methods were used (a personality test, a job-related knowledge test, or an interview). Among their findings, the authors found that having an in-house HR department was a significant predictor for all three selection methods. The authors also found that nonprofit organizations were more likely to use an interview (while for-profit organizations were more likely to use a personality test), and that the application process for permanent (as opposed to temporary) positions was more likely to contain an interview. On the other hand, the number of employees that an organization had did not predict use of any of the three selection methods.
While the results of this study were generated using only Canadian employees, the employees came from a wide range of occupations and organizational settings; as such, it is likely that the authors’ findings should generalize to other country’s organizations and employees quite well. Practitioners might use the results of this study to better understand the organizations that would be particularly likely to incorporate tests into their selection practice, such as those organizations with an in-house HR department and those with unionized employees.
human resource management, organizational industrial psychology, organizational management
source for picture: http://www.freedigitalphotos.net/images/search.php?search=canada&cat=
