Category Archives: Selection
Making personality tests better for selection (IO Psychology)
Topic: Personality, Selection
Publication: Journal of Applied Psychology (AUTUMN 2012)
Article: A matter of context: A meta-analytic investigation of the relative validity of contextualized and noncontextualized personality measures
Authors: J. A. Shaffer & B. E. Postlethwaite
Reviewed by: Alexandra Rechlin
Whether or not you agree with it, your organization likely uses personality assessments as part of the selection process. Personality assessments do appear to be valid predictors of job performance, but can we do anything to make them be even better predictors? Recent research indicates that the answer is yes.
Jonathan Shaffer and Bennett Postlethwaite conducted a meta-analysis that compared generic (noncontextualized) personality measures with those that were presented in a work context (contextualized). For example, noncontextualized personality measures might include an item such as “I am on good terms with everyone,” whereas a contextualized version might be “I am on good terms with everyone at work.” The authors found that contextualized personality assessments were much better predictors of job performance than were noncontextualized assessments.
Making personality assessments contextualized is incredibly easy; it’s as simple as adding “at work” to the end of each item. This study found that contextualized measures designed for general use were just as good as those designed for use in the workplace, so you can use a publicly available measure and make it contextualized to improve its validity. Finally, one of the best implications of using contextualized measures is that they will be better accepted by applicants, as it’s clearer to the applicant how the personality tests are relevant to the workplace.
human resource management, organizational industrial psychology, organizational management
Can’t we just get along? Team personality and conflict (IO Psychology)
Topics: Teams, Personality, Selection
Publication: Journal of Applied Psychology (SEP 2012)
Article: Ready to rumble: How team personality composition and task conflict interact to
improve performance.
Authors: Bret H. Bradley, Anthony C. Klotz, Bennett E. Postlethwaite, & Kenneth G. Brown
Reviewed By: Aaron Manier
Team members need to get along in order to perform well. Unfortunately, we’re all different people, so sometimes conflict arises. Often this conflict arises around different takes on the team’s task. However, scientific understanding of the relationship between task conflict and effective team performance has been inconclusive.
Personality impacts team dynamics and processes. Specifically, openness to experience and emotional stability can help or hinder team communication and conflict resolution. Team members who are open to experience are generally open-minded and curious, resulting in greater adaptability and a willingness to discuss conflict openly. Members with emotional stability have a steady sense of composure and generally have a positive view of others that allows them to use others effectively in conflict resolution.
Teams with members high in emotional stability and openness to experience perform stronger in the face of task conflict than teams with members low in these personality characteristics. Because of these findings, management should consider personality when building teams for unique tasks. Employees with high levels of emotional stability and openness will be able to tackle non-routine, challenging tasks with more grace and dignity, effectively resolving task conflict as it arises.
Makes sense, right? Who wants to work with a neurotic, closed-minded team member? Unless you’re just into that kind of thing.
Bradley, B. H., Klotz, A. C., Postlethwaite, B. E., & Brown, K. G. (2012). Ready to rumble:
How team personality composition and task conflict interact to improve performance.
Journal of Applied Psychology, Advance online publication. Doi: 10.1037/a0029845
human resource management, organizational industrial psychology, organizational management
Score one for standardized tests!
Topic: Selection
Publication: Psychological Bulletin (2001)
Article: A comprehensive meta-analysis of the predictive validity of the graduate record examinations: Implications for graduate student selection and performance
Authors: N. R. Kuncel, S. A. Hezlett, & D. S. Ones
Reviewed By: Scott Charles Sitrin, M.A.
Have you taken the Graduate Record Examinations (GRE)? This test is like the SAT on steroids, and it, just like the SAT, is thought to predict academic achievement. Unlike the SAT, the GRE is thought to predict academic performance in graduate school and not college. So, if you have taken the GRE, you may be aware the bone-chilling anxiety that it can induce, as it will determine which graduate school you get into and if you get into graduate school at all. For those of you who have not had the pleasure of taking this test, imagine working really hard in college in order to get good grades in the hopes of attending graduate school. While your friends were at the bar, you were in a cubicle in the library next to the guy whom thinks that a bag of Sun Chips is an acceptable snack for a quiet environment. Now, after four years of sacrifice, you have to take one more test, and the scores, at least in the eyes of some graduate programs, is more important than your academic achievement over the past four years. For a test with such significance, the question becomes, “does it work?”
The Graduate Record Examination (GRE) is a set of standardized tests that assess verbal, quantitative, and analytic capacities as well as knowledge of a particular field, such as psychology. In examining its validity, Kuncel, Hezlett, and Ones performed a massive meta-analysis that analyzed 1,753 samples and over 80,000 graduate students. They wanted to know the relationship between GRE scores and graduate school performance as indicated by graduate grade point average (GPA), 1st-year graduate GPA, scores on the comprehensive exam, number of publications, and faculty ratings. In short, the results indicated that the GRE is a valid predictor of graduate school performance, as it had a relationship with all of the performance indicators.
In relating these results to other domains, such as business, it appears that there is a role for standardized tests in predicting performance. What that role should be and how much reliance is placed on the instrument should be examined on a case-by-case basis. As other studies have indicated, other variables, such as personality and intelligence, also appear to predict performance, and in an ideal setting, all of these characteristics in addition to standardized tested should be utilized in trying to predict how well someone is going to do.
human resource management, organizational industrial psychology, organizational management
source for picture: http://www.freedigitalphotos.net/images/Education_g314-Test_p13636.html
Are you getting enough sleep?
Topic: Selection
Publication: Sleep (2011)
Article: The effects of sleep extension on the athletic performance of collegiate basketball players
Authors: C. D. Mah, K. E. Mah, E. J. Kezirian, & W. C. Dement
Reviewed By: Scott Charles Sitrin, M.A.
It’s 6:00 a.m., and the shrill of the alarm fills your room. You are now faced with one of the biggest decision of the day, and it must be decided whether you should hit the snooze for 15 more heavenly minutes or pull yourself out of bed and face the day. If you get up, you may be able to get more done during the day. In contrast, if you snooze, it will feel really good. What if you could have the best of both worlds, and it would be possible to not only sleep more but also perform better. This magical combination may exist
In looking at the relationship between sleep and performance, Mah, Mah, Kezirian, and Dement investigated the amount of sleep and performance of 11 NCAA Division I collegiate basketball players. The baseline measure of the typical sleep-wake cycles of the athletes was taken for two to four weeks, and the athletes were then encouraged to sleep as much as possible and for at least 10 hours per night. Daily sleep logs and journals as well as actigraphy, a method that utilizes a device worn on the subject’s wrist, assessed daily sleep-wake activity. The Epworth Sleepiness Scale measured daytime sleepiness and the Profile of Mood States examined mood states. The performance indicators were speed as determined by a timed, 282 feet sprint; free throw and three-point shooting accuracy; reaction time as measured by the Psychomotor Vigilance Task; and subjective responses of physical and mental well being. Results indicated that increases in the amount of sleep increased athletic performance. Specifically, with more sleep, the players had a faster timed sprint, improved shooting accuracy, a better mood, decreased fatigue, improved reaction time, and increased subjective ratings of physical and mental well-being.
So, hit that snooze button, as increased sleep appears to improve performance. Not only was the physical performance of the athletes improved, but the psychological well being was too. The athletes were in a better mood, moved faster, felt less fatigued, and had an improved overall sense of well being. These improvements would seem to better not only athletic organizations, but organizations as a whole. If you were a manager, would you appreciate it if your workforce smiled a bit more and completed tasks quicker? There’s no guarantee, but it appears that burning the midnight oil may not be all it’s cracked up to be.
human resource management, organizational industrial psychology, organizational management
Smile, you may pitch better
Topic: Selection
Publication: Journal of Applied Sport Psychology (2003)
Article: The relationship between emotional intelligence and performance among college
baseball players
Authors: S. Zizzi, H. Deaner, & D. Hirschhorn
Reviewed By: Scott Charles Sitrin, M.A.
Emotional intelligence refers to the capacity to recognize and use emotions. If I smile, do you recognize this as happiness? Or do you believe that me laughing means that I am sad? If you knew that a smile typically indicates a state of happiness, give yourself a gold star. Now that you are the Albert Einstein of recognizing emotions, can you use them? For instance, if you want to non-verbally convey how you are feeling, can you alter your body language so as to communicate your mood? If so, two gold stars for you. Since recognizing and using emotions seems important to the performance of everyday activities and communications, does recognizing and using emotions affect performance in other domains such as collegiate sports?
In investigating the relationship between emotional intelligence and athletic performance,
Zizzi, Deaner, and Hirschhorn examined the emotional intelligence and performance of 61
NCAA Division I collegiate baseball players. The players were divided into pitchers, 21,
and hitters, 40. The Emotional Intelligence Scale – a 33-item measure that is comprised of
the domains of appraisal and expression of emotion, regulation of emotion, and utilization of emotion – assessed emotional intelligence. The performance indicators of the pitchers were the number of earned runs, walks, hits, strikeouts, and wild pitches; and the performance indicators of the hitters were the number of hits, doubles, walks, and strikeouts. Results indicated that emotional intelligence was positively correlated with the total number of strikeouts by pitchers, but it was not related to any of the other indicators of pitching performance or any of the indicators of hitting performance.
Hmm, the results are less than clear. Emotional intelligence appears to predict
performance, but it only does so with one out of the nine performance indicators; it appears that the jury is still out. If I were a gambling man, I would bet that emotional intelligence not only predicts the performance of collegiate athletes, but also individuals in other settings such as the boardroom. Since emotional intelligence appears vital to effective communication, and communication is an important aspect of most tasks that involve more than one person, it should follow that the more emotionally intelligence someone is, the better they will be able to communicate, and the more effective and efficient they will perform. At least that’s what I think.
Zizzi, S., Deaner, H., & Hirschhorn, D. (2003). The relationship between emotional
intelligence and performance among college baseball players. Journal of Applied Sport
Psychology, 15:3, 262-269.
human resource management, organizational industrial psychology, organizational management
What are those emerging markets thinking? (IO Psychology)
Topic: Selection, Assessment
Publication: International Journal of Selection and Assessment (JUN 2012)
Article: Cross-cultural Examination of Applicant Reactions to Selection Methods: United States and Vietnam
Authors: Hoang, T.G., Truxillo, D.M., Erdogan, B., and Bauer, T.N.
Reviewer: Neil Morelli
Over the past several years I-O psychologists have become more interested in understanding applicant reactions to selection tools. Of course we still care about the reliability and validity of the selection tool, but we know that how an applicant reacts to the process could influence how fair or trusting the company is perceived to be, how well the newly hired employee adjusts to the job, or how vulnerable the selection tools are to legal challenge. And, as organizations become more global and enter emerging markets, it’s important to understand what the reactions of people in these new, unexamined candidate pools may be.
A good example of an emerging market is Vietnam—economic reports show that companies are moving operations to Vietnam in greater numbers and Vietnam’s economy is expected to become the 17th largest by 2025. To understand the potential reactions to selection methods in a country with very different cultural, legal, and socioeconomic conditions, the authors compared the reactions of 225 Vietnamese business students to 151 American business students. Examples of the selection methods included interviews, work samples, resumes, biodata, and graphology, while the specific reactions were process favorability and fairness (i.e.,
face validity, personal privacy, and legality).
Hoang et al. discovered that perceptions in either country were fairly similar to each other: interviews and work samples were viewed more positively than biodata, cognitive ability tests, and personality tests, which were viewed more positively than honesty tests and graphology. However, the finding that Americans viewed personal contacts as more positive than Vietnamese, while Vietnamese found written ability tests more favorable than Americans reveals that cultural differences do exist. Another important finding was that the perceived face validity and legality of the selection method were the strongest fairness predictors of process favorability—apparently no matter where you go the test should still look and feel appropriate to the job.
Overall, organizations should be mindful of the culture they’re moving to, but using more favorable methods while making sure they are as face valid as possible is a best practice for any organization looking to select candidates overseas.
Hoang, T.G., Truxillo, D.M., Erdogan, B., & Bauer, T.N.. (2012). Cross-cultural
examination of applicant reactions to selection methods: United States and Vietnam.
International Journal of Selection and Assessment, 20(2), 209-219.
human resource management, organizational industrial psychology, organizational management
Discrimination in selection: Who’s most at risk? (IO Psychology)
Topic: Selection, Discrimination
Publication: Journal of Organizational Behavior (MAY 2012)
Article: Multiple categorization in resume screening: Examining effects on hiring discrimination against Arab applicants in field and lab settings
Authors: Eva Derous, Ann Marie Ryan, & Hannah-Hanh D. Nguyen
Reviewed by: Alexandra Rechlin
You’re probably aware that discrimination can occur during selection. However, in recent years, various predictions have been made regarding who is most likely to be discriminated against, and why. The multiple minority status hypothesis (MMS) posits that someone who is a member of more than one minority group (e.g., an Arab woman in the United States) is more likely to be discriminated against than someone who is only part of one minority group (e.g., an Arab man in the U.S.). Another perspective is the ethnic prominence hypothesis (EP), which suggests that numerical minority status (in other words, women are not counted as a minority) leads to stereotyping. In a series of recent studies in the Netherlands, Eva Derous and her colleagues studied discrimination against Arabs and tested the MMS and EP perspectives.
In these studies, Arabs were discriminated against in selection practices, supporting the EP hypothesis. Arabs were much more likely to be rejected based on resumes that differed only in the name of the applicants, and not in experience or other qualifications. However, raters who were motivated to control their prejudice did not show discrimination based on the ethnicity of the applicant. Arabs were discriminated against more when raters’ prejudice was high, as would be expected. In addition, when the job was stereotypically held by white men (i.e., HR manager), Arab women were discriminated against more than Arab men, supporting the MMS hypothesis.
The authors found support for the ES hypothesis for lower-status jobs and support for the MMS hypothesis for higher-status jobs. In other words, discrimination may depend on how the applicant, job, and recruiter characteristics interact.
human resource management, organizational industrial psychology, organizational management
Unproctored Testing: Increasing the Applicant Pool to Combat Concerns with Cheating (IO Psychology)
Topic: Selection, Evidence Based Management, Assessment
Publication: International Journal of Selection and Assessment (JUN 2012)
Article: Offsetting Performance Losses Due to Cheating in Unproctored Internet-Based Testing by Increasing the Applicant Pool
Authors: Richard N. Landers & Paul R. Sackett
Reviewed By: Thaddeus Rada
Unproctored Internet testing (UIT) has been a hot topic in IO psychology over the past several years. In a nutshell, UIT allows organizations to post some of their selection tests online, allowing applicants to access them from virtually anywhere, so they can complete them on their own time. Some of the research on UIT has confirmed its strengths, such as its accessibility and efficiency, but other research has highlighted some of its limitations; in particular, there remains widespread concern about cheating in UIT. Because UIT is unproctored (it’s right there in the name), applicants are not under any supervision when they take such tests, so it’s possible that cheating could occur in a wide variety of ways.
While many authors have debated the extent to which cheating in UIT is a problem (e.g., how prevalent it is, how to cope with it, etc.), Landers and Sackett argue that if UIT increases the size of the applicant pool, even if some of these individuals do cheat, the organization may still derive benefit from a UIT program. Their logic goes something like this: if an organization is only looking to hire a set number of people (e.g., 50), then increasing the size of the applicant pool allows the organization to increase the cut score that an individual needs to exceed in order to be hired, thus making it harder for cheaters to get job offers. To test their ideas, Landers and Sackett conducted a computer simulation, which confirmed their belief in the power of a large applicant pool to compensate for some of UIT’s limitations.
While it does not remove all the concerns that exist about UIT, Landers and Sackett’s study does demonstrate that some of UIT’s benefits outweigh its limitations. This study’s findings hinge, to a large extent, on the capacity of a UIT intervention to increase the size of the applicant pool. As such, the authors suggest that practitioners carefully assess whether or not they anticipate that this will be the case before recommending that an organization adopt UIT.
human resource management, organizational industrial psychology, organizational management
A Sequel to the Ring of Fire: How Internal and External Candidates React to Employment Testing (I/O Psychology)
Topic: Assessment, Personality Assessment, Selection
Publication: International Journal of Selection and Assessment (JUN 2012)
Article: Don’t you know me well enough yet? Comparing reactions of internal and external candidates to employment testing
Authors: G. W. Giumetti and E. F. Sinar
Reviewed By: Megan Leasher
Employment testing is gaining in popularity at all levels within organizations, leading internal candidates to complete assessment tests to be considered for promotion or lateral moves. When you test employees you already hired, you might expect some pushback!
When competing for the same job, do internal and external candidates react differently to employment tests? Gary Giumetti and Evan Sinar explored this question with over 2300 candidates across 12 organizations and found several key differences. As compared to external candidates, internal candidates reported lower perceptions regarding information they received about what to expect in the tests, but held more positive views on both the job- relatedness of the tests and their overall recommendation of the organization to others.
This study focused on candidates’ perceptions of the tests themselves immediately after completing the tests. But what if you asked them later on? As a practitioner, the article led me to a spiraling of even more questions; all of which could impact applicant reactions at various points in the hiring process. How were the scores on the tests used? Was there a hard cutoff that required a “passing” score (on one or more of the tests) to advance in the hiring process? If so, were exceptions ever made because of something else amazing in a candidate’s background? What were candidates told about the hiring process and how their test scores were used? Did any of the internal candidates take the same or similar tests when they vied for
their current job? What had internal candidates heard, if anything, about the ease or difficulty of the tests from their coworkers? Were interviewers made aware of their test scores?
When it comes to any hiring process, consistency in both process and communication is paramount, no matter who comprises the applicant pool. An organization can choose to be transparent, opaque, or somewhere in the middle in what they share about a hiring process to candidates. But they need to be consistent to ensure that all applicant pools are treated justly.
human resource management, organizational industrial psychology, organizational management
Integrity Tests May Have Lower Performance Validity (IO Psychology)
Topic: Selection, Measurement
Publication: Journal of Applied Psychology
Article: The Criterion-Related Validity of Integrity Tests: An Updated Meta-Analysis
Authors: Van Iddekinge, C.H., Roth, P.L., Raymark, P.H., & Odle-Dusseau, H.N.
Reviewer: Neil Morelli
According to a recent meta-analysis by Van Iddekinge and colleagues, integrity tests may not be as predictive of job performance as once thought. Integrity tests have become popular with organizations and practitioners due to their high correlations with job performance and few differences between groups (based on race, gender, etc.). But, Van Iddekinge et al. were concerned that past meta-analytic results drew too heavily on unpublished studies authored by test publishers. In fact, only 10% of one meta-analysis’ sample was made up of studies published in peer-reviewed journals (pro-tip: we like things that are peer reviewed).
The authors used 104 studies (42 were published and 62 were unpublished) to investigate if including more “neutral” primary studies whose methodology has been more rigorously vetted would change the test’s validity. They reported that the overall job performance validity is .13 to .16 depending on whether it’s corrected for unreliability. In other words, this validity coefficient is much lower than originally reported and indicates that the integrity test is not as predictive of job performance as once thought. Although the test has a higher validity coefficient for counterproductive work behaviors (.26 to .32), this is still lower than originally reported in previous meta-analyses.
So what does this all mean? On the one hand integrity tests are still viable options for
predicting counterproductive work behaviors while maintaining low sub-group differences. On the other hand, integrity tests’ predictive validity is weaker than previously thought and practitioners may not be able to rely on meta-analytic results in lieu of a local validation study. The authors pointed out that one could argue test publisher data is overly optimistic while data from independent researchers is overly pessimistic. Regardless of your position the authors suggest that practitioners should consider the source when reporting integrity test validity and researchers may need to develop more primary studies on the standard integrity test’s true capability to predict future job performance.
Van Iddekinge, C.H., Roth, P.L., Raymark, P.H., & Odle-Dusseau, H.N. (2012). The
criterion-related validity of integrity tests: An updated meta-analysis. Journal of Applied
Psychology, 97, 499-530.
human resource management, organizational industrial psychology, organizational management
