Category Archives: Selection

So you think they are a bunch of dumb jocks?

Reply

Topic: Selection
Publication: PLoS ONE (2012)
Article: Executive functions predict the success of top-soccer players
Authors: T. Vestberg, R. Gustafson, L. Maurex, M. Ingvar, & P. Petrovic
Reviewed By: Scott Charles Sitrin, M.A.

What do sport announcers mean when they say that an athlete does or does not have
the mental game? Personally, I think that announcers use this term as a catch-all to describe any attribute or characteristic that is not physical or visually perceivable. This vagueness may represent the lack of understanding of the exact mental characteristics and abilities that are necessary for elite sport performance. The exploration of these characteristics and abilities has been limited at best, and more resources are being devoted to formal investigations as of late.
In Sweden, a study was recently published in which the investigators assessed the
cognitive abilities of 57 male and female professional, Swedish soccer players. The cognitive tests measured the ability to visually problem solve in a timed setting, the capacity to inhibit impulses, and the ability to remember visual images and patterns. It was found that soccer players in the most elite division in Sweden had higher cognitive abilities than players in lower divisions, and that both groups performed better than a control group. Further, a positive correlation between cognitive ability and the number of goals scored was found. In other words, the higher the cognitive ability, the more goals scored, and vice versa.
Employers might want to consider this information when deciding which employees to
select. Though neuropsychological tests can be expensive and cost thousands of dollars for each assessment, the money spent may be less than the money saved. Costs associated with a bad hire include that employee’s salary and the potential decrease in production associated with that employee’s less than stellar performance. After all, an ounce of prevention can be worth more
than a pound of cure.

Vestberg, T., Gustafson, R., Maurex, L., Ingvar, M., & Petrovic, P. (2012). Executive functions predict the success of top-soccer players. PLoS ONE 7(4): e34731. doi:10.1371/journal.pone.0034731

human resource management, organizational industrial psychology, organizational management

 


source for picture: http://www.freedigitalphotos.net/images/Healthy_Living_g284Fit_Man_Drinking_Water_Isolated_On_White_p81268.html

Job-Relevant Prediction: Cognitive Ability Tests with High Criterion and Content Validity

Reply

Topic: Selection
Publication: International Journal of Selection and Assessment (MAR 2012)
Article: Cognitive Tests Used in Selection Can Have Content Validity as Well as Criterion Validity: A Broader Research Review and Implications for Practice
Authors: Frank L. Schmidt
Reviewed By: Thaddeus Rada

It is a well-known fact in IO psychology that cognitive ability is the one of the single best predictors of job performance, in a vast array of occupations. As such, cognitive ability tests are commonly created and used as a personnel selection tool for organizations. Such tests are typically validated using a criterion-related validity strategy, meaning that the usefulness of the test for predicting subsequent job performance is assessed. However, content validity, another important “type” of validity that refers to the extent to which a test can be considered to adequately sample the domain of interest, is often ignored in the validation process. In a recent article, Frank Schmidt argues that both types of validity can (and should) be assessed when creating a new cognitive ability measure.

Schmidt suggests that, in addition to taking advantage of the existing literature concerning the predictive power of various selection methods, practitioners should conduct well-designed content validation studies when creating a cognitive ability test for an organization. Such a validation would typically consist of a job analysis to identify both the narrower cognitive skills and broader learning capabilities that employees need to be successful. A test can then be designed that will assess cognitive ability using questions that are related to the actual mental qualities that an employee will be using on the job if they are hired.

In summary, Schmidt argues that a content validity model can be applied to cognitive ability tests in addition to a criterion-related validity model. As Schmidt notes, the strengths of this framework are considerable, in that a cognitive ability test validated in this way is more legally defensible and professionally satisfying. In addition, applicant reactions to a content-valid cognitive ability test are likely to be higher than they would be a more generic cognitive ability test that is not built with an eye towards the job in question.

Schmidt, F. L. (2012). Cognitive tests used in selection can have content validity as well as criterion validity: A broader research review and implications for practice. International Journal of Selection and Assessment, 20, 1-13.

human resource management, organizational industrial psychology, organizational management


source for picture: http://www.freedigitalphotos.net/images/Office_and_Stationer_g145-Pencil_And_Brown_Note_Paper_p30787.html

Are Five Choices Better Than Three? (IO Psychology)

Reply

Topic: Selection, Testing
Publication: International Journal of Selection and Assessment
Article: The three option format for knowledge and ability multiple-choice tests: A case
for why it should be more commonly used in personnel testing
Authors: Edwards, B. D. Arthur, W. Jr., and Bruce, L. L.
Reviewer: Neil Morelli

When it comes to deciding how many response options should be given on a multiple choice test, many might argue that three versus four or five options is splitting hairs. But, Edwards, Arthur, and Bruce would argue this issue is a perfect example of the gap between science and practice in I-O psychology.

According to their review of the response option literature, most of the previous studies on this issue have discovered that three-response options is preferable for a number of practical and empirical reasons. For example, tests with three response options take less time to complete, are less susceptible to “testwise” test takers, take less time and resources to develop, and have equal psychometric characteristics like reliability, discriminability, and difficulty. Similar measurement characteristics and lower time requirements mean more bang for the buck when administering these tests. In other words, test takers can be tested on a wider body of knowledge when using three options.

Given that three-option tests measure knowledge or traits just like their four and five-option
cousins with less hassle, the authors questioned why practitioners have yet to broadly accept three-option tests. Guessing that practitioners still have concerns about test-taker reactions, criterion-related validity, and subgroup differences, the authors tested the assumption that there are no differences among the response-option types on these variables.

Edwards et al. gave groups of students either three-option or five-option items from the ACT college-entry test and compared students’ perceptions, race and age differences, and the test score relationship with GPA. Not only were the three-option equivalent to the five-option on their measurement characteristics, three-option tests were perceived to be just as fair, had similar correlations with GPA, and did not differ on their magnitude of race or gender-based differences. Perhaps these findings can help put practitioner fears to rest when considering the practical, financial, and administrative benefits of three-option multiple choice tests.

Edwards, B. D. Arthur, W. Jr., & Bruce, L. L. (2012). The three option format for
knowledge and ability multiple-choice tests: A case for why it should be more commonly
used in personnel testing. International Journal of Selection and Assessment, 20(1), 65-
81.

human resource management, organizational industrial psychology, organizational management

 

source for picture: http://www.freedigitalphotos.net/images/Ideas_and_Decision_M_g409-Choices_p22024.html

What Does Your Credit Score Say About You? (IO Psychology)

Reply

Topic: Selection, Personality
Publication: Journal of Applied Psychology (2012)
Article: An Empirical Investigation of Dispositional Antecedents and Performance-
Related Outcomes of Credit Scores
Authors: Bernerth, J.B., Taylor, S.G., Walker, H.J. and Whitman, D.S.
Reviewer: Neil Morelli

You’ve no doubt heard the catchy jingles asking you to check your credit score. You may have also heard that a bad credit score could potentially cost you a new job, but have you wondered if companies should actually be looking at applicant credit scores? Recognizing that “60% of employers conduct credit checks of at least some of their new hires”, according to a recent SHRM poll, Bernerth and colleagues investigated if credit scores are indeed related to dispositional traits and job performance—as many organizations assume.

They specifically hypothesized that conscientiousness, task performance, and organizational citizenship behaviors (OCBs) would be positively related to employee credit scores, while neuroticism, extraversion, agreeableness, and workplace deviance would be negatively related.

After gathering personality information, FICO scores, and performance feedback from direct supervisors for over 100 people, Bernerth et al. discovered that credit scores were in fact positively related to conscientiousness, task performance, and OCBs, whereas agreeableness was negatively related as expected. Surprisingly, the authors did not find a relationship between extraversion, neuroticism, and workplace deviance, this suggested that people with a poor credit score may not deserve as much stigma as popularly thought.

So, while credit scores do seem to be a viable tool for screening applicants, the evidence does not support the generally held belief that applicants with poor credit scores are “up to no good.” However, the authors qualified this conclusion by suggesting different results might be found if self-report deviance data were available from the financial industry in particular. Another word of caution: although a credit score could be used as a selection tool it may also be subject to adverse impact on protected groups and negative impressions by applicants; thus, more research may be needed to determine if the practice is really worth it.
Bernerth, J.B., Taylor, S.G., Walker, H.J., & Whitman, D.S. (2012). An empirical investigation of dispositional antecedents and performance-related outcomes of credit scores. Journal of Applied Psychology, 97(2), 469-478.

human resource management, organizational industrial psychology, organizational management

source for picture: http://www.freedigitalphotos.net/images/Emotions_g96-Serious_Little_Boy_With_Folded_Arms_p76015.html

Why was Michael Jordan a Starter?

Reply

Topic: Selection
Publication: Athletic Insight – The Online Journal of Sport Psychology (2007)
Article: Predicting athletic success: Factors contributing to the success of NCAA Division I AA collegiate football players
Authors: M. Spieler, D. R. Czech, A. B. Joyner, B. Munkasy, N. Gentner, & J. Long  
Reviewed By: Scott Charles Sitrin, M.A.

What are the characteristics that allow an athlete to succeed?  The physical characteristics appear to include strength, speed, agility, and endurance.  The mental characteristics that enable an athlete to excel are less understood.

In addressing this void, the investigators assessed the demographic information, psychological characteristics, and athletic accomplishments of 108 male football players.  The demographic variables included age, height, weight, high school academic performance, and high school size; the psychological characteristics were neuroticism, extraversion, openness to experience, agreeableness, conscientiousness, and coping ability; and the primary athletic accomplishment was starting status.  It was found that starters and non-starters differed in age, high school size, and ability to cope with adversity.  Starters were older, came from a larger high school, and had a greater ability to cope.   

Employers might want to consider this information when deciding which employees to select.  Though age and high school size may not be pertinent to success of employees, the ability to cope with adversity likely is.  This characteristic appears to predict success across domains, from the gridiron to the boardroom.

Spieler, M., Czech, D. R., Joyner, A. B., Munkasy, B., Gentner, N., & Long, J. (2007). Predicting athletic success: Factors contributing to the success of NCAA Division I AA collegiate football players. Athletic Insight – The Online Journal of Sport Psychology, 9(2), 22-33.

human resource management, organizational industrial psychology, organizational management

 source for picture: http://www.freedigitalphotos.net/images/Indoor_Sports_g219-Basket_Ball_p11540.html

Making an A-Team? (Human Resource Management)

Reply

Topic: Selection, Talent Management
 Publication: Harvard Business Review (JAN 2012)
Article: Gilt Groupe’s CEO on Building a Team of A Players
Author: Kevin Ryan
Reviewed by: Liz Brashier

In a recent article by the CEO of the flash sales company the Gilt Groupe, Ryan (2012) discusses what makes a company truly successful. (Hint: it’s something we focus on the most!) According to Ryan, a business idea is worth next to nothing – without the right people to implement it. While most companies claim to put people first, Ryan asks a compelling question: “do most CEOs spend more time on recruiting and managing people than any other activity?” The answer is more often than not a “no,” and for this CEO, that’s a big problem. He also challenges CEOs to consider their relationship with the head of HR in their own companies, and to make sure that this person has a real seat at the executive table. The most important thing a CEO can do is build and maintain a top caliber “A-Team” of employees.

Ryan offers other key suggestions for building a high caliber team:

1) Add by subtracting: if employees are no longer productively contributing, then it’s time to let them go and bring new talent in. There is no where to put a stellar new hire if all the desks are occupied – not being utilized productively.
2) Check those references! According to Ryan, most hiring managers value the resume over the reference check, while checking a reference can often provide the most valuable insight into what this candidate’s work is like.
3) Make sure you understand why people choose to leave. Often, it’s because of a manager. If talented employees are choosing to leave, make sure that their manager understands an evaluation for him or her is up next.

The clear message from this article? Put your people first, especially when in the role of CEO. Obsessing over talent while serving as CEO could provide the greatest returns for the company.

Ryan, K. (2012). Gilt Groupe’s CEO on building a team of A players. Harvard Business Review, 90, 43-46.

human resource management, organizational industrial psychology, organizational management

 

source for picture: http://www.freedigitalphotos.net/images/Business_People_g201-Multi_Ethnic_Team_During_Meeting_p66399.html

Goldilocks and the Three Levels of Adversity (IO Psychology)

Reply

Topic: Selection
Publication: Current Directions in Psychological Science (2011)
Article: Resilience: A Silver Lining to Experiencing Adverse Life Events?
Authors: Mark Seery
Reviewed By: Scott Charles Sitrin, M.A.

What is the right amount of adversity? Is it best to have a carefree life that minimizes stress and allows the person to focus on his or her job? Could a life filled with adversity build character and enable the person to function better in an unpredictable work environment? Or is something in between the best choice, where there is not too little or too much adversity and the amount is just right?

In addressing this Goldilocks dilemma, Seery (2011) reviewed the literature on several studies that examined adversity, resilience, performance, and level of functioning. It was found that individuals who had a history of some adversity had better outcomes than those who had lots and minimal amounts of adversity. Employers might want to consider this information when deciding which employees to select. In addition to educational background, previous work experience, and letters of recommendation, an applica

A New Weapon in the Fight Against Faking on Personality Tests (IO Psychology)

2

Topic: Faking, Personality, Selection
Publication: Journal of Applied Psychology
Article: Testing the efficacy of a new procedure for reducing faking on personality tests within selection contexts
Authors: Fan, J. Gao., D., Carroll, S.A., Lopez, F.J., Tian, T.S., & Meng, H.
Reviewer: Neil Morelli

Has your organization ever used, or ever considered using a personality test as part of their selection battery? Due to personality tests’ predictive validity and relatively low subgroup differences, you’re not alone. However, one controversial issue still plagues the use of personality tests in selecting applicants: faking. Faking is defined as the intentional distortion of responses to portray a more positive image, and it can negatively affect the validity of the selection device. Fortunately, Fan et al. have recently tested a new method for identifying and reducing faking on personality tests that uses a computer-based warning system.

Fan et al. admits that there’s nothing new about warning applicants about faking, but the novel component of Fan et al.’s system is how the warning is provided. Instead of a reactive system for reducing faking via statistical controls, the Fan et al. method proactively mitigates faking by first testing for the likelihood of faking on an “initial item block” (this block consists of impression management items, a bogus scale, and a subset of items from the actual personality test). After comparing the scores from this block to a cutoff level for faking, the computer provides “a polite warning” to respondents flagged as potential fakers while non-flagged applicants are given a control message. All respondents are then given the “main item block” (a second testing of the faking items and the full personality measure).

This method’s utility rests in combining best practices from the faking literature: using a proactive mitigation strategy, only providing a warning to potential fakers, and allowing an opportunity for retest. In an organizational quasi-experiment, and a student-based true experiment, Fan et al. was able to demonstrate that flagged applicants lowered their scores after the warning was provided. Another benefit was that the perception of the test was not significantly affected. Admittedly, some of the kinks still need to be
ironed out, but as selection methods become more technologically advanced, new opportunities for reducing faking, such as the Fan et al. method, will be recommended.

Fan, J. Gao., D., Carroll, S.A., Lopez, F.J., Tian, T.S., & Meng, H. (2011). Testing the efficacy of a new procedure for reducing faking on personality tests within selection contexts. Journal of Applied Psychology.

human resource management, organizational industrial psychology, organizational management

source for picture: http://www.freedigitalphotos.net/images/Ideas_and_Decision_M_g409-Choices_p22024.html

Using Facebook profiles to assess personality (IO Psychology)

Reply

Topic: Personality Assessment, Selection
Publication: Journal of Applied Social Psychology (in press)
Article: Social networking websites, personality ratings, and the organizational context: More than meets the eye?
Authors: Kluemper, D. H., Rosen, P. A., & Mossholder, K. W.
Reviewed by: Alexandra Rechlin

As Facebook becomes increasingly more popular, employers are starting to look at the profiles of applicants. Numerous pictures of drunken debauchery may be informative to employers, but can Facebook profiles be used to assess an applicant’s personality? A recent study by Donald Kluemper and his colleagues suggests that they can.

In this study, raters used 15 questions from the IPIP (International Personality Item Pool) to rate participants’ Facebook profiles for the Big Five (openness, conscientiousness, extraversion, agreeableness, neuroticism) in 2007 and 2008. According to the authors, many indicators of personality can be found on a person’s Facebook page. For example, the number of friends that a person has is related to extraversion, and someone high in conscientiousness may be more careful regarding the types of posts he writes or comments on. Someone high in agreeableness may be more trusting and therefore post more personal information.

The authors found that raters showed good agreement about the personality ratings, and that they were fairly consistent. Also, other-ratings (the ratings based on Facebook profiles) showed pretty good agreement (r = .23 – .44) with self-ratings, which is about what would be expected based on past research looking at the accuracy of ratings from friends and family. In other words, Facebook profiles seem to be a pretty good way of getting personality ratings. In addition, the Facebook ratings were more strongly related to job performance than were self-ratings. The authors also found that of the Big Five, agreeableness and conscientiousness were the most important predictors of hirability ratings, and those ratings in turn were related to actual job performance.

These findings are interesting, but what do they really mean? We now know that Facebook profiles can be used as a source of personality ratings. However, should we really use them that way? As you can imagine, there are many potential legal and ethical issues related to using Facebook profiles in selection. In addition, many users now make their profiles private and therefore inaccessible to employers. It appears that using Facebook profiles in personality assessment could be useful, but at this point there remain potential legal risks.

Kluemper, D. H., Rosen, P. A., & Mossholder, K. W. (in press). Social networking websites, personality ratings, and the organizational context: More than meets the eye? Journal of Applied Social Psychology. doi: 10.1111/j.1559-1816.2011.00881.x

human resource management, organizational industrial psychology, organizational management

 

source for picture: http://www.freedigitalphotos.net/images/Internet_g170-Click_Like_p39776.html

Are cognitive ability tests insulting your applicants? (IO Psychology)

Reply

Topic: Organizational Justice, Fairness, Interviewing, Assessment, Selection
Publication: Personnel Psychology (WINTER 2011)
Article: Status and organizational entry: How organizational and individual status affect justice perceptions of hiring systems
Authors: Sumanth, J. J., & Cable, D. M.
Reviewed by: Alexandra Rechlin

It is well known in the field of IO psychology that cognitive ability tests are very predictive of employee performance.  However, applicants often see them as unfair and do not like taking them; more informal and much less valid methods (like informal interviews) tend to be preferred by applicants. In this study, Sumanth and Cable (2011) investigated the effect that the status of the organization and the career status of the applicant would have on applicants’ perceptions of the selection system’s fairness.

In this quasi-experimental study, the authors tested their hypotheses with two samples of adults (one of MBA alumni in the United States, one of executives in the United Kingdom). All participants were told that the hiring organization would use behavioral interviews, and half of the participants were told that they would also need to complete a cognitive ability test.

Organizational status is the reputation of an organization; a high-status organization (e.g., Google) is seen as being very reputable and prestigious. High-status organizations tend to be known for the rigor of their selection systems. The authors found that when low-status organizations (as opposed to high-status organizations) used cognitive ability tests as part of their selection system, applicants were less likely to view the organization attractively.

Individual career status refers to one’s beliefs about his or her career accomplishments and status (i.e., respect, prominence) within and outside of an organization. It is plausible that applicants with high status might be insulted by having to take a cognitive ability test when applying for a job; for example, they may think that their accomplishments should speak for themselves. Procedural justice is the fairness of the procedures used to make decisions. The authors found that when an individual’s status was high and cognitive ability tests were included, that person was more likely than low-status individuals to have lower perceptions of procedural justice. In other words, the process was seen as unfair. High-status individuals also were more insulted by the inclusion of a cognitive ability test.

These results indicate that even though cognitive ability tests are highly valid, they may repel high-status applicants who may see the process as unfair and insulting. My recommendation based on these findings would be to make it clear to applicants why cognitive ability tests are being used (i.e., their high validity) and that the exact same procedure will be used for all applicants.

Sumanth, J. J., & Cable, D. M. (2011). Status and organizational entry: How organizational and individual status affect justice perceptions of hiring systems. Personnel Psychology, 64, 963-1000. doi: 10.1111/j.1744-6570.2011.01233.x

human resource management, organizational industrial psychology, organizational management


source for picture: http://www.freedigitalphotos.net/images/Business_People_g201-Businessman_Holding_Hands_Up_p65923.html