Category Archives: Personality Assessment
When Reading Research Leads to a Brain Full of “What?!” (IO Psychology)
When you read scientific research, you should be left feeling as though you gained knowledge and/or have something new and shiny that can be applied to the real world. But once in a while you finish an article and there is nothing but unpoppable “What did I just read?!” bubbles floating in your brain.
This article focused on how applicants’ personality types might impact their reactions to assessment tests within a hiring process. Specifically, candidates for firefighter, dispatcher, and rescue management roles had to complete a series of personality and cognitive assessments as a part of the selection process. Immediately after, they were asked to complete a voluntary survey asking about their reactions to the tests. The researchers found that personality types had no impact on applicants’ perceptions that the assessments were related to the job and that the tests could predict future job performance. One personality type did perceive the tests as less fair than those with other personality types, but the difference may not have been large enough to have real meaning.
As I kept reading the article, I kept wondering how this information would be applied, or even how it would be useful. I kept wondering this because the authors never told me. The authors briefly mention previous research stating that applicant reactions can impact whether or not a candidate might accept a job offer and/or impact their future performance on the job. Yet they never relate their own findings to this previous research. I was left hanging.
The study also had a number of confounds, a few of which the authors acknowledged. Looking solely at rescue applicants isn’t representative of most jobs and applicants. Candidates had to first pass a physical test before they were allowed to begin the personality and cognitive assessments. The reactions survey only asked for their reactions to the personality and cognitive tests, but wouldn’t their perceptions of the physical test muck up their thoughts a bit?
Also, participants voluntarily completed the reactions survey, and not everyone completed it. Wouldn’t the thoughts of those who did NOT want to share their reactions be critical? Finally, their research found different reactions to the assessments based on gender and age, but they never investigated further, which I found disappointing.
Now I have to be fair and say that no research is perfect. All research has confounds. But when you feel as though you don’t get the “so what?” of the entire study and there are also lots of confounds, how are you supposed to react?
After reading this article I was left feeling a little icky inside. But it reminded me that reading research with a discerning amount of skepticism is not only healthy, it is mandatory. It reminded me of a wonderful quote by the philosopher George Santayana: “Skepticism, like chastity, should not be relinquished too readily.”
Facebook friends and your personality?
Publication: Proceedings of Association for Computing Machinery Conference on Computer Supported Cooperative Work (2012)
Article: The personality of popular Facebook users
Authors: Daniele Quercia, Renaud Lambiotte, David Stillwell, Michal Kosinski, & Jon Crowcroft
Reviewed By: Scott Charles Sitrin
Imagine that you’re trying to fill a sales position and are looking for someone who is outgoing, comfortable around people, and the life of the party. Sure, in the interview, you’ll be able to get a sense of how cordial someone is, but you have 10 applications on your desk, and you are only able to interview two of them.
A possible solution is Facebook, as Quercia, Lambiotte, Stillwell, Kosinski, and Crowcroft found that extraversion predicts the number of social contacts on the social networking site as well as the number of friends in the real world. Thus, if one of your candidates has 30 social contacts and the other has 900, the latter would have a higher likelihood of being extraverted. To find this result, the investigators calculated the number of social contacts and personality – as measured by an online application called myPersonality – of 172,952 Facebook users.
human resource management, organizational industrial psychology, organizational management
source for picture: http://www.freedigitalphotos.net/images/Computing_g368-Man_With_Laptop_p63129.html
A Sequel to the Ring of Fire: How Internal and External Candidates React to Employment Testing (I/O Psychology)
Topic: Assessment, Personality Assessment, Selection
Publication: International Journal of Selection and Assessment (JUN 2012)
Article: Don’t you know me well enough yet? Comparing reactions of internal and external candidates to employment testing
Authors: G. W. Giumetti and E. F. Sinar
Reviewed By: Megan Leasher
Employment testing is gaining in popularity at all levels within organizations, leading internal candidates to complete assessment tests to be considered for promotion or lateral moves. When you test employees you already hired, you might expect some pushback!
When competing for the same job, do internal and external candidates react differently to employment tests? Gary Giumetti and Evan Sinar explored this question with over 2300 candidates across 12 organizations and found several key differences. As compared to external candidates, internal candidates reported lower perceptions regarding information they received about what to expect in the tests, but held more positive views on both the job- relatedness of the tests and their overall recommendation of the organization to others.
This study focused on candidates’ perceptions of the tests themselves immediately after completing the tests. But what if you asked them later on? As a practitioner, the article led me to a spiraling of even more questions; all of which could impact applicant reactions at various points in the hiring process. How were the scores on the tests used? Was there a hard cutoff that required a “passing” score (on one or more of the tests) to advance in the hiring process? If so, were exceptions ever made because of something else amazing in a candidate’s background? What were candidates told about the hiring process and how their test scores were used? Did any of the internal candidates take the same or similar tests when they vied for
their current job? What had internal candidates heard, if anything, about the ease or difficulty of the tests from their coworkers? Were interviewers made aware of their test scores?
When it comes to any hiring process, consistency in both process and communication is paramount, no matter who comprises the applicant pool. An organization can choose to be transparent, opaque, or somewhere in the middle in what they share about a hiring process to candidates. But they need to be consistent to ensure that all applicant pools are treated justly.
human resource management, organizational industrial psychology, organizational management
You’ve Been Tagged: On the Potential Risks and Rewards of Obtaining Applicant Information from Social Networking Sites (Human Resource Management)
Topic: Selection, Evidence Based Management, Personality Assessment
Publication: Journal of Managerial Psychology (2009)
Article: Future Employment Selection Methods: Evaluating Social Networking Web Sites
Authors: Donald H. Kluemper & Peter A. Rosen
Reviewed By: Thaddeus Rada
As social networking web sites (SNWs) such as Facebook and LinkedIn become ever more popular, the field of IO psychology has begun to turn its attention towards understanding the impact these web sites have on human resource management. On the one hand, these SNWs offer a tempting opportunity for organizations to obtain information about applicants. At the same time, there are concerns about the legality of obtaining this information; if information that is not job-relevant is obtained through the examination of SNWs and used to make hiring decisions, then organizations who use such methods may violate employment laws and put themselves at risk of having lawsuits filed against them.
In a 2009 paper in the Journal of Managerial Psychology, Donald Kluemper and Peter Rosen examined the consistency and accuracy of student judges (in an employment selection course) who examined six different profiles in SNWs. Overall, they found that judges were consistent with one another and fairly accurate at evaluating the intelligence and personality of the individuals in the profiles they viewed. While these results suggest that it may be possible for SNWs to provide useful information about job applicants, concerns remain about obtaining applicant information in this fashion, particularly as it concerns the legality of such practices.
This article was published less than three years ago, but it is likely that additional research in this area has been conducted between then and now. As such, practitioners should “stay tuned” for further research in this area, and caution organizations about relying too heavily on the information that SNWs may provide about applicants until more is known about the usefulness and legality of this information. At the same time, given the relatively short time that SNWs have been popular, this topic may be ripe for academic-practitioner collaboration; the large field samples that practitioners often have access to may be particularly useful for conducting research in this area.
human resource management, organizational industrial psychology, organizational management
Using Facebook profiles to assess personality (IO Psychology)
Topic: Personality Assessment, Selection
Publication: Journal of Applied Social Psychology (in press)
Article: Social networking websites, personality ratings, and the organizational context: More than meets the eye?
Authors: Kluemper, D. H., Rosen, P. A., & Mossholder, K. W.
Reviewed by: Alexandra Rechlin
As Facebook becomes increasingly more popular, employers are starting to look at the profiles of applicants. Numerous pictures of drunken debauchery may be informative to employers, but can Facebook profiles be used to assess an applicant’s personality? A recent study by Donald Kluemper and his colleagues suggests that they can.
In this study, raters used 15 questions from the IPIP (International Personality Item Pool) to rate participants’ Facebook profiles for the Big Five (openness, conscientiousness, extraversion, agreeableness, neuroticism) in 2007 and 2008. According to the authors, many indicators of personality can be found on a person’s Facebook page. For example, the number of friends that a person has is related to extraversion, and someone high in conscientiousness may be more careful regarding the types of posts he writes or comments on. Someone high in agreeableness may be more trusting and therefore post more personal information.
The authors found that raters showed good agreement about the personality ratings, and that they were fairly consistent. Also, other-ratings (the ratings based on Facebook profiles) showed pretty good agreement (r = .23 – .44) with self-ratings, which is about what would be expected based on past research looking at the accuracy of ratings from friends and family. In other words, Facebook profiles seem to be a pretty good way of getting personality ratings. In addition, the Facebook ratings were more strongly related to job performance than were self-ratings. The authors also found that of the Big Five, agreeableness and conscientiousness were the most important predictors of hirability ratings, and those ratings in turn were related to actual job performance.
These findings are interesting, but what do they really mean? We now know that Facebook profiles can be used as a source of personality ratings. However, should we really use them that way? As you can imagine, there are many potential legal and ethical issues related to using Facebook profiles in selection. In addition, many users now make their profiles private and therefore inaccessible to employers. It appears that using Facebook profiles in personality assessment could be useful, but at this point there remain potential legal risks.
human resource management, organizational industrial psychology, organizational management
source for picture: http://www.freedigitalphotos.net/images/Internet_g170-Click_Like_p39776.html
Cheating on Unproctored Internet-Based Tests – is it a big deal?
Topic: Personality Assessment
Publication: International Journal of Selection and Assessment (MAR 2010)
Article: The magnitude and extent of cheating and response distortion effects on unproctored internet-based tests of cognitive ability and personality
Authors: W. Arthur, R.M. Glaze, A.J. Villado, and J.E. Taylor
Reviewed By: Benjamin Granger
The future of employment testing is upon us and many organizations have turned to unproctored internet-based testing in lieu of proctored paper-and-pencil testing.
Among its many advantages, internet-based testing is often faster, more efficient, and more convenient than proctored paper-and-pencil methods (e.g., can be scored immediately, distributed to geographically dispersed applicants). One concern, however, is that unproctored internet-based testing allows for cheating or response distortion (i.e., faking). But is this a realistic concern? Is cheating really more prevalent in unproctored internet-based settings?
Arthur, Glaze, Villado, and Taylor (2010) recently explored this question using a speeded cognitive ability test and two different personality tests. Although the authors note that the speeded nature of the cognitive ability test may have curtailed cheating in and of itself, the pattern of their results was very similar to the pattern observed in proctored, paper-and-pencil contexts.
Ultimately, although cheating/response distortion is well known to occur, Arthur and colleagues did not find that it is necessarily more common in unproctored internet-based testing situations. In concluding this review, it should be noted that the behavior of cheating or response distortion is very difficult to measure. In fact, as Arthur et al. note, response distortion or cheating was not measured directly in their study. Nevertheless, the results of this study suggest that response distortion/cheating is not necessarily more common in unproctored internet-based testing situations.
Arthur, W., Glaze, R.M., Villado, A.J., & Taylor, J.E. (2010). The magnitude and extent of cheating and response distortion effects on unproctored internet-based tests of cognitive ability and personality. International Journal of Selection and Assessment, 18(1), 1-16.
What Makes a “Good Faker”? And Do We Want Them?
Topic: Faking, Personality Assessment
Publication: Human PerformanceArticle: Individual differences in the ability to fake on personality measures.
Author: P.H. Raymark, T.L. Tafero
Featured by: Benjamin Granger
One common criticism of personality testing is its susceptibility to faking. Faking (i.e., response distortion) occurs when job applicants intentionally misrepresent themselves (e.g., respond in ways that present themselves as more attractive job candidates).
In a recent study, Raymark and Tafero (2009) investigated the role of several individual differences thought to explain why certain job applicants are more able to fake on personality measures than others. Specifically, the authors investigated:
(1) Openness to ideas (individuals high in openness are described as being curious, intelligent, and having a high need for cognition)
(2) Self-monitoring (the extent to which individuals actively monitor their self-presentation and behavior – social chameleons)
(3) Prior knowledge of the personality factors being measured
(4) Prior knowledge of the job an applicant is applying for
Raymark and Tafero utilized a sample of 342 students from a university in the U.S., roughly half of which were instructed to fake their personality in a generally “good” direction and the other half to fake “good” for a specific job – accountant. In addition to completing the personality test while faking, all participants were also instructed to complete the test by responding honestly at a different time during the study.
As expected, the results of the study suggest that certain individual differences are related to the ability to fake. While self-monitoring was not related to faking, individuals who reported being more open to ideas and having more prior knowledge of the personality characteristics being measured tended to have increased scores on the personality measure when to instructed to fake “good” in general. Moreover, openness to ideas predicted faking for the specific job (accountant).
BUT…It is unknown whether this type of faking is actually a “bad” thing (reduces the validity and usefulness of the personality tests) or “good” (actually relates to job performance). Now wait a second… How can faking be “good?”
Consider this: Although we may assume that faking is always a “bad” thing, it is plausible (and some have argued this) that the ability to fake personality tests is actually predictive of job performance. That is, perhaps it is beneficial from a performance point of view to have curious, intelligent and open applicants who are able to fake! So, which is it? According to Raymark and Tafero the answer is still up for debate.
Stop Faking Before It Happens
Topic: Assessment, Personality AssessmentPublication: International Journal of Selection and Assessment
Article: Comparing personality test formats and warnings: Effects on criterion-related validity and test-taker reactions
Authors: P.D. Converse
Reviewed by: Benjamin Granger
Although personality testing in employee selection settings is a common practice, it hasn’t gone without critique. The reason for this is simple: personality tests can be faked. (Let’s see, I really want this job so, yea I’m conscientious and agreeable). A quick glance at many of the commonly used personality test items will corroborate this concern (e.g., “I am always prepared”, “I make friends easily”). Seriously, why would a job applicant even consider disagreeing with such statements in a high stakes situation? Despite the obvious problem of faking, personality tests have been shown to predict employee job performance. Thus, there is a major dilemma over what to do about personality testing in employee selection settings.
In general, most personality measures allow job applicants to read a series of items and indicate the extent to which they agree/disagree with each statement.
Due to the transparency of such items, several methods are currently being investigated which aim to ameliorate the faking issue. Two of the most promising methods include using (1) forced choice (FC) items (i.e., force applicants to choose a personality trait characteristic of themselves over another) and (2) warnings to test takers about faking (i.e., inform applicants of the consequences of faking). Importantly, these two options effect faking in different ways. FC methods reduce an applicant’s ability to misrepresent him or herself while warnings effect an applicant’s motivation to fake his or her personality.
And although both have been shown to reduce faking, relatively little work has looked at these two methods simultaneously. Until now!
In an article published in the International Journal of Selection and Assessment, Converse et al. (2008) were interested in the combined effects of FC and warnings on two important outcomes:
(1) criterion-related validity (how well personality measures predict job performance) and
(2) test-taker reactions.
Converse and colleagues found that FC personality tests not only predicted important work-related criteria (e.g., leadership, perseverance), but they did so above and beyond cognitive ability. However,
their findings suggest that in some situations, job applicants may have more negative reactions toward FC formats compared to more traditional formats (e.g., Likert scale). On the other hand, it was not completely clear whether warning applicants of the consequences of faking showed acceptable criterion-related validity. Thus the authors warned against drawing firm conclusions. Moreover, when warnings were used and were framed negatively, applicants reported high levels of test anxiety. But, positively framed warnings did not have such an effect on applicants.
So what do organizations need to know about these methods?
Firstly, both FC and warnings reduce the occurrence of faking. Second, FC methods, although more time-consuming than traditional personality tests, predict workplace criteria over and above cognitive ability. Thirdly, if organizations choose to use warnings against faking, they may consider wording the warnings in a positive way (e.g., If you respond honestly, it is more likely that you will be placed in a job that suits you well) since negatively worded warnings (e.g., we’re going to catch you and you’ll no longer be considered for the job!) may cause test-taker anxiety.
Converse, P, D., Oswald, F. L., Imus, A., Hedricks, C., Roy, R., & Butera, H. (2008). Comparing personality test formats and warnings: Effects on criterion-related validity and test-taker reactions. International Journal of Selection and Assessment, 16(2), 155-169.
