Category Archives: Evidence Based Management
Handle with Care: Ensuring That Downsizing Decisions are Made for the Right Reasons (IO Psychology)
Among the many personnel-related processes that organizations engage in, downsizing is undoubtedly one of the most dreaded by everyone involved. The employees themselves who lose their jobs often face financial hardships, while the employees who remain employed by the organization may face an increased workload and pressure from management during the transition. Although it is clear that downsizing is an important process that can have major impacts on many people, research in I-O psychology has generally neglected to study the way in which downsizing decisions are actually made (i.e. how it is determined which employees will stay and which will be let go). However, a new study by Dale Dwyer and Morgan Arbelo has begun to shed some light on this area.
Using a mixed sample of both MBA students and human resource professionals, the authors instructed participants to read 25 hypothetical employee profiles, and then asked each participant to choose the 5 employees that they would select to lay off. Participants were also asked why they would choose to lay off the employees they selected. The authors were interested in the criteria that participants would use to make their downsizing decisions; would they choose employees based on objective, job-relevant criteria, such as performance ratings and absenteeism records, or would they base their choices on personal characteristics, such as the gender or race of the employee?
In general, the authors found that demographic characteristics were quite influential in making downsizing decisions. Specifically, older and minority employees were more likely to be laid off than younger or non-minority employees; in addition, participant demographic characteristics (e.g. age, sex, and race of the research participant) had an influence on downsizing decisions. Unfortunately, downsizing decisions did not appear to be impacted by performance ratings at all; lower-performing employees were no more likely to be laid off than high-performing employees.
In all, the findings of this study suggest that there may be substantial opportunities for I-O practitioners to influence the way in which downsizing decisions are made. Specifically, practitioners might emphasize that individuals in organizations who are making downsizing decisions should focus on job-relevant criteria, to avoid both poor downsizing decisions and possible legal challenges.
Using data to make smart decisions: 1 + 1 = It’s Not That Simple
Topic: Business Strategy, Decision Making, Evidence Based Management, Statistics
Publication: Harvard Business Review (APR 2012)
Article: Good Data Won’t Guarantee Good Decisions
Authors: S. Shah, A. Horne, and J. Capellá
Reviewed By: Megan Leasher
When we were in grade school, we learned that 1 + 1 = 2. We quickly realized and celebrated the immediate success in figuring out what came after the equal sign. This celebration built faith; blind faith that we should always believe in the result of an analysis.
But in business, it’s not quite so simple. We should not automatically rejoice in what we see after an equal sign, because we need to judge what went into the numbers in the first place. This concept is the focus of a study conducted by the Corporate Executive Board, which classified 5,000 employees at 22 global companies into one of three categories: Those who always trust analysis over judgment, those who always rely on their gut, and those who balance analysis and judgment together. The Board advocates the latter “balanced” group, as their research found that this group demonstrated higher productivity, effectiveness, market-share growth, and engagement than those in the other two groups. However, the Board also found that only 38% of employees and 50% of senior executives fell into this “balanced” group. Taken together, their findings advocate cultivating both analysis and judgment in decision-making at all levels of organizations.
The authors present several ideas as to how organizations can begin to make a shift toward a culture of applying appropriate insight and judgment to their data analysis. First and foremost, they argue that data must be made accessible and presented in usable formats that enable analysis. A dual-focus must be placed on the both the data and the judgment; increase data literacy and statistical expertise while simultaneously training employees how to correctly use the data, encouraging both dialogue and dissent throughout the interpretation.
But this is easier said than done. You have to know what to trust and distrust in data. You have to learn if and how metrics support the strategy and growth of an organization. You have to learn what types of caveats and error can be found within the data. You have to learn how the data was collected, what might be wrong with the collection process, and what important information might have been ignored. You have to know how to interpret and proceed when you find that multiple metrics of performance are giving you competing answers; not all data play nice with each other. You have to know what data is worth analyzing and what data should be abandoned altogether. Sometimes running away screaming is the appropriate response.
Analysis isn’t just about writing a formula and clicking “run” or “execute” to crunch the numbers. After all, data without method is just numbers in columns and rows. It’s about a series of critical, incremental, and ethical judgment calls before and after each iteration within an analysis. Some of the judgment calls come from understanding the content and context of the data, some come from a grounding in organizational and industry knowledge, and some come from an understanding of the past, present, and future strategy of the organization. And yes, some judgment calls come from pure statistical knowledge. The true expertise comes from a constant interplay and interdependence of all of these factors.
Regardless of the challenges presented, the authors are clear that decisions should never be made by data or one’s gut alone; analysis is critical, but so is applying corresponding judgment.
human resource management, organizational industrial psychology, organizational management
Unproctored Testing: Increasing the Applicant Pool to Combat Concerns with Cheating (IO Psychology)
Topic: Selection, Evidence Based Management, Assessment
Publication: International Journal of Selection and Assessment (JUN 2012)
Article: Offsetting Performance Losses Due to Cheating in Unproctored Internet-Based Testing by Increasing the Applicant Pool
Authors: Richard N. Landers & Paul R. Sackett
Reviewed By: Thaddeus Rada
Unproctored Internet testing (UIT) has been a hot topic in IO psychology over the past several years. In a nutshell, UIT allows organizations to post some of their selection tests online, allowing applicants to access them from virtually anywhere, so they can complete them on their own time. Some of the research on UIT has confirmed its strengths, such as its accessibility and efficiency, but other research has highlighted some of its limitations; in particular, there remains widespread concern about cheating in UIT. Because UIT is unproctored (it’s right there in the name), applicants are not under any supervision when they take such tests, so it’s possible that cheating could occur in a wide variety of ways.
While many authors have debated the extent to which cheating in UIT is a problem (e.g., how prevalent it is, how to cope with it, etc.), Landers and Sackett argue that if UIT increases the size of the applicant pool, even if some of these individuals do cheat, the organization may still derive benefit from a UIT program. Their logic goes something like this: if an organization is only looking to hire a set number of people (e.g., 50), then increasing the size of the applicant pool allows the organization to increase the cut score that an individual needs to exceed in order to be hired, thus making it harder for cheaters to get job offers. To test their ideas, Landers and Sackett conducted a computer simulation, which confirmed their belief in the power of a large applicant pool to compensate for some of UIT’s limitations.
While it does not remove all the concerns that exist about UIT, Landers and Sackett’s study does demonstrate that some of UIT’s benefits outweigh its limitations. This study’s findings hinge, to a large extent, on the capacity of a UIT intervention to increase the size of the applicant pool. As such, the authors suggest that practitioners carefully assess whether or not they anticipate that this will be the case before recommending that an organization adopt UIT.
human resource management, organizational industrial psychology, organizational management
You’ve Been Tagged: On the Potential Risks and Rewards of Obtaining Applicant Information from Social Networking Sites (Human Resource Management)
Topic: Selection, Evidence Based Management, Personality Assessment
Publication: Journal of Managerial Psychology (2009)
Article: Future Employment Selection Methods: Evaluating Social Networking Web Sites
Authors: Donald H. Kluemper & Peter A. Rosen
Reviewed By: Thaddeus Rada
As social networking web sites (SNWs) such as Facebook and LinkedIn become ever more popular, the field of IO psychology has begun to turn its attention towards understanding the impact these web sites have on human resource management. On the one hand, these SNWs offer a tempting opportunity for organizations to obtain information about applicants. At the same time, there are concerns about the legality of obtaining this information; if information that is not job-relevant is obtained through the examination of SNWs and used to make hiring decisions, then organizations who use such methods may violate employment laws and put themselves at risk of having lawsuits filed against them.
In a 2009 paper in the Journal of Managerial Psychology, Donald Kluemper and Peter Rosen examined the consistency and accuracy of student judges (in an employment selection course) who examined six different profiles in SNWs. Overall, they found that judges were consistent with one another and fairly accurate at evaluating the intelligence and personality of the individuals in the profiles they viewed. While these results suggest that it may be possible for SNWs to provide useful information about job applicants, concerns remain about obtaining applicant information in this fashion, particularly as it concerns the legality of such practices.
This article was published less than three years ago, but it is likely that additional research in this area has been conducted between then and now. As such, practitioners should “stay tuned” for further research in this area, and caution organizations about relying too heavily on the information that SNWs may provide about applicants until more is known about the usefulness and legality of this information. At the same time, given the relatively short time that SNWs have been popular, this topic may be ripe for academic-practitioner collaboration; the large field samples that practitioners often have access to may be particularly useful for conducting research in this area.
human resource management, organizational industrial psychology, organizational management
Can You Hear Me Now? The Impact of Power on Advice-Taking (IO Psychology)
Topic: Evidence Based Management
Publication: Organizational Behavior and Human Decision Processes (JAN 2012)
Article: Power, Competitiveness, and Advice Taking: Why the Powerful Don’t Listen
Authors: Tost, L. P., Gino, F., & Larrick, R. P.
Reviewed By: Thaddeus Rada
It will come as no surprise to individuals working in IO psychology that people, when making decisions, often have the opportunity to seek advice from others about the choice they might make. Researchers have found that a variety of factors can influence whether or not an individual will listen to such advice, including the nature of the decision to be made, the “status” of the person giving the advice, relative to the decision maker, and individual-level characteristics of the decision maker. In a recent paper, Leigh Tost and colleagues assess the impact that one of these individual-level variables has on advice taking: the perceived power that the decision maker personally feels.
In the course of a series of experiments, the researchers found that power appears to have a negative impact on advice taking, in that individuals who perceive themselves to be more powerful are less responsive to the advice of others. The researchers found this to be the case both when the advice giver was a novice, as well as when they were an expert. In contrast, while decision makers who perceived themselves to be less powerful were generally unlikely to follow the advice of novices, they were likely to follow the advice of experts.
Given the high degrees of power that many individuals in organizations may wield (e.g. executives, managers, boards of directors, etc.), understanding the ways in which these individuals handle advice is important for practitioners. Studies like the ones conducted by these authors can continue to contribute to our knowledge of these processes, ideally for the benefit of decision makers, advisors, and the people impacted by the decisions themselves.
human resource management, organizational industrial psychology, organizational management
Telling More than You Can Say: The Relationship Between Résumé Characteristics and Hiring Recommendations (Human Resources)
Topic: Evidence-Based Management, Recruiting
Publication: International Journal of Selection & Assessment (DEC 2011)
Article: Test of a Model Linking Applicant Résumé Information and Hiring Recommendations
Authors: Chen, C.- C., Huang, Y.- M., & Lee, M.- I.
Reviewed By: Thaddeus Rada
A common feature of the job application process that many practitioners involved with human resource management are familiar with is the résumé. As the first contact that human resources personnel, managers, and recruiters often have with a given applicant, résumés are generally considered to be an important part of the application for many jobs. However, while the overall importance of this document is well-recognized, the reasons for its importance remain something of a mystery. Specifically, uncertainty remains as to how specific information communicated in résumés is used by personnel involved in the hiring process to make evaluations about applicants. To help address this issue, a new study by Chien-Cheng Chen and colleagues proposes a model that specifies how certain résumé characteristics, including content and style, influence recruiters’ perceptions of the knowledge, skills, abilities, and other characteristics (KSAOs) of job applicants, and, consequently, how these perceptions impact recruiters’ evaluations of job applicants.
The authors’ proposed model examines how four features of résumés (academic qualifications, work experience, extracurricular activities, and résumé aesthetics) are related to recruiters’ perceptions of applicants’ job-related knowledge, interpersonal skills, GMA, and conscientiousness. In turn, the authors then examined the link between these perceptions and recruiters’ evaluations (i.e. if they recommended hiring a particular employee). Using structural equation modeling, the authors found that each of the four résumé characteristics they examined are substantially related to recruiters’ perceptions of applicants. In particular, it was found that academic qualifications were strongly related to perceptions of applicants’ general mental ability, while work experience and extracurricular activities were related to job-related knowledge and interpersonal skills, respectively. In terms of the perceptions that recruiters held about applicants, interpersonal skills were more strongly related to hiring recommendations than job-related knowledge or interpersonal skills, though these perceptions appear to be important as well.
The results of this study point to the fact that recruiters (and likely other professionals who review résumés as well) can and do make attributions about the characteristics of job applicants based on certain features of applicants’ résumés. As our understanding of how certain features of résumés impact perceptions of job applicants increases, the use of résumé screening as an initial selection tool may be increased.
human resource management, organizational industrial psychology, organizational management
Tension at the Top: Why Women at the Executive Level May Not Welcome Other Women to the Club
Topic: Evidence-Based Management, Teams
Publication: Organizational Behavior & Human Decision Processes (SEP 2011)
Article: Female Tokens in High-Prestige Work Groups: Catalysts or Inhibitors of Group Diversification?
Authors: Duguid, M.
Reviewed By: Thaddeus Rada
Workplace diversity is a goal that many (and hopefully most or all) organizations aspire to. In recent years, there has been an increased emphasis on incorporating previously underrepresented groups, and especially women, into high-prestige work groups at the highest levels of the organization. Among the many benefits that might come from the addition of women to high-prestige groups, it is commonly believed that other women may be inspired by the level of authority and success that women in elite positions have achieved. However, a new paper by Michelle Duguid challenges this logic, suggesting that this “inspirational” role is one that women may embrace with some reservations.
Duguid conducted three studies to examine various ways in which women, as minority members in high-prestige work groups (i.e. “female tokens”), may, at times, be unsupportive of the efforts of other women to join a high-prestige group. More specifically, Duguid explained that two types of value threat may lead to women resisting the inclusion of other women in elite work groups. The first type of value threat, competitive threat, may occur when another highly-qualified female candidate is being considered for a high-level position. In this situation, the incumbent female may feel threatened by the possible addition of another woman in the high-level group. The second type of value threat is collective threat, which may occur when a moderately-qualified female candidate is being considered. In this situation, the incumbent female may be concerned that the applicant will be a poor performer, and hence perpetuate negative stereotypes about women, perhaps making salient the fact that the incumbent herself is a woman.
The results of Duguid’s studies suggest that organizations would be wise to consider the feelings that incumbent women may have about the addition of other women to high-prestige groups. While the fact that women may have negative reactions to other female applicants should not discourage organizations from adding qualified women to high-prestige work groups, it seems advantageous for organizations to consider the dynamics of the group and the feelings of current female incumbents when considering such additions.
human resource management,organizational industrial psychology, organizational management
Life Isn’t Always Fair: Using Inducements & Contributions to Predict Employee Satisfaction
Topic: Employee Satisfaction, Evidence-Based Management, Rewards
Publication: Journal of Applied Psychology (JUL 2011)
Article: Promised and Delivered Inducements and Contributions: An Integrated View of Psychological Contract Appraisal
Authors: Lambert, L. S.
Reviewed By: Thaddeus Rada
One of the most common complaints an employee may have with their employing organization is that they are not be fairly or adequately compensated for the contributions that they are putting into the company. A complaint of this type gets down to the concept of a psychological contract, which consists of inducements and contributions. Both of these come in two “varieties,” promised and delivered. Promised inducements or contributions are commitments that an organization or an employee, respectively, commit to providing to the other. Delivered inducements or contributions are what the organization or employee actually provide to the other, which may deviate from the promised inducement or contribution.
Together, the balance, or lack thereof, between these four components determines the overall quality of the psychological contract between an employee and the organization they work for. The current study, by Lisa Schurer Lambert, addresses a gap in the psychological contract literature: the comparison, by employees, of inducements to contributions, particularly with respect to the weight that employees give to each component.
Smarter Employees Perform Better, but Will They Stay?
Topic: Evidence Based Management, Talent Management, Turnover
Publication: Journal of Applied Psychology (AUG 2010)
Article: A conceptual and empirical analysis of the cognitive ability-voluntary turnover relationship
Authors: M.A. Maltarich, A.J. Nyberg, and G. Reilly
Reviewed By: Benjamin Granger
Cognitive ability is one of the best predictors of employee job performance across jobs, but there are other important organizational outcomes besides job performance that cognitive ability may not predict as favorably. One such possibility is voluntary turnover. Unfortunately, previous attempts at linking cognitive ability to voluntary turnover have shown that the relationship is not as simple as “high cognitive ability employees are more likely to leave voluntarily than low cognitive ability employees”. Maltarich et al. (2010) argue that this important relationship is better understood when the cognitive demands of employees’ jobs are considered.
In their study of over 5,300 adults from the “National Longitudinal Survey of Youth, 1979 cohort”, Maltarich et al. analyzed employment data collected in 1994, 1996, 1998 and turnover data up until 2000. As expected, the cognitive demands of employees’ jobs plays an important role in determining how cognitive ability impacts voluntary turnover.
For jobs high in cognitive demands, low cognitive ability employees were the much more likely to voluntarily leave than those with moderate levels of cognitive ability, suggesting that low cognitive ability employees get pushed out of jobs that have high cognitive demands.
Is Evidence-Based Management Actually Based on Evidence?
Topic: Evidence Based Management
Publication: Academy of Management Perspectives
Article: What’s the
evidence on evidence-based management? (NOV 2009)
Author: T. Reay, W. Berta, and M. K. Kohn
Reviewed by: Sarah Teague
In the last century, many significant advances have
been made in the study and application of Industrial/Organizational (I/O)
Psychology and Organizational Behavior (OB). Unfortunately, there remains a
substantial communication gap between researchers and practitioners. But research
and practice do not have to be mutually exclusive – as evidenced by this very
website!
One
approach that has been put forth in attempts to promote such collaboration is
that of Evidence-Based Management (EBMgt). This approach emphasizes the
importance of utilizing various sources of solid evidence in the process of
organizational decision-making (e.g., back decisions and actions with solid
evidence, rather than lore, biases, false assumptions, or a “that’s the way
we’ve always done things around here” mentality). While this is still a
relatively new concept, many in the community have begun to take note.
In
a recent literature review, Reay, Berta, and Kohn (2009) sought to assess the
quantity and quality of the existing evidence supporting evidence-based
management. This evidence was divided into six different levels according to
empirical quality ranging from an author’s opinion (Level 6 – weakest) to
large-scale studies involving a high degree of empirical rigor (Level 1 –
strongest). The majority of the literature fell into a very weak Level 5
categorization, and no studies were given the strongest rating of Level 1.
Further, none of the studies in their review addressed the question of whether
or not evidence-based management will actually improve organizational
performance. More research is needed in order to discuss the validity of such
practices.
The
results of the current review indicate that while the empirical quality of the
existing literature is certainly lacking, individual study findings suggest
promise for the EBMgt approach.. The authors convey optimism for the futureof EBMgt
research but echo the calls from other researchers (including Rousseau and
colleagues) for greater emphasis on knowledge transfer (KT; disseminating the
information gleaned from research to facilitate implementation in the workplace).
Accordingly, the authors of the current review provide suggestions to promote KT,
including placing more value on literature reviews and replication studies in
the research domain, teaching management students to value – but also critically
evaluate – evidence, and encouraging researchers to think practically about
their findings and practitioners to consider how their needs and experiences
might further research, all helping to bridge the science-practice gap.
We could think of evidence-based management
as a medication that has shown great potential in initial testing but needs
further trials to validate long-term use.
