Tag Archives: performance appraisal

Customer Satisfaction Surveys: A Measure of Race and Gender. A Measure of Performance? Not So Much

1

Topic: Fairness, Diversity, Performance Appraisal

Publication: Academy of Management Journal
Article: An examination of whether and how
racial and gender biases influence customer satisfaction

Authors: D. R. Hekman, K. Aquino, B. P.
Owens, T. R. Mitchell, P. Schilpzand, & K. Leavitt

Reviewed By: Katie Bachman


Balance  There’s this great
line in the 1980 movie, 9 to 5, when Jane Fonda says to Dabney Coleman: “You’re a sexist,
egotistical, lying, hypocritical bigot” and he replies: “So I have a few
faults; who doesn’t?” Keep that in mind when you think about the Average Joe on
the street, filling out a survey. Untrained raters don’t rate accurately—that’s
why they need training! Customer satisfaction surveys are the epitome of using
untrained raters to measure employee performance.

For organizations
attempting to become more customer service-oriented, customer satisfaction
surveys seem like a good way to measure performance. You might get an accurate
rating if you’re White and male, but minorities and women can be hurt by these
types of ratings. Bias seeps into the rating process, which is a big time legal
no-no, particularly if such ratings are used as criteria for promotion and
compensation. In three separate studies—two in the field and one in the
lab—researchers determined that women and minorities were consistently rated
lower on customer satisfaction, even when performance was the same. Obviously,
this relationship was even stronger when the rater held negative attitudes
toward these groups. Additionally, the negative ratings given to minority and
female employees also affect customer ratings of the organization. It’s not
enough that customers like your employees a little less for being non-White or
female, they also like your company a little less.

Why is this happening?
Anonymity probably has something to do with it. Surveys almost never ask
customer raters to identify themselves so people feel freer to let their
attitudes affect their judgments. Also, there’s a lack of standards and
training for most of these surveys. Raters without training may rely on their
gut reactions more so than individuals trained to focus on observed behavior.

So what does this mean
for an organization? Customer satisfaction surveys need to be taken with a
grain of salt and probably not used for employment decisions. Customers will rate identical work
as less satisfactory if a woman or minority performs it (no word in this
article about the double whammy: female minorities). If you are going to use
them, customer satisfaction surveys should be tailored to ask for behavioral
episodes, not gut reactions and should only be used in conjunction with other,
less biased measures of employee performance.

 

Hekman,
D. R., Aquino, K., Owens, B. P., Mitchell, T. R., Schilpzand, P., &
Leavitt, K. (2010). An examination of whether and how racial and gender biases
influence customer satisfaction. Academy of Management Journal, 53, 238-264

Performance Ratings = More Than Just Performance

Reply

Topic: Performance Appraisal
Publication: Journal of Organizational Behavior (MAY 2010)
Article: The impact of non-performance information on ratings of job performance: A
policy-capturing approach
Authors: J.R. Spence and L.M. Keeping
Reviewed By: Benjamin Granger

Though they are intended to reflect employees’ performance on the job, performance appraisal ratings are well known to reflect things that are irrelevant to performance.  Even more troubling is the fact that the sources of irrelevant information that lead to inaccurate performance ratings are numerous.  Recently, Spence and Keeping (2010) identified three, often neglected, sources of performance rating inaccuracy:

1.Potential consequences of ratings (e.g., interpersonal conflicts, confrontations)

2. Norms of the organization (e.g., high/low ratings across the board, accurate ratings)

3. The advancement of self interests (e.g., rater’s own performance rating contingent upon those of subordinates)

Although the authors presented scenarios of a fictitious employee to university alumni in a non-work setting, their results confirm that all three sources of potential inaccuracy can contribute to performance ratings above and beyond performance itself. Specifically, Spence and Keeping’s results point to several tendencies of performance raters. First, raters are more likely to give higher performance ratings when the norms of the organization are to give uniformly high ratings. Secondly (and not surprisingly), raters are more likely to give higher performance ratings when their own performance ratings are contingent upon their subordinates’ performance.

Finally, contrary to the authors’ prediction, the possibility that an employee will confront the rater following a poor performance appraisal, leads to lower performance ratings.  Spence and Keeping
suggest that the possibility of confrontation may lead raters to “dislike the ‘ratee’”, leading to a lower performance rating.

These findings are particularly important for those organizations that have strong norms of providing high/low ratings across the board.  Such norms can ultimately lead to the detriment of the performance appraisal system.  Organizations should also consider if (and how) they reward supervisors for providing performance ratings to their subordinates, as self-interests also appears to be a source of rating inaccuracy.  This potential problem may be addressed with the use of multi-rater performance ratings (e.g., 360 degree feedback).

Spence, J.R., & Keeping, L.M. (2010). The impact of non-performance information on ratings of job performance: A policy-capturing approach. Journal of Organizational Behavior, 31, 587-608.

Is it Fair to Include “Citizenship” in Performance Appraisals?

Reply

Topic: Citizenship Behavior, Performance Appraisal
Publication: Journal of Business and Psychology (DEC 2009)
Article: Organizational citizenship behavior in performance evaluations: Distributive justice or injustice
Authors: S.K., Johnson, C.L. Holladay, & M.A. Quinones
Reviewed By: Benjamin Granger

Organizational Citizenship Behaviors (OCBs) are volitional work behaviors that go above and beyond the call of duty and are intended to benefit the organization and/or its members.  Though OCBs are not  formally required of employees (e.g., don’t show up in the job description), they are highly valued by organizations. Thus, supervisors (and peers) often consider employees’ OCBs in formal performance appraisals.  But, how do employees feel about this?  In other words, since OCBs are not absolutely required of employees, do employees find this practice fair?

Johnson, Holladay, and Quinones (2009) investigated the extent to which employees consider including OCBs in formal performance appraisals fair. The authors conducted two separate experiments, one employing a sample of 78 employees from diverse organizations and industries and the other employing a large sample of undergraduate students.  In general, the findings of both studies were similar.

Overall, employees reported that it is fairer to include OCBs in performance appraisals than to not include them. Importantly, employees felt that it is most fair to include OCBs in performance appraisals when they constitute about 30 – 50% of the total performance rating (While the remaining represents Core Task Behaviors).

Johnson et al. also found that while females generally preferred higher weightings of OCBs (between 25 and 50%) men found a 20 – 30% weighting of OCBs to be most fair. Johnson et al. speculated that the findings for gender represent differences in the typical gender roles of males and females. For example, they suggested that in general, females are expected to engage in more helping behaviors (i.e., OCBs) than men at work and they want their performance ratings to account for this. 

Whether this is true or not (and we welcome your personal insight on this matter!), it is clear that employees  consider OCBs to be an important and rate-able portion of their job performance. Ultimately, employees do find it fair to include OCBs in performance appraisals, but the extent to which OCBs makeup the total performance appraisal depends largely on gender.

Johnson, S.K., Holladay, C.L., & Quinones, M.A. (2009). Organizational citizenship behavior in performance evaluations: Distributive justice or injustice. Journal of Business and Psychology, 24, 409-418.