Publication: The Journal of Applied Psychology (2008)
Article: Using frame-of-reference training to understand the implications of rater idiosyncrasy for rating accuracy.
Blogger: Rob Stilson
Frame of Reference (FOR) training is intended to get all raters on the same metric to mitigate idiosyncrasies caused by raters whose ideas about what is important when rating performance differ from the organization’s standards. Someone is high on idiosyncrasies if his/her dimensions for performance (what they consider important to performance) do not match up with the organization’s standards of performance. An individual is low on idiosyncrasies if his/her dimensions for performance are similar to the organization’s standards of performance.
The authors of this article wanted to determine how FOR training would affect two types of raters, those who were high on idiosyncrasies and those who were low on idiosyncrasies.
In this study, undergraduate students were asked to rate teacher performance. These students were identified as being high in idiosyncrasies or low in idiosyncrasies (you’ll need to read the article to learn how). The undergraduates were then randomly assigned to FOR training or control training. Results from the study indicate that raters higher in idiosyncrasies benefited the most from FOR training, but overall everyone who received the FOR training, regardless of the level or type of idiosyncrasy, improved. The control group showed that there was no mere exposure effect for training and that those receiving the FOR training did indeed improve significantly in accuracy. Another interesting note from the study was that every trainee (n = 236) was idiosyncratic to some extent about what dimensions are needed to be an effective teacher.
In conclusion, FOR training appears to be effective for everyone and not just people with alternate schemas about performance. Hooray for unintended, applicable findings!