Do you receive multisource (or 360 degree) feedback at work? Based on its extreme popularity, my guess is that you do. An important question, therefore, is how to make the ratings more accurate and thus more informative for development. Researchers (Hoffman et al., 2012) recently conducted two studies in which they developed and evaluated the efficacy of a new type of scale, called frame-of-reference scale (FORS) to use in multisource feedback systems.
WHAT IS A FRAME-OF-REFERENCE SCALE?
FORS start with a definition of the dimension being rated, as well as examples of both effective and ineffective behavior for that dimension. The actual items are then presented. FORS differ from normal rating scales in that a definition and behavioral examples are provided. In this way, FORS are similar to frame-of-reference training, except that the information is presented in written format along with the items (as opposed to being provided in a training program). FORS are different from behaviorally anchored rating scales (BARS) in that BARS link specific behaviors with each scale point (whereas FORS provide examples of effective and ineffective behavior).
RESULTS OF THE STUDY
The authors compared FORS with standard rating scales (which were similar to behavioral observation scales) and found that using FORS increased accuracy and led to greater differentiation among dimensions. When compared to frame-of-reference training, using FORS led to similar levels of accuracy. FORS therefore seem to be an important development in the improvement of multisource feedback ratings; FORS leads to increased accuracy and are not as expensive or time-consuming as BARS or frame-of-reference training.
Hoffman, B. J., Gorman, C. A., Blair, C. A., Meriac, J. P., Overstreet, B., & Atchley, E. K. (2012). Evidence for the effectiveness of an alternative multisource performance rating methodology. Personnel Psychology, 65, 531-563.