Blackwell Reference Online

Extract

Rating accuracy refers to the correspondence between performance appraisal ratings and actual performance levels. Because acceptable measures of actual performance (often referred to as “true scores”) are often difficult to obtain in field settings, rating accuracy is often inferred on the basis of the psychometric characteristics of the ratings. For example, a number of rating errors measures (e.g., measures of halo effects, leniency effects, central tendency effects ) might be used to evaluate ratings. In the absence of such errors, accuracy is sometimes assumed. However, research on rater error measures and other indirect measures of accuracy suggests that these measures cannot be used to determine the accuracy of performance ratings ( Sulsky and Balzer, 1988 ; Murphy and Balzer, 1989 ). The measurement of rating accuracy requires the development of some normative standard or true score. Rating accuracy measures are most often encountered in laboratory research studies (e.g., Borman 1977 ; Murphy et al., 1982 ), where it is possible to videotape the performance being evaluated and develop normative true scores. These scores are obtained from multiple expert raters, who have opportunities to observe and evaluate ratings under optimal conditions (e.g., multiple observations of each tape, freedom from distractions). If adequate evidence of convergent and discriminant validity ... log in or subscribe to read full text