Inter-rater reliability

Another way of performing reliability testing is to use the intra-class correlation coefficient (ICC) . There are several types of this and one is defined as, "the proportion of variance of an observation due to between-subject variability in the true scores". The range of the ICC may be between 0.0 and 1.0 (an early definition of ICC could be between −1 and +1). The ICC will be high when there is little variation between the scores given to each item by the raters, e.g. if all raters give the same, or similar scores to each of the items. The ICC is an improvement over Pearson's and Spearman's , as it takes into account the differences in ratings for individual segments, along with the correlation between raters.