This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Fleming, J. A. Articles by Carran, D. Search for Related Content Social Bookmarking                         What's this?

A Comparison of Two Methods of Determining Interrater Reliability

Seabrook Island, SC

Janeen McCracken

Johns Hopkins University

Deborah Carran, Ph.D.

Johns Hopkins University

This article offers an alternative methodology for practitioners and researchers to use in establishing interrater reliability for testing purposes. The majority of studies on interrater reliability use a traditional methodology where by two raters are compared using a Pearson product-moment correlation. This traditional method of estimating interrater reliability uses a correlation of the two raters' scores for the same subject. This study used an observer-rater paradigm in which 20 observers' scores were compared to the scores of an expert rater. Results bring into question the common practice of examining interrater reliability. The correlation of two raters may not be a reliable way of determining if every evaluator using the instrument will obtain scores that accurately report the developmental status of young children.

Assessment for Effective Intervention, Vol. 29, No. 2, 39-51 (2004)
DOI: 10.1177/073724770402900204


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?