This is for IE7 to hold div open
I have computed Cohen's kappa to assess agreement among raters, corrected for chance agreement. What is a reasonable kappa level? What are good and poor values of kappa?
The information that follows was derived from posts by Judith Saebel and Scott McNary on the Structural Equation Modeling LISTSERV e-mail discussion group on August 19, 1999.
Although there are no absolute cutoffs for kappa coefficients, two sources provide some rough guidelines for the interpretation of kappa coefficients. According to J. L. Fleiss (1981), p. 218, values exceeding .75 suggest strong agreement above chance, values in the range of .40 to .75 indicate fair levels of agreement above chance, and values below .40 are indicative of poor agreement above chance levels.
A journal article by Landis & Koch, p. 159, suggests the following kappa interpretation scale may be useful:
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
In addition, Gardner (1995) recommends that kappa exceed .70 before you proceed with additional data analyses.
References
Fleiss, J. L. (1981). Statistical Methods for Rates and Proportions. New York: John Wiley & Sons. (Second Edition).
Gardner, W. (1995). On the reliability of sequential data: measurement, meaning, and correction. In John M. Gottman (Ed.), The analysis of change. Mahwah, N.J.: Erlbaum.
Landis, J. & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, (33), 159-174.
If you have further questions, send E-mail to stats@ssc.utexas.edu.