The QuadraticWeightedKappa command accepts two lists or arrays of numeric data and computes the quadratic weighted kappa value.

The quadratic weight kappa value is useful in the context of essay scoring where you want to measure how well one set of grades correlates to another set of grades.  A value of 1 indicates perfect agreement between the two grade sets.  Typically two sets of grades generated by independent human raters scoring the same sets of essays will achieve a quadratic weight kappa value of .8 or better (for open-ended long-form essay responses).

A negative score indicates an inverse correlation.  The two data sets agree somewhat, but in an opposite trend.

The Min and Max values are computed from the score data if not given.  If the score data does not cover the full range of possible scores then the minimum and maximum possible values can be specified.

This function is part of the EssayTools package, so it can be used in the short form QuadraticWeightedKappa(..) only after executing the command with(EssayTools). However, it can always be accessed through the long form of the command by using EssayTools[QuadraticWeightedKappa](..).

$\mathrm{with}\left(\mathrm{EssayTools}\right)\:$

$\mathrm{QuadraticWeightedKappa}\left(\left[9\,7\,8\,7\,6\,8\,3\,5\,10\right]\,\left[8\,8\,6\,6\,7\,6\,1\,4\,8\right]\right)$

$0.758620689655172$

$\mathrm{QuadraticWeightedKappa}\left(\left[9\,7\,8\,7\,6\,8\,3\,5\,10\right]\,\left[3\,5\,6\,6\,7\,6\,1\,4\,8\right]\right)$

$0.438095238095238$

$\mathrm{QuadraticWeightedKappa}\left(\left[9\,7\,8\,7\,6\,8\,3\,5\,10\right]\,\left[1\,2\,3\,4\,5\,6\,7\,8\,9\right]\right)$

$\mathrm{-0.166666666666667}$

$\mathrm{QuadraticWeightedKappa}\left(\left[9\,7\,8\,7\,6\,8\,3\,5\,10\right]\,\left[9\,7\,8\,7\,6\,8\,3\,5\,10\right]\right)$

$1.$

$\mathrm{with}\left(\mathrm{Statistics}\right)\:$

$N≔1000$

$N≔1000$

$\mathrm{R1}≔\mathrm{LinearAlgebra}:-\mathrm{RandomVector}\left[\mathrm{row}\right]\left(N\,\mathrm{generator}=\mathrm{rand}\left(0..10\right)\right)$

R2 := zip( proc(a,b) local v := trunc(a+b); `if`(v<0,0,`if`(v>N,N,v)) end,R1,Sample(Normal(0,2),N));

$\mathrm{QuadraticWeightedKappa}\left(\mathrm{R1}\&comma;\mathrm{R2}\right)$

$0.837098824348283$

$\mathrm{Correlation}\left(\mathrm{R1}\&comma;\mathrm{R2}\right)$

$0.845135744799198$

The EssayTools[QuadraticWeightedKappa] command was introduced in Maple 17.

For more information on Maple 17 changes, see Updates in Maple 17.