Suppose you analyze data for a group of 50 people applying for a grant. Each grant proposal was read by two readers, and each reader said “yes” or “no” to the proposal. Suppose the discrepancy count data were as follows: A and B are the drives, the data on the main diagonal of the matrix (a and d) the number of chords and the non-diagonal data (b and c) the number of disagreements: in the large area and in the self, a Kappa under 0.2 indicates a bad concordance, and a Kappa above 0.8 indicates a very good concordance beyond. Theoretically, confidence intervals are represented by the kappa subtraction of the desired DE level value times the standard kappa error. As the most frequently desired value is 95%, Formula 1.96 uses as a constant to multiply the standard error of Kappa (SE). The formula for a confidence interval is as follows: as Marusteri and Bacarea (9) have found, there is never 100% certainty about the results of the research, even if the statistical significance is reached. The statistical results used to test hypotheses about the relationship between independent and dependent variables are meaningless when there are inconsistencies in the evaluation of variables by evaluators. If the agreement is less than 80%, more than 20% of the data analysed is wrong. With a reliability of only 0.50 to 0.60, it is understandable that 40 to 50% of the data analyzed is wrong. If Kappa values are less than 0.60, the confidence intervals around the received kappa are so wide that it can be assumed that about half of the data may be false (10).

It is clear that statistical significance does not mean much when there are so many errors in the results tested. The higher the accuracy of the observer, the better the overall agreement. The relationship between the level of compliance in each level of prevalence, with different accuracy of observation. The level of compliance depends mainly on the accuracy of the observer and the prevalence of the code. The “perfect” agreement only takes place when observers .90 and 95 are accurate, while all categories obtain a substantial and higher approval majority. Kappa`s P value is rarely reported, probably because even relatively low Kappa values may differ by zero, but not large enough to satisfy investigators. [8]:66 Nevertheless, its default error has been described[9] and is calculated by different computer programs. [10] Kappa will only address its maximum theoretical value of 1 if the two observers distribute codes in the same way, i.e. if the corresponding totals are the same. Everything else is less than a perfect match. Nevertheless, the maximum value Kappa could achieve helps, as uneven distributions help interpret the actual value received from Kappa.

The equation for the maximum of:[16] Creates a ranking table from raw data in the calculation table for two observers and calculates a statistic from the Inter-Council Agreement (Kappa) to assess the agreement between two classifications on ordinal or nominal scales. If the observed agreement is due only to chance, i.e. if the evaluations are completely independent, then each diagonal element is a product of the two marginalized groups. – where xi is the number of positive reviews of mi raters for the subject i of the subjects, and therefore the default error for the tests is that the kappa statistics are zero. Weighted Kappa partly compensates for a problem with unweighted kappa, namely that it is not suited to the degree of disagreement. Disagreement is weighted as a decreasing priority by the upper left (origin) of the table.