measures of reliability and multivariate analysis
DESCRIPTION
Measures of Reliability and Multivariate Analysis. Objectives. To discuss measures of reliability To discuss multivariate analysis. Kappa Statistic. p o = observed probability of concordance between the two surveys p e = expected probability of concordance between the two surveys - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/1.jpg)
Measures of Reliabilityand
Multivariate Analysis
![Page 2: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/2.jpg)
Objectives
• To discuss measures of reliability
• To discuss multivariate analysis
![Page 3: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/3.jpg)
Kappa Statistic
• po = observed probability of concordance between the two surveys
• pe = expected probability of concordance between the two surveys
• The standard error of the Kappa statistic is calculated by:
• To test the hypothesis Ho:=0 vs. H1:0, use the test statistic:
![Page 4: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/4.jpg)
![Page 5: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/5.jpg)
Percent agreement
![Page 6: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/6.jpg)
![Page 7: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/7.jpg)
Figure 1. Association of average faculty performance rating (from 1, bottom 20%, to 5, top 20%) and absolute rank on the National Resident Matching Program (NRMP) list (r = 0.19; P =.11).
![Page 8: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/8.jpg)
Table 1. Discrepancy Between the DIS and SCAN for the Lifetime Occurrence of Depressive Disorder in the
Baltimore ECA Follow-up*
![Page 9: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/9.jpg)
Table 1-Comparison of WHO and ADA diagnostic categories for undiagnosed diabetes From: Lee: Diabetes Care, Volume 23(2).February 2000.181-186
![Page 10: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/10.jpg)
Kappa score calculation
PlaqueNormal Expected chance agreementPlaque 140 52 192 40.70 Normal 69 725 794 625.70
209 777 986
Pe= 40.7 626 0.676Po= 140 725 0.877
k = 0.877 - .676 = 0.62----------------1.0 - .676
![Page 11: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/11.jpg)
![Page 12: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/12.jpg)
Kappa and prevalence
• You have two populations of 1000 each being compared by interview A and interviewer B
• One has a prevalence of 5 and the other has a prevalence of 40
• Sensitivities and specificities for observer A and B are .90 and .90 and .80 and .95 respectively
• Create a table for each population
![Page 13: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/13.jpg)
Prevalence and Kappa
• Kappa tends toward 0 as the prevalence approaches either 0 or 1
• High values of kappa can be obtained when the marginals of the contingency table are unbalanced. (tends to be higher when the positivity prevalence is different between A and B
![Page 14: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/14.jpg)
![Page 15: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/15.jpg)
Kappa scores for table 8-18
![Page 16: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/16.jpg)
![Page 17: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/17.jpg)
![Page 18: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/18.jpg)
![Page 19: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/19.jpg)
![Page 20: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/20.jpg)
![Page 21: Measures of Reliability and Multivariate Analysis](https://reader036.vdocuments.us/reader036/viewer/2022062321/56813787550346895d9f2067/html5/thumbnails/21.jpg)