wittgenstein in heidelberg
TRANSCRIPT
-
7/29/2019 Wittgenstein in Heidelberg
1/20
Martin Diller and Paul Artes, IMAGE 2013Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
2/20
Data are complex.Analyses are complex.
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
3/20
Please write down yourjudgements!
(A to F, yes or no)
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
4/20
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
5/20
A: The likelihood that this testresult is abnormal is >99.5%.
B: The likelihood that this patienthas a normal visual field is
-
7/29/2019 Wittgenstein in Heidelberg
6/20
beyond 99.9% P.L.
between 95% and 99.9% P.L.
above 95% P.L.
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
7/20
E: The likelihood that this testresult is abnormal is >99.9%.
F: The likelihood that this patienthas a healthy optic disc is 99.9% likely.
H: The likelihood that this patient
has glaucoma is >99.9%.
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
8/20
A - H are allwrong.
(Some are dangerous.)
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
9/20
Why do we need L?Studies show ~50% undiagnosed disease.
Patients still go blind from glaucoma.
A lot of these patients present late.
Many diagnostic and therapeutic decisions are poor.All this despite a lot of technological advances.
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
10/20
positive rate in VFnegative group
positive
rate
in
VFpositive
group
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
A
B
C
D
AUC=0.82
87%
r=0.84
mean.Diff=0.38
criterion=0.67
A
!0
5
10
15
20
25
30
response
re
sponse
latency(s)
d h p h n s p d d d
0
5
10
15
20
25
30
29 40 10 11 10
positive rate in VFnegative group
positive
rate
in
VFpositive
group
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
A
B
C
D
AUC=0.79
79%
r=0.61
mean.Diff=0.27
criterion=1.3
A
0
5
10
15
20
25
30
response
re
sponse
latency(s
d h p h n s p d d d
0
5
10
15
20
25
30
62 18 9 8 3
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
11/20
Case Example
Assessment of Glaucomatous Optic Disc Damage
by Ophthalmology Residents at the University of So PauloJayme A. Vianna1, Alexandre S. Reis1,2, Lucas P. Vicente1, Marcelo Hatanaka1, Paul H. Artes2
Title
Results
Purpose
Conclusion
1Department of Ophthalmology, University of So Paulo, So Paulo, Brazil; 2Ophthalmology and Visual Sciences, Dalhousie University, Halifax, Canada
Jayme A Vianna, ARVO 2012, #644
To examine performance at diagnosingglaucomatous optic disc damage in residents at
different stages of training.
MethodsAt the end of the academic year, 40
ophthalmology residents (14, 14, 12 in the 1st,
2nd, and 3rd year or residency training) tested
themselves with the Discus software.1
The software randomly displayed 100 non-
stereoscopic optic disc photographs. Twenty(20%) were from patients with glaucoma and
confirmed visual field defect, and 80 (80%) from
patients with suspected glaucoma or ocular
hypertension with reproducibly normal visual
fields. Twenty-six (26%) of images were repeated
to evaluate consistency.
Each image was displayed for 10 seconds, and
observers had unlimited time to rate it (Figure 1).Fig 2. Graphic results of Discus, containing:
User ROC curve (colored)
Expert reference ROC curve (gray)
Area under ROC (AUC)
User AUC / Expert AUC (percentage)
Rank correlation of user and expert responsesMean difference of repeated images
Likelihood to diagnose damage as mean of responses (criterion)
Response latency for each category (inset boxplot)
Graphic representation of criterion (red line)
Expert reference of criterion (gray dashed line)
Median AuROC was smaller in the first year.
Median response latency was larger in the second
year. Decision criteria and correlation with experts
were similar among the 3 years (Table 1).
There was moderate correlation between the two
performance measures (AuROC and correlation
with experts, Spearmans = 0.61, P< 0.001), but
no relationship between either performance
measure and the decision criteria ( = -0.13 and
-0.01, P> 0.10).
Table1. Perfor
(numbers are
mance of resi
ean [standar
ents at differe
d deviation])
nt years of traiining
1st Year 2nd Year 3rd Year P*
AuROC
Correlationw/ Experts
DecisionCriteria
ResponseLatency (s)
0.69 (0.07) 0.74 (0.05) 0.74 (0.07) 0.61
0.66 (0.15) 0.65 (0.10) 0.65 (0.11) 0.86
1.92 (0.34) 1.85 (0.30) 1.81 (0.26) 0.52
5.6 (2.42) 7.6 (2.82) 5.4 (2.17) 0.04
*Kruskal-Walli
AuROC, Area
- test
, under the Rec
-
, eiver Operatin
-
, g Characterist
-
, ic
There were considerable differences in
performance, criterion, and speed, between
residents in each year of training.
Residents in the second and third year tended to
perform better than those at the first year of
training. These differences were not statistically
significant.
Discus provides a simple, rapid and objective
assessment of performance that should be useful
in many training programs. Our results will be
useful as a reference for comparing other trainees.
Fig 1. Screenshot of Discus software, showing an optic disc
photograph, and the rating scale from definitely healthy (+2) to
definitely damaged (-2). Images are displayed for up to 10 sec.
Reference
Denniss et al. Discus: investigating subjective
judgment of optic disc damage. Optometry and
Vision Science. 2011; 88(1):E93-101.
Diagnostic performance (AuROC), response
latency, decision criteria for individual
participants, and summary statistics by year, are
shown in Figure 3.
Try Discus yourself!www.discusproject.blogspot.com
Fig 3. Discus results stratified by year of residency training.
Each circle represents a single resident.
Sizes of the circles are proportional to the response latency,
colors are coded according to criterion,
bold circles and horizontal dashed lines are group medians,
and vertical dashed lines give the 25th and 75th percentiles.
The horizontal gray line and shaded area give the mean and
range from of a reference group of 10 experts (Denniss, 2011)
After completion of the test, each resident sresponses were automatically analyzed and
the results presented in a graphic (Figure 2).
unda, pril 9, 0Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
12/20
Imaging really has a place
in primary and secondaryglaucoma care.
We need to provide better
guidance on use andinterpretation.
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
13/20Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
14/20
Aim
jargon-free statements
simple, but technically accurateunderstandable to non-experts
useful rather than trivial
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
15/20
Guiding Principles
"Was sich berhaupt sagen lt, das kann manklar sagen; und wovon man nicht reden kann,
darber mu man schweigen."
"What can be said, can be said clearly,
and what cannot be said clearly, thereof weshould be silent."
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
16/20
Things should be madeas simple as possible.but no simpler.
Man muss die Dinge so einfachwie mglich machen.
Aber nicht einfacher.
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
17/20
Categories
Changetopographical analysisrate of rim area change (mm2/y * 10-3)
power (to detect a rate of -20.0 units)
Relevance of model / validity of assumptionsoutlierstests of nonlinearity & autocorrelation
Data qualityImage quality (MPHSD)Overlap of images across the series
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
18/20
Principles
Visualisationtransform each statistic via its reference distribution.show quantiles [0-10, .., 90-100] on a simple 10-segment bar chart.
Translation into verbal statementsThe data are of good quality. Typically, only 1 of 10 patients havebetter images. No change exceeding chance was statistically detectableover 5 years of follow-up (10 tests). If rapid deterioration had
occurred, it would almost certainly have been detected. Another testshould be obtained within the next 18 months.
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
19/20
Open Questions
Combining different indices to single metrice.g: series quality = median MPHSD + 85th percentile MPHSD + overlapweighted mean, linear & non-linear discriminant functions
Reference datapublished research studies vs large clinical databaseswhose data is it anyway?
Friday, March 22, 2013
-
7/29/2019 Wittgenstein in Heidelberg
20/20
We hide it effectively.