how well does a screen actually measure what the authors claim?
TRANSCRIPT
How to spot a quality screening measure
Frances Page Glascoewww.pedstest.comwww.forepath.org
STANDARDIZATION
Children with disabilities ( 16% - 18% of total)
Naturalistic sample
Large, nationally representativeProportionate Demographics: minorities, parents’ level of education, broad geographic locations, Spanish speakers, etc.
Inter-rater--at least 85%
Test-retest--at least 85%
Internal consistency > .80
RELIABILITY
How well does a screen actually measure what
the authors claim?
Good
Poor
Good
Poor
SCREEN DX TEST
?
?
DX TESTGood
Poor
SCREEN
VALIDITYConcurrent: battery should sample same domains as screen Discriminant: information on how well various kinds of problems are detectedPredictive: optional but helpful for showing that items sample enduring aspects of development
Criterion Measure/Gold Standard
SCREEN
CRITERION-RELATED VALIDITY
=
ACCURACY
Screening sorts those who probably have
problems from those who probably don’t
ACCURACYSPECIFICITY = % of children without problems correctly detected (e.g., by passing, above cutoffs scores)SENSITIVITY = % of children with problems correctly detected70% to 80% is the standard
SPECIFICITY
SENSITIVITY
Accuracy of the Denver-II
Denver-II PASS 86
FAIL 27
69 8
Specificity = 69/86 = 80%
17 10
86 18
Developmental DX
NO YES
Sensitivity = 10/18 = 56%
ACCURACY:ACROSS AGE RANGES
AGE SENSITIVITY SPECIFICITY
N % N %0 - 18 mos 3/4 75 66/82 8018 mos - 3 yrs 27/34 79 117/149 793 - 4 1/2 yrs 26/35 74 118/165 724 1/2 - 8 yrs 42/57 74 172/245 70TOTAL 98/130 75 473/641 74
Rigorously Peer-reviewed
Because of the enormous expense involved in standardization, reliabilty and validity research and accuracy studies, tests are expensive to create, keep current, provide translations, and to ensure that users and researchers have support and guidance. For these reasons, tests costs money and it is why few measures can be photocopied. The money, however small, you spend when purchasing measures goes to their continued development.
Detection rates WITH Screening Tests
70% to 80% of children with developmental disabilities correctly identified
Squires et al, JDBP. 1996;17:420 - 427
80% to 90% of children with mental health problems correctly identified
Sturner, JDBP . 1991; 12: 51-64
Most over-referrals on standardized screens are children with below average development and psychosocial risk factors
Glascoe, APAM. 2001; 155:54-59. -