Testing the NIH Toolbox Cognitive Battery in Individuals with Traumatic Brain Injury, Spinal Cord

Injury, and Stroke

David Tulsky, Ph.D.Center for Health Assessment Research and

Translation

This project is funded in whole or in part with Federal funds from National Institute on Disability, Independent Living & Rehabilitation Research under grant number H133B090024 and the Blueprint for Neuroscience Research and the Office of Behavioral and Social Sciences Research, National Institutes of Health, under Contract No. HHS-N-260-2006-00007-C.

DisclosureI am a faculty member at the University of Delaware.

I had received grant funding from several funding agencies including NIDILRR and NIH.

I had been a member of the NIH Toolbox Steering Committee and Cognition Team.

I have no conflicts of interest to disclose

Validating Toolbox in Rehabilitation Populations

• One of the primary research studies within the Rehabilitation Research and Training Center (RRTC) grant on Improving Measurement of Medical Rehabilitation Outcomes

• R2: Validation of New Measures of Cognitive Function and Performance in Individuals with Stroke, SCI, and TBI

• NIDILRR Grant Number: H133B090024

NIHTB-CB was published as a Research Tool• NIH Toolbox Cognition Battery (NIHTB-CB) has been

“validated as a research test battery and not for clinical use, nor would it substitute for a comprehensive clinical neuropsychological examination of patients with neurobehavioral symptoms and disorders.

• It has several potential applications in clinical research and in longitudinal, large-scale epidemiologic studies where there is a need for brief instruments that tap different cognitive constructs within a very large age range …”

• Weintraub, Dikmen, Heaton, Tulsky, Zelazo, et al. 2014. P. 568

At the Same Time …• NIH Toolbox Cognition Battery (NIHTB-CB) was developed

using traditional test development techniques.• The battery was developed with extensive item

development stages, tryout, validation, and standardization.• The NIHTB-CB has state-of-the-art normative scores.• The NIHTB has demographically corrected normative data.• All point to the potential to use the NIHTB-CB in clinical

practice.However, all of the development work was with “healthy” individuals.…. the NIHTB-CB had not been tested or validated in clinical populations.

NIDILRR RRTC Final Priorities• “NIDILRR has made the development of measures of

cognitive function … a priority because adequate measures … have not been developed for systemic application in the field of medical rehabilitation.

• Cognition is both a rehabilitation outcome and a factor related to broader functional and community outcomes for individuals with a wide variety of disabling conditions.”

RRTC Collaborators

•Allen Heinemann, PhD (RIC)•David Tulsky, PhD (Delaware)•David Gray, PhD (Wash U)•Noelle Carlozzi, PhD (Michigan)• Pam Kisala, MA (Delaware)• Sofia Garcia, PhD (NUMSS)• Beth Hahn, MA (NUMSS)• Jin-Shei Lai, PhD (NUMSS)• Matt Cohen, PhD (Delaware)

• Carolyn Baum, PhD (Wash U)• Holly Hollingsworth, PhD (Wash U)• Joy Hammel, PhD (UIC)• Susan Magasi, PhD (UIC)• Bob Heaton, PhD (UCSD)• Kaitlin Casaletto, PhD (UCSD)• Anne Deutsch, PhD (RIC)• Alex Wong, PhD (Wash U)• Jim Holdnack, PhD (Delaware/Pearson)

Some Aims of the RRTC Studies

• To allow examination of the applicability of the NIH Toolbox in individuals with differing impairments

• To enable the development of standardized accommodations to ensure accessibility of the NIHTB-CB tests

• Provide the essential data base from which we could examine sensitivity, specificity, clinical utility, and construct validity of the measures.

Evaluation of New Cognitive Tests• New examiner-administered cognitive

test (Toolbox)• Traditional “Gold Standard”

neuropyschological/cognitive tests• New tests of motor, sensory, emotional

functioning (Toolbox)• New performance-based test

(Executive Function Performance Test)• New self-reported scale of cognitive

functioning (TBI-QOL/Neuro-QOL)

RRTC Field Testing Overview• Goal: 600 participants (TBI, SCI, Stroke)• Two Days: ~10 hour testing battery

• Three sites participated in data collection• In-Person Examiner Training and Certification

Completed for all Staff Members

R2/R3 Data Collection• R2 (Cognitive Tests) Data Collection

– NIH Toolbox (2.5 Hours)– Executive Function Performance Test (45 minutes)– “Gold Standard” tests (2 hours)– Neuro-QOL/TBI-QOL (15 minutes)

• R3 (Environmental Factors) Data Collection– Self-Report measures (~ 3.5 hours)

• COMBINED TESTING – = 10 hours over 2 days

11

Training and Certification

• Critical that all data collectors at each site followed standardized test administration protocols.

• To be trained in the NIH Toolbox & Neuropsychology Battery, each data collector had to go through extensive training on test administration.

• At least 5 Practice Cases• For certification - data collector was observed administering

the complete NIH Toolbox and Neuropsychological Assessment Battery

• Data Collector - 3 scored assessments for review. • Recertification – 1 year later

RRTC Validation Sample• Inclusion Criteria

– Diagnosis of traumatic TBI – Glasgow Coma Score(Severe (8 or lower), Moderate (9-12), complicated Mild (13-15 with imaging) – Glasgow Coma Score

– Diagnosis of traumatic SCI • Injury Diagnosis (Paraplegia/Tetraplegia) • Severity (Complete/Incomplete Injuries) • ASIA Neurological Exam

– Diagnosis of Stroke• Severe, Moderate, Mild (Rankin)

Step 1: Look for best GCS score within 24 hours. If GCS is rated an 8 or less, subject is classified as SEVERE.

If you see "decerebrate posturing" in MR, classify as SEVERE. Stop MR Review. Classify as MODERATE if GCS is between 9-12.Classify as MILD if GCS is between 13-15, then proceed to STEP 4 to determine if pt is complicated mild.

Stop MR Review. If no GCS, go to Step 2

Step 2:Look for report of duration of posttraumatic amnesia (PTA).(This is defined as ‘duration from time of injury to the time when medical personnel observed a return of continousmemory.’)

Rate as SEVERE if PTA is > 7 days

Rate as MODERATE if PTA is between 1 – 7 days

Rate as MILD if PTA < 1 day, then proceed to STEP 4 to determine if pt is complicated mild.

Stop MR Review. If no PTA, go to step 3 Step 3: Look for report of loss of conciousness (LOC) or coma. (LOC is defined as ‘duration from time of injury to time when the subject first followed commands’)

Rate as SEVERE if LOC / Coma duration is > 6 hours.

Rate as MODERATE if LOC / Coma duration is between 20 min – 6 hrs.

Rate as MILD if LOC / Coma duration is <20 min, then proceed to STEP 4 to determine if pt is complicated mild.

Stop MR Review. If no LOC, move to step 5. If no PTA is documented, OR if PTA is less than 24 hrs, continue to Step 5.

Step 4:FOR ALL MILD CASES Look for report of brain imaging.

If imaging is positive for any of the following, rate as complicated mild:Intracerebral hematoma (ICH) Cerebral contusionSubdural hematoma (SDH) Subarachnoid hemorrhage (SAH)Epidural hematoma (EDH) Brain stem injuryHemorrhagic contusionPenetrating TBI (including any GSW to the brain)

If brain imaging is negative, continue to Step 5.

Step 5:If no GCS, PTA or LOC is documented, but data collector feels there is evidence to infer TBI, hand off record to the neuropsychologist on staff for a rating.

Include highlighted portions of MR that include Hx, subjective notes, progress notes, appointment documentation and and imaging reports.

Prj. Manager will keep team up to date on numbers without confirmations, only 10-20% accepted.

Study Totals

SCIParaplegia Complete 48

209Paraplegia Incomplete 57Tetraplegia Complete 52

Tetraplegia Incomplete 52

TBIComplicated Mild 67 (37%)

184Moderate 16 (9%)Severe 99 (54%)

StrokeMild 60 (29%)

211Moderate 57 (27%)

Severity of Injury

Sample Demographic InformationSCI TBI CVA Total

N=209 N=184 N=211 N=604Years since injury mean (SD) 12 (10) 6 (6) 3 (2) 7 (8)

GenderMale 164 (78%) 118 (64%) 105 (50%) 387 (64%)Female 45 (22%) 66 (36%) 106 (50%) 217 (36%)

Highest EducationLess than HS 18 (9%) 24 (11%) 20 (11%) 62 (10%)HS/GED 48 (23%) 46 (22%) 42 (23%) 136 (22%)Some college 76 (36%) 80 (38%) 65 (35%) 221 (37%)College degree 67 (32%) 61 (29%) 57 (31%) 185 (31%)

Employed Yes 52 (27%) 57 (33%) 36 (18%) 145 (25%)No 146 (74%) 118 (68%) 168 (82%) 432 (75%)

Construct Validation of the NIH Toolbox• Group Differences

– Compare performance between TBI & control.– Compare performance between SCI & control.– Compare performance between stroke and demographic norms.

• Control group and matching procedure– Obtain samples of toolbox scores from the standardization sample– Match participants on Age, Education Level, Gender, & Ethnicity– Examine group differences on subtests and composite scores

• Convergent & Discriminant Validity• Replicating the Factor Structure in Clinical Groups

Toolbox & Criterion Measures

NIH Toolbox Cognition Tests Criterion Measure(s)

Picture Memory Rey Auditory Verbal Learning Test (RAVLT);Brief Visuospatial Memory Test – Revised

List Sorting Letter Number Sequence / PASAT

Pattern Comparison WAIS-IV Processing Speed Index Score (PSI)

Dimensional Change Card Sorting Wisconsin Card Sorting Test (WCST);Delis-Kaplan Executive Functioning (DKEFS)

Flanker Inhibitory Control & Attn. WCST, DKEFS

Picture Vocabulary Peabody Picture Vocabulary Test – PPVT

Reading Wide Range Achievement Test – WRAT

Additionally, there are three Composite Scores

Using the NIHTB-CB in individuals with TBI• Compare performance on individuals with complicated mild, moderate, severe TBI

against matched controls.

• Describe the sensitivity of the NIHTB-CB in individuals with TBI

• Hypothesis is that participants with TBI will produce lower scores than controls on processing speed, working memory, episodic memory, and executive functioning

• More frequent scores (<1 SD) in TBI samples with individuals who have more severe injuries scoring lower.

NIH Toolbox in TBI – Complicated Mild/Moderate, Severe, Matched Controls NIHTB Scores

Mild/Mod Severe TBI ControlaF

Mild/mod Severe

Mean (SD) Mean (SD) Mean (SD) d d

NIHTB Composite Scores N=74 N=84 N=158

Fluid 92.2 (18.7) 82.4 (18.7) 101.6 (16.0) 34.4** -0.56 -1.14

Crystalized 103.2 (15.9) 96.2 (14.7) 101.7 (15.2) 4.9* 0.09 -0.35

NIHTB Subtest Scores N=74 N=84 N=158

Picture Sequence Memory Test 92.5 (16.2) 84.2 (15.5) 100.2 (15.5) 31.6** -0.52 -1.08

Picture Vocabulary 102.9 (15.4) 97.1 (14.2) 101.5 (15.4) 3.5+ 0.09 -0.29

Oral Reading Recognition 103.2 (15.8) 95.4 (15.3) 102.0 (15.5) 4.7+ 0.08 -0.36

Pattern Comparison 96.7 (15.5) 89.8 (15.9) 101.8 (15.9) 15.0** -0.32 -0.75

List Sorting 96.3 (15.5) 91.2 (15.0) 102.1 (15.3) 14.5** -0.38 -0.71

Flanker 94.0 (17.2) 86.2 (16.4) 100.8 (14.6) 24.1** -0.44 -0.94

DCCS 94.8 (16.7) 90.1 (16.6) 101.1 (15.5) 13.5** -0.39 -0.69

NIH Toolbox in TBI – Complicated Mild/Moderate, Severe, Matched Controls NIHTB Scores

Mild/Mod Severe TBI ControlaF

Mild/mod Severe

Mean (SD) Mean (SD) Mean (SD) d d

NIHTB Composite Scores N=74 N=84 N=158

Fluid 92.2 (18.7) 82.4 (18.7) 101.6 (16.0) 34.4** -0.56 -1.14

Crystalized 103.2 (15.9) 96.2 (14.7) 101.7 (15.2) 4.9* 0.09 -0.35

NIHTB Subtest Scores N=74 N=84 N=158

Picture Sequence Memory Test 92.5 (16.2) 84.2 (15.5) 100.2 (15.5) 31.6** -0.52 -1.08

Picture Vocabulary 102.9 (15.4) 97.1 (14.2) 101.5 (15.4) 3.5+ 0.09 -0.29

Oral Reading Recognition 103.2 (15.8) 95.4 (15.3) 102.0 (15.5) 4.7+ 0.08 -0.36

Pattern Comparison 96.7 (15.5) 89.8 (15.9) 101.8 (15.9) 15.0** -0.32 -0.75

List Sorting 96.3 (15.5) 91.2 (15.0) 102.1 (15.3) 14.5** -0.38 -0.71

Flanker 94.0 (17.2) 86.2 (16.4) 100.8 (14.6) 24.1** -0.44 -0.94

DCCS 94.8 (16.7) 90.1 (16.6) 101.1 (15.5) 13.5** -0.39 -0.69

NIH Toolbox in TBI Base Rates of Low Scores and Sensitivity & Specificity

Crystalized Fluid Oral Reading Recognition

Picture Vocabulary

List Sorting

Picture Sequence Memory

Pattern Comparison Flanker DCCS

T-Score < 40Mild Complicated/Moderate

(MCMO) 12.2 33.8 17.6 10.8 27.0 36.5 25.7 32.4 31.1

Severe (SEV) 17.9 50.0 25.0 20.2 32.1 52.4 32.1 42.9 44.0

Controls 12.0 12.7 16.5 13.9 11.4 12.7 12.7 15.2 12.0

Sensitivity MCMO 0.12 0.34 0.18 0.11 0.27 0.36 0.26 0.32 0.31

Sensivity SEV 0.18 0.50 0.25 0.20 0.32 0.52 0.32 0.43 0.44

Specificity 0.88 0.87 0.84 0.86 0.89 0.87 0.87 0.85 0.88

NIH Toolbox in TBI Base Rates of Low Scores and Sensitivity & Specificity

Crystalized Fluid Oral Reading Recognition

Picture Vocabulary

List Sorting

Picture Sequence Memory

Pattern Comparison Flanker DCCS

T-Score < 40Mild Complicated/Moderate

(MCMO) 12.2 33.8 17.6 10.8 27.0 36.5 25.7 32.4 31.1

Severe (SEV) 17.9 50.0 25.0 20.2 32.1 52.4 32.1 42.9 44.0

Controls 12.0 12.7 16.5 13.9 11.4 12.7 12.7 15.2 12.0

Sensitivity MCMO 0.12 0.34 0.18 0.11 0.27 0.36 0.26 0.32 0.31

Sensivity SEV 0.18 0.50 0.25 0.20 0.32 0.52 0.32 0.43 0.44

Specificity 0.88 0.87 0.84 0.86 0.89 0.87 0.87 0.85 0.88

Convergent ValidityNIH ToolBox Test r Established Measure Languag

eWorking Memory

Episodic Memory

Processing Speed

Executive Functioning

Oral Reading Recognition 0.83 Wide Range Achievement Test-4th Edition

Reading 0.75 0.47 0.27 0.30 -0.25

Picture Vocabulary 0.80 Peabody Picture Vocabulary Test-4th Edition 0.71 0.40 0.26 0.27 -0.20

List Sorting 0.56 WAIS-IV Letter Number Sequencing 0.28 0.54 0.58 0.59 -0.46

Picture Sequence Memory 0.68 Brief Visualspatial Memory Test-Revised 0.24 0.50 0.68 0.65 -0.40

Pattern Comparison 0.69 WAIS-IV Coding 0.13 0.45 0.49 0.69 -0.43Flanker Inhibitory Control -0.46 DKEFS Color-Word Interference -

Inhibition 0.21 0.40 0.40 0.59 -0.44

DCCS -0.42 Wisconsin Cord Sort Test 0.10 0.48 0.48 0.55 -0.49

Using the NIHTB-CB in individuals with TBI• Examine the pattern of correlations with other measures

• Convergent validity – magnitude of correlations should be moderate or high correlations (e.g., > .40) with tests that are similar

• Discriminant validity – magnitude of correlations should be low with tests that are different (e.g., < .40).

Convergent ValidityNIH ToolBox Test r Established Measure Languag

eWorking Memory

Episodic Memory

Processing Speed

Executive Functioning

Oral Reading Recognition 0.83 Wide Range Achievement Test-4th Edition

Reading 0.75 0.47 0.27 0.30 -0.25

Picture Vocabulary 0.80 Peabody Picture Vocabulary Test-4th Edition 0.71 0.40 0.26 0.27 -0.20

List Sorting 0.56 WAIS-IV Letter Number Sequencing 0.28 0.54 0.58 0.59 -0.46

Picture Sequence Memory 0.68 Brief Visualspatial Memory Test-Revised 0.24 0.50 0.68 0.65 -0.40

Pattern Comparison 0.69 WAIS-IV Coding 0.13 0.45 0.49 0.69 -0.43Flanker Inhibitory Control -0.46 DKEFS Color-Word Interference -

Inhibition 0.21 0.40 0.40 0.59 -0.44

DCCS -0.42 Wisconsin Cord Sort Test 0.10 0.48 0.48 0.55 -0.49

Convergent ValidityNIH ToolBox Test r Established Measure Languag

eWorking Memory

Episodic Memory

Processing Speed

Executive Functioning

Oral Reading Recognition 0.83 Wide Range Achievement Test-4th Edition

Reading 0.75 0.47 0.27 0.30 -0.25

Picture Vocabulary 0.80 Peabody Picture Vocabulary Test-4th Edition 0.71 0.40 0.26 0.27 -0.20

List Sorting 0.56 WAIS-IV Letter Number Sequencing 0.28 0.54 0.58 0.59 -0.46

Picture Sequence Memory 0.68 Brief Visualspatial Memory Test-Revised 0.24 0.50 0.68 0.65 -0.40

Pattern Comparison 0.69 WAIS-IV Coding 0.13 0.45 0.49 0.69 -0.43Flanker Inhibitory Control -0.46 DKEFS Color-Word Interference -

Inhibition 0.21 0.40 0.40 0.59 -0.44

DCCS -0.42 Wisconsin Cord Sort Test 0.10 0.48 0.48 0.55 -0.49

Discriminant ValidityNIH ToolBox Test Established Measure Language Working

MemoryEpisodic Memory

Processing Speed

Executive Functioning

Oral Reading Recognition Wide Range Achievement Test-4th Edition Reading 0.75 0.47 0.27 0.30 -0.25

Picture Vocabulary Peabody Picture Vocabulary Test-4th Edition 0.71 0.40 0.26 0.27 -0.20

List Sorting WAIS-IV Letter Number Sequencing 0.28 0.54 0.58 0.59 -0.46

Picture Sequence Memory Brief Visualspatial Memory Test-Revised 0.24 0.50 0.68 0.65 -0.40

Pattern Comparison WAIS-IV Coding 0.13 0.45 0.49 0.69 -0.43

Flanker Inhibitory Control DKEFS Color-Word Interference - Inhibition 0.21 0.40 0.40 0.59 -0.44

DCCS Wisconsin Cord Sort Test 0.10 0.48 0.48 0.55 -0.49

Convergent and Discriminant Validity• We observed convergent correlations between .42 and .83.

– High correlations between language tests (.80 and .83).– High within-construct correlations for episodic memory and processing speed (.68 and .69).– Moderate correlations between working memory tests (.56).– Moderate correlations between executive function tests (-.46 and -.40).

• Fluid tests are correlated with other fluid constructs.

• Discriminant validity – magnitude of correlations between tests that are not related are low.

Replicating the Factor Structure• Mungas et al., 2014 examined the factor structure of the NIHTB-CB. Journal of

the International Neuropsychological Society• Performed a confirmatory factor analyses of a sample of 268 adults (20-85) in

NIHTB-CB Validation sample.• Concluded that a 5 factor model had the best fit to the data –• Vocabulary, Reading, Episodic Memory, Working Memory, Executive

Function/Processing Speed• The Executive Function tests required fast performance, so it was logical that the

executive functioning and processing factors would be combined

Can these results be replicated in a sample of individuals with Acquired Brain Injury?

Replicating the Factor Structure

Variable TBI (N=182) Stroke (N=210)

Age(Years)

M (SD) 39.1 (17.0) 56.2 (12.9)Time Since Injury (years)

M(SD) 6.0 (5.5) 2.8 (2.5)Injury Severity (%)

Complicated-Milda 36.8 28.6Moderate 8.8 27.1Severe 54.4 44.3

Cognitive Models Evaluated with Joint NIH Toolbox and Established Neuropsychological Tests CFA

1f– Global Cognition2fa– Crystalized, Fluid2fb– Memory, Non-Memory3fa– Language, Memory/Working Memory, Executive/Speed3fb – Language, Memory, Working Memory/Executive/Speed4fa – Language, Memory, Working Memory, Executive/Speed4fb– Vocabulary, Reading, Memory, Working Memory/Executive/Speed4fc– Vocabulary, Reading, Memory/Working Memory, Executive/Speed5fa– Language, Memory, Working Memory, Executive, Speed5fb– Vocabulary, Reading, Memory, Working Memory, Executive/Speed6f – Vocabulary Reading Memory Working Memory Executive Speed

Model χ2 df χ2/df CFI TLI RMSEA SRMR BIC1f 1365.71 117 11.67 0.73 0.68 0.17 0.10 1679.412a 722.31 116 6.23 0.87 0.84 0.12 0.07 1041.932b 1295.30 116 11.17 0.74 0.70 0.17 0.10 1614.923a 594.99 114 5.22 0.90 0.88 0.11 0.06 926.443b 632.37 114 5.55 0.89 0.86 0.11 0.07 963.824a 521.17 111 4.70 0.91 0.89 0.10 0.06 870.394b 440.36 111 3.97 0.93 0.91 0.09 0.06 789.584c 402.93 111 3.63 0.94 0.92 0.08 0.05 752.145a 477.46 107 4.46 0.92 0.90 0.10 0.05 850.355b 330.59 107 3.09 0.95 0.94 0.08 0.04 703.486f 287.66 102 2.82 0.96 0.95 0.07 0.04 690.15

Model χ2 df χ2/df CFI TLI RMSEA SRMR BIC1f 1365.71 117 11.67 0.73 0.68 0.17 0.10 1679.412a 722.31 116 6.23 0.87 0.84 0.12 0.07 1041.932b 1295.30 116 11.17 0.74 0.70 0.17 0.10 1614.923a 594.99 114 5.22 0.90 0.88 0.11 0.06 926.443b 632.37 114 5.55 0.89 0.86 0.11 0.07 963.824a 521.17 111 4.70 0.91 0.89 0.10 0.06 870.394b 440.36 111 3.97 0.93 0.91 0.09 0.06 789.584c 402.93 111 3.63 0.94 0.92 0.08 0.05 752.145a 477.46 107 4.46 0.92 0.90 0.10 0.05 850.355b 330.59 107 3.09 0.95 0.94 0.08 0.04 703.486f 287.66 102 2.82 0.96 0.95 0.07 0.04 690.15

Support for a 5-factor model• Results replicated model of cognition outlined by Mungas et al., 2014 in general

population sample.

• Observed support for a 6-factor model where executive functioning and processing speed are distinct factors.

• However, the correlation between executive functioning and processing speed is too high (.95) to justify distinct factors.

• The emphasis of timed performance in the scores of the Flanker and DCCS creates de facto processing speed tests.

• The results replicate those found by Mungas et al., 2014.

Special Section – In Press – Rehabilitation Psychology1. Carlozzi et al. - Construct Validity of the NIHTB-CB for use with individuals with Stroke2. Tulsky et al. – Using the NIHTB-CB in Individuals with TBI3. Cohen et al. – Cognition Among Community-Dwelling Individuals with Spinal Cord Injury4. Tulsky et al. - Confirmatory Factor Analysis of the NIH Toolbox Cognition Battery in individuals with

TBI and Stroke5. Holdnack et al. - NIH Toolbox Cognition Tests Following Traumatic Brain Injury: Base Rates of Low

Scores6. Nitsch - Uncorrected versus Demographically-corrected Scores on the NIH Toolbox Cognition

Battery: What is the Difference?7. Magasi - Test Accommodations for Individuals with Neurological Conditions Completing the NIH

Toolbox - Cognition Battery: An Evaluation of the Frequency and Appropriateness8. Carlozzi - Motor-Free composites from the NIH Toolbox Cognition Battery for people with

disabilities

Conclusions• Even though the NIHTB-CB was originally developed as a research tool, these

new studies and new normative information (demographic corrected scores, base rates) starts to provide normative information and validity to make then useful in clinical settings

• Need to test Toolbox in individuals following injury or at start of disease process and follow over time.

• Need validation efforts in other clinical groups.

Further information can be found at:

(Presentation) – Rehabilitation Psychology (likely Nov. 2017).(NIH Toolbox) - http://www.healthmeasures.net/explore-

measurement-systems/nih-toolbox

Thank you!