Mark Meterko, PhD1

Errol Baker, PhD1

Kelly L. Stolzmann, MS1

Ann Hendricks, PhD1

Keith D. Cicerone, PhD, ABPP-Cn2

Henry L. Lew, MD, PhD3

Psychometric Assessment of the Neurobehavioral

Symptom Inventory (NSI-22)

This work supported by VA HSR&D Grant: SDR 08-405

1VA Boston Healthcare System2JFK-Johnson Rehabilitation Institute3Defense and Veterans Brain Injury

Center

1

Background 1: Postconcussive Syndrome?

Studies of postconcussive symptoms have raised several issues: Is there a postconcussive syndrome (PCS)? If yes, is there a single cluster of symptoms, or

several? What symptom patterns distinguish among them?

Previous studies suggest different answers, depending on: Etiology of injury Evaluation instrument Target population Statistical procedures

2

Background 2: The VA Context

VA Policy & Process Regarding TBI Screening Comprehensive TBI Evaluation (CTE)

Includes Neurobehavioral Symptom Inventory (NSI-22)

Clinical evaluator’s overall judgment regarding history and course “consistent with a diagnosis of TBI”

3

Background 3: Prior Work

Cicerone KD & Kalmar K, 1995 (JHTR)22-item self-report inventory of symptoms50 mostly vehicular accident patientsUsing cluster analysis, 17 items grouped into

4 factors: Cognitive Affective Somatic Sensory

5 orphan items

4

Background 4: NSI22 Recent Work

Benge JF, Pastorek NJ & Thornton GM, 2009. Postconcussive symptoms in OEF-OIF Veterans: Factor

structure and impact of posttraumatic stress. Rehab Psych, 54(3), 270-278.

Exploratory factor analysis yielded 6-factor model:1. Cognitive 4. Sensory2. Vestibular 5. Headaches, Sensitivity to light3. Affective 6. Hearing, Sensitivity to noise

Caplan LJ, Ivins B, Poole JH, Vanderploeg RD, Jaffee MS, Schwab K, 2010. The structure of postconcussive symptoms in 3 US military

samples. JHTR, 25(6), 447-458. Exploratory & confirmatory factor analysis

Three models supported: 2,3 and 9 factors Endorsed 3-factor solution

1. Somatic/sensory2. Affective3. Cognitive

5

Purpose: Study Aims

1. Examine the factor structure of NSI-22 In large sample of deployed veterans judged to have

mTBI

2. Examine whether & how pain related to other symptoms in the NSI-22

3. Assess utility of the factor-based NSI-22 scales

Compare subgroups defined by: Etiology of concussion Presence/absence of PTSD

6

Methods 1: Sample7

VA National CTE database for FY08 & most of FY09 N=36,919

Random split into 2 samples Derivation (n=18,459) Confirmation (n=18,460)

Applied inclusion/exclusion criteria to both samples Keep only those with “symptoms consistent with TBI”

(n=18,649) Drop those with either pre- or post-deployment concussion

history (n=5945) Drop cases missing on pain (n=663) Drop duplicate and invalid (test case) entries (n=53)

Final samples Derivation (n=6001) Confirmation (n=5987)

Methods 2: Analyses8

Check success of randomization Compare derivation & confirmation samples on

Demographics & etiology (chi-square) NSI-22 items and pain item (MANOVA)

Examine factor structure of NSI-22 Derivation sample Exploratory factor analysis (EFA) – four runs

Empirical criteria for n of factors retained, NSI-22 only Empirical criteria, NSI-22 plus pain Specify 4 factors, NSI-22 only Specify 4 factors, NIS-22 plus pain

Methods 3: Analyses9

Confirm factor structure Confirmation sample Confirmatory factor analysis (CFA)

Utility of proposed factor-based scales Confirmation sample Stratified respondents by:

Etiology Blast, Non-Blast, Both (“Blast Plus”)

PTSD co-morbidity Dichotomous based on clinical evaluator judgment during

CTE Two-way MANOVA

Grouping factors (IV): Etiology, PTSD, Etiology x PTSD Dependent variables: NSI-22 factor scores, with and without

pain

Results 1: Randomization Success10

No significant differences, derivation vs. confirmation samples on: NSI-22 symptoms Pain Blast injury exposure TBI diagnosis Marital status Education

Borderline exception (p=.07): Employment status Derivation sample: 7.2% working part time Confirmation sample: 7.9% working part time Very small effect size (Cramer’s V = .02)

Results 2: EFA in Derivation Sample11

Using empirical criteria for N of factors to retain Three criteria

Percent variance accounted for Horn’s parallel analysis Velicer’s Minimum Average Partial (MAP) test

2- and 3-factor models emerged Same results with and without pain

Preponderance of evidence favored 3-factor model Somatosensory (11 or 12 items)

Pain loaded cleanly here when included Affective (6 items) Cognitive (4 items)

Orphan items (2 items) Hearing difficulties (no loading >=.40) Change in appetite (equal loadings < .40 on two factors)

Results 3: EFA in Derivation Sample12

Specify 4 factors a-prioriThree items from Somato-sensory form a

separate, Vestibular factor Loss of balance Dizziness Poor coordination/clumsiness

Pain remained affiliated with Somato-sensoryOrphans – same as before

Results 5: CFA in Confirmation Sample13

Table 1 Goodness of Fit Statistics for Four Confirmatory Factor Analysis Models: Confirmation Sample

Model χ2 df SRMR RMSEA CFI TLI (NNFI)

3 Factors, Without Pain 7112.91 167 0.048 0.083 0.974 0.970

3 Factors, Including Pain 7640.93 186 0.048 0.082 0.973 0.970

4 Factors, Without Pain 5578.12 164 0.044 0.074 0.979 0.976

4 Factors, Including Pain 6005.99 183 0.044 0.073 0.979 0.976

Results 6: Utility Analyses in Confirmation Sample

14

Significant main effects for both Etiology and PTSD co-morbidity Regardless of whether 3 or 4 factors were compared Regardless of whether pain was/was not included

No significant Etiology x PTSD interactions Results for PTSD and no-PTSD respondents the same

across Etiology groups

Results 7: Utility Analyses in Confirmation Sample

15

1

1.2

1.4

1.6

1.8

2

Blast Only Blast Plus Non - Blast

Etiology of Injury

Figure 1a: Somato-sensory Scale

Mean +/ - 95% CI

PTSD non - PTSD

Results 8: Utility Analyses in Confirmation Sample

16

0.7

0.9

1.1

1.3

1.5

1.7

Blast Only Blast Plus Non - Blast

Etiology of Injury

Figure 1b: Vestibular Scale

Mean +/ - 95% CI

PTSD non - PTSD

Conclusions

Scale k α Scale Content

Affective 6 .88

• Low frustration tolerance

• Irritability• Anxiety/tension

• Fatigue • Difficulties sleeping• Depressed or sad

Somatosensory

8 .81

• Light sensitivity• Noise sensitivity• Vision problems• Headaches

• Nausea• Numbness/tingling • Change in taste,

smell• Pain

Cognitive 4 .89

• Difficulties getting organized/can’t finish things • Poor concentration • Forgetfulness • Difficulties making decisions

Vestibular 3 .82• Loss of balance• Feeling dizzy• Poor coordination/clumsy

Unassigned 2 NA• Loss or increase in appetite • Hearing difficulty

17

Conclusions

18

PCS for Veterans injured during deployment as measured by NSI-22 are multi-dimensional

Pain associated with Somto-sensory factor in all solutions

By technical criteria, no substantial difference between 3- and 4-factor models

Prefer 4-factor model In EFA: No dual-loading items in EFA In CFA: Fit statistics marginally but consistently better Interpretability and utility of 4-factor model

Increased potential for differentiation among clinical groups