ARTICLE OPEN ACCESS

Evaluation of multiple sclerosis disabilityoutcome measures using pooled clinical trial dataMyla D Goldman MD MSc Nicholas G LaRocca PhD Richard A Rudick MD Lynn D Hudson PhD

Peter S Chin MD Gordon S Francis MD Adam Jacobs PhD Raj Kapoor FRCP Paul M Matthews MD

Ellen M Mowry MD MCR Laura J Balcer MD Michael Panzara MD Glenn Phillips PhD

Bernard MJ Uitdehaag MD and Jeffrey A Cohen MD on behalf of the Multiple Sclerosis Outcome

Assessments Consortium

Neurologyreg 201993e1921-e1931 doi101212WNL0000000000008519

Correspondence

Dr Cohen

cohenjccforg

AbstractObjectiveWe report analyses of a pooled database by the Multiple Sclerosis Outcome AssessmentsConsortium to evaluate 4 proposed components of a multidimensional test battery

MethodsStandardized data on 12776 participants comprising demographics multiple sclerosis diseasecharacteristics Expanded Disability Status Scale (EDSS) score performance measures andShort Formndash36 Physical Component Summary (SF-36 PCS) were pooled from control andtreatment arms of 14 clinical trials Analyses of Timed 25-Foot Walk (T25FW) 9-Hole PegTest (9HPT) Low Contrast Letter Acuity (LCLA) and Symbol Digit Modalities Test(SDMT) included measurement properties construct convergent and known group validityand longitudinal performance of the measures individually and when combined into a multi-dimensional test battery relative to the EDSS and SF-36 to determine sensitivity and clinicalmeaningfulness

ResultsThe performance measures had excellent testndashretest reliability and showed expected differencesbetween subgroups based on disease duration and EDSS level Progression rates in detectingtime to 3-month confirmed worsening were lower for T25FW and 9HPT compared to EDSSwhile progression rates for LCLA and SDMTwere similar to EDSS When the 4 measures wereanalyzed as a multidimensional measure rather than as individual measures progression on anyone performance measure was more sensitive than the EDSS Worsening on the performancemeasures analyzed individually or as a multidimensional test battery was associated withclinically meaningful SF-36 PCS score worsening supporting clinical meaningfulness of des-ignated performance test score worsening

ConclusionThese results support the use of the 4 proposed performance measures individually or com-bined into a multidimensional test battery as study outcome measures

RELATED ARTICLE

EditorialMeasuring disability inmultiple sclerosis Walkingplus much more

Page 919

MORE ONLINE

CME CourseNPuborgcmelist

From the University of Virginia (MDG) Charlottesville National Multiple Sclerosis Society (NGL) New York NY Biogen (RAR GP) Cambridge MA Critical Path Institute (LDH)Tucson AZ Genentech (PSC) South San Francisco CA Independent Neurology Clinical Development Consultant (GSF) Premier Research (AJ) Wokingham UK UCL Institute ofNeurology (RK) London UK Imperial College London and UK Dementia Research Institute (PMM) Johns Hopkins (EMM) Baltimore MD New York University School of Medicine(LJB) NY Wave Life Sciences (MP) Cambridge MA VU University Medical Center (BMJU) Amsterdam the Netherlands and Cleveland Clinic (JAC) OH

Go to NeurologyorgN for full disclosures Funding information and disclosures deemed relevant by the authors if any are provided at the end of the article

Multiple Sclerosis Outcome Assessments Consortium coinvestigators are listed at linkslwwcomWNLA995

The Article Processing Charge was funded by the National Multiple Sclerosis Society

This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License 40 (CC BY-NC-ND) which permits downloadingand sharing the work provided it is properly cited The work cannot be changed in any way or used commercially without permission from the journal

Copyright copy 2019 The Author(s) Published by Wolters Kluwer Health Inc on behalf of the American Academy of Neurology e1921

Recognition of the limitations of existing measures of mul-tiple sclerosis (MS)ndashrelated disability1 led to developmentof the Multiple Sclerosis Functional Composite (MSFC) asan alternative clinical outcome measure23 The MSFC in-tegrated 3 quantitative performance measures Timed 25-Foot Walk (T25FW) 9-Hole Peg Test (9HPT) and PacedAuditory Serial Addition Test (PASAT)23 Component testscores were normalized by conversion to z scores usinga reference sample and averaged to create the MSFC scoreSubsequent utilization of the MSFC in clinical trials andother studies highlighted several advantages high reliabilitycoverage of cognition which is not adequately captured byother outcome measures a single score on a continuousscale ease of use and capacity to be administered bya trained technician rather than a physician However whileutilization of z scores offered statistical advantages it im-peded MSFC acceptance and adoption Both regulatoryagencies and clinicians had difficulty interpreting the clinicalmeaningfulness of the MSFC score

Subsequent research provided guidance around thresholds ofMSFC measure variability and clinically meaningful change4ndash7

In addition researchers recommended adding Low ContrastLetter Acuity (LCLA) a test of visual impairment validated inMS89 and replacing the PASAT with the Symbol Digit Mo-dalities Test (SDMT)10 In this article we report the initialanalyses of a pooled database to assess measurement proper-ties sensitivity construct convergent and known group val-idity and clinical meaningfulness of 4 performance tests(T25FW 9HPT LCLA and SDMT) individually and com-bined into a multidimensional outcome measure of MS-relateddisability

MethodsStudy designThe Multiple Sclerosis Outcome Assessments Consortium(MSOAC) was established in 2012 to accelerate developmentand validation of improved clinical outcome measures of MS-related disability1112 The organization of MSOAC and overallapproach to develop and validate clinical outcome measures ofMS-related disability have been reported previously13 Fre-quent interactions with the European Medicines Agency andUS Food and Drug Administration contributed to MSOACrsquosresearch approach13 After executing data use agreementsMSOAC obtained prospectively acquired patient-level data

from 16 clinical trials including 14370 participants and com-bined these into a single database Fourteen of these trialscomprising 12776 patients were used for the analyses reportedhere

To allow aggregation of data from distinct datasets a commondata standard was developed through the Coalition for Ac-celerating Standards and Therapies and data from eachclinical trial were remapped to the Clinical Data InterchangeStandards Consortium data standard to create a single pooleddata set14 The standardized data comprising the control andtreatment arms of clinical trials formed the MSOAC databaseData elements included demographics MS disease charac-teristics treatment relapse data Expanded Disability StatusScale (EDSS) score performance measures and patient-reported outcomes13 TheMSOAC database does not includethe MRI data

Outcome measuresThese analyses focused on 4 performance measures and theirrelation to other measures in the database T25FW (shortdistance walking speed to measure ambulation)15 9HPT(dexterity and upper extremity motor function)16 LCLA(vision)9 and SDMT (cognitive processing speed and sus-tained attention)10 Data on PASAT are presented else-where17 The EDSS is an ordinal scale ranging from 0 to 10based on the severity of findings on the neurologic examina-tion walking ability and ability to carry out activities of dailyliving with higher scores indicating worse disability18 TheShort Formndash36 (SF-36) is a 36-item questionnaire thatincludes 8 multi-item health concepts (Physical FunctioningRole-Physical Bodily Pain General Health Vitality SocialFunctioning Role-Emotional and Mental Health)19 Scoresare a mean of subsetted questions and range from 0 to 100higher scores indicate better health-related quality of life(HRQoL) The SF-36 has 2 summary scales the PhysicalComponent Summary (PCS) and the Mental ComponentSummary (MCS) whose calculation produces a T-score witha mean score of 50 and SD of 10 representing the referencescore for the US general population The Beck DepressionInventory (BDI) is a 21-item self-report measure of de-pression with scores ranging from 0 to 62 and higher scoreindicating more severe depression symptoms20

For these analyses worsening was defined as follows T25FW(20 increase)15 9HPT (20 increase)16 LCLA with 25contrast (20 or 7-letter decrease)9 SDMT (4-point

Glossary9HPT = 9-Hole Peg Test BDI = Beck Depression Inventory CI = confidence interval EDSS = Expanded Disability StatusScale HRQoL = health-related quality of life ICC = intraclass correlation coefficient LCLA = Low Contrast Letter AcuityMCS =Mental Component SummaryMS = multiple sclerosisMSFC =Multiple Sclerosis Functional CompositeMSOAC =Multiple Sclerosis Outcome Assessments Consortium PASAT = Paced Auditory Serial Addition Test PCS = PhysicalComponent Summary RR = relapsing-remitting SDMT = Symbol Digit Modalities Test SFndash36 = Short Form-36 T25FW =Timed 25-Foot Walk

e1922 Neurology | Volume 93 Number 21 | November 19 2019 NeurologyorgN

decrease)10 EDSS (baseline score 0 15-point increasebaseline score 10ndash55 1-point increase baseline score ge6005-point increase)21 and SF-36 PCS Score (5-point wors-ening)22 For all measures except SF-36 PCS the worseninghad to be sustained for at least 3 months

Statistical methodsNo imputation was done for missing data other than forparticipants unable to complete the T25FW or 9HPT becauseof disability Following convention imputation for patientsunable to perform was 180 seconds for T25FW and 300seconds for 9HPT23 The MSFC administration and scoringmanual states that for T25FW testing patients should use theirusual assistive devices and an effort should be made to use thesame device over the course of the study Summary scores ofthe SF-36 MCS and PCS were calculated using standardmethods which provide T-scores for analysis For the SF-36 8health concept scores Quality Metrics Health OutcomesScoring Software was utilized The maximum data recoverymethod was used to handle missing data If any individualitem was missing for the BDI score the total score was notcalculated for that participant and time point

Testndashretest reliability was assessed by intraclass correlationcoefficient (ICC) of all administrations of each test (2ndash6compared with test 1) based on periods in which patientstatus on the EDSS did not change and not exceeding 6months from baseline24 Correlations among the EDSS andperformance tests were assessed by Spearman rank correla-tion coefficient Time to confirmed clinically meaningfulworsening was analyzed by Kaplan-Meier methods Cohenkappa coefficient was used to assess agreement in worseningin different disability measures Baseline outcome measurescores were compared The baseline score for each perfor-mance measure was compared between the groups of patientsbased on disease duration and EDSS score using an analysis ofvariance model adjusting for age in 5-year age bands

Data availabilityPer data use agreements analyses were done through a con-tract research organization (Premier Research) under theoversight of the Critical Path Institute Pooled data were notavailable to individual sponsors or academic members al-though per agreement placebo data were made availablepublicly25 Consortiummembers contributed to developmentof the statistical analysis plan and had access to results from allanalyses The authors had full access to all the data generatedin this fashion

ResultsTable 1 summarizes the data available baseline demographicsdisease characteristics EDSS score performance test resultsand participant self-reported measures Overall the pop-ulation was relatively young with a recent diagnosis of MSpredominantly relapsing-remitting (RR) course and mild

disability Although fewer studies included LCLA SDMTand self-reported measures substantial data were available forall outcome measures

The frequency distributions of the T25FW and 9HPT werepositively skewed and showed floor effects with scorestending to be clustered at shorter times (figure 1) Bothpossessed the ability to distinguish gradations of performancein the middle of the scale LCLA distribution appeared mildlynegatively skewed without floor or ceiling effects SDMTscores showed no evidence of skewing or floor or ceilingeffects

Table 2 summarizes trends over the first 6 assessments forthe performance tests T25FW 9HPT and LCLA tended toworsen over time and showed minimal or no practice effectswhile the SDMT demonstrated modest practice effectsTestndashretest reliability was estimated by calculating the ICCaccounting for practice effects where needed All of themeasures showed good testndashretest reliability though theICC for T25FW was somewhat lower (071) compared tothe other tests (084ndash088)

To compare sensitivity to change of the performance meas-ures with EDSS time from baseline to 3-month confirmedworsening over 24 months was analyzed (figure 2) The studypopulations available for each comparison differed leading todiffering proportions with 3-month confirmed worsening onEDSS Using a 20 threshold for T25FW 65 worsenedcompared to 202 on EDSS Using a 20 threshold for9HPT 29 worsened compared to 202 on EDSS Using7-point threshold for LCLA 131 worsened compared to161 on EDSS Using 4-point threshold for SDMT 150worsened at 18 months compared to 114 on EDSS at 18months and 145 at 24months Thus progression rates werelower for T25FW and 9HPT compared to that of EDSS whileprogression rates for LCLA and SDMT were similar to orhigher than compared to EDSS When the performance testswere combined into a multidimensional outcome measurethe proportion of participants worsening on any one perfor-mance test was substantially greater than the proportionworsening on EDSS When worsening on 2 performance testswas required sensitivity to disability progression was some-what reduced compared to the EDSS The progression eventsdefined by the performance tests were weakly associated withor independent of those defined by the EDSS T25FW(Cohen κ = 002 95 confidence interval [CI] minus000 to 003)9HPT (κ = 000 95CI minus001 to 001) LCLA (κ = 011 95CI 008 to 014) and SDMT (κ = minus002 95 CI minus006to 002)

To investigate construct and convergent validity correlationsbetween the performance measures and EDSS were analyzed(table 3) The T25FW and 9HPT correlated strongly withone another and demonstrated the strongest correlation tothe EDSS relative to other performance measures LCLA andSDMT were weakly correlated to the other performance

NeurologyorgN Neurology | Volume 93 Number 21 | November 19 2019 e1923

measures and EDSS Between the two the SDMT hadsomewhat stronger correlation to the EDSS Cross-sectionalcorrelations among outcomes at baseline were notablystronger than the correlations among changes from baselineto endpoint which had a similar pattern of correlativestrength (T25FW gt 9HPT gt SDMT gt LCLA) but whollyweaker in magnitude

Known group validity was assessed as a function of diseaseduration and disability level (table 4) At baseline values forall 4 performance measures were better in participants withMS of shorter duration (lt10 years since symptom onset)compared to those with disease of longer duration (ge10years) Similarly the results on all 4 performance tests werebetter in participants with lower EDSS scores (0ndash35) vs thosewith higher EDSS scores (40ndash10)

To explore clinical meaningfulness correlations were cal-culated between performance measures and participant self-reported measures of HRQoL and depression (table 3) Atbaseline T25FW 9HPT and SDMT correlated moderatelywith SF-36 PCS and significantly but weakly with MCS andBDI LCLA correlated weakly with SF-36 PCS and MSCand BDI Correlations between change baseline to endpoint

in the performance measures and change on SF-36 PCS orMCS or BDI were generally not significant and weak at best(table 3) Among participants with worsening from baselineto endpoint on the T25FW 9HPT or SDMT the mean SF-36 PCS also worsened (p lt 0001 p lt 0001 and p = 00308respectively) (table 5) Similarly among participants whoshowed baseline to endpoint worsening on the T25FW9HPT or SDMT the proportions of participants with5-point PCS worsening on SF-36 PCS were greater Non-significant trends were seen for mean PCS change and theproportion with 5-point PCS change among participantswho did or did not experience baseline to endpoint wors-ening on LCLA Mean PCS worsened among participantswith baseline to endpoint worsening in each of 2 groupsthose with worsening on any one measure and those wors-ening on 2 or more performance measures The SF-36 PCSwas improved or stable respectively among participantswho did not worsen on 1 or on 2 or more performancemeasures Similarly the proportions of participants with5-point SF-36 PCS worsening were greater among partic-ipants who showed baseline to endpoint worsening on 1 oron 2 or more performance measures suggesting that wors-ening on any single performance measure was clinicallyrelevant

Table 1 Baseline characteristics

Measure N Mean (SD) or n () Median (range)

Age y 12776 395 (992) 400 (17ndash72)

Sex n () 12776

Female 8799 (689)

Male 3977 (311)

Disease course 12776

Relapsing-remitting 10789 (844)

Secondary progressive 1044 (82)

Primary progressive 943 (74)

Disease duration y 6641 65 (726) 40 (0ndash48)

EDSS 12776 29 (163) 25 (0ndash80)

T25FW s 11649 76 (984) 54 (10ndash2310)a

9HPT s 11653 243 (1430) 213 (50ndash3310)a

LCLA 25 contrast number correct 5669 346 (1165) 370 (0ndash60)

SDMT number correct 2583 479 (1590) 480 (0ndash110)

PCS 7766 415 (995) 409 (10ndash73)

MCS 7766 477 (1153) 495 (minus5 to 74)

BDI 2824 89 (862) 70 (0ndash53)

Abbreviations 9HPT = 9-Hole Peg Test BDI = Beck Depression Inventory EDSS = ExpandedDisability Status Scale LCLA = LowContrast Letter Acuity with 25contrast MCS = Short Formndash36 Mental Component Summary PCS = Short Formndash36 Physical Component Summary SDMT = Symbol Digit Modalities TestT25FW = Timed 25-Foot Walka The extreme values were verified as being present in the database

e1924 Neurology | Volume 93 Number 21 | November 19 2019 NeurologyorgN

DiscussionWe present analyses to characterize the measurement prop-erties sensitivity construct convergent and known groupvalidity and clinical meaningfulness of 4 performancemeasuresmdashT25FW 9HPT LCLA and SDMTmdashto permituse individually or combined into a multidimensional testbattery as a primary or co-primary outcomemeasure We haveassessed the components of the proposed multidimensionaltest battery in relation both to the EDSS and self-reportedmeasures of health-related quality of life and depressionThese results based on a database of 14 datasets comprising12776 participants represent the largest pooled analysis ofprospectively acquired clinical trial data in MS to date Thedemographics of the pooled dataset largely reflect the type of

patents historically enrolled in MS clinical trials for whichtrials in RRMS have predominated

The distributions of the T25FW and 9HPT were positivelyskewed and both measures demonstrated floor effects Thepotential for a high proportion of patients to perform thesemeasure as well as can be performed by a healthy control canresult in reduced ability to distinguish gradations of perfor-mance at the lower end of the scale (demonstrated by far leftpeaks in figure 1 A and B) Baseline LCLA and SDMT scoreswere more normally distributed without evidence of floor orceiling effects The T25FW 9HPT and LCLA showed noclearcut evidence of practice effects As is typical of mostcognitive measures the SDMT exhibited some practiceeffects but these appeared not to affect the normality of the

Table 2 Practice effects and testndashretest reliability of performance measures with tests 2ndash6 each compared to test 1

Test N Test 2 Test 3 Test 4 Test 5 Test 6 ICC

T25FW 7971 008 008 005 008 013 071

9HPT 7973 002 000 minus003 minus004 000 084

LCLA 4611 minus002 minus003 minus001 000 000 088

SDMT 2094 003 010 015 028 037 085

Abbreviations 9HPT = 9-Hole Peg Test ICC = intraclass correlation coefficient (ameasure of reliability higher is better 1 is themaximumpossible score) LCLA= Low Contrast Letter Acuity with 25 contrast SDMT = Symbol Digit Modalities Test T25FW = Timed 25-Foot WalkThe values for tests 2ndash6 are the regression coefficients for the 2nd to 6th test expressed as an effect size tomake them comparable For example with T25FWthe 2nd test was on average 008 SDs higher than the first test

Figure 1 Distribution of performance measure scores at baseline

(A) Timed 25-Foot Walk (T25FW) (seconds) (B) 9-Hole Peg Test (9HPT) (seconds) (C) Low Contrast Letter Acuity (LCLA) with 25 contrast (number correct) (D)Symbol Digit Modalities Test (SDMT) (number correct)

NeurologyorgN Neurology | Volume 93 Number 21 | November 19 2019 e1925

SDMTrsquos frequency distribution All 4 performance measuresdemonstrated good testndashretest reliability indicating they yieldreproducible scores if there is no change in the participantrsquoscondition As a result changes in a score can be assumed to bedue to the participantrsquos condition rather than measurementvariability These results support the advantageous measure-ment properties of the 4 performance measures

These results provide a cautionary note regarding the pop-ulation for which these measures will be most useful The

majority of participants represented in the pooled dataset hadRRMS with relatively mild disability In turn the T25FW and9HPT exhibited floor effects which may explain the decreasedsensitivity of 3-month confirmed worsening of T25FW and9HPT compared to EDSS More sensitive tests may be neededin studies enrolling participants with mild gait and upper ex-tremity impairments21 One might question whether con-firmed worsening on EDSS at the low end of the scale inpatients with MS with mild impairment represents increasingdisability or simply new signs on the neurologic examination

Figure 2 Kaplan-Meier graphs of time to 3-month confirmed disability worsening on performancemeasures compared toExpanded Disability Status Scale

(A) Timed 25-Foot Walk (T25FW) (B) 9-Hole Peg Test (9HPT) (C) Low Contrast Letter Acuity (LCLA) with 25 contrast (D) Symbol Digit Modalities Test (SDMT)(E) Any 1 or any 2 performance measures

e1926 Neurology | Volume 93 Number 21 | November 19 2019 NeurologyorgN

At baseline the T25FW and 9HPT had stronger correlationswith the EDSS and with each other than with the other per-formance measures These results support the construct val-idity of the T25FW and 9HPT as both are measures ofphysical functions that overlap with the EDSS in its lowerrange (EDSS 0ndash40) as seen in this population In compari-son LCLA and SDMT correlated less strongly with EDSS andthe other performance measures supporting their additivevalue to assess functions not captured by the other perfor-mance measures and EDSS Compared to correlations atbaseline all the correlations for change from baseline toendpoint were much weaker Cohen kappa coefficients

showed that the confirmed worsening events defined by the 4performance measures were largely independent of thosedefined by EDSS Taken together these results suggest thatthe 4 performance measures assess overlapping but somewhatdifferent aspects of disability and disability worsening thandoes the EDSS

All 4 performance measures were worse in participants withlonger MS disease duration and with worse disability mea-sured by EDSS supporting known group validity Exploratoryanalyses were undertaken to assess the clinical meaningfulnessof worsening on the performance measures using the SF-36

Table 3 Correlations between outcome measures

9HPT LCLA SDMT EDSS PCS MCS BDI

Baseline correlations

T25FW 052(051 to 053)

minus030(minus032 to minus027)

minus042(minus046 to minus038)

056(055 to 058)

minus040(minus042 to minus038)

minus013(minus016 to minus011)

022(018 to 026)

9HPT minus033(minus035 to minus031)

minus047(minus051 to minus043)

054(053 to 056)

minus033(minus036 to minus031)

minus014(minus016 to minus011)

020(016 to 024)

LCLA 034(030 to 039)

minus029(minus031 to minus027)

012(009 to 014)

019(016 to 022)

minus016(minus020 to minus012)

SDMT minus034(minus038 to minus029)

036(032 to 041)

021(016 to 026)

minus020(minus024 to minus015)

9HPTchange LCLA change SDMT change EDSS change PCS change MCS change BDI change

Correlations of changefrom baseline toendpoint

T25FW change 030(028 to 032)

minus008(minus011 to minus006)

minus014(minus019 to minus009)

029(027 to 031)

minus020(minus023 to minus018)

minus009(minus012 to minus006)

010(005 to 014)

9HPT change minus006(minus009 to minus004)

minus020(minus025 to minus015)

023(022 to 025)

minus016(minus019 to minus013)

minus007(minus010 to minus005)

011(007 to 016)

LCLA change 006(001 to 011)

minus011(minus013 to minus008)

002(minus001 to 005)

006(003 to 010)

minus002(minus007 to 003)

SDMT change minus012(minus016 to minus008)

000(minus001 to 005)

006(003 to 010)

minus009(minus013 to minus004)

Abbreviations 9HPT = 9-Hole Peg Test CI = confidence interval LCLA = Low Contrast letter Acuity with 25 contrast SDMT = Symbol Digit Modalities TestT25FW = Timed 25-Foot WalkValues are Spearman correlation coefficients (95 CI)

Table 4 Known group analysis of baseline values based on disease duration and disability level

Disease duration y EDSS

lt10 ge10 Difference (95 CI) p value 0ndash35 40ndash10 Difference (95 CI) p Value

T25FW s 77 133 N = 5597 557 (474 to 640) p lt 00001 61 127 N = 11595 663 (621 to 706) p lt 00001

9HPT s 243 299 N = 5599 557 (448 to 665) p lt 00001 217 318 N = 11594 1010 (948 to 1072) p lt 00001

LCLA number correct 332 307 N = 3579 minus250 (minus375 to minus125) p lt 00001 348 274 N = 5787 minus746 (minus833 to minus660) p lt 00001

SDMT number correct 485 452 N = 2543 minus331 (minus485 to minus177) p lt 00001 498 412 N = 2583 minus860 (minus1009 to minus712) p lt 00001

Abbreviations 9HPT = 9-Hole Peg Test CI = confidence interval EDSS = ExpandedDisability Status Scale LCLA = LowContrast Letter Acuity with 25 contrastSDMT = Symbol Digit Modalities Test T25FW = Timed 25-Foot Walk

NeurologyorgN Neurology | Volume 93 Number 21 | November 19 2019 e1927

Table 5 Change in Short Formndash36 Physical Component Summary (PCS) in participants with and without worsening on Expanded Disability Status Scale (EDSS) andperformance measures

Disabilitymeasure N

Absolute changein PCS (SD)amongparticipants withdisabilitymeasureworsening

Absolute changein PCS (SD)amongparticipantswithoutdisabilitymeasureworsening p Value

Percent (95 CI)with 5-point PCSworseningamongparticipants withdisabilitymeasureworsening

Percent (95 CI)with 5-point PCSworseningamongparticipantswithoutdisabilitymeasureworsening

Odds ratio (95CI) p Value

EDSS Total 7455 minus275 (821) 043 (754) p lt 00001 366 (341 to 391) 205 (195 to 216) 223 (198 to 253)p lt 00001

Worse 1479

Not worse 5976

T25FW (20) Total 7455 minus218 (783) 037 (767) p lt 00001 334 (311 to 357) 209 (199 to 220) 189 (168 to 213)p lt 00001

Worse 1666

Not worse 5789

9HPT (20) Total 7455 minus286 (833) 004 (768) p lt 00001 386 (347 to 425) 223 (214 to 234) 218 (184 to 259)p lt 00001

Worse 622

Not worse 6833

LCLA (7 point) Total 4678 003 (795) 038 (749) p = 02907 221 (188 to 257) 200 (188 to 213) 113 (092 to 140)p = 02662

Worse 570

Not worse 4108

SDMT (4-point) Total 1467 minus115 (819) minus004 (769) p = 00308 288 (237 to 344) 222 (199 to 247) 142 (106 to 189)p = 00201

Worse 288

Not worse 1179

Worse on any 1 performancemeasure (T25FWor 9HPT or LCLA or SDMT)

Total 7455 minus163 (794) 051 (760) p lt 00001 308 (290 to 327) 202 (190 to 213) 177 (158 to 197)p lt 00001

Worse 2478

Not Worse 4977

Continued

e1928Neu

rology

|Vo

lume93N

umber

21|

Novem

ber

192019NeurologyorgN

PCS as an anchor SF-36 PCS correlated moderately atbaseline with T25FW 9HPT and SDMT and weakly withLCLA SF-36 MCS and BDI correlated weakly with all 4performance measures at baseline Group aggregate changesfrom baseline to endpoint in the performance measures andself-report measures correlated weakly or not at all when di-rectionality was not considered Importantly for participantsexperiencing confirmed worsening from baseline to endpointon the T25FW 9HPT and SDMT the SF-36 PCS was sig-nificantly worse and the likelihood of a 5-point worsening onSF-36 PCS considered clinically important22 was signifi-cantly greater The LCLA results mirrored these findings butwith statistically nonsignificant trends These results indicatethat the T25FW 9HPT and SDMT assess clinically mean-ingful aspects of MS-related disability and that the proposedthresholds for clinically meaningful change for each are rea-sonable The nonsignificant trend of concomitant worseningin the LCLA and SF-36 PCS provides some support for theclinical meaningfulness of 7-letter change in LCLA Theclinical meaningfulness of the 7-letter LCLA worsening hasbeen demonstrated previously using validated visual qualityof life scales9

Limitations of this work include the availability of somewhatfewer data for LCLA SDMT and self-reported measuresRelatively few datasets contained all 4 performance measuresand EDSS limiting the analyses of their relative sensitivityincluding in RR vs progressive MS as a function of baselinedisability level and to capture worsening disability in-dependent of relapse In addition our ability to fully exploreclinical meaningfulness of the performance measures usingself-report measures also was restricted by lack of self-reportmeasures used across studies beyond SF-36 and BDI Othermeasures of self-report have been applied to the MS pop-ulation but these analyses were limited by the surveys in theexisting dataset that is the SF-36 Also although the datasetincluded the full range of disability the majority of partic-ipants had RRMS and relatively mild disability with medianEDSS of 25 reflecting the over-representation of clinical trialsin RRMS in the MS field This point may limit a full un-derstanding of the performance tests in more disabled pro-gressive populations Finally for these analyses we pooledtreatment groups and focused on 3-month confirmed dis-ability worsening (rather than 6-month) which could haveaffected the results

These results confirm the advantageous measurementproperties of the T25FW 9HPT LCLA and SDMT andsupport their construct convergent and known group val-idity and sensitivity particularly when combined intoa multidimensional test battery The associations withestablished measures of disability (EDSS) and HRQoL (SF-36) indicate that they evaluate clinically meaningful aspectsof MS-related disability These findings support the use ofthese measures either alone or together as a multidimen-sional test battery as primary or key secondary endpoints inMS studiesTa

ble

5Chan

gein

ShortFo

rmndash36

PhysicalComponen

tSummary(PCS)

inparticipan

tswithan

dwithoutw

orsen

ingonEx

pan

ded

Disab

ility

StatusSc

ale(EDSS

)andperform

ance

mea

sures(con

tinue

d)

Disability

measu

reN

Abso

lute

change

inPCS(SD)

among

participants

with

disability

measu

reworsening

Abso

lute

change

inPCS(SD)

among

participants

without

disability

measu

reworsening

pValue

Percent(95

CI)

with5-pointPCS

worsening

among

participants

with

disability

measu

reworsening

Percent(95

CI)

with5-pointPCS

worsening

among

participants

without

disability

measu

reworsening

Oddsra

tio(95

CI)p

Value

Worseonany2ormore

perform

ance

measu

res

Total745

5minus243

(826)

minus000

(771)

plt000

0135

4(316to

393)

226

(217to

237)

187

(157to

223

)plt000

01

Worse

616

NotWorse

683

9

Abbreviations

9HPT=9-Hole

Peg

TestC

I=co

nfiden

ceintervalL

CLA

=Lo

wContras

tLe

tter

Acu

itywith25

contras

tSD

MT=Sy

mbolD

igitModalitiesTe

stT

25FW

=Timed

25-FootWalk

NeurologyorgN Neurology | Volume 93 Number 21 | November 19 2019 e1929

AcknowledgmentPMM acknowledges support for activities related to thiswork from the NIHR Imperial Biomedical Research Centre

Study fundingMSOAC is funded through the National Multiple SclerosisSociety grant RG 4869-A-1 to the Critical Path Institutesupplemented by annual dues from industry sponsors

DisclosureM Goldman reports grant support from Biogen MedDayand Novartis Pharmaceuticals and consulting fees from Ada-mas EMD Serono Novartis Pharmaceuticals Sanofi Gen-zyme and TEVA Pharmaceuticals N LaRocca is a salariedemployee of the National MS Society R Rudick is a salariedemployee of Biogen L Hudson is a salaried employee of thenonprofit Critical Path Institute P Chin is a salaried em-ployee of Genentech G Francis reports that he was a salariedemployee of Novartis at inception of the project but is now anindependent consultant including for Novartis GenentechRoche EMD Serono and GeNeuro A Jacobs is a salariedemployee of Premier Research which was paid a commercialfee by the Critical Path Institute for analysis of the data de-scribed in this article R Kapoor receives support from the UKNational Institute of Health Research UCLH BiomedicalResearch Centre and fees for consultancies lectures andorsupport for travel to medical meetings from Actelion BiogenGenzyme Novartis Roche and Teva P Matthews has re-ceived honoraria from Biogen Roche Novartis Evelo andIpsen Pharmaceuticals for advisory board participation re-search grants or studentships from Biogen Nodthera Nestleand GE Healthcare and support for educational meetingsfrom Biogen and Novartis E Mowry reports serving as site PIfor clinical trials and studies sponsored by Biogen RocheNovartis Evelo and Ipsen Pharmaceuticals for advisory boardparticipation and SunPharma She has research grants orstudentships support from Sanofi Genzyme and BiogenNodthera Nestle and GE Healthcare and support for edu-cational meetings for investigator-initiated trials and shereceives free medication for a clinical trial of which she is PIfrom Teva Neuroscience She receives royalties for editorialduties from Biogen and Novartis M Panzara was an em-ployee of Sanofi Genzyme during the project G Phillips waspreviously and employee of and is currently a stockholder ofBiogen L Balcer reports research funding from Biogen BUitdehaag reports consultancy fees from Sanofi-GenzymeBiogen Idec Teva Merck Serono and Roche J Cohenreports personal compensation for consulting for AlkermesBiogen Convelo EMD Serono ERT Gossamer Bio MapiNovartis and ProValuate speaking for Mylan and Synthonand serving as an Editor of Multiple Sclerosis Journal Go toNeurologyorgN for full disclosures

Publication historyReceived by Neurology February 19 2019 Accepted in final formJune 24 2019

Appendix Authors

Name Location Role Contribution

Myla DGoldmanMD MSc

University of VirginiaCharlottesville

Author Conceptualized anddesigned the studyinterpreted theresults prepared thefirst draft reviewedand edited themanuscript forintellectual content

Nicholas GLaRoccaPhD

National MultipleSclerosis Society NewYork NY

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

Richard ARudick MD

Biogen CambridgeMA

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

Lynn DHudsonPhD

Critical Path InstituteTucson AZ

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

Peter ChinMD

Genentech SanFrancisco CA

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

Gordon SFrancisMD

At onset of MSOACemployee of Novartiscurrentlyindependentneurology clinicaldevelopmentconsultant SanFrancisco CA

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

AdamJacobs PhD

Premier ResearchQuincy MA

Author Conceptualized anddesigned the studyperformed thestatistical analysesinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

Raj KapoorFRCP

UCL Institute ofNeurology LondonUK

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

Paul MMatthewsMD

Imperial CollegeLondon UK

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

e1930 Neurology | Volume 93 Number 21 | November 19 2019 NeurologyorgN

References1 Whitaker JN McFarland HF Rudge P Reingold SC Outcomes assessment in mul-

tiple sclerosis clinical trials a critical analysis Mult Scler 1995137ndash472 Rudick R Antel J Confavreux C et al Clinical outcomes assessment in multiple

sclerosis Ann Neurol 199640469ndash479

3 Rudick R Antel J Confavreux C et al Recommendations from the National MultipleSclerosis Society clinical outcomes assessment task force Ann Neurol 199742379ndash382

4 Schwid SR Goodman AD McDermott MP Bever CF Cook SD Quantitativefunctional measures inMS what is a reliable change Neurology 2002581294ndash1296

5 Kragt J Van der Linden F Nielsen J Uitdehagg B Polman C Clinical impact of 20worsening on Timed 25-Foot Walk and 9-Hole Peg Test in multiple sclerosis MultScler 200612594ndash598

6 van Winsen L Kragt J Hoogervorst E Polman C Uitdehaag B Outcome measure-ment in multiple sclerosis detection of clinically relevant improvement Mult Scler201016604ndash610

7 Bosma L Kragt J Brieva L et al Progression on the Multiple Sclerosis FunctionalComposite in multiple sclerosis what is the optimal cut-off for the three componentsMult Scler 201016862ndash867

8 Balcer LJ Miller DH Reingold SC Cohen JA Vision and vision-related outcomemeasures in multiple sclerosis Brain 201513811ndash27

9 Balcer LJ Raynowska J Nolan R et al Validity of low-contrast letter acuity as a visualperformance outcome measure for multiple sclerosis Mult Scler J 201723734ndash747

10 Benedict RHB DeLuca J Phillips G et al Validity of the Symbol Digit ModalitiesTest as a cognition performance outcome measure for multiple sclerosis Mult Scler J201723721ndash733

11 Ontaneda D LaRocca N Coetzee T Rudick RA Revisiting the multiple sclerosisfunctional composite proceedings from the National Multiple Sclerosis Society(NMSS) task force on clinical disability outcomes Mult Scler J 2012181074ndash1080

12 Rudick RA LaRocca N Hudson LD MSOAC Multiple Sclerosis Outcome Assess-ments Consortium genesis and initial project plan Mult Scler J 20142012ndash17

13 LaRocca NG Hudson LD Rudick R et al The MSOAC approach to developingperformance outcomes to measure and monitor multiple sclerosis disability MultScler J 2018241469ndash1484

14 Multiple Sclerosis Outcome Assessments Consortium and the CFAST MultipleSclerosis Development Team Therapeutic area data standards user guide for multiplesclerosis version 10 [online] Available at cdiscorgtherapeuticMS AccessedFebruary 22 2019

15 Motl RW Cohen JA Benedict R et al Validity of the timed 25-foot walk as anambulatory performance outcome measure for multiple sclerosis Mult Scler J 201723704ndash710

16 Feys P Lamers I Francis G et al The Nine-Hole Peg Test as a manual dexterityperformance measure for multiple sclerosis Mult Scler J 201723711ndash720

17 Strober L DeLuca J Benedict RHB et al Symbol Digit Modalities Test a validclinical trial endpoint for measuring cognition in multiple sclerosis Mult Scler J 2018Epub ahead of print

18 Kurtzke JF Rating neurologic impairment in multiple sclerosis an expanded disabilitystatus scale (EDSS) Neurology 1983331444ndash1452

19 Ware JE Snow KK Kosinski M Gandek B SF-36 Health Survey Manual and In-terpretation Guide Boston The Health Institute New England Medical Center1993

20 Beck AT Ward CH Mendelson M Mock J Erbaugh J An inventory for measuringdepression Arch Gen Psychiatry 19614561ndash571

21 Cohen JA Reingold SC Polman CH Wolinsky JS International Advisory Com-mittee on Clinical Trials in Multiple Sclerosis Disability outcome measures in mul-tiple sclerosis trials current status and future prospects Lancet Neurol 201211467ndash476

22 Norman GR Sloan JA Wyrwich KW Interpretation of changes in health-relatedquality of life the remarkable universality of half a standard deviation Med Care 200341582ndash592

23 Fischer JS Jak AJ Knicker JE Rudick RA Cutter G Multiple Sclerosis FunctionalComposite (MSFC) Administration and Scoring Manual Unitech National Multi-ple Sclerosis Society 2001

24 Winer BJ Statistical Principles in Experimental Design 2nd ed New York McGraw-Hill 1971

25 Critical Path Institute Available at c-pathorg Accessed February 19 2019

Appendix (continued)

Name Location Role Contribution

Ellen MMowryMD MCR

Johns HopkinsBaltimore MD

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

MichaelPanzaraMD

Wave Life SciencesCambridge MA

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

GlennPhillipsPhD

Biogen CambridgeMA

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

Laura JBalcer MDMSCE

New York UniversityNY

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

BernardMJUitdehaagMD

VU University MedicalCenter Amsterdamthe Netherlands

Author Conceptualized anddesigned the studyinterpreted theresults reviewed andedited the manuscriptfor intellectualcontent

Jeffrey ACohen MD

Cleveland Clinic OH Author Conceptualized anddesigned the studyinterpreted theresults prepared thefirst draft reviewedand edited themanuscript forintellectual content

NeurologyorgN Neurology | Volume 93 Number 21 | November 19 2019 e1931

DOI 101212WNL0000000000008519201993e1921-e1931 Published Online before print October 22 2019Neurology

Myla D Goldman Nicholas G LaRocca Richard A Rudick et al data

Evaluation of multiple sclerosis disability outcome measures using pooled clinical trial

This information is current as of October 22 2019

ServicesUpdated Information amp

httpnneurologyorgcontent9321e1921fullincluding high resolution figures can be found at

References httpnneurologyorgcontent9321e1921fullref-list-1

This article cites 19 articles 2 of which you can access for free at

Citations httpnneurologyorgcontent9321e1921fullotherarticles

This article has been cited by 1 HighWire-hosted articles

Subspecialty Collections

httpnneurologyorgcgicollectionmultiple_sclerosisMultiple sclerosis

_design_httpnneurologyorgcgicollectionclinical_trials_methodology_studyClinical trials Methodologystudy designfollowing collection(s) This article along with others on similar topics appears in the

Permissions amp Licensing

httpwwwneurologyorgaboutabout_the_journalpermissionsits entirety can be found online atInformation about reproducing this article in parts (figurestables) or in

Reprints

httpnneurologyorgsubscribersadvertiseInformation about ordering reprints can be found online

ISSN 0028-3878 Online ISSN 1526-632XWolters Kluwer Health Inc on behalf of the American Academy of Neurology All rights reserved Print1951 it is now a weekly with 48 issues per year Copyright Copyright copy 2019 The Author(s) Published by

reg is the official journal of the American Academy of Neurology Published continuously sinceNeurology