13 tiv4 Q1

5

6

7 Q2

8 , Bar9 a

1011

1 3

1415 Keywords:16 Machine learning17 Algorithms18 Statistics19

2 021Cognitive rehabilitation (CR) treatment consists of hierarchically organized tasks that require repetitive22use of impaired cognitive functions in a progressively more demanding sequence. Active monitoring of2324252627282930313233343536373839

40

41

42

43

44

45

46

47

48

49

50

51

52 Despite new techniques for early intervention and intensive53 ABI, both of which increase the survival rate, there is still no surgi-54 cal or pharmacological treatment for the re-establishment of lost55 functions following brain injury. Cognitive rehabilitation (CR) is

56

57

58

59

60

61

62

63ning schedule re-64t stimulation and65uition, and66(Green & B672005). It is difcult to identify this maximum effective l68stimulation and therapists use their expertise in daily p69without precise guidelines on these issues.70In this work, the NeuroRehabilitation Range (NRR) is introduced71as the conceptual framework to describe the degree of perfor-72mance of a CR task that produces maximum rehabilitation effects.73A data mining approach is used to induce an operational model for74the NRR of CR tasks. The aim is to help create useful guidelines for75CR therapists that can help them select the most appropriate tasks

Corresponding author. Tel.: +34 93 401 73 23; fax: +34 93 401 58 55.E-mail addresses: agarciar@guttmann.com (A. Garca-Rudolph), karina.gibert@

upc.edu (K. Gibert).1 Tel.: +34 93 497 77 00; fax: +34 93 497 77 07.

Expert Systems with Applications xxx (2014) xxxxxx

Contents lists availab

Expert Systems w

.e

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1of all trauma deaths (Walsh, Donal, Stephen, & Muldoon, 2012).In Europe, brain injuries from trauma are responsible for moreyears of disability than any other cause (Maas, Stocchetti, &Bullock, 2008).

the task itself. Thus, determining the correct traiquires a quite precise trade-off between sufciensufciently achievable tasks, which is far from intan open issue, both empirically and theoreticallyhttp://dx.doi.org/10.1016/j.eswa.2014.03.0010957-4174/ 2014 Elsevier Ltd. All rights reserved.

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data mining approach to identify cognitive NeuroRehabilitation Range in TraBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.org/10.1016/j.eswa.2014.03.001is stillavelier,evel ofractice,1. Introduction

Acquired Brain Injury (ABI) of either vascular or traumatic nat-ure is one of the most important causes of neurological disabilities.According to the World Health Organization, Traumatic Brain In-jury (TBI) is the leading cause of death and disability in childrenand young adults around the world and is a factor in nearly half

currently considered the therapeutic process for re-establishingfunctioning in everyday life (Pascual-Leone & et al., 2005). A typicalCR programmainly provides exercises which require repetitive useof the impaired cognitive system in a progressively more demand-ing (Sohlberg, 2001) sequence of tasks. The rehabilitating impact ofa task or exercise depends on the ratio between the skills of thetreated patient and the challenges involved in the execution ofthe progress of the subjects is therefore required, and the difculty of the tasks must be progressivelyincreased, always pushing the subjects to reach a goal just beyond what they can attain. There is animportant lack of well-established criteria by which to identify the right tasks to propose to the patient.In this paper, the NeuroRehabilitation Range (NRR) is introduced as a means of identifying formaloperational models. These are to provide the therapist with dynamic decision support information forassigning the most appropriate CR plan to each patient. Data mining techniques are used to builddata-driven models for NRR. The Sectorized and Annotated Plane (SAP) is proposed as a visual tool bywhich to identify NRR, and two data-driven methods to build the SAP are introduced and compared.Application to a specic representative cognitive task is presented. The results obtained suggest thatthe current clinical hypothesis about NRR might be reconsidered. Prior knowledge in the area is takeninto account to introduce the number of task executions and task performance into NRR models and anew model is proposed which outperforms the current clinical hypothesis. The NRR is introduced as akey concept to provide an operational model identifying when a patient is experiencing activities inhis or her Zone of Proximal Development and, consequently, experiencing maximum improvement.For the rst time, data collected through a CR platform has been used to nd a model for the NRR.

2014 Elsevier Ltd. All rights reserved.A data mining approach to identify cogniin Traumatic Brain Injury patients

Alejandro Garca-Rudolph a,1, Karina Gibert b,a Institut Guttmann, Hospital de Neurorehabilitaci, Cami Can Ruti s/n, 08916 BadalonabDepartament dEstadstica i Investigaci Operativa, Universitat Politcnica de Cataluny

a r t i c l e i n f o a b s t r a c t

journal homepage: wwwe NeuroRehabilitation Range

celona, SpainBarcelonaTech, Jordi Girona 1-3, 08034 Barcelona, Spain

le at ScienceDirect

ith Applications

lsevier .com/locate /eswaumatic

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106 CR, as part of neuropsychological rehabilitation, tries to im-107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178Q3

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

tem

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1prove the decits caused by ABI in daily living activities (Bernabeu& Roig, 1999) by retraining attention, memory, reasoning/problemsolving, and executive functions. The plasticity of the central ner-vous system plays a central role (Pascual-Leone et al., 2005) inCR, based on therapeutic plans to stimulate that non-damagedneurons can modify their structure by learning from experiencethe damaged functions, through repetition (Luria, 1978). Plasticitymay represent a surrogate marker of functional recovery, indicat-ing behavioral change that is resistant to decay. In Kleim and Jones(2008) is suggested that a sufcient level of rehabilitation is likelyto be required in order to get the subject over the hump i.e. repeti-tion may be needed to obtain a sufcient level of improvement andbrain reorganization for the patient to continue using the affectedfunction outside of therapy and to achieve and maintain furtherfunctional gains. A great deal of research indicates that behavioralexperience can enhance behavioral performance and optimizerestorative brain plasticity after brain damage. Simply engaging aneural circuit in task performance is not sufcient to drive plastic-ity. Repetition of a newly learned (or relearned) behavior may berequired to induce lasting neural changes. In fact, from the expertspoint of view, there is a clear perception that the effectiveness ofthe task also depends on the replication, as Luria also asserts.

A typical CR program mainly provides exercises that requirerepetitive use of the impaired cognitive system in a progressivelymore demanding (Sohlberg, 2001) sequence of tasks. Each task tar-gets a principal cognitive function and can be performed at differ-ent levels of difculty, according to the response of the patient. Thedesign of a CR program has become an essential issue for patientrecovery.for each single patient at a given moment in their rehabilitationplan, as well as correctly to determine the most appropriate levelof difculty for the proposed task.

The Sectorized and Annotated Plane (SAP) is proposed here as avisual tool to nd both the NRR and operational denitions to beused in real clinical practice.

Two data-driven methods to build the SAP are introduced andcompared. One of them (DT-SAP) is based on a decision tree model,the other (Vis-SAP) on a visualization of available data that pro-motes model induction from a graphical representation. A qualitycriterion to assess NRR models is also introduced, based on the cor-rect prediction ratio provided by the tool.

The performance of NRR model obtained with both DT-SAP andVis-SAP approaches is evaluated and the advantages and draw-backs are analyzed over a real application.

Data comes from the PREVIRNEC platform (Tormos, Garcia-Molina, Garcia Rudolph, & Roig, 2009) which contains rich datamonitoring the CR process on real neurorehabilitation patients.The real performance of a representative cognitive task is analyzedunder both approaches and discussed for a sample of patients fol-lowing a CR treatment at Institut Guttmann (IG) hospital de Neuro-rehabilitaci, Barcelona, Spain.

The structure of the paper is: Section 2 briey presents the stateof the art. Section 3 presents the IG conceptual framework for theresearch of NRR. Section 4 introduces the analysis methodologyand Section 5 its application to a typical cognitive rehabilitationtask in the proposed framework. Section 6 presents a discussionof the results obtained and Section 7 the conclusions and futurelines of research.

2. State of the art

2 A. Garca-Rudolph, K. Gibert / Expert SysAs said before, the rehabilitating effect of a task or exercise de-pends on the ratio between the skills of the treated patient andthe challenges involved in the execution of the task itself. The dif-

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgculty is related to the level of stimulation of cognitively involvedfunctions; maximum activation occurs when the task is just barelytoo difcult (Green & Bavelier, 2005). If the task is too easy for thepatient, or too hard, it appears to be less effective. Active monitor-ing of the subjects progress is therefore required to adapt the dif-culty of the tasks to the potential capacities and progress of thesubject, always pushing them to reach a goal just beyondwhat theycan attain, but not too far. Thus, determining the correct trainingschedule requires a very precise trade-off between sufciently stim-ulating and sufciently achievable tasks, which is far from intuitive,and is still an open problem, both empirically and theoretically.

In the early 1930s, Vygotsky introduced the concept of Zone ofProximal Development (ZPD) (Vygotsky, 1934) in the eld of childlearning, being the distance between the actual capacities of the childby himself and their potential capacities when being guided(Vygotsky, 1978). In 1986, Cicerone and Tupper (1986) transferredZPD ideas to the neurorehabilitation eld by introducing the zoneof rehabilitation potential (ZRP), i.e. the zone in which maximumrecovery of cognitive functions might occur, provided that theproper help is given to the subject. They propose the use of ZPDas a guiding principle in CR. This zone is supposed to reect the pa-tients region of potential restoration thanks to cognitive plasticity(Calero & Navarro, 2007). Current neurorehabilitation practice triesto design therapeutic plans that keep the subject working in thisarea during treatment. However, determining when the patientworks in ZPD or not is still an open issue. Thus in most cases CRtherapists design CR plans from scratch, determining clinical set-tings for specic patients based mainly on their own expertise.Each specic plan evolves according to each therapists own crite-ria and evaluation of the patients follow-up. There is as yet not en-ough in-eld knowledge regarding which specic intervention(task or exercise assignation) is more appropriate to help CR ther-apists design their clinical therapeutic plans.

There is a common belief that CR is effective for TBI patients,based on a large number of studies and extensive clinical experi-ence. Different statistical methodologies and predictive data min-ing methods have been applied to predict clinical outcomes ofTBI rehabilitation (Rughani et al., 2010; Ji, Smith, Huynh,& Najari-an, 2009; Pang et al., 2007; Segal et al., 2006; Brown et al., 2005;Rovlias & Kotsou, 2004; Andrews et al., 2002). Most of these stud-ies focus on determining survival, predicting disability or therecovery of patients, and looking for the factors that better predictthe patients condition after an ABI.

However, current knowledge about the factors that determine afavorable outcome is mainly empirical and the benet of suchinterventions is still controversial (Ecri, 2011; Rohling, Faust,et al., 2009). The development of new tools to evaluate scienticevidence of such effectiveness will contribute to a better under-standing of CR.

Several meta-analyses (Cicerone, Langenbahn, Braden, Malec, &Kalmar, 2011) identify structural limitations to nd scientic evi-dence under classical approaches, related mainly to the existenceof uncontrolled factors and the intrinsic difculty of guaranteeingthe sample heterogeneity. Classical approaches tend to generateevidence about effectiveness by comparing two or more interven-tions in selected and comparable groups. Determining the compa-rable groups relies on identifying the factors that inuencerecovery or chronicity, which should be controlled during thestudy, and these factors are unknown in neurorehabilitation. Itseems that patient improvement might depend inter alia on thelocation of the injuries, cognitive prole, duration, and intensityof proposed treatments and their level of completion (Ciceroneet al., 2011; Norea et al., 2010; Whyte & Hart, 2003).

s with Applications xxx (2014) xxxxxx202However, these seem to be only some of the determining factors203and they cannot by themselves explain the overall phenomenon.204Although these factors are considered in the design of rehabilitation

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295retraining of individuals with special health-related problems296(such as young disabled or elder people) involving the nervous sys-297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

tem

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1treatments, other relevant factors exist that aremuchmore difcultto control, and which are related to the high variability of the le-sions, the complexity of cognitive functions, and the lack of properinstrumentation by which to systematize interventions. This pro-duces intrinsic group heterogeneity and the classical comparativestudies do not performwell (Gibert & Garca-Rudolph, 2007), whichmakes it difcult to advances knowledge on the pathophysiology ofcognitive neurorehabilitation.

In Serra et al. (2013) basic machine learning algorithms wereused to predict the probability of improvement of a patient giventheir initial neuropsychological assessment. This approach,although it is able to identify subpopulations of patients more suit-able for improvement using CR treatments, did not provide anyinformation to help CR therapists adapt CR programs to increasethe improvement itself or to enlarge the subpopulations that mightactivate improvement to CR treatments. Going a little bit further, inMarcano, Chausa, Garcia-Rudolph, Cceres, and Tormos (2013) theperformance obtained by the patient in a certain task has been in-cluded in the model together with the initial assessment. Machinelearning methods signicantly improved predictive capacity. Thiswork provided evidence that task performance is involved in pa-tient improvement; However, it did not provide information onsuccessful patterns of tasks to be proposed to the patients so thatthey improved as much as possible.

For these reasons, other approaches have to be found to betterunderstand the CR process, with the aim of obtaining scientic evi-dence about its effectiveness and providing relevant informationfor the establishment of general guidelines for CR program designthat can assist CR therapists in clinical practice.

Analyzing data from new perspectives can contribute to thiseld (Jagaroo, 2009). Our proposal is trying to approach the prob-lem from a data-driven perspective, by developing new tools thatcan reduce uncertainty in the eld. This paper introduces elementsto assess when a patient is performing a task under a NeuroReha-bilitation Range, as an indicator that maximum improvement ofthe patient might be expected on the targeted cognitive function.This contributes to a better understanding of the role over clinicalimprovement of a particular degree of performance of a CR task. Infact, the NRR helps provide an operational denition for the zone ofmaximum rehabilitation potential and represents an operational-ization of the ZPD.

Thework is based on the experience of Institut Guttmann Neu-rorehabilitation Hospital (IG) regarding the introduction of Infor-mation and Communication Technologies (ICTs) in CR. Data usedin this work comes from a CR computerized platform conceivedby IG and developed in collaborationwith clinical and technologicalpartners, namely PREVIRNEC a serious game platform for CR (Tor-mos et al., 2009). PREVIRNEC is specically designed to managethe CR plans assigned to subjects, as well as obtaining precise fol-low-up information about the process. At the time of submission(December 2013) PREVIRNEC has already been integrated intothe clinical practices of more than 24 clinical centers. As the wholebehavior of the patients working in PREVIRNEC is registered in acentral server, this provides a unique database of on-eld interven-tions, with detailed information that is potentially valuable for abetter understanding of the circumstances under which CR therapywould be most benecial to individual patients. The PREVIRNECdatabase permits knowledge extraction from data, and provides aframework in which new knowledge can be introduced into thesystem to be veried and continuously rened, also contributingto elaborate data-driven personalized treatments.

2.1. Serious games in cognitive rehabilitation

A. Garca-Rudolph, K. Gibert / Expert SysAccording to Lurias theory outlined above, repeated taxing of thesame neurological system facilitates and guides the reorganization

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgtem were also highlighted. An improvement in the spatial resolu-tion of attention in videogame players has been observed (Green& Bavelier, 2007).

A persistent difculty is that training can be more or less ef-cient depending on how it is administered and this is directly re-lated with tasks difculty management (Linkenhoker & Knudsen,2002).

2.2. Flow in computer-mediated environments

Learning is enhanced when the match between the skills of thelearner and the challenges of the subject matter are optimized(Whalen, 1998). Csikszentmihalyis Flow Theory (Csikszentmihalyi,1991) provides a framework and vocabulary for understanding theexperiential nexus between the active person and the facilitativeenvironment. The experience of Flow creates information thatmelds actor and activity into one transactive system. In this sense,Flow may be seen as the experiential dimension of the ZPD (Wha-len, 1998).

Flow or optimal experiences, also referred to as the zone(Csikszentmihalyi, 1991) represents a state of consciousness wherea person is so absorbed in an activity that he or she excels in per-formance without consciously being aware of his or her everymovement.

Within a Computer Mediated Environment (CME), theexperience of ow in the past 20 years has demonstrated an in-crease in communication, ofce productivity software on desktopcomputers, learning, general web activity, online consumer set-tings, and online search experiences among others (Finneran &Zhang, 2005).

The practical implications of the consequences of ow experi-ences are clear, important, and promising. It is expected that agood understanding of the ow phenomenon would guide ICTsdesigners to build products that lead users to ow experiences. Lit-of the targeted cognitive function. This approach requires imple-mentationof repetitive exerciseswithin theplannedprogramwhichrequire patients to use their impaired cognitive skills at a productivelevel. In this work therefore, given the initial assessment of thepatient impairments, the therapists assign repetitive tasks to be exe-cuted in PREVIRNEC, a serious game platform for cognitiverehabilitation.

The emergence of serious games broadens the discipline ofentertainment-education in numerous dimensions. Serious gameshave recently been applied in diverse areas e.g. military training,health, higher education, city planning (Rego, Moreira, & Reis,2010). Prior research demonstrates that videogame attributes, suchas task difculty, realism, and interactivity, affect learning out-comes in game-based learning environments (Orvis, Horn, & Bela-nich, 2008). These prior works suggest that in order to be mosteffective, instructional games should present an optimal level ofdifculty to learners. This optimal range of difculty is alignedwith the Vygotskys concept of ZPD, where training should be dif-cult to the learner, but not beyond his or her capabilities.

Videogames involving the sensory-motor system and problem-solving skills are serious candidates for neuro-rehabilitation andmotor or cognitive training. In Green and Bavelier (2007) severalimprovements in gaming activity were identied, from reactiontimes to spatial skills. The opportunities for using this kind of med-ia to improve cognitive functions in individuals with particularneeds (as reviewed for surgeons and soldiers) or for training and

s with Applications xxx (2014) xxxxxx 3329tle research is available concerning the application of data mining330techniques in Flow. In Mathwick and RigdonPlay (2004) cluster331analysis is used to identify a ow cluster comprised of individu-

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351 tempt to re-conceptualization, in Finneran and Zhang (2005) a con-352 ceptual model for ow antecedents is proposed: the Person-353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371 range of drug concentrations within which the probability of the372 desired clinical response is relatively high and the probability of373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397Q4

398

399

400

401

402

403405405

406407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

tem

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1unacceptable toxicity is relatively low. Within this therapeuticrange the desired effects of the drug are observed. Below it thereis a greater probability that the therapeutic benets are not real-ized (non-response or treatment-resistance); above it, toxic effectsmay occur (DiPiro & Spruill, 2010).

In this work, the concept of NeuroRehabilitation Range (NRR) isintroduced as a translation of the classical therapeutic range frompharmacology to the eld of neurorehabilitation. The role of phar-macs in disease treatment is assumed in neurology by the role ofneurorehabilitation tasks. The effect of treatment corresponds hereto the restoration of cognitive functions.

Using this analogy, we will consider that a cognitive rehabilita-tion treatment task behaves in NRR if the desired clinical responseis obtained i.e. if an observable improvement in the targeted cogni-tive function is registered for the patient. As nding therapeuticArtifact-Task (PAT) model.PAT removes the ambiguities among the ow antecedents by

considering the task and the artifact as separate entities whenlooking at the factors that lead to a ow state.

The PAT model considers each of the three main components ofperson, artifact, and task independently and their interactions, tounderstand the holistic picture of ow antecedents.

The intention is therefore to conceptualize the major compo-nents of a person working on a computer-related activity thatcan inuence the ow experience the person may have. Individualdifferences, which are shown to be important in early non-com-puter-mediated ow studies, are probably even more importantin CME. According to Finneran and Zhang (2005) much empiricalresearch is needed to validate or clarify which individual factorsinuence the ow experience and where they occur in the process.

3. Conceptual framework

3.1. NeuroRehabilitation Range

In Clinical Pharmacokinetics, therapeutic range is dened as aals with high Internet search skills and a search task that presents ahigh navigational challenge.

From a research perspective however, ow is poorly dened inCME because of the numerous ways it is conceptualized, opera-tionalized, and measured. Flow experience is associated with a per-son doing an activity. In traditional ow studies, the activities tendto be very clear: playing music, climbing a cliff, playing chess orreading a book. Most existing ow studies in CME do not clearlydifferentiate between factors that are related to the task and thosethat are related to the artifact.

Thus, there is a need to re-conceptualize ow in CME to con-sider the uniqueness of the artifacts and the complexity they addto the ow phenomenon. Indeed one of the aims of this work isto use PREVIRNEC to produce ow experiences in the subject,thus incrementing the benets of the neurorehabilitation process.

2.3. PAT model

As introduced in the previous section, there is a need to re-con-ceptualize ow in CME to consider the uniqueness of the artifactsand the complexity they add to the ow phenomenon. As an at-

4 A. Garca-Rudolph, K. Gibert / Expert Sysrange in pharmacokinetics consists of determining the proper drugconcentration to be administered to a patient, nding NRR of a cog-nitive rehabilitation task is dened as determining the proper level

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.org3.2. PAT model

In this section we identify the elements of the PAT model usedin our application.

3.2.1. PersonOne hundred and twenty-three Traumatic Brain Injury (TBI) pa-

tients with moderate to severe cognitive affectation (according toGlasgow Coma Scale) and who underwent rehabilitation at IG wereincluded in this analysis. All subjects gave informed consent to theneuropsychological procedure, which was approved by IGs EthicalCommittee. The patient (mean age 36.56 6.5, range 1868 years;91 male and 32 female) diagnosis was made according to theclinical protocols of the IG Neuropsychological department. Allpatientsmet criteria to initiate IG neuropsychological rehabilitationtreatment. It includes a Neuropsychological Assessment Battery(NAB), 28 items covering the major cognitive domains (language,attention, memory and learning, and executive functions)measured using standardized cognitive tests.

After NAB initial evaluation all patients started a CR programlasting four to six months based on personalized interventions,where patients worked in each one of the specic cognitivedomains, considering the degree of the decit and the residualfunctional capacity. All patients were administered the same NABneuropsychological assessment at the end of the rehabilitationprogram. All NAB items are normalized to a 04 scale (where0 = no affectation, 1 = mild affectation, 2 = moderate affectation,3 = severe affectation and 4 = acute affectation). Differencesbetween pre- and post-treatment NAB test scores were used tomeasure particular patient improvement in the elds of attention,Currently, some hypotheses are being tested for the values ofr, r+. The aim of this work is to use data-driven models and PRE-VIRNEC database to extract useful knowledge to determine aproper model for NRR(T).of task difculty to be proposed to the patient to obtain an optimalcognitive improvement of the targeted cognitive function.

We presume that being able to determine the NRR will providea model that can help us to know better the relevant factors deter-mining the ZRP proposed by Cicerone and Tupper (1986).

In PREVIRNEC, following the execution of a given task T thesubject gets a result RT ranging from 0 to 100. Section 3.3.3 detailshow this result is obtained, in this section we merely remark that a0 result denotes the lowest level of task completion and a 100 thehighest. Being the NRR of task T dened as NRR(T) = [r,r+], andbeing r, r+ in [0,100], using a simple test it is easy to determinewhether or not the patient performed the task in NRR:

in NRR RT iff RT 2 NRRT r 6 RT 6 r Tasks that are too easy will produce results higher than r+ andare probably out of ZRP because they only involve undamagedbrain areas and do not demand impaired cognitive functionsto be activated. In this case, we say the task has been executedin SupraNeuroRehabilitation Range (SNRR).

Tasks that are too difcult will produce results lower thanr and are also likely to be out of ZRP. This is because theyintensively required the implication of the impaired brain areasthat cannot react to the excessively difcult cognitive stimulus.In this case we talk about InfraNeuroRehabilitationRange(INRR).

s with Applications xxx (2014) xxxxxx451memory, and executive functions. Improvement criteria in the452respective cognitive functions are dened in IG cognitive rehabili-453tation protocols.

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485 Q5

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537For each person or entity using the system in a determined con-538

539

540

541nitive functions affected, as a consequence of suffering ABI. The542

543

544

545rehabilitation in patients with ABI, who will have a number of as-546

547

548

549each center, both patients and therapists and their assignments,550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

tem

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q13.2.2. ArtifactPREVIRNEC (Tormos et al., 2009) comprises a series of rehabil-

itation tasks for training different cognitive functions: attention,memory, executive functions, and language. For each specic taskthat the patient executes, the result of the task (0100 real num-ber) is assigned into one of the following three ranges: NRR, INRR,and SNRR. As a rst hypothesis, PREVIRNEC is currently assumingthat NRR(T) = [65, 85] These limits have been dened according tothe expertise of the CR therapists of making the task difcult en-ough not to be innocuous, but not so difcult as to be blocking.

To maintain the interest of the patient throughout the execu-tion of tasks, PREVIRNEC includes an algorithm that proposestasks to patients while trying to keep them inside NRR limits. PRE-VIRNEC is a cognitive tele-rehabilitation platform, developedover an architecture based on web 2.0 technologies. It is conceivedas a tool for the enhancement of cognitive rehabilitation, thestrengthening of the relationship between the neuropsychologistand the patient, the personalization of treatment, the monitoringof results, and the performance of tasks. The platform architectureconsists of four main modules that group related functionalitiesvertically, sharing the user interface that is personalized dependingon the users role. This interface is also multi-language, with Cata-lan, Spanish and English already implemented, but being open tosupport any other language. The system also has a help module,which guides the user in order to complete each action. Securityaspects are transversal and have to be taken into account in everymodule to keep information and all connections safe, due to thecondentiality concerns of medical applications. The security mod-ule is responsible for controlling every access, including the onesrelated to the patients Electronic Health Record (EHR). The fourmodules are briey described below:

3.2.2.1. Information management. This module groups functional-ities related to the generation and edition of information that de-pends on the patients EHR, as well as the tests used todetermine the grade of affection of each cognitive function. Thesetests are used to dene the affection prole of the patient. In addi-tion, this module controls the assignation of therapies to the pa-tients, determining which computerized tasks a patient has to doon a certain day. The results of the execution of these tasks are reg-istered in the system, and can then be used by the clinicians to seethe evolution of the therapy, as well as showing graphics and re-ports related to the completion of the sessions, tasks that havebeen used, both global and individual results, and much more.

3.2.2.2. Monitoring. To comply with data protection laws, every ac-tion carried out by a user is stored in both the database and also ina log le, so that the administrator can track every action related toa patient and their data. The system also offers a module for mon-itoring the execution of the tasks, so the therapist can then repro-duce a task as it was done by the patient. This allows the therapistto see exactly what a patient did in the monitored task. This is veryuseful because sometimes merely seeing the numeric results is notenough.

3.2.2.3. Administration. This module, although it is the one withfewer functionalities and users, includes very important function-alities such as the users management and their proles, as wellas system monitoring (using logs).

3.2.2.4. Communication. The main element of this module is the vi-deo conference that allows users to communicate using video,audio, and chat. Using the videoconference therapists can hold

A. Garca-Rudolph, K. Gibert / Expert Systele-appointments with patients or other therapists, removingthe distance barriers between users, and helping the patients tofeel closer to the clinical team. In addition to the videoconference,

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgapart from other management and control functions applied tothe supervising center/s.

3.2.2.8. Administrator. Apart from all the typical administrationtasks for every informatics system, the administrator will be theperson responsible for managing the categories, functions andtasks dened in the system, which is the content the therapist willuse to schedule therapy sessions for their patients.

The content used in the tele-rehabilitation sessions consists ofcomputerized tasks, grouped by cognitive functions and categories.The neuropsychologist creates a tele-rehabilitation session byassigning a set of tasks to a certain day. He or she is able to cong-ure the difculty of each task because they all have a set of inputparameters.

PREVIRNEC comprises a series of rehabilitation tasks for train-ing different cognitive functions: attention, memory, executivefunctions and language. For each specic task that the patient exe-cutes, the result of the task (0100 real number) is assigned to oneof the following three ranges: the NRR, INRR, and SNRR. In PREVIR-NEC the initial hypothesis is that the patient has completed a taskwithin NRR if they obtained result is higher than 65 and lower than85, in INRR if it is less than 65 and in SNRR if it is higher than 85.PREVIRNEC dynamically adjusts each task level of difcultyaccording to the above dened ranges, aiming to scaffold them insigned patients and be responsible for their treatment, schedulingand the monitoring of personalized and individualized therapies.

3.2.2.7. Supervisor. Person in charge of the user management forcaregiver role appears here, considered a secondary actor that willhelp the patient use the system when necessary.

3.2.2.6. Therapist. A neuropsychologist who specializes in cognitivetext and for a specic goal, four different user proles have beendened:

3.2.2.5. Patients. Man or woman of any age with one or some cog-this module has a mailing service for the exchange of internalasynchronous messages and an alert service that lets users knowwhat tasks they have to accomplish.

The platform is based on open source web 2.0 technologies. Themain architecture of the platform is based on a clientserver com-munication using HTTP and XML-RPC, as it is shown in Fig. 1.

A Model-View-Controller pattern was followed during thedevelopment phase. As a result, the view and the logic to accessand process data are separated.

The new web application requires Java (jdk 1.6, jre 6.x) and itruns over Apache Tomcat 6.X, as it is based on Servlet/JSP. Thedatabase used is MySQL Server 5.X and MySQL Java Connector5.X (JDBC).

With regard to the programming languages used, all the envi-ronment is Java 2 Platform (J2EE, Enterprise Edition), using Java-Script and AJAX (SACK library) to dynamically change the datashowed on the HTML pages, thereby avoiding the need to reloadthe page every time the user wants to show or edit content.

For the videoconference module, OpenMeetings has been used.This implements the Real Time Multimedia Protocol (RTMP) usinga red5 server for audio and video streaming.

s with Applications xxx (2014) xxxxxx 5573NRR. Therefore if the actual task is performed in INRR its difculty574level is automatically lowered, and if it is performed in SNRR it is575automatically raised.

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629631631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

6464.1. Sectorized and Annotated Plane (SAP)

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662lows identication of logical restrictions (rules) determining the663

664

erv

tem

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q13.2.3. TasksAs introduced in Section 3.2.1, following NAB initial evaluation

all analyzed subjects initiated a CR program lasting four to sixmonths based on personalized interventions in PREVIRNECplatform.

The therapeutic content used in PREVIRNEC tele-rehabilita-tion sessions consists of computerized tasks, grouped by cognitivefunctions. The neuropsychologist creates a tele-rehabilitation ses-sion by assigning a set of tasks to a certain session day. He orshe is able to congure the difculty of each task because theyall have a set of input parameters.

At the time of this analysis, PREVIRNEC includes one hundredand fteen rehabilitations tasks. Each task is dened by a series ofparameters that determine its level of difculty. The therapist se-lects for each task the parameter to be used for the automaticadjustment of the difculty level described above. This dynamicadjustment of the difculty level is performed twice for each taskas necessary. This means that if the patient does not obtain a taskresult in NRR in the rst execution, PREVIRNEC automaticallygenerates the task with the adjusted difculty level once; if againthe obtained result is not in TR, PREVIRNEC likewise generatesa second version of the task.

3.2.3.1. Visual memory task description. For illustrative purposesone such task designed for visual memory treatment is describedbelow in more detail. This task (identied as idTask = 151) hasbeen one of the most extensively administrated by neuropsychol-ogists and executed by participants during the analyzed period(described in Section 3.2.1.) and will be used throughout the differ-ent sections of this paper.

The objective of the task is to recall the position of pairs of iden-tical images in a grid. A grid of xed size (e.g. 5 5 dark coloredcells) is presented to the participant at the start. When the partic-ipant left-clicks on a cell in the grid, an image of an object on awhite background appears in the cell. This image remains until asecond cell is clicked, then both images are shown for a period oftime (e.g. 1500 ms) for the participant to remember them; after-wards both images are covered. Only two cells can be simulta-neously discovered in one go. When two identical images are

Fig. 1. PREVIRNEC client/s

6 A. Garca-Rudolph, K. Gibert / Expert Sysdiscovered, both of them remain visible in their cells. The aim ofthe task is to discover all the images in the grid with the minimumnumber of clicks. The parameters that determine the different dif-culty levels are shown in Table 1.

The quantied result parameters for the evaluation of task com-pletion are: the total execution time, the total number of discover-ing clicks, the total number of wrong clicks (this number increasesif the participant clicks on an image already discovered before,meaning that errors are computed after an initial explorationphase), the total number of correct clicks (in this case, althoughit is computed for homogeneity with other tasks, the number ofclicks for all participants is constant because the task is consideredunnished until all the images are discovered; this also means thattask151 does not produce omissions, and they are presumed to bezero). The task result is computed as:

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgdifferent outcomes of treatment.The SAP is built following two methodologies that implementGiven three variables Y, X1, X2, where Y is a qualitative responsevariable, with values {y1,y2, . . .}, and X1, X2 numerical explanatoryvariables, the SAP is a 2-dimensional plot with X1 in the x axis, X2in the y axis and rectangular regions with constant Y displayed andlabeled with Y values as outlined in Fig. 2. An SAP is therefore agraphical support tool aimed at visualization, where the responsevariable is constant in certain regions of the X1 X2 space. Eventu-ally, allowing a relaxation of strict constant Y in the marked re-gions, the SAP might include an indicator of region purity, addingthe probability of occurrence of the labeling value.

Given a particular CR task, and assuming Y as a binary variablereporting improvement of the patient in the cognitive function tar-geted by the task (yes, no), the SAP leads to response zones whereparticipants show similar response to treatment. The SAP shows aplane sectorization directly related to treatment response. This al-Task result correct=correctwrong omissions 100

4. Methods

As a rst attempt to nd the NeuroRehabilitation Range of acognitive task executed by means of PREVIRNEC, the result ob-tained by the patient in the execution of the task is used. The num-ber of executions of the task performed by the patient isconsidered, as it is known that repetition is highly related to acti-vation of brain plasticity, which is in the core of cognitive functionsre-establishment, as discussed in the State of The Art Section.

The proposed methodology presents two strategies for the ana-lytical and graphical identication/visualization of neurorehabili-tation and non-NeuroRehabilitation Ranges based on the notionof Sectorized and Annotated Plane introduced below. The twomodels for NRR obtained are compared and discussed for the spe-cic case of task151, related to visual memory cognitive function.

er communication schema.s with Applications xxx (2014) xxxxxx665two different strategies, the rst one based on graphical visualiza-666tion and the second based on the plane partitions induced by deci-667sion trees, as introduced below.

6684.1.1. Visualization-based SAP (Vis-SAP)669Data is plotted regarding X1 and X2, and each point is marked670with different colors according to the values of Y. This categorized671scatterplot (sometimes known as letterplot) is an exploratory tech-672nique for investigating relationships between X1 and X2 within the673sub-groups determined by Y. For the particular application pre-674sented here, X1 is the number of executions of the task performed675by the subject (Execs151), X2 is the result obtained at every single676execution (Results), while Y is the effect of the neurorehabilitation677process (improvement/non-improvement).

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

678 This exploratory analysis is used to identify systematic relation-679 ships between variables when there is no previous knowledge680 about the nature of those relationships. The constant-Y regions de-681 tected in the plot can be expressed in the form of logical rules682 involving the implied variables. The SAP is built on the basis of683 these rules.

684 4.1.2. Decision tree-based SAP685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734735

736

737738740740

741743743

744745

747747

748

749750

752752

753

755755

756

758758

759

760

761

Table 1Parameters that determine idTask = 151 level of difculty.

Number ofcells

Stimulus type Proximity ofthesecond image

Presentation time(ms)

4 4 Abstract objects 2 Cells 15005 5 Numbers 3 Cells 30006 6 Animals 4 Cells 40008 8 Colors Random

A. Garca-Rudolph, K. Gibert / Expert System

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1The tuple (X1, X2, Y) = (Execs151, Results, Improvement) can betreated as a classical classication problem, where Improvementhas to be recognized on the basis of Execs151 and Results. Thetraining dataset is used to induce a decision tree classier whichis later evaluated with the test set in the usual way. For the testing,the class label is ignored and predicted by the classier. Perfor-mance of classier is evaluated by comparing both predicted andreal class. The confusion matrix and taxes of misclassication canbe provided.

Here the Weka (Hall, Frank, Holmes, Pfahringer Reutemann, &Witten, 2009) software has been used to apply the J48 (Witten &Frank, 2005) decision tree algorithm which implements QuinlansC4.5 algorithm (Quinlan, 1993) building an unpruned tree.

Once the decision tree has been constructed, it is converted intoan equivalent set of rules in the usual way.

4.2. Quality indicators

4.2.1. Sector condenceGiven a sector S from the SAP graph, labeled with Y = y, the sec-

tor condence corresponds to the empirical probability of occur-rence of event y inside the sector. P(Y = y|S) is computed as theratio between the number of positive cases and the sector size. Thisis in one sense a measurement of the purity of the sector andprovides the quality of the assignment of class y to all elementsin the sector. The higher the condences of the SAP sectors, thebetter the model is considered.762

763

764

765

766

767

768

Fig. 2. General Sectorized Annotated Plane (SAP) description.

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgWhen Y is a binary variable P(Y = yes | S) it provides the sensi-tivity of S, while P(Y = No | S) provides the specicity. As usual,the higher the sensibility and specicity, the higher the quality of S.

We dene the global quality of the SAP as the pooled condenceof all sectors. Additionally for the SAP of binary variables a pooledspecicity and a pooled sensitivity can be used as qualityindicators.

4.2.2. Hypothesis testingThus, the range of results determining those sectors where S is

labeled as Y = yes (Improvement) determine the NRR of the task.The sensitivity of the NRR is related to the fact that patients in

NRR improve, or in a more relaxed formulation, that there is a highproportion of patient improvement within NRR.

The specicity is related to the fact that patients out of NRR donot improve. This can be measured by the high proportion of non-improving patients in INRR or SNRR or equivalently by the low pro-portion of improving patients in INRR or SNRR.

A classic 2-sample probability test is used to see whether theresponse to the CR therapy is signicantly different for those exe-cuting tasks in NRR than those obtaining results out of NRR. It isexpected that the probability of improvement is signicantly high-er for those in NRR. SAP models that provide sectors without signif-icant differences should be disregarded, as they provide NRR withpoor identication of the improving population. Thus, being

pMR = Probability of improving being in NRRpMR = Probability of improving being out of NRR

the hypothesis tested is

H0 : pMR pMR

H1 : pMR > pMR

The statistics used are

e pMR pMRp01 p0 1nMR 1nMNoR

r H0z

where p0 is the weighted common estimator of pMR and pMR underthe Ho,

p0 nMRpMR nMRpMR

nMR nMR

pMR p^MR number of patients in NRR that improve

nMR

pMR p^MR number of patients out of NRR that improve

nMR

The test is solved under the z-distribution, with alpha = 0.05. Thegreater the difference between pMR and pMR (pMR > pMR) the moresensitive and specic is the NRR criterion tested, the lower the p-va-lue of the test, and better performs over real patients.

5. Application and results

For the 123 patients followed, we have 3366 executions of theidTask = 151 relative to visual memory. Each task execution is de-scribed by an instance. Table 2 shows an instance representing anexecution of a task by a patient, the number of previous executionsof the task by the participant is 23, the result obtained in that spe-

s with Applications xxx (2014) xxxxxx 7769cic task execution is 83.4, and an improvement in the addressed770cognitive sub-function was obtained according to NAB pre and post771differences.

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

772 Execs151. Number of previous executions of the task already773 performed by the participant at the time of current execution.774 Result. Is the result obtained by the participant in the execution775 of the task as described above; it is a [0,100] real value.776 Improvement. Improvement is a binary variable with values777 (YES, NO) and indicates whether or not the patient improved778 with the neurorehabilitation treatment. It is dened as the class779 label Y described in Section 4.1.2. The improvement of a patient780 is determined according to the criteria described in IG clinical781 protocols. Those protocols consider the comparison of the782 NAB neuropsychological assessment both before and after783 treatment. For each cognitive function the protocol is dened784 with the relevant NAB scales.

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

804

805

806The test provided a result that is not statistically signicant807(z = 0.7316, p = 0.2323). This appears to be evidence that using808the single result of the task is not enough to detect either the809NRR or the ZRP zone. In the next sections, a model including the810number of executions per task is tested.

8115.2. Analysis of PREVIRNEC visual memory task using visualization-812based SAP (Vis-SAP)

813The relationship between Results and Number of executions of814patients is shown in Fig. 4. Improving patients improving are815shown in green and non-improving in red. Areas with a single cat-816egory of patients (improving or not improving) are visually817identied.818Looking at Fig. 4, an area with no improvement is obtained for819participants obtaining results higher than 70 and number of execu-820tions lower than 40 at the top left of the scatter plot. Furthermore,821if the number of executions is higher than 60 and results higher822than 20, a large region can be easily identied in which every par-823ticipant is labeled in the improvement group, leading to the iden-824

825

826

827

828

829

830

831

832

833

834

835

836

837

838

839

840

841

842

843

844

Table 2Attributes of classied instances.

Execs151 Result Improvement

23 83.4 YES

8 A. Garca-Rudolph, K. Gibert / Expert Systems with Applications xxx (2014) xxxxxx

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q15.1. Analysis of PREVIRNEC visual memory task using basic criterion

As introduced in Section 3.2.2, nowadays in PREVIRNEC a rsthypothesis is being tested which considers that any participant pa-tient has completed any task within NRR if they obtained result isin the 6585 range, in INRR if it is less than 65, and in SNRR if it ishigher than 85. As a reference, the current NRR used (Re-sult e [65,85]) is visualized in a manually built SAP shown inFig. 3 with an overall sensitivity = 0.5660, overall specic-ity = 0.5012 and overall quality = 0.5022. The percentages repre-sented in the SAP provide the empirical proportion of patientsthat improved after the treatment in every area. About 60% of pa-tients performing idTask = 151 in NRR really improved. This is farfrom random improvement. However, to evaluate the quality ofthe basic NRR used as a reference, a 2-sample probability test isused as described in Section 4.2.2

This enables verication of whether being in NRR really impliesa signicantly higher probability of improvement.

Table 3 contains the relevant information to compute the testby crossing the classication of the patients regarding two factors.Improving/not improving and performing the idTask = 151 withinNRR or not (according to the basic criterion currently used).Fig. 3. Vis-SAP for [68,85

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgtication of a therapeutic range of results depending on thenumber of executions.

Two neat regions emerge from the above rules which can be ex-pressed in the form of logical restriction rules, and visualized in anSAP diagram (shown in Fig. 5 with an overall sensitivity = 0.939,overall specicity = 0.5523 and overall quality = 0.994).

Identied rules are:

(Execs151 < = 40) AND (Res > 70)? Not NRR(Execs151 > 60) AND (Res > 20)? NRR

The quality of the induced denition for NRR is assessed bymeans of the 2-sample proportion test (described in Section4.2.2). Table 4 contains the relevant information to compute thetest by crossing the classication of the patients regarding two fac-tors. Improving/not improving and performing the idTask = 151within NRR or not (according to the rules directly induced overthe SAP). The results are z = 12.42, p

845 ability of improving visual memory function. Thus, upon the Vis-846 SAP criterion, NRR(task151) = Execs151 > 60 and Results > 20.

847 5.3. Analysis of PREVIRNEC visual memory task using DT-SAP

848 Although the visual-based SAP method seems to produce good849 results, the NRR has been dened on the basis of the visual exper-850 tise of the data miner, and this approach is totally dependent of the851

852

853

854

855

856

857

858

859

860

861

862

863

864

865

866

867

868

869

870

871

872With this information, the test is statistically signicant873(z = 13.45 p

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925rarely the tax of improvement goes down to 71.4%. This lower926bound is much higher than the upper bound of the general group,

P fo

Q12

10 A. Garca-Rudolph, K. Gibert / Expert Systems with Applications xxx (2014) xxxxxx

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1Fig. 5. Vis-SA

Table 4Contingency table for VIS-SAP TR for idTask = 151.

Improvement In NRR

Yes No Total

Yes 199 1483 1682No 13 1671 1684of obtaining measurable improvements in the participatingpatients.

The authors are aware that the sample size is small enough toguarantee improvement, but it can be claimed a guarantee ofincreasing the probabilities of improvement of the participatingpatients. For the standard treatment group, an improvement taxof 59% was found with a 95% CI: [0.536,0.644]. This means thatthe tax of improvement would rarely be higher than the 64% of pa-tients. The whole set of 10 patients submitted to the NRR recom-mendations improved. The 95% CI with a 100% improvement tax

927thus proving the efcacy of the recommendation. We can also con-928

929

930

931

932

Total 212 3154 3366^p(YES) 0.9386 0.4701 0.4997

Fig. 6. DT-SAP fo

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgrm this issue by the standard test of comparing two proportionsas shown in Table 6.

6. Discussion

This work aims to identify the conditions in which performing acertain cognitive rehabilitation task maximizes the potential tois [1,1], since the length of the CI is computed as a function ofp(1 p). However, even in the hypothetical case of having one pa-tient without improvement in this small size set, this would lead toa 90% of improvement with a 95% CI of [0.714,1], meaning that

r idTask 151.933activate brain plasticity and to establish the damaged cognitive934function as much as possible. The NeuroRehabilitation Range has935been introduced as a formal concept to express these conditions936and it has been induced on the basis of the SAP tool. As far as we

r idTask 151.

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

937 know, this is the rst time that the number of repetitions of the938 task is considered for modeling the NRR of the task.939 This goes one step ahead on the current state of the art. Most of940 the work done on maximizing the rehabilitative effect of a task has941 been oriented to a proper management of the level of difculty of942 the tasks presented to the patient. In the context of rehabilitation943 through a serious game strategy, the determination of tasks and944 game difculty is made statically by the therapists in most thera-945 peutic games proposed in the literature. For example (Heuser946 et al., 2006)Q6 suggest ve therapeutic games exercises, each one947

948

949

950

951

952

953

954

955

956

957

958

959

960

961

962

963

964

965

966

967

968

969Two methods to induce the NRR are proposed, one based in970visualization of raw data, the other based on DT. Both provide an971easy data-driven tool to determine the NRR of a task.972The Vis-SAP method induced the NRR of task15: (NRR(151) = Ex-973ecs151 > 60 and Res > 20) The DT-SAP provides a more complex974model:

975NRR151 Execs151 44 AND Res > 4ANDRes < 32 OR Execs151 45ANDRes > 57 OR Execs151> 58 AND Res > 8 977977

978But both methods determine a region for Results and Number of979Executions where improvements are concentrated. Bounds for both980Results and Number of Executions dene the conditions of execut-981ing the task that maximizes probability of improvement.982Statistical test assessed that indeed, patients executing the983

984

985

986

987

988

989

990

991

992

993

994

995

996

997

998

999

1000

1001

1002

1003

1004

Table 5Contingency table for DT-SAT TR for idTask = 151.

Improvement In NRR

YES NO Total

Yes 375 1307 1682NO 71 1613 1684Total 446 2920 3366^p(YES) 0.8408 0.4476 0.4997

A. Garca-Rudolph, K. Gibert / Expert Systems with Applications xxx (2014) xxxxxx 11

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1involving a set of difculty levels following patients recovery;the difculty level is statically designed in an increasing way andthe simulation stops the exercise when the patient fails. In Maet al. (2007) the therapeutic game trains visual discriminationand selective attention using three difculty levels: Beginner,Intermediate and Expert; the system includes a matrix assigningsuitable difculty levels for a set of patient proles, that is usedto suggest a difculty level to each patient.

PREVIRNEC is assuming an expert-based constant NRR for thewhole set of available tasks: [65,85]; the system automatically in-creases difculty if the patient performs over the NRR and de-creases it if he/she performs below NRR. All these systems try todeal with the degree of difculty of the task to be proposed tothe patient, but none of them include any kind of guide aboutthe number of executions required to empower the rehabilitationeffect.

According to Lurias theory (Luria, 1978), present researchshows that repetition is important in CR. The main result emergedfrom this analysis is that the number of executions of the CR tasksis clearly relevant to determine NRR and that including them in themodel provides signicant improvement with regards to currentpractice.Fig. 7. DT-SAP for

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgtasks in these conditions, are expected to improve mucheasily (with signicant higher probability) than those executingthe task out of NRR (pVis-SAP(improvement|NRR) = 0.9386, pDT-SAP(improvement|NRR) = 0.8408).

Even if both models involve both Results and Number of execu-tions, the conditions provided by the DT-SAP are more complex,less intuitive for the end-user, and from the numerical point ofview, provide lower quality results. Both specicity and sensitivityas well as the global quality of the SAP are lower than those pro-vided by Vis-SAP. For this reason, the Vis-SAP criterion seems tobe more appropriated for real use.

Despite of that, it is interesting to remark that the denitionsprovided by Vis-SAP and DT-SAP are consistent. Indeed DT-SAPcorrectly recognizes the NRR zone identied by Vis-SAP fortask151. However, the decision tree procedure, cannot detect thenon-improvement zone. In fact, partitions induced by decisiontrees on the plane are complete; thus, the procedure tries to assignimprovement or not to every area in the SAP diagram, thus forcingassignments in areas where the behavior of patients is not sohomogeneous. This, generates a decreasing of the quality indica-tors as predictive power decreases in those zones where heteroge-neity of treatment outcome is higher, leading to a lower global rate.idTask 151.

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

1005

1006

1007

1008

1009

1010

1011

1012

1013

1014

1015

1016

1017

1018

1019

1020

1021

1022

1023

1024

1025

1026

1027

1028

1029

1030

1031

1032

1033

1034

1035

1036

1037

1038

1039

1040

1041

1042

1043

1044

1045

1046

1047

1048

1049

1050

1051

1052

1053

1054

1055

nt h

12 A. Garca-Rudolph, K. Gibert / Expert System

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1In fact, VIS-SAP strategy identied smaller zones showing higherhomogeneity, and this impacts into higher predictive power.

Same kind of behavior has been observed when analyzing otherneurorehabilitation tasks, addressed to different cognitive func-tions (e.g. idTask = 146 for sustained attention, idTask = 153 forverbal memory).

For this particular application the Vis-SAP induced a NRR forTask 151 that suggests more than 60 executions of the task withresults scoring over 20 for a maximum benecial effect. From thepractical point of view, the proposed tools are able to give clini-cians clear criteria to decide how many times the task must be re-peated by the patient and which is the range of performance thatcan produce better improvement. In fact, the NRR conditionsemerged from the SAP were rarely envisaged by therapists alongtypical CR treatments. Also it was observed that assuming theseguidelines to design the CR program signicantly increased thepossibilities of improving the targeted cognitive functions of theparticipating patients.

The main drawbacks of the Vis-SAP proposal are two. On theone hand, the lack of completeness of the Vis-SAP criterion pro-

Fig. 8. ROC curves comparison for VIS-SAP curreposed. Indeed, looking at the SAP diagram, VIS-SAP it is not assign-ing improvement or non improvement to the whole surface, butonly to small parts of the diagram corresponding to concrete andreduced areas where improvement can be ensured, or non re-sponse can be ensured. This means that, even though the probabil-ity of improving is very high inside the NRR, this is not necessarilyimplying that people out of this area do not improve. In fact, theVis-SAP is determining a small area where the non improvementis dominant, but, out of the NRR area almost half of the patientsalso improve, being this close to random response to the treat-ment. So, one could say that Vis-SAP provides a semi-deterministicprocedure where a particular conguration for both results andrepetitions ensure improvement, a second conguration where

Table 6Contingency table for validation of idTask = 151.

Improvement In NRR

YES NO Total

Yes 10 189 199NO 0 128 128Total 10 317 327^p(YES) 1.000 0.5962 0.6100

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgThis work is a contribution towards the personalized, predict-able, and data driven CR design from both a theoretical and practi-cal point of view.

From the theoretical point of view, the paper introduces a newconcept, the NeuroRehabilitation Range (NRR) as the framework todescribe the degree of performance of a CR task which producesmaximum rehabilitation effects. The NRR contributes to providean operational denition for the zone of maximum rehabilitationpotential and represents an operationalization of the Zone of Prox-the task do not produce patients improvement, and out of theseregions the outcome is undetermined.

On the other hand, the proposed analysis considers each taskindividually, being NRR dened for every single task. However, itis already known that the CR is based on sequences of tasks thatinteract among them, and taking in consideration the whole se-quence of tasks involved in the treatment is likely to improve thequality of the model.

7. Conclusions and future work

ypothesis, VIS-SAP proposed NRR and DT -SAP.

s with Applications xxx (2014) xxxxxx1056imal Development referred in Vygotsky (1934).1057Analytical and visual tools are also proposed in this paper, de-1058ned and validated, to nd an operational denition of a NRR from1059a data driven approach. On the one hand, the SAP has been intro-1060duced as a general visualization tool to nd areas with high prob-1061ability of occurrence of a target event. A particular case of SAP for1062detecting cognitive improvement in relation with results and rep-1063etitions of a certain cognitive rehabilitation task is presented in the1064paper. For this particular application, the SAP identies areas with1065high probability of cognitive improvement. Although SAP is not a1066complex concept, it has shown a great potential to nd the NRR re-1067gion of a cognitive rehabilitation task in a quick, simple and very1068intuitive way, which has shown to be highly useful at clinical prac-1069tice level. Also, for the rst time, the NRR is dened as a bivariate1070structure involving conditions in both results and repetitions of the1071tasks.1072Another contribution of the paper is to propose two different1073methodologies to build the SAP in a given real problem: Direct con-1074struction of SAP by visualization of raw data (Vis-SAPmethod); and1075DT-SAP, which is based on decision-tree induction and could be1076automated. Decision trees have been considered because their1077inherent structure is directly providing the NRR model, which is1078built as the OR of all branches bringing to a leaf labeled as improve-1079ment. Both methods effectively determine the areas where proba-

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

1080

1081

1082

1083

1084

1085

1086

1087

1088

1089

1090

1091

1092

1093

1094

1095

1096

1097

1098

1099

1100

1101

1102

1103

1104

1105

1106

1107

1108

1109

1110

1111

1112

1113

1114

1115

1116

1117

1118

1119

1120

1121

1122

1123

1124

1125

1126

1127

1128

1129

1130

1131

1132

1133

1134

1135

1136

1137

1138

1139

1140

1141

1142

1143

1144

1145

1146

1147

1148

1149

1150

1151each task individually1152

1153

1154

1155

1156

1157

1158

1159

1160

1161

1162

1163

1164

1165

1166

1167to be faced from scratch.1168

1169

1170

1171

1172

1173

1174

1175

1176

1177

1178

1179

1180

1181

1182

1183

1184

1185

1186

1187

1188

1189

1190

1191

1192

1193

1194

1195

1196

1197

1198

1199

1200

12011202120312041205

tem

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1bility of improvement are higher; a statistical two-proportion testhas been used to assess the goodness of the NRR models, by check-ing whereas the probability of improvement is signicantly higherwhen tasks are performed according to NRR than out of it. WhereasDT-SAP is a deterministic method that can be automated, the Vis-SAP is a semi-deterministic method that requires visual inspectionin its last step. However, it seems to produce better results in prac-tical applications, as the incomplete sectorization of the plane invery homogeneous areas provided by Vis-SAP outperforms the re-sults induced from a DT where the leaves are often contaminated,containing both improving and non-improving patients.

The last theoretical contribution of this work is the denition ofa quality criterion to assess NRR models, based on pooled con-dence and pooled specicity. The dened criterion is based onthe capacity of a NRR model to detect the patients improving withthe execution of a task. This is somehow giving a global perfor-mance indicator, although ROC curves have also been used to testthe quality of obtained models, and it conrms that both proposedmethods outperform the univariate and static NRR [65,85] cur-rently used by the experts, as well as that Vis-SAP performs slightlybetter than DT-SAP.

Finally, all those elements have been applied to a real casestudy. The application shows how the proposed methodologycould identify NRR for a given cognitive rehabilitation task, andhow the NRR obtained provides clear guidelines to the therapistsabout the number of repetitions of the task to be proposed to thepatient together with the acceptable range of performance desir-able to maximize the effect of the rehabilitation. From the clinicalpoint of view, main contributions of this paper are to show thatrepetition of tasks is really relevant for rehabilitation (as statedby Luria in 1978), to provide an intuitive tool that permits thetherapists to obtain guidelines about how much repetitions ofthe task must be proposed to the patient, and to show that thedesirable difculty level of a task is specic for each task. Thisis an excellent complement to the previous state of the art inwhich some advances were done regarding how to manage thedifculty level of the tasks, but no works assessing repetitionswere addressed.

Clinicians established an initial hypothesis about the NRR,assuming it is xed and task independent (NRR(T) = [65,85]); thesebounds have been dened according to CR therapists expertise.PREVIRNEC allows a systematic pre and post evaluation of partic-ipants covering the major cognitive domains. This provides empir-ical data useful to validate or clarify clinical hypothesis. For therst time, data collected through PREVIRNEC platform has beenused to learn more about the NRR. Although the ratio of improve-ment of patients in that initial NRR was not low, this work pro-vided evidence that a simple formulation for NRR regarding onlythe Results obtained is insufcient to identify the group of patientswith better response to CR treatment. According to our results NRRcannot be dened by means of univariant analysis (consideringonly the Result of performing a task). A predictive model consider-ing other implied covariables needs to be developed. The presentanalysis is a rst attempt into that direction. It has been shownthat the number of repetitions that a patient performs of a certaintask is also relevant for the patients outcome, according to litera-ture. Bidimensional NRR, depending not only on performances, butalso on repetition, signicantly improve the CR treatment design.On the other hand, the range of therapeutic performances mightchange from task to task. This work points to target a specic per-formance-range for each task, instead of the current [65,85] rangeused for the whole set of cognitive tasks available.

An old wondering of the Institute Guttmann was to better

A. Garca-Rudolph, K. Gibert / Expert Sysunderstand how NRR could be found, and since 2007, the institutehas been leading research in this line. This work is providing objec-tive criteria for NRR that can be integrated in daily clinical practice

Please cite this article in press as: Garca-Rudolph, A., & Gibert, K. A data miniBrain Injury patients. Expert Systems with Applications (2014), http://dx.doi.orgThis work is currently being enriched by analyzing how patientswalk through the SAP areas (or sectors) during their rehabilitationprocess. This can be analyzed by connecting the points correspond-ing to a same patient in the SAP and nding prototypical patternsaccording to the form of the paths designed on the SAP. This dy-namic analysis can be later generalized to nd dynamic patternson the global treatment of the patient involving the whole se-quence of tasks performed during the treatment, and providinginformation about the possible positive interactions among tasksthat empower the improvement capacity.

Although the NRR models using number of executions and re-sults seem to provide quite high sensitiveness and specicity, thereare other factors supposed to be highly determinant of cognitiveimprovement, like task difculty. Extension of the current propos-als to include such other factors is currently being explored.

Finally, obtained results are expected to be more interpretableby clinicians when other demographic and clinical variables are in-cluded in the model, e.g. participants educational level, age, timesince injury, obtained results in pre-treatment evaluation.

Acknowledgments

This research was supported by: Ministry of Industry, Tourismand Trade (Spain) AVANZA PLAN-Digital Citizen Subprogram (PT:NEUROLEARNING Grant Nr: TSI-020501-2008-0154). Institute ofHealth Carlos III (Spain) Strategic Action Healths Call (PT: Clinicalimplantation of PREVIRNEC platform in TBI and stroke patients/Grant Nr: PI08/900525). Ministry of Science and Innovation (Spain)INNPACTO Program (PT NEUROCONTENT - Grant Nr 300000-2010-30). Ministry of Education Social Policy and Social Services (Spain)IMSERSO Program (PT COGNIDAC Grant Nr 41/2008). MARATTV3 Foundation (PT: Improving Social Cognition and meta-cogni-tion in schizophrenia: A tele-rehabilitation project Nr 091330)and EU CIP-ICT-PSP-2007-1 (PT: CLEAR Grant Nr.: 224985)

References

Bernabeu, M., & Roig. T. (1999). La rehabilitacin del traumatismo craneoenceflico:un enfoque interdisciplinar. Barcelona: Fundaci Institut Guttmann.

Calero, M. D., & Navarro, E. (2007). Cognitive plasticity as a modulating variable onthe effects of memory training in elderly persons. Archives of ClinicalNeuropsychology, 22, 6372.Till now, CR plans are mainly built from scratch for every pa-tient, on the basis of the expertise of therapist and the follow-upof the patient, as no standard guidelines are available in this do-main yet. The ndings from the present study led to new action-able knowledge in the eld of rehabilitation practice, opening thedoor towards more precise, predictable and powerful CR treat-ments, customized for the individual patient. Some clinicalhypothesis are being formulated by specialists on the basis of theseresults and currently under validation, as a previous step to theestablishment of a methodology for personalized therapeuticinterventions based on clinical evidence.

As future research lines, the automatic construction of SAP stillrequires more work since decision trees imply, by construction,some intrinsic error taxes in every branch that will be always prop-agated to the NRR performance and automation from Vis-SAP hasof the institution, as well as operationalized for PREVIRNEC plat-form, that provides the support to verify clinical hypothesis.

As discussed in previous section, the Vis-SAP provides a semi-deterministic criterion that outperforms DT-SAP, but the later isautomatable. On the other hand, the proposed analysis considers

s with Applications xxx (2014) xxxxxx 131206Cicerone, K. D., & Tupper, D. E. (1986). Cognitive assessment in the1207neuropsychological rehabilitation of head-injured adults. In B. P. Uzzell & Y.1208Gross (Eds.) (pp. 5984). Boston: Martinus Nijhoff. Q7

ng approach to identify cognitive NeuroRehabilitation Range in Traumatic/10.1016/j.eswa.2014.03.001

1209 Cicerone, K. D., Langenbahn, D. M., Braden, C., Malec, J. F., & Kalmar, K. (2011).1210 Evidence-based cognitive rehabilitation: Updated review of the literature from1211 2003 through 2008. Archives of Physical Medicine and Rehabilitation, 92(5),1212 1930.1213 Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience. Harper1214 perennial1215 DiPiro, J. T., & Spruill, W. J. (2010). Concepts in clinical pharmacokinetics (5th ed.)1216 American Society of Health-System Pharmacists (January 1) ISBN-13: 978-1217 1585282418.1218 Ecri. Cognitive rehabilitation therapy for traumatic brain injury: What we know and1219 dont know about its efcacy. EDITORIAL NOTE 10/11/11: IOMs New Report on1220 Brain Injury Treatments Draws Conclusions Similar to ECRI Institutes Earlier1221 Findings.Q81222 Finneran, C. M., & Zhang, P. (2005). Flow in computer-mediated environments:1223 Promises and challenges. Communications of the Association for Information1224 Systems, 15, 82101.1225 Gibert, K., & Garca-Rudolph, A. (2007). Desarrollo de herramientas para evaluar el1226 resultado de las tecnologas aplicadas al proceso rehabilitador Estudio a partir1227 de dos modelos concretos: Lesin Medular y Dao Cerebral Adquirido. Cap 6:1228 Posibilidades de aplicacin de minera de datos para el descubrimiento de1229 conocimiento a partir de la prctica clnica. Madrid: Plan Nacional para el1230 Sistema Nacional de Salud del Ministerio de Sanidad y Consumo. Barcelona:1231 Agncia dAvaluaci de Tecnologia I Recerca Mdiques. Informes de Evaluacin1232 de Tecnologias Sanitarias. AATRM nm. 2006/11. Barcelona.1233 Green, C., & Bavelier, D. (2005). In Messaris & Humphreys (Eds.), Digital media:1234 Transformations in human communication.1235 Green, C. S., & Bavelier, D. (2007). Action-video-game experience alters the spatial1236 resolution of vision. Psychological Science, 18(1), 8894.1237 Hall, M., Frank, E., Holmes, G., Pfahringer Reutemann, P., & Witten, I. H. (2009). The1238 WEKA data mining software: An update. SIGKDD Explorations, 11(1).1239 Heuser, H. A., Kourtev, H., Winter, S., Fensterheim, D., Burdea, G., Hentz, V., et al.12401241124212431244124512461247124812491250125112521253125412551256125712581259

1260Marcano, A., Chausa, P., Garcia-Rudolph, A., Cceres, C., & Tormos, J. M. (2013). Data1261mining applied to the cognitive rehabilitation of patients with acquired brain1262injury. Data mining applied to the cognitive rehabilitation of patients with1263acquired brain injury. Expert Systems with Applications, 40(4), 10541060. ISSN12640957-4174.1265Mathwick, C., & RigdonPlay, E. (2004). Flow, and the online search experience.1266Journal of Consumer Research, 31(2), 324332.1267Norea, D., Rios-Lago, M., Bombin-Gonzalez, I., Sanchez-Cubillo, I., Garcia-Molina,1268A., & Tirapu-Ustarroz, J. (2010). Efectividad de la rehabilitacin neuropsicolgica1269en el dao cerebral adquirido (I): atencin, velocidad de procesamiento,1270memoria y lenguaje. Revista de Neurologia, 51, 687698.1271Orvis, K., Horn, D., & Belanich, J. (2008). The roles of task difculty and prior1272videogame experience on performance and motivation in instructional1273videogames. Computers in Human Behavior, 24, 24152433.1274Pascual-Leone, A. et al. (2005). The plastic human brain cortex. Annual Review of1275Neuroscience, 28, 377401.1276Quinlan, R. (1993). C4.5: Programs for machine learning. San Mateo, CA: Morgan1277Kaufmann Publishers.1278Rego, P., Moreira, P. M. & Reis, L. P. (2010). Serious games for rehabilitation a survey1279and a classication towards a taxonomy. In Information systems and technologies1280(CISTI) (pp. 16).1281Rohling, M. L., Faust, M. E., et al. (2009). Effectiveness of cognitive rehabilitation1282following acquired brain injury: A meta-analytic re-examination of Cicerone1283et al.s (2000, 2005) systematic reviews. Neuropsychology, 23(1), 2039.1284Serra, J., Arcos, J. Ll., Garcia-Rudolph, A., Garca-Molina, A., Roig, T., & Tormos, J. M.1285(2013). Cognitive prognosis of acquired brain injury patients using machine1286learning techniques. In International conference on advanced cognitive1287technologies and applications (COGNITIVE), IARIA (pp. 108113). Valencia, Spain.1288Sohlberg, M. M. (2001). In A. Mateer (Ed.), Cognitive rehabilitation. An interactive1289neuropsychological approach. Catherine. ISBN: 9781572306134.1290Tormos, J. M., Garcia-Molina, A., Garcia Rudolph, A., & Roig, T. (2009). Information1291and communications technology in learning devel Q9opment and rehabilitation.1292Q101293129412951296129712981299130013011302130313041305130613071308Q11

1309

14 A. Garca-Rudolph, K. Gibert / Expert Systems with Applications xxx (2014) xxxxxx

ESWA 9208 No. of Pages 14, Model 5G

20 March 2014

Q1(2006). Tele-rehabilitation using the Rutgers master II glove foll