stable long-term bci-enabled communication in als …shenoy/grouppublications/...85 crossings have...

Stable long-term BCI-enabled communication in ALS and locked-in syndrome using 1

LFP signals 2

Tomislav Milekovic1,2,3*†#, Anish A. Sarma2,4,5†, Daniel Bacher2,4, John D. Simeral5,2,4, Jad Saab2,4, Chethan Pandarinath6,11,15, 3

Brittany L. Sorice9, Christine Blabe6, Erin M. Oakley9, Kathryn R. Tringale9, Emad Eskandar7,8, Sydney S. Cash8,9, Jaimie M. 4

Henderson6,10,15‡, Krishna V. Shenoy11,12,13,14,15,16‡, John P. Donoghue5,1,2, Leigh R. Hochberg5,2,4,8,9 5

1Department of Neuroscience, Brown University, Providence, RI, USA. 6 2Brown Institute for Brain Science, Brown University, Providence, RI, USA. 7 3Department of Fundamental Neuroscience, Faculty of Medicine, University of Geneva, Geneva, Switzerland. 8 4School of Engineering, Brown University, Providence, RI, USA. 9 5Center for Neurorestoration and Neurotechnology, Rehabilitation Research & Development, Department of Veterans Affairs, 10

Providence, RI, USA. 11 6Department of Neurosurgery, Stanford University, Stanford, CA, USA. 12 7Department of Neurosurgery, Massachusetts General Hospital, Boston, MA, USA. 13 8Harvard Medical School, Boston, MA, USA. 14 9Department of Neurology, Massachusetts General Hospital, Boston, MA, USA. 15 10Department of Neurology and Neurological Sciences, Stanford University, Stanford, CA, USA. 16 11Department of Electrical Engineering, Stanford University, Stanford, CA, USA. 17 12Neurosciences Program, Stanford University, Stanford, CA, USA. 18 13Department of Neurobiology, Stanford University, Stanford, CA, USA. 19 14Department of Bioengineering, Stanford University, Stanford, CA, USA. 20 15Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA. 21 16Howard Hughes Medical Institute at Stanford University, Stanford, CA USA. 22

†,‡ These authors contributed equally. 23

* Corresponding author: [email protected] 24 # Current address: Department of Basic Neuroscience 25

Faculty of Medicine 26

University of Geneva 27

Campus Biotech, 9 chemin des Mines, 1202 Geneva, Switzerland 28

Running Head: Long-term communication BCI in people with tetraplegia 29

Downloaded from www.physiology.org/journal/jn by ${individualUser.givenNames} ${individualUser.surname} (171.066.209.004) on April 30, 2018.Copyright © 2018 American Physiological Society. All rights reserved.

ABSTRACT 30

Restoring communication for people with locked-in syndrome remains a challenging clinical problem without a reliable 31

solution. Recent studies have shown that people with paralysis can use brain-computer interfaces (BCIs) based on intracortical 32

spiking activity to efficiently type messages. However, due to neuronal signal instability, most intracortical BCIs have required 33

frequent calibration and continuous assistance of skilled engineers to maintain performance. Here, an individual with locked-in 34

syndrome due to brainstem stroke and an individual with tetraplegia secondary to amyotrophic lateral sclerosis (ALS) used a 35

simple communication BCI based on intracortical local field potentials (LFPs) for 76 and 138 days, respectively, without 36

recalibration and without significant loss of performance. BCI spelling rates of 3.07 and 6.88 correct characters/minute allowed 37

the participants to type messages and write emails. Our results indicate that people with locked-in syndrome could soon use a 38

slow but reliable LFP-based BCI for everyday communication without ongoing intervention from a technician or caregiver. 39

NEW & NOTEWORTHY 40

This study demonstrates, for the first time, stable repeated use of an intracortical brain-computer interface by people with 41

tetraplegia over up to four and a half months. The approach uses local field potentials (LFPs), signals that may be more stable 42

than neuronal action potentials, to decode participants’ commands. Throughout the several months of evaluation, the decoder 43

remained unchanged; thus no technical interventions were required to maintain consistent brain-computer interface operation. 44

KEYWORDS 45

brain-computer interface; communication; people with locked-in syndrome; local field potentials; long-term stability 46

INTRODUCTION 47

Communication is a major goal for people with locked-in syndrome (LIS) (Plum and Posner 1972), who due to their motor 48

impairment cannot move their limbs or speak but remain conscious and mentally engaged (Bauer et al. 1979). Enabling people 49

with LIS to communicate has a positive effect on their quality of life (Bach 1993; Hecht et al. 2002), can promote reintegration 50

into society, improving the ability to lead a fulfilling, productive life (Doble et al. 2003; Laureys et al. 2005; Young and 51

McNicoll 1998). 52

Brain-computer interfaces (BCIs) – devices that translate a user’s neural activity into computer commands – have emerged as 53

promising tools for helping people with LIS communicate. Previous studies have used BCIs for communication with people 54

with LIS (Bacher et al. 2014; Birbaumer et al. 1999; Chaudhary et al. 2017; Gallegos-Ayala et al. 2014; Jarosiewicz et al. 2015; 55

Kennedy et al. 2000; Kubler et al. 1999; Kubler et al. 2001; Kubler et al. 2005; McCane et al. 2014; Nijboer et al. 2008; Sellers 56

et al. 2014; Vansteensel et al. 2016). Some of these studies relied on shaping of neural responses using biofeedback to achieve 57

accuracy permitting communication, a lengthy process lasting for weeks and, in some cases, requiring EEG preparation by 58

technicians before each use (Birbaumer et al. 1999; Chaudhary et al. 2017; Kubler et al. 1999; Kubler et al. 2005; Vansteensel et 59

al. 2016). Other studies relied on daily calibration of BCIs, which required intermittent assistance of highly skilled engineers 60

(Bacher et al. 2014; Gallegos-Ayala et al. 2014; Nijboer et al. 2008; Sellers et al. 2014). A BCI system capable of providing 61

independent communication to people with LIS, where technical support or interaction with a caretaker is not required 62

frequently, still needs to be developed. 63

Among these approaches, BCIs based on intracortically-recorded action potentials have been used to achieve communication by 64

a person with incomplete LIS (Bacher et al. 2014; Jarosiewicz et al. 2015) or ALS (Gilja et al. 2015; Jarosiewicz et al. 2015; 65

Pandarinath et al. 2017); and enable people with tetraplegia to control robotic arms (Aflalo et al. 2015; Collinger et al. 2013; 66

Hochberg et al. 2012; Hochberg et al. 2006; Wodlinger et al. 2015) or their own muscles using functional electrical stimulation 67

(Ajiboye et al. 2017; Bouton et al. 2016). In addition to spiking signals, intracortically implanted microelectrodes also record 68

local field potentials (LFPs), neuronal signals thought to represent the summed activity of neuronal populations in the vicinity of 69

the microelectrode (Buzsaki et al. 2012). LFPs recorded in the motor cortex of people with paralysis modulate with attempted 70

movements in multiple frequency bands (Ajiboye et al. 2012; Hochberg et al. 2006) and have enabled a person with LIS to spell 71

using a BCI (Kennedy et al. 2000). Information about arm actions has been found in non-human primates in both signal 72

modalities (Ajiboye et al. 2012; Bansal et al. 2012; Bansal et al. 2011; Flint et al. 2012a; Flint et al. 2012b; Flint et al. 2013; 73

Heldman et al. 2006; Mehring et al. 2003; Rickert et al. 2005; Stark and Abeles 2007). 74


Sorted spike recordings from intracortical implants can be unstable across periods of days to months (Dickey et al. 2009; Perge 75

et al. 2013; Perge et al. 2014; Simeral et al. 2011). Signals from individual neurons can be lost due to array movements as small 76

as a few tens of µm in relation to the brain. The number of recorded well-discriminated neurons can also diminish over time, 77

due to degradation of the electrode materials, biological responses to the sensor, or other factors (Barrese et al. 2013; Bjornsson 78

et al. 2006; Shain et al. 2003; Simeral et al. 2011). Partly in response, previous studies have predominantly used frequent 79

recalibration of decoders, often by highly trained engineers and researchers on site, to maintain a high level of performance 80

despite signal instability. While recent studies showed that able-bodied non-human primates can use BCIs with an unchanged 81

decoder for over six months without a significant drop in performance (Flint et al. 2013; Nuyujukian et al. 2014), other recent 82

studies pointed towards comparatively greater intracortical signal variation in people (Jarosiewicz et al. 2015; Perge et al. 2014). 83

One approach to this problem is to build an adaptive decoder (Jarosiewicz et al. 2015). In addition, multi-unit threshold 84

crossings have been proposed as a more stable alternative to sorted spikes (Flint et al. 2016; Fraser et al. 2009; Gilja et al. 2015). 85

Alternatively, LFP signals could be employed to achieve long-term reliable BCI control. 86

Here we demonstrate stable, long-term LFP-based communication in two people with tetraplegia. Our study included an 87

individual with LIS, unable to move his limbs or speak, and an individual with progressive tetraplegia secondary to ALS. They 88

used an LFP-based BCI with a decoder unchanged for 76 and 138 days to communicate with family members, type messages 89

and send emails, without a loss of performance (Figure 1). Our study demonstrates that an intracortical LFP-based BCI can be 90

used for independent communication without the need for re-calibration, thereby reducing the need for caregiver and/or family 91

intervention during communication. 92

Our results provide evidence that intracortical BCIs could be used in-home, for months and with minimal technical oversight, all 93

of which are necessary for a practical communication system. With this new benchmark for stable, LFP-based neural control, 94

future work can continue to address more complex control modalities and higher-resolution neural signals to achieve stable 95

control of more complex neuroprosthetic systems. This study is a step towards a reliable and robust BCI that will allow people 96

with LIS to communicate independently and, therefore, provide greater and more extensive interactions with their friends, 97

family, and caregivers. 98

MATERIALS AND METHODS 99

Participants 100

Permission for these studies was granted by the US Food and Drug Administration (Investigational Device Exemption) and 101

Institutional Review Boards of Stanford University (protocol # 20804), Partners Healthcare/Massachusetts General Hospital 102

(2011P001036), Providence VA Medical Center (2011-009), and Brown University (0809992560). The participants in this study 103

were: T2, a man with classic LIS secondary to a brainstem stroke; and T6, a woman with tetraplegia resultant from ALS. They 104

were enrolled in a pilot clinical trial of the BrainGate Neural Interface System (http://www. 105

clinicaltrials.gov/ct2/show/NCT00912041). As part of the trial, the participants received one 96-channel intracortical multi-106

electrode array (Blackrock Microsystems) in the arm area of the dominant precentral gyrus (Yousry et al. 1997) using a 107

previously described procedure (Hochberg et al. 2006). Both participants were right handed. At the time of the study, T2 was 108

fed through a percutaneous gastrostomy tube and had nighttime-only volume-control lung ventilation to support his breathing. 109

His respiratory drive remained intact throughout the study. T6 had a tracheostomy and on-demand ventilation, though she did 110

not require constant ventilatory support (mostly at night, similar to T2). She was able to eat. Her ALSFRS-R score was 16. T6 111

could speak and use accessibility devices to control a computer with limited hand movement. Neural recordings were collected, 112

stored and processes for online BCI control as described previously (Simeral et al. 2011). 113

Task 114

Participants took part in research sessions in their homes during which they interacted with the FlashSpeller text-entry 115

application through selections: discrete events identified their attempts to perform a particular movement based on their 116

neuronal activity. The FlashSpeller scanned through character-entry options and other options such as backspace, space, word 117

completion, text-to-speech, or email. An option was presented for 1.5s for T6 and 2s for T2. Presentations of two options were 118

separated by a period of 0.1s for T6 and 0.3s for T2. If selected, the option button turned green for 0.9s. As selected movements, 119


referred to as “click actions” in the remainder of the text, T2 attempted to squeeze his right hand and T6 flexed her right index 120

finger. 121

Each individual session was divided into blocks lasting up to twelve minutes each. We asked the participant to perform one of 122

four tasks, depending on the block type: (i) normalization – observe the FlashSpeller automatically typing a phrase while not 123

interacting in any way; (ii) open-loop calibration – observe the FlashSpeller automatically typing a phrase while trying to 124

perform click actions as if to spell the phrase; (iii) copy-phrase – control the FlashSpeller using a BCI with either a same-day 125

decoder or the historical decoder to type an assigned phrase, and (iv) free-spelling - control the FlashSpeller using a BCI with a 126

historical decoder to type a message of choice. 127

Three types of sessions were run in this study: collection, comparison and communication (Figure 2). All sessions started with a 128

normalization block. In collection sessions, we collected open-loop data through open-loop calibration blocks that were used 129

afterwards to calibrate a historical decoder. In comparison sessions, after three open-loop calibration blocks were used to 130

calibrate a same-day decoder, the historical and same-day decoder were compared in a double-blind fashion using copy-phrase 131

blocks. Occasionally, comparison sessions would end with one or more free-spelling blocks. In communication sessions, the 132

historical decoder was used for free typing. 133

Preprocessing 134

Neuronal recordings were first down sampled to 15kHz. We then computed power spectral densities for each individual channel 135

to exclude channels that exhibited large peaks at a line noise frequency (60Hz) or its harmonics. We then referenced the 136

recordings to the common average calculated over the remaining low-noise channels (80 channels for T2 and 33 channels for 137

T6). 138

Spectral amplitude normalization 139

At the beginning of every session, the participant completed a normalization block during which he was instructed to observe 140

the monitor during the presentation of an open-loop calibration block, but not to respond to any cues. Recordings from this 141

block were first pre-processed, then low-pass filtered using a 30th order windowed linear-phase finite impulse response filter 142

(WLPFIR) and then downsampled to 1kHz. A short time Fourier transform (STFT; 256ms window; using a Hamming window) 143

was then used to estimate spectral amplitudes in 3.91Hz wide frequency bins in 20ms steps. For each frequency bin, we 144

calculated the average amplitude recorded by a single channel over the normalization block. This produced a normalization 145

matrix of size 96 x 129, where 96 was the number of channels and 129 was the number of frequency bins. 146

Determining frequency bands of the intermediate and high frequency components 147

To estimate the appropriate frequency bands that would provide selection-related neuronal responses, we processed the 148

recordings from all open-loop calibration blocks by pre-processing the data, low-pass filtering it using a 30th order WLPFIR, 149

down sampling it to 1kHz and then using a STFT (256ms window; using a Hamming window) to estimate amplitudes for each 150

of the frequency bins in 50ms steps. Amplitudes were then normalized using the spectral amplitude normalization calculated 151

from the normalization block. Selection-related epochs were then defined as the time from the selection cue until 1.8 seconds 152

after the cue for T2 and until 1.5 seconds after the cue for T6. Amplitudes calculated during the remaining time (no cue or cue 153

that should not be responded to) were taken as baseline. We then calculated the signal to noise ratio (SNR) between cue-related 154

neuronal responses and the baseline activity for each electrode el and for each of the frequency bands, defined by the starting 155

and ending bin of a given frequency band, bS and bE respectively (Milekovic et al. 2012). 156

1ˆ ; , ;

E

S

bel el

i S E i

b bE S

X t b b X t tc bb b

157


1

2

1

2

1 ˆ; , ; ,

1 ˆ; , ; , ; ,

1 ˆ, ; ,

1 ˆ, ; , ,

B

B

Kel el

C S E i S E

i

Kel el

C S E i S E C S E

i

el el

B S E S E

t T

el el

B S E S E B S E

t T

t b b X t b bK

t b b X t b b t b bK

b b X t b bT

b b X t b b b bT

158

; , ,; ,

; , ,

el el

C S E B S Eel

S E el el

C S E B S E

t b b b bSNR t b b

t b b b b

159

where Xel(t;b) is the amplitude in the frequency bin b and estimated from a window that ends at time t on an electrode el; K is 160

the number of all selection-related epochs in a session; TB is the set of all time points that belong to the baseline; and T is the 161

cardinal size of TB. To reach a single value representing the signal to noise ratio as a function of a frequency band, we evaluated 162

the frequency band SNR by calculating the maximum SNR over time and averaging it over all electrodes: 163

1, ; ,

max arg ; ,

el

S E MAX S E

elel

el

MAX S Et

SNR b b SNR t b bN

t SNR t b b

164

This procedure was performed for collection sessions from session days 452, 453, 460, 467 and 474 for T2; and session days 165

313, 322, 336, 348 and 355 for T6. 166

These same sessions were used to build the historical decoders for each of the participants. All of these sessions showed three 167

well isolated local maxima of the SNR(bS,bE) (Figure 3). One maximum was present in the very low frequencies, typically 0-168

6Hz. Selection-related neuronal responses in this band were captured by the low frequency component (LFC) obtained by low-169

pass filtering the common—average referenced signals using the filter with a -3dB cutoff frequency of 4.63Hz. We determined 170

the start and end bins for the other two frequency bands for each of the five sessions and used the average as the start and end 171

bins for the intermediate and high frequency components (IFC and HFC, Table 1). 172

Extraction of neural features 173

We then extracted three different components of the LFPs: (i) low-pass filtered component (LFC), (ii) intermediate frequency 174

component (IFC) and (iii) high frequency component (HFC). LFC was extracted by low-pass filtering the neural recordings (-175

3dB cutoff frequency of 4.63Hz). To extract IFC and HFC, we first calculated the spectral amplitudes of neural recordings using 176

a short time Fourier transform (STFT). We then divided the amplitude in each frequency bin by its mean in the normalization 177

block and averaged these normalized amplitudes across the IFC and HFC bands. We then defined a neuronal feature, Φf, at time 178

t as one of LFC, IFC or HFC derived from the electrode el: 179

el

el

f

el

LFC t

t IFC t

HFC t

180

To account for the drifts of the neuronal features, we z-scored the features using recursive estimates of the mean and variance 181

(see “Adaptive feature normalization” section in the Appendix). 182

Decoder calibration and feature selection 183

Data for historical decoder calibration were collected over several collection sessions spanning up to 42 days. This data was then 184

used to rank all 288 available features according to the signal-to-noise ratio (SNR) between peri-click and baseline feature 185

values (see “SNR used for selection of neural features” section in the Appendix). We then selected the top 5 LFC, top 5 IFC, 186


and top 5 HFC features, followed by the top 35 remaining features for a total of 50. This procedure selected 16 LFC, 5 IFC and 187

29 HFC features for T2, and 5 LFC, 5 IFC and 40 HFC features for T6. 188

These features were then used to calibrate a regularized linear discriminant analysis (rLDA) classifier that calculated the 189

selection probability for a given set of feature values (see “Calibration of decoding algorithms” section in the Appendix). 190

Parameters of this historical decoder were optimized using the normalized mutual information CYX between the decoded 191

selection and selection cues provided during the open-loop calibration blocks of the collection sessions. A same-day decoder 192

was calibrated to data from three open-loop calibration blocks recorded on that day following the same procedure. 193

Decoding algorithm 194

A regularized linear discriminant analysis (rLDA) classifier, i.e. either a historical or a same-day decoder, was used to calculate 195

the probability of a selection every 20ms. When the probability crossed a threshold of 95%, a selection command was sent to the 196

text-entry application. After the selection was detected, a refractory period of 0.5 seconds was imposed, during which another 197

selection would not be detected. In addition, the probability had to drop to 75% or below before another selection could be 198

detected. 199

Cross-session stability of LFP responses 200

To investigate the stability of LFP signals over time, we calculated the correlation coefficient between mean single-feature LFP 201

responses during click actions recorded on two sessions s1 and s2, rf(s1,s2) (see “Measuring cross-session stability of LFP 202

responses” section in the Appendix). To extract an overall estimate of stability of LFP signals between two sessions, we then 203

calculated the median of rf(s1,s2) over all features, Mr(s1,s2). To estimate the trends of LFP signal stability, we examined the 204

relationship between the Mr(s1,s2) and the number of days between two sessions. A slope b of the line fitted to Mr(s1,s2) as a 205

function of time using ordinary least squares gave us the estimate on the rate of instability of the LFP signals. 206

Performance measures 207

Decoding accuracy was measured using normalized mutual information CYX (Coombs et al. 1970), a metric based on 208

information theory, between the decoded selections and either the selection cues provided to the participants, in open-loop 209

blocks, or the assumed selections a participant wanted to perform based on the final spelled phrase in copy-phrase and free-210

spelling blocks. CYX was used to optimize the parameters of a decoding algorithm during calibration and to obtain a measure of 211

spelling performance that was independent of the spelled phrase. The spelling performance of our communication interface was 212

measured by counting the number of correctly typed characters per minute. For a more elaborate derivation of these measures, 213

see “Measuring spelling and decoding performance” section in the Appendix. 214

Retrospective comparison to a hypothetical single electrode 215

It has been shown that people can use BCIs based on neural signals recorded from the surface of the cortex 216

(electrocorticography) for rapid spelling (Brunner et al. 2011) and control of computer cursors and robotic arms (Wang et al. 217

2013). For a simple, switch-based communication demonstrated in this study, a BCI based on single electrocorticographic 218

electrode might perform comparably to a BCI based on LFPs recorded from the whole implanted intracortical array, while 219

covering a similar cortical surface. We explored this secondary question by simulating the case where only one broadband 220

signal - the average of LFP signals recorded across the intracortical array - was used for neural decoding. In a retrospective 221

analysis, this average LFP signal was used to define intermediate and high frequency bands and to calibrate an average signal 222

decoder based on open-loop calibration blocks of the data collection sessions. We then used this decoder to decode participant’s 223

actions during the open-loop blocks of the comparison sessions. Since participant did not receive any decoding feedback during 224

these blocks, this allowed for a fair comparison between this average signal decoder and the historical decoder that used LFPs 225

from multiple electrodes. Furthermore, while the closed-loop blocks could vary in size, the open-loop calibration sequence was 226

standardized for each participant over all sessions. Therefore, the same amount of data was used to calibrate (fifteen OLC 227

blocks) and test (three OLC blocks) both historical and average signal decoders on each session. We compared the decoders by 228

calculating CYX from offline decoding. In order to prevent an undue penalty on the average signal decoder from particularly 229


noisy signals originating from one or more electrodes, the average LFP was calculated using only the LFPs from the channels 230

that contributed to the common average in the multiple-electrode case. 231

RESULTS 232

Interaction of participants with a FlashSpeller communication BCI 233

Our study shows that two people with tetraplegia, one of whom was locked-in, can spell text at a rate of about a word per minute 234

using a FlashSpeller BCI with a decoder that remained unchanged, without a drop in performance, for time periods of several 235

months. This BCI decoder was calibrated on data collected during five one-day sessions (spanning a period of up to 42 days) 236

and was then used unchanged for communication for up to 138 days afterwards. To spell messages on a computer monitor, two 237

participants - an individual with LIS secondary to brainstem stroke (T2) and an individual with tetraplegia secondary to ALS 238

(T6) - used a custom text-entry application, called FlashSpeller, with an LFP-based BCI (Figure 1, 4; Table 2). FlashSpeller was 239

designed for individuals with limited or no eye movement, which occurs commonly in people with LIS. In the middle of the 240

screen, a large grey button displayed options for 1.5-2s. To select the currently displayed option, i.e. to “click”, the participant 241

attempted or executed a chosen movement. LFP responses elicited during this “click actions” were recorded from the motor 242

cortex and decoded as clicks by the BCI. 243

The study lasted for 99 and 181 days in total for T2 and T6, respectively. At the beginning of the study, we ran “data collection” 244

sessions to accumulate a large number of neuronal responses during participants’ click actions, which were used to calibrate a 245

historical decoder. Data collection sessions were followed by “comparison” sessions, in which the historical decoder was 246

compared to a decoder calibrated on data recorded on the same day (“same-day” decoder) by measuring the participants’ 247

performance in spelling assigned phrases during “copy- spelling” blocks. Often, at the end of comparison sessions, participants 248

used the historical decoder to freely spell messages during “free-spelling” blocks. We also ran separate “communication” 249

sessions, in which the participants used the historical decoder to spell messages. The historical decoder was calibrated only once 250

for each participant and was used in all comparison sessions and all communication sessions. 251

To create BCI decoders, we identified three different features of LFPs that modulated in relation to participants’ attempts to 252

perform click actions (Figure 5, 6a). The low-pass filtered component (continuous signal filtered using a low-pass filter below 253

4.63Hz) reflected a slow, large-amplitude deflection related to the selection actions. Activity in the intermediate frequency 254

component (normalized spectral amplitude in 14-33Hz for T2 and 21-41Hz for T6) showed a decrease following the cue, while 255

the high frequency component (normalized spectral amplitude in 53-193Hz for T2 61-283Hz for T6) showed an increase in 256

activity. Modulations of these features could be distinguished from LFPs recorded on individual trials and were used to detect 257

participants’ intentions to select the presented options (Figure 4, Movie 1-8). 258

In addition to quantifying the performance and stability of our BCI system, we investigated different causes that led to stable 259

long-term spelling performance, formulated in the following questions. (i) Was the stable performance a direct consequence of 260

the stability of LFP responses that occurred during click actions? (ii) Does the resolution of intracortical recordings substantially 261

alter the decoding performance as compared to a hypothetical electrocortigraphic electrode? (iii) Was the design of our decoder 262

responsible for stable decoding performance despite the instability of LFP responses during click actions? 263

Stability of LFP responses to click actions over the whole microelectrode array 264

LFP responses during click actions in features used for decoding were similar over the duration of the study (Figure 5, 6a). We 265

also measured the stability of LFP signals recorded over the whole MEA over time as follows. For each session, we calculated 266

the mean LFP response time-course for each feature, ranging from the click cue to 1.5s after the click cue, across all click 267

actions recorded during that session’s open-loop calibration blocks (34 and 68 cue responses for sessions T2 466 and T2 467, 268

respectively; 51 cue responses for all other sessions). For each feature’s mean,we then calculated the correlation coefficient 269

between any pair of sessions, hereafter referred to as “single-feature cross-session correlation coefficient”. Finally, we 270

calculated the median cross-session correlation coefficient across all 288 features (96 electrodes times three feature types), thus 271

obtaining a single value for each session pair. 272

This median single-feature cross-session correlation coefficient did not change significantly with inter-session period for T2 (b 273

= -0.02year-1; P = 0.78; Spearman’s rank correlation test) while it significantly diminished with the inter-session period for T6 274


(T6: b = -0.21year-1; P < 0.05; testing non-zero correlation between the correlation coefficient and inter-session period, 275

Spearman’s rank correlation test; Figure 6b). 276

Stability of detection based on LFP neuronal responses 277

We tested the stability of the LFP-based detection of clicks by using a decoder, calibrated on all open-loop calibration blocks 278

from one collection or comparison session, to detect clicks on open-loop calibration blocks from another session in the future or 279

in the past. The standardized calibration sequence for each participant over all sessions ensured that the same amount of data 280

was used to calibrate (three OLC blocks) and test (three OLC blocks) each decoder. Since participants did not control the 281

spelling during open-loop blocks and, therefore, did not receive any feedback on their performance, they could not adapt their 282

strategy to modify the decoding performance. Decoding performance was measured using normalized mutual information (CYX) 283

between the decoded clicks and the clicks that would lead to errorless spelling (Coombs et al. 1970; Milekovic et al. 2013). 284

For both participants, offline detection performance, as measured by CYX, was variable, but did not degrade as a function of the 285

number of days since decoder calibration, ΔT (Figure 7). Correlation between CYX and ΔT was not significant (T2: P=0.82; T6: 286

P=0.46, Spearman’s rank test). Average CYX calculated this way was 0.68 and 0.72 for T2 and T6, respectively. 287

Spelling performance and other performance measures 288

Both study participants controlled the FlashSpeller using LFP signals and a historical decoder to communicate by spelling 289

messages (Figure 8). The average spelling performance, as measured by the number of correct characters per minute (CCPM), 290

during free-spelling blocks using word completion was 3.07±0.20CCPM and 6.88±0.33CCPM for T2 and T6, respectively. 291

These rates enabled participants to spell full sentences within several minutes. The period over which the unchanged historical 292

decoder was used by the participants for communication spanned 76 and 138 days for T2 and T6, respectively. During this 293

period the participants’ spelling performance did not change significantly (P value for testing non-zero correlation between 294

spelling performances during free-spelling blocks and historical decoder age: T2: P = 0.40; T6: P = 0.73; Spearman’s rank 295

correlation test). 296

Spelling performance takes into account both the accuracy of the selection detections and the difficulty of the typed phrases. It 297

may be possible that participants chose to type easier phrases during the sessions in which the decoders were not as accurate, 298

thus masking the instability of LFP responses used for detection of selections. We used CYX to compare the performance of 299

decoders irrespective of the typed phrases (Figure 9). Over all sessions with free-spelling blocks in which word completion was 300

used, CYX in those blocks did not change significantly (P value for testing non-zero correlation between CYX during free-spelling 301

blocks and historical decoder age: T2: P = 0.33; T6: P = 0.13; Spearman’s rank correlation test). The historical decoder spelling 302

performance was not significantly different from the same-day decoder performance (mean spelling performance difference 303

during copy-phrase blocks over all comparison session: T2: 0.19±0.18CCPM, P = 0.10; T6: 0.03±0.22CCPM, P = 0.85; t-test). 304

In addition, spelling performance of both the historical and same-day decoders during copy-phrase blocks did not change 305

significantly over the time we conducted the comparison sessions (P value for testing non-zero correlation between spelling 306

performances during copy-phrase blocks and historical decoder age: same-day decoder: T2: P = 0.68; T6: P = 0.95; historical 307

decoder: T2: P = 0.18; T6: P = 0.48; Spearman’s rank correlation test). Further, CYX did not change significantly over the same 308

time (P value for testing non-zero correlation between CYX during copy-phrase blocks and historical decoder age: same-day 309

decoder: T2: P = 0.78; T6: P = 0.67; historical decoder: T2: P = 0.42; T6: P = 0.76; Spearman’s rank correlation test), thus 310

further confirming that LFP responses remained stable over the tested period. 311

The participants self-assessed the performance of both historical and same-day decoders on a scale from 1 to 10 (best) after each 312

copy-phrase block (Figure 10). Scores were the same for both decoders for all participants. On average, T2 scored the historical 313

decoder with 9.50±0.15 and the same-day decoder with 9.34±0.37, T6 scored the historical decoder with 9.23±0.43 and the 314

same-day decoder with 8.94±0.47. The score differences in all participants were not significant (T2: P = 1; Wilcoxon signed 315

rank test; T6: P = 0.47; Wilcoxon ranksum test). The score in one of the blocks in session T6 473 was not recorded. 316


Contribution of intracortical resolution 317

We compared decoding performance of the average signal decoder, which used the average LFP across the array and 318

represented a simulated single electrocorticographic electrode signal, to the historical decoder. For both participants, the 319

historical decoder was more effective (P < 0.05, bootstrap test; Figure 11). 320

DISCUSSION 321

Our results demonstrate stable, long-term LFP-based communication with an unchanged decoder in two people with severe 322

paralysis. The unchanged decoder was effective for as long as 138 days. Our participants included one person with long-323

standing locked-in syndrome (T2), who was unable to move or verbally communicate, and one person with progressive 324

tetraplegia secondary to ALS (T6). Importantly, over the duration of the study, spelling performance did not change 325

significantly. Both participants used the BCI to communicate with family members, caretakers and researchers by typing full 326

sentences. T2 also used the BCI to write and send emails - the participant first selected the email recipient and title using a 327

caretaker-assisted spelling board, then wrote the email body using FlashSpeller. When writing emails a “send” button was 328

present at the end of each FlashSpeller cycle, which could be selected by T2. T2 requested the use of a send button, because he 329

enjoyed using the FlashSpeller BCI system for personal email communication. T6 used other electronic communication systems 330

for email, and preferred to use the FlashSpeller to communicate messages to the research community. Decoders calibrated on 331

the same day did not achieve higher performance when compared to the unchanged historical decoder in double-blind 332

comparisons. During these comparisons, participants scored the performance of historical and same-day decoders equally well. 333

Together, these results show that an LFP-based communication BCI could soon provide people with locked-in syndrome with a 334

slow but reliable ability to communicate with minimum technical supervision or caretaker intervention. This LFP-based 335

decoding could also be used to enhance higher-throughput, intracortical spike-based communication interfaces (Bacher et al. 336

2014; Gilja et al. 2015; Jarosiewicz et al. 2015; Pandarinath et al. 2017). 337

While severely paralyzed, our participants were still able to move their eyes and head and neck muscles. Since eye movements 338

and head and neck muscle activity were not controlled for, artifacts originating from eye, head and neck movements could have 339

been introduced into our recordings. Nonetheless, such artifacts were likely to be common across signals recorded from a small 340

patch of motor cortex covered by one MEA. Therefore, we minimized their influence by using common-average re-referencing 341

step in our signal processing. 342

Stability of LFP signals 343

We show that LFPs recorded from a chronically implanted microelectrode array can serve as the basis for stable long-term 344

control of a switch-based communication interface. A major concern with chronic multielectrode recordings has been their 345

suitability for long-term prosthetic control. It is difficult to reliably detect and isolate activity of single units within a day or from 346

one day to the next (Dickey et al. 2009; Perge et al. 2013). Furthermore, changes to the recording environment can lead to a 347

long-term decline in the number of detectable single units (Chestek et al. 2011; Flint et al. 2012b; Moran 2010). However, when 348

available, isolated single units appear to have a predictable long-term relationship to behaviour, suggesting a stable underlying 349

map (Chestek et al. 2007). As such, two signal types have been studied as potentially more stable than isolated single units: 350

multi-unit activity derived from threshold crossings of a high-frequency band (0.5-7.5kHz) signal (Chestek et al. 2011; Flint et 351

al. 2013), and LFPs (Flint et al. 2012b; Perge et al. 2014). At present, the evidence suggests that while threshold crossings are 352

more stable neuroprosthetic control signal than was expected a decade ago, LFPs are still more stable from day to day (Flint et 353

al. 2016) and may persist when threshold crossings are lost (Flint et al. 2012b; Simeral et al. 2011). Therefore, for studies of 354

long-term reliability of neuroprosthetic control, LFPs continue to be a signal of interest. 355

In our study, it was important to distinguish between contributions to stable communication performance from signal 356

processing, signal selection, and changes in participant control strategy. We found that LFP signals recorded over the whole 357

microelectrode array showed long-term stability only for participant T2, suggesting that T2's stable communication rates 358

resulted from signal processing. For participant T6, we found significant changes of LFP modulations over time across the entire 359

array (Figure 6b). This result shows that the underlying signals, i.e. the processed LFPs, could not account alone for T6's stable 360

communication rates. Either decoder design (that is, feature selection and feature weighting) or T6's control strategy might 361


account for this discrepancy. We next tested decoding performance on open-loop blocks, during which participants could not 362

adapt control strategies due to lack of feedback (Figure 7). We found that open-loop decoding performance did not degrade over 363

time for T2 or T6, leading us to conclude that signal selection and feature weights, rather than a change in control strategy, 364

accounted for T6's stable communication rates. Therefore, population-level approaches can continue to decode a stable 365

behaviour-to-neuroprosthetic-command map even when the fidelity of the signal has declined on average. 366

Selection of LFP features to achieve BCI control 367

One objective of this study was to develop an LFP feature selection method that readily generalized across participants. 368

Following the data-driven process to identify informative LFP frequency bands, we derived three features on each channel (288 369

in total). We considered the case where features in a given frequency band were separately informative but highly cross-370

correlated, while separately less informative features from other frequency bands might provide independent novel information. 371

One way to capture the diversity of information over all available features is to use all features for decoding. However, this 372

approach increases the risk of overfitting the decoder to the training data. Furthermore, we were constrained by the need to 373

perform all signal processing and decoding computations online, which made it necessary to perform feature selection. 374

Previous studies in decoding movement direction from motor cortical spiking activity and LFPs in monkeys and humans 375

showed that the marginal benefit of added features diminishes substantially in the 30-50 range (Bansal et al. 2012; Bansal et al. 376

2011; Gilja et al. 2015; Wessberg et al. 2000). Therefore, we made a conservative estimate that 50 features will still contain the 377

majority of movement attempt information. We scored the “informativeness” of each feature by its SNR. To capture diverse 378

information potentially available in different frequency bands, we first selected five features with the highest SNR from each 379

frequency band. We supplemented those 15 features with additional 35 features with the highest SNR irrespective of the 380

frequency band to select the total of 50 features. We further mitigated the risk of overfitting the decoder to the training data by 381

using regularization. 382

Utility of switch-based BCIs 383

Although the primary finding of this study relates to the stability of neurophysiological signals, the efficacy of simple switch-384

based communication BCIs is also worth underscoring. ALS at moderate and severe stages may result in cortical and/or other 385

changes that could disrupt BCI function, including attention deficits (Pinkhardt et al. 2008; Schreiber et al. 2005; Vieregge et al. 386

1999) and other cognitive impairments (Cipresso et al. 2012). Furthermore, people with ALS can experience problems with 387

vision due to poor conjugate gaze control and corneal injury (Murguialday et al. 2011). People with LIS secondary to brainstem 388

stroke can also develop oculomotor and visual impairments (Caplan 1980; Sellers et al. 2014). Lastly, people with 389

communication deficits related to neurological disease or injury (for instance, related to traumatic brain injury) may have a 390

range of sensory and cognitive abilities, but could still benefit from a reliable method of communication for basic needs. The 391

switch-based communication method described here could, for example, also be used with an auditory interface. For all these 392

reasons, simple switch-based communication BCIs may be better suited for some people with LIS, ALS, or other neurological 393

disorders, than more complex communication BCIs designed for high communication throughput, including those previously 394

reported by our group. Additionally, approaches and algorithms that can reliably decode state information from intracortical 395

recordings may point towards more expansive use of BCIs for non-typing communication goals such as control of household 396

technology and emergency alarms, for which reliability is paramount (Kageyama et al. 2014). 397

The development of a fully implantable amplification and telemetry system is a likely precursor to the widespread adoption of 398

BCI technologies based on intracortical recordings. Safety and efficacy of such devices are currently being studied in animal 399

models (Borton et al. 2013; Capogrosso et al. 2016; Yin et al. 2014); and translation for use in people is ongoing. Future work 400

should also aim to implement the signal normalization block as an ongoing, self-updating process. 401

Comparison to other communication BCI studies 402

Other approaches using different signal modalities have investigated communication BCIs (Chaudhary et al. 2016; Wolpaw and 403

Wolpaw 2012). Nevertheless, recent studies have demonstrated the superior performance of intracortical communication BCIs 404

in people with tetraplegia when compared to prior studies (Gilja et al. 2015; Jarosiewicz et al. 2015; Pandarinath et al. 2017). 405

However, for successful clinical applications, these systems have to provide stable performance with minimal technical 406


oversight, despite the instability of intracortical signal recordings (Jarosiewicz et al. 2015; Perge et al. 2014). An approach taken 407

by a recent study from our group used an adaptive decoder that conformed to the signal changes to generate stable performance, 408

which allowed the participant with tetraplegia to use a communication BCI without a loss in performance and without 409

reinitializing the decoder over a period of 42 days (Jarosiewicz et al. 2015). In the current study, we followed an alternate 410

approach of harnessing intracortical LFP signals, which are thought to be more stable than the neuronal spiking signals. Despite 411

changes in LFP signals recorded across the entire array in one participant, the online decoding performance remained stable for 412

both participants throughout the duration of the study, over a period of 76 days for one and 138 days for the other. 413

In a recent study, a person with LIS used a BCI based on ECoG signals to spell text at a rate of 1.82 CCPM over 70 days with an 414

unchanged decoder and without a drop in performance (Vansteensel et al. 2016). In our current study, a person with LIS used an 415

intracortical LFP-based BCI with an unchanged decoder and without a drop of performance over a similar period (76 days), but 416

achieved 69% higher performance (3.07 CCPM). A previous study from our group has shown that people with LIS can use a 417

communication BCI based on intracortical spiking signals, but used daily calibration of decoders to preserve performance 418

(Bacher et al. 2014). Another study from our group demonstrated stable spelling performance using an intracortical BCI in an 419

individual with tetraplegia who could still speak (Jarosiewicz et al. 2015). This study thus complements recent intracortical BCI 420

communication studies. While a previous study (Jarosiewicz et al. 2015) adapted the decoder based on data collected on the 421

current day, in this study the historical decoder was calibrated with data pooled across about a month. Therefore, our calibration 422

strategy inherently favored features that remained stable over longer periods. This strategy might also benefit spike-based 423

control. The adaptation strategy used in the previous study relies on reasonable neural control in a point-and-click task after a 424

days-long period of disuse, and would likely benefit from the methods developed here. LFP signals and the associated data 425

collection, decoder calibration, signal selection and signal processing strategies developed in our study could contribute not only 426

to point-and-click BCI communication but also to more general neuroprosthetic control. 427

Other studies have investigated the use of communication BCIs based on other signal modalities as a communication tool for 428

people with LIS (Bacher et al. 2014; Birbaumer et al. 1999; Kubler et al. 1999; Kubler et al. 2005; Nijboer et al. 2008; Sellers et 429

al. 2014). A prior study showed that two individuals with LIS secondary to ALS could communicate using an EEG-based BCI, 430

with one of the participants typing 510 characters for over 16 hours (Birbaumer et al. 1999). Another recent study has shown 431

that an individual with LIS secondary to brainstem stroke (state similar to that of T2) can communicate using an EEG-based 432

BCI with a performance of below 1 CCPM (Sellers et al. 2014). In our study, all participants, including a person with LIS, typed 433

at a higher rate. 434

Our system provided higher performance than reported by any EEG- or ECoG-based spellers tested on a similar population of 435

subjects, i.e. people with Amyotrophic Lateral Sclerosis and people with Locked-in Syndrome (Arya et al. 2013; Bai et al. 2010; 436

Brunner et al. 2010; Chen et al. 2015; Fazel-Rezai et al. 2012; Hamer et al. 2002; Hedegard et al. 2014; Hill et al. 2006; 437

Hinterberger et al. 2005; Kubler and Birbaumer 2008; Mainsah et al. 2015; Marchetti and Priftis 2014; McCane et al. 2014; 438

Miner et al. 1998; Nijboer et al. 2008; Pires et al. 2012; Vansteensel et al. 2016) (see Table 3 for detailed list of spelling rates 439

reported in other studies). In the two above studies in which the communication rates were similar (McCane et al. 2014; Pires et 440

al. 2012), our BCI communication system was used to type considerably longer and more complex messages, demonstrating that 441

people with paralysis can use our BCI system for spelling, and that intracortical recording may be an effective solution. 442

Two recent studies showed that near infrared spectroscopy (NIRS) can be used to detect recognition of the veracity of a Boolean 443

choice in people with complete LIS (Chaudhary et al. 2017; Gallegos-Ayala et al. 2014), which suggests that some people with 444

complete LIS might remain capable of communication with adequate technology. However, the rate of yes/no detections was 445

slow (around 20-25s per response, responses about 75% correct), possibly reflecting the low signal-to-noise ratio of NIRS 446

responses. Other attempts to provide individuals with complete LIS secondary to ALS with the ability to communicate using 447

electrocorticography-based BCIs were not yet successful (Hill et al. 2006; Marchetti and Priftis 2014). It remains to be seen 448

whether a LFP-based communication BCI could be used by someone with complete LIS, but our study with participants who 449

have ALS or classic LIS suggests that this approach could be effective. 450


ACKNOWLEDGMENTS 451

We thank participants T2 and T6 for their dedication to this research. We also thank L. Barefoot, D. Rosler, S. Mernoff and M. 452

Bowker for their contributions to this research. The research was supported by the National Institutes of Health: National 453

Institute on Deafness and Other Communication Disorders - NIDCD (R01DC009899); Rehabilitation Research and 454

Development Service, Department of Veterans Affairs (B6453R and B6459L); Massachusetts General Hospital (MGH) - Deane 455

Institute for Integrated Research on Atrial Fibrillation and Stroke; Joseph Martin Prize for Basic Research; The Executive 456

Committee on Research (ECOR) of Massachusetts General Hospital; Doris Duke Charitable Foundation and the Swiss National 457

Science Foundation Ambizione program (PZOOP2_168103/1). The content of this paper is solely the responsibility of the 458

authors and does not necessarily represent the official views of the National Institutes of Health, the Department of Veterans 459

Affairs or the United States Government. 460

AUTHOR CONTRIBUTIONS 461

TM designed the experiment with input from AAS, DB, JDS, JPD and LRH. TM and AAS performed pilot analyses to develop 462

the historical decoder approach. TM conceived and implemented the decoder calibration method, with input from AAS. AAS 463

implemented the adaptive feature normalization method, with input from TM. DB designed and implemented the 464

communication interface. TM and AAS performed the data analyses and drafted the manuscript, which was further edited by all 465

authors. SSC is clinical co-investigator of the pilot clinical trial and assisted in the clinical oversight of the participants. EE and 466

JH planned and executed the electrode array implants and supported the clinical research components of the study. AAS, JDS, 467

JS and DB made key contributions to the BrainGate software and hardware infrastructure supporting these research sessions. 468

EG, KT, BLS, CB, CP, KVS, and JMH contributed to data collection. JPD and LRH conceived, planned and continue to direct 469

the ongoing BrainGate research. LRH is IDE sponsor-investigator of the BrainGate2 pilot clinical trial. 470

471


FIGURE CAPTIONS 472

Fig. 1. LFP-based communication brain-computer interface enables a person with a locked-in syndrome (T2) and a person with 473

tetraplegia and ALS (T6) to write messages over long time periods without decoder recalibration and with minimal technical 474

intervention. Top. The BCI used a historical decoder that was calibrated to data recorded over a period of 23 and 42 days (blue) 475

and then remained unchanged during the use of the BCI for communication for 76 and 138 days afterwards (red) for T2 and T6, 476

respectively. Middle. The participants sat in front of a computer monitor that displayed a BCI-controlled speller FlashSpeller. 477

The participants selected the options by attempting or executing selected movements, which were accompanied by LFP 478

responses decoded as selections. The grey arrow traces the processing of the intracortical neural signals. Raw signals from each 479

of 96 electrodes (first box) were processed into low-pass filtered, intermediate frequency and high frequency component 480

features (second box). Out of these 288 features, we selected 50 (third box). These features were analyzed by the decoder that 481

outputted a “click” probability (fourth box). If the probability crossed the threshold, the option was selected (fifth box). If no 482

selections were detected during the option presentation time (1.5s-2s), the FlashSpeller presented the next option in the cue. 483

Bottom. The FlashSpeller is a two tier speller. The first tier is composed of groups of characters, space and backspace options. 484

Once a group of characters is selected, FlashSpeller enters into the second tier composed of individual characters. Selecting a 485

character adds it to the entered text and restarts the first tier cycle. If all second tier options have been passed, the FlashSpeller 486

continues the first tier cycle. If all first tier options have been passed, the first tier cycle restarts from the backspace option. 487

Additional “word completion” option was added during free-spelling, which allowed completion of a partially spelled word 488

(middle; purple panel). 489

Fig. 2. Schemes of the sessions used in our study. Top left panel shows the scheme of the data collection sessions. These started 490

with a normalization block and proceeded with two to four open-loop calibration blocks. Recordings from five consecutive data 491

collection sessions were used to calibrate the historical decoder, which remained unchanged throughout all subsequent sessions. 492

Bottom panel shows the scheme for the comparison sessions. These started with a normalization block and three open-loop 493

calibration blocks. Same-day decoder was calibrated to the three open-loop calibration blocks and the historical decoder was 494

loaded from file. The two decoders were then blind randomized and used in the 4 pairs of copy phrase blocks, each decoder 495

once in each pair. The historical decoder was used for typing in the free-spelling blocks that followed. Top right panel shows the 496

scheme of the Communication session in which a normalization block was followed by free-spelling blocks in which the 497

historical decoder was used for typing. Comparison and communication sessions ended when the participant did not express his 498

intention to start a new block. 499

Fig. 3. SNR for five collection sessions for both participants, averaged over all channels, which have been used to determine 500

intermediate and high frequency component (IFC and HFC) frequency bands. SNR has been normalized to the maximum value 501

for each session. The black circles mark the frequency band that gave the highest SNR in the intermediate and high frequency 502

ranges. 503

Fig. 4. Example of selection probabilities and selected options throughout the first 316s of the first free-spelling block, session 504

T6 day 493. During this time, T6 spelled the phrase, “Free speech is one of the most important rights”. Top to bottom. 505

Successive panels show increasing time resolution of a region outlined in red. Three top rows in each panel display the periods 506

during which the FlashSpeller button was: green after selecting an option (Feedback), presenting an option (Option), and empty 507

during transition between options (Transition). Fourth row shows the click probability with a probability threshold. Rows five 508

through seven show three neural features selected for decoding, one for each component (LFC – low-pass filtered component; 509

IFC – intermediate frequency component; HFC – high frequency component). The last row shows all fifty neural features 510

selected for decoding. 511

Fig. 5. Examples of T2’s neuronal responses to attempts to squeeze his right fist recorded on a single electrode (el. 95) during 512

three open-loop calibration blocks for three sessions 48 or more days apart. Panels show average spectrogram (top row of 513

panels) and single trials of low-pass filtered (second row), intermediate frequency (third row) and high frequency (fourth row) 514

component LFP features. Single trials are shown in units of standard deviation (s.d.). For better visualization, the colorbar scale 515

spans the values from 0.5th to 99.5th percentile. 516


Fig. 6. Long term stability of participants’ LFP responses during click actions. (A) Mean (thick lines) and 95% confidence 517

intervals (thin lines) of single electrode low-pass filtered component (LFC; top panels), intermediate frequency component (IFC; 518

middle panels) and high frequency component (HFC; bottom panels) LFP features calculated over all trials recorded in open-519

loop calibration blocks are shown for three selected sessions at least one month apart (blue, red and black traces in temporal 520

order). Left and right panels show signals for participant T2, electrode 95; and participant T6, electrode 80, respectively. 521

Average spectrograms and single trial LFP responses for these sessions are shown in Figure 5. s.d. – standard deviation; el. – 522

electrode. (B) Black dots show the median of single-feature cross-session LFP response correlation coefficients (rf(s1,s2)) over 523

all channels for all compared session pairs s1 and s2 against the inter-session period. Grey dots show the mean Mr(s1,s2) when 524

the LFP responses are triggered on random events. Grey lines show the linear ordinary least squares fit to Mr(s1,s2) with the 525

slope b shown at the bottom of the panel. 526

Fig. 7. Demonstration of the detection stability based on stability of LFPs. Top panels show the CYX calculated by calibrating a 527

decoder on all open-loop calibration blocks from one session and validating it on all open-loop calibration blocks of another 528

sessions. The same amount of data was used to calibrate and validate each decoder. Bottom panels show the same CYX values as 529

a function of the decoder age ΔT (time between the session used to calibrate the decoder and the session used to validate it). 530

Black lines show the linear ordinary least squares fit to the CYX values. P value of testing for significant correlation between ΔT 531

and CYX is given. 532

Fig. 8. Spelling performance of participants when using LFP-based FlashSpeller communication BCI. Dots on the left show 533

participants’ spelling performance, as measured by correct characters per minute, as a function of decoder age when using same-534

day and historical decoders during copy-phrase blocks (yellow and blue dots, respectively) and when using historical decoder 535

during free spelling with the word completion option (purple dots). Broken lines show the linear ordinary least squares fit over 536

all sessions with P values for testing significant correlation between the spelling performance and the age of the historical 537

decoder shown below the lines. Bars on the right side show the mean spelling performance over all sessions. Error bars show 538

95% confidence intervals. ns – non significant difference (P ≥ 0.05); * - P < 0.05. 539

Fig. 9. Performance of participants’ interaction with the FlashSpeller application as measured by the normalized mutual 540

information (CYX). Left panels show CYX when using same-day and historical decoders during copy-phrase blocks (yellow and 541

blue bars, respectively) and when using historical decoder during free spelling with and without the word completion option 542

(full and empty purple bars). Maximum theoretical performance, described as participant always selecting the intended spelling 543

option or character, is 1. Right panels show the CYX as a function of time (colored dots) and its linear ordinary least squares fit 544

(broken lines). Top right corner displays p values for testing significant correlation between the CYX and time. Performance is 545

shown as mean CYX. Error bars show 95% confidence intervals. Error bars show 95% confidence intervals. ns – non significant 546

difference (P ≥ 0.05); * - P < 0.05. 547

Fig. 10. Participants’ self-assessed quantitative scores of historical and same-day decoders during copy-phrase blocks. Each 548

colored dot shows the participant’s score for a single copy-phrase block. Score was given on a scale from 1 to 10 (best). Bars on 549

the right show the mean score over all scored blocks and error bars show the 95% confidence intervals. ns – non significant 550

difference (P ≥ 0.05). 551

Fig. 11. Hypothetical performance of an average signal decoder compared to the performance of the historical decoder. We 552

simulated the case where the intracortical microelectrode array was replaced with a single electrocorticographic electrode by 553

averaging the recorded signal over all electrodes. (A) As with the historical decoder, we identified IFC and HFC frequency 554

bands for each of the participants from the peri-click SNR in five collection sessions. Note that, to construct the simulated single 555

electrocorticographic electrode signal, we averaged the signals over all low-noise microelectrode array channels prior to 556

computing the SNR. For T6, the HFC frequency band did not emerge in any of the sessions. Therefore, we used only LFC and 557

IFC features to calibrate and use the average signal decoder for T6. (B) Bar plots show decoding accuracy, as measured by CYX, 558

of the historical and average signal decoder for each participant on individual session days and the mean across all sessions 559

(overall). For all participants, the historical decoder outperformed the simulated average signal decoder (p<.05, bootstrap test). 560

The same amount of data was used to calibrate and validate both decoders. 561

562


TABLES AND TABLE CAPTIONS 563

564

Sessions

Intermediate frequency SNR local maximum

High frequency SNR local maximum

First bin Last bin First bin Last bin

T2

452 5 (14Hz) 9 (33Hz) 14 (49Hz) 42 (162Hz)

453 5 (14Hz) 9 (33Hz) 16 (57Hz) 60 (232Hz)

460 5 (14Hz) 8 (29Hz) 14 (49Hz) 45 (174Hz)

467 5 (14Hz) 9 (33Hz) 15 (53Hz) 41 (158Hz)

474 6 (18Hz) 8 (29Hz) 14 (49Hz) 61 (236Hz)

Average 5 (14Hz) 9 (33Hz) 15 (53Hz) 50 (193Hz)

T6

313 6 (18Hz) 9 (33Hz) 16 (57Hz) 55 (213Hz)

322 7 (21Hz) 9 (33Hz) 18 (64Hz) 90 (350Hz)

336 7 (21Hz) 14 (53Hz) 18 (64Hz) 81 (314Hz)

348 7 (21Hz) 9 (33Hz) 17 (61Hz) 86 (334Hz)

355 7 (21Hz) 13 (49Hz) 17 (61Hz) 53 (205Hz)

Average 7 (21Hz) 11 (41Hz) 17 (61Hz) 73 (283Hz)

565

Table 1. SNR local maxima for five collection sessions used to determine the intermediate and high frequency component (IFC 566

and HFC) frequency bands (average) for each participant. Numbers in parentheses show the rounded lower edge frequency of 567

the first frequency bin or the rounded top edge frequency of the last frequency bin. 568

569

Session Phrase Time (min) Number of correct

characters

T2 510 Thankyou for your time friday. We had a g ood time. ****. 21.8 56

T2 524 A tale told by an idiot, full of sound and fury, signifying nothing 15.0 67

T2 529 Would it help if i came saturday in smaller chair? 13.3 50

T6 403 Embracing als is an important step towards living with als. 8.5 59

T6 441 I want to thank all my caregivers who made the trip to hawaii possible 11.0 70

T6 493 Free speech is one of the most important rights we have whether you are able or

disabled, and w

11.9 95

Where technolgy is a 3.8 20

Available, there should be a constitutional mandate to make available such techn 11.9 80

Technology to people in need free of charge. 6.2 43

570

Table 2. Examples of sentences typed by participants T2 and T6 using the historical decoder during the free-spelling blocks. In 571

the phrase from session T2 510, four characters that contained personal information have been redacted (*). In session T6 493, 572

the participant typed a sentence over four free-spelling blocks. Typing was interrupted either due to the block maximum length 573

of 12 minutes or by the participant. 574

575


Able-bodied participants Participants with ALS without LIS Participants with LIS

Speller type Switch P300 / SSVEP Switch P300 / SSVEP Switch P300 / SSVEP

Scalp EEG 0-0.86* (Bai et al. 2010) 2.6 (McCane et al. 2015);

5.48 (Pires et al. 2012);

5.83 (Chen et al. 2015)

0 (Hill et al. 2006);

1.2* (Miner et al. 1998)

<2.1 (Nijboer et al. 2008);

4.97 (Pires et al. 2012);

1.16 (Mainsah et al. 2015)

0.5 (Birbaumer et al. 1999);

<2 (Kubler et al. 2001)

2.1 (McCane et al. 2014);

<1 (Sellers et al. 2014)

ECoG 0.51 (Hinterberger et al. 2005) 17 (Brunner et al. 2011) 0 (Hill et al. 2006);

1.82 (Vansteensel et al. 2016)

0 (Murguialday et al. 2011)

Intracortical

LFP-based

(this study)

6.88 3.07

Intracortical

spike-based

using other

speller types

10.44-17.40 (Jarosiewicz et al. 2015);

13.5-31.6 (Pandarinath et al. 2017)

9.4 (Bacher et al. 2014);

1.95-11.75 (Jarosiewicz et al. 2015)

Table 3. Comparison of spelling performance between our study (red) and other spelling BCI studies, including all studies listed in three recent reviews (Fazel-576

Rezai et al. 2012; Kubler and Birbaumer 2008; Marchetti and Priftis 2014). Performance of BCI-based spellers in able-bodied people, people with ALS who 577

retain the ability to speak, and people with LIS resulting from ALS or stroke, as measured in correct characters per minute. When only selections per minute were 578

reported, we include this value as an upper bound. When only binary selections were allowed we estimated a typing rate based on a simple switch keyboard 579

design, with five selections required per character (marked with *s). 580

581


APPENDIX 582

Adaptive Feature Normalization 583

To account for the drifts of the neuronal features, Φf, we used a recursive algorithm to estimate the their mean, μf , and variance, 584

σf, using a time constant τ of 240 seconds (Bruce 1969). 585

f f f

tt t t t

586

22

f f f f

tt t t t t

587

where Δt was the temporal resolution of our online data processing set to 20ms. Estimated mean of each feature was subtracted 588

from the current feature value and the result was divided by the estimated standard deviation, thereby normalizing all features to 589

zero-mean and unit-variance. To prevent division by zero, a value of 10-6 was added to the variance estimate. 590

6ˆ

10

f f

f

f

t tt

t

591

SNR used for selection of neural features 592

To reduce the time needed for algorithm calibration and to reduce the effect of excessive dimensionality (Hastie et al. 2009), we 593

selected a subset of neural features that were used for decoder calibration. Selection was done by ranking the features according 594

to the SNR between peri-action and baseline feature values (see Milekovic et al. 2012 for similar use of SNR): 595

ˆ

ˆ

ˆ

ˆ

SQUEEZE SQUEEZE

SQUEEZE SQUEEZE

BASELINE BASELINE

BASELINE BASELINE

i

i

t E t tc

t std t tc

E t

std t

596

SQUEEZE BASELINE

SQUEEZE BASELINE

t tSNR t

t t

597

.

max max max arg ,SQUEEZE

VSBASELINE

S Et

SNR SNR t t SNR t t t t 598

Where tci is the time of the trial i, E is the expectation operator and std is the standard deviation operator, both acting over trials, 599

tS (300ms) and tE (1.8s for T2 and 1.5s for T6) are the start and the end of the investigated epoch. 600

Calibration of decoding algorithms 601

The rLDA classifier was calibrated in a following way. First, we calibrated a set of classifiers that captured the neuronal features 602

specific to the participant’s selection actions, recorded in the selected set of open-loop calibration blocks. In these blocks, the 603

participant was cued to perform selection click action at times tci (see Task section for details). Given a set of normalized 604

features (Φf1, …, Φfm), the peri-selection feature vector Fi was defined as: 605


1 1

1

2 1

2

ˆ

ˆ

ˆ

ˆ

ˆ

f i

f i n

f i

i

f i n

fm i n

tc t

tc t

tc tF

tc t

tc t

M

M

M

606

where t1, ..., tn are the selected time points relative to the time of the cue. Therefore, the feature vector contained n∙m features. 607

The time points t1, ..., tn were always equidistant and defined by a set of parameters: (i) the time of the first feature in relation to 608

the time of the cue, t1, (ii) the number of time points, n and (iii) the temporal distance between the first and the last feature, tn-t1. 609

Each classifier was built using two classes of feature vectors: the “select” class, which contained peri-select feature vectors 610

(Sclass), and the baseline class, which contained feature vectors between selections (Bclass). 611

1 1

1

2 1

2

, 1,...,

ˆ

ˆ

ˆ,

, | 0.5

ˆ

ˆ

class i

f

f n

f

class

i

f n

fm n

S F i N

k t t

k t t

kk t tB

i k tc k t s

k t t

k t t

M

¥

M

M

612

where δt is the time resolution of baseline feature vectors, set to 80ms; and N is the number of selection cues present in the data 613

used to train the classifier. 614

These classes were used to build a regularized linear discriminant analysis (rLDA) classifier (Friedman 1989). Regularization 615

was implemented in a same way as in Milekovic et al. (Milekovic et al. 2013). The probability of every feature vector F(t) in the 616

test dataset to belong to the “select” class was then calculated using the classifier. 617

1 1 1 2 1ˆ ˆ ˆ ˆ,..., , ,...,f f n f fm nF t t t t t t t t t 618

Due to the internal latency of the decoder, feature vector F(t) could only lead to a selection detection at time t+tn. Based on the 619

probability of the selection, refractory period and the detection hysteresis, we assigned “detected selection” (ds) labels to all 620

times in which the selection was detected and assigned “detected as non-selection” (dns) labels to all remaining time points. 621

To validate each of the classifiers, we divided the set of blocks used to calibrate the classifiers into two equally long subsets, one 622

containing the first halves and another containing the second halves of each block. First, we chose a set of parameter values 623

consisting of: (i) a time of the first feature in relation to the selection cue, t1, (ii) a number of features, n and (iii) a time distance 624

between the first and the last feature, tn-t1, and (iv) regularization coefficient, γ. An rLDA classifier was then calibrated to the 625

first part of the dataset and used to detect the events on the second part of the dataset and vice-versa. Detection performance was 626

estimated by calculating the normalized mutual information CYX (see section “Performance measures” for details) for both 627

combinations and then averaging between the two. Values of the parameters were then changed and the process was repeated, 628

until all parameter values from the parameter grid were tested. The parameter values that were used to calibrate the decoder that 629

gave the maximum CYX were then used to calibrate a new decoder to the whole dataset. This decoder was then used to detect 630


selections during the copy-phrase and free-spelling blocks of the comparison and communication sessions (online) or to detect 631

selections in the open-loop calibration blocks of all completed sessions in a posterior analysis (offline). 632

Performing the selection of parameter values that give the highest performance is a computationally demanding process. The 633

historical decoder used during the participants’ comparison and communication sessions and all decoders used for offline 634

analysis were calibrated using a parameter grid with numerous values. This was possible since there were no restrictions on the 635

amount of time needed for calibration. However, same-day decoder used during comparison sessions had to be calibrated during 636

the session itself, within several minutes. Therefore, instead of selecting the parameter values from a grid of values, online 637

same-day decoders were calibrated using the parameter values selected through the calibration of the historical decoder. 638

Measuring cross-session stability of LFP responses 639

rf(s1,s2) was calculated as follows: 640

1 2

1 2

1 11 2

1 1ˆ ˆ1, 2 , ,s sN N

s s

f f f

n n

r s s CC t tN N

641

where CC is the correlation coefficient operator calculated over time t running from 0 to 1.5 seconds after the cue, and Ns1, Ns2, 642

and are the numbers of responses and the features for sessions s1 and s2, respectively. rf(s1,s2) was calculated for 643

all 288 possible features. We used only the recordings from the open-loop calibration blocks for the calculation. Since the 644

calibration was standardized for each participant, the same amount of data from each session - three equally long OLC blocks 645

containing 17 cues each – was used to calculate CC. 646

To estimate the Mr(s1,s2) values for no evoked LFP responses, we randomly drew as many “random” cues from a given block 647

as there were active cues. The draws were made from a uniform distribution. These random cues were then used to calculate 648

Mr(s1,s2). This procedure was repeated 1000 times to calculate the mean Mr(s1,s2) for random cues, termed RMr(s1,s2). 649

Measuring spelling and decoding performance 650

Accuracy of a decoding algorithm is often described by its positive and negative predictive values (Fletcher et al. 2014). 651

However, using these measures to obtain a single metric that can be used to optimize a decoder requires establishing an arbitrary 652

tradeoff between these two values. Another way to measure decoding performance, which takes both positive and negative 653

predictive values into account, is to use a metric based on information theory. We used the normalized mutual information CYX 654

(Coombs et al. 1970), which was calculated using the assumed intended actions of the participant, “real selection” (rs) and “real 655

non-selection” (rns), and the actions detected by BrainGate system, “detected selection” (ds) and “detected non-selection” (dns). 656

We first divided the corresponding blocks into epochs of the same length as the presentation time that we labeled as “rs” and 657

“rns”. During the open-loop calibration blocks, we assumed that the participant always responded to the cues. Therefore, every 658

epoch after the selection cue was labeled as “rs” and all other epochs were labeled as “rns”. According to the selection 659

detections, we labeled each epoch with one of the four labels: true positive – (rs,ds); false positive – (rns,ds); false negative - 660

(rs,dns); and true negative – (rns,dns). We calculated CYX by computing the mutual information between detections and real 661

events, divided by the entropy of real events, as described previously (Milekovic et al. 2013). 662

The spelling performance of our text-entry interface was calculated by counting the total number of correctly typed characters 663

(NCC) and dividing it by the time that participant spent from the start of the spelling until s/he finished the phrase (value given in 664

correct characters per minute, CCPM). To calculate the confidence intervals of the measure, we split the spelling period into 665

epochs that the participant spent to type every single character, τi(char). Occasionally, even though the participants could erase 666

the incorrectly typed character, they would continue spelling. For the purpose of calculating spelling performance, this character 667

was not counted as having been typed, and the time taken to type it was added to the time it took to type the next character. Due 668

to the design of the text-entry interface, there is a minimum time necessary to reach each of the characters, τMIN(char). Note that 669

τMIN(char) can be different for different characters. Once the character is reached, the time it takes for the participant to type it, 670

κi, is independent of the character, even if the character has been missed. The spelling performance (SP) can then be written as: 671

1 ˆs

f 2 ˆs

f


1 1 1

CC CC CC

CC CC CC

N N N

i MIN i MIN i

i i i

N N NSP

char char phrase

672

We estimated the standard deviation of the spelling performance calculated from a given dataset by bootstrapping with 100,000 673

re-samples (Moore et al. 2009). We assumed the normal distribution of the bootstrapped spelling performance values and used 674

the t-distribution to calculate the 95% confidence intervals (Bohm and Zech 2010). During the free-typing blocks, by instantly 675

completing the word through word completion, the participant was able to select several characters at once. To calculate the 676

confidence intervals in these blocks, we bootstrapped the pairs, consisting of the number of the characters spelled in this 677

particular step nCCi and the corresponding κi. When comparing the performance of two different detection algorithms, we again 678

used bootstrapping with 100,000 re-samples on a SPA - SPB statistic and used the t-distribution to calculate the 95% confidence 679

intervals. 680

681


REFERENCES 682

Aflalo T, Kellis S, Klaes C, Lee B, Shi Y, Pejsa K, Shanfield K, Hayes-Jackson S, Aisen M, Heck C, Liu C, and Andersen 683

RA. Neurophysiology. Decoding motor imagery from the posterior parietal cortex of a tetraplegic human. Science 348: 906-910, 684

2015. 685

Ajiboye AB, Simeral JD, Donoghue JP, Hochberg LR, and Kirsch RF. Prediction of imagined single-joint movements in a 686

person with high-level tetraplegia. IEEE Trans Biomed Eng 59: 2755-2765, 2012. 687

Ajiboye AB, Willett FR, Young DR, Memberg WD, Murphy BA, Miller JP, Walter BL, Sweet JA, Hoyen HA, Keith 688 MW, Peckham PH, Simeral JD, Donoghue JP, Hochberg LR, and Kirsch RF. Restoration of reaching and grasping 689

movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration. The 690

Lancet 2017. 691

Arya R, Mangano FT, Horn PS, Holland KD, Rose DF, and Glauser TA. Adverse events related to extraoperative invasive 692

EEG monitoring with subdural grid electrodes: a systematic review and meta-analysis. Epilepsia 54: 828-839, 2013. 693

Bach JR. Amyotrophic lateral sclerosis. Communication status and survival with ventilatory support. American journal of 694

physical medicine & rehabilitation / Association of Academic Physiatrists 72: 343-349, 1993. 695

Bacher D, Jarosiewicz B, Masse NY, Stavisky SD, Simeral JD, Newell K, Oakley EM, Cash SS, Friehs G, and Hochberg 696 LR. Neural Point-and-Click Communication by a Person With Incomplete Locked-In Syndrome. Neurorehabil Neural Repair 697

2014. 698

Bai O, Lin P, Huang D, Fei DY, and Floeter MK. Towards a user-friendly brain-computer interface: initial tests in ALS and 699

PLS patients. Clin Neurophysiol 121: 1293-1303, 2010. 700

Bansal AK, Truccolo W, Vargas-Irwin CE, and Donoghue JP. Decoding 3D reach and grasp from hybrid signals in motor 701

and premotor cortices: spikes, multiunit activity, and local field potentials. J Neurophysiol 107: 1337-1355, 2012. 702

Bansal AK, Vargas-Irwin CE, Truccolo W, and Donoghue JP. Relationships among low-frequency local field potentials, 703

spiking activity, and three-dimensional reach and grasp kinematics in primary motor and ventral premotor cortices. J 704

Neurophysiol 105: 1603-1619, 2011. 705

Barrese JC, Rao N, Paroo K, Triebwasser C, Vargas-Irwin C, Franquemont L, and Donoghue JP. Failure mode analysis 706

of silicon-based intracortical microelectrode arrays in non-human primates. J Neural Eng 10: 066014, 2013. 707

Bauer G, Gerstenbrand F, and Rumpl E. Varieties of the locked-in syndrome. Journal of neurology 221: 77-91, 1979. 708

Birbaumer N, Ghanayim N, Hinterberger T, Iversen I, Kotchoubey B, Kubler A, Perelmouter J, Taub E, and Flor H. A 709

spelling device for the paralysed. Nature 398: 297-298, 1999. 710

Bjornsson CS, Oh SJ, Al-Kofahi YA, Lim YJ, Smith KL, Turner JN, De S, Roysam B, Shain W, and Kim SJ. Effects of 711

insertion conditions on tissue strain and vascular damage during neuroprosthetic device insertion. J Neural Eng 3: 196-207, 712

2006. 713

Bohm G, and Zech G. Introduction to statistics and data analysis for physicists. Verlag Deutsches Elektronen-Synchrotron, 714

2010. 715

Borton DA, Yin M, Aceros J, and Nurmikko A. An implantable wireless neural interface for recording cortical circuit 716

dynamics in moving primates. J Neural Eng 10: 026010, 2013. 717

Bouton CE, Shaikhouni A, Annetta NV, Bockbrader MA, Friedenberg DA, Nielson DM, Sharma G, Sederberg PB, 718

Glenn BC, Mysiw WJ, Morgan AG, Deogaonkar M, and Rezai AR. Restoring cortical control of functional movement in a 719

human with quadriplegia. Nature 533: 247-250, 2016. 720

Bruce MM. Estimation of variance by a recursive equation National Aeronautics and Space Administration, 1969. 721

Brunner P, Joshi S, Briskin S, Wolpaw JR, Bischof H, and Schalk G. Does the 'P300' speller depend on eye gaze? J Neural 722

Eng 7: 056013, 2010. 723

Brunner P, Ritaccio AL, Emrich JF, Bischof H, and Schalk G. Rapid Communication with a "P300" Matrix Speller Using 724

Electrocorticographic Signals (ECoG). Front Neurosci 5: 5, 2011. 725

Buzsaki G, Anastassiou CA, and Koch C. The origin of extracellular fields and currents--EEG, ECoG, LFP and spikes. Nat 726

Rev Neurosci 13: 407-420, 2012. 727

Caplan LR. "Top of the basilar" syndrome. Neurology 30: 72-79, 1980. 728

Capogrosso M, Milekovic T, Borton D, Wagner F, Moraud EM, Mignardot J-B, Buse N, Gandar J, Barraud Q, Xing D, 729

Rey E, Duis S, Jianzhong Y, Ko WKD, Li Q, Detemple P, Denison T, Micera S, Bezard E, Bloch J, and Courtine G. A 730

brain–spine interface alleviating gait deficits after spinal cord injury in primates. Nature 539: 284-288, 2016. 731

Chaudhary U, Birbaumer N, and Ramos-Murguialday A. Brain-computer interfaces for communication and rehabilitation. 732

Nature Reviews Neurology 12: 513-525, 2016. 733

Chaudhary U, Xia B, Silvoni S, Cohen LG, and Birbaumer N. Brain-Computer Interface-Based Communication in the 734

Completely Locked-In State. Plos Biology 15: 2017. 735

Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, and Gao S. High-speed spelling with a noninvasive brain-computer 736

interface. Proc Natl Acad Sci U S A 112: E6058-6067, 2015. 737


Chestek CA, Batista AP, Santhanam G, Yu BM, Afshar A, Cunningham JP, Gilja V, Ryu SI, Churchland MM, and 738

Shenoy KV. Single-neuron stability during repeated reaching in macaque premotor cortex. J Neurosci 27: 10742-10750, 2007. 739

Chestek CA, Gilja V, Nuyujukian P, Foster JD, Fan JM, Kaufman MT, Churchland MM, Rivera-Alvidrez Z, 740

Cunningham JP, Ryu SI, and Shenoy KV. Long-term stability of neural prosthetic control signals from silicon cortical arrays 741

in rhesus macaque motor cortex. J Neural Eng 8: 045005, 2011. 742

Cipresso P, Carelli L, Solca F, Meazzi D, Meriggi P, Poletti B, Lule D, Ludolph AC, Silani V, and Riva G. The use of 743

P300-based BCIs in amyotrophic lateral sclerosis: from augmentative and alternative communication to cognitive assessment. 744

Brain and behavior 2: 479-498, 2012. 745

Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, McMorland AJ, Velliste M, Boninger 746

ML, and Schwartz AB. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381: 557-564, 747

2013. 748

Coombs CH, Dawes RM, and Tversky A. Mathematical psychology. An elementary introduction. [By] Clyde H. Coombs ... 749

Robin M. Dawes ... Amos Tversky. Englewood Cliffs, New Jersey: Prentice-Hall, 1970. 750

Dickey AS, Suminski A, Amit Y, and Hatsopoulos NG. Single-unit stability using chronically implanted multielectrode 751

arrays. J Neurophysiol 102: 1331-1339, 2009. 752

Doble JE, Haig AJ, Anderson C, and Katz R. Impairment, activity, participation, life satisfaction, and survival in persons 753

with locked-in syndrome for over a decade: follow-up on a previously reported cohort. The Journal of head trauma 754

rehabilitation 18: 435-444, 2003. 755

Fazel-Rezai R, Allison BZ, Guger C, Sellers EW, Kleih SC, and Kubler A. P300 brain computer interface: current 756

challenges and emerging trends. Front Neuroeng 5: 14, 2012. 757

Fletcher RH, Fletcher SW, and Fletcher GS. Clinical epidemiology : the essentials. Philadelphia: Wolters Kluwer/Lippincott 758

Williams & Wilkins Health, 2014, p. 253 p. 759

Flint RD, Ethier C, Oby ER, Miller LE, and Slutzky MW. Local field potentials allow accurate decoding of muscle activity. 760

J Neurophysiol 108: 18-24, 2012a. 761

Flint RD, Lindberg EW, Jordan LR, Miller LE, and Slutzky MW. Accurate decoding of reaching movements from field 762

potentials in the absence of spikes. J Neural Eng 9: 046006, 2012b. 763

Flint RD, Scheid MR, Wright ZA, Solla SA, and Slutzky MW. Long-Term Stability of Motor Cortical Activity: Implications 764

for Brain Machine Interfaces and Optimal Feedback Control. J Neurosci 36: 3623-3632, 2016. 765

Flint RD, Wright ZA, Scheid MR, and Slutzky MW. Long term, stable brain machine interface performance using local field 766

potentials and multiunit spikes. J Neural Eng 10: 056005, 2013. 767

Fraser GW, Chase SM, Whitford A, and Schwartz AB. Control of a brain-computer interface without spike sorting. J Neural 768

Eng 6: 055004, 2009. 769

Friedman JH. Regularized Discriminant-Analysis. J Am Stat Assoc 84: 165-175, 1989. 770

Gallegos-Ayala G, Furdea A, Takano K, Ruf CA, Flor H, and Birbaumer N. Brain communication in a completely locked-771

in patient using bedside near-infrared spectroscopy. Neurology 82: 1930-1932, 2014. 772

Gilja V, Pandarinath C, Blabe CH, Nuyujukian P, Simeral JD, Sarma AA, Sorice BL, Perge JA, Jarosiewicz B, 773 Hochberg LR, Shenoy KV, and Henderson JM. Clinical translation of a high-performance neural prosthesis. Nature medicine 774

2015. 775

Hamer HM, Morris HH, Mascha EJ, Karafa MT, Bingaman WE, Bej MD, Burgess RC, Dinner DS, Foldvary NR, Hahn 776 JF, Kotagal P, Najm I, Wyllie E, and Luders HO. Complications of invasive video-EEG monitoring with subdural grid 777

electrodes. Neurology 58: 97-103, 2002. 778

Hastie T, Tibshirani R, and Friedman JH. The elements of statistical learning : data mining, inference, and prediction. New 779

York: Springer, 2009. 780

Hecht M, Hillemacher T, Grasel E, Tigges S, Winterholler M, Heuss D, Hilz MJ, and Neundorfer B. Subjective 781

experience and coping in ALS. Amyotrophic lateral sclerosis and other motor neuron disorders : official publication of the 782

World Federation of Neurology, Research Group on Motor Neuron Diseases 3: 225-231, 2002. 783

Hedegard E, Bjellvi J, Edelvik A, Rydenhag B, Flink R, and Malmgren K. Complications to invasive epilepsy surgery 784

workup with subdural and depth electrodes: a prospective population-based observational study. J Neurol Neurosurg Psychiatry 785

85: 716-720, 2014. 786

Heldman DA, Wang W, Chan SS, and Moran DW. Local field potential spectral tuning in motor cortex during reaching. 787

IEEE Trans Neural Syst Rehabil Eng 14: 180-183, 2006. 788

Hill NJ, Lal TN, Schroder M, Hinterberger T, Wilhelm B, Nijboer F, Mochty U, Widman G, Elger C, Scholkopf B, 789

Kubler A, and Birbaumer N. Classifying EEG and ECoG signals without subject training for fast BCI implementation: 790

comparison of nonparalyzed and completely paralyzed subjects. IEEE Trans Neural Syst Rehabil Eng 14: 183-186, 2006. 791

Hinterberger T, Birbaumer N, and Flor H. Assessment of cognitive function and communication ability in a completely 792

locked-in patient. Neurology 64: 1307-1308, 2005. 793


Hochberg LR, Bacher D, Jarosiewicz B, Masse NY, Simeral JD, Vogel J, Haddadin S, Liu J, Cash SS, van der Smagt P, 794

and Donoghue JP. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485: 372-375, 795

2012. 796

Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner A, Chen D, Penn RD, and 797

Donoghue JP. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442: 164-171, 2006. 798

Jarosiewicz B, Sarma AA, Bacher D, Masse NY, Simeral JD, Sorice B, Oakley EM, Blabe C, Pandarinath C, Gilja V, 799

Cash SS, Eskandar EN, Friehs G, Henderson JM, Shenoy KV, Donoghue JP, and Hochberg LR. Virtual typing by people 800

with tetraplegia using a self-calibrating intracortical brain-computer interface. Science translational medicine 7: 313ra179, 2015. 801

Kageyama Y, Hirata M, Yanagisawa T, Shimokawa T, Sawada J, Morris S, Mizushima N, Kishima H, Sakura O, and 802

Yoshimine T. Severely affected ALS patients have broad and high expectations for brain-machine interfaces. Amyotrophic 803

Lateral Sclerosis and Frontotemporal Degeneration 15: 513-519, 2014. 804

Kennedy PR, Bakay RA, Moore MM, Adams K, and Goldwaithe J. Direct control of a computer from the human central 805

nervous system. IEEE Trans Rehabil Eng 8: 198-202, 2000. 806

Kubler A, and Birbaumer N. Brain-computer interfaces and communication in paralysis: extinction of goal directed thinking 807

in completely paralysed patients? Clin Neurophysiol 119: 2658-2666, 2008. 808

Kubler A, Kotchoubey B, Hinterberger T, Ghanayim N, Perelmouter J, Schauer M, Fritsch C, Taub E, and Birbaumer 809 N. The thought translation device: a neurophysiological approach to communication in total motor paralysis. Exp Brain Res 124: 810

223-232, 1999. 811

Kubler A, Kotchoubey B, Kaiser J, Wolpaw JR, and Birbaumer N. Brain-computer communication: unlocking the locked 812

in. Psychol Bull 127: 358-375, 2001. 813

Kubler A, Nijboer F, Mellinger J, Vaughan TM, Pawelzik H, Schalk G, McFarland DJ, Birbaumer N, and Wolpaw JR. 814

Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. Neurology 64: 1775-1777, 2005. 815

Laureys S, Pellas F, Van Eeckhout P, Ghorbel S, Schnakers C, Perrin F, Berre J, Faymonville ME, Pantke KH, Damas 816

F, Lamy M, Moonen G, and Goldman S. The locked-in syndrome : what is it like to be conscious but paralyzed and 817

voiceless? Prog Brain Res 150: 495-511, 2005. 818

Mainsah BO, Collins LM, Colwell KA, Sellers EW, Ryan DB, Caves K, and Throckmorton CS. Increasing BCI 819

communication rates with dynamic stopping towards more practical use: an ALS study. J Neural Eng 12: 016013, 2015. 820

Marchetti M, and Priftis K. Brain-computer interfaces in amyotrophic lateral sclerosis: A metanalysis. Clin Neurophysiol 821

2014. 822

McCane LM, Heckman SM, McFarland DJ, Townsend G, Mak JN, Sellers EW, Zeitlin D, Tenteromano LM, Wolpaw 823

JR, and Vaughan TM. P300-based brain-computer interface (BCI) event-related potentials (ERPs): People with amyotrophic 824

lateral sclerosis (ALS) vs. age-matched controls. Clin Neurophysiol 126: 2124-2131, 2015. 825

McCane LM, Sellers EW, McFarland DJ, Mak JN, Carmack CS, Zeitlin D, Wolpaw JR, and Vaughan TM. Brain-826

computer interface (BCI) evaluation in people with amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal 827

Degener 15: 207-215, 2014. 828

Mehring C, Rickert J, Vaadia E, Cardosa de Oliveira S, Aertsen A, and Rotter S. Inference of hand movements from local 829

field potentials in monkey motor cortex. Nat Neurosci 6: 1253-1254, 2003. 830

Milekovic T, Ball T, Schulze-Bonhage A, Aertsen A, and Mehring C. Detection of Error Related Neuronal Responses 831

Recorded by Electrocorticography in Humans during Continuous Movements. PLoS One 8: e55235, 2013. 832

Milekovic T, Ball T, Schulze-Bonhage A, Aertsen A, and Mehring C. Error-related electrocorticographic activity in humans 833

during continuous movements. J Neural Eng 9: 026007, 2012. 834

Miner LA, McFarland DJ, and Wolpaw JR. Answering questions with an electroencephalogram-based brain-computer 835

interface. Arch Phys Med Rehabil 79: 1029-1033, 1998. 836

Moore DS, McCabe GP, and Craig B. Introduction to the practice of statistics. Basingstoke: W.H.Freeman, 2009. 837

Moran D. Evolution of brain-computer interface: action potentials, local field potentials and electrocorticograms. Curr Opin 838

Neurobiol 20: 741-745, 2010. 839

Murguialday AR, Hill J, Bensch M, Martens S, Halder S, Nijboer F, Schoelkopf B, Birbaumer N, and Gharabaghi A. 840

Transition from the locked in to the completely locked-in state: a physiological analysis. Clin Neurophysiol 122: 925-933, 2011. 841

Nijboer F, Sellers EW, Mellinger J, Jordan MA, Matuz T, Furdea A, Halder S, Mochty U, Krusienski DJ, Vaughan TM, 842

Wolpaw JR, Birbaumer N, and Kubler A. A P300-based brain-computer interface for people with amyotrophic lateral 843

sclerosis. Clin Neurophysiol 119: 1909-1916, 2008. 844

Nuyujukian P, Kao JC, Fan JM, Stavisky SD, Ryu SI, and Shenoy KV. Performance sustaining intracortical neural 845

prostheses. J Neural Eng 11: 066003, 2014. 846

Pandarinath C, Nuyujukian P, Blabe CH, Sorice BL, Saab J, Willett FR, Hochberg LR, Shenoy KV, and Henderson JM. 847

High performance communication by people with paralysis using an intracortical brain-computer interface. Elife 6: 2017. 848


Perge JA, Homer ML, Malik WQ, Cash S, Eskandar E, Friehs G, Donoghue JP, and Hochberg LR. Intra-day signal 849

instabilities affect decoding performance in an intracortical neural interface system. J Neural Eng 10: 036004, 2013. 850

Perge JA, Zhang S, Malik WQ, Homer ML, Cash S, Friehs G, Eskandar EN, Donoghue JP, and Hochberg LR. 851

Reliability of directional information in unsorted spikes and local field potentials recorded in human motor cortex. J Neural Eng 852

11: 046007, 2014. 853

Pinkhardt EH, Jurgens R, Becker W, Molle M, Born J, Ludolph AC, and Schreiber H. Signs of impaired selective 854

attention in patients with amyotrophic lateral sclerosis. Journal of neurology 255: 532-538, 2008. 855

Pires G, Nunes U, and Castelo-Branco M. Comparison of a row-column speller vs. a novel lateral single-character speller: 856

assessment of BCI for severe motor disabled patients. Clin Neurophysiol 123: 1168-1181, 2012. 857

Plum F, and Posner JB. The diagnosis of stupor and coma. Contemp Neurol Ser 10: 1-286, 1972. 858

Rickert J, Oliveira SC, Vaadia E, Aertsen A, Rotter S, and Mehring C. Encoding of movement direction in different 859

frequency ranges of motor cortical local field potentials. J Neurosci 25: 8815-8824, 2005. 860

Schreiber H, Gaigalat T, Wiedemuth-Catrinescu U, Graf M, Uttner I, Muche R, and Ludolph AC. Cognitive function in 861

bulbar- and spinal-onset amyotrophic lateral sclerosis. A longitudinal study in 52 patients. Journal of neurology 252: 772-781, 862

2005. 863

Sellers EW, Ryan DB, and Hauser CK. Noninvasive brain-computer interface enables communication after brainstem stroke. 864

Science translational medicine 6: 257re257, 2014. 865

Shain W, Spataro L, Dilgen J, Haverstick K, Retterer S, Isaacson M, Saltzman M, and Turner JN. Controlling cellular 866

reactive responses around neural prosthetic devices using peripheral and local intervention strategies. IEEE Trans Neural Syst 867

Rehabil Eng 11: 186-188, 2003. 868

Simeral JD, Kim SP, Black MJ, Donoghue JP, and Hochberg LR. Neural control of cursor trajectory and click by a human 869

with tetraplegia 1000 days after implant of an intracortical microelectrode array. J Neural Eng 8: 025027, 2011. 870

Stark E, and Abeles M. Predicting movement from multiunit activity. J Neurosci 27: 8387-8394, 2007. 871

Vansteensel MJ, Pels EGM, Bleichner MG, Branco MP, Denison T, Freudenburg ZV, Gosselaar P, Leinders S, Ottens 872

TH, Van Den Boom MA, Van Rijen PC, Aarnoutse EJ, and Ramsey NF. Fully Implanted Brain–Computer Interface in a 873

Locked-In Patient with ALS. New Engl J Med 0: null, 2016. 874

Vieregge P, Wauschkuhn B, Heberlein I, Hagenah J, and Verleger R. Selective attention is impaired in amyotrophic lateral 875

sclerosis--a study of event-related EEG potentials. Brain research Cognitive brain research 8: 27-35, 1999. 876

Wang W, Collinger JL, Degenhart AD, Tyler-Kabara EC, Schwartz AB, Moran DW, Weber DJ, Wodlinger B, 877

Vinjamuri RK, Ashmore RC, Kelly JW, and Boninger ML. An Electrocorticographic Brain Interface in an Individual with 878

Tetraplegia. PLoS One 8: e55344, 2013. 879

Wessberg J, Stambaugh CR, Kralik JD, Beck PD, Laubach M, Chapin JK, Kim J, Biggs SJ, Srinivasan MA, and 880

Nicolelis MA. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408: 361-365, 2000. 881

Wodlinger B, Downey JE, Tyler-Kabara EC, Schwartz AB, Boninger ML, and Collinger JL. Ten-dimensional 882

anthropomorphic arm control in a human brain-machine interface: difficulties, solutions, and limitations. J Neural Eng 12: 883

016011, 2015. 884

Wolpaw JR, and Wolpaw EW. Brain-computer interfaces : principles and practice. Oxford; New York: Oxford University 885

Press, 2012. 886

Yin M, Borton DA, Komar J, Agha N, Lu Y, Li H, Laurens J, Lang Y, Li Q, Bull C, Larson L, Rosler D, Bezard E, 887 Courtine G, and Nurmikko AV. Wireless Neurosensor for Full-Spectrum Electrophysiology Recordings during Free Behavior. 888

Neuron 84: 1170-1182, 2014. 889

Young JM, and McNicoll P. Against all odds: Positive life experiences of people with advanced amyotrophic lateral sclerosis. 890

Health Soc Work 23: 35-43, 1998. 891

Yousry TA, Schmid UD, Alkadhi H, Schmidt D, Peraud A, Buettner A, and Winkler P. Localization of the motor hand 892

area to a knob on the precentral gyrus. A new landmark. Brain 120 ( Pt 1): 141-157, 1997. 893

894


SUPPORTING INFORMATION 895

Movie 1. Session T6 473, 3rd copy-phrase block using historical decoder. T6 used the LFP-based BCI with a historical decoder 896

to control the FlashSpeller in order to type “Pack my box with”. Upon selection, the button with options turned green. Left 897

panel. Video showing the participant monitor during the block as seen by the participant. Right panels. Neural features recorded 898

during the block and the probability of selection for the historical decoder synchronized with the video shown on the left panel. 899

Each panel shows a 3s epoch with the frame of the video synchronized to the middle of the epoch marked by the white line. Top 900

right panel. All 50 features selected for decoder calibration are shown as the color image. Bright colors shows higher amplitude 901

and darker colors show lower amplitude. Three middle right panels. One of each low-filtered component (LFC), intermediate 902

frequency component (IFC) and high-frequency component (HFC) chosen out of 50 features selected for decoder calibration. 903

Bottom right panel. Probability of selection for the historical decoder. Cyan line marks the threshold set at 0.95. 904

Movie 2. Session T2 523, 1st copy-phrase block using historical decoder. T2 used the LFP-based BCI with a historical decoder 905

to control the FlashSpeller in order to type “The quick brown fox”. Upon selection, the button with options turned green. Left 906


during the block and the probability of selection for the historical decoder synchronized with the video shown on the left panel. 908





Bottom right panel. Probability of selection for the historical decoder. Cyan line marks the threshold set at 0.95. 913

Movie 3. Session T6 473, 3rd copy-phrase block using same-day decoder. T6 used the LFP-based BCI with a same-day decoder 914

to control the FlashSpeller in order to type “Pack my box with”. Upon selection, the button with options turned green. Left 915


during the block and the probability of selection for the same-day decoder synchronized with the video shown on the left panel. 917





Bottom right panel. Probability of selection for the same-day decoder. Cyan line marks the threshold set at 0.95. 922

Movie 4. Session T2 523, 1st copy-phrase block using same-day decoder. T2 used the LFP-based BCI with a same-day decoder 923

to control the FlashSpeller in order to type “The quick brown fox”. Upon selection, the button with options turned green. Left 924


during the block and the probability of selection for the same-day decoder synchronized with the video shown on the left panel. 926





Bottom right panel. Probability of selection for the same-day decoder. Cyan line marks the threshold set at 0.95. 931

Movie 5. Session T2 524, 1st free-spelling block without word completion. T2 used the LFP-based BCI with a historical 932

decoder to control the FlashSpeller in order to type “A story told by an idiot” (Shakespeare paraphrase, Macbeth, Act V, Scene 933

V). Upon selection, the button with options turned green. Left panel. Video showing the participant monitor during the block as 934

seen by the participant. Right panels. Neural features recorded during the block and the probability of selection for the historical 935

decoder synchronized with the video shown on the left panel. Each panel shows a 3s epoch with the frame of the video 936

synchronized to the middle of the epoch marked by the white line. Top right panel. All 50 features selected for decoder 937

calibration are shown as the color image. Bright colors shows higher amplitude and darker colors show lower amplitude. Three 938

middle right panels. One of each low-filtered component (LFC), intermediate frequency component (IFC) and high-frequency 939

component (HFC) chosen out of 50 features selected for decoder calibration. Bottom right panel. Probability of selection for the 940

historical decoder. Cyan line marks the threshold set at 0.95. 941

Movie 6. Session T2 523, 1st free-spelling block with word completion. T2 used the LFP-based BCI with a historical decoder to 942

control the FlashSpeller in order to type and send an email. The beginning of the email spelled “Tell team thanks for fixing”. 943

Upon selection, the button with options turned green. Left panel. Video showing the participant monitor during the block as seen 944

by the participant. Right panels. Neural features recorded during the block and the probability of selection for the historical 945








Movie 7. Session T6 493, 1st free-spelling block with word completion. T6 used the LFP-based BCI with a historical decoder to 952

control the FlashSpeller in order to type “Free speech is one of the most important rights”. Upon selection, the button with 953

options turned green. Left panel. Video showing the participant monitor during the block as seen by the participant. Right 954

panels. Neural features recorded during the block and the probability of selection for the historical decoder synchronized with 955

the video shown on the left panel. Each panel shows a 3s epoch with the frame of the video synchronized to the middle of the 956

epoch marked by the white line. Top right panel. All 50 features selected for decoder calibration are shown as the color image. 957

Bright colors shows higher amplitude and darker colors show lower amplitude. Three middle right panels. One of each low-958

filtered component (LFC), intermediate frequency component (IFC) and high-frequency component (HFC) chosen out of 50 959

features selected for decoder calibration. Bottom right panel. Probability of selection for the historical decoder. Cyan line marks 960

the threshold set at 0.95. 961

Movie 8. Session T6 493 normalization block. Each session started with a normalization block during which the participants 962

looked at the monitor while the FlashSpeller automatically spelled the phrase “The brain”. Data from the normalization block 963

were used to calculate the STFT normalization matrix and to initialize the parameters for the automatic feature normalization. 964

Left panel. Video showing the participant monitor during the normalization block as seen by the participant. Right panels. 965

Neural features recorded during the block and the probability of selection for the historical decoder synchronized with the video 966

shown on the left panel. Each panel shows a 3s epoch with the frame of the video synchronized to the middle of the epoch 967

marked by the white line. Top right panel. All 50 features selected for decoder calibration are shown as the color image. Bright 968

colors shows higher amplitude and darker colors show lower amplitude. Three middle right panels. One of each low-filtered 969

component (LFC), intermediate frequency component (IFC) and high-frequency component (HFC) chosen out of 50 features 970

selected for decoder calibration. Bottom right panel. Probability of selection for the historical decoder. Cyan line marks the 971

threshold set at 0.95. 972

Movie 9. Session T6 473, 2nd open-loop calibration block. In the open-loop calibration blocks, the FlashSpeller automatically 973

spelled the phrase “The brain”. The participants observed the spelling and were asked to perform a click action when the correct 974

options appeared on the screen. The correct options were presented with a highlighted blue background of the button. Data from 975

the open-loop calibration blocks was used to calibrate the historical decoder (collection sessions) or to calibrate the same-day 976

decoder (comparison sessions). Left panel. Video showing the participant monitor during the open-loop calibration block as 977

seen by the participant. Right panels. Neural features recorded during the block and the probability of selection for the historical 978








TimeDecoder calibrated from data recorded over up to 42 days ...

... then used unchanged for up to 138 days laterHistorical decoder

Feature 1Feature 2

0

1

Prob

abili

ty Historical decoder

50 selected features...

Wideband recordingsfrom 96 electrodes

Volta

ge

...

Time

96 high frequencycomponent features

...

96 low-pass �lteredcomponent features

...

96 intermediate frequencycomponent features

...

Decoded selection

Prob

abili

ty

Threshold

Time

Selection

No selection

e

t

Word completion

<< BK

CL DRET

SP >> ETAO GYPBV KJXQZ .,’?:!-^RLDC UMWFINSH

<< BK

FlashSpeller Custom text-entry application designed for selection-based BCIs

Signal processing for LFP-based Flash Speller brain-computer interface


Cued aut

Samdecoder

Load from Historicaldecoder

m Ov laz g Pac m o i Five doze li uor g

decoder 1

decoder 2

decoder 1

decoder 2

decoder 1

decoder 2

decoder 1

decoder 2

Double bli

Build fr

decoder…

Par t rela ed

Comparison session

OLC

Cued autPar t rela ed…

Data collection session

Par t rela ed…

Communication session


T2 45249-162Hz

14-33Hz

T2 45357-232Hz

14-33Hz

T2 46049-174Hz

14-29Hz

T2 46753-158Hz

14-33Hz

T2 47449-236Hz

18-29Hz

T6 31357-213Hz

18-33Hz

T6 32264-350Hz

21-33Hz

T6 33664-330Hz

21-53Hz

T6 34861-334Hz

21-33Hz

T6 35561-205Hz

21-45Hz

0

Bot

tom

ban

d fr

eque

ncy

Top band frequency

0100Hz

0 0.2 0.4 0.6 0.8 1

Frequency band SNR normalized to maximum

100H

z

- Frequency band that gave the highest SNR in the intermediate and high frequency ranges


-5 0 5 10

Normalized feature

FeedbackOptionTransition

DUFP F nrcw r

Probability

LFCchannel 65

IFCchannel 14

HFCchannel 80

18 20 22 24 26 28 30 32 34 36 38

Selectedfeatures

Block time (s)

Threshold


DUFP

F nrcw

r [wor

ds]

Free

SP >>

etos

s dufp

p etose et

ose [w

ords

]

spee

ch

Probability

LFCchannel 65

IFCchannel 14

HFCchannel 80

Selectedfeatures

20 30 40 50 60 70 80 90 100

Threshold

D F nr r

4


Probability

LFCchannel 65

IFCchannel 14

HFCchannel 80

Selectedfeatures

50 100 150 200 250 300

Free speech is one of the most important rights

Threshold

os tos

[w sp

T6 493, first free-spelling block


0

50

100

150

200

T2 452, el. 95

Fre

quen

cy (

Hz)

Spe

ctro

gram

T2 500, el. 95 (+48 days) T2 550, el. 95 (+98 days)

10

20

30

40

50

Low

-pas

s fil

tere

dco

mpo

nent

Tria

ls

10

20

30

40

50

Inte

rmed

iate

freq

uenc

yco

mpo

nent

Tria

ls

0 1 2

10

20

30

40

50

Time from click cue (s)

Hig

h fr

eque

nncy

com

pone

nt

Tria

ls

0 1 2Time from click cue (s)

0 1 2Time from click cue (s)

Fea

ture

(s.

d.)

Fea

ture

(s.

d.)

Fea

ture

(s.

d.)

0246

02468

0

2

4

1

2

3

4

Nor

mal

ized

am

plitu

de


0 25 50 75 100

0

0.2

0.4

0.6

0.8

1

Cor

rela

tion

coef

ficie

nt

25 50 75 100 125 150

Number of days between compared sessions (day)

b = -0.02 year-1 b = -0.21 year-1

b, bC Slope of the linear fitLinear fit over Mr(s1,s2)Mean of Mr(s1,s2) for random events - RMr(s1,s2)Median of rf(s1,s2) over all channels - Mr(s1,s2)

T2 T6

0123

Fea

ture

(s.

d.)

012

0

Fea

ture

(s.

d.)

0

1

0 1 2

0

1

2

Fea

ture

(s.

d.)

0 1 2

0

1

2

3

Time from click cue (s)

el. 95 el. 80T2 T6

+98 days (T2 550)+48 days (T2 500)0 days (T2 452)

+151 days (T6 473)+75 days (T6 397)0 days (T6 322)

LFC

IFC

HFC

A

B


452453460467474487500510515523529545550

452

453

460

467

474

487

500

510

515

523

529

545

550

313

322

336

348

355

369

376

397

418

441

474

Cal

libra

tion

sess

ion

313

322

336

348

355

369

376

397

418

441

473

Test session Test session

0

0.2

0.4

0.6

0.8

1

0 25 50 75 1000

0.2

0.4

0.6

0.8

1

25 50 75 100 125 1500

0.2

0.4

0.6

0.8

1

Decoder age T (day)

P = 0.82 P = 0.46

CYX

T2 T6

Nor

mal

ized

mut

ual

info

rmat

ion

C YX

Decoder age T (day)


42 48 63 86 96 118 1380

2

4

6

8

Spe

lling

per

form

ance

(co

rrec

t cha

ract

ers

per

min

ute,

CC

PM

)

T6

Overall

*ns*

Decoder age (days)

13 26 36 41 49 55 56 71 760

1

2

3

4

T2

Overall

*ns*

Historical decoder with word completionFree Spelling blocks

Historical decoderCopy Phrase blocks

Same day decoderCopy Phrase blocks

P = 0.68 P = 0.18 P = 0.40

P = 0.95 P = 0.48 P = 0.73


487 500 510 515 523 524 529 530 545 550 Overall0

0.5

1

[13] [26] [36] [41] [49] [50] [55] [56] [71] [76]

T2

13 26 36 41 49 55 56 71 760.4

0.6

0.8

397 403 418 441 451 474 493 Overall0

0.5

1

[42] [48] [63] [86] [96] [119] [138]

T6

42 48 63 86 96 119 138

0.7

0.8

0.9

Nor

mal

ized

mut

ual i

nfor

mat

ion

C YX

Session (days from implantation) [Decoder age (days)] Decoder age (days)

P = 0.78 P = 0.42 P = 0.33

P = 0.67 P = 0.76 P = 0.13

Historical decoderCopy Phrase blocks

Historical decoder with word completionFree Spelling blocks

Same day decoderCopy Phrase blocks

Historical decoderFree Spelling blocks

ns **

ns **


487 500 510 515 523 529 545 550 Overall0

5

10

[13] [26] [36] [41] [49] [55] [71] [76]

T2

ns

397 418 441 473 Overall[42] [63] [86] [118]

T6

ns

Par

ticip

ant's

sco

res

Session (days from implantation) Decoder age [days]

Historical decoderSame day decoderCopy Phrase blocks:


T2 45257-209Hz

18-29Hz

T2 45353-229Hz

21-29Hz

T2 46053-287Hz

18-29Hz

T2 46749-330Hz

21-29Hz

T2 474127-174Hz

21-29Hz

T6 313

21-29Hz

T6 322

21-33Hz

T6 336

21-29Hz

T6 348

21-29Hz

T6 355

21-25Hz

00

Bot

tom

ban

d fr

eque

ncy

Top band frequency

100H

z

0 0.2 0.4 0.6 0.8 1

Frequency band SNR normalized to maximum100Hz

369

376

397

418

441

474 Overall

*

0

1

487

500

510

515

523

529

545

550 Overall

*

Nor

mal

ized

mut

ual

info

rmat

ion C YX

T2 T6

Average signal decoder

Historical decoder

A

B

Session (days from implant placed)


stable long-term bci-enabled communication in als …shenoy/grouppublications/...85 crossings have...

Documents