Neuron, Volume 95

Supplemental Information

Motor Cortical Visuomotor Feedback Activity

Is Initially Isolated from Downstream Targets

in Output-Null Neural State Space Dimensions

Sergey D. Stavisky, Jonathan C. Kao, Stephen I. Ryu, and Krishna V. Shenoy

SUPPLEMENTAL FIGURES

Figure S1, related to Figures 2C and 3D. Monkeys showed minimal improvement on the Cursor Jump Task across days (A) Each marker and vertical bar shows one experiment session’s mean ± STD time to target increase, which was calculated by subtracting a given session’s no jump trials’ mean time to target from that session’s cursor jump trials’ times to target. Experiment sessions are numbered by chronological order. The slope and p-value of a linear regression between experiment session number and time to target increase is shown for each monkey’s jump event type. (B) Same for BMI controlled experiments. Note that all four jump event types were presented during each experiment session.

Tim

e To

Tar

get

Incr

ease

(ms)

Tim

e To

Tar

get

Incr

ease

(ms)

1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8

0

400

800

1200

Experiment Session Number

1 2 3 4 5 6 7 81 2 3 4 5 6 71 2 3 4 5 6 70

200

400

600

800

Experiment Session Number

Monkey J

Monkey J

rm-ControlledA

MI-ControlledB

Monkey L

Monkey L

Monkey R

-2.5 ms/session, p=0.897-16.7 ms/session, p=0.145

-10.1 ms/session, p=0.452-40.0 ms/session, p=0.032-18.7 ms/session, p=0.262-15.5 ms/session, p=0.150

5.3 ms/session, p=0.435-5.7 ms/session, p=0.720-7.7 ms/session, p=0.503

-12.5 ms/session, p=0.226

-19.4 ms/session, p=0.098-20.2 ms/session, p=0.392

slope = 5.4 ms/session, p=0.478 -3.1 ms/session, p=0.956

Jump at 3 cmJump at 4cmJump at 6 cmJump at 10 cmJump during hold

Figure S2, related to Figure 4A. Example perturbation-evoked firing rate changes (A) Example neural responses following a cursor jump. Each plot shows one electrode’s threshold crossing spike rate on a particular experiment session, trial-averaged within a specific target/jump-event-type/jump-direction condition (‘dataset-condition’). Mean ± STD firing rates are shown for no jump (gray) and perturbed (red) trials. (Insets) provide context of this response with respect to the task. The gray vector points to the location of the radial target. The colored arrow shows the direction of perturbation, and its color denotes the jump event type. The black arrow and text show this electrode’s neural push direction and decoder contribution weight. Thus, the first plot shows that following a jump at 6 cm, the firing rate on an electrode that pushes the cursor in a similar direction as the cursor jump briefly increases and then decreases back to the no jump rate. Examples were chosen to illustrate a variety of common response patterns. (B) Signed firing rate changes following a cursor jump. Data were analyzed as in Figure 4A, except that at each time point we plot the mean, rather than vector norm, of all electrodes’ firing rate changes. This reveals that the overall population firing rates increased in response to the perturbation. ‘Early’ and ‘Late’ analysis epochs (see panel C) are shaded. (C) Histograms showing the distribution of firing rate changes at two example epochs: ‘Early’ (125 to 150 ms after the cursor jump) and ‘Late’ (325 to 350 ms). Since the jump during hold event’s faux jump-subtracted time series is truncated, the Late epoch for this condition is defined earlier, from 215 to 240 ms. Each dataset-condition-electrode contributes one datum to each histogram, and the heights of all bins add up to one. Distribution medians are shown with vertical ticks along the horizontal axis. Across conditions and electrodes, firing rates both increased and decreased following the perturbation.

0 500

0

220 1.3% ofdecode

0 500

220 1.2%

0 5000

260 1.9%

0 5000

340

3.2%

0 5000

0

1905.1%

0 500

0

110

1.3%

0 5000

160

5.1%

0 500

0

1601.3%

-50 0 50 -50 0 50 -50 0 50 -50 0 50

0 100 200 300 400 500-20

7

0 100 200 300 400 5000

10

PerturbationNo Jump

Firing

Rat

e (H

z)J.2015.01.09 Elec.103 J.2015.01.09 Elec.132 L.2015.01.07 Elec.94 L.2015.01.14 Elec.36

L.2015.01.07 Elec.94 L.2015.01.14 Elec.17J.2015.01.09 Elec.113 J.2015.01.12 Elec.162

Early Late Early Late

A

B

C

Monkey J Monkey L∆

Firing

Rat

e (H

z)

∆ Firing Rate (Hz)

Time After Cursor Jump (ms)

Jump at 3 cmJump at 6 cmJump at 10 cmJump during hold

Early: 125 to 150 ms Late: 325 to 350 ms Early: 125 to 150 ms Late: 325 to 350 ms

Time After Cursor Jump (ms)

Figure S3, related to Figure 4B. Examples of neural push following a cursor jump Each figure column corresponds to the neural push analysis of trials from one example BMI Cursor Jump Task dataset-condition. All four target locations and jump event types are represented, and there are two examples from each monkey. Note that axis limits differ across plots. (A) Cursor trajectories for all perturbed trials of each example dataset-condition that met the trial inclusion criteria. The cursor started from near coordinate (0,0), shown by the gray cross, and proceeded towards the target whose acquisition area is by the dashed gray square. (B) The neural push component along the Perturbation Axis. Positive push corresponds to the direction opposing the cursor jump. Unperturbed responses were generated by aligning to faux jumps. Note that the duration of the jump during hold unperturbed series is limited because these trials ended shortly after the faux jump. (C) Same as (B) but for the neural push component along the Task Axis. Positive values denote push in the direction from workspace center towards the target. (D) Neural push differences were calculated by subtracting the mean neural push for unperturbed trials to a given target from jump trials’ neural push to the same target. Figure 4B was generated in the same manner, but averaged across all target locations, jump directions, and datasets.

0 500-1

0

3

0 500-2

0

6

0 500-2

0

6

Task Axis

PerturbationAxis

-1

0

1

2

3

4

-1

1

3

0 500-1

0

1

-1.5

0

1.5

2

10

0

6

-1

0

3

-4

0

4

-4

0

4

X Position4 cm

Y Po

sition

start

Perturbation

target

A

B

C

D

Time After Cursor Jump (ms)

Δ Ne

ural

Push

(cm

/s)

Task

Axis

(cm

/s)

Pertu

rbat

ion A

xis (c

m/s

)

No Jump

Perturbation

Jump at 3 cm

J.2015.01.20 J.2015.01.12 L.2015.01.07 L.2015.01.08

Jump at 6 cm Jump at 10 cm Jump During Hold

Δ Neural Push

Figure S4, related to Figure 5A. Controls for interpretation (A) To rule out separate early responding and output-potent neural subpopulations, data from Figure 5A were re-analyzed using only electrodes with high decoder weights. Electrodes were sorted from greatest to least decoder contribution weight (red). Cumulative decoder weights are shown in black. Thinner faded traces show individual experiment session curves, while thicker traces show the means of these individual days’ curves. Dashed vertical lines mark when each monkey’s mean cumulative weight exceeds the threshold used for inclusion in panel B. Depending on the dataset, 24 to 30 electrodes (together contributing > 0.5 of the decoded velocity) were included for monkey J. For monkey L, 28 to 38 electrodes with a cumulative weight > 0.8 were included. (B) Perturbation-evoked output-null activity changes restricted to high decoder weight electrodes. The initial output-null response is still present, indicating that this early feedback-related response is not restricted to a neural subpopulation with minimal influence on BMI output. (C) The early jump-evoked neural response was not output-null simply because the ReFIT protocol trained decoders to “ignore” responses to naturally occurring errors in the closed-loop recalibration data. This could happen if decoder refitting mapped error-evoked neural activity to zero velocity. Here, Figure 5A neural data were re-analyzed with the output-potent and output-null subspaces calculated with respect to a different velocity Kalman filter that was trained using data from the first block of the ReFIT protocol. During this block the monkey passively watched automated error-free movements of the cursor to targets. Jump-evoked responses projected into these dimensions are similar to those shown in Figure 5A, which were projected into the output-null and output-potent subspaces of the final (closed-loop re-calibrated) ReFIT decoder. (D) Similarly, the early perturbation-evoked response was not output-null due to the decoder being trained using data immediately following target appearance. Neural activity during these epochs may reflect a sudden arrival of inputs signaling a new difference between cursor and target positions, and this activity was regressed against near-zero velocities. A potential concern is that the decoder was thereby indirectly optimized to ignore cursor jump-evoked neural responses. To rule out this possibility, Figure 5A data were re-analyzed with ReFIT decoders trained identically to those used in that figure, except here we excluded the initial 150 ms following target presentation for each training trial. This exclusion applied to both outward and return-to-center trials. We found that as in panel C and Figure 5A, the early perturbation-evoked response was largely confined to output-null dimensions.

0

0 50 100 150 200 250 0 50 100 150 200 250Time After Cursor Jump (ms)

ReFIT Decoder With Trials’ First 150 ms Skipped

Passive Observation-Trained Decoder

∆ O

utpu

t-Po

tent

(AU)

∆ Ou

tput

-Nu

ll (AU

)

100 ms 126 ms

122 ms

122 ms

177 ms

67 ms

66 ms

103 ms

126 ms79 ms

High Decoder Weight Electrodes Only

961 400

0.1

Cum

ulat

ive

Deco

der W

eight

Electrode Rank0

1

1921 100

Frac

tion

Deco

der

Weig

ht

0

0.05Monkey J Monkey L

0.5 0.8

Jump at 3 cmJump at 6 cmJump at 10 cmJump during hold

C

B

A

D

Figure S5, related to Figure 5B. Directional tuning differs between early jump responses and unperturbed movement initiation (A) BMI Radial 8 Target Task PSTHs for example electrode J.2015.01.20 Elec.109. Each plot corresponds to one target. This electrode showed significant leftward direction preference in the early epoch (higher rates across the three leftward target conditions) and significant rightward preference in the late epoch. (B) The same example electrode’s perturbation-evoked responses for all four target/jump-direction conditions in this dataset’s BMI Cursor Jump Task. To orient the reader, the panel center shows each condition’s kinematics just before and after the cursor jump. The four corner plots show the corresponding target/jump-direction condition’s firing rate changes. The red PSTH shows the grand mean of all four individual jump event type’s PSTHs. This electrode increases its firing rate more for leftward perturbations during both early and late epochs. Its late corrective response is consistent with its late unperturbed tuning (faster firing when generating rightward movement). The early response, however, is inconsistent between the perturbation response (faster firing when initiating rightward corrections) and unperturbed movements (faster firing when initiating leftward movements). (C) Counts of how many electrode-target pairs (mean ± SEM across datasets) had consistent (filled bars) or inconsistent (open bars) directional preference between the early and late epochs of unperturbed and corrective movements. The example electrode from panels A and B contributed two electrode-target datums to the inconsistent early epoch category and two to the consistent late epoch category. Most electrodes showed consistent direction preference during the late epochs of unperturbed and corrective behaviors; in contrast, almost half showed inconsistent direction preference during the early epoch of these different behaviors. (D) Perturbation-evoked activity did not substantially explore neural dimensions not traversed during unperturbed BMI movements. Bright red and gray curves show the cumulative variance explained when projecting early epoch BMI jump and faux-jump PSTHs into the top PCs of the BMI Radial 8 Task data (mean ± SEM across days). To provide a sense of the dimensionalities of the unperturbed arm-controlled and BMI Radial 8 Task neural data, as well as the early perturbation-evoked activity itself, we also show variance explained when performing PCA directly on these data (‘self’ curves).

1 5 10 151 5 10 150

1Fr

actio

n Va

rianc

e Ex

plain

ed

Number of Principal Components

Monkey J Monkey L

Arm Radial 8 (self)BMI Radial 8 (self)∆BMI Jump (self)BMI Faux JumpBMI Jump

J Arm L Arm J BMI L BMI

232749 53 140

65 23 76 30 3 428 146

76 2018

-20

0

40

17025 340

-20

0

40

Jump Left

Target Up

Target Down

Jump Right

4 cm

10 Hz 19 Hz 8.9 Hz 2.8 Hz

4.0 Hz2.6 Hz25 Hz14 Hz

56 Hz 71 Hz

Early Late

Early Late

p=3.47e-6 86 Hzp=0.008

52 Hz

100

017025 340

Time AfterTarget On (ms)

Time After Cursor Jump (ms)

Consistent Inconsistent

Firing

Rat

e (H

z)

Δ Fir

ing R

ate

(Hz)

Target XTa

rget

Y

# El

ectro

de-T

arge

t Pair

sA B

C D

Jump at 3 cmJump at 6 cmJump at 10 cmJump during hold

Figure S6, related to Figures 4B and 6C. No learning-related changes in output-potent leakage of the early perturbation response (A) We did not find evidence of learning-related change in how strongly Early epoch (150 ms after perturbation) firing rate changes projected into decoder output-potent dimensions. Trials were grouped either by within-session quartile (top row) or individual experiment number (bottom row). Quartiles were determined based on trial order within a given condition (target/jump-direction/jump-event-type). If subdividing a condition led to fewer than 5 trials per quartile, we excluded it from analysis. Faux jump-aligned firing rate subtraction was performed against unperturbed trials from the same quartile. The ratio of output-null to output-potent response magnitude was then calculated within each quartile or session as in Figure 6C. Thin colored curves show this ratio for each jump event type, while red curves show this ratio when aggregating across all jump event types. If there had been learning, we would expect to see significant increases in the null:potent ratio. Note that comparing the ratio of null to potent activity makes this analysis robust against within- and across-session differences in overall firing rates. Red text shows the result of a least-squares linear regression between quartile or session number and the combined across jump events null:potent ratio. The slope and significance test p-value (against a null hypothesis of zero slope) are reported. For the quartile analysis, each experiment session provided one datum per quartile, and error bars show ± SEM across sessions. (B) Even if the relative magnitude of early output-potent activity did not diminish over time, learning could have caused this activity to become more beneficial in terms of pushing the cursor towards the target. To look for this, we again divided the same data by quartile or session as in the previous panel, but instead measured change in neural push (calculated as in Figure 4B). Positive values indicate neural push in the task-beneficial direction (opposing the perturbation along the Perturbation Axis, and along the center-to-target Task Axis vector). The small amount of early neural push did not show evidence of systematically becoming more beneficial over time.

1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8−2

0

2

Quartile

Δ Ne

ural

Push

(cm

/s)

Experiment Session Number

1 2 3 41 2 3 4−2

0

2

Quartile

1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 80

10

20

30

1 2 3 41 2 3 40

10

20

30

Experiment Session Number

Null:P

oten

t Rat

io

Task AxisPerturbation Axis

slope = 0.35/quartile, p=0.282 0.13/quartile, p=0.538

0.26/session, p=0.067

Perturb. Axis: -0.020/quartile, p=0.602Task Axis: 0.090/quartile, p=0.018

Perturb. Axis: 0.002/session, p=0.930Task Axis: -0.029/session, p=0.195

Perturb. Axis: -0.009/session, p=0.782Task Axis: -0.021/session, p=0.439

-0.046/quartile, p=0.353-0.059/quartile, p=0.198

0.03/session, p=0.916

A

B

Monkey J Monkey LJump at 3 cmJump at 6 cmJump at 10 cmJump during hold