integrated brain-machine-body interfacess3.amazonaws.com/sdieee/210-integrated+brain... · gert...

Gert Cauwenberghs [email protected] Integrated Brain-Machine-Body Interfaces

Integrated Brain-Machine-Body Interfaces

Gert Cauwenberghs Department of Bioengineering

Institute for Neural Computation UC San Diego

http://isn.ucsd.edu


Integrated Systems Neuroengineering

Silicon Microchips

Neural Systems

Neuromorphic/ Neurosystems

Engineering

Learning &

Adaptation

Environment Human/Bio Interaction

Sensors and Actuators

METRIC fitness function Q

MIMO parameters !"

thalamocortical/BG model

EE

G

EM

G, k

inet

ics,

gaz

e

force

PD markers

MoBI

MoCap

synaptic plasticity

CyberGlove

PNS

PNS

PNS PNS

CNS

CNS

CNS

adaptive control

MoCap

Computational modeling

G. Cauwenberghs, K. Kreutz-Delgado, T.P. Jung, S. Makeig, H. Poizner, T. Sejnowski, F. Broccard, D. Peterson, M. Arnold, A. Akinin, C. Stevenson, J. Menon

Distributed Brain Dynamics of Human Motor Control NSF EFRI 2012 – Mind, Machines and Motor Control (M3C)

IFAT

IFAT IFAT

IFAT

SRT

SRT

SRT

SRT SRT

SRT

SRT

SRT

SRT SRT

Level 1 HiAER

Level 1 HiAER

Level 1 HiAER

Level 1 HiAER

Level 2 HiAER

Connector

JTAG JTAG

JTAG

JTAG

EEG brain dynamics and Parkinson s

Force feedback

Neuromorphic emulation of brain dynamics in motor control

MoBI


Neuromorphic Engineering in silico neural systems design

VLSI Microchips

Neuromorphic Engineering

Neural Systems

Learning &

Adaptation

g 1 g 0

g 2 g 0

g 0

g 2 g 2

g 1

g 1


Silicon Model of Visual Cortical Processing

g!1!g!0!

g!2!g!0!

g!0!

g!2!g!2!

g!1!

g!1!

C!0!

C!+z! C!+y!

C!+x!C!-x!

C!-y! C!-z!

I!0!

I!0!

I!0!I!0!I!0!

I!0!I!+y!I!+z!

I!-x!I!+x!

I!-y! I!-z!LGN

V2

6

4

3

2

6

V1

Optic Nerve

Single-chip focal-plane implementation (Cauwenberghs and Waskiewicz, 1999)

Bipole cells (diffusive network)

Complex and hypercomplex

cells (lateral

inhibition)

Neural model of boundary contour representation in V1, one orientation shown (Grossberg, Mingolla, and Williamson, 1997)

88 transistors/pixel (including

photodetector)


Silicon Learning Machines for Embedded Sensor Adaptive Intelligence

ASP A/D Sensory Features

Digital Analog

Large-Margin Kernel Regression Class Identification

Kerneltron: massively parallel support vector machine (SVM) in

silicon (JSSC 2007)

MAP Forward Decoding Sequence Identification

Sub-microwatt speaker verification and phoneme recognition (NIPS 2004)

GiniSVM


Kerneltron: Adiabatic Support Vector Machine Karakiewicz, Genov and Cauwenberghs , 2007

•! 1.2 TMACS / mW –! adiabatic resonant clocking

conserves charge energy –! energy efficiency on par with

human brain (1015 SynOP/S at 15W)

Karakiewicz, Genov, and Cauwenberghs, VLSI 2006; CICC 2007

x i

x SIGN

! i MVM

SUPPORT VECTORS

INPUT y

KE

RN

EL K

(x ,x i )

)),((sign bKyySi

iii += !"

xx#

Classification results on MIT CBCL face detection data

resonance

capacitive load


Silicon support vector machine (SVM) and forward decoding kernel machine (FDKM)

x s

x NORMALIZATION

! i1 s

14 24x24

30x24 30x24

1 2 24

" j[n-1] " i[n] 24

MVM MVM

SUPPORT VECTORS

INPUT f i1 (x)

24 FORWARD DECODING

P i1 P i2 4 K

ER

NE

L K(x,x s )

24x24

Forward decoding MAP sequence estimation Biometric verification

840 nW power

Sub-Micropower Analog VLSI Adaptive Sequence Decoding Chakrabartty and Cauwenberghs , 2004

GiniSVM

X[n] X[n-1] X[n+1]

j!

i!


Neuron-Silicon and Brain-Machine Interfaces

MicropowerMixed-Signal

VLSI Neuro

Bio

Neurosystems Engineering

Biosensors,

Neural Prostheses and Brain Interfaces

Adaptive Sensory Feature

Extraction and Pattern Recognition

Neuromorphic Engineering

Learning &

Adaptation


Brain Machine Interfaces and Motor Control

•! The brain s motor commands ! –! Parietal/frontal cortex

•! Implanted electrodes •! Electroencephalogram (EEG)

–! Cortical signals, noninvasive –! Low bandwidth (seconds)

–! Nerve signals •! Spinal cord electrodes •! Electromyogram (EMG)

–! Muscle signals, noninvasive –! Higher bandwidth (milliseconds)

! translated into motor actions –! Machine learning/signal processing –! Neuromorphic approaches

•! Central pattern generators (CPGs)

Nicolelis, Nature Rev. Neuroscience 4, 417, 2003


Wireless Non-Invasive, Orthotic Brain Machine Interfaces

–! Mind-machine interfaces for augmented human-computer interaction

–! Body sensor networks for mobile health monitoring and augmented situation awareness

Calit2 StarCAVE immersive 3-D virtual reality environment Yu Mike Chi, 2010 TATRC Grand Challenge


Scalp EEG Recording

•! State of the art EEG recording –! 32-256 channels –! Gel contact electrodes –! Tethered to acquisition box –! Off-line analysis

BioSemi Active2 www.biosemi.com


Envisioned High-Res EEG/ICA Neurotechnology

RF Wireless Link

EEG/ICA Silicon Die

Dry Electrode

Flex Printed Circuit

T.J. Sullivan, S.R. Deiss, T.-P. Jung, and G. Cauwenberghs, 2008

•! Integrated EEG/ICA wireless EEG recording system –! Scalable towards 1000+ channels –! Dry-contact MEMS electrodes (NCTU, Taiwan) –! Wireless, lightweight –! Integrated, distributed independent component analysis (ICA)


Independent Component Analysis •! The task of blind source separation (BSS) is to separate and recover

independent sources from (instantaneously) mixed sensor observations, where both the sources and mixing matrix are unknown.

•! Independent component analysis (ICA) minimizes higher-order statistical dependencies between reconstructed signals to estimate the unmixing matrix.

•! Columns of the unmixing matrix yield the spatial profiles for each of the estimated sources of brain activities, projected onto the scalp map (sensor locations). Inverse methods yield estimates for the location of the centers of each of the dipole sources.

A W s(t)

M N N

x(t) y(t)

Source signals Sensor

observations Reconstructed source signals

Mixing matrix Unmixing matrix s1

s2

x1

x2 x3


EEG Independent Component Analysis

–! ICA on single-trial EEG array data identifies and localizes sources of brain activity.

–! ICA can also be used to identify and remove unwanted biopotential signals and other artifacts. •! EMG muscle activity •! 60Hz line noise

Left: 5 seconds of EEG containing eye movement artifacts. Center: Time courses and scalp maps of 5 independent component processes, extracted from the data by decomposing 3 minutes of 31-channel EEG data from the same session and then

applied to the same 5-s data epoch. The scalp maps show the projections of lateral eye movement and eye blink (top 2) and temporal muscle artifacts (bottom 3) to the scalp signals. Right: The same 5 s of data with the five mapped component

processes removed from the data [Jung et al., 2000].

Swartz Center for Computational Neuroscience, UCSD http://sccn.ucsd.edu/


Envisioned High-Density EEG Embedded in Elastic Fabric

•! Non-contact electrode –! No skin/subject

preparation –! Insulated, embeddable

in elastic fabric

•! Fully integrated –! On board power, signal

processing, wireless transceiver

•! Applications –! Brain computer interface –! Mobile, health

monitoring


Wireless Non-Contact Biopotential Sensors Mike Yu Chi and Gert Cauwenberghs, 2010

EEG alpha and eye blink activity recorded on the occipital lobe over

haired skull


[1] C.J. Harland, T.D. Clark, and R.J. Prance. Electric potential probes - new directions in the remote sensing of the human body. Measurement Science and Technology, 2:163–169, February 2002. [2] A. Lopez and P. C. Richardson. Capacitive electrocardiographic and bioelectric electrodes. IEEE Transactions on Biomedical Engineering, 16:299–300, 1969. [3] P. Park, P.H. Chou, Y. Bai, R. Matthews, and A. Hibbs. An ultra- wearable, wireless, low power ECG monitoring system. Proc. IEEE International Conference on Complex Medical Engineering, pages 241–244, Nov 2006.

Capacitive Non-Contact Electrodes •! Senses biopotential signals without

contact –! Capacitive signal coupling –! No electro-gel –! Through clothing and hair

•! Basic idea is well-known –! First patent in 1968 (Richardson) –! Several groups (Prance) and one company

(Quasar) have pursued this

•! Technology still problematic –! Noise, interference pickup, artifacts –! Circuit complexity, materials, construction,

cost –! Nothing beyond lab prototype


Challenges in Non-Contact Sensors

•! Amplifier parasitic input capacitance –! Reduces gain as electrode-skin distance changes –! Severely degrades CMRR –! Increases the effect of amplifier voltage noise

•! Integrates current noise at biopotential signal frequencies –! Amplifier input biasing –! Large resistance required for adequate low frequency response adds

further current noise

Capacitive coupling, rather than ohmic contact, between

scalp/skin and electrode#

skull

skin

unity gain buffer amplifier active

shield

electrode


Non-Contact Sensor Noise


•! Non-contact sensor fabricated on a printed circuit board substrate

•! Advantages: –! Robust circuit –! Inexpensive production –! Safe, no sharp edges or fingers, can be made flexible –! Very low power (<100µW/sensor) –! Strong immunity to external noise

Standard 4-layer PCB!

Sensing Plate!

Active Shield!

Amplifier!

Non-Contact Sensor Design

Chi and Cauwenberghs, 2010


EEG Hand-band! ECG Chest Harness! Electronics!

Wearable Wireless EEG/ECG System

•! Prototype non-contact sensor system with 4-channels –! Bluetooth wireless telemetry and microSD data storage –! Rechargeable battery

•! Mounted in both head and chest harnesses

Y. M. Chi, E. Kang, J. Kang, J. Fang and G. Cauwenberghs, 2010


Simultaneously acquired ECG in laboratory setting!No 60Hz Filter!

ECG Comparison



Derived 12-lead ECG from 4 electrodes mounted in chest harness!

Sample ECG Data



Sitting! Walking!

Running! Jumping!

ECG Under Motion



Non-Contact EEG Recording over Haired Scalp

•! Easy access to hair-covered areas of the head without gels or slap-contact

•! EEG data available only from the posterior –! P300 (Brain-computer control, memory

recognition) –! SSVP (Brain-computer control)



Subject s eyes closed showing alpha wave activity!Full bandwidth, unfiltered, signal show (.5-100Hz)!

Non-Contact vs. Ag/AgCl Comparison Y. M. Chi, E. Kang, J. Kang, J. Fang and G. Cauwenberghs, 2010


Wireless Interfaces

Digitization Wireless Telemetry

Energy and noise efficiency metrics


Energy and Noise Efficiency Metrics

•! Noise Efficiency Factor (NEF): –! Relative measure of energy cost of a biopotential amplifier,

relative to that of an ideal amplifier with same input referred noise power

–! Thermal noise fundamental limit: NEF = 1 –! Practical limit for CMRR > 80 dB: NEF > 2 (2.3 demonstrated)

•! Energy per Conversion Level Figure of Merit (FoM): –! Energy cost of an analog-to-digital converter, per conversion, and

divided by the number of quantization levels –! State of the art: FoM = ~ 10 fJ at 10b and 100ksps

•! Range Efficiency: –! Energy per bit, per squared meter of wireless transmission –! Depends on target BER and power at the receiver –! State of the art: ~ 10 fJ/m2


EEG/ECoG/EMG Amplification, Filtering and Quantization Mollazadeh, Murari, Cauwenberghs and Thakor (2009)

–! Low noise •! 21nV/!Hz input-referred noise •! 2.0µVrms over 0.2Hz-8.2kHz

–! Low power •! 100µW per channel at 3.3V

–! Reconfigurable •! 0.2-94Hz highpass, analog adjustable •! 140Hz-8.2kHz lowpass, analog

adjustable •! 34dB-94dB gain, digitally selectable

–! High density •! 16 channels •! 3.3mm X 3.3mm in 0.5µm 2P3M CMOS •! 0.33 sq. mm per channel


Implantable Wireless Telemetry and Energy Harvesting

•! Transcutaneous wires limit the application of implantable sensing/actuation technology to neural prostheses –! Risk of infection

•! Opening through the skin reduces the body s natural defense against invading microorganisms

–! Limited mobility •! Tethered to power source and data logging instrumentation

•! Wireless technology is widely available, however: –! Frequency range of radio transmission is limited by the body s

absorption spectra and safety considerations •! Magnetic (inductive) coupling at low frequency, ~1-4 MHz •! Very low transmitted power requires efficient low-power design

Sauer, Stanacevic, Cauwenberghs, and Thakor, 2005


Sensor Interface Conditioning Telemetry Sauer, Stanacevic, Cauwenberghs, and Thakor (2005)

Regulation

Modulation Data Encoding

Clock Extraction CLK

VDD

GND

Data

Data

Power

Data Receiver

Rectification Power Transmitter

Biopotential acquisition

Telemetry

Inductor Coil

Electrodes

SoS released probe body

Implantable probe with biopotential electrodes, VLSI acquisition, microbatteries, and power harvesting telemetry chip.

Power delivery and data transmission over the same inductive link

Telemetry chip (1.5mm X 1.5mm)


Silicon-on-Sapphire (SOS) Ultra-Wide Band (UWB) RF Transmission

Tang, Andreou, and Culurcielo (2009) •! Pulse-based UWB radio transmitter

–! operates with sub-milliwatt power at multi-megahertz data rates and at microwatts of power for kilohertz data rates

–! body area networks and sensor networks

•! Implemented in silicon-on-sapphire (SOS) process –! optimizes its operation at high-speed and low-power consumption

UWB transmitter integrated circuit in silion-on-sapphire (SOS)

UWB transmitter antenna


Emerging Technologies •! Alternatives to EEG Wireless Brain Interfaces

–! NIR (near infrared spectroscopy) –! Miniaturized fMRI (functional magnetic resonance

imaging) –! Miniaturized MEG (magnetoencephalography)

•! Optogenetics –! ChR2 optical activation of targeted neurons –! NPhR optical inactivation of targeted neurons

•! Others !


180 !m

Minute 0 Minute 12 Minute 30 Minute 60

CMOS Imaging in Awake Behaving Rats Murari, Etienne-Cummings, Cauwenberghs, and Thakor (2010)

–! First simultaneous behavioral and cortical imaging from untethered, freely-moving rats.


Integrated Systems Neuroengineering

Silicon Microchips

Neural Systems

Neuromorphic/ Neurosystems

Engineering

Learning &

Adaptation

Environment Human/Bio Interaction

Sensors and Actuators

integrated brain-machine-body interfacess3.amazonaws.com/sdieee/210-integrated+brain... · gert...

Documents