hacking brain computer interfaces
TRANSCRIPT
HackingBrain-Computer Interfaces
SMSI talk by Mike SchäkermannBerlin - March 17, 2015
Overview
1. About BCIs
2. OpenBCI
3. Live Demo
Overview
1. About BCIs
2. OpenBCI
3. Live Demo
Brain-Computer Interfaces
A brain-computer interface is a directcommunication pathway between the
brain and an external device.
Brain Imaging Techniques
EEG
fMRI
NIRS
Invasive techniques
Brain Imaging Techniques
EEG
fMRI
NIRS
Invasive techniques
Pros and Cons of EEG
High temporal resolution
Level of insight
Sensitivity
Low spatial resolution
Discomfort
Sensitivity
EEG Data: (1/3) Presentation
Time domain plot
Frequency domain plot
Spectrogram
Head plot / topography
EEG Data: (1/3) Presentation
TIME DOMAIN PLOT
FREQUENCY DOMAIN PLOT
HEAD PLOT
EEG Data: (1/3) PresentationSPECTROGRAM
EEG Data: (2/3) Preprocessing
Removal of noise / artifacts, caused by:
blinkslateral eye movementsmuscle activityhead/body movementelectrical currents (powerline interference)
EEG Data: (3/3) Analysis
Frequency analysis (Alpha, Beta, Gamma, Delta, Theta, Mu)
Event-related potentials (P300, ERN, and many more)
Steady-state topography (SSVEP, ASSR)
Others (Hemispheric asymmetry, Coherence)
EEG Data: (3/3) Analysis
Frequency analysis (Alpha, Beta, Gamma, Delta, Theta, Mu)
Event-related potentials (P300, ERN, and many more)
Steady-state topography (SSVEP, ASSR)
Others (Hemispheric asymmetry, Coherence)
EEG Data: (3/3) Analysis
GAMMA (> 31 Hz)
DETECTED OVER THE SOMATOSENSORY CORTEX
DURING CROSS-MODAL SENSORY PROCESSING
FREQUENCY ANALYSIS
EEG Data: (3/3) Analysis
BETA (15 - 31 Hz)
DETECTED SYMMETRICALLY IN FRONTAL REGIONS DURING
PHASES OF FOCUS AND CONCENTRATION
FREQUENCY ANALYSIS
EEG Data: (3/3) Analysis
ALPHA (8 - 15 Hz)
DETECTED IN POSTERIOR REGIONS DURING RELAXED
STATES AND WHILE EYES ARE CLOSED
FREQUENCY ANALYSIS
EEG Data: (3/3) Analysis
MU (8 - 12 Hz)
DETECTED OVER MOTOR CORTEX DURING IMAGINED
OR ACTUAL MOVEMENT
FREQUENCY ANALYSIS
EEG Data: (3/3) Analysis
THETA (4 - 7 Hz)
DETECTED DURING IDLING, RELAXED, MEDITATIVE AND
CREATIVE STATES
FREQUENCY ANALYSIS
EEG Data: (3/3) Analysis
DELTA (< 4 Hz)
DETECTED IN FRONTAL REGIONS DURING SLOW-
WAVE SLEEP + CONTINUOUS ATTENTION TASKS
FREQUENCY ANALYSIS
EEG Data: (3/3) Analysis
Frequency analysis (Alpha, Beta, Gamma, Delta, Theta, Mu)
Event-related potentials (P300, ERN, and many more)
Steady-state topography (SSVEP, ASSR)
Others (Hemispheric asymmetry, Coherence)
EEG Data: (3/3) AnalysisEVENT-RELATED
POTENTIALS (EX: P300)
EEG Data: (3/3) AnalysisEVENT-RELATED
POTENTIALS (EX: P300)
EEG Data: (3/3) AnalysisEVENT-RELATED
POTENTIALS (EX: P300)
EEG Data: (3/3) Analysis
Frequency analysis (Alpha, Beta, Gamma, Delta, Theta, Mu)
Event-related potentials (P300, ERN, and many more)
Steady-state topography (SSVEP, ASSR)
Others (Hemispheric asymmetry, Coherence)
EEG Data: (3/3) Analysis
Frequency analysis (Alpha, Beta, Gamma, Delta, Theta, Mu)
Event-related potentials (P300, ERN, and many more)
Steady-state topography (SSVEP, ASSR)
Others (Hemispheric asymmetry, Coherence)
Some fun examples
BrainBall: Relax to Win (1999)Based on alpha andtheta activation
BrainBall: Relax to Win (1999)Based on alpha andtheta activation
SharkAttack: mind-controlled shark (2015)Based on multiple people’s alpha activation
HexBug: mind-controlled robot (2015)
Based on alpha activation and visual entrainment
HexBug: mind-controlled robot (2015)
HexBug: mind-controlled robot (2015)
HexBug: mind-controlled robot (2015)
Unity3D SDKs for Emotiv and Neurosky
Overview
1. About BCIs
2. OpenBCI
3. Live Demo
Overview
1. About BCIs
2. OpenBCI
3. Live Demo
Crowdfunded open source consumer BCI
Open source hardware
Open source hardware
Open source software
Open source headware
EmotivNeuroSky
Muse
OpenEEGSystems for research or clinical use, cost thousands of dollars
Prices range from $50 to $300,for a comparison, see here
Bridging a gap ...
Open-source:
SoftwareHardwareHeadware
Access to raw data
High sample rate: 256 Hz
SDKs / connectors available for:
ProcessingPythonOpenVIBEBrainBayOpenFrameworks (coming soon)Node.js (coming soon)
Comes pre-assembled
Affordable price:
$450 for 8-channel kit$800 for 16-channel kit
Bridging a gap ...
Overview
1. About BCIs
2. OpenBCI
3. Live Demo
Live Demo
Alpha wave trigger for meditation music
Alpha wave trigger for meditation music
EYES OPEN
PLAY ENERGETIC MUSIC
Alpha wave trigger for meditation music
EYES OPEN
PLAY ENERGETIC MUSIC EYES CLOSED
PLAY CALMING MUSIC
Alpha wave trigger for meditation music
Alpha wave trigger for meditation music
Alpha wave trigger for meditation musicFUTURE WORK:
GUARD BANDS
Alpha wave trigger for meditation musicFUTURE WORK:
GUARD BANDS
Alpha wave trigger for meditation musicFUTURE WORK:
GUARD BANDS
Thanks for your attention!Questions, please!
References & Further ReadingOpenBCI (http://openbci.com/):
Docs:
Tutorials: http://docs.openbci.com/tutorials/01-GettingStarted
Software: http://docs.openbci.com/software/01-OpenBCI_SDK
Hardware: http://docs.openbci.com/hardware/01-OpenBCI_Hardware
Blogs (http://openbci.com/community/):
Omphaloskeptic: http://www.autodidacts.io/
Chip Audette: http://eeghacker.blogspot.com/
Conor Russomanno: http://conorrussomanno.com/
Jeremy Frey: http://blog.jfrey.info/
GitHub: https://github.com/OpenBCI
References & Further ReadingAmbinder, Mike. 2011. Biofeedback in Gameplay: How Valve Measures Physiology to Enhance Gaming Experience. Presentation at Game Developers Conference (GDC) 2011. http://www.gdcvault.com/play/1014734/Biofeedback-in-Gameplay-How-Valve.
Cacioppo, John T., Louis G. Tassinary, and Gary Berntson. 2007. Handbook of Psychophysiology. Cambridge University Press. isbn: 9780521844710.
Hakvoort, Gido, Hayrettin Gurkok, Danny Plass-Oude Bos, Michel Obbink, and Mannes Poel. 2011. Measuring immersion and affect in a brain-computer interface game. In Proceedings of the 13th ifip tc 13 international conference on human-computer interaction - volume part i, 115–128. INTERACT’11. Berlin, Heidelberg: Springer-Verlag. isbn: 978-3-642-23773-7, http://dl.acm.org/citation.cfm?id=2042053.2042069.
Kivikangas, J. Matias, Inger Ekman, Guillaume Chanel, Simo Järvelä, Ben Cowley, Pentti Henttonen, and Niklas Ravaja. 2010. Review on psychophysiological methods in game research. In Proc. of 1st nordic digra, digra.
Luck, Stephen J. 2005. An introduction to the event-related potential technique. Cognitive neuroscience. MIT Press. isbn: 9780262621960.
References & Further ReadingMandryk, Regan. 2008. Physiological Measures for Game Evaluation. In Game usability:
advice from the experts for advancing the player experience.
Nacke, Lennart E. 2011. Directions in Physiological Game Evaluation and Interaction.
In chi 2011 bbi workshop proceedings.
———. 2013. An Introduction to Physiological Player Metrics for Evaluating Games. Chap. 26 in Game analytics: maximizing the value of player data, 585–620. London: Springer-Verlag. isbn: 978-1-4471-4768-8, doi:10.1007/978-1-4471-4769-5.
Nacke, Lennart E., Mark N. Grimshaw, and Craig A. Lindley. 2010. More Than a Feeling: Measurement of Sonic User Experience and Psychophysiology in a First-person Shooter Game. Interact. Comput. (New York, NY, USA) 22, no. 5 (Sept.): 336–343. issn: 0953-5438, doi:10.1016/j.intcom.2010.04.005, http://dx.doi.org/10. 1016/j.intcom.2010.04.005.
Wehbe, Rina R., Dennis L. Kappen, David. Rojas, Matthias. Klauser, Bill. Kapralos, and Lennart E. Nacke. 2013. EEG-based Assessment of Video and In-game Learning. In Chi ’13 extended abstracts on human factors in computing systems, 667–672. CHI EA ’13. New York, NY, USA: ACM. isbn: 978-1-4503-1952-2, doi:10.1145/2468356.2468474, http://doi.acm.org/10.1145/2468356.2468474.