trends in bcis for usdesa/brendanslides.pdf · and 2011 “trends in bci research”. video...

Trends in BCIs for us

Brendan Allison, PhD

University of California, San Diego

COGS 189

January 8, 2019

Brain-Computer-Interface (BCI) components

BCI components (Allison, 2011)

Applications: RISER model

From Wolpaw and Wolpaw (2012); since widely adopted

This broadens the applications of BCIs from the

definitions in the 2010 “Gentle Introduction” chapter

and 2011 “Trends in BCI Research”.

Video courtesy of G. Schalk.

Work is from Schalk et al., 2017.

Common view

BCI/AN is for users with disabilities

Sci-fi view

BCI and AN is from, and for, evil

Star Trek (1966)

This is still among the mostrealistic and altruistic

portrayals of BCIs.

Widespread BCI adoption requires replacing

conventional interfaces for conventional

users in conventional settings.

Common view

TU Graz,

2010

Replacing or supplementing conventional

interfaces for conventional users in

specific settings.

→ Any user may be like a disabled user due

to situational disability and laziness.

People might enjoy a BCI in some settings

just for fun.

Emerging view

When a healthy person cannot use some natural means of communication and control in a certain situation.

Leeb et al (2007), Faller et al (2010, 2017), Aloise et al (2011) – virtual navigation

Middendorf et al (2000), Trejo et al (2006), Menon al (2011) – pilots or astronauts

Pineda et al (2003), Nijholt et al (2008, 2011,2016), Navarro et al (2011) – gamers

Allison (2008, 2010, 2011), Millan et al (2010), Zander and Kothe (2011) – other examples

Also cellphone users, mechanics, surgeons, soldiers, drivers aka Tesla seatwarmers

And (sadly) disability by laziness: remote control or cellphone users

Trejo et al (2006) Scherer et al (2008)

Sit. Disability

Widespread BCI adoption requires

dramatically new capabilities, such that gel-

based wired “ugly” systems are appealing.

Common view

Disruptive TechnologiesPractical electrodesWireless systems

Emerging view

Cosmesis

PERCEIVED Cosmesis

Common view

Schalk (2008)

Moderately disabled

Eager healthy

Mainstream healthy

Speed = critical for broader adoption

Emerging view

Ease, design, utility = key for broader adoption

• Improved hardware (esp. wireless, dry electrodes)

• Easy, available software (BCI2000 and others)

• Easy, available systems and classes (Hack-a-thons, majors)

• Improved math and signal processing

• Improved knowledge of EEGs and cognitive correlates

• Wearable computing ubiquity

• Perceived cosmesis

• Positive media coverage of AN and related systems

Catalysts

Catalysts of Broader Adoption

Why not use something else?• Faster than an unavailable interface

– That requires impractical hardware

– That cannot be easily used

– That would take longer to provide the same information (if at all)

• Easier to use than other interfaces– More portable, accessible, or convenient in real world settings

– Induced disability is major, even from laziness (TV remote control)

– More natural and intuitive

– Less training?

• The only interface capable of total privacy?

• May seem novel or fun

How much useful additional information would remain in the output of the brain-, other body-organ- , or behavior-based workload gauges after regressing out variation directly measurable [from photodiodes, a software agent counting mouse clicks, and a microphone]?

Commentary on DARPA AugCog report by Gevins and Smith, 2003.

**Note: They referred to workload monitoring here.

Why not use something else?

• “Enhance” = User experience in games

• “Supplement” = Google Glass (this was 2015)

• “Improve” = Upper Limb therapy after stroke

• “Research tool” = Cognitive neuroscience research

Case Scenarios

Image Triaging• Images shown to a user 10x per second

– EEG can identify a minority of images that are of interest

– Could reduce time for photo analysis work

From Paul Sajda’s group

Neurofeedback• Has a “bad rep” due largely to unethical claims

– Done properly, NF is good for (at least) relaxation and attention

– Many, many companies offer NF.

– How can people identify the best ones?

From company websites

“Lie” detection• P300 and related signals can detect image familiarity

– Does the accused have “guilty knowledge”?

From Farwell and colleagues

NeuromarketingCan detect how focus groups react to sounds/images

Error detection• ERN = Error-related negativity

– Real-time error correction requires good single-trial performance. Most systems are integrated with BCIs – not other interfaces

– Non-real-time HCI, usability testing

– Still a popular topic at the BCI Meeting 2018

From Schalk et al. (2000)

Alertness monitoring• Based on only 2 EEG channels, 20 years ago

– Red = missed targets; green = hit targets

– Blue = actual error rate; red = predicted error rate

From Scott Makeig and T-P Jung

Task adaptation

From Alan Pope’s group

EEG-controlled World of Warcraft (g.tec)

4 controls:

Turn left, right, move forward, perform actions like grasping

objects, attacking other objects

60 Hz LCD display with 15, 12, 10 and 8.75 Hz.

BCI overlay based on OpenGL –

can be used with any graphics application

Game adaptation

From Ewing et al. (2016)

Necomimi

Brainball

Star Wars Force Trainer

Cockroach racing

Towards “Big BCI”

BCI Meeting

2018

Sponsors

Emerging “Big BCI”

Galvani

NeuralinkFacebook

Emerging “Big BCI”• Facebook

– Announced a noninvasive 100 word-per-minute BCI.

– They announced at our 2018 BCI Meeting that it will be in 10 years.

– What’s new: BCI? Optical imaging? Natural language processing?

• Neuralink– New effort popularized by Elon Musk

– Focuses on invasive approaches, including patients

Good progress on BCI training and infrastructure, but we need more:

• Classes, majors, degrees focused on BCI, neurotech

• Many classes on BCIs; NCAN summer schools

• Publicly available training videos, websites, exercises, lectures

• Quite a lot available online

• Open Access software for BCIs

• BCI2000, OpenVibe, OpenEEG, BF++, others

• Official conferences and societies

• BCI Conferences, BCI Meetings, BCI Society!!!

• Official journals and special issues

• BCI Journal, Special Issues in Frontiers

• Make BCIs fun and available

• Museums, hackathons

Public activities for students and enthusiasts to make their own BCI applications!

Wolpaw et al., 2002

Thanks to Prof. De Sa, and to all of you for coming!

For questions, including image credts, please email me: ballison@ucsd.edu