Brain Products Press Release December 2010, Volume 37

www.brainproducts.com

>> Introduction

In his 1973 publication ‚Towards direct brain-computer

communication‘, Jacques J. Vidal outlined the notion of

communication between a human being and a technical system

that is based directly on neural activity. Since then many ideas,

approaches and publications inspired by his proposition have

followed, and the term Brain-Computer Interface (BCI) has

entered into common use. Some other terms, like Brain-Machine

Interface (BMI), Direct Neural Interface (DNI) and Brain/Neuronal

Computer Interaction (BNCI), are also used to describe this

approach. One of the most well-known BCI-based systems is the

P300 Speller developed by Farwell and Donchin.

Figure 1: The approach used by the P300 Speller is very common in BCI research. Usually a matrix of 6 rows and 6 columns displaying all 26 letters of the alphabet plus the 10 numerals is presented to the subject. While the subject focuses his or her attention on a single letter or numeral, the rows and columns are randomly highlighted several times. The highlighting of the symbol being focused on deviates from the previous perception and should evoke (at minimum) a more pronounced novelty-related potential than the subject’s peripheral perception of the other flashes. Taking this assumption as its starting point, the BCI system then selects the symbol corresponding to the highest amplitude in the averaged positive component.

The P300 Speller’s approach utilizes brain signals indirectly via

a reaction to an external stimulus. Hence the signal detected by

the BCI is not directly generated by the user. Other approaches

that involve a BCI-based speller use directly generated brain

signals, often involving motor imagery. One example is the Hex-

O-Spell developed by the Berlin Brain-Computer Interface group.

Until now, research in the field of BCI has mainly focused on

applications to provide severely disabled users with new

channels for communication and control. In this article we

will discuss the capabilities of the

currently available BCI technology. I

will start with an overview of recent

developments in the field of BCI

research.

>> Timeline of BCI research

The timeline of research into BCI can be divided into two

successive phases. The first has chiefly been influenced by

researchers working in psychology and neuroscience who

established the main purpose of classical BCI research. This

consisted of building up communication channels based solely on

brain activity for the purpose of supporting patients. The second

phase has been shaped by the introduction into the BCI field of

modern signal processing and machine learning techniques. The

great efforts made by researchers in the fields of mathematics

and computer science have enabled a major breakthrough in the

applicability and performance of BCI systems.

>> The era of user-trained BCI systems

This era started in the early 1980s with work by several

researchers on BCI projects that has mainly focused on defining

new communication channels for severely disabled people.

Some research groups have achieved major advances. In the

following I will briefly discuss the results and approaches of

three groups that have significantly influenced the direction of

BCI research.

One research group was led by Gert Pfurtscheller at the

Technical University of Graz, Austria. It is currently being led

by Christa Neuper. Their investigation of changes in frequency

bands induced by imaginary limb movements has been highly

inspirational for subsequent research. Their approach was

initially based on the usage of a small number of electrodes

and simple data processing but was later extended to larger and

more complex sets. Its main assumption is that motor imagery

leads to a change in frequency recorded at specific electrode

sites which is consistent across subjects and is easy for any

subject to control.

A second group, led by Niels Birbaumer at the University of

Tuebingen, Germany, trained their subjects to control the polarity

of the EEG signals produced by specific areas of their brains. The

resulting slow waves were used as input for a spelling device.

The approach involving the utilization of features from EEG

time series as input commands has subsequently led to several

different approaches for designing BCI-based communication

devices.

A third group, led by Jonathan R. Wolpaw of the Wadsworth

Center, USA, has utilized several BCI technology-based

Did you know ...!?

Brain Computer Interfaces (Part 2) by Thorsten Zander

Brain Products Press Release December 2010, Volume 37

www.brainproducts.com

approaches for developing communication and control systems.

The theory emerging from BCI research has been strongly

influenced by the work on definitions carried out by Jonathan

Wolpaw.

In principle, all these approaches have given the users of the

respective systems basic control over them. However, this could

only be achieved through a tremendous effort on the part of the

user that required several month of training.

>> The era of machine learning-based BCI systems

The end of the 1990s saw the emergence of a new approach to

BCI-based applications. The main change was the introduction

of more complex data analysis methods that allowed subject-

specific patterns in the brain to be detected. This accordingly

made it possible to detect patterns induced by natural brain

states in a highly reliable way. The main drawback of these

methods is that the calibration of the system requires a

significant amount of data for the generation of the prototypes

of the expected signals. Due to the high degree of variance

between data sets from different EEG recording sessions – even

for the same subject – this calibration procedure usually has

to be repeated at the beginning of every BCI session. However,

once the system has been calibrated, the effort needed from

the subject to control a device via a BCI channel is greatly

reduced. In this way, the burden involved in realizing reliable

BCI communication has switched from the user to the machine.

The above-mentioned group led by Prof. Pfurtscheller has

developed and implemented several algorithms for detecting

specific patterns connected with the imagination of limb

movements. The most familiar of these is the optimization

method known as Common Spatial Patterns (CSP), which has

given rise to several more complex derivates.

Using this method, Pfurtscheller’s group has focused on

applications for severely disabled persons. The newest approach

developed by Pfurtscheller’s group is known as Hybrid BCI and

combines BCI with at least one additional input stream.

The group led by Prof. Jose del Milan in Martigny, Switzerland

has also utilized several methods for machine learning-based

BCI approaches that have mainly focused on event detection

and Gaussian Mixture models. His group too has focused on new

applications, like shared robotic control, adding a secondary BCI

channel (e.g. for error correction), or concepts for non-medical

applications of BCI.

The largest input into the development of machine learning-

based BCIs has come from Klaus-Robert Mueller‘s group

at the Berlin Brain-Computer Interface, Berlin, Germany.

Here, multiple methods have been developed and applied

in different application scenarios (see www.bbci.de).

>> Main goals of current BCI research

From the time it was first defined, BCI research has focused on

the development of systems for supporting severely disabled

patients suffering from conditions like amyotrophic lateral

sclerosis (ALS), quadriplegia or other impairments that restrict

natural communication channels. The main goal has been to

restore some of the independence taken away by the symptoms

of the impairment in question.

One important challenge is to construct systems that provide

asynchronous communication which allows a user to decide

when information is exchanged. By contrast, most laboratory

equipment operates at the pace of the system: stimuli are

presented at given points in time, and corresponding responses

are required from the subjects. In typical human-machine

interactions the user can set the pace by manually pressing

buttons that initiate interaction cycles. But this becomes

difficult when it is limited to BCI communication. Some signals,

like event-related potentials (ERPs), relate directly to external

events. Additionally, not all signals are reliably detectable,

leading to delays in interaction or to false-positive selections.

For long-term use, a so-called ‘brain switch’ is needed. This

consists of a secondary BCI-based input channel defined

in relation to one of the most reliably detectable signals that

indicates whether or not the subject wants to communicate via

the main BCI channel. The main issue is the definition of the

underlying accurately detectable brain signal.

Additionally, most algorithms used are predicated on the

assumption that the underlying data is stationary, i.e. that their

statistical properties do not vary over time. But partly due to

its plasticity – in other words, its capability for learning and

adapting – the human brain generates highly non-stationary

data. Fortunately, the data produced by an EEG system in a

given working scenario is piecewise stationary, i.e. changes in

the statistical properties are bounded in a particular, restricted

time frame.

Another branch of BCI research focuses on developing new kinds

of data acquisition. The main goals here are to reduce the effort

needed for implementing the sensor system, improving the

quality of the recordings, and improving the acceptance of the

equipment by users.

>> The current state of the art

To what degree have the goals of BCI been achieved in the past

three decades?

Currently, a few locked-in patients are working with BCI-based

systems. Some of these systems use invasive techniques, while

others rely on EEG-based methods even fMRI, MEG and NIRS have

been used. But applying a BCI to those individuals who most

stand to benefit, namely completely locked-in patients, is still

the most complicated challenge. Advances in combining BCI-

based input with robotic support or prostheses can potentially

be achieved. But the fact must be faced that there still is a huge

difference between laboratory studies using healthy volunteers

Brain Products Press Release December 2010, Volume 37

www.brainproducts.com

and real-world applications involving actual patients. In this

connection, the definition of a proper ’brain switch’ would be

very helpful. Most of the initial steps towards such a system

have been taken by Gert Pfurtscheller’s group. Currently, a novel

hybrid approach combining a NIRS-based brain switch with a

motor imagery BCI has recently been presented. However, its

applicability in real-world scenarios still has to be proven.

Even though a BCI-based system might work, there are still major

drawbacks to be overcome. The fastest non-invasive systems

provide bit rates of several dozen bits/min.; in an invasive

scenario it might be several times greater than that.

While it represents a big step away from a total lack of

communication, this is still very limited compared to the bit

rate that prevails in standard communication systems. One

reason for this is the fact of each BCI channel usually being

limited to binary communication. Approaches developed in

user-trained BCI systems have led to the possibility of eight-bit

communication, but these have been felt to be highly exhausting

and would therefore only be feasible for short-term usage.

Additionally, due to the assumption of statistical stationarity of

the data in relation to the machine learning algorithms used, the

accuracy of a BCI system is not robust overtime. Fluctuations

in the performance of the systems used have been reported

that require the latter to be recalibrated. A different solution

to this problem could involve employing an auto-adaptive

system that continuously learns while it is in use.

But here the resulting human-machine system would consist of

at least two learning systems: the BCI classifier and the human

brain. It would be necessary to ensure the convergence of their

learning, as otherwise the result could be a vicious circle that

produces a downward spiral in terms of performance and effort.

Even with non-invasive data acquisition, long-term use might

lead to disadvantages for the user. The abrasive gel used with

EEG electrodes could lead to painful skin irritation. One solution

to this problem might be to develop dry electrode systems. The

first dry electrode prototypes have recently been developed,

including a highly promising one from Brain Products.

While this article has focused on the central applications and

questions that BCI research has been addressing in recent

decades, in the next installment of the series on BCI (to be

published in the next press release) we will discuss novel

approaches in BCI research that will allow BCI technology to

enter other application areas, mainly in the fields of human-

machine systems and cognitive neuroscience.