iv. bci software ch 20. biosig : an open-source software library for bci research ch 21. bci2000 : a...
TRANSCRIPT
IV. BCI Software
CH 20. BioSig : An Open-Source Software Library for BCI Re-
search
CH 21. BCI2000
: A General-Purpose Software Platform for BCI
2010.11.19. JO, INAH
Introduction Introduce two software packages for BCI
◦ Freely available◦ Include a set of useful tools to process EEG signals for real-time
BCI Chapter 20 : Graz BioSig toolbox.
◦ Interface to many different data formats of EEG recordings◦ Supports several-signal processing◦ Artifact reduction◦ Feature extraction◦ Classification methods
Chapter 21 : BCI2000 toolbox◦ Wadsworth center in Albany , the University of Tubingen.◦ Mainly for real-time BCI application◦ General data acquisition techniques and a general framework◦ Freely available◦ Windows platforms and C++ programming.
BioSig An Open-Source software Library for BCI research
Chapter 20.
1. Abstract Common interface to many different data formats Artifact processing, quality control Adaptive signal processing, Feature extraction methods :
very suitable for online and real-time processing. Supports handling of missing values. Includes several classification methods Overview of the current status and an outline of future pos-
sibilities. Licensed with the GNU General Public License.(GNU GPL or
GPL) Open development platform
: ideal tool for research and development.
2. OverviewCurrent, available open-source software tools
◦ BioSig, OpenEEG, Bioelectromagnetismtoolbox,
EMEGS, FieldTrip, LORETA,BCI2000◦ GPL(General Public License) : free run, copy, distribute
study, change,
and improve the software is available
A Major aim : Provide an open-source software library for BCI research
Available from http://biosig.sf.net
3. BioSig for Octave and MAT-LAB
The BioSig project was started as a library for Matlab with an attempt to be compatible with Octave ( http://www.octave.org )
SubProject(rtsBCI, C/C++, SigViewer, etc)
=> Renamed as “Biosig for octave and MATLAB”
3-1. Data Formats and Storage
• Currently more than 40 different data formats.
• Common interface to various data formats
• Automated detection of the file format
• Do not need to care • which data format is actually used • how the proper scaling of the amplitude values is obtained • From where to obtain the sampling rate.
• Supports also simple preprocessing • selection of specific data channels, overflow detection
• However, none of these data formats contains all the features needed for BCI research.
• GDF(General Data Format):• Constructed with advantages of various data formats for biomedical signal pro-
cessing
3-1. Data Formats and Storage
GDF
◦ table20.3◦ Codes are grouped
According to different application areas
◦ EEG artifact code range:
0x0100 – 0x010f
◦ BCI experiment code range
: 0x0300 – 0x03ff
3-2. Preprocessing Intermediate state between data access and fea-
ture extraction.
Functions◦ Triggering the data : to extract data segments relative to a given
trigger
◦ Extracting the trigger time points from a continuous trigger channel
◦ Advanced algorithms for artifact processing quality control method(histogram analysis), reducing EOG artifacts algorithm(based on regression analysis), detecting muscle artifacts(based on inverse filtering)
Many signal processing methods are available Tool for handling of missing data is rare. Exception : TSA(time series analysis) toolbox, NaN toolbox,
part of OctaveForge, distributed with BioSig.
◦ NaN toolbox Represent missing values as not a number according to IEEE754 sumskipnan.m : skips all NaNs, and sums and counts all numerical val-
ues. Return value SUM, COUNT : calculate the mean of the data. expectation value of any statistic including covariance matrices and
more advanced statistics. more advanced ? Investigated.
◦ TSA(Time Series Analysis) toolbox Many functions for parametric modeling(autoregressive estimator) Burg and Levinson-Durbin algorithms for estimating autoregressive
parameter Adaptive autoregressive estimators like Kalman filtering RLS, LMS algorithms Multivariate autoregressive estimators like the Nuttal-strand method Multichannel Levinson algorithm
3-3. Signal Processing and Feature Extraction
3-4. Classification Various need for classifier
◦ Combines various features into a single output
◦ cross validation to prevent overfitting
◦ Time lag of classification should be as small as possible.
◦ Optimal time point for applying the classifier must be identified.
Wrapper function(findclassifier.m)◦ First version : LDA for two classes
◦ Later : Quadratic classifier, svm for more than 2 classes
◦ Selection of the Proper time segment for estimating a classifier. Data were triggered, seperability were calculated for each time segment. Length of each segment can be varied from 1 to N samples Typically, fixed segment length of a fraction of a second is used.
Cross validation : trial based LOOM(Leave-One-Out Method)◦ Samples from the chosen segment of all but one trial
◦ This classifier used to all samples of testing trials.
◦ Repeated until each trial has been used once as test trial.
◦ used in the BCI competition 2005 on data Set I(rank7),Iva(rank 8)
3-5. Evaluation Criteria Different evaluation criteria
◦ accuracy(error rate), information transfer, cohen’s kappa coef-ficient, others.
◦ Analyzing the time courses of the different criteria are provided.
◦ These functions were used for analyzing results of the BCI competition 2003(datasetIII), BCI competition 2005(datasetIIIa,IIIb),
Plota.m(Wrapper funtion)◦ for displaying various results.
◦ Generate a certain result depends on input data
◦ ex> outcome of findclassifer.m for two classes
: time course of the error rate and SNR
◦ ex > outcome of findclassifier.m for N>2 classes
: time course of the accuracy and the kappa coefficient.
◦
Sview ◦ Present the Raw EEG data
◦ in a compressed form on a single screen.
- Useful to get an overview of the available data
- Scoring of EEG data.
- Two packages : Sviewer, SigViewer.
3-6. Visualization
3-7. SViewer : a viewing and scoring software
Written in MatlabCommon interface
◦ Can read any data format supported by BioSig for Octave and Matlab
annotation, markers, event can be edited.
Not a stand-alone program
4. Real-time BCI System Based on MATLAB/Simulink-rtsBCI
rtsBCI : framework for the development and rapid prototyp-ing of real-time BCI system
To enable hard real-time computing on Windows platforms◦ Real-time Windows Target(RTWT)◦ Real-Time Workshop(RTW)◦ BioSig for Octave and Matlab(for data format handling)◦ TCP/UDP/IP toolbox(for network communication support)
Data Acquisition Card is required to sample the biosignals.
Licensed under GNU GPL Based on Matlab/Simulink, running on Microsoft Windows Used toolbox
◦ Real-time Windows Target(enable hard real-time computing on win-dows platforms)
◦ Real-Time Workshop(for the generation of stand-alone C code)◦ Tunable parameters: INI file
5. SigViewerViewing and scoring program for biosignals.Multichannel signal(EEG,ECT,EMG,EOG)Scoring : user can annotate to the signal written in C++
◦ Able to be run under Linux, windows, MacOS◦ designed as platform-independent
stand alone program Now only GDF file, but
can be extended to other file format
6. BioProFeed Data acquisition module Can be used with following 3 data acquisition hardware now
◦ G. mobilab from g.tec
◦ NI-60243 PCMCIA from National Instruments
◦ ModularEEG from OpenEEG
The name “BioProFeed”Stand for◦ Portable Biosignal acquisition
◦ Real-time processing
◦ Feedback generation framework
Part of BioSig project, it is a software framework for processing and storage unit.
Specifications◦ Platform independent
◦ Support different data acquisition hardware
◦ Real-time data processing should be possible
◦ Storage of the data in GDF format should be possible
Runs on : Linux, MacOs, Windows XP with Cygwin Autotools are used to build software framework
7. Summary Plan to improve Almost every part of BioSig
Artifact processing, quality control should be evaluated offline -> on-line.
Additional signal processing, feature extraction methods need to be developed including the ability to handle missing value.
Further data formats should be supported in SigViewer.
Biosig contains modules for data acquisition, BCI experiments with on-line feedback, viewing and scoring software for reviewing the raw data, functions for offline analysis of BCI data, obtaining a classifier.
Almost all components can be used with free software. GNU GPL
Chap 21.
BCI2000 A General-Purpose software Plat-form for BCI
1. Abstract BCI2000 is a flexible general-purpose platform for brain-com-
puter interface research Aim : Reducing the complexity and cost of implementing BCI sys-
tems. Joint Project : since 2000 Wadsworth center of the New York State Department of Health in Albany, New York
Institute of Medical Psychology and Behavioral Neurobiology at the University of Tub-ingen, Germany.
Used in more than 110 Laboratories around the world BCI2000 Supports
◦ Variety of data acquisition systems
◦ Brain signals
◦ Feedback modalities
By reducing labor and cost
=> different BCI system, psychophysiological experiment http://www.bci2000.org : freely available!
2. Overview Goals of the BCI2000 Project:
◦ 1. To create a system that can facilitate the implementation of any BCI system
◦ 2. to incorporate into this system support for the most commonly used BCI methods
◦ 3. to disseminate the system and associated documentation to other laboratories.
◦ Reduce time, effort, expense of testing new BCI methods, by providing a standard-ized data format, engage BCI research.
2007.02, 110 Labs Consist of 4 modules (independent and interchangeable modules)
◦ 1. Signal acquisition
◦ 2. Signal processing
◦ 3. User feedback
◦ 4. Operating protocol
To date,◦ Variety of data acquisition
Sensorimotor rhythms, cortical surface rhythms, slow cortical potentials, p300 potentials
◦ different outputs Several kinds of cursor, robotic arm control , sequential menu selection, selection from a matrix.
Written in C++
3. Dissemination of BCI2000
Available free of charge for research and educational pur-poses
http://www.bci2000.org 2006, 01 70 Labs
◦ Use the existing BCI2000 system without changing the software
◦ Implement new methods or system capabilities into BCI 2000
◦ Use BCI2000as a real-time signal acquisition platform to develop systems for research not related to brain-computer interfaces.
Example configurations, descriptive documentation, tutorial introductions.
Online bulletin board, question and answer
4. BCI Model and Modules BCI2000 is based on a model that can describe any BCI
system.◦ Source(data acquisition and storage)
Brain signals are processed synchronously in blocks containing a fixed number of samples that are acquired by the Source module.
◦ Signal processing Each time a new block of data is acquired, the source module sends it to signal
processing, which extracts signal features, translates those features into con-trol signals and sends them on to the User Application module.
◦ User application application module sends the resulting event markers back to the source mod-
ule where they and the raw signals are stored to disc.
◦ Operator interface
4-1. Module1. Source Module and File Format
Digitizes and stores brain signals◦ Passes them on without any further preprocessing to
Signal Processing◦ Consists of data acquisition component, data storage
component
BCI2000 file format Consist of ASCII header, binary signal sample, event marker
value. Accommodate any number of signal channels, system pa-
rameters, event markers.
4-2. Module2. Signal Processing Module
Converts signal from the brain into signals that control an output de-vice.
Realized using a chain of signal filters.
◦ Feature extraction (Three filters) 1. A linear calibration routine that converts signals from A/D units into uV 2. A linear spatial filtering operation (calculating a matrix multiplication of the input signals with a spatial filtering
matrix) 3. Temporal filter (6)
Slow wave filter : process slow cortical potentials Autoregressive spectral estimation Spectral estimation based on the FFT A general finite impulse response filter : process sensorimotor rhythms A peak detection routine : extracts firing rates from neuronal action potentials Filter that averages evoked responses(p300)
◦ Feature Translation Translates the extracted signal features into device independent control
signals. ( 2 filters) Linear classifier Filter normalizes the output signals such that they have zero mean and a specific value range
4-3. Module3. User Application Module
Receives control signals from signal processing
and uses them to drive an application
◦ 1. Sensorimotor rhythm amplitude◦ 2. Slow cortical potential amplitude
Visually on a computer screen Selection of targets, letters, or icons. 1~3 dimensional cursor movement. Two dimensional robotic arm control
◦ 3. Evoked potential classification Auditory and visual stimuli
◦ 4. p300 evoked potentials A matrix spelling application based on.
4-4. Module4. Operator Interface Module
Provides a graphical interface that displays ◦ Current system parameters ◦ Real-time analysis results communicated to it from
other modules.Allows investigator to start, stop, suspend,
resume, reconfigure system operation.
Monitor 1 : User feedback is displayed Monitor 2 : Operator module’s graphical interface
4-5. System Variables BCI2000 incorporates three types of system
variables : parameters, event markers, signals.◦ System Parameters : do not change throughout a data file
◦ Event markers: record events that occur during operation, can change from one data sample to the next.
◦ System signals: functions of the user’s brain signals that are re-ceived and modified by the modules.
Each module can request that the operator module create any number of system parameters or event markers.◦ EX> Source module : parameter for signal’s sampling rate.
◦ This parameter is constant during some defined period of online operation, and all other modules, and is recorded automatically in the data file.
◦ EX> Signal processing filter designed to detect artifacts : an event marker to mark artifacts in the signal
◦ EX> Application module : an event marker to record stimulus conditions.
4-6. System Requirements and Real-Time Processing
Microsoft Windows 2000 or a more recent windows-based operating system.
Any recent desktop or laptop computer, One of the data acquisition devices that BCI2000 sup-
ports Real-Time Processing
◦ windows is not a real-time operating system, thus does not guarantee specific timing of events
◦ System latency and latency jitter easily satisfied the real-time requirements of BCI operation.
(latencies are well below 20ms)◦ Processor load was sufficiently low to guarantee reliable
operation.
4-7. Offline Analysis Tools
We currently provide tools◦ To visualize signals in the time domain
◦ To get information on the configuration stored in a data file
◦ To convert data files to ASCII and Matlab
Mario◦ sensorimotor rhythm , p300evoked potential
Created Framework ◦ To compile and link BCI2000 signal processing components
◦ As command-line programs that read BCI2000 parameter
BCI2000 filters may also be compiled into dynamically loaded li-braries(DLLs)
4-8. Rapid Prototyping with MATLAB Matlab code can
Act on blocks of brain signal data in real-time Request its own parameters and event markers Read and modify parameter and marker values Report configuration and run-time errors
5. Future Development Hardware support
◦ Data acquisition systems tend to be the most expensive components.◦ Add support for additional devices.
Platform independence◦ BCI2000 depends on the Borland C++ compiler and the Borland VCL application
framework.◦ By moving toward platform independent libraries, and by replacing non-portable por-
tion from the code base,◦ More compatible with multiple compilers and operating systems
Clinical implementations◦ To make BCI setup, operation, and analyses available to users who are not experts on
BCI technology is critical.◦ Less complex and flexible version is needed.◦ Support simple menu selection using sensorimotor rhythms and p300-evoked poten-
tials.
Integration with other software◦ Interfacing BCI2000 with such external software is an important goal of its future de-
velopment.
6. Availability More information is available at Web Site
BCI2000 source code and executables is free for re-search and educational purposes
But Wadsworth requires a Material Transfer Agree-ment between a user’s institution and Wadsworth.(Prohibits commercial u se and limits liability.)
Compilation of the full BCI2000 source code requires ◦ version 5.4 of the Borland C++ ◦ Version 6.0 of the Borland VCL application framework is
sufficient to build the offline signal processing environ-ment.
<Four-Target Selection Using Tactile Feedback> -The subject controls movement of a cursor on the screen to select one of four targets. - The subject cannot see the screen, - Receives feedback on the target cur-rent cursor position through eight vi-brotactile actuators placed on his shoulders.