DIPARTIMENTO DI INGEGNERIA DELL’INFORMAZIONE E SCIENZE MATEMATICHE

Niccolò Meucci

a.a. 2017/2018

EEG Data Analysis

and Motion

Recognition

Introduction

Where

➢ Liquidweb is a startup company operating within the Information and

Communication Technologies (ICT) field, with special focus in

Ubiquitous Computing, Mobile Services and Cloud computing

technologies.

➢ BrainControl:

a framework

based on Artificial

Intelligence for

human-machine

interaction

through bio-

feedback.

Introduction

What is BrainControl?

Introduction

BC: how it looks like (1/3)

Introduction

BC: how it looks like (2/3)

Introduction

BC: how it looks like (3/3)

Introduction

BC: why do we need it

➢ ALS (Amyotrophic Lateral Sclerosis)

➢ Locked In Syndrome

BC: a multi-sensory platform

Introduction

➢ Augmentative and Alternative Communication (AAC)

Hand Gesture

Recognition

Accelerometer-based

gesture recognition

model

Hand Gesture Recognition

Introduction

➢ Current use: the accelerometer is connected via Bluetooth to the platform.

Once “MotionSensor” interaction is selected by the user, then a threshold-

based algorithm (very sophisticated ! ) is able to discriminate among 3 different

gestures:

➢ gesture UP;

➢ gesture RIGHT;

➢ gesture LEFT.

➢ Research Question: Create a model, Machine Learning- based, that

is able to classify such gestures.

Hand Gesture Recognition

Structure of research

Record data

Segmentation

Data smoothing

Dataset Creation

Feat. Extraction

Classification

Input user

Gesture

Recognition

Hand Gesture Recognition

Recording data (Up, Right, Left);

➢ 3 files .txt, containing only gestures belonging to one class (Up, Right, Left);

➢ file format:

➢Reference frame:

Hand Gesture Recognition

Segmentation

➢ Total number of gestures: 275

➢ 80 Up;

➢ 105 Right;

➢ 90 Left;

Hand Gesture Recognition

Applying Gaussian Filter (Pre-processing)

Hand Gesture Recognition

Dataset Creation

➢ number of examples: 80 per class;

➢ example dimensions: [ 20 x 3 ];

➢ dataset dimensions: [ 240 x 20 x 3 ];

➢ target each record with a label:

➢ class 1: Up;

➢ class 2: Right;

➢ class 3: Left.

Dataset label

1: [ 20 x 3 ] 1 = GestUp

… …

81: [ 20 x 3 ] 2 = GestRight

… …

161: [ 20 x 3 ] 3 = GestLeft

… …

Hand Gesture Recognition

Feature Extraction

➢ for each gesture:

➢ for each axis:

➢ compute (Area, Max, Min)

➢ 9 features, for each record

Dataset

dimension

FeatureSet

dimension

240 x 60 240 x 9

Hand Gesture Recognition

Classifier: Suppport Vector Machine

➢ (Multi-class) Classification problem: 3 Classes

➢ Model: Support Vector Machine

➢ It constructs a hyper-plane or set of hyper-planes, which can be used for

classification, regression, ..

➢ Intuitively, a good separation is achieved by the hyper-plane that has the

largest distance to the nearest training data points of any class (margin).

x2

x1

x2

x1

Hand Gesture Recognition

Train a Classifier (Supervised Learning)

➢ Split the Feature Set in Training and Test Set;

➢ Training Set dimensions: [ 160 x 9 ] 67% of data (2/3)

➢ Test Set dimensions: [ 80 x 9 ] 33% of data (1/3)

SVM

+ Class Label

OK

NO

SVM Prediction vs Class

Label

➢ Training

➢ Test

Hand Gesture Recognition

Results

Training accuracy

Test accuracy

Hand Gesture Recognition

Conclusions

➢ Off-line classification of gestures (Up, Right, Left)

➢ Good set of features

➢ Good generalization capability

Challenges and further works

➢ Different features (and feature set?) ?

➢ Data augmentation

➢ Real time classification (what about segmentation??)

➢ Build up an interface (GUI) able to classify gestures in real time better than

threshold-based algorithm ( classification or verification?? )

BCI and EEG

Analysis

Resting vs Push

Motor Imagery

Classification

Brain Computer Interface

definition

➢ A system which allows someone to communicate information about

their mental state without the use of the peripheral nervous system.

➢ signal must come directly from the brain

BCI

How does it work?

➢ BCI cycle:

12

3 4

BCI

The cycle: Generation (induced)

➢ Thinking of (for instance) a left-hand movement, or actually performing it,

leads you to the same (time-frequency) patterns

➢ Motor imagery: mental process by which an individual rehearses or

simulates a given action in his/her mind without actually performing the

movement (Dickstein and Deutsch, 2007; Ge et al., 2014; Sharma et al.,

2006)

1

BCI

The cycle: Measurement

2

BCI

The cycle: Decoding

Ch

t

Resting or Push?

➢ In general, we don‘t know how the users mental-task is encoded

in the measured signal

BCI

The cycle: Decoding C

htY

e

s

o

r

n

o

?

User A User B User C

Session 1

Session 2

Session 3

➢ Inter-subject and inter-session variability

BCI

The cycle: Output

BCI

BrainControl output

EEG Analysis

Research Question

➢ Understand what are the differences between resting and push, and how

to discriminate them from EEG point of view

Ch

t

Resting/push?

EEG Analysis

Hardware

➢ Epoc vs OpenBCI

➢ OpenBCI:

➢ 8 channels

➢ fs = 250 Hz

➢ data raw (unfiltered)

EEG Analysis

Protocol and Paradigm for Data Recording

➢ Paradigm:

rp

rp

rp

rp

rp

r = Resting

p = Push

Set up Set down

t (sec)4

5 2

➢ Resting = relaxation

➢ Push = ‘ thinking about moving something ’

EEG Analysis

Resting vs Push

r

p

EEG Analysis

Patterns in brain signal

➢ Source signals:

EEG Analysis

Patterns in brain signal

Delta δ

Theta θ

Beta β

Alpha α

noise

EEG approximation:

EEG Analysis

Evidence

➢ While performing a motor imagery thought (PUSH), the power in the alpha (or

beta) band decreases (Event-related desynchronization (ERD)), over sensory

motor cortex

➢ It gets back to «normal» when the user starts thinking of relaxing (RESTING)

(Event-related synchronization (ERS))

➢ Pattern evidence:

➢ Where: sensory motor cortex.

➢ When: during motor imagery

➢ What: decreasing in power over alpha/beta bands

➢ How: pre-processing, feature extraction

Pfurtscheller G, Neuper C, Mohl W.

EEG Analysis

Pipeline

Record data

Pre-processing

Filtering

Channel Selection

Power Spetrum

Plotting

Input user EEG

Resting VS

Push

EEG Analysis

Record data: Notch Filter + Band Pass (1-30 Hz)

EEG Analysis

Channel Selection

➢ Only sensors from somatosensory motor cortex are needed

T3

C3

C4

T4

EEG Analysis

Results and Conclusions

➢ My results

➢ Conclusions:

Resting and Push can be discriminated by computing the power spectrum of

alpha and beta bands, according to the literature

➢ Literature

➢ Power spectrum

A.Sivakami and

S.Shenbaga Devi

EEG Analysis

Application

Resting

Push

EEG Analysis

Further works

➢ Gather more data for Statistical analysis

➢ Change features ( Spatial Filtering, …)

➢ Gather more (and more) data to train a Classifier (SVM)

➢ Eventually Real-time classification with Machine Learning Models ( BCI )

Innovative Human

Machine Interfaces