a high-security eeg-based login system with rsvp stimuli ......to date, a number of different...

1556-6013 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIFS.2016.2577551, IEEETransactions on Information Forensics and Security

1

A high-security EEG-based login system withRSVP stimuli and dry electrodes

Yiyu Chen, Ayalneh Dessalegn Atnafu, Isabella Schlattner, Wendimagegn Tariku Weldtsadik, Myung-Cheol Roh,Hyoung Joong Kim, Seong-Whan Lee, Benjamin Blankertz, Siamac Fazli

Abstract—Lately, EEG-based authentication has received con-siderable attention from the scientific community. However, thelimited usability of wet EEG electrodes as well as low accuracyfor large numbers of users have so far prevented this newtechnology to become commonplace. In this study a novel EEG-based authentication system is presented, which is based onthe RSVP paradigm and uses a knowledge-based approach forauthentication. 29 subjects’ data were recorded and analyzedwith wet EEG electrodes as well as dry ones. A true acceptancerate of 100% can be reached for all subjects with an averagerequired login time of 13.5 s for wet and 27.0 s for dry electrodes.Average false acceptance rates for the dry electrode setup wereestimated to be 3.33·10-5.

Index Terms—biometrics, EEG, brain-computer interfaces,computer security, authentication, RSVP, dry electrodes, ERP

I. INTRODUCTION

ELECTRONIC authentication is a process of establishingconfidence in user identities electronically presented to an

information system [1]. Conventional authentication systemsare based on passwords and user input on a keyboard, touchscreen or with a mouse. More recent approaches employ eyetracking or biometric mechanisms.

To overcome some of the shortcomings of these classicauthentication techniques, novel authentication approaches arebeing explored that rely on the acquisition of neural activity,see, for example, [2], [3], [4], [5], [6], [7], [8], [9].

Non-invasive measurements of neural activity are mostcommonly performed by Electroencephalography (EEG) [10],Magnetoencephalography (MEG) [11], Near-Infrared Spec-troscopy (NIRS) [12], [13] and functional Magnetic Resonance

This work was supported by the National Research Foundationof Korea (NRF) grant funded by the Korea government (NRF-2015R1A2A1A05001867). The work of author Benjamin Blankertz wassupported by a grant from the BMBF under contract #01GQ0850. Thispublication only reflects the authors views. Funding agencies are not liablefor any use that may be made of the information contained herein.

Corresponding author: S.-W. Lee, Department of Brain and CognitiveEngineering, Korea University, Anam-dong, Seongbuk-ku, Seoul 136-713,Korea.

Y. Chen, I. Schlattner, S.-W. Lee and S. Fazli are with theDepartment of Brain and Cognitive Engineering, Korea University,Seoul, South-Korea. emails: [email protected], [email protected],[email protected], [email protected]

A.D. Atnafu, W.T. Weldtsadik and H.J. Kim are with the Departmentof Information Security, Korea University, Seoul, South-Korea. emails:dessalegn atne, adetariku, [email protected]

M.-C. Roh is with S-1 Corporation, Seoul, South-Korea. email:[email protected]

B. Blankertz is with the Neurotechnology Group, Technische UniversitatBerlin, Berlin, Germany. email: [email protected]

Imaging (fMRI) [14], [15]. Due to the ease-of-use, portability,relative low cost and its high temporal resolution EEG presentsthe most viable imaging technology for biometric authentica-tion purposes.

To date, a number of different EEG-based authenticationtechniques have been proposed. Most of them rely on thefinding that EEG characteristics are unique for every per-son [16], and can therefore be used as a biometric featurefor identification. Since the first EEG-based biometric systemwas proposed in the late 90s [17], a whole range of subject-specific EEG features have been examined for a variety ofconditions, such as eyes-open/eyes closed [17], [18], visualstimulus presentations [2], [7], motor imagery [4], [19], wordgeneration [4] and imagined speech [20], among others.

Poulos et al. [17] used spectral features in combinationwith an autoregressive (AR) model of resting state EEG andemployed learning vector quantification (LVQ) to achievean identification accuracy of up to 84%. Paranjape et al.used single-channel EEG, recorded during a simple eyes-closed/eyes-open task for the identification of individualsubjects and achieved an out-of-sample accuracy of over80% [18]. Palaniappan and Raveendran [2] used visual evokedpotentials (VEP) to identify users. They developed a taskin which pictures of gray-scale objects were presented andanalyzed corresponding power changes within the gammarange, which lead to an average classification accuracy of95% across subjects. Marcel and Millan [4] proposed anauthentication system, which is based on inherent features andused Gaussian mixture models. Subjects were cued to performmotor imagery tasks and word generation. Their results showthat motor imagery is the more appropriate mental task forauthentication and also that classification performance maydegrade across multiple sessions. Rocca et al. [8] suggested aninherence-based authentication method using PhysioNet datacollected from 108 subjects with eye-open and eye-closedresting conditions. The authors estimate the robustness of func-tional connectivity measures between EEG sensors and use itas a feature for biometric recognition. Yeom et al. [7] designedan authentication system, where EEG responses to self-faceand non-self-face visual stimuli are analyzed by means ofnon-linear support-vector machines (SVMs). Their averageperformance is reported as 86.1% across 10 subjects. For amore detailed review on recent EEG-based biometric userrecognition approaches we would like to refer the interestedreader to [21].

To date, most of the neural activity-based authenticationsystems employ gel-based EEG electrodes. While gel-based



2

electrodes offer the best signal quality, they are also inconve-nient to use in daily life as they need to be set-up and leave thehair full of gel. For the authentication process which shouldtake less than a minute, this is not feasible. To this end, dry andwater-based electrodes can offer a viable solution. While dryelectrode technology has first been proposed in the late 60’sand early 70’s [22], [23], it has only recently been validatedin online BCI paradigms and thus gained more attentionfrom the scientific community [24], [25], [26]. Besides, recentdevelopments for deployable setups use electrodes that areplaced hardly visible in-ear [27], [28], on the ear [29] or, asprinted electrode arrays, around the ears [30].

Early stage authentication systems, based on commerciallyavailable dry electrodes and wireless EEG, have recently beenreported [31], [6]. These existing dry-electrode authenticationsystems (and most others) are based on oscillatory features.From the neuroscientific and Brain-Computer Interface (BCI)literature it is known that oscillatory features are highlysubject-dependent and that so-called BCI illiteracy occurs inapproximately 15-30% of subjects [32]. In short, BCI illiteracymeans that subjects are not able to change their oscillatorybrain activity willingly upon command. As a result, at leastleast some subjects will not be able to use these types ofsystems meaningfully.

An Event-Related Potential (ERP) is the electrical brainactivity, as measured by EEG, that is time-locked to someevent. Typically this event is an external sensory stimulus,but an ERP can also result from the execution of a motor,cognitive or psychophysiological task [33]. ERPs reflect, e.g.,sensory perception and cognitive processing and they aremodulated by various factors, e.g., user states like attention. Incontrast to oscillatory features, ERP-based signals are knownto have a much better reproducibility across subjects and canbe employed in spelling devices [34], [35], [36].

In this paper we propose a novel EEG-based authenticationsystem which is based on ERPs that are elicited by a rapidserial visual presentation (RSVP) paradigm [37]. In the RSVPparadigm, different symbols are presented one-by-one in aserial manner and in the same location of display, exploitingonly the foveal visual field. The RSVP paradigm allows topresent a large number of stimuli within a short time. Despiteof the fast stimulus presentation, the RSVP elicits strongERPs [36]. As a result the RSVP paradigm can lead to highspelling speeds [36] or short login times as we will show.

In contrast to most previous approaches, which extractspecific inherence factors from the EEG, here we providea knowledge-based approach: users choose a combination ofthree images as their personal password. A large number ofimages, which include the password images, are then presentedin rapid succession at a single central location. Selectiveattention of the images which comprise the password leads toan enhancement of event-related potential (ERP) componentswhich can be robustly detected with the help of machinelearning techniques in the EEG. The usability of this systemis furthermore validated with dry electrodes.

For a detailed discussion on alternative methods, that do notrely on EEG as well as specific advantages of the proposedauthentication system are discussed in Sec. V.

Fig. 1. Left: g.Sahara dry electrodes. Center: Dry electrodes are mounted on astandard EEG cap. Right: Electrode positions for wet and dry electrodes. Redcircles denote common channels, blue circles denote channels that occurredonly for the wet electrode setup.

II. EXPERIMENTAL DESIGN

A. Participants

29 participants (20 males and 9 females, aged 21-44, mean= 29.97 yrs , S.D. = ± 4.54) took part in the experiment.All had normal or correct-to-normal visual acuity and noneof them had a history of neurological disease or injury. Allsubjects were members of Korea University, South Koreaor Technical University Berlin, Germany and volunteered toparticipate in the study. Out of 29 participants, 26 were naıvein respect to RSVP paradigms. The study was performed inaccordance with the declaration of Helsinki and all participantsgave written informed consent.

B. Apparatus

EEG with wet electrodes was recorded with a samplingfrequency of 1000 Hz, using BrainAmp amplifiers and Easy-Caps with a passive electrode system (Brain Products, Munich,Germany). The measurements were performed with 31 EEGelectrodes, namely: Fp2, F9,7,3,z,4,8,10, FC5,1,2,6, T7,8,C3,z,4, CP5,1,2,6, P7,3,z,4,8, PO3,4, O1,z,2 as well as oneEOG electrode below the right eye (EOGv1). Two bipolarEOG channels were computed with channels F9,10 and Fp2,EOGv1 for horizontal and vertical EOG, respectively. All other28 EEG electrodes were nasion-referenced and a foreheadground was used (Fpz). Impedances of all electrodes werekept below 10 kΩ during the experiment. Setup time for thewet electrode configuration was 34 minutes on average.

EEG with dry electrodes was recorded with a samplingfrequency of 1200 Hz, using a g.USBamp amplifier with com-mercial g.Sahara dry electrodes (g.tec, Schiedlberg, Austria;see Fig. 1) mounted on a flexible EEG cap. Measurements of16 channels were recorded: F3,4, FC1,2, C3,z,4, CP5,1,2,6,P3,z,4, and O1,2. Electrodes were nasion-referenced and aforehead ground was used (Fpz). Setup time for dry electrodesrequired 2 minutes on average.

All stimuli were presented on a 27” TFT monitor (Dell,Texas, USA) with a refresh rate of 60 Hz and a resolution of2560 × 1440 px2. The experiment was implemented in Pythonwith TkInter. Data analysis and classification were performedwith MATLAB (The MathWorks, Natick, MA, USA) usingthe Berlin BCI toolbox [38].



3

Model Estimation

AccessDenied

Access Granted

Required TAR

No

Yes

Pre-processing

EEG data acquisition

Model parameters

-A-B-C-

Model parameters

Pre-processoring

EEG data acquisition

-C-

Classifier

Registration System Login System

Authentication Server

Fig. 2. Flowchart of a potential system deployment. During registration the user chooses a password, consisting of three symbols. These symbols are displayedin random order together with 22 other non-password symbols in fast progression (i.e. RSVP). The EEG data is evaluated and a model estimated. During thelogin process, all symbols are displayed in RSVP fashion and the login decision, is based on the classifier’s output of the EEG data.

C. Paradigm

1) Key Idea: In order to introduce the main idea, we wouldlike to explain the system in terms of a potential deploy-ment (see Fig. 2): During registration, the subject choosesa username as well as a combination of three symbols as apassword (see also Fig. 3A). Password symbols are flashedin rapid succession together with other non-password symbolsin randomized order (Fig. 3C) and a subject-dependent modelis estimated from the EEG data. During the login phase firstthe username is entered, then the same password and non-password symbols are flashed in rapid succession. The modelthat was derived from the registration phase is applied to theEEG data and the authentication is performed.

2) Detailed experimental paradigm: A total of six runswas performed. At the beginning of each run, subjects wereinstructed to select three password symbols (PS) from acollection of 260 pictures, which would subsequently act astheir password. For the stimuli, a standardized set of 260pictures was used [39], which are loosely based on Snodgrassand Vanderwart’s object set [40] (see part A of Fig. 3). Thepassword symbols were selected by clicking the icon with amouse (see part A of Fig. 3). Once three icons were chosen,the selected PS were displayed at the center of the screen for15 s and the subject was asked to memorize them (part B ofFig. 3). 22 non-target symbols were randomly sampled fromthe data base and kept constant during each run.

During the RSVP phase, 25 different icons (including thethree PS as well as 22 unrelated icons) were presented inRSVP with an inter-stimulus interval (ISI) of 200 ms. Theywere presented in bursts of 150 presentations (lasting 30s).Each burst contained a variable number of target symbols,which the user was instructed to count. After each burst a shortbreak occurred. The break consisted of two phases: during the

first phase, the subject was asked to type the number of targets(i.e. PS) he was able to count (see Fig. 3D). His actual inputwas not used in any further analysis, but merely to ensure hisattention to the task. In the second phase, the target stimuli anda countdown of 15 s were shown (Fig. 3B). The countdownindicated the start of a new presentation period. This procedurewas repeated four times until a total of 600 trials, consisting of72 targets and 528 non-targets were presented. The sequenceof stimuli was designed such that there was always a minimumgap of two other icons between two same icons.

Please note, that for deployment, only one run would besufficient for registration. However, as mentioned earlier a totalof six runs was performed. The additional runs were includedto examine various scenarios, as described below.

a) Runs 1-3: Login with novel and specific symbols:During the first run, subjects had to choose three icons astheir password. In the second run they had to choose threedifferent icons. The icons of the first run were blocked duringthe selection process. Similarly, in run 3, icons of runs 1 &2 could not be chosen. If an icon had been selected in anyof the previous runs, it was not part of the stimulus set inthe current run. Runs 2 and 3 were performed to simulate apassword change on the user side.

b) Run 4: Influence of old password symbols: A differentsetting was chosen in run 4 in order to investigate whether oldPS would still elicit specific ERPs that the login system mightmistake as the actual PS. This issue is relevant for the caseof changing passwords, as well as in situations when tryingto resist a forced login. To this end, the old PS of run 1 wereintroduced as nontargets in run 4 to be tested against the newlychosen PS.

c) Runs 5-6: Categories vs. specific symbols: In the lasttwo runs, we introduced some variability in the target symbolsin order to challenge the permissiveness of the proposed login



4

EB C DB C DB C D

A B C

A B CRun 1

D E F

D F15s 30s 15s 30s 15s 30s 30s15s 3m

Run 6

NT

150 stimuli/sequence5 stimuli/sec

Start of RSVPsequence

NT

NTNT

T

TNT

NTT

200ms

End of RSVPsequence

Fig. 3. Flowchart of the experimental paradigm. A: User chooses three symbols as his password. B: Chosen password is shown and a counter indicates thestart of the RSVP. C: 25 randomized symbols (including the password symbols) are presented in rapid succession. D: User is asked to input the total numberof password symbol occurrences. E,F: Distractors (not used in further analysis).

system. As in runs 1-4, the participants were ask to choosethree PS from a collection of 33 options. But this time, eachchosen symbol was taken as a representative for a category(such as ‘car’ or ‘elephant’). During stimulus presentation, thesymbol that represented the category in the selection processwas exchanged with other examples from this category. Threeexamples per category were used during stimulus presentation.If an elephant was chosen as one PS, also pictures of otherelephants were considered as (target) PS. These symbols willbe called categorical symbols in the remainder of this article.The average ratio of targets was the same in runs 5 and 6, i.e.12%. By using categorical symbols instead of specific symbolsthe size and complexity of the stimulus set is increased andthereby the system security.

Note, that in runs 1-4, the data base contained only onesymbol from each category. However, there were some pic-tures that were quite similar, e.g., tiger, leopard, lion, fox, seeFig. 3A.

The comparatively large number of runs can lead to ex-haustion and/or boredom on the subject side. To counter

these effects, distractors were included in the experimentalparadigm. After each run, short movie clips were shown tothe subject (Fig. 3E). Furthermore, the subjects were requiredto perform six mathematical tasks, which consisted of additionand subtraction (Fig. 3F). Prior to the six actual runs a shorttest run was performed, such that participants could becomefamiliar with the task.

For the dry electrode study, only the first three runs wereperformed, since we were interested to show that the basicsetup is viable for dry electrodes. Also the movie clipswere not shown during dry measurements. As a result thetotal recording time for the dry electrode measurements wasreduced to approx. 12 minutes. Three subjects performedthe experiment twice, once with each electrode configuration,while all others participated in only one experiment.

III. DATA ANALYSIS

For the ERP analysis, the EEG data was low-pass filteredby a Chebyshev digital filter with a passband of 40 Hz anda stopband at 49 Hz. The data was then downsampled to



5

100 Hz from 1000 Hz (and 1200 Hz dry electrode data) byaveraging consecutive blocks of 10 (of 12) samples. The datawas epoched to a range of -200 ms to 1000 ms with respectto stimulus onset. A baseline correction was performed on thepre-stimulus interval from -200 ms to 0 ms. Epochs containingstrong eye movements were detected and rejected using aminmax criterion of 75 μV on the channels F9, Fz, F10 andFp2 for wet electrode, and 150 μV on all channels for dryelectrode data.

Features were calculated from 28 wet (16 dry) electrodechannels by averaging voltages in nine non-overlapping time-windows with a width of 100 ms, starting from 100 ms to1000 ms with respect to stimulus onset. This resulted in 28× 9 = 252 (dry: 16 × 9 = 144) dimensional feature vectors.Channels F9,10 and Fp2 were removed in the wet electrodesetup, while none of the dry electrodes were removed.

Topographical maps of significant features were calcu-lated by point-biserial correlation coefficients [41]. The point-biserial correlation coefficient is a special case of the Pear-son product-moment correlation coefficient, and measures theassociation of a binary random variable (in this case ’targetsymbols’ and ’non-target symbols’) to a continuous randomvariable (here channel-wise ERP data). It is defined as:

rpb =M1 −M0

sn

√n1n0

n2with sn =

√√√√ 1

n

n∑i=1

(xi − x)2

(1)where M1 and M0 are the mean values of the data points ingroups 1 and 0, respectively, n1/0 the number of examples ingroups 1 and 0 and n the total sample size. Using Fisher’stransformation the correlations were transformed into unitvariance z-scores for each subject j [42], and grand average z-scores were obtained by a weighted sum of individual z-scoresover all subjects:

zj =tanh−1(rj)√

mj − 3and z =

∑N

j=1 zj√N

(2)

where mj is the sample size of subject j, and N = 16 thetotal number of subjects. p-values for the hypothesis of zerocorrelation in the grand average were computed by meansof a two-sided z-test. All reported p-values were Bonferroni-corrected to account for multiple hypothesis testing [43].

Classification of ERP components was performed withregularized linear discriminant analysis (RLDA), also knownas shrinkage LDA [44], [45]. Let x1, . . . ,xn ∈ Rd be n featurevectors, and µ and Σ be the unbiased estimator of the meanand covariance matrix, respectively. If the dimensionality ofthe data is high with respect to the number of data points, theestimation of the covariance matrix can be imprecise and leadto a systematic bias [46], [47]. Shrinkage (i.e. regularization)of the estimated covariance matrix is a way of correcting thissystematic bias:

Σ(γ) := (1− γ)µ + γνI, (3)

where γ ∈ [0, 1] is the shrinkage parameter, and ν being theaverage eigenvalue of the estimated covariance matrix (ν =Tr(Σ)/d with d being the dimensionality of the data) and I the

identity matrix.For shrinkage LDA, an analytical solution is available [48].

The analytical solution penalizes large sample-to-sample vari-ance of entries in the empirical covariance, leading to strongershrinkage. Let the i-th element of vectors xk and µ be denotedas (xk)i and (µ), respectively, and sij be the i-th row and j-thcolumn of Σ and, define

zij = ((xk)i − (µ)i)(xk)j − (µ)j), (4)

then the optimal shrinkage parameter γ∗ in terms of general-ization error can be calculated by

γ∗ =n

(n− 1)2·

∑d

i,j=1vark(zij(k))∑

i6=js2ij +

∑i(sii − ν)2

. (5)

While this parameter could in principle also be found by nestedcross-validation, the analytical solution is computationally lessexpensive and was used in our analysis.

In order to investigate the minimum time required for apotential login with a predefined desired true acceptance rate(TAR), cross-validation was performed. Please note that threedifferent targets were present in each run (with 22 non-targets;see Sec. II-C for more details). Furthermore, targets changedduring individual runs (similarly for non-targets). However, alldata was grouped into target and non-target trials, regardlessof the actual symbol and run. During the cross-validationprocedure, 21 target trials with corresponding non-target trials,i.e. 7 sequences consisting of 175 trials (21 targets and 154nontargets), were left out for testing, while all other data wasused for classifier training. These left out trials were fed to theclassifier and the resulting output was written into a matrix.The trials were arranged in the 21 × 8 matrix such that thefirst column consisted of all 21 targets, while the other 7columns consisted of 147 non-targets. Note, that the outputs of7 nontargets were discarded. This arrangement of the classifieroutputs as a matrix was chosen to analyze the recognitionperformance depending on the length of the symbol sequencethat is presented for the login process. The true labels of eachrow are therefore C = [1 0 0 0 0 0 0 0]. Only if the predictedtrials matched the true labels of C exactly, the login attemptwas granted.

To examine the trade-off of TAR vs. login time, the clas-sification outputs of n sequences (i.e. n matrix rows) wereaveraged, where n was varied from 1 to 21. The number ofrequired sequences (and thus login time) was calculated forthree predefined TARs, namely 90%, 99% and 100%.

Due to the unbalanced number of trials per class, anappropriate loss function needs to be chosen, which considersthe different class prior probabilities [49]. A typical choiceis the area under the receiver operator characteristic (ROC)curve [50], which we have employed in the single trialclassification analysis.

The false acceptance rate (FAR) of an authentication sys-tem is defined as the fraction of the number of successfulauthentications by impostors divided by the total number ofimpostor authentication attempts. Let us consider the followingsituation: person B is trying to gain access to the account ofperson A. To simulate this situation, the EEG data of person



6

B is classified by the model, which was derived from the dataof person A. We examine two cases.

Case 1: Person B does not know the password of person A.We therefore map a permuted version of person B’s classesto those of subject A. In practical terms this means personB randomly selects three symbols as his password symbols.The class permutation was performed 50 times and sequencedata used for the login procedure were randomly sampled 100times. Thus a total of 5000 login attempts were simulatedfor person B trying to enter the account of person A. Notonly person B, but all other subjects were also consideredas potential attackers for the account of person A, resultingin 15× 5000 = 75000 attempts in total. This analysis wasrepeated for each subject. The same number of sequences n,which were estimated through the predefined TAR levels, wereused for averaging.

Case 2: Person B knows the username and password ofperson A. We apply the same procedure as for case 1, exceptthat now no permutation of classes is performed. As a result,15× 100 = 1500 attempts are simulated in total.

IV. RESULTS

A. ERP

The grand average ERPs for normal and categorical datacan be obtained from Figure 4. The top row shows ERPtimecourses, where the green line represents the average targetresponse, and the gray line the non-target response. For bothconditions electrode P7 shows a N2 peak at 350 ms, P3a peaksat 540-550 ms in channel Cz and P3b peaks at 630 ms in bothdepicted channels (Cz and P7). The first two rows of scalpmaps indicate the average target and non-target amplitudesfor selected intervals. These intervals can be obtained fromthe gray shaded areas in the ERP timecourses. In the interval300-420 ms, the N2 can be seen in the occipital area. Thethird row of scalp maps as well as the plots underneath showthe distribution of the sgn(r2) as a measure of the correlationbetween the target and non-target classes. The correlationsin occipito-parietal areas are higher for specific symbols inthe time interval between 480-580 ms, while correlations infronto-central area are higher for categorical symbols in thetime interval from 580 to 680 ms. Both conditions show a latenegative peak around 800 ms.

Classwise averaged timecourses of grand average ERPs withthe dry electrode configuration can be seen in Figure 5.

B. Including the old password as a distractor

As described in Sec. II-C the PS of run 1 were includedas non-targets in run 4. Figure 6 shows the time evolutionof statistical significance for the target symbols versus non-targets (top) and for the old target symbols of run 1 versusnon-targets (bottom). As can be seen, new target symbols showhighly significant differences as compared to non-targets intime windows starting from 400 ms and ending at 700 ms.Old target symbols do not show any significant differences tonon-target symbols.

C3

target

non−target

C4

CP1 CP2CP5 CP6

Cz

F3 F4

FC1 FC2

O1 O2

P3 P4Pz

5 µV

+

500 ms−0.01

0

0.01

sgn r2

Fig. 5. Grand average timecourses for dry electrodes.

C. Single-trial classification

Single trial classification results of the specific symbolcondition can be seen in Table I. As mentioned before, areaof ROC is used as a loss function here. The first two columnsshow classification accuracy for wet electrode data with thefull 28 channel setup and a 16 channel setup, respectively. Thelast column shows single-trial accuracy for 16 dry electrodes.Both 16 channel setups contain the same channels. Averageaccuracy across 16 subjects for the 28- and 16-channel wetconfigurations are 87.8±5.1 % and 85.9±5.0 %. For the dryelectrode setup, the average accuracy is 78.2±5.7 %. A paired,two-sided sign test with the hypothesis that the differencebetween the matched samples for the 16 wet and 16 dryelectrode setups comes from a distribution whose median iszero, yielded significantly lower accuracy for the dry electrodesetup (p < 0.001).

Categorical symbols scored an average accuracy of82.62±4.78 % across 16 participants with the full, wet channelsetup (not shown in Table I). The result of a paired two-sidedsign test shows significantly lower accuracy for the categoricalsymbol condition compared to the specific symbol condition(p < 0.05).

D. Estimation of required login time

The estimated login times were calculated as a functionof required accuracy. Table II summarizes the results for thespecific-symbol condition with wet and dry electrode data.If the required accuracy of the system is set to 100%, theaverage login time for the wet electrode setup is 10.7±4.6 sand 27.0±11.7 s for the dry electrode setup.

E. False acceptance rate

Results for the simulations of impostor login attempts, whenthe impostor does not know the password of the user (case 1)can be obtained from Table III. False acceptance rates aregiven for each subject and for three levels of true acceptance



7

−100 0 100 200 300 400 500 600 700 800 [ms]

−2

0

2

4

Cz (thick) P7 (thin)

[µV

]

target

non−target

targ

et

no

n−

targ

et

specific symbols

−100 0 100 200 300 400 500 600 700 800 [ms]

−2

0

2

4

Cz (thick) P7 (thin)

[µV

]

target

non−target

targ

et

no

n−

targ

et

categorical symbols

[µV

]

−3

−2

−1

0

1

2

3

−100 0 100 200 300 400 500 600 700 800 [ms]

−2

0

2

4

6

8

10x 10

−3Cz (thick) P7 (thin)

[sg

n r

2]

sg

n r

2(T

,NT

)

−100 0 100 200 300 400 500 600 700 800 [ms]

−2

0

2

4

6

8

10x 10

−3Cz (thick) P7 (thin)

[sg

n r

2]

s

gn

r2(T

,NT

)

[sg

n r

2]

−0.01

0

0.01

Fig. 4. Grand average ERP analysis of specific- and categorical-symbol conditions. The top row shows timecourses of two EEG electrodes, namely Cz andP7. Rows 2 and 3 show scalp maps of time-averaged amplitudes. Below the distribution of the sgn(r2) as a measure of the correlation between the targetand non-target classes is shown as time-averaged scalp maps (row 4) and timecourse (row 5).

rate. The average FARs across subjects are 1.69 · 10−4,4.00·10−5 and 3.33·10−5 for TAR being 90%, 99% and 100%,respectively. The system design is such that the probabilityof guessing another subject’s password symbols correctly isp = 3

25× 2

24× 1

23≈ 4.35 · 10−4. One sample t-tests with

the hypothesis of equal means show that FAR values obtainedfrom our simulation are significantly lower for all three testedTAR values (p = 3.8·10−6, p = 1.7·10−14 and 4.1·10−15). Thereason for this is the reduced classification accuracy due to thesubject-to-subject transfer of a given classifier. If the impostorknows the user’s password symbols (case 2), the average FARsacross subjects are 7.8±8.2, 15.6±8.4 and 19.7±10.1 for TARbeing 90%, 99% and 100%, respectively. While these numbersmay seem high at first sight, please note that the impostor isin the posession of the username and password. Since theimpostor knows the password symbols, these symbols willelicit an ERP with a strong deflection when displayed duringthe RSVP procedure. The neural signature of the ERP isonly slightly different for every person and the system is not

designed to discriminate between inter-subject differences ofERPs. Please note, that a combination with an inherence-basedapproach would reduce FAR rates for impostors with passwordknowledge (see also Sec. VI, where this is discussed further).

V. DISCUSSION

We proposed a knowledge-based authentication approachthat relies on the acquisition of brain signals via EEG. Wedemonstrated that the actual login time is relatively short forthe established high security level even when employing noveldry electrode EEG caps which provide lower signal quality.However, the additional effort of requiring an EEG headsetand mounting it needs to be considered. In order to trade-offthe pros and cons of the proposed brain-based authentication,we review existing approaches in more detail.

A. Existing authentication systems

Existing authentication techniques can be categorized intothree factors: knowledge factor, possession factor and inher-



8

[−lo

g1

0(p

)]

−10

0

10

PS

vs

. N

T

[−lo

g1

0(p

)]

−10

0

10

OP

S v

s.

NT

−100 − 0[ms]

0 − 300[ms]

300 − 400[ms]

400 − 500[ms]

500 − 600[ms]

600 − 700[ms]

700 − 900[ms]

[−lo

g1

0(p

)]

−10

0

10

PS

vs

. N

T

dry

ele

ctr

od

es

Fig. 6. Top and bottom rows show grand average statistical significance of differences between password-symbols (PS) and non-target (NT) symbols for wetand dry electrodes, respectively. The middle compares old target symbols (OPS) of an earlier password to other non-targets (i.e. data from run 4, see Sec. II-Cfor more details).

TABLE IISHOWS THE MINIMUM NUMBER OF TARGETS THAT NEED TO BE AVERAGED (#) AND THE LOGIN TIME (T) REQUIRED TO ACHIEVE A TRUE ACCEPTANCE

RATE (TAR) OF 90%, 99% AND 100%. RESULTS ARE GIVEN FOR EACH SUBJECT AND THEIR MEAN FOR THE WET (LEFT) AND DRY ELECTRODECONFIGURATIONS (RIGHT).

wet electrodes dry electrodesTAR 90% 99% 100% TAR 90% 99% 100%ID # t # t # t ID # t # t # tzk 3 5.0 7 11.7 9 15.0 zk 5 8.3 14 23.3 22 36.7eal 2 3.3 3 5.0 5 8.3 eal 3 5.0 7 11.7 11 18.3lh 2 3.3 4 6.7 6 10.0 lh 3 5.0 9 15.0 10 16.7xk 2 3.3 5 8.3 6 10.0 jak 2 3.3 6 10.0 13 21.7jw 3 5.0 9 15.0 14 23.3 ocb 2 3.3 5 8.3 7 11.7ad 2 3.3 5 8.3 8 13.3 rsv 4 6.7 14 23.3 18 30.0yy 2 3.3 3 5.0 4 6.7 sn 2 3.3 7 11.7 12 20.0er 2 3.3 4 6.7 5 8.3 jeg 6 10.0 16 26.7 23 38.3vi 1 1.7 2 3.3 3 5.0 fat 7 11.7 20 33.3 30 50.0sc 2 3.3 5 8.3 8 13.3 hjh 3 5.0 8 13.3 11 18.3hw 2 3.3 5 8.3 8 13.3 sd 3 5.0 8 13.3 11 18.3th 2 3.3 5 8.3 8 13.3 nsk 2 3.3 4 6.7 7 11.7hj 1 1.7 2 3.3 3 5.0 bv 4 6.7 10 16.7 16 26.7tu 1 1.7 3 5.0 5 8.3 dk 8 13.3 21 35.0 32 53.3gz 1 1.7 3 5.0 4 6.7 mg 6 10.0 12 20.0 21 35.0sk 1 1.7 4 6.7 7 11.7 jys 3 5.0 9 15.0 15 25.0

mean 1.8 3.0 4.3 7.2 6.4 10.7 mean 3.9 6.6 10.6 17.7 16.2 27.0



9

TABLE ISINGLE TRIAL CLASSIFICATION ACCURACY FOR WET AND DRY

ELECTRODES

ID wet ID dry28 ch 16 ch 16 ch

zk 78.8 80.4 zk 76.6eal 90.2 87.6 eal 78.6lh 86.9 83.9 lh 77.5xk 84.4 84.2 jak 83.3jw 78.0 75.9 ocb 86.8ad 85.6 81.6 rsv 73.0yy 89.6 86.7 sn 82.5er 88.7 84.2 jeg 72.3vi 95.8 95.4 fat 67.8sc 83.7 82.5 hjh 81.9hw 86.6 84.9 sd 78.6th 88.0 85.2 nsk 84.2hj 95.4 94.0 bv 76.4tu 89.0 87.2 dk 67.6gz 94.3 91.1 mg 70.5sk 90.0 89.1 jys 82.4

mean 87.8 ± 5.1 85.9 ± 5.0 mean 77.5 ± 5.9

TABLE IIISHOWS FALSE ACCEPTANCE RATE FOR THREE TAR LEVELS. A TOTAL OF

75000 IMPOSTOR LOGIN ATTEMPTS WERE SIMULATED FOR EACHSUBJECT. DRY EEG ELECTRODE DATA WAS USED FOR SIMULATIONS.

ID False acceptance rate forTAR=90% TAR=99% TAR=100%

zk 2.93e-04 9.33e-05 4.00e-05eal 4.00e-05 0.00e+00 0.00e+00lh 1.47e-04 6.67e-05 2.67e-05jak 1.73e-04 1.33e-05 0.00e+00ocb 1.33e-05 0.00e+00 0.00e+00rsv 1.60e-04 0.00e+00 1.33e-05sn 6.67e-05 1.33e-05 0.00e+00jeg 2.40e-04 5.33e-05 9.33e-05fat 1.33e-05 0.00e+00 0.00e+00hjh 2.93e-04 1.33e-05 4.00e-05sd 4.00e-04 1.07e-04 8.00e-05nsk 4.00e-04 1.87e-04 1.87e-04bv 0.00e+00 0.00e+00 0.00e+00dk 0.00e+00 0.00e+00 0.00e+00mg 6.67e-05 0.00e+00 0.00e+00jys 4.00e-04 9.33e-05 5.33e-05

mean 1.69e-04 4.00e-05 3.33e-05

ence factor [51]. Password-based authentication is a knowl-edge factor authentication that is most widely used in com-puter systems. Recent major leaks from giant IT companieslike Twitter, LinkedIn, Dropbox and Yahoo are an indicatorthat password-based authentication systems do not providesufficient security [52]. The major attacks on password-basedauthentication systems are grouped into 6 types. They arebrute-force, dictionary, guessing, spyware, social engineeringand shoulder surfing [53], [54]. One of the basic rules intext-based password security to protect against guessing anddictionary attacks is that passwords should be random andstrong, but this makes it difficult for the user to remember. Theconventional way of measuring the strength of a password hasbeen improved in a number of ways. Malone and Maher [55]proposed a method to identify dangerously common passwordsamong different standard statistics. Komanduri et al. [56]invented a novel way of identifying weak passwords by pre-dicting the next character that the user will type. The authorsindicate that their method creates fewer weak passwords thanconventional character composition methods. On the otherhand, Schechter et al. [57] claim passwords made of simplewords are not always weak. The strength of passwords dependson their popularity. The authors propose an oracle that canidentify popular (undesirable) passwords using the count-minsketch method [58]. In addition, Florencio et al. [59] argue thatthe common way of ruling out weak passwords and passwordre-use are suboptimal. They propose an optimal solution thatallows the re-use of passwords by grouping accounts.

The difficulty of recalling strong text passwords by userslead researchers to look for better ways of representing pass-words. Since the 19th century, psychologists have hypothesizedthat visual information is easier to recall than verbal or textualinformation [60], [61]. The main reason for the superiorityof visual memory is that pictures are more easily associatedwith previous knowledge about the object they represent [62],[63]. Two decades ago, the first picture-based password systemwas proposed by Blonder [64]. Since then, a variety ofgraphical password systems have been introduced to improvesecurity and usability [65]. Although graphical passwords haveimproved memorability and security as compared to text-basedpasswords, they are still vulnerable against shoulder surfingattacks. Several techniques have been suggested to overcomeshoulder surfing on graphical password systems. Suo [66]designed a system that blurs all parts of the image excepta small portion, which he termed decoy region. In this systemthe user only needs to respond ’yes’ or ’no’ by using mouseclicks to indicate whether his password appears in the decoyregion. Sobrado et al. [67] and Wiedenbeck [68] suggesteda convex hull-based shoulder surfing protection scheme forgraphical passwords. The convex hull is an imaginary polygon,formed by connecting the password images, which are scat-tered among a large number of other pictures. This methodavoids the need of revealing the password images directly,since the user is instructed to click anywhere within the convexhull. Shoulder surfing can also be prevented by changing thepassword input mechanism from mouse, keyboard or touchscreen to biometric based mechanisms [69], [70]. Kumar etal. [71] suggest to use an eye-tracking system with an on-



10

screen keyboard to input the password. This makes it difficultfor a peeper to identify the user’s password. Later Wu etal. [72] designed a method that combines the Convex Hullclick method [67], [68] with the eye-tracker input method [71]in order to provide better security as compared to the individ-ual techniques.

Osborn et al. [73] introduced a novel graphical passwordsystem, where the user chooses a number of picture-basedcategories as his password. During the login process, a set ofpictures is shown to the user on a grid. Each picture belongsto a predefined category (such as car, house, flower, etc.) andappears with an associated letter. The user will then find thepictures, which belong to his password categories and type thecorresponding letters. The pictures as well as the associatedletters will be different for each login process. Matching twopictures of the same category is a computationally expensivetask, rendering brute-force attacks useless. Furthermore, thesecurity level of the system can be controlled by adjusting thetotal number of categories as well as the number of displayedpictures during the login process.

Biometric authentication has several advantages over thetraditional knowledge-based and possession-based authentica-tion techniques, however most biometric-based authenticationrequire special hardware to capture biometric data, which re-sults in increased implementation costs. Some of the biometricauthentication techniques found today are based on fingerprintscans [74], hand geometry [75], [76], iris scans [77], [78], facerecognition [79], [80] and keystroke dynamics [81], [82].

B. The proposed brain-based authentication

The results indicate that our knowledge-based authenticationapproach is appropriate for high security EEG-based logintechnology. Attack modes, such as shoulder surfing or eye-tracking are impossible to apply. While previous approachesfocused on inherence-based features, a knowledge-based ap-proach offers a number of benefits. One of them is thatthe required TAR levels can be adjusted to the particularapplication. For example, if only low-security access (e.g.TAR ≥ 90%) is required, the average login time with wetelectrodes is only 3.0 s. If needed, the system can be set to100% authentication accuracy for every subject by averagingmore trials and thereby increasing the signal-to-noise ratio.However, this happens at the expense of increased login time.Even under maximum security (i.e. TAR = 100%) situationsan average required login time of 10.7 s seems viable (seeTab. II for more results). Furthermore, we would like topoint out that login times could be reduced by so-calledearly stopping methods, which adaptively stop the stimuluspresentation sequence when enough evidence for a presetaccuracy is reached, see [83] for an overview. Finally, it isan important fact that no subjects were excluded from theanalysis which indicates that our system is applicable to thegeneral public.

C. Stimulus complexity and ERP morphology

In an earlier study, the RSVP paradigm was employed for aspelling application [36]. The stimuli consisted of individual

letters and ERPs showed much earlier positive deflections,when compared to our study. In this study we comparedthe effect of specific as well as categorical symbols. Theresults of these two studies indicate that the delay of thelargest positive ERP deflection is positively correlated withthe stimulus complexity. In a future study, we are planning tolook further into this interesting finding. By deepening ourknowledge in this domain it may be possible to not onlyincrease our neurophysiological understanding, but to also findoptimal parameters for RSVP-based applications [36], [84].

D. Simulation of an abduction scenario

If a given person is abducted and forced to login, aninherence-based biometric can be conducted forcefully. Tofind out whether it is possible to suppress the passwordsymbols, we asked the subjects to think of a new passwordand tested whether the old password symbols would stillelicit a discriminative response. The analysis did not show adiscriminative response for old password symbols (see middlerow of Fig. 6). Therefore, we conclude that it is possible toeffectively ’hide’ the password, if needed.

VI. CONCLUSION

We tested the proposed system with commercially availabledry electrodes. While wet EEG electrodes generally havebetter signal quality as compared to dry ones, wet electrodesneed to be prepared by inserting a conductive gel, whichcan require up to 30 minutes for a whole head setup. Inaddition, the hair needs to be washed after the experiment (i.e.login process). As a result, wet electrodes are not feasible forauthentication systems. Our analysis shows that while single-trial classification is significantly lower for dry electrodesas compared to wet ones, they show sufficient accuracy forhigh security applications. We conclude that the use of dryelectrodes in a commercial authentication system is possibleand furthermore highlight their practicality. A whole rangeof novel EEG-based technologies are currently being devel-oped. Among those are commercial products, but also manyresearch-based prototypes. They include mobile, low-cost andalso dry EEG solutions. In our study, the application of thedry electrodes took 2 minutes on average. Future mobile anddry EEG technology will reduce this time even further andcould presumably be shortened to 15 s with next generationEEG headsets.

While most previous approaches are based on inherencefactors, EEG-based biometrics can also serve as a knowledge-based or hybrid authentication system [3], [9]. As a nextstep, we plan to extend our approach to a one-step, two-factor authentication system, where inherence and knowledge-based factors are combined. An inherence-based authenticationsystem could be applied to identify the user and be combinedwith the knowledge-based approach we introduced here byemploying advanced multi-modal fusion techniques, whichhave recently been developed [85], [86].

In our analysis we calculated the number of required se-quence repetitions to achieve three levels of TAR, namely90%, 99% and 100%. As described earlier these results



11

were obtained through cross-validation with data of the samesession. However, earlier studies observed the degradationof classification performance, when a so-called session-to-session transfer is performed, i.e. when the classifier of anearlier session is transferred to a newer session [4]. While wehave not estimated this type of degradation here, we wouldlike to mention that this degradation could in principle becounterbalanced by raising the number of averaged sequences,which would in turn lead to increased login times. In addition,recent technological advances in BCI have shown that session-to-session transfers are possible for expert users with onlyminimal increased loss [87]. These type of techniques alleviatethe need for recalibrating the system. In addition, a recentin-depth study on the stability of EEG-based features forbiometrics came to the conclusion, that in fact EEG signalscontain discriminative information which are stable acrosstime [88].

Another current limitation of the presented technology isthe need of calibration data on the subject side. Subject-independent decoding [89], [90], [91], [92], [93] has recentlybeen addressed in BCI research and allows any user to start areal-time BCI feedback session instantaneously. In the future,we plan to adopt these type of techniques for the presentedparadigm and thereby hope to further increase the usability ofthe proposed system significantly.

APPENDIX A. SUPPLEMENTARY MATERIAL

A video of the stimulus presentation is available.

REFERENCES

[1] W. E. Burr, D. F. Dodson, E. M. Newton, R. A. Perlner, W. T. Polk,S. Gupta, and E. A. Nabbus, Sp 800-63-2. Electronic AuthenticationGuideline. National Institute of Standards & Technology, 2013.

[2] R. Palaniappan and P. Raveendran, “Individual identification techniqueusing visual evoked potential signals,” Electronics Letters, vol. 38,no. 25, pp. 1634–1635, 2002.

[3] J. Thorpe, P. C. van Oorschot, and A. Somayaji, “Pass-thoughts: authen-ticating with our minds,” in Proceedings of the 2005 workshop on Newsecurity paradigms, pp. 45–56, ACM, 2005.

[4] S. Marcel and J. d. R. Millan, “Person authentication using brainwaves(EEG) and maximum a posteriori model adaptation,” Pattern Analysisand Machine Intelligence, IEEE Transactions on, vol. 29, no. 4, pp. 743–752, 2007.

[5] R. Palaniappan and D. P. Mandic, “Biometrics from brain electricalactivity: A machine learning approach,” Pattern Analysis and MachineIntelligence, IEEE Transactions on, vol. 29, no. 4, pp. 738–742, 2007.

[6] J. Chuang, H. Nguyen, C. Wang, and B. Johnson, “I think, thereforeI am: Usability and security of authentication using brainwaves,” inFinancial Cryptography and Data Security, pp. 1–16, Springer, 2013.

[7] S.-K. Yeom, H.-I. Suk, and S.-W. Lee, “Person authentication fromneural activity of face-specific visual self-representation,” Pattern Recog-nition, vol. 46, no. 4, pp. 1159–1169, 2013.

[8] D. Rocca, P. Campisi, B. Vegso, P. Cserti, G. Kozmann, F. Babiloni,and F. V. Fallani, “Human brain distinctiveness based on EEG spectralcoherence connectivity,” IEEE Transactions on Biomedical Engineering,vol. 61, no. 9, pp. 2406–2412, 2014.

[9] B. Johnson, T. Maillart, and J. Chuang, “My thoughts are not yourthoughts,” in Proceedings of the 2014 ACM International Joint Con-ference on Pervasive and Ubiquitous Computing: Adjunct Publication,pp. 1329–1338, ACM, 2014.

[10] H. Berger, “Uber das Elektroenkephalogramm des Menschen,” Archivfur Psychiatrie und Nervenkrankheiten, vol. 87, pp. 527–570, 1929.

[11] D. Cohen, “Magnetoencephalography: evidence of magnetic fields pro-duced by alpha-rhythm currents,” Science, vol. 161, no. 3843, pp. 784–786, 1968.

[12] F. F. Jobsis, “Noninvasive, infrared monitoring of cerebral and myocar-dial oxygen sufficiency and circulatory parameters,” Science, vol. 198,no. 4323, pp. 1264–1267, 1977.

[13] A. Villringer, J. Planck, C. Hock, L. Schleinkofer, and U. Dirnagl, “Nearinfrared spectroscopy (NIRS): a new tool to study hemodynamic changesduring activation of brain function in human adults,” Neuroscienceletters, vol. 154, no. 1, pp. 101–104, 1993.

[14] S. Ogawa, D. W. Tank, R. Menon, J. M. Ellermann, S. G. Kim,H. Merkle, and K. Ugurbil, “Intrinsic signal changes accompanyingsensory stimulation: functional brain mapping with magnetic resonanceimaging,” Proceedings of the National Academy of Sciences, vol. 89,no. 13, pp. 5951–5955, 1992.

[15] K. K. Kwong, J. W. Belliveau, D. A. Chesler, I. E. Goldberg, R. M.Weisskoff, B. P. Poncelet, D. N. Kennedy, B. E. Hoppel, M. S.Cohen, and R. Turner, “Dynamic magnetic resonance imaging of humanbrain activity during primary sensory stimulation.,” Proceedings of theNational Academy of Sciences, vol. 89, no. 12, pp. 5675–5679, 1992.

[16] W. Lennox, E. Gibbs, and F. Gibbs, “The brain-wave pattern, anhereditary trait; evidence from 74 normal pairs of twins,” Journal ofHeredity, 1945.

[17] M. Poulos, M. Rangoussi, V. Chrissikopoulos, and A. Evangelou,“Person identification based on parametric processing of the EEG,” inElectronics, Circuits and Systems, 1999. Proceedings of ICECS’99. The6th IEEE International Conference on, pp. 283–286, IEEE, 1999.

[18] R. Paranjape, J. Mahovsky, L. Benedicenti, and Z. Koles, “The electroen-cephalogram as a biometric,” in Electrical and Computer Engineering,2001. Canadian Conference on, vol. 2, pp. 1363–1366 vol.2, 2001.

[19] S. Sun, “Multitask learning for EEG-based biometrics,” in PatternRecognition, 2008. ICPR 2008. 19th International Conference on, pp. 1–4, IEEE, Dec 2008.

[20] K. Brigham and B. Kumar, “Subject identification from electroen-cephalogram (EEG) signals during imagined speech,” in Biometrics:Theory Applications and Systems (BTAS), 2010 Fourth IEEE Interna-tional Conference on, pp. 1–8, IEEE, 2010.

[21] P. Campisi and D. La Rocca, “Brain waves for automatic biometric-based user recognition,” Information Forensics and Security, IEEETransactions on, vol. 9, pp. 782–800, May 2014.

[22] P. C. Richardson, F. K. Coombs, and R. M. Adams, “Some new electrodetechniques for long-term physiologic monitoring,” Aerospace medicine,vol. 39, pp. 745–750, Jul 1968.

[23] G. E. Bergey, R. D. Squires, and W. C. Sipple, “Electrocardiogramrecording with pasteless electrodes,” IEEE Transactions on BiomedicalEngineering, vol. BME-18, pp. 206–211, May 1971.

[24] F. Popescu, S. Fazli, Y. Badower, B. Blankertz, and K.-R. Muller, “Singletrial classification of motor imagination using 6 dry EEG electrodes,”PLoS ONE, vol. 2, no. 7, p. e637, 2007.

[25] C. Grozea, C. Voinescu, and S. Fazli, “Bristle-sensors - Low-costFlexible Passive Dry EEG Electrodes for Neurofeedback and BCIapplications,” Journal of Neural Engineering, vol. 8, p. 025008, 2011.

[26] J. J. S. Norton, D. S. Lee, J. W. Lee, W. Lee, O. Kwon, P. Won, S.-Y. Jung, H. Cheng, J.-W. Jeong, A. Akce, S. Umunna, I. Na, Y. H.Kwon, X.-Q. Wang, Z. Liu, U. Paik, Y. Huang, T. Bretl, W.-H. Yeo,and J. A. Rogers, “Soft, curved electrode systems capable of integrationon the auricle as a persistent braincomputer interface,” Proceedings ofthe National Academy of Sciences, vol. 112, no. 13, pp. 3920–3925,2015.

[27] D. Looney, P. Kidmose, and D. P. Mandic, “Ear-EEG: user-centeredand wearable BCI,” in Brain-Computer Interface Research, pp. 41–50,Springer, 2014.

[28] V. Goverdovsky, D. Looney, P. Kidmose, and D. P. Mandic, “In-earEEG from viscoelastic generic earpieces: robust and unobtrusive 24/7monitoring,” Sensors Journal, IEEE, vol. 16, no. 1, pp. 271–277, 2016.

[29] J. J. Norton, D. S. Lee, J. W. Lee, W. Lee, O. Kwon, P. Won, S.-Y.Jung, H. Cheng, J.-W. Jeong, A. Akce, et al., “Soft, curved electrodesystems capable of integration on the auricle as a persistent brain–computer interface,” Proceedings of the National Academy of Sciencesof the United States of America, vol. 112, no. 13, pp. 3920–3925, 2015.

[30] S. Debener, R. Emkes, M. De Vos, and M. Bleichner, “Unobtrusiveambulatory EEG using a smartphone and flexible printed electrodesaround the ear,” Scientific reports, vol. 5, 2015.

[31] C. Ashby, A. Bhatia, F. Tenore, and J. Vogelstein, “Low-cost elec-troencephalogram (EEG) based authentication,” in Neural Engineering(NER), 2011 5th International IEEE/EMBS Conference on, pp. 442–445,IEEE, 2011.

[32] B. Blankertz, C. Sannelli, S. Halder, E. M. Hammer, A. Kubler, K.-R.Muller, G. Curio, and T. Dickhaus, “Neurophysiological predictor of



12

SMR-based BCI performance,” NeuroImage, vol. 51, no. 4, pp. 1303–1309, 2010.

[33] J. Polich, “Updating P300: an integrative theory of P3a and P3b,” Clin-ical Neurophysiology : Official Journal of the International Federationof Clinical Neurophysiology, vol. 118, pp. 2128–2148, Oct 2007.

[34] M. S. Treder and B. Blankertz, “(C)overt attention and visual spellerdesign in an ERP-based brain-computer interface,” Behavioral and BrainFunctions, vol. 6, p. 28, May 2010.

[35] M. Schreuder, B. Blankertz, and M. Tangermann, “A new auditorymulti-class brain-computer interface paradigm: Spatial hearing as aninformative cue,” PLoS ONE, vol. 5, no. 4, p. e9813, 2010.

[36] L. Acqualagna and B. Blankertz, “Gaze-independent BCI-spelling usingrapid visual serial presentation (RSVP),” Clinical Neurophysiology :Official Journal of the International Federation of Clinical Neurophys-iology, vol. 124, pp. 901–908, May 2013.

[37] M. C. Potter, “Rapid serial visual presentation (RSVP): A method forstudying language processing,” New methods in reading comprehensionresearch, vol. 118, pp. 91–118, 1984.

[38] B. Blankertz, M. Tangermann, C. Vidaurre, S. Fazli, C. Sannelli,S. Haufe, C. Maeder, L. E. Ramsey, I. Sturm, G. Curio, and K.-R.Muller, “The Berlin Brain-Computer Interface: Non-medical uses of BCItechnology,” Frontiers in Neuroscience, vol. 4, p. 198, 2010.

[39] B. Rossion and G. Pourtois, “Revisiting Snodgrass and Vanderwart’sobject pictorial set: The role of surface detail in basic-level objectrecognition,” Perception, vol. 33, no. 2, pp. 217–236, 2004.

[40] J. G. Snodgrass and M. Vanderwart, “A standardized set of 260 pictures:norms for name agreement, image agreement, familiarity, and visualcomplexity,” Journal of experimental psychology: Human learning andmemory, vol. 6, no. 2, p. 174, 1980.

[41] R. F. Tate, “Correlation between a discrete and a continuous vari-able. Point-biserial correlation,” The Annals of Mathematical Statistics,vol. 25, pp. 603–607, Sep. 1954.

[42] H. Hotelling, “New light on the correlation coefficient and its Trans-forms,” Journal of the Royal Statistical Society. Series B (Methodolog-ical), vol. 15, no. 2, pp. 193–232, 1953.

[43] C. E. Bonferroni, Teoria statistica delle classi e calcolo delle probabilita.Libreria internazionale Seeber, 1936.

[44] R. A. Fisher, “The use of multiple measurements in taxonomic prob-lems,” Annals of eugenics, vol. 7, no. 2, pp. 179–188, 1936.

[45] J. H. Friedman, “Regularized discriminant analysis,” Journal of theAmerican Statistical Association, vol. 84, no. 405, pp. 165–175, 1989.

[46] C. Stein, “Inadmissibility of the usual estimator for the mean of amultivariate normal distribution,” in Proc. 3rd Berkeley Sympos. Math.Statist. Probability, vol. 1, pp. 197–206, 1956.

[47] B. Blankertz, S. Lemm, M. S. Treder, S. Haufe, and K.-R. Muller,“Single-trial analysis and classification of ERP components – a tutorial,”NeuroImage, vol. 56, pp. 814–825, 2011.

[48] J. Schafer and K. Strimmer, “A shrinkage approach to large-scalecovariance matrix estimation and implications for functional genomics,”Statistical applications in genetics and molecular biology, vol. 4, p. Ar-ticle32, 2005.

[49] S. Lemm, B. Blankertz, T. Dickhaus, and K.-R. Muller, “Introduction tomachine learning for brain imaging,” NeuroImage, vol. 56, pp. 387–399,2011.

[50] T. Fawcett, “An introduction to ROC analysis,” Pattern recognitionletters, vol. 27, no. 8, pp. 861–874, 2006.

[51] R. Sandhu and P. Samarati, “Authentication, access control, and audit,”ACM Computing Surveys (CSUR), vol. 28, no. 1, pp. 241–243, 1996.

[52] S. H. Khan, M. A. Akbar, F. Shahzad, M. Farooq, and Z. Khan, “Securebiometric template generation for multi-factor authentication,” PatternRecognition, vol. 48, no. 2, pp. 458–472, 2015.

[53] M. D. Hafiz, A. H. Abdullah, N. Ithnin, and H. K. Mammi, “Towardsidentifying usability and security features of graphical password inknowledge based authentication technique,” in Modeling & Simulation,2008. AICMS 08. Second Asia International Conference on, pp. 396–403, IEEE, 2008.

[54] T. Khodadadi, M. Alizadeh, S. Gholizadeh, M. Zamani, and M. Darvishi,“Security analysis method of recognition-based graphical password,”Jurnal Teknologi, vol. 72, no. 5, pp. 57–62, 2015.

[55] D. Malone and K. Maher, “Investigating the distribution of passwordchoices,” in Proceedings of the 21st international conference on WorldWide Web, pp. 301–310, ACM, 2012.

[56] S. Komanduri, R. Shay, L. F. Cranor, C. Herley, and S. Schechter,“Telepathwords: Preventing weak passwords by reading users’ minds,”in 23rd USENIX Security Symposium (USENIX Security 14), (San Diego,California), pp. 591–606, USENIX Association, 2014.

[57] S. Schechter, C. Herley, and M. Mitzenmacher, “Popularity is every-thing: A new approach to protecting passwords from statistical-guessingattacks,” in Proceedings of the 5th USENIX conference on Hot topics insecurity, pp. 1–8, USENIX Association, 2010.

[58] G. Cormode and S. Muthukrishnan, “An improved data stream summary:the count-min sketch and its applications,” Journal of Algorithms,vol. 55, no. 1, pp. 58 – 75, 2005.

[59] D. Florencio, C. Herley, and P. C. Van Oorschot, “Password portfoliosand the finite-effort user: Sustainably managing large numbers of ac-counts,” in Proc. USENIX Security, pp. 575–590, USENIX Association,2014.

[60] R. Biddle, S. Chiasson, and P. C. Van Oorschot, “Graphical passwords:Learning from the first twelve years,” ACM Computing Surveys (CSUR),vol. 44, no. 4, pp. 1–41, 2012.

[61] E. A. Kirkpatrick, “An experimental study of memory,” PsychologicalReview, vol. 1, pp. 602–609, November 1894.

[62] R. N. Shepard, “Recognition memory for words, sentences, and pic-tures,” Journal of Verbal Learning and Verbal Behavior, vol. 6, no. 1,pp. 156–163, 1967.

[63] I. Jermyn, A. J. Mayer, F. Monrose, M. K. Reiter, A. D. Rubin, et al.,“The design and analysis of graphical passwords,” in Usenix Security,pp. 1–14, USENIX Association, 1999.

[64] G. E. Blonder, “Graphical password,” 1996. US Patent 5,559,961.[65] X. Suo, Y. Zhu, and G. Owen, “Graphical passwords: a survey,” in

Computer Security Applications Conference, 21st Annual, pp. 10 pp.–472, Dec 2005.

[66] X. Suo, “A design and analysis of graphical password,” Master’s thesis,Georgia State University, Atlanta, Georgia, 2006.

[67] L. Sobrado and J.-C. Birget, “Graphical passwords,” The RutgersScholar, an electronic Bulletin for undergraduate research, vol. 4,p. 2002, 2002.

[68] S. Wiedenbeck, J. Waters, L. Sobrado, and J.-C. Birget, “Design andevaluation of a shoulder-surfing resistant graphical password scheme,”in Proceedings of the working conference on Advanced visual interfaces,pp. 177–184, ACM, 2006.

[69] A. K. Jain, R. Bolle, and S. Pankanti, Biometrics: personal identificationin networked society. Kluwer Academic Publishers, 1999.

[70] A. K. Jain, A. Ross, and S. Prabhakar, “An introduction to biometricrecognition,” Circuits and Systems for Video Technology, IEEE Trans-actions on, vol. 14, no. 1, pp. 4–20, 2004.

[71] M. Kumar, T. Garfinkel, D. Boneh, and T. Winograd, “Reducingshoulder-surfing by using gaze-based password entry,” in Proceedingsof the 3rd symposium on Usable privacy and security, pp. 13–19, ACM,2007.

[72] T.-S. Wu, M.-L. Lee, H.-Y. Lin, and C.-Y. Wang, “Shoulder-surfing-proof graphical password authentication scheme,” International Journalof Information Security, vol. 13, no. 3, pp. 245–254, 2014.

[73] S. L. Osborn III, N. A. Davis, J. L. Sontag, and J. Norvell, “Methods andsystems for graphical image authentication,” Feb. 14 2012. US Patent8,117,458.

[74] A. K. Jain and D. Maltoni, Handbook of fingerprint recognition.Springer-Verlag New York, Inc., 2003.

[75] A. Ross and A. Jain, “A prototype hand geometry-based verificationsystem,” in Proceedings of 2nd Conference on Audio and Video BasedBiometric Person Authentication, pp. 166–171, 1999.

[76] R. Sanchez-Reillo, C. Sanchez-Avila, and A. Gonzalez-Marcos, “Bio-metric identification through hand geometry measurements,” PatternAnalysis and Machine Intelligence, IEEE Transactions on, vol. 22,no. 10, pp. 1168–1171, 2000.

[77] K. W. Bowyer, K. Hollingsworth, and P. J. Flynn, “Image understandingfor iris biometrics: A survey,” Computer vision and image understand-ing, vol. 110, no. 2, pp. 281–307, 2008.

[78] Y. Wang, T. Tan, and A. K. Jain, “Combining face and iris biometricsfor identity verification,” in Audio-and Video-Based Biometric PersonAuthentication, pp. 805–813, Springer, 2003.

[79] M. Turk, A. P. Pentland, et al., “Face recognition using eigenfaces,” inComputer Vision and Pattern Recognition, 1991. Proceedings CVPR’91.,IEEE Computer Society Conference on, pp. 586–591, IEEE, 1991.

[80] W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, “Face recog-nition: A literature survey,” ACM computing surveys (CSUR), vol. 35,no. 4, pp. 399–458, 2003.

[81] F. Monrose and A. Rubin, “Authentication via keystroke dynamics,” inProceedings of the 4th ACM conference on Computer and communica-tions security, pp. 48–56, ACM, 1997.

[82] F. Bergadano, D. Gunetti, and C. Picardi, “User authentication throughkeystroke dynamics,” ACM Transactions on Information and SystemSecurity (TISSEC), vol. 5, no. 4, pp. 367–397, 2002.



13

[83] M. Schreuder, J. Hohne, B. Blankertz, S. Haufe, T. Dickhaus, andM. Tangermann, “Optimizing ERP based BCI - a systematic evaluationof dynamic stopping methods,” Journal of Neural Engineering, vol. 10,no. 3, p. 036025, 2013.

[84] L. F. Seoane, S. Gabler, and B. Blankertz, “Images from the mind: BCIimage evolution based on rapid serial visual presentation of polygonprimitives,” Brain-Computer Interfaces, vol. 2, no. 1, pp. 40–56, 2015.

[85] S. Dahne, F. Bießman, W. Samek, S. Haufe, D. Goltz, C. Gundlach,A. Villringer, S. Fazli, and K.-R. Muller, “Multivariate machine learningmethods for fusing functional multimodal neuroimaging data,” Proceed-ings of the IEEE, vol. 103, no. 9, pp. 1507–1530, 2015.

[86] S. Fazli, S. Dahne, W. Samek, F. Bießmann, and K.-R. Muller, “Learningfrom more than one data source: data fusion techniques for sensorimotorrhythm-based Brain-Computer Interfaces,” Proceedings of the IEEE,vol. 103, no. 6, pp. 891–906, 2015.

[87] M. Krauledat, M. Tangermann, B. Blankertz, and K.-R. Muller, “To-wards zero training for brain-computer interfacing,” PLoS ONE, vol. 3,p. e2967, Aug 2008.

[88] E. Maiorana, D. La Rocca, and P. Campisi, “On the permanence of EEGsignals for biometric recognition,” Information Forensics and Security,IEEE Transactions on, vol. PP, no. 99, pp. 1–1, 2015. in press.

[89] S. Fazli, F. Popescu, M. Danoczy, B. Blankertz, K.-R. Muller, andC. Grozea, “Subject-independent mental state classification in singletrials,” Neural networks : the official journal of the International NeuralNetwork Society, vol. 22, pp. 1305–1312, Jun 2009.

[90] M. Alamgir, M. Grosse-Wentrup, and Y. Altun, “Multitask learning forbrain-computer interfaces,” in JMLR Workshop and Conference Proceed-ings Volume 9: AISTATS 2010 (M. T. Teh, Y.W., ed.), (Cambridge, MA,USA), pp. 17–24, Max-Planck-Gesellschaft, JMLR, May 2010.

[91] S. Fazli, M. Danoczy, J. Schelldorfer, and K.-R. Muller, “L1-penalizedLinear Mixed-Effects Models for high dimensional data with applicationto BCI,” NeuroImage, vol. 56, no. 4, pp. 2100 – 2108, 2011.

[92] P.-J. Kindermans, D. Verstraeten, and B. Schrauwen, “A bayesian modelfor exploiting application constraints to enable unsupervised training ofa P300-based BCI,” PLoS ONE, vol. 7, no. 4, p. e33758, 2012.

[93] H. Morioka, A. Kanemura, J.-i. Hirayama, M. Shikauchi, T. Ogawa,S. Ikeda, M. Kawanabe, and S. Ishii, “Learning a common dictionaryfor subject-transfer decoding with resting calibration,” NeuroImage,vol. 111, pp. 167–178, 2015.

Yiyu Chen received her B.S. degree in Computerand Communication Engineering from Korea Uni-versity, Seoul, Korea, in 2014. Since 2014, shehas enrolled in the integrated Master’s and Ph.D.course at the Department of Brain and CognitiveEngineering, Korea University, Seoul, Korea. Hercurrent research interests include Brain ComputerInterfaces, Machine Learning, decision making andCognitive Neuroscience.

Ayalneh Dessalegn Atnafu received his B.Techin Electrical Engineering from Defense UniversityCollege, Debre zeit, Ethiopia, in 2003, MSc in Com-puter Engineering from Addis Ababa University,Addis Ababa, Ethiopia, in 2008. He is currentlya PhD candidate in Information Security at KoreaUniversity, since 2012. His research interests includeDigital forensics, information hiding, Authenticationand Information Security Management.

Isabella Schlattner is in the integrated Master’sand Ph.D. program at the Department of Brain andCognitive Engineering at Korea University. She re-ceived her Bachelor’s degree in Biology at the FreieUniversity Berlin, Germany. Her current researchinterests include cognitive processes, especially in-telligence and creativity and their underlying brain-based mechanisms. Furthermore, she is interested inbrain imaging techniques, such as EEG, NIRS aswell as fMRI.

Wendimagegn Tariku Weldtsadik received hisBSC in Mathematics from Addis Ababa University,Ethiopia, in 1998. MSc in Computer Science fromAddis Ababa University, Ethiopia, in 2009. He iscurrently a PhD candidate in Information Securityat Korea University, since 2013. His research inter-ests include Face Recognition, information hiding,Authentication and EEG signal analysis.

Myung-Cheol Roh received his B.S. degree inComputer Engineering from Kangwon University,Chun-Choen, Korea, in 2001, and the MS and PhDdegrees in Computer Science and Engineering fromKorea University, Seoul, Korea, in 2003 and 2008.Currently, he is working as a managing researcher atS1, Seoul, Korea. He won the best paper award of the25th annual paper competition which is supervisedby the Korea Information Science Society and issponsored by Microsoft in 2006. He worked at theCenter for Vision, Speech and Signal Processing in

the University of Surrey, UK, as a collaborate researcher in 2004 and at theRobotics Institue in Carnegie Mellon University, US, as a researcher from2008 to 2012. His present research interests include face alignment, face andgesture recognition, robot vision and the pattern recognition related fields.

Hyoung Joong Kim received his B.S., M.S., andPh.D. degrees from Seoul National University, Ko-rea, in 1978, 1986, and 1989, respectively. He joinedthe faculty of Kangwon National University, Korea,in 1989. He is currently a Professor of Korea Uni-versity, Korea. He invented fast lossless compressionalgorithm using reversible data hiding technique. Healso invented image hash algorithm which is robustagainst histogram equalization attack. He publishednumerous papers including more than 40 peer-reviewed journal papers. He served Guest Editor of

several journals including IEEE Transactions on Circuits and Systems forVideo Technology. He is a Vice Editor-in-Chief of the LNCS Transactions onData Hiding and Multimedia Security. He was a prime investigator of severalnational R&D projects. His main research interests include high-performancecomputing and multimedia computing.



14

Seong-Whan Lee received the B.S. degree in com-puter science and statistics from Seoul National Uni-versity, Korea, in 1984, and the M.S. and Ph.D. de-grees in computer science from the Korea AdvancedInstitute of Science and Technology (KAIST), Seoul,Korea, in 1986 and 1989, respectively. Currently,he is the Hyundai-Kia Motor Chair Professor atKorea University, Seoul, where he is the Head ofthe Department of Brain and Cognitive Engineering.His research interests include artificial intelligence,pattern recognition, and brain engineering. He is a

Fellow of the IEEE, the IAPR, and the Korean Academy of Science andTechnology.

Benjamin Blankertz received his PhD in mathemat-ics in 1998 and pursued several studies in music cog-nition. He started Brain-Computer Interface researchin 2000 and became chair for Neurotechnology atTechnische Universitat Berlin in 2012. The BerlinBCI group is known for innovative machine learningapproaches in the field of BCI and the developmentof novel experimental paradigms. This includes, thetransfer of BCI technology from the lab to real worldapplications.

Siamac Fazli received his B.Sc. Physics degree fromthe University of Exeter in 2002, his M.Sc. in Med-ical Neurosciences from the Humboldt UniversityBerlin in 2004 and his Ph.D. from the Berlin Instituteof Technology in 2011. From 2011-2013 he workedas a Postdoc researcher at the Berlin Institute ofTechnology for the Bernstein Focus Neurotechnol-ogy. Since 2013 he works as an Assistant Professorat Korea University. His current research interestsinclude neuroscience, machine learning, multi-modalneuroimaging and brain-computer interfacing.

a high-security eeg-based login system with rsvp stimuli ......to date, a number of different...

Documents