Research ArticleCovert Intention to Answer ldquoYesrdquo or ldquoNordquo Can BeDecoded from Single-Trial Electroencephalograms (EEGs)

Jeong Woo Choi 12 and Kyung Hwan Kim 1

1Department of Biomedical Engineering Yonsei University Wonju 26493 Republic of Korea2Department of Neurosurgery University of California Los Angeles CA 90095 USA

Correspondence should be addressed to Kyung Hwan Kim khkim0604yonseiackr

Received 19 March 2019 Revised 14 May 2019 Accepted 13 June 2019 Published 10 July 2019

Guest Editor Hyun J Baek

Copyright copy 2019 JeongWoo Choi and KyungHwan Kim-is is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in anymedium provided the original work isproperly cited

Interpersonal communication is based on questions and answers and the most useful and simplest case is the binary ldquoyes or nordquoquestion and answer -e purpose of this study is to show that it is possible to decode intentions on ldquoyesrdquo or ldquonordquo answers frommultichannel single-trial electroencephalograms which were recorded while covertly answering to self-referential questions witheither ldquoyesrdquo or ldquonordquo -e intention decoding algorithm consists of a common spatial pattern and support vector machine whichare employed for the feature extraction and pattern classification respectively after dividing the overall time-frequency range intosubwindows of 200mstimes 2Hz -e decoding accuracy using the information within each subwindow was investigated to finduseful temporal and spectral ranges and found to be the highest for 800ndash1200ms in the alpha band or 200ndash400ms in the thetaband When the features from multiple subwindows were utilized together the accuracy was significantly increased up to sim86-e most useful features for the ldquoyesnordquo discrimination was found to be focused in the right frontal region in the theta band andright centroparietal region in the alpha band which may reflect the violation of autobiographic facts and higher cognitive load forldquonordquo compared to ldquoyesrdquo Our task requires the subjects to answer self-referential questions just as in interpersonal conversationwithout any self-regulation of the brain signals or high cognitive efforts and the ldquoyesrdquo and ldquonordquo answers are decoded directly fromthe brain activities -is implies that the ldquomind readingrdquo in a true sense is feasible Beyond its contribution in fundamentalunderstanding of the neural mechanism of human intention the decoding of ldquoyesrdquo or ldquonordquo from brain activities may eventuallylead to a natural brain-computer interface

1 Introduction

-emost fundamental linguistic communication consists ofquestions and answers and the simplest one is the binaryldquoyes or nordquo question and answer -is enables fundamentalinterpersonal communications (eg ldquoIs your name JohnrdquoldquoYesrdquo or ldquoDo you want to drink waterrdquo ldquoNordquo) So bydecoding the intentions to answer either ldquoyesrdquo or ldquonordquo frombrain activities a natural interpersonal communication toolwhich does not require any operant training or heavycognitive efforts may be developed As the first step towardthis here we tried to demonstrate that it is possible to decodethe intentions to answer ldquoyesrdquo or ldquonordquo in response to self-referential questions from noninvasive electroencephalo-grams (EEGs) on single-trial basis -is was motivated by

our recent studies which showed that the intentions toanswer ldquoyesrdquo or ldquonordquo to self-referential questions is repre-sented significantly differently in event-related EEGs par-ticularly in alpha-band activities [1 2]

Direct decoding of ldquoyesrdquo and ldquonordquo intentions mayeventually lead to advancement of the brain-computer in-terface (BCI) which is a technological means to deliveruserrsquos intention to the external world (device or otherpeople) without behavioral outputs by direct interpretationof brain activities -e most important target of the BCI isthe patients with severe motor impairment who are unableto communicate with others including those in the com-pletely locked-in state (CLIS) due to amyotrophic lateralsclerosis spinal cord injury and brainstem stroke [3ndash6] Oneof the most crucial technologies to enable the BCI is to read

HindawiComputational Intelligence and NeuroscienceVolume 2019 Article ID 4259369 11 pageshttpsdoiorg10115520194259369

or ldquodecoderdquo the usersrsquo intention from their brain activitiesTwo major approaches have been pursued for the intentiondecoding -e first is based on voluntary self-regulation ofspecific brain signals such as slow cortical potential [7] andsensorimotor rhythms [8] -is requires extensive operanttraining using feedback and reward Unfortunately manypeople are unable to regulate the brain activities as requiredwhich is known as ldquoBCI illiteracyrdquo [9 10]-e other approachutilizes evoked brain activities such as P300 event-relatedpotential (ERP) [11 12] and steady-state evoked potential[13 14] -e operant training is not required but sustainedattention is needed to induce discriminable brain responseincreases resulting in significant cognitive workload

Both approaches may not be so successful for the pa-tients with CLIS [15] It is speculated that the failure is due tothe extinction of goal-directed cognition and thought in theCLIS patients [15] An alternative approach for the mindreading is crucial which does not require volitional andhighly cognitive efforts Birbaumer and colleagues suggestedan approach based on classical conditioning [16ndash18] -eytried to associate language stimuli with unpleasant andpainful sensory stimuli so that cortical responses to thesenonlanguage stimuli are conditioned according to the lan-guage stimuli -is is remarkable considering that languageis the most natural means of communication

-e specific aim of this study is to show that it is feasibleto decode ldquoyesrdquo and ldquonordquo answers in mind from single-trialEEGs We demonstrated that mind reading in a true sensewhich is based on the prediction of the intentions to answerthe questions from brain activities is achievable For theintention decoding the discriminative characteristics ofEEGs that we found in our previous study were utilized tofind the time-frequency features for ldquoyesnordquo decoding -edecoding algorithm was developed based on the same dataused in our previous study [2]

2 Materials and Methods

21 Subjects 23 subjects with no record of neurological orpsychiatric illness participated in the experiment (age2313plusmn 297 years 12 males) All the subjects were un-dergraduate students of Yonsei University and right-handednative Korean speakers Written informed consent wasobtained from each subject before the experiment -eexperimental procedure was approved by the Yonsei Uni-versity Wonju Institutional Review Board (IRB) All ex-periments were performed in accordance with the guidelinesand regulations of the IRB

22 Experimental Task Before the experiment all subjectscompleted a written questionnaire on their autobiographicalfacts (eg job name age and date of birth) We generatedtwo opposite types of questions from a single autobio-graphical fact one question should be answered ldquoyesrdquo andthe other (ie autobiographical fact violation (AFV)) shouldbe answered ldquonordquo-ese two questions were almost the sameexcept one critical word (italicized word in the examplebelow) which determined whether the question agreed with

the subjectrsquos identity or not For example if the subjectrsquos jobwas a student the two questions were as follows

Type (a) to be answered ldquoyesrdquo Is your job a studentType (b) to be answered ldquonordquo Is your job a teacher

In total 40 type (a) questions and 40 type (b) questionswere generated based on the questionnaire for each subjectEach question was composed of 2 or 3 Korean words andthe average number of characters (Korean ldquoHangulrdquo) in eachcritical word was 318plusmn 102 Each character had 33 cmwidth and 427 cm height

All questions were presented visually through com-mercial software (PRESENTATION Neurobehavioral sys-tems Berkeley CA) After explaining the detailed procedureof the experimental task we requested the subjects to watcheach word presented on a 17 inch computer screen carefullyso that they can make immediate response as soon aspossible to the critical words -e distance between thesubjectrsquos eyes and the monitor was set to sim075m Each wordin a question was presented sequentially one by one on thecenter of the monitor as described below

Figure 1(a) illustrates the experimental procedure Across mark (ldquo+rdquo) for the fixation appeared for 1000ms and ablack screen followed for 300ms And then each word in aquestion was presented sequentially for 300ms with a blackscreen for 300ms between the words -e last word in thequestion is referred to the critical word (CW) which waspresented for 300ms along with a question mark Althoughthis question mark may naturally induce decision of ldquoyesrdquo orldquonordquo and thus evoke answer automatically we instructed thesubjects not to make any response neither covertly norovertly but to retain the answer in mind during the 1 s blankperiod -is would enable us to explore the cortical activityduring retaining the information on ldquoyesrdquo or ldquonordquo inworking memory (WM) Finally when ldquoPlease respondrdquo (inKorean) was presented for 300ms the subjects wererequested to respond covertly in mind with either ldquoyesrdquo orldquonordquo without any behavioral responses

Figure 1(b) illustrates expected temporal sequence ofcognitive processing following the CW onset until theldquoPlease respondrdquo cue appeared which was based on ourprevious studies on cortical information processing of in-tention [1 2] which showed that the brain activities differedbetween ldquoyesrdquo and ldquonordquo answers at both early (0sim600ms)and late periods (600sim1300ms) relative to the CW onset Wefound that the early period was associated with semanticprocessing and automatic decision to answer [1] (denoted bya red box in Figure 1(b)) while the late period was involvedin the retention of the answer in memory (denoted by a bluebox in Figure 1(b)) until the ldquoRespond cuerdquo appeared(denoted by a yellow box in Figure 1(b)) [2] -us thetemporal period of interest for decoding the intentions toanswer ldquoyesrdquo or ldquonordquo was the late period corresponding toretain the intention in mind (600sim1300ms)

Each subject performed two blocks of tasks Each blockincluded all questions generated based on the questionnaire(ie 40 type (a) and 40 type (b) questions) and 10 of 40questions for each question type were randomly selected and

2 Computational Intelligence and Neuroscience

presented once again Consequently each block included 50type (a) and 50 type (b) questions in total e averageduration of each single trial (ie one question and answer)was 4380plusmn 27495ms e total time for performing thetasks was approximately 20minutes including at least5minutes of rest between blocks

23 Electroencephalogram (EEG) Recording and DataAnalysis Sixty channel EEGs were recorded based on the10ndash10 system using an EEG amplier (Brain ProductsGmbH Munich Germany) with an AgAgCl electrode cap(EASYCAP FMS Munich Germany) e ground andreference electrodes were at AFz and linked mastoids re-spectively e impedances of all electrodes were kept under10 kΩ e sampling rate was 500 sampless A bandpasslter (003ndash100Hz) and a notch lter (60Hz) were applied inorder to reduce background noise and powerlineinterferences

An open source toolbox EEGLAB was used for the wholeprocedure of preprocessing [19] First single-trial EEGswere segmented during the minus500sim1300ms period relative tothe critical word onset By visual inspection we removed thesingle-trial waveforms contaminated severely from non-stereotyped artifacts such as drifts and discontinuity enan independent component analysis (ICA) was employed tothe remaining single-trial EEGs in order to correct the ocularand muscular artifacts [20] e group-averaged percentageof the number of epochs remaining per subject was 9888plusmn308 and 9796plusmn 586 for ldquoyesrdquo and ldquonordquo questionsrespectively

24 YesNo Decoding Figure 2(a) illustrates the structure ofldquoyesnordquo intention decoding algorithm using local time-frequency information First we selected 29 channels out of60 (ie Fp1 Fpz Fp2 F7 F3 Fz F4 F8 FC5 FC1 FC2 FC6T7 C3 Cz C4 T8 CP5 CP1 CP2 CP6 P7 P3 Pz P4 P8O1 Oz and O2) following the standard 10ndash20 system iswas based on a recent simulation study which showed thatthe decoding accuracy with the common spatial pattern(CSP) spatial ltering was optimized when sim30 channelswere used and decreased for more channels [21] e overall

time-frequency range (0ndash1200ms 4ndash50Hz) was divided intosubwindows of 200mstimes 2Hz e intention decodingwithin each of the local time-frequency subwindows wasperformed as follows Single-trial EEGs were bandpass l-tered in the frequency range of the subwindow using alinear-phase nite impulse response lter (the number of thelter order 512 bandwidth 2Hz) e multichannelbandpass-ltered signals within the temporal period of thesubwindow were subsequently projected to the lower di-mension (four dimensions) by the CSP algorithm [22] efour time series obtained from the CSP spatial lter wereused to construct a four-dimensional feature vector whichwas passed to a support vectormachine (SVM) classierenal output of the classier was either ldquoyesrdquo or ldquonordquo adecision of answer for each single trial

e performance of the trained classier was validatedby 10-fold cross-validation as follows First for each classwe randomly split all the trials into 10-folds with the samenumber of trials (ie sim10 trials per fold for each class)en we randomly selected one fold (kth where k 1 2 10) as a test data (10) and trained the classier usingthe rest of data (ie 9 folds excepting the kth fold 90) Inorder to keep a balance between the numbers of ldquoyesrdquo andldquonordquo trials the trainingtesting data were selected withineach class (ie ldquoyesrdquo or ldquonordquo) as shown in Figure 2(b) eground truth for each single trial was determined whetherthe question in the single trial was including AFV or note decoding accuracy for each subject was estimated byaveraging the ratio of correct classication from 10 rep-etitions (ie k 1 2 10) of this procedure

Additionally we also made ecentective use all the featuresobtained from multiple time-frequency subwindows inorder to investigate whether more accurate decoding ispossible by combining useful features each of which waslocalized in the time-frequency domain e time-fre-quency subwindows were selected if the decoding accu-racies for a specic subwindow were higher than apredetermined threshold (2 times standard deviation above themean among all time-frequency subwindows) And thenthe classier was trained and tested as described abovewith input feature vectors obtained by combining all theselected subwindows

PleaserespondStudentJob+

Fixation

1000 ms 300 ms 300 ms 700 ms300 ms 1000 ms

Blank Critical word

Cue

0 1000 500 1500 Time relative to the critical word (CW) onset (ms)

Semantic processingautomatic decision

Retainingldquoyesnordquo in mind

Covertresponse

Stimuli

300 ms300 ms

CW Blank Cue

(b)

(a)1st word

Expected cognitive processing

Figure 1 Experimental task (a) presentation of stimuli (b) expected temporal sequence of cognitive processing

Computational Intelligence and Neuroscience 3

25 Event-Related Spectral Perturbation (ERSP) Analysise time-frequency activation patterns ie ERSPs wereinvestigated to reveal statistical dicenterences between ldquoyesrdquoand ldquonordquo to nd the time and frequency ranges of interestsfor ecentective classication A` continuous wavelet transform(CWT) based on a complex Morlet wavelet was used for theERSP analysis [23] e number of cycles for the CWTlinearly increased according to the frequency from 4 to 135at the lowest (1Hz) and the highest frequencies (100Hz)respectively [19] is method provides better frequencyresolution at high frequencies and it is better matched to thelinear scale that we adopted to visualize the time-frequencymap [19] e induced spectral power was calculated byaveraging the ERSP patterns of each single trial [24] etime-frequency distribution of ERSP patterns was repre-sented as the ratio of the relative change to the power in abaseline interval from minus300 to 0ms prior to stimulus onsetto reduce intersubject variability and to normalize powerchanges across dicenterent frequency bands

We employed the mass-univariate approach with thecluster-based permutation test for correcting multiple com-parisons [25] in order to nd the time frequency and elec-trode showing signicant dicenterences between ldquoyesrdquo and ldquonordquowithout a priori knowledge Detailed procedure is as follows

(1) A large number of paired-sample t-tests were appliedto the data for all time-frequency-electrode binswithin the range of 0ndash1200ms (time) 5ndash30Hz(frequency) and 29 electrodes e number of binswas 181714 241times 26times 29 since there were 241 timesamples 26 frequency points and 29 electrodes e

electrodes showing high t values were selected andthe average power spectral power was calculated overthe selected electrodes as follows First from spatialdistribution of the t values averaged within thefrequency band of interest (eg theta band 4ndash8Hzalpha band 8ndash13Hz) during the overall time period(0ndash1200ms) the electrodes showing higher p valuesabove a predetermined threshold (the upper 10highest value) were selected e average powerspectral power was calculated over the selectedelectrodes for the next step

(2) After signicant locations were found in step 1 time-frequency bins were screened to be signicantamong all 6266 (241times 26) bins if p values werebelow a predetermined threshold (plt 005) Acluster of time-frequency bins was formed if morethan two successive bins were selected along eithertime or frequency axis Sum of t values within thecluster tmass was then calculated and compared withthe null distribution of surrogate data to determinestatistical signicance of the cluster (above the highest5 of the null distribution) e null distribution oftmass was obtained from the largest values of tmass foreach of 5000 surrogate data which were derived byrandom permutation of ldquoyesrdquo and ldquonordquo answers

26 Feature Extraction by Common Spatial Pattern (CSP)Filtering CSP is a mathematical procedure to derive aspatial lter which separates a multichannel signal into

Selecting time segment

Spatial filtering(CSP)

Training Filtered 4 time series

Spatial filtering

Spatial filter matrix (W)Test

29-channelsingle-trial EEGs

(90)

YesNo

Bandpassfiltering

Training classifier(SVM)

Selecting time segment

Bandpassfiltering

Classification(SVM)

29-channelsingle-trial EEGs

(10)Filtered

4 time series

Feature construction

Feature construction

4 features

4 features

(a)

Num

ber o

f tria

ls

0

20

40

60

80

100

5 10 15 20Subject

Yes (train)No (train)

Yes (test)No (test)

(b)

Figure 2 (a) Block diagram of the intention decoding algorithm (b) the number of trials which was selected as training and testing datawithin each answer type for each subject

4 Computational Intelligence and Neuroscience

additive subcomponents so that the differences of variancesare maximized between two classes -at is the most dis-criminative features between two classes are obtained bymaximizing the variance of the spatially filtered signal of oneclass while minimizing that of the other class [22] -e CSPalgorithm is recognized to be effective for the discriminationof mental states from event-related EEG spectral powers[26] -e results of the CSP can be visualized as a topo-graphic map on the scalp which facilitates interpretation offunctional neuroanatomical meanings [26]

-e CSP spatial filter W can be obtained by simulta-neous diagonalization of two covariance matrices of classes 1and 2 as follows

WTΣ1W Λ1

WTΣ2W Λ2(1)

where Λ1 + Λ2 I Σ1 and Σ2 represent the spatial co-variance matrices averaged over all single-trial EEGs foreach class and Λ1 and Λ2 denote the diagonal matrices -eprojection vector w (column vectors of W) can be ob-tained from a generalized eigenvalue decomposition asfollows

Σ1wk λkΣ2wk (2)

wherewk (k 1 C where C is the number of channels) isthe generalized eigenvector and λ1k wT

kΣ1wk and λ2k

wTkΣ2wk are defined as the k

th diagonal element ofΛ1 andΛ2respectively where λk λ1kλ2k Importantly λ1k and λ2k

(ranges from 0 to 1) reflect the variance for each class andλ1k + λ2k 1-us if λ1k is close to 1 λ2k should be close to0 -is means that corresponding projection vector wkshows high variance in class 1 but low variance in class 2-edifference in variances between these two classes enablesdiscriminating one class from another -e eigenvalues aresorted in the descending order during calculation meaningthat the first projection vector yields the highest variance forclass 1 (but the lowest for class 2) whereas the last projectionvector yields the highest variance for class 2 (but lowest forclass 1) -us the first and last projection vectors are themost useful for the discrimination [22]

-e spatial filter W provides the decomposition of asingle-trial multichannel EEG E as ZWTE where E isrepresented as a matrix with C (the number of channels)rows and T (the number of time samples) columns -ecolumns of Wminus1 form the common spatial patterns and canbe visualized as topographies on scalp -e variances of thespatially filtered time-series Z are calculated as features forthe classification as follows

fp logvar Zp1113872 1113873

11139362mi1var Zi( 1113857

⎛⎝ ⎞⎠ where p 1 2 2m (3)

where p is the number of features m was set to 2 whichmeans that the first 2 and last 2 projection vectors were usedas features and thus the number of features p was 4 for allclassifications -e log transformation was adopted to ap-proximate the normal distribution of the data

27 Pattern Classification Using Support Vector Machine(SVM) SVM has been recognized to be a practical androbust method for the classification of human brain signals[27 28] -e SVM is trained to determine an optimal hy-perplane by which the distance to the support vectors(closest to the separating boundary) is maximized [29 30]In the case of the linear SVM classification the hyperplaneaTx+ b satisfies

yi aTxi + b1113872 1113873ge 1minus ξi for i 1 N (4)

where xi fpi denotes a feature vector (in which p 1 4) which can be obtained from the CSP algorithm andyi isin +1minus1 denotes its correct class label N and ξi denotethe total number of training samples and the deviation fromthe optimal condition of linear separability respectively-epair of hyperplanes that provide the maximum separatingmargin can be found by minimizing the cost function(12)aTa + P1113936

Ni1ξi subject to the constraints

yi aTxi + b1113872 1113873ge 1minus ξi

ξi ge 0 for i 1 N(5)

where Pgt 0 represents a regularization penalty parameterof the error term By transforming this optimizationproblem into its dual problem the solution may be de-termined as a 1113936

Ni1αiyixi and achieves equality for

nonzero values of αi only -e corresponding data samplesare referred to as support vectors which are crucial toidentify the decision boundary Instead of the basic linearSVM we used a radial basis function (RBF) kernel whichnonlinearly projects the feature vectors onto a higherdimensional space and thus is better suited for nonlinearrelationships between features and class labels [29] -edetailed parameters of the SVM including the RBF kernelparameter and regularization penalty were determined bytrial-and-error

3 Results

31 YesNo Decoding Figure 3 shows the time-frequencyrepresentation of the ldquoyesnordquo decoding accuracy averagedover all subjects for each time-frequency subwindow -etime-frequency map of decoding accuracy was generated byrepresenting the decoding accuracies averaged over allsubjects within each time-frequency subwindow whichenables estimation of the decoding accuracies over all time-frequency ranges We used two criteria to define the mostimportant time-frequency subwindows showing highdecoding accuracies -e first was to use the threshold levelof a decoding accuracy of 75 determined by the theoretical95 confidence limits of the chance level when 10 trials perclass are used for testing [31] Another criterion was that thedecoding accuracy should be above the mean + 2times standarddeviation value (7934 here) -e high decoding accuraciesabove these two threshold levels were obtained for threesubwindows in the alpha and theta bands (as denoted by thethree boxes in Figure 3) for both early and late periods -ehighest and second highest decoding accuracies were found

Computational Intelligence and Neuroscience 5

in the upper alpha band (10ndash12Hz) at late epoch (box ①8108plusmn 889 at the 1000ndash1200ms box② 7999plusmn 899 atthe 800ndash1000ms) Also the third highest decoding accuracywas found in the upper theta band (6ndash8Hz) at the earlyperiod (box③ 7976plusmn 1021 at the 200ndash400ms) When all12 features within these three best time-frequency sub-windows were used together the decoding accuracy wasdrastically enhanced compared to the best subwindow(10ndash12Hz 1000ndash1200ms) as shown in Figure 4(a) (single8108plusmn 889 combined 8603plusmn 869 t(22)minus595plt 0001 by the paired-sample t-test) e individualdecoding accuracies are presented in Table 1 e sensitivityand specicity values for each time-frequency subwindoware presented in Supplementary Figure 1

32 Spatial Patterns Figure 5 shows the dicenterence betweenthe most important common spatial patterns for ldquonordquo andldquoyesrdquo answers within the three time-frequency subwindows

(averaged over all subjects) Each topography was obtainedfrom the dicenterence between the last (ldquonordquo answer) and rstcolumns (ldquoyesrdquo answer) of the inverse matrix of the pro-jection matrix W for each subject (Supplementary Fig-ure 2) which was calculated in each time-frequencysubwindow and then averaged over all subjects e dif-ference between the most important common spatial pat-terns in the alpha band showed the strongest coecopycient atthe right centroparietal region at both 1000ndash1200ms and800ndash1000ms periods (the leftmost and middle panels inFigure 5 respectively) e dicenterence between the mostimportant common spatial pattern in the theta band at the200ndash400ms period was focused in the right frontal regions(the rightmost panel in Figure 5)

33 Event-Related Spectral Perturbation (ERSP) AnalysisFigure 6(a) shows the topographical distributions of t valuesaveraged within the theta band (4ndash8Hz) in 0ndash1200mse 3

0 600400200 1200800Time (ms)

70

75

80

85

Dec

odin

g ac

cura

cy (

)

10

20

30

40

50

Freq

uenc

y (H

z)

1000

Semantic processingautomatic decision

Retainingldquoyesnordquo in mind

32 1

Figure 3 Decoding accuracies of time-frequency subwindows Color code denotes the decoding accuracy averaged over 23 subjects withineach time-frequency subwindow e best three subwindows are denoted by dashed boxes (①10ndash12Hz 1000ndash1200ms ② 10ndash12Hz800ndash1000ms ③ 6ndash8Hz 200ndash400ms)

Feature construction

29-channelsingle-trial

EEGs

TFspatialfiltering 1

Feature construction

Classificationusing

combined features

Feature construction

TFspatialfiltering 3

Yes

No

TFspatialfiltering 2

4 features

4 features

4 features

(a)

Sing

le

Com

bine

d

lowastlowastlowast

50

60

70

80

90

100

Dec

odin

g ac

cura

cy (

)

(b)

FIGURE 4 Intention decoding using the features from multiple time-frequency subwindows (a) intention decoder using the features frommultiple time-frequency subwindows (b) statistical comparison of the decoding accuracies between the cases of using single and multipletime-frequency subwindows (lowastlowastlowastplt 0001 by the paired-sample t-test error bar standard deviation)

6 Computational Intelligence and Neuroscience

electrodes (FC2 FC6 and C4) showing high t values above apredetermined threshold (tgt 162 corresponding to thehighest 10) were selected over the right frontal region(denoted by black dots in the left panel in Figure 6(a))Significant ldquoyesnordquo difference was found within a singletime-frequency range around 200ndash800ms in the upper thetaand lower alpha bands (6ndash10Hz) which was stronger forldquonordquo compared to ldquoyesrdquo (denoted by a solid contour in theleft panel in Figure 6(b))

In the alpha band 3 electrodes in right parietal area(CP2 Pz and P4) with high t values were selected as de-scribed above (tgt 152 the highest 10) as denoted by blackdots in the right panel in Figure 6(a)-e ldquoyesnordquo differencein spectral power in this region was significant within asingle time-frequency range (300ndash1200ms 9ndash12Hz) wherethe alpha-band power was stronger for ldquonordquo compared toldquoyesrdquo (denoted by a solid contour in the right panel inFigure 6(b))

Table 1 Decoding accuracies for individual subjects

Subject TF subwindow ① TF subwindow ② TF subwindow ③ Combined features

1 8689plusmn 652(7895 100)

8895plusmn 435(8000 9500)

8737plusmn 947(7000 100)

9450plusmn 350(9000 100)

2 7100plusmn 663(5500 8000)

7450plusmn 1011(5500 9000)

7800plusmn 980(6500 9500)

8000plusmn 1049(6000 9500)

3 7600plusmn 1020(6000 9000)

7900plusmn 663(6500 9000)

8200plusmn 714(6500 9000)

8550plusmn 789(6500 9500)

4 8050plusmn 879(6500 9500)

7550plusmn 723(6500 9000)

7100plusmn 800(6000 8500)

8400plusmn 374(8000 9000)

5 8800plusmn 332(8500 9500)

8900plusmn 1044(6500 100)

9200plusmn 748(7500 100)

9600plusmn 490(8500 100)

6 9000plusmn 707(7500 100)

9050plusmn 415(8500 9500)

9200plusmn 557(8500 100)

9300plusmn 400(8500 100)

7 7847plusmn 1321(4500 9474)

7745plusmn 1000(6500 100)

7742plusmn 975(6000 9000)

8445plusmn 817(6500 9500)

8 8300plusmn 557(7500 9500)

7550plusmn 757(6500 8500)

8050plusmn 757(6500 9000)

8800plusmn 872(7500 100)

9 7350plusmn 976(5500 8500)

7000plusmn 837(6000 8500)

6800plusmn 640(5500 8000)

7700plusmn 812(6500 9000)

10 7864plusmn 1095(6000 9474)

6854plusmn 860(6000 8947)

7609plusmn 1277(5000 9474)

8273plusmn 625(7000 9000)

11 7654plusmn 758(6316 8500)

7634plusmn 954(6000 8947)

6759plusmn 600(6000 7500)

7954plusmn 625(6667 9000)

12 9295plusmn 337(8947 100)

9400plusmn 735(7500 100)

9597plusmn 375(9000 100)

9797plusmn 248(9474 100)

13 8789plusmn 654(7895 100)

8284plusmn 976(6500 9500)

7442plusmn 1227(5000 9000)

8545plusmn 817(7000 100)

14 6000plusmn 775(4500 7500)

6500plusmn 1025(4500 7500)

6950plusmn 1404(5000 9000)

7000plusmn 1049(5000 8500)

15 6958plusmn 1498(4444 8889)

7953plusmn 1070(5556 9444)

5896plusmn 1375(3333 7778)

7455plusmn 1076(5789 9444)

16 6895plusmn 849(6000 9000)

6451plusmn 680(5000 7500)

6745plusmn 821(5500 8000)

6948plusmn 700(6000 8500)

17 7900plusmn 943(6500 9000)

7700plusmn 714(6500 8500)

8300plusmn 900(7000 9500)

8950plusmn 789(8000 100)

18 8150plusmn 867(6500 100)

8000plusmn 922(6000 9000)

8800plusmn 714(7000 9500)

9350plusmn 550(8000 100)

19 9300plusmn 510(8500 100)

9700plusmn 400(9000 100)

9450plusmn 269(9000 100)

9800plusmn 245(9500 100)

20 9050plusmn 472(8500 100)

8950plusmn 610(8000 100)

8850plusmn 673(7500 100)

9650plusmn 450(8500 100)

21 8700plusmn 748(7000 9500)

8300plusmn 927(6500 100)

8350plusmn 550(7500 9500)

9150plusmn 594(8000 100)

22 7800plusmn 872(6000 9000)

7350plusmn 808(5500 8500)

7000plusmn 866(5500 8500)

7700plusmn 400(7000 8500)

23 9382plusmn 551(8750 100)

8862plusmn 834(6667 100)

8877plusmn 686(7500 100)

9061plusmn 901(7333 100)

Ave 8108plusmn 889(6000 9382)

7999plusmn 899(6451 9700)

7976plusmn 1021(6745 9597)

8603plusmn 869(6948 9800)

Value meanplusmn standard deviation (SD) of decoding accuracy ()-e range of decoding accuracies was in parenthesis Abbreviation TF time frequency Aveaverage values over all subjects

Computational Intelligence and Neuroscience 7

4 Discussion

We showed that it is possible to decode the intentions toanswer ldquoyesrdquo and ldquonordquo with high accuracy from single-trialEEGs e best decoding accuracy averaged over 23 subjectswas as high as 8603 when useful features in multipletime-frequency subwindows were all combined edecoding accuracy was above 70 for most of the subjects

(22 out of 23 subjects) which is considered as a reasonableaccuracy for the binary classication [32] We decoded theldquoyesrdquo and ldquonordquo answers directly from the brain activitiesrepresenting the two dicenterent answers which implies thatthe ldquomind readingrdquo in a true sense is feasible e exper-imental paradigm of our study is based on a natural taskwhich required the subjects to answer self-referentialquestions as in conversation with others without any self-

6ndash8 Hz200ndash400 ms

10ndash12 Hz800ndash1000 ms

10ndash12 Hz 1000ndash1200 ms

1

ndash1

0

Figure 5 Dicenterence between the most important common spatial patterns for ldquonordquo and ldquoyesrdquo answers averaged over all subjects within 3time-frequency subwindows e topography was obtained from the dicenterence between the last (ldquonordquo answer) and rst (ldquoyesrdquo answer)columns of the inverse of the matrix W for each subject and then averaged over all subjects

4ndash8 Hz 0ndash1200 ms

eta power

4

ndash4

8ndash13 Hz 0ndash1200 ms

Alpha power

t val

ue

(a)

0 600400200 1200800Time (ms)

10

15

20

25

30

Freq

uenc

y (H

z)

1000

Right frontal

5

Nondashyes

p lt 005

0 600400200 1200800Time (ms)

ndash5

5

t val

ue10

15

20

25

30

Freq

uenc

y (H

z)

1000

Nondashyes

Right parietal p lt 005

5

(b)

Figure 6 Statistical comparisons in spectral power between ldquoyesrdquo and ldquonordquo (a) Topographical distributions of t values (left theta band(4ndash8Hz) 0ndash1200ms right alpha band (8ndash13Hz) 0ndash1200ms) Black dots (FC2 FC6 and C4 in the left panel and CP2 Pz and P4 in the rightpanel) denote high t values above a predetermined threshold corresponding to the highest 10 (b) e clusters in the time-frequencydomain showing signicant dicenterences between ldquoyesrdquo and ldquonordquo in right frontal (left panel) and right parietal regions (right panel) (denotedby black contours)

8 Computational Intelligence and Neuroscience

regulation of the brain signals or high cognitive efforts Nounpleasant stimuli and volition or high cognitive efforts arerequired since our approach is based on a direct decodingof ldquoyesrdquo and ldquonordquo without any self-regulation of the brainsignals Birbaumerrsquos group has suggested a new alternativeapproach based on classical conditioning to solve theproblem of conventional BCI in the CLIS patients [16ndash18]For the training two distinct unconditioned stimuli arepresented to the subjects immediately after the simple ldquoyesnordquo questions (corresponding to the conditioned stimuli)so that the cortical responses can be conditioned differentlyfor yes and no -e unconditioned stimuli include auditorypink noise and white noise [16 18] and weak electricalstimulation to the thumb [17] -e main idea of this ap-proach is to modulate the usersrsquo brain activities indirectlythrough the unconditioned stimuli so that ldquoyesrdquo and ldquonordquocan be easily discriminated from neural signals respondingto the sensory stimuli rather than to read the usersrsquo an-swers from neural signals -is approach may provide analternative to the conventional BCI approaches in thatvolition or high cognitive efforts are not required How-ever it remains unclear how long the conditioned corticalresponse can be maintained considering the extinctioneffect of classical conditioning [33] Moreover un-conditioned stimuli such as auditory noise or electricalstimulation can evoke significant displeasure

Recently a more natural approach for the ldquoyesnordquodecoding was demonstrated based on functional near-in-frared spectroscopy (fNIRS) in the CLIS patients [34] -eyachieved ldquoyesnordquo decoding accuracy over 70 based onfNIRS signals which were recorded while the patientsanswered ldquoyesrdquo or ldquonordquo to personal and open questions inminds repeatedly Interestingly for the same experimentalprotocol they reported that EEG-based decoding yieldedaccuracy below the chance level -is study employed anatural questionanswer task which does not require highcognitive efforts or volition just as ours But due to the slownature of hemodynamics the duration of each trial for thedecoding was quite long (gt10 sec) Here we showed thepossibility of ldquoyesnordquo decoding from considerably shortersignal recording which is more beneficial for a practicalBCI communication tool

We took a systematic approach of finding features ofbrain activities reflecting ldquoyesnordquo answers in minds andthen developing the decoding algorithm by utilizing thesefeatures Further studies may be necessary to investigatewhether the patients who would potentially benefit fromthe BCI can hold the intentions to answer in minds for ashort time and to validate our method on the patientsrsquo data

In this study the intentions regarding self-referentialquestions based on the autobiographic facts were in-vestigated It is important to further try decoding the in-tentions to answer various types of questions includingdesire feeling and preference In addition our questionswere presented only in visual stimuli Neurological patientsmay have an abnormal visual function such as disability tofix their gaze on specific visual stimuli [35] Different sensorymodality such as auditory stimuli has been tried for the BCIcommunication tools [34 36] It would be beneficial if our

approach can be validated with auditory stimuli such asvoice considering that a high decoding accuracy above 80was obtained even when the brain activities during theperiod of retaining the decision in minds (10ndash12Hz1000ndash1200ms) used for the decoding-us we expect that itis possible to decode the ldquoyesrdquo and ldquonordquo intentions in asimilar way even if other types of questions andor theauditory stimuli are employed in the further studies Inaddition here we did not try to optimize the detailed pa-rameters of the SVM including the RBF kernel parameterand regularization penalty -e use of the best parameters ofthe SVM for example by using the ldquogrid-searchrdquo method[37] may be obviously helpful for better results

We found two time-frequency regions containing usefulinformation for the ldquoyesnordquo decoding in early theta and latealpha bands-e useful features for the ldquoyesnordquo decoding inthe alpha band were found to be concentrated in the parietalregion at 800ndash1200ms from the CSP algorithm Recently weshowed that the alpha rhythms in the right parietal regionare differentiated between the intentions to answers eitherldquoyesrdquo or ldquonordquo in minds presumably due to the difference incognitive loads for the WM retention [2] Several previousstudies showed that the higher parietal alpha power reflectsincreased memory load [38 39] or attentional demand[40 41] during WM retention -e higher alpha power isattributed to active inhibitory control to block incomingstimuli during WM retention for efficient cortical in-formation processing [38 39 42 43] Our results showedhigher parietal alpha power for ldquonordquo compared to ldquoyesrdquowhich may imply higher cognitive load during retainingldquonordquo in minds compared to ldquoyesrdquo [2] -e greater increase inalpha-band activity for ldquonordquo may reflect the increased WMload during the intention retention In Korean languageldquoyesrdquo is the one-character word ldquo네rdquo and ldquonordquo is three-character word ldquo아니오rdquo It is plausible that the higherWMload is required to represent intention to respond ldquonordquo thanldquoyesrdquo due to the length of the Korean words resulting in thehigher alpha rhythm -is assumption is supported by anERP study which reported that greater alpha-band powerwas induced for retaining longer word [44]

It can also be interpreted that the significantly higheralpha-band activity in the centroparietal region for ldquonordquo isdue to the higher attentional demand [40 45] and thiscontributed to the high decoding accuracy -is is also inagreement with a recent study [46] which reported that ahigher alpha rhythm was identified in the right parietalcortex for a higher internal attention condition during adivergent thinking task Our result of greater alpha powerfor ldquonordquo than for ldquoyesrdquo may imply a stronger inhibition of theouter stimuli by the bottom-up attention network for ldquonordquoinduced by higher internal attentional demand -is issupported by psychophysical which showed that saying ldquonordquorequires more effortful reconsideration after comprehendinga sentence and a longer response time for saying ldquonordquo thanldquoyesrdquo [47 48]

-e theta-band activity in the frontal region in 200ndash500ms was another major feature for ldquoyesnordquo decoding-e theta ERS showed topography focused on midlinefrontal and lateral temporal regions -e difference

Computational Intelligence and Neuroscience 9

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

presented once again Consequently each block included 50type (a) and 50 type (b) questions in total e averageduration of each single trial (ie one question and answer)was 4380plusmn 27495ms e total time for performing thetasks was approximately 20minutes including at least5minutes of rest between blocks

23 Electroencephalogram (EEG) Recording and DataAnalysis Sixty channel EEGs were recorded based on the10ndash10 system using an EEG amplier (Brain ProductsGmbH Munich Germany) with an AgAgCl electrode cap(EASYCAP FMS Munich Germany) e ground andreference electrodes were at AFz and linked mastoids re-spectively e impedances of all electrodes were kept under10 kΩ e sampling rate was 500 sampless A bandpasslter (003ndash100Hz) and a notch lter (60Hz) were applied inorder to reduce background noise and powerlineinterferences

An open source toolbox EEGLAB was used for the wholeprocedure of preprocessing [19] First single-trial EEGswere segmented during the minus500sim1300ms period relative tothe critical word onset By visual inspection we removed thesingle-trial waveforms contaminated severely from non-stereotyped artifacts such as drifts and discontinuity enan independent component analysis (ICA) was employed tothe remaining single-trial EEGs in order to correct the ocularand muscular artifacts [20] e group-averaged percentageof the number of epochs remaining per subject was 9888plusmn308 and 9796plusmn 586 for ldquoyesrdquo and ldquonordquo questionsrespectively

24 YesNo Decoding Figure 2(a) illustrates the structure ofldquoyesnordquo intention decoding algorithm using local time-frequency information First we selected 29 channels out of60 (ie Fp1 Fpz Fp2 F7 F3 Fz F4 F8 FC5 FC1 FC2 FC6T7 C3 Cz C4 T8 CP5 CP1 CP2 CP6 P7 P3 Pz P4 P8O1 Oz and O2) following the standard 10ndash20 system iswas based on a recent simulation study which showed thatthe decoding accuracy with the common spatial pattern(CSP) spatial ltering was optimized when sim30 channelswere used and decreased for more channels [21] e overall

time-frequency range (0ndash1200ms 4ndash50Hz) was divided intosubwindows of 200mstimes 2Hz e intention decodingwithin each of the local time-frequency subwindows wasperformed as follows Single-trial EEGs were bandpass l-tered in the frequency range of the subwindow using alinear-phase nite impulse response lter (the number of thelter order 512 bandwidth 2Hz) e multichannelbandpass-ltered signals within the temporal period of thesubwindow were subsequently projected to the lower di-mension (four dimensions) by the CSP algorithm [22] efour time series obtained from the CSP spatial lter wereused to construct a four-dimensional feature vector whichwas passed to a support vectormachine (SVM) classierenal output of the classier was either ldquoyesrdquo or ldquonordquo adecision of answer for each single trial

e performance of the trained classier was validatedby 10-fold cross-validation as follows First for each classwe randomly split all the trials into 10-folds with the samenumber of trials (ie sim10 trials per fold for each class)en we randomly selected one fold (kth where k 1 2 10) as a test data (10) and trained the classier usingthe rest of data (ie 9 folds excepting the kth fold 90) Inorder to keep a balance between the numbers of ldquoyesrdquo andldquonordquo trials the trainingtesting data were selected withineach class (ie ldquoyesrdquo or ldquonordquo) as shown in Figure 2(b) eground truth for each single trial was determined whetherthe question in the single trial was including AFV or note decoding accuracy for each subject was estimated byaveraging the ratio of correct classication from 10 rep-etitions (ie k 1 2 10) of this procedure

Additionally we also made ecentective use all the featuresobtained from multiple time-frequency subwindows inorder to investigate whether more accurate decoding ispossible by combining useful features each of which waslocalized in the time-frequency domain e time-fre-quency subwindows were selected if the decoding accu-racies for a specic subwindow were higher than apredetermined threshold (2 times standard deviation above themean among all time-frequency subwindows) And thenthe classier was trained and tested as described abovewith input feature vectors obtained by combining all theselected subwindows

PleaserespondStudentJob+

Fixation

1000 ms 300 ms 300 ms 700 ms300 ms 1000 ms

Blank Critical word

Cue

0 1000 500 1500 Time relative to the critical word (CW) onset (ms)

Semantic processingautomatic decision

Retainingldquoyesnordquo in mind

Covertresponse

Stimuli

300 ms300 ms

CW Blank Cue

(b)

(a)1st word

Expected cognitive processing

Figure 1 Experimental task (a) presentation of stimuli (b) expected temporal sequence of cognitive processing

Computational Intelligence and Neuroscience 3

25 Event-Related Spectral Perturbation (ERSP) Analysise time-frequency activation patterns ie ERSPs wereinvestigated to reveal statistical dicenterences between ldquoyesrdquoand ldquonordquo to nd the time and frequency ranges of interestsfor ecentective classication A` continuous wavelet transform(CWT) based on a complex Morlet wavelet was used for theERSP analysis [23] e number of cycles for the CWTlinearly increased according to the frequency from 4 to 135at the lowest (1Hz) and the highest frequencies (100Hz)respectively [19] is method provides better frequencyresolution at high frequencies and it is better matched to thelinear scale that we adopted to visualize the time-frequencymap [19] e induced spectral power was calculated byaveraging the ERSP patterns of each single trial [24] etime-frequency distribution of ERSP patterns was repre-sented as the ratio of the relative change to the power in abaseline interval from minus300 to 0ms prior to stimulus onsetto reduce intersubject variability and to normalize powerchanges across dicenterent frequency bands

We employed the mass-univariate approach with thecluster-based permutation test for correcting multiple com-parisons [25] in order to nd the time frequency and elec-trode showing signicant dicenterences between ldquoyesrdquo and ldquonordquowithout a priori knowledge Detailed procedure is as follows

(1) A large number of paired-sample t-tests were appliedto the data for all time-frequency-electrode binswithin the range of 0ndash1200ms (time) 5ndash30Hz(frequency) and 29 electrodes e number of binswas 181714 241times 26times 29 since there were 241 timesamples 26 frequency points and 29 electrodes e

electrodes showing high t values were selected andthe average power spectral power was calculated overthe selected electrodes as follows First from spatialdistribution of the t values averaged within thefrequency band of interest (eg theta band 4ndash8Hzalpha band 8ndash13Hz) during the overall time period(0ndash1200ms) the electrodes showing higher p valuesabove a predetermined threshold (the upper 10highest value) were selected e average powerspectral power was calculated over the selectedelectrodes for the next step

(2) After signicant locations were found in step 1 time-frequency bins were screened to be signicantamong all 6266 (241times 26) bins if p values werebelow a predetermined threshold (plt 005) Acluster of time-frequency bins was formed if morethan two successive bins were selected along eithertime or frequency axis Sum of t values within thecluster tmass was then calculated and compared withthe null distribution of surrogate data to determinestatistical signicance of the cluster (above the highest5 of the null distribution) e null distribution oftmass was obtained from the largest values of tmass foreach of 5000 surrogate data which were derived byrandom permutation of ldquoyesrdquo and ldquonordquo answers

26 Feature Extraction by Common Spatial Pattern (CSP)Filtering CSP is a mathematical procedure to derive aspatial lter which separates a multichannel signal into

Selecting time segment

Spatial filtering(CSP)

Training Filtered 4 time series

Spatial filtering

Spatial filter matrix (W)Test

29-channelsingle-trial EEGs

(90)

YesNo

Bandpassfiltering

Training classifier(SVM)

Selecting time segment

Bandpassfiltering

Classification(SVM)

29-channelsingle-trial EEGs

(10)Filtered

4 time series

Feature construction

Feature construction

4 features

4 features

(a)

Num

ber o

f tria

ls

0

20

40

60

80

100

5 10 15 20Subject

Yes (train)No (train)

Yes (test)No (test)

(b)

Figure 2 (a) Block diagram of the intention decoding algorithm (b) the number of trials which was selected as training and testing datawithin each answer type for each subject

4 Computational Intelligence and Neuroscience

additive subcomponents so that the differences of variancesare maximized between two classes -at is the most dis-criminative features between two classes are obtained bymaximizing the variance of the spatially filtered signal of oneclass while minimizing that of the other class [22] -e CSPalgorithm is recognized to be effective for the discriminationof mental states from event-related EEG spectral powers[26] -e results of the CSP can be visualized as a topo-graphic map on the scalp which facilitates interpretation offunctional neuroanatomical meanings [26]

-e CSP spatial filter W can be obtained by simulta-neous diagonalization of two covariance matrices of classes 1and 2 as follows

WTΣ1W Λ1

WTΣ2W Λ2(1)

where Λ1 + Λ2 I Σ1 and Σ2 represent the spatial co-variance matrices averaged over all single-trial EEGs foreach class and Λ1 and Λ2 denote the diagonal matrices -eprojection vector w (column vectors of W) can be ob-tained from a generalized eigenvalue decomposition asfollows

Σ1wk λkΣ2wk (2)

wherewk (k 1 C where C is the number of channels) isthe generalized eigenvector and λ1k wT

kΣ1wk and λ2k

wTkΣ2wk are defined as the k

th diagonal element ofΛ1 andΛ2respectively where λk λ1kλ2k Importantly λ1k and λ2k

(ranges from 0 to 1) reflect the variance for each class andλ1k + λ2k 1-us if λ1k is close to 1 λ2k should be close to0 -is means that corresponding projection vector wkshows high variance in class 1 but low variance in class 2-edifference in variances between these two classes enablesdiscriminating one class from another -e eigenvalues aresorted in the descending order during calculation meaningthat the first projection vector yields the highest variance forclass 1 (but the lowest for class 2) whereas the last projectionvector yields the highest variance for class 2 (but lowest forclass 1) -us the first and last projection vectors are themost useful for the discrimination [22]

-e spatial filter W provides the decomposition of asingle-trial multichannel EEG E as ZWTE where E isrepresented as a matrix with C (the number of channels)rows and T (the number of time samples) columns -ecolumns of Wminus1 form the common spatial patterns and canbe visualized as topographies on scalp -e variances of thespatially filtered time-series Z are calculated as features forthe classification as follows

fp logvar Zp1113872 1113873

11139362mi1var Zi( 1113857

⎛⎝ ⎞⎠ where p 1 2 2m (3)

where p is the number of features m was set to 2 whichmeans that the first 2 and last 2 projection vectors were usedas features and thus the number of features p was 4 for allclassifications -e log transformation was adopted to ap-proximate the normal distribution of the data

27 Pattern Classification Using Support Vector Machine(SVM) SVM has been recognized to be a practical androbust method for the classification of human brain signals[27 28] -e SVM is trained to determine an optimal hy-perplane by which the distance to the support vectors(closest to the separating boundary) is maximized [29 30]In the case of the linear SVM classification the hyperplaneaTx+ b satisfies

yi aTxi + b1113872 1113873ge 1minus ξi for i 1 N (4)

where xi fpi denotes a feature vector (in which p 1 4) which can be obtained from the CSP algorithm andyi isin +1minus1 denotes its correct class label N and ξi denotethe total number of training samples and the deviation fromthe optimal condition of linear separability respectively-epair of hyperplanes that provide the maximum separatingmargin can be found by minimizing the cost function(12)aTa + P1113936

Ni1ξi subject to the constraints

yi aTxi + b1113872 1113873ge 1minus ξi

ξi ge 0 for i 1 N(5)

where Pgt 0 represents a regularization penalty parameterof the error term By transforming this optimizationproblem into its dual problem the solution may be de-termined as a 1113936

Ni1αiyixi and achieves equality for

nonzero values of αi only -e corresponding data samplesare referred to as support vectors which are crucial toidentify the decision boundary Instead of the basic linearSVM we used a radial basis function (RBF) kernel whichnonlinearly projects the feature vectors onto a higherdimensional space and thus is better suited for nonlinearrelationships between features and class labels [29] -edetailed parameters of the SVM including the RBF kernelparameter and regularization penalty were determined bytrial-and-error

3 Results

31 YesNo Decoding Figure 3 shows the time-frequencyrepresentation of the ldquoyesnordquo decoding accuracy averagedover all subjects for each time-frequency subwindow -etime-frequency map of decoding accuracy was generated byrepresenting the decoding accuracies averaged over allsubjects within each time-frequency subwindow whichenables estimation of the decoding accuracies over all time-frequency ranges We used two criteria to define the mostimportant time-frequency subwindows showing highdecoding accuracies -e first was to use the threshold levelof a decoding accuracy of 75 determined by the theoretical95 confidence limits of the chance level when 10 trials perclass are used for testing [31] Another criterion was that thedecoding accuracy should be above the mean + 2times standarddeviation value (7934 here) -e high decoding accuraciesabove these two threshold levels were obtained for threesubwindows in the alpha and theta bands (as denoted by thethree boxes in Figure 3) for both early and late periods -ehighest and second highest decoding accuracies were found

Computational Intelligence and Neuroscience 5

in the upper alpha band (10ndash12Hz) at late epoch (box ①8108plusmn 889 at the 1000ndash1200ms box② 7999plusmn 899 atthe 800ndash1000ms) Also the third highest decoding accuracywas found in the upper theta band (6ndash8Hz) at the earlyperiod (box③ 7976plusmn 1021 at the 200ndash400ms) When all12 features within these three best time-frequency sub-windows were used together the decoding accuracy wasdrastically enhanced compared to the best subwindow(10ndash12Hz 1000ndash1200ms) as shown in Figure 4(a) (single8108plusmn 889 combined 8603plusmn 869 t(22)minus595plt 0001 by the paired-sample t-test) e individualdecoding accuracies are presented in Table 1 e sensitivityand specicity values for each time-frequency subwindoware presented in Supplementary Figure 1

32 Spatial Patterns Figure 5 shows the dicenterence betweenthe most important common spatial patterns for ldquonordquo andldquoyesrdquo answers within the three time-frequency subwindows

(averaged over all subjects) Each topography was obtainedfrom the dicenterence between the last (ldquonordquo answer) and rstcolumns (ldquoyesrdquo answer) of the inverse matrix of the pro-jection matrix W for each subject (Supplementary Fig-ure 2) which was calculated in each time-frequencysubwindow and then averaged over all subjects e dif-ference between the most important common spatial pat-terns in the alpha band showed the strongest coecopycient atthe right centroparietal region at both 1000ndash1200ms and800ndash1000ms periods (the leftmost and middle panels inFigure 5 respectively) e dicenterence between the mostimportant common spatial pattern in the theta band at the200ndash400ms period was focused in the right frontal regions(the rightmost panel in Figure 5)

33 Event-Related Spectral Perturbation (ERSP) AnalysisFigure 6(a) shows the topographical distributions of t valuesaveraged within the theta band (4ndash8Hz) in 0ndash1200mse 3

0 600400200 1200800Time (ms)

70

75

80

85

Dec

odin

g ac

cura

cy (

)

10

20

30

40

50

Freq

uenc

y (H

z)

1000

Semantic processingautomatic decision

Retainingldquoyesnordquo in mind

32 1

Figure 3 Decoding accuracies of time-frequency subwindows Color code denotes the decoding accuracy averaged over 23 subjects withineach time-frequency subwindow e best three subwindows are denoted by dashed boxes (①10ndash12Hz 1000ndash1200ms ② 10ndash12Hz800ndash1000ms ③ 6ndash8Hz 200ndash400ms)

Feature construction

29-channelsingle-trial

EEGs

TFspatialfiltering 1

Feature construction

Classificationusing

combined features

Feature construction

TFspatialfiltering 3

Yes

No

TFspatialfiltering 2

4 features

4 features

4 features

(a)

Sing

le

Com

bine

d

lowastlowastlowast

50

60

70

80

90

100

Dec

odin

g ac

cura

cy (

)

(b)

FIGURE 4 Intention decoding using the features from multiple time-frequency subwindows (a) intention decoder using the features frommultiple time-frequency subwindows (b) statistical comparison of the decoding accuracies between the cases of using single and multipletime-frequency subwindows (lowastlowastlowastplt 0001 by the paired-sample t-test error bar standard deviation)

6 Computational Intelligence and Neuroscience

electrodes (FC2 FC6 and C4) showing high t values above apredetermined threshold (tgt 162 corresponding to thehighest 10) were selected over the right frontal region(denoted by black dots in the left panel in Figure 6(a))Significant ldquoyesnordquo difference was found within a singletime-frequency range around 200ndash800ms in the upper thetaand lower alpha bands (6ndash10Hz) which was stronger forldquonordquo compared to ldquoyesrdquo (denoted by a solid contour in theleft panel in Figure 6(b))

In the alpha band 3 electrodes in right parietal area(CP2 Pz and P4) with high t values were selected as de-scribed above (tgt 152 the highest 10) as denoted by blackdots in the right panel in Figure 6(a)-e ldquoyesnordquo differencein spectral power in this region was significant within asingle time-frequency range (300ndash1200ms 9ndash12Hz) wherethe alpha-band power was stronger for ldquonordquo compared toldquoyesrdquo (denoted by a solid contour in the right panel inFigure 6(b))

Table 1 Decoding accuracies for individual subjects

Subject TF subwindow ① TF subwindow ② TF subwindow ③ Combined features

1 8689plusmn 652(7895 100)

8895plusmn 435(8000 9500)

8737plusmn 947(7000 100)

9450plusmn 350(9000 100)

2 7100plusmn 663(5500 8000)

7450plusmn 1011(5500 9000)

7800plusmn 980(6500 9500)

8000plusmn 1049(6000 9500)

3 7600plusmn 1020(6000 9000)

7900plusmn 663(6500 9000)

8200plusmn 714(6500 9000)

8550plusmn 789(6500 9500)

4 8050plusmn 879(6500 9500)

7550plusmn 723(6500 9000)

7100plusmn 800(6000 8500)

8400plusmn 374(8000 9000)

5 8800plusmn 332(8500 9500)

8900plusmn 1044(6500 100)

9200plusmn 748(7500 100)

9600plusmn 490(8500 100)

6 9000plusmn 707(7500 100)

9050plusmn 415(8500 9500)

9200plusmn 557(8500 100)

9300plusmn 400(8500 100)

7 7847plusmn 1321(4500 9474)

7745plusmn 1000(6500 100)

7742plusmn 975(6000 9000)

8445plusmn 817(6500 9500)

8 8300plusmn 557(7500 9500)

7550plusmn 757(6500 8500)

8050plusmn 757(6500 9000)

8800plusmn 872(7500 100)

9 7350plusmn 976(5500 8500)

7000plusmn 837(6000 8500)

6800plusmn 640(5500 8000)

7700plusmn 812(6500 9000)

10 7864plusmn 1095(6000 9474)

6854plusmn 860(6000 8947)

7609plusmn 1277(5000 9474)

8273plusmn 625(7000 9000)

11 7654plusmn 758(6316 8500)

7634plusmn 954(6000 8947)

6759plusmn 600(6000 7500)

7954plusmn 625(6667 9000)

12 9295plusmn 337(8947 100)

9400plusmn 735(7500 100)

9597plusmn 375(9000 100)

9797plusmn 248(9474 100)

13 8789plusmn 654(7895 100)

8284plusmn 976(6500 9500)

7442plusmn 1227(5000 9000)

8545plusmn 817(7000 100)

14 6000plusmn 775(4500 7500)

6500plusmn 1025(4500 7500)

6950plusmn 1404(5000 9000)

7000plusmn 1049(5000 8500)

15 6958plusmn 1498(4444 8889)

7953plusmn 1070(5556 9444)

5896plusmn 1375(3333 7778)

7455plusmn 1076(5789 9444)

16 6895plusmn 849(6000 9000)

6451plusmn 680(5000 7500)

6745plusmn 821(5500 8000)

6948plusmn 700(6000 8500)

17 7900plusmn 943(6500 9000)

7700plusmn 714(6500 8500)

8300plusmn 900(7000 9500)

8950plusmn 789(8000 100)

18 8150plusmn 867(6500 100)

8000plusmn 922(6000 9000)

8800plusmn 714(7000 9500)

9350plusmn 550(8000 100)

19 9300plusmn 510(8500 100)

9700plusmn 400(9000 100)

9450plusmn 269(9000 100)

9800plusmn 245(9500 100)

20 9050plusmn 472(8500 100)

8950plusmn 610(8000 100)

8850plusmn 673(7500 100)

9650plusmn 450(8500 100)

21 8700plusmn 748(7000 9500)

8300plusmn 927(6500 100)

8350plusmn 550(7500 9500)

9150plusmn 594(8000 100)

22 7800plusmn 872(6000 9000)

7350plusmn 808(5500 8500)

7000plusmn 866(5500 8500)

7700plusmn 400(7000 8500)

23 9382plusmn 551(8750 100)

8862plusmn 834(6667 100)

8877plusmn 686(7500 100)

9061plusmn 901(7333 100)

Ave 8108plusmn 889(6000 9382)

7999plusmn 899(6451 9700)

7976plusmn 1021(6745 9597)

8603plusmn 869(6948 9800)

Value meanplusmn standard deviation (SD) of decoding accuracy ()-e range of decoding accuracies was in parenthesis Abbreviation TF time frequency Aveaverage values over all subjects

Computational Intelligence and Neuroscience 7

4 Discussion

We showed that it is possible to decode the intentions toanswer ldquoyesrdquo and ldquonordquo with high accuracy from single-trialEEGs e best decoding accuracy averaged over 23 subjectswas as high as 8603 when useful features in multipletime-frequency subwindows were all combined edecoding accuracy was above 70 for most of the subjects

(22 out of 23 subjects) which is considered as a reasonableaccuracy for the binary classication [32] We decoded theldquoyesrdquo and ldquonordquo answers directly from the brain activitiesrepresenting the two dicenterent answers which implies thatthe ldquomind readingrdquo in a true sense is feasible e exper-imental paradigm of our study is based on a natural taskwhich required the subjects to answer self-referentialquestions as in conversation with others without any self-

6ndash8 Hz200ndash400 ms

10ndash12 Hz800ndash1000 ms

10ndash12 Hz 1000ndash1200 ms

1

ndash1

0

Figure 5 Dicenterence between the most important common spatial patterns for ldquonordquo and ldquoyesrdquo answers averaged over all subjects within 3time-frequency subwindows e topography was obtained from the dicenterence between the last (ldquonordquo answer) and rst (ldquoyesrdquo answer)columns of the inverse of the matrix W for each subject and then averaged over all subjects

4ndash8 Hz 0ndash1200 ms

eta power

4

ndash4

8ndash13 Hz 0ndash1200 ms

Alpha power

t val

ue

(a)

0 600400200 1200800Time (ms)

10

15

20

25

30

Freq

uenc

y (H

z)

1000

Right frontal

5

Nondashyes

p lt 005

0 600400200 1200800Time (ms)

ndash5

5

t val

ue10

15

20

25

30

Freq

uenc

y (H

z)

1000

Nondashyes

Right parietal p lt 005

5

(b)

Figure 6 Statistical comparisons in spectral power between ldquoyesrdquo and ldquonordquo (a) Topographical distributions of t values (left theta band(4ndash8Hz) 0ndash1200ms right alpha band (8ndash13Hz) 0ndash1200ms) Black dots (FC2 FC6 and C4 in the left panel and CP2 Pz and P4 in the rightpanel) denote high t values above a predetermined threshold corresponding to the highest 10 (b) e clusters in the time-frequencydomain showing signicant dicenterences between ldquoyesrdquo and ldquonordquo in right frontal (left panel) and right parietal regions (right panel) (denotedby black contours)

8 Computational Intelligence and Neuroscience

regulation of the brain signals or high cognitive efforts Nounpleasant stimuli and volition or high cognitive efforts arerequired since our approach is based on a direct decodingof ldquoyesrdquo and ldquonordquo without any self-regulation of the brainsignals Birbaumerrsquos group has suggested a new alternativeapproach based on classical conditioning to solve theproblem of conventional BCI in the CLIS patients [16ndash18]For the training two distinct unconditioned stimuli arepresented to the subjects immediately after the simple ldquoyesnordquo questions (corresponding to the conditioned stimuli)so that the cortical responses can be conditioned differentlyfor yes and no -e unconditioned stimuli include auditorypink noise and white noise [16 18] and weak electricalstimulation to the thumb [17] -e main idea of this ap-proach is to modulate the usersrsquo brain activities indirectlythrough the unconditioned stimuli so that ldquoyesrdquo and ldquonordquocan be easily discriminated from neural signals respondingto the sensory stimuli rather than to read the usersrsquo an-swers from neural signals -is approach may provide analternative to the conventional BCI approaches in thatvolition or high cognitive efforts are not required How-ever it remains unclear how long the conditioned corticalresponse can be maintained considering the extinctioneffect of classical conditioning [33] Moreover un-conditioned stimuli such as auditory noise or electricalstimulation can evoke significant displeasure

Recently a more natural approach for the ldquoyesnordquodecoding was demonstrated based on functional near-in-frared spectroscopy (fNIRS) in the CLIS patients [34] -eyachieved ldquoyesnordquo decoding accuracy over 70 based onfNIRS signals which were recorded while the patientsanswered ldquoyesrdquo or ldquonordquo to personal and open questions inminds repeatedly Interestingly for the same experimentalprotocol they reported that EEG-based decoding yieldedaccuracy below the chance level -is study employed anatural questionanswer task which does not require highcognitive efforts or volition just as ours But due to the slownature of hemodynamics the duration of each trial for thedecoding was quite long (gt10 sec) Here we showed thepossibility of ldquoyesnordquo decoding from considerably shortersignal recording which is more beneficial for a practicalBCI communication tool

We took a systematic approach of finding features ofbrain activities reflecting ldquoyesnordquo answers in minds andthen developing the decoding algorithm by utilizing thesefeatures Further studies may be necessary to investigatewhether the patients who would potentially benefit fromthe BCI can hold the intentions to answer in minds for ashort time and to validate our method on the patientsrsquo data

In this study the intentions regarding self-referentialquestions based on the autobiographic facts were in-vestigated It is important to further try decoding the in-tentions to answer various types of questions includingdesire feeling and preference In addition our questionswere presented only in visual stimuli Neurological patientsmay have an abnormal visual function such as disability tofix their gaze on specific visual stimuli [35] Different sensorymodality such as auditory stimuli has been tried for the BCIcommunication tools [34 36] It would be beneficial if our

approach can be validated with auditory stimuli such asvoice considering that a high decoding accuracy above 80was obtained even when the brain activities during theperiod of retaining the decision in minds (10ndash12Hz1000ndash1200ms) used for the decoding-us we expect that itis possible to decode the ldquoyesrdquo and ldquonordquo intentions in asimilar way even if other types of questions andor theauditory stimuli are employed in the further studies Inaddition here we did not try to optimize the detailed pa-rameters of the SVM including the RBF kernel parameterand regularization penalty -e use of the best parameters ofthe SVM for example by using the ldquogrid-searchrdquo method[37] may be obviously helpful for better results

We found two time-frequency regions containing usefulinformation for the ldquoyesnordquo decoding in early theta and latealpha bands-e useful features for the ldquoyesnordquo decoding inthe alpha band were found to be concentrated in the parietalregion at 800ndash1200ms from the CSP algorithm Recently weshowed that the alpha rhythms in the right parietal regionare differentiated between the intentions to answers eitherldquoyesrdquo or ldquonordquo in minds presumably due to the difference incognitive loads for the WM retention [2] Several previousstudies showed that the higher parietal alpha power reflectsincreased memory load [38 39] or attentional demand[40 41] during WM retention -e higher alpha power isattributed to active inhibitory control to block incomingstimuli during WM retention for efficient cortical in-formation processing [38 39 42 43] Our results showedhigher parietal alpha power for ldquonordquo compared to ldquoyesrdquowhich may imply higher cognitive load during retainingldquonordquo in minds compared to ldquoyesrdquo [2] -e greater increase inalpha-band activity for ldquonordquo may reflect the increased WMload during the intention retention In Korean languageldquoyesrdquo is the one-character word ldquo네rdquo and ldquonordquo is three-character word ldquo아니오rdquo It is plausible that the higherWMload is required to represent intention to respond ldquonordquo thanldquoyesrdquo due to the length of the Korean words resulting in thehigher alpha rhythm -is assumption is supported by anERP study which reported that greater alpha-band powerwas induced for retaining longer word [44]

It can also be interpreted that the significantly higheralpha-band activity in the centroparietal region for ldquonordquo isdue to the higher attentional demand [40 45] and thiscontributed to the high decoding accuracy -is is also inagreement with a recent study [46] which reported that ahigher alpha rhythm was identified in the right parietalcortex for a higher internal attention condition during adivergent thinking task Our result of greater alpha powerfor ldquonordquo than for ldquoyesrdquo may imply a stronger inhibition of theouter stimuli by the bottom-up attention network for ldquonordquoinduced by higher internal attentional demand -is issupported by psychophysical which showed that saying ldquonordquorequires more effortful reconsideration after comprehendinga sentence and a longer response time for saying ldquonordquo thanldquoyesrdquo [47 48]

-e theta-band activity in the frontal region in 200ndash500ms was another major feature for ldquoyesnordquo decoding-e theta ERS showed topography focused on midlinefrontal and lateral temporal regions -e difference

Computational Intelligence and Neuroscience 9

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

25 Event-Related Spectral Perturbation (ERSP) Analysise time-frequency activation patterns ie ERSPs wereinvestigated to reveal statistical dicenterences between ldquoyesrdquoand ldquonordquo to nd the time and frequency ranges of interestsfor ecentective classication A` continuous wavelet transform(CWT) based on a complex Morlet wavelet was used for theERSP analysis [23] e number of cycles for the CWTlinearly increased according to the frequency from 4 to 135at the lowest (1Hz) and the highest frequencies (100Hz)respectively [19] is method provides better frequencyresolution at high frequencies and it is better matched to thelinear scale that we adopted to visualize the time-frequencymap [19] e induced spectral power was calculated byaveraging the ERSP patterns of each single trial [24] etime-frequency distribution of ERSP patterns was repre-sented as the ratio of the relative change to the power in abaseline interval from minus300 to 0ms prior to stimulus onsetto reduce intersubject variability and to normalize powerchanges across dicenterent frequency bands

We employed the mass-univariate approach with thecluster-based permutation test for correcting multiple com-parisons [25] in order to nd the time frequency and elec-trode showing signicant dicenterences between ldquoyesrdquo and ldquonordquowithout a priori knowledge Detailed procedure is as follows

(1) A large number of paired-sample t-tests were appliedto the data for all time-frequency-electrode binswithin the range of 0ndash1200ms (time) 5ndash30Hz(frequency) and 29 electrodes e number of binswas 181714 241times 26times 29 since there were 241 timesamples 26 frequency points and 29 electrodes e

electrodes showing high t values were selected andthe average power spectral power was calculated overthe selected electrodes as follows First from spatialdistribution of the t values averaged within thefrequency band of interest (eg theta band 4ndash8Hzalpha band 8ndash13Hz) during the overall time period(0ndash1200ms) the electrodes showing higher p valuesabove a predetermined threshold (the upper 10highest value) were selected e average powerspectral power was calculated over the selectedelectrodes for the next step

(2) After signicant locations were found in step 1 time-frequency bins were screened to be signicantamong all 6266 (241times 26) bins if p values werebelow a predetermined threshold (plt 005) Acluster of time-frequency bins was formed if morethan two successive bins were selected along eithertime or frequency axis Sum of t values within thecluster tmass was then calculated and compared withthe null distribution of surrogate data to determinestatistical signicance of the cluster (above the highest5 of the null distribution) e null distribution oftmass was obtained from the largest values of tmass foreach of 5000 surrogate data which were derived byrandom permutation of ldquoyesrdquo and ldquonordquo answers

26 Feature Extraction by Common Spatial Pattern (CSP)Filtering CSP is a mathematical procedure to derive aspatial lter which separates a multichannel signal into

Selecting time segment

Spatial filtering(CSP)

Training Filtered 4 time series

Spatial filtering

Spatial filter matrix (W)Test

29-channelsingle-trial EEGs

(90)

YesNo

Bandpassfiltering

Training classifier(SVM)

Selecting time segment

Bandpassfiltering

Classification(SVM)

29-channelsingle-trial EEGs

(10)Filtered

4 time series

Feature construction

Feature construction

4 features

4 features

(a)

Num

ber o

f tria

ls

0

20

40

60

80

100

5 10 15 20Subject

Yes (train)No (train)

Yes (test)No (test)

(b)

Figure 2 (a) Block diagram of the intention decoding algorithm (b) the number of trials which was selected as training and testing datawithin each answer type for each subject

4 Computational Intelligence and Neuroscience

additive subcomponents so that the differences of variancesare maximized between two classes -at is the most dis-criminative features between two classes are obtained bymaximizing the variance of the spatially filtered signal of oneclass while minimizing that of the other class [22] -e CSPalgorithm is recognized to be effective for the discriminationof mental states from event-related EEG spectral powers[26] -e results of the CSP can be visualized as a topo-graphic map on the scalp which facilitates interpretation offunctional neuroanatomical meanings [26]

-e CSP spatial filter W can be obtained by simulta-neous diagonalization of two covariance matrices of classes 1and 2 as follows

WTΣ1W Λ1

WTΣ2W Λ2(1)

where Λ1 + Λ2 I Σ1 and Σ2 represent the spatial co-variance matrices averaged over all single-trial EEGs foreach class and Λ1 and Λ2 denote the diagonal matrices -eprojection vector w (column vectors of W) can be ob-tained from a generalized eigenvalue decomposition asfollows

Σ1wk λkΣ2wk (2)

wherewk (k 1 C where C is the number of channels) isthe generalized eigenvector and λ1k wT

kΣ1wk and λ2k

wTkΣ2wk are defined as the k

th diagonal element ofΛ1 andΛ2respectively where λk λ1kλ2k Importantly λ1k and λ2k

(ranges from 0 to 1) reflect the variance for each class andλ1k + λ2k 1-us if λ1k is close to 1 λ2k should be close to0 -is means that corresponding projection vector wkshows high variance in class 1 but low variance in class 2-edifference in variances between these two classes enablesdiscriminating one class from another -e eigenvalues aresorted in the descending order during calculation meaningthat the first projection vector yields the highest variance forclass 1 (but the lowest for class 2) whereas the last projectionvector yields the highest variance for class 2 (but lowest forclass 1) -us the first and last projection vectors are themost useful for the discrimination [22]

-e spatial filter W provides the decomposition of asingle-trial multichannel EEG E as ZWTE where E isrepresented as a matrix with C (the number of channels)rows and T (the number of time samples) columns -ecolumns of Wminus1 form the common spatial patterns and canbe visualized as topographies on scalp -e variances of thespatially filtered time-series Z are calculated as features forthe classification as follows

fp logvar Zp1113872 1113873

11139362mi1var Zi( 1113857

⎛⎝ ⎞⎠ where p 1 2 2m (3)

where p is the number of features m was set to 2 whichmeans that the first 2 and last 2 projection vectors were usedas features and thus the number of features p was 4 for allclassifications -e log transformation was adopted to ap-proximate the normal distribution of the data

27 Pattern Classification Using Support Vector Machine(SVM) SVM has been recognized to be a practical androbust method for the classification of human brain signals[27 28] -e SVM is trained to determine an optimal hy-perplane by which the distance to the support vectors(closest to the separating boundary) is maximized [29 30]In the case of the linear SVM classification the hyperplaneaTx+ b satisfies

yi aTxi + b1113872 1113873ge 1minus ξi for i 1 N (4)

where xi fpi denotes a feature vector (in which p 1 4) which can be obtained from the CSP algorithm andyi isin +1minus1 denotes its correct class label N and ξi denotethe total number of training samples and the deviation fromthe optimal condition of linear separability respectively-epair of hyperplanes that provide the maximum separatingmargin can be found by minimizing the cost function(12)aTa + P1113936

Ni1ξi subject to the constraints

yi aTxi + b1113872 1113873ge 1minus ξi

ξi ge 0 for i 1 N(5)

where Pgt 0 represents a regularization penalty parameterof the error term By transforming this optimizationproblem into its dual problem the solution may be de-termined as a 1113936

Ni1αiyixi and achieves equality for

nonzero values of αi only -e corresponding data samplesare referred to as support vectors which are crucial toidentify the decision boundary Instead of the basic linearSVM we used a radial basis function (RBF) kernel whichnonlinearly projects the feature vectors onto a higherdimensional space and thus is better suited for nonlinearrelationships between features and class labels [29] -edetailed parameters of the SVM including the RBF kernelparameter and regularization penalty were determined bytrial-and-error

3 Results

31 YesNo Decoding Figure 3 shows the time-frequencyrepresentation of the ldquoyesnordquo decoding accuracy averagedover all subjects for each time-frequency subwindow -etime-frequency map of decoding accuracy was generated byrepresenting the decoding accuracies averaged over allsubjects within each time-frequency subwindow whichenables estimation of the decoding accuracies over all time-frequency ranges We used two criteria to define the mostimportant time-frequency subwindows showing highdecoding accuracies -e first was to use the threshold levelof a decoding accuracy of 75 determined by the theoretical95 confidence limits of the chance level when 10 trials perclass are used for testing [31] Another criterion was that thedecoding accuracy should be above the mean + 2times standarddeviation value (7934 here) -e high decoding accuraciesabove these two threshold levels were obtained for threesubwindows in the alpha and theta bands (as denoted by thethree boxes in Figure 3) for both early and late periods -ehighest and second highest decoding accuracies were found

Computational Intelligence and Neuroscience 5

in the upper alpha band (10ndash12Hz) at late epoch (box ①8108plusmn 889 at the 1000ndash1200ms box② 7999plusmn 899 atthe 800ndash1000ms) Also the third highest decoding accuracywas found in the upper theta band (6ndash8Hz) at the earlyperiod (box③ 7976plusmn 1021 at the 200ndash400ms) When all12 features within these three best time-frequency sub-windows were used together the decoding accuracy wasdrastically enhanced compared to the best subwindow(10ndash12Hz 1000ndash1200ms) as shown in Figure 4(a) (single8108plusmn 889 combined 8603plusmn 869 t(22)minus595plt 0001 by the paired-sample t-test) e individualdecoding accuracies are presented in Table 1 e sensitivityand specicity values for each time-frequency subwindoware presented in Supplementary Figure 1

32 Spatial Patterns Figure 5 shows the dicenterence betweenthe most important common spatial patterns for ldquonordquo andldquoyesrdquo answers within the three time-frequency subwindows

(averaged over all subjects) Each topography was obtainedfrom the dicenterence between the last (ldquonordquo answer) and rstcolumns (ldquoyesrdquo answer) of the inverse matrix of the pro-jection matrix W for each subject (Supplementary Fig-ure 2) which was calculated in each time-frequencysubwindow and then averaged over all subjects e dif-ference between the most important common spatial pat-terns in the alpha band showed the strongest coecopycient atthe right centroparietal region at both 1000ndash1200ms and800ndash1000ms periods (the leftmost and middle panels inFigure 5 respectively) e dicenterence between the mostimportant common spatial pattern in the theta band at the200ndash400ms period was focused in the right frontal regions(the rightmost panel in Figure 5)

33 Event-Related Spectral Perturbation (ERSP) AnalysisFigure 6(a) shows the topographical distributions of t valuesaveraged within the theta band (4ndash8Hz) in 0ndash1200mse 3

0 600400200 1200800Time (ms)

70

75

80

85

Dec

odin

g ac

cura

cy (

)

10

20

30

40

50

Freq

uenc

y (H

z)

1000

Semantic processingautomatic decision

Retainingldquoyesnordquo in mind

32 1

Figure 3 Decoding accuracies of time-frequency subwindows Color code denotes the decoding accuracy averaged over 23 subjects withineach time-frequency subwindow e best three subwindows are denoted by dashed boxes (①10ndash12Hz 1000ndash1200ms ② 10ndash12Hz800ndash1000ms ③ 6ndash8Hz 200ndash400ms)

Feature construction

29-channelsingle-trial

EEGs

TFspatialfiltering 1

Feature construction

Classificationusing

combined features

Feature construction

TFspatialfiltering 3

Yes

No

TFspatialfiltering 2

4 features

4 features

4 features

(a)

Sing

le

Com

bine

d

lowastlowastlowast

50

60

70

80

90

100

Dec

odin

g ac

cura

cy (

)

(b)

FIGURE 4 Intention decoding using the features from multiple time-frequency subwindows (a) intention decoder using the features frommultiple time-frequency subwindows (b) statistical comparison of the decoding accuracies between the cases of using single and multipletime-frequency subwindows (lowastlowastlowastplt 0001 by the paired-sample t-test error bar standard deviation)

6 Computational Intelligence and Neuroscience

electrodes (FC2 FC6 and C4) showing high t values above apredetermined threshold (tgt 162 corresponding to thehighest 10) were selected over the right frontal region(denoted by black dots in the left panel in Figure 6(a))Significant ldquoyesnordquo difference was found within a singletime-frequency range around 200ndash800ms in the upper thetaand lower alpha bands (6ndash10Hz) which was stronger forldquonordquo compared to ldquoyesrdquo (denoted by a solid contour in theleft panel in Figure 6(b))

In the alpha band 3 electrodes in right parietal area(CP2 Pz and P4) with high t values were selected as de-scribed above (tgt 152 the highest 10) as denoted by blackdots in the right panel in Figure 6(a)-e ldquoyesnordquo differencein spectral power in this region was significant within asingle time-frequency range (300ndash1200ms 9ndash12Hz) wherethe alpha-band power was stronger for ldquonordquo compared toldquoyesrdquo (denoted by a solid contour in the right panel inFigure 6(b))

Table 1 Decoding accuracies for individual subjects

Subject TF subwindow ① TF subwindow ② TF subwindow ③ Combined features

1 8689plusmn 652(7895 100)

8895plusmn 435(8000 9500)

8737plusmn 947(7000 100)

9450plusmn 350(9000 100)

2 7100plusmn 663(5500 8000)

7450plusmn 1011(5500 9000)

7800plusmn 980(6500 9500)

8000plusmn 1049(6000 9500)

3 7600plusmn 1020(6000 9000)

7900plusmn 663(6500 9000)

8200plusmn 714(6500 9000)

8550plusmn 789(6500 9500)

4 8050plusmn 879(6500 9500)

7550plusmn 723(6500 9000)

7100plusmn 800(6000 8500)

8400plusmn 374(8000 9000)

5 8800plusmn 332(8500 9500)

8900plusmn 1044(6500 100)

9200plusmn 748(7500 100)

9600plusmn 490(8500 100)

6 9000plusmn 707(7500 100)

9050plusmn 415(8500 9500)

9200plusmn 557(8500 100)

9300plusmn 400(8500 100)

7 7847plusmn 1321(4500 9474)

7745plusmn 1000(6500 100)

7742plusmn 975(6000 9000)

8445plusmn 817(6500 9500)

8 8300plusmn 557(7500 9500)

7550plusmn 757(6500 8500)

8050plusmn 757(6500 9000)

8800plusmn 872(7500 100)

9 7350plusmn 976(5500 8500)

7000plusmn 837(6000 8500)

6800plusmn 640(5500 8000)

7700plusmn 812(6500 9000)

10 7864plusmn 1095(6000 9474)

6854plusmn 860(6000 8947)

7609plusmn 1277(5000 9474)

8273plusmn 625(7000 9000)

11 7654plusmn 758(6316 8500)

7634plusmn 954(6000 8947)

6759plusmn 600(6000 7500)

7954plusmn 625(6667 9000)

12 9295plusmn 337(8947 100)

9400plusmn 735(7500 100)

9597plusmn 375(9000 100)

9797plusmn 248(9474 100)

13 8789plusmn 654(7895 100)

8284plusmn 976(6500 9500)

7442plusmn 1227(5000 9000)

8545plusmn 817(7000 100)

14 6000plusmn 775(4500 7500)

6500plusmn 1025(4500 7500)

6950plusmn 1404(5000 9000)

7000plusmn 1049(5000 8500)

15 6958plusmn 1498(4444 8889)

7953plusmn 1070(5556 9444)

5896plusmn 1375(3333 7778)

7455plusmn 1076(5789 9444)

16 6895plusmn 849(6000 9000)

6451plusmn 680(5000 7500)

6745plusmn 821(5500 8000)

6948plusmn 700(6000 8500)

17 7900plusmn 943(6500 9000)

7700plusmn 714(6500 8500)

8300plusmn 900(7000 9500)

8950plusmn 789(8000 100)

18 8150plusmn 867(6500 100)

8000plusmn 922(6000 9000)

8800plusmn 714(7000 9500)

9350plusmn 550(8000 100)

19 9300plusmn 510(8500 100)

9700plusmn 400(9000 100)

9450plusmn 269(9000 100)

9800plusmn 245(9500 100)

20 9050plusmn 472(8500 100)

8950plusmn 610(8000 100)

8850plusmn 673(7500 100)

9650plusmn 450(8500 100)

21 8700plusmn 748(7000 9500)

8300plusmn 927(6500 100)

8350plusmn 550(7500 9500)

9150plusmn 594(8000 100)

22 7800plusmn 872(6000 9000)

7350plusmn 808(5500 8500)

7000plusmn 866(5500 8500)

7700plusmn 400(7000 8500)

23 9382plusmn 551(8750 100)

8862plusmn 834(6667 100)

8877plusmn 686(7500 100)

9061plusmn 901(7333 100)

Ave 8108plusmn 889(6000 9382)

7999plusmn 899(6451 9700)

7976plusmn 1021(6745 9597)

8603plusmn 869(6948 9800)

Value meanplusmn standard deviation (SD) of decoding accuracy ()-e range of decoding accuracies was in parenthesis Abbreviation TF time frequency Aveaverage values over all subjects

Computational Intelligence and Neuroscience 7

4 Discussion

We showed that it is possible to decode the intentions toanswer ldquoyesrdquo and ldquonordquo with high accuracy from single-trialEEGs e best decoding accuracy averaged over 23 subjectswas as high as 8603 when useful features in multipletime-frequency subwindows were all combined edecoding accuracy was above 70 for most of the subjects

(22 out of 23 subjects) which is considered as a reasonableaccuracy for the binary classication [32] We decoded theldquoyesrdquo and ldquonordquo answers directly from the brain activitiesrepresenting the two dicenterent answers which implies thatthe ldquomind readingrdquo in a true sense is feasible e exper-imental paradigm of our study is based on a natural taskwhich required the subjects to answer self-referentialquestions as in conversation with others without any self-

6ndash8 Hz200ndash400 ms

10ndash12 Hz800ndash1000 ms

10ndash12 Hz 1000ndash1200 ms

1

ndash1

0

Figure 5 Dicenterence between the most important common spatial patterns for ldquonordquo and ldquoyesrdquo answers averaged over all subjects within 3time-frequency subwindows e topography was obtained from the dicenterence between the last (ldquonordquo answer) and rst (ldquoyesrdquo answer)columns of the inverse of the matrix W for each subject and then averaged over all subjects

4ndash8 Hz 0ndash1200 ms

eta power

4

ndash4

8ndash13 Hz 0ndash1200 ms

Alpha power

t val

ue

(a)

0 600400200 1200800Time (ms)

10

15

20

25

30

Freq

uenc

y (H

z)

1000

Right frontal

5

Nondashyes

p lt 005

0 600400200 1200800Time (ms)

ndash5

5

t val

ue10

15

20

25

30

Freq

uenc

y (H

z)

1000

Nondashyes

Right parietal p lt 005

5

(b)

Figure 6 Statistical comparisons in spectral power between ldquoyesrdquo and ldquonordquo (a) Topographical distributions of t values (left theta band(4ndash8Hz) 0ndash1200ms right alpha band (8ndash13Hz) 0ndash1200ms) Black dots (FC2 FC6 and C4 in the left panel and CP2 Pz and P4 in the rightpanel) denote high t values above a predetermined threshold corresponding to the highest 10 (b) e clusters in the time-frequencydomain showing signicant dicenterences between ldquoyesrdquo and ldquonordquo in right frontal (left panel) and right parietal regions (right panel) (denotedby black contours)

8 Computational Intelligence and Neuroscience

regulation of the brain signals or high cognitive efforts Nounpleasant stimuli and volition or high cognitive efforts arerequired since our approach is based on a direct decodingof ldquoyesrdquo and ldquonordquo without any self-regulation of the brainsignals Birbaumerrsquos group has suggested a new alternativeapproach based on classical conditioning to solve theproblem of conventional BCI in the CLIS patients [16ndash18]For the training two distinct unconditioned stimuli arepresented to the subjects immediately after the simple ldquoyesnordquo questions (corresponding to the conditioned stimuli)so that the cortical responses can be conditioned differentlyfor yes and no -e unconditioned stimuli include auditorypink noise and white noise [16 18] and weak electricalstimulation to the thumb [17] -e main idea of this ap-proach is to modulate the usersrsquo brain activities indirectlythrough the unconditioned stimuli so that ldquoyesrdquo and ldquonordquocan be easily discriminated from neural signals respondingto the sensory stimuli rather than to read the usersrsquo an-swers from neural signals -is approach may provide analternative to the conventional BCI approaches in thatvolition or high cognitive efforts are not required How-ever it remains unclear how long the conditioned corticalresponse can be maintained considering the extinctioneffect of classical conditioning [33] Moreover un-conditioned stimuli such as auditory noise or electricalstimulation can evoke significant displeasure

Recently a more natural approach for the ldquoyesnordquodecoding was demonstrated based on functional near-in-frared spectroscopy (fNIRS) in the CLIS patients [34] -eyachieved ldquoyesnordquo decoding accuracy over 70 based onfNIRS signals which were recorded while the patientsanswered ldquoyesrdquo or ldquonordquo to personal and open questions inminds repeatedly Interestingly for the same experimentalprotocol they reported that EEG-based decoding yieldedaccuracy below the chance level -is study employed anatural questionanswer task which does not require highcognitive efforts or volition just as ours But due to the slownature of hemodynamics the duration of each trial for thedecoding was quite long (gt10 sec) Here we showed thepossibility of ldquoyesnordquo decoding from considerably shortersignal recording which is more beneficial for a practicalBCI communication tool

We took a systematic approach of finding features ofbrain activities reflecting ldquoyesnordquo answers in minds andthen developing the decoding algorithm by utilizing thesefeatures Further studies may be necessary to investigatewhether the patients who would potentially benefit fromthe BCI can hold the intentions to answer in minds for ashort time and to validate our method on the patientsrsquo data

In this study the intentions regarding self-referentialquestions based on the autobiographic facts were in-vestigated It is important to further try decoding the in-tentions to answer various types of questions includingdesire feeling and preference In addition our questionswere presented only in visual stimuli Neurological patientsmay have an abnormal visual function such as disability tofix their gaze on specific visual stimuli [35] Different sensorymodality such as auditory stimuli has been tried for the BCIcommunication tools [34 36] It would be beneficial if our

approach can be validated with auditory stimuli such asvoice considering that a high decoding accuracy above 80was obtained even when the brain activities during theperiod of retaining the decision in minds (10ndash12Hz1000ndash1200ms) used for the decoding-us we expect that itis possible to decode the ldquoyesrdquo and ldquonordquo intentions in asimilar way even if other types of questions andor theauditory stimuli are employed in the further studies Inaddition here we did not try to optimize the detailed pa-rameters of the SVM including the RBF kernel parameterand regularization penalty -e use of the best parameters ofthe SVM for example by using the ldquogrid-searchrdquo method[37] may be obviously helpful for better results

We found two time-frequency regions containing usefulinformation for the ldquoyesnordquo decoding in early theta and latealpha bands-e useful features for the ldquoyesnordquo decoding inthe alpha band were found to be concentrated in the parietalregion at 800ndash1200ms from the CSP algorithm Recently weshowed that the alpha rhythms in the right parietal regionare differentiated between the intentions to answers eitherldquoyesrdquo or ldquonordquo in minds presumably due to the difference incognitive loads for the WM retention [2] Several previousstudies showed that the higher parietal alpha power reflectsincreased memory load [38 39] or attentional demand[40 41] during WM retention -e higher alpha power isattributed to active inhibitory control to block incomingstimuli during WM retention for efficient cortical in-formation processing [38 39 42 43] Our results showedhigher parietal alpha power for ldquonordquo compared to ldquoyesrdquowhich may imply higher cognitive load during retainingldquonordquo in minds compared to ldquoyesrdquo [2] -e greater increase inalpha-band activity for ldquonordquo may reflect the increased WMload during the intention retention In Korean languageldquoyesrdquo is the one-character word ldquo네rdquo and ldquonordquo is three-character word ldquo아니오rdquo It is plausible that the higherWMload is required to represent intention to respond ldquonordquo thanldquoyesrdquo due to the length of the Korean words resulting in thehigher alpha rhythm -is assumption is supported by anERP study which reported that greater alpha-band powerwas induced for retaining longer word [44]

It can also be interpreted that the significantly higheralpha-band activity in the centroparietal region for ldquonordquo isdue to the higher attentional demand [40 45] and thiscontributed to the high decoding accuracy -is is also inagreement with a recent study [46] which reported that ahigher alpha rhythm was identified in the right parietalcortex for a higher internal attention condition during adivergent thinking task Our result of greater alpha powerfor ldquonordquo than for ldquoyesrdquo may imply a stronger inhibition of theouter stimuli by the bottom-up attention network for ldquonordquoinduced by higher internal attentional demand -is issupported by psychophysical which showed that saying ldquonordquorequires more effortful reconsideration after comprehendinga sentence and a longer response time for saying ldquonordquo thanldquoyesrdquo [47 48]

-e theta-band activity in the frontal region in 200ndash500ms was another major feature for ldquoyesnordquo decoding-e theta ERS showed topography focused on midlinefrontal and lateral temporal regions -e difference

Computational Intelligence and Neuroscience 9

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

additive subcomponents so that the differences of variancesare maximized between two classes -at is the most dis-criminative features between two classes are obtained bymaximizing the variance of the spatially filtered signal of oneclass while minimizing that of the other class [22] -e CSPalgorithm is recognized to be effective for the discriminationof mental states from event-related EEG spectral powers[26] -e results of the CSP can be visualized as a topo-graphic map on the scalp which facilitates interpretation offunctional neuroanatomical meanings [26]

-e CSP spatial filter W can be obtained by simulta-neous diagonalization of two covariance matrices of classes 1and 2 as follows

WTΣ1W Λ1

WTΣ2W Λ2(1)

where Λ1 + Λ2 I Σ1 and Σ2 represent the spatial co-variance matrices averaged over all single-trial EEGs foreach class and Λ1 and Λ2 denote the diagonal matrices -eprojection vector w (column vectors of W) can be ob-tained from a generalized eigenvalue decomposition asfollows

Σ1wk λkΣ2wk (2)

wherewk (k 1 C where C is the number of channels) isthe generalized eigenvector and λ1k wT

kΣ1wk and λ2k

wTkΣ2wk are defined as the k

th diagonal element ofΛ1 andΛ2respectively where λk λ1kλ2k Importantly λ1k and λ2k

(ranges from 0 to 1) reflect the variance for each class andλ1k + λ2k 1-us if λ1k is close to 1 λ2k should be close to0 -is means that corresponding projection vector wkshows high variance in class 1 but low variance in class 2-edifference in variances between these two classes enablesdiscriminating one class from another -e eigenvalues aresorted in the descending order during calculation meaningthat the first projection vector yields the highest variance forclass 1 (but the lowest for class 2) whereas the last projectionvector yields the highest variance for class 2 (but lowest forclass 1) -us the first and last projection vectors are themost useful for the discrimination [22]

-e spatial filter W provides the decomposition of asingle-trial multichannel EEG E as ZWTE where E isrepresented as a matrix with C (the number of channels)rows and T (the number of time samples) columns -ecolumns of Wminus1 form the common spatial patterns and canbe visualized as topographies on scalp -e variances of thespatially filtered time-series Z are calculated as features forthe classification as follows

fp logvar Zp1113872 1113873

11139362mi1var Zi( 1113857

⎛⎝ ⎞⎠ where p 1 2 2m (3)

where p is the number of features m was set to 2 whichmeans that the first 2 and last 2 projection vectors were usedas features and thus the number of features p was 4 for allclassifications -e log transformation was adopted to ap-proximate the normal distribution of the data

27 Pattern Classification Using Support Vector Machine(SVM) SVM has been recognized to be a practical androbust method for the classification of human brain signals[27 28] -e SVM is trained to determine an optimal hy-perplane by which the distance to the support vectors(closest to the separating boundary) is maximized [29 30]In the case of the linear SVM classification the hyperplaneaTx+ b satisfies

yi aTxi + b1113872 1113873ge 1minus ξi for i 1 N (4)

where xi fpi denotes a feature vector (in which p 1 4) which can be obtained from the CSP algorithm andyi isin +1minus1 denotes its correct class label N and ξi denotethe total number of training samples and the deviation fromthe optimal condition of linear separability respectively-epair of hyperplanes that provide the maximum separatingmargin can be found by minimizing the cost function(12)aTa + P1113936

Ni1ξi subject to the constraints

yi aTxi + b1113872 1113873ge 1minus ξi

ξi ge 0 for i 1 N(5)

where Pgt 0 represents a regularization penalty parameterof the error term By transforming this optimizationproblem into its dual problem the solution may be de-termined as a 1113936

Ni1αiyixi and achieves equality for

nonzero values of αi only -e corresponding data samplesare referred to as support vectors which are crucial toidentify the decision boundary Instead of the basic linearSVM we used a radial basis function (RBF) kernel whichnonlinearly projects the feature vectors onto a higherdimensional space and thus is better suited for nonlinearrelationships between features and class labels [29] -edetailed parameters of the SVM including the RBF kernelparameter and regularization penalty were determined bytrial-and-error

3 Results

31 YesNo Decoding Figure 3 shows the time-frequencyrepresentation of the ldquoyesnordquo decoding accuracy averagedover all subjects for each time-frequency subwindow -etime-frequency map of decoding accuracy was generated byrepresenting the decoding accuracies averaged over allsubjects within each time-frequency subwindow whichenables estimation of the decoding accuracies over all time-frequency ranges We used two criteria to define the mostimportant time-frequency subwindows showing highdecoding accuracies -e first was to use the threshold levelof a decoding accuracy of 75 determined by the theoretical95 confidence limits of the chance level when 10 trials perclass are used for testing [31] Another criterion was that thedecoding accuracy should be above the mean + 2times standarddeviation value (7934 here) -e high decoding accuraciesabove these two threshold levels were obtained for threesubwindows in the alpha and theta bands (as denoted by thethree boxes in Figure 3) for both early and late periods -ehighest and second highest decoding accuracies were found

Computational Intelligence and Neuroscience 5

in the upper alpha band (10ndash12Hz) at late epoch (box ①8108plusmn 889 at the 1000ndash1200ms box② 7999plusmn 899 atthe 800ndash1000ms) Also the third highest decoding accuracywas found in the upper theta band (6ndash8Hz) at the earlyperiod (box③ 7976plusmn 1021 at the 200ndash400ms) When all12 features within these three best time-frequency sub-windows were used together the decoding accuracy wasdrastically enhanced compared to the best subwindow(10ndash12Hz 1000ndash1200ms) as shown in Figure 4(a) (single8108plusmn 889 combined 8603plusmn 869 t(22)minus595plt 0001 by the paired-sample t-test) e individualdecoding accuracies are presented in Table 1 e sensitivityand specicity values for each time-frequency subwindoware presented in Supplementary Figure 1

32 Spatial Patterns Figure 5 shows the dicenterence betweenthe most important common spatial patterns for ldquonordquo andldquoyesrdquo answers within the three time-frequency subwindows

(averaged over all subjects) Each topography was obtainedfrom the dicenterence between the last (ldquonordquo answer) and rstcolumns (ldquoyesrdquo answer) of the inverse matrix of the pro-jection matrix W for each subject (Supplementary Fig-ure 2) which was calculated in each time-frequencysubwindow and then averaged over all subjects e dif-ference between the most important common spatial pat-terns in the alpha band showed the strongest coecopycient atthe right centroparietal region at both 1000ndash1200ms and800ndash1000ms periods (the leftmost and middle panels inFigure 5 respectively) e dicenterence between the mostimportant common spatial pattern in the theta band at the200ndash400ms period was focused in the right frontal regions(the rightmost panel in Figure 5)

33 Event-Related Spectral Perturbation (ERSP) AnalysisFigure 6(a) shows the topographical distributions of t valuesaveraged within the theta band (4ndash8Hz) in 0ndash1200mse 3

0 600400200 1200800Time (ms)

70

75

80

85

Dec

odin

g ac

cura

cy (

)

10

20

30

40

50

Freq

uenc

y (H

z)

1000

Semantic processingautomatic decision

Retainingldquoyesnordquo in mind

32 1

Figure 3 Decoding accuracies of time-frequency subwindows Color code denotes the decoding accuracy averaged over 23 subjects withineach time-frequency subwindow e best three subwindows are denoted by dashed boxes (①10ndash12Hz 1000ndash1200ms ② 10ndash12Hz800ndash1000ms ③ 6ndash8Hz 200ndash400ms)

Feature construction

29-channelsingle-trial

EEGs

TFspatialfiltering 1

Feature construction

Classificationusing

combined features

Feature construction

TFspatialfiltering 3

Yes

No

TFspatialfiltering 2

4 features

4 features

4 features

(a)

Sing

le

Com

bine

d

lowastlowastlowast

50

60

70

80

90

100

Dec

odin

g ac

cura

cy (

)

(b)

FIGURE 4 Intention decoding using the features from multiple time-frequency subwindows (a) intention decoder using the features frommultiple time-frequency subwindows (b) statistical comparison of the decoding accuracies between the cases of using single and multipletime-frequency subwindows (lowastlowastlowastplt 0001 by the paired-sample t-test error bar standard deviation)

6 Computational Intelligence and Neuroscience

electrodes (FC2 FC6 and C4) showing high t values above apredetermined threshold (tgt 162 corresponding to thehighest 10) were selected over the right frontal region(denoted by black dots in the left panel in Figure 6(a))Significant ldquoyesnordquo difference was found within a singletime-frequency range around 200ndash800ms in the upper thetaand lower alpha bands (6ndash10Hz) which was stronger forldquonordquo compared to ldquoyesrdquo (denoted by a solid contour in theleft panel in Figure 6(b))

In the alpha band 3 electrodes in right parietal area(CP2 Pz and P4) with high t values were selected as de-scribed above (tgt 152 the highest 10) as denoted by blackdots in the right panel in Figure 6(a)-e ldquoyesnordquo differencein spectral power in this region was significant within asingle time-frequency range (300ndash1200ms 9ndash12Hz) wherethe alpha-band power was stronger for ldquonordquo compared toldquoyesrdquo (denoted by a solid contour in the right panel inFigure 6(b))

Table 1 Decoding accuracies for individual subjects

Subject TF subwindow ① TF subwindow ② TF subwindow ③ Combined features

1 8689plusmn 652(7895 100)

8895plusmn 435(8000 9500)

8737plusmn 947(7000 100)

9450plusmn 350(9000 100)

2 7100plusmn 663(5500 8000)

7450plusmn 1011(5500 9000)

7800plusmn 980(6500 9500)

8000plusmn 1049(6000 9500)

3 7600plusmn 1020(6000 9000)

7900plusmn 663(6500 9000)

8200plusmn 714(6500 9000)

8550plusmn 789(6500 9500)

4 8050plusmn 879(6500 9500)

7550plusmn 723(6500 9000)

7100plusmn 800(6000 8500)

8400plusmn 374(8000 9000)

5 8800plusmn 332(8500 9500)

8900plusmn 1044(6500 100)

9200plusmn 748(7500 100)

9600plusmn 490(8500 100)

6 9000plusmn 707(7500 100)

9050plusmn 415(8500 9500)

9200plusmn 557(8500 100)

9300plusmn 400(8500 100)

7 7847plusmn 1321(4500 9474)

7745plusmn 1000(6500 100)

7742plusmn 975(6000 9000)

8445plusmn 817(6500 9500)

8 8300plusmn 557(7500 9500)

7550plusmn 757(6500 8500)

8050plusmn 757(6500 9000)

8800plusmn 872(7500 100)

9 7350plusmn 976(5500 8500)

7000plusmn 837(6000 8500)

6800plusmn 640(5500 8000)

7700plusmn 812(6500 9000)

10 7864plusmn 1095(6000 9474)

6854plusmn 860(6000 8947)

7609plusmn 1277(5000 9474)

8273plusmn 625(7000 9000)

11 7654plusmn 758(6316 8500)

7634plusmn 954(6000 8947)

6759plusmn 600(6000 7500)

7954plusmn 625(6667 9000)

12 9295plusmn 337(8947 100)

9400plusmn 735(7500 100)

9597plusmn 375(9000 100)

9797plusmn 248(9474 100)

13 8789plusmn 654(7895 100)

8284plusmn 976(6500 9500)

7442plusmn 1227(5000 9000)

8545plusmn 817(7000 100)

14 6000plusmn 775(4500 7500)

6500plusmn 1025(4500 7500)

6950plusmn 1404(5000 9000)

7000plusmn 1049(5000 8500)

15 6958plusmn 1498(4444 8889)

7953plusmn 1070(5556 9444)

5896plusmn 1375(3333 7778)

7455plusmn 1076(5789 9444)

16 6895plusmn 849(6000 9000)

6451plusmn 680(5000 7500)

6745plusmn 821(5500 8000)

6948plusmn 700(6000 8500)

17 7900plusmn 943(6500 9000)

7700plusmn 714(6500 8500)

8300plusmn 900(7000 9500)

8950plusmn 789(8000 100)

18 8150plusmn 867(6500 100)

8000plusmn 922(6000 9000)

8800plusmn 714(7000 9500)

9350plusmn 550(8000 100)

19 9300plusmn 510(8500 100)

9700plusmn 400(9000 100)

9450plusmn 269(9000 100)

9800plusmn 245(9500 100)

20 9050plusmn 472(8500 100)

8950plusmn 610(8000 100)

8850plusmn 673(7500 100)

9650plusmn 450(8500 100)

21 8700plusmn 748(7000 9500)

8300plusmn 927(6500 100)

8350plusmn 550(7500 9500)

9150plusmn 594(8000 100)

22 7800plusmn 872(6000 9000)

7350plusmn 808(5500 8500)

7000plusmn 866(5500 8500)

7700plusmn 400(7000 8500)

23 9382plusmn 551(8750 100)

8862plusmn 834(6667 100)

8877plusmn 686(7500 100)

9061plusmn 901(7333 100)

Ave 8108plusmn 889(6000 9382)

7999plusmn 899(6451 9700)

7976plusmn 1021(6745 9597)

8603plusmn 869(6948 9800)

Value meanplusmn standard deviation (SD) of decoding accuracy ()-e range of decoding accuracies was in parenthesis Abbreviation TF time frequency Aveaverage values over all subjects

Computational Intelligence and Neuroscience 7

4 Discussion

We showed that it is possible to decode the intentions toanswer ldquoyesrdquo and ldquonordquo with high accuracy from single-trialEEGs e best decoding accuracy averaged over 23 subjectswas as high as 8603 when useful features in multipletime-frequency subwindows were all combined edecoding accuracy was above 70 for most of the subjects

(22 out of 23 subjects) which is considered as a reasonableaccuracy for the binary classication [32] We decoded theldquoyesrdquo and ldquonordquo answers directly from the brain activitiesrepresenting the two dicenterent answers which implies thatthe ldquomind readingrdquo in a true sense is feasible e exper-imental paradigm of our study is based on a natural taskwhich required the subjects to answer self-referentialquestions as in conversation with others without any self-

6ndash8 Hz200ndash400 ms

10ndash12 Hz800ndash1000 ms

10ndash12 Hz 1000ndash1200 ms

1

ndash1

0

Figure 5 Dicenterence between the most important common spatial patterns for ldquonordquo and ldquoyesrdquo answers averaged over all subjects within 3time-frequency subwindows e topography was obtained from the dicenterence between the last (ldquonordquo answer) and rst (ldquoyesrdquo answer)columns of the inverse of the matrix W for each subject and then averaged over all subjects

4ndash8 Hz 0ndash1200 ms

eta power

4

ndash4

8ndash13 Hz 0ndash1200 ms

Alpha power

t val

ue

(a)

0 600400200 1200800Time (ms)

10

15

20

25

30

Freq

uenc

y (H

z)

1000

Right frontal

5

Nondashyes

p lt 005

0 600400200 1200800Time (ms)

ndash5

5

t val

ue10

15

20

25

30

Freq

uenc

y (H

z)

1000

Nondashyes

Right parietal p lt 005

5

(b)

Figure 6 Statistical comparisons in spectral power between ldquoyesrdquo and ldquonordquo (a) Topographical distributions of t values (left theta band(4ndash8Hz) 0ndash1200ms right alpha band (8ndash13Hz) 0ndash1200ms) Black dots (FC2 FC6 and C4 in the left panel and CP2 Pz and P4 in the rightpanel) denote high t values above a predetermined threshold corresponding to the highest 10 (b) e clusters in the time-frequencydomain showing signicant dicenterences between ldquoyesrdquo and ldquonordquo in right frontal (left panel) and right parietal regions (right panel) (denotedby black contours)

8 Computational Intelligence and Neuroscience

regulation of the brain signals or high cognitive efforts Nounpleasant stimuli and volition or high cognitive efforts arerequired since our approach is based on a direct decodingof ldquoyesrdquo and ldquonordquo without any self-regulation of the brainsignals Birbaumerrsquos group has suggested a new alternativeapproach based on classical conditioning to solve theproblem of conventional BCI in the CLIS patients [16ndash18]For the training two distinct unconditioned stimuli arepresented to the subjects immediately after the simple ldquoyesnordquo questions (corresponding to the conditioned stimuli)so that the cortical responses can be conditioned differentlyfor yes and no -e unconditioned stimuli include auditorypink noise and white noise [16 18] and weak electricalstimulation to the thumb [17] -e main idea of this ap-proach is to modulate the usersrsquo brain activities indirectlythrough the unconditioned stimuli so that ldquoyesrdquo and ldquonordquocan be easily discriminated from neural signals respondingto the sensory stimuli rather than to read the usersrsquo an-swers from neural signals -is approach may provide analternative to the conventional BCI approaches in thatvolition or high cognitive efforts are not required How-ever it remains unclear how long the conditioned corticalresponse can be maintained considering the extinctioneffect of classical conditioning [33] Moreover un-conditioned stimuli such as auditory noise or electricalstimulation can evoke significant displeasure

Recently a more natural approach for the ldquoyesnordquodecoding was demonstrated based on functional near-in-frared spectroscopy (fNIRS) in the CLIS patients [34] -eyachieved ldquoyesnordquo decoding accuracy over 70 based onfNIRS signals which were recorded while the patientsanswered ldquoyesrdquo or ldquonordquo to personal and open questions inminds repeatedly Interestingly for the same experimentalprotocol they reported that EEG-based decoding yieldedaccuracy below the chance level -is study employed anatural questionanswer task which does not require highcognitive efforts or volition just as ours But due to the slownature of hemodynamics the duration of each trial for thedecoding was quite long (gt10 sec) Here we showed thepossibility of ldquoyesnordquo decoding from considerably shortersignal recording which is more beneficial for a practicalBCI communication tool

We took a systematic approach of finding features ofbrain activities reflecting ldquoyesnordquo answers in minds andthen developing the decoding algorithm by utilizing thesefeatures Further studies may be necessary to investigatewhether the patients who would potentially benefit fromthe BCI can hold the intentions to answer in minds for ashort time and to validate our method on the patientsrsquo data

In this study the intentions regarding self-referentialquestions based on the autobiographic facts were in-vestigated It is important to further try decoding the in-tentions to answer various types of questions includingdesire feeling and preference In addition our questionswere presented only in visual stimuli Neurological patientsmay have an abnormal visual function such as disability tofix their gaze on specific visual stimuli [35] Different sensorymodality such as auditory stimuli has been tried for the BCIcommunication tools [34 36] It would be beneficial if our

approach can be validated with auditory stimuli such asvoice considering that a high decoding accuracy above 80was obtained even when the brain activities during theperiod of retaining the decision in minds (10ndash12Hz1000ndash1200ms) used for the decoding-us we expect that itis possible to decode the ldquoyesrdquo and ldquonordquo intentions in asimilar way even if other types of questions andor theauditory stimuli are employed in the further studies Inaddition here we did not try to optimize the detailed pa-rameters of the SVM including the RBF kernel parameterand regularization penalty -e use of the best parameters ofthe SVM for example by using the ldquogrid-searchrdquo method[37] may be obviously helpful for better results

We found two time-frequency regions containing usefulinformation for the ldquoyesnordquo decoding in early theta and latealpha bands-e useful features for the ldquoyesnordquo decoding inthe alpha band were found to be concentrated in the parietalregion at 800ndash1200ms from the CSP algorithm Recently weshowed that the alpha rhythms in the right parietal regionare differentiated between the intentions to answers eitherldquoyesrdquo or ldquonordquo in minds presumably due to the difference incognitive loads for the WM retention [2] Several previousstudies showed that the higher parietal alpha power reflectsincreased memory load [38 39] or attentional demand[40 41] during WM retention -e higher alpha power isattributed to active inhibitory control to block incomingstimuli during WM retention for efficient cortical in-formation processing [38 39 42 43] Our results showedhigher parietal alpha power for ldquonordquo compared to ldquoyesrdquowhich may imply higher cognitive load during retainingldquonordquo in minds compared to ldquoyesrdquo [2] -e greater increase inalpha-band activity for ldquonordquo may reflect the increased WMload during the intention retention In Korean languageldquoyesrdquo is the one-character word ldquo네rdquo and ldquonordquo is three-character word ldquo아니오rdquo It is plausible that the higherWMload is required to represent intention to respond ldquonordquo thanldquoyesrdquo due to the length of the Korean words resulting in thehigher alpha rhythm -is assumption is supported by anERP study which reported that greater alpha-band powerwas induced for retaining longer word [44]

It can also be interpreted that the significantly higheralpha-band activity in the centroparietal region for ldquonordquo isdue to the higher attentional demand [40 45] and thiscontributed to the high decoding accuracy -is is also inagreement with a recent study [46] which reported that ahigher alpha rhythm was identified in the right parietalcortex for a higher internal attention condition during adivergent thinking task Our result of greater alpha powerfor ldquonordquo than for ldquoyesrdquo may imply a stronger inhibition of theouter stimuli by the bottom-up attention network for ldquonordquoinduced by higher internal attentional demand -is issupported by psychophysical which showed that saying ldquonordquorequires more effortful reconsideration after comprehendinga sentence and a longer response time for saying ldquonordquo thanldquoyesrdquo [47 48]

-e theta-band activity in the frontal region in 200ndash500ms was another major feature for ldquoyesnordquo decoding-e theta ERS showed topography focused on midlinefrontal and lateral temporal regions -e difference

Computational Intelligence and Neuroscience 9

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

in the upper alpha band (10ndash12Hz) at late epoch (box ①8108plusmn 889 at the 1000ndash1200ms box② 7999plusmn 899 atthe 800ndash1000ms) Also the third highest decoding accuracywas found in the upper theta band (6ndash8Hz) at the earlyperiod (box③ 7976plusmn 1021 at the 200ndash400ms) When all12 features within these three best time-frequency sub-windows were used together the decoding accuracy wasdrastically enhanced compared to the best subwindow(10ndash12Hz 1000ndash1200ms) as shown in Figure 4(a) (single8108plusmn 889 combined 8603plusmn 869 t(22)minus595plt 0001 by the paired-sample t-test) e individualdecoding accuracies are presented in Table 1 e sensitivityand specicity values for each time-frequency subwindoware presented in Supplementary Figure 1

32 Spatial Patterns Figure 5 shows the dicenterence betweenthe most important common spatial patterns for ldquonordquo andldquoyesrdquo answers within the three time-frequency subwindows

(averaged over all subjects) Each topography was obtainedfrom the dicenterence between the last (ldquonordquo answer) and rstcolumns (ldquoyesrdquo answer) of the inverse matrix of the pro-jection matrix W for each subject (Supplementary Fig-ure 2) which was calculated in each time-frequencysubwindow and then averaged over all subjects e dif-ference between the most important common spatial pat-terns in the alpha band showed the strongest coecopycient atthe right centroparietal region at both 1000ndash1200ms and800ndash1000ms periods (the leftmost and middle panels inFigure 5 respectively) e dicenterence between the mostimportant common spatial pattern in the theta band at the200ndash400ms period was focused in the right frontal regions(the rightmost panel in Figure 5)

33 Event-Related Spectral Perturbation (ERSP) AnalysisFigure 6(a) shows the topographical distributions of t valuesaveraged within the theta band (4ndash8Hz) in 0ndash1200mse 3

0 600400200 1200800Time (ms)

70

75

80

85

Dec

odin

g ac

cura

cy (

)

10

20

30

40

50

Freq

uenc

y (H

z)

1000

Semantic processingautomatic decision

Retainingldquoyesnordquo in mind

32 1

Figure 3 Decoding accuracies of time-frequency subwindows Color code denotes the decoding accuracy averaged over 23 subjects withineach time-frequency subwindow e best three subwindows are denoted by dashed boxes (①10ndash12Hz 1000ndash1200ms ② 10ndash12Hz800ndash1000ms ③ 6ndash8Hz 200ndash400ms)

Feature construction

29-channelsingle-trial

EEGs

TFspatialfiltering 1

Feature construction

Classificationusing

combined features

Feature construction

TFspatialfiltering 3

Yes

No

TFspatialfiltering 2

4 features

4 features

4 features

(a)

Sing

le

Com

bine

d

lowastlowastlowast

50

60

70

80

90

100

Dec

odin

g ac

cura

cy (

)

(b)

FIGURE 4 Intention decoding using the features from multiple time-frequency subwindows (a) intention decoder using the features frommultiple time-frequency subwindows (b) statistical comparison of the decoding accuracies between the cases of using single and multipletime-frequency subwindows (lowastlowastlowastplt 0001 by the paired-sample t-test error bar standard deviation)

6 Computational Intelligence and Neuroscience

electrodes (FC2 FC6 and C4) showing high t values above apredetermined threshold (tgt 162 corresponding to thehighest 10) were selected over the right frontal region(denoted by black dots in the left panel in Figure 6(a))Significant ldquoyesnordquo difference was found within a singletime-frequency range around 200ndash800ms in the upper thetaand lower alpha bands (6ndash10Hz) which was stronger forldquonordquo compared to ldquoyesrdquo (denoted by a solid contour in theleft panel in Figure 6(b))

In the alpha band 3 electrodes in right parietal area(CP2 Pz and P4) with high t values were selected as de-scribed above (tgt 152 the highest 10) as denoted by blackdots in the right panel in Figure 6(a)-e ldquoyesnordquo differencein spectral power in this region was significant within asingle time-frequency range (300ndash1200ms 9ndash12Hz) wherethe alpha-band power was stronger for ldquonordquo compared toldquoyesrdquo (denoted by a solid contour in the right panel inFigure 6(b))

Table 1 Decoding accuracies for individual subjects

Subject TF subwindow ① TF subwindow ② TF subwindow ③ Combined features

1 8689plusmn 652(7895 100)

8895plusmn 435(8000 9500)

8737plusmn 947(7000 100)

9450plusmn 350(9000 100)

2 7100plusmn 663(5500 8000)

7450plusmn 1011(5500 9000)

7800plusmn 980(6500 9500)

8000plusmn 1049(6000 9500)

3 7600plusmn 1020(6000 9000)

7900plusmn 663(6500 9000)

8200plusmn 714(6500 9000)

8550plusmn 789(6500 9500)

4 8050plusmn 879(6500 9500)

7550plusmn 723(6500 9000)

7100plusmn 800(6000 8500)

8400plusmn 374(8000 9000)

5 8800plusmn 332(8500 9500)

8900plusmn 1044(6500 100)

9200plusmn 748(7500 100)

9600plusmn 490(8500 100)

6 9000plusmn 707(7500 100)

9050plusmn 415(8500 9500)

9200plusmn 557(8500 100)

9300plusmn 400(8500 100)

7 7847plusmn 1321(4500 9474)

7745plusmn 1000(6500 100)

7742plusmn 975(6000 9000)

8445plusmn 817(6500 9500)

8 8300plusmn 557(7500 9500)

7550plusmn 757(6500 8500)

8050plusmn 757(6500 9000)

8800plusmn 872(7500 100)

9 7350plusmn 976(5500 8500)

7000plusmn 837(6000 8500)

6800plusmn 640(5500 8000)

7700plusmn 812(6500 9000)

10 7864plusmn 1095(6000 9474)

6854plusmn 860(6000 8947)

7609plusmn 1277(5000 9474)

8273plusmn 625(7000 9000)

11 7654plusmn 758(6316 8500)

7634plusmn 954(6000 8947)

6759plusmn 600(6000 7500)

7954plusmn 625(6667 9000)

12 9295plusmn 337(8947 100)

9400plusmn 735(7500 100)

9597plusmn 375(9000 100)

9797plusmn 248(9474 100)

13 8789plusmn 654(7895 100)

8284plusmn 976(6500 9500)

7442plusmn 1227(5000 9000)

8545plusmn 817(7000 100)

14 6000plusmn 775(4500 7500)

6500plusmn 1025(4500 7500)

6950plusmn 1404(5000 9000)

7000plusmn 1049(5000 8500)

15 6958plusmn 1498(4444 8889)

7953plusmn 1070(5556 9444)

5896plusmn 1375(3333 7778)

7455plusmn 1076(5789 9444)

16 6895plusmn 849(6000 9000)

6451plusmn 680(5000 7500)

6745plusmn 821(5500 8000)

6948plusmn 700(6000 8500)

17 7900plusmn 943(6500 9000)

7700plusmn 714(6500 8500)

8300plusmn 900(7000 9500)

8950plusmn 789(8000 100)

18 8150plusmn 867(6500 100)

8000plusmn 922(6000 9000)

8800plusmn 714(7000 9500)

9350plusmn 550(8000 100)

19 9300plusmn 510(8500 100)

9700plusmn 400(9000 100)

9450plusmn 269(9000 100)

9800plusmn 245(9500 100)

20 9050plusmn 472(8500 100)

8950plusmn 610(8000 100)

8850plusmn 673(7500 100)

9650plusmn 450(8500 100)

21 8700plusmn 748(7000 9500)

8300plusmn 927(6500 100)

8350plusmn 550(7500 9500)

9150plusmn 594(8000 100)

22 7800plusmn 872(6000 9000)

7350plusmn 808(5500 8500)

7000plusmn 866(5500 8500)

7700plusmn 400(7000 8500)

23 9382plusmn 551(8750 100)

8862plusmn 834(6667 100)

8877plusmn 686(7500 100)

9061plusmn 901(7333 100)

Ave 8108plusmn 889(6000 9382)

7999plusmn 899(6451 9700)

7976plusmn 1021(6745 9597)

8603plusmn 869(6948 9800)

Value meanplusmn standard deviation (SD) of decoding accuracy ()-e range of decoding accuracies was in parenthesis Abbreviation TF time frequency Aveaverage values over all subjects

Computational Intelligence and Neuroscience 7

4 Discussion

We showed that it is possible to decode the intentions toanswer ldquoyesrdquo and ldquonordquo with high accuracy from single-trialEEGs e best decoding accuracy averaged over 23 subjectswas as high as 8603 when useful features in multipletime-frequency subwindows were all combined edecoding accuracy was above 70 for most of the subjects

(22 out of 23 subjects) which is considered as a reasonableaccuracy for the binary classication [32] We decoded theldquoyesrdquo and ldquonordquo answers directly from the brain activitiesrepresenting the two dicenterent answers which implies thatthe ldquomind readingrdquo in a true sense is feasible e exper-imental paradigm of our study is based on a natural taskwhich required the subjects to answer self-referentialquestions as in conversation with others without any self-

6ndash8 Hz200ndash400 ms

10ndash12 Hz800ndash1000 ms

10ndash12 Hz 1000ndash1200 ms

1

ndash1

0

Figure 5 Dicenterence between the most important common spatial patterns for ldquonordquo and ldquoyesrdquo answers averaged over all subjects within 3time-frequency subwindows e topography was obtained from the dicenterence between the last (ldquonordquo answer) and rst (ldquoyesrdquo answer)columns of the inverse of the matrix W for each subject and then averaged over all subjects

4ndash8 Hz 0ndash1200 ms

eta power

4

ndash4

8ndash13 Hz 0ndash1200 ms

Alpha power

t val

ue

(a)

0 600400200 1200800Time (ms)

10

15

20

25

30

Freq

uenc

y (H

z)

1000

Right frontal

5

Nondashyes

p lt 005

0 600400200 1200800Time (ms)

ndash5

5

t val

ue10

15

20

25

30

Freq

uenc

y (H

z)

1000

Nondashyes

Right parietal p lt 005

5

(b)

Figure 6 Statistical comparisons in spectral power between ldquoyesrdquo and ldquonordquo (a) Topographical distributions of t values (left theta band(4ndash8Hz) 0ndash1200ms right alpha band (8ndash13Hz) 0ndash1200ms) Black dots (FC2 FC6 and C4 in the left panel and CP2 Pz and P4 in the rightpanel) denote high t values above a predetermined threshold corresponding to the highest 10 (b) e clusters in the time-frequencydomain showing signicant dicenterences between ldquoyesrdquo and ldquonordquo in right frontal (left panel) and right parietal regions (right panel) (denotedby black contours)

8 Computational Intelligence and Neuroscience

regulation of the brain signals or high cognitive efforts Nounpleasant stimuli and volition or high cognitive efforts arerequired since our approach is based on a direct decodingof ldquoyesrdquo and ldquonordquo without any self-regulation of the brainsignals Birbaumerrsquos group has suggested a new alternativeapproach based on classical conditioning to solve theproblem of conventional BCI in the CLIS patients [16ndash18]For the training two distinct unconditioned stimuli arepresented to the subjects immediately after the simple ldquoyesnordquo questions (corresponding to the conditioned stimuli)so that the cortical responses can be conditioned differentlyfor yes and no -e unconditioned stimuli include auditorypink noise and white noise [16 18] and weak electricalstimulation to the thumb [17] -e main idea of this ap-proach is to modulate the usersrsquo brain activities indirectlythrough the unconditioned stimuli so that ldquoyesrdquo and ldquonordquocan be easily discriminated from neural signals respondingto the sensory stimuli rather than to read the usersrsquo an-swers from neural signals -is approach may provide analternative to the conventional BCI approaches in thatvolition or high cognitive efforts are not required How-ever it remains unclear how long the conditioned corticalresponse can be maintained considering the extinctioneffect of classical conditioning [33] Moreover un-conditioned stimuli such as auditory noise or electricalstimulation can evoke significant displeasure

Recently a more natural approach for the ldquoyesnordquodecoding was demonstrated based on functional near-in-frared spectroscopy (fNIRS) in the CLIS patients [34] -eyachieved ldquoyesnordquo decoding accuracy over 70 based onfNIRS signals which were recorded while the patientsanswered ldquoyesrdquo or ldquonordquo to personal and open questions inminds repeatedly Interestingly for the same experimentalprotocol they reported that EEG-based decoding yieldedaccuracy below the chance level -is study employed anatural questionanswer task which does not require highcognitive efforts or volition just as ours But due to the slownature of hemodynamics the duration of each trial for thedecoding was quite long (gt10 sec) Here we showed thepossibility of ldquoyesnordquo decoding from considerably shortersignal recording which is more beneficial for a practicalBCI communication tool

We took a systematic approach of finding features ofbrain activities reflecting ldquoyesnordquo answers in minds andthen developing the decoding algorithm by utilizing thesefeatures Further studies may be necessary to investigatewhether the patients who would potentially benefit fromthe BCI can hold the intentions to answer in minds for ashort time and to validate our method on the patientsrsquo data

In this study the intentions regarding self-referentialquestions based on the autobiographic facts were in-vestigated It is important to further try decoding the in-tentions to answer various types of questions includingdesire feeling and preference In addition our questionswere presented only in visual stimuli Neurological patientsmay have an abnormal visual function such as disability tofix their gaze on specific visual stimuli [35] Different sensorymodality such as auditory stimuli has been tried for the BCIcommunication tools [34 36] It would be beneficial if our

approach can be validated with auditory stimuli such asvoice considering that a high decoding accuracy above 80was obtained even when the brain activities during theperiod of retaining the decision in minds (10ndash12Hz1000ndash1200ms) used for the decoding-us we expect that itis possible to decode the ldquoyesrdquo and ldquonordquo intentions in asimilar way even if other types of questions andor theauditory stimuli are employed in the further studies Inaddition here we did not try to optimize the detailed pa-rameters of the SVM including the RBF kernel parameterand regularization penalty -e use of the best parameters ofthe SVM for example by using the ldquogrid-searchrdquo method[37] may be obviously helpful for better results

We found two time-frequency regions containing usefulinformation for the ldquoyesnordquo decoding in early theta and latealpha bands-e useful features for the ldquoyesnordquo decoding inthe alpha band were found to be concentrated in the parietalregion at 800ndash1200ms from the CSP algorithm Recently weshowed that the alpha rhythms in the right parietal regionare differentiated between the intentions to answers eitherldquoyesrdquo or ldquonordquo in minds presumably due to the difference incognitive loads for the WM retention [2] Several previousstudies showed that the higher parietal alpha power reflectsincreased memory load [38 39] or attentional demand[40 41] during WM retention -e higher alpha power isattributed to active inhibitory control to block incomingstimuli during WM retention for efficient cortical in-formation processing [38 39 42 43] Our results showedhigher parietal alpha power for ldquonordquo compared to ldquoyesrdquowhich may imply higher cognitive load during retainingldquonordquo in minds compared to ldquoyesrdquo [2] -e greater increase inalpha-band activity for ldquonordquo may reflect the increased WMload during the intention retention In Korean languageldquoyesrdquo is the one-character word ldquo네rdquo and ldquonordquo is three-character word ldquo아니오rdquo It is plausible that the higherWMload is required to represent intention to respond ldquonordquo thanldquoyesrdquo due to the length of the Korean words resulting in thehigher alpha rhythm -is assumption is supported by anERP study which reported that greater alpha-band powerwas induced for retaining longer word [44]

It can also be interpreted that the significantly higheralpha-band activity in the centroparietal region for ldquonordquo isdue to the higher attentional demand [40 45] and thiscontributed to the high decoding accuracy -is is also inagreement with a recent study [46] which reported that ahigher alpha rhythm was identified in the right parietalcortex for a higher internal attention condition during adivergent thinking task Our result of greater alpha powerfor ldquonordquo than for ldquoyesrdquo may imply a stronger inhibition of theouter stimuli by the bottom-up attention network for ldquonordquoinduced by higher internal attentional demand -is issupported by psychophysical which showed that saying ldquonordquorequires more effortful reconsideration after comprehendinga sentence and a longer response time for saying ldquonordquo thanldquoyesrdquo [47 48]

-e theta-band activity in the frontal region in 200ndash500ms was another major feature for ldquoyesnordquo decoding-e theta ERS showed topography focused on midlinefrontal and lateral temporal regions -e difference

Computational Intelligence and Neuroscience 9

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

electrodes (FC2 FC6 and C4) showing high t values above apredetermined threshold (tgt 162 corresponding to thehighest 10) were selected over the right frontal region(denoted by black dots in the left panel in Figure 6(a))Significant ldquoyesnordquo difference was found within a singletime-frequency range around 200ndash800ms in the upper thetaand lower alpha bands (6ndash10Hz) which was stronger forldquonordquo compared to ldquoyesrdquo (denoted by a solid contour in theleft panel in Figure 6(b))

In the alpha band 3 electrodes in right parietal area(CP2 Pz and P4) with high t values were selected as de-scribed above (tgt 152 the highest 10) as denoted by blackdots in the right panel in Figure 6(a)-e ldquoyesnordquo differencein spectral power in this region was significant within asingle time-frequency range (300ndash1200ms 9ndash12Hz) wherethe alpha-band power was stronger for ldquonordquo compared toldquoyesrdquo (denoted by a solid contour in the right panel inFigure 6(b))

Table 1 Decoding accuracies for individual subjects

Subject TF subwindow ① TF subwindow ② TF subwindow ③ Combined features

1 8689plusmn 652(7895 100)

8895plusmn 435(8000 9500)

8737plusmn 947(7000 100)

9450plusmn 350(9000 100)

2 7100plusmn 663(5500 8000)

7450plusmn 1011(5500 9000)

7800plusmn 980(6500 9500)

8000plusmn 1049(6000 9500)

3 7600plusmn 1020(6000 9000)

7900plusmn 663(6500 9000)

8200plusmn 714(6500 9000)

8550plusmn 789(6500 9500)

4 8050plusmn 879(6500 9500)

7550plusmn 723(6500 9000)

7100plusmn 800(6000 8500)

8400plusmn 374(8000 9000)

5 8800plusmn 332(8500 9500)

8900plusmn 1044(6500 100)

9200plusmn 748(7500 100)

9600plusmn 490(8500 100)

6 9000plusmn 707(7500 100)

9050plusmn 415(8500 9500)

9200plusmn 557(8500 100)

9300plusmn 400(8500 100)

7 7847plusmn 1321(4500 9474)

7745plusmn 1000(6500 100)

7742plusmn 975(6000 9000)

8445plusmn 817(6500 9500)

8 8300plusmn 557(7500 9500)

7550plusmn 757(6500 8500)

8050plusmn 757(6500 9000)

8800plusmn 872(7500 100)

9 7350plusmn 976(5500 8500)

7000plusmn 837(6000 8500)

6800plusmn 640(5500 8000)

7700plusmn 812(6500 9000)

10 7864plusmn 1095(6000 9474)

6854plusmn 860(6000 8947)

7609plusmn 1277(5000 9474)

8273plusmn 625(7000 9000)

11 7654plusmn 758(6316 8500)

7634plusmn 954(6000 8947)

6759plusmn 600(6000 7500)

7954plusmn 625(6667 9000)

12 9295plusmn 337(8947 100)

9400plusmn 735(7500 100)

9597plusmn 375(9000 100)

9797plusmn 248(9474 100)

13 8789plusmn 654(7895 100)

8284plusmn 976(6500 9500)

7442plusmn 1227(5000 9000)

8545plusmn 817(7000 100)

14 6000plusmn 775(4500 7500)

6500plusmn 1025(4500 7500)

6950plusmn 1404(5000 9000)

7000plusmn 1049(5000 8500)

15 6958plusmn 1498(4444 8889)

7953plusmn 1070(5556 9444)

5896plusmn 1375(3333 7778)

7455plusmn 1076(5789 9444)

16 6895plusmn 849(6000 9000)

6451plusmn 680(5000 7500)

6745plusmn 821(5500 8000)

6948plusmn 700(6000 8500)

17 7900plusmn 943(6500 9000)

7700plusmn 714(6500 8500)

8300plusmn 900(7000 9500)

8950plusmn 789(8000 100)

18 8150plusmn 867(6500 100)

8000plusmn 922(6000 9000)

8800plusmn 714(7000 9500)

9350plusmn 550(8000 100)

19 9300plusmn 510(8500 100)

9700plusmn 400(9000 100)

9450plusmn 269(9000 100)

9800plusmn 245(9500 100)

20 9050plusmn 472(8500 100)

8950plusmn 610(8000 100)

8850plusmn 673(7500 100)

9650plusmn 450(8500 100)

21 8700plusmn 748(7000 9500)

8300plusmn 927(6500 100)

8350plusmn 550(7500 9500)

9150plusmn 594(8000 100)

22 7800plusmn 872(6000 9000)

7350plusmn 808(5500 8500)

7000plusmn 866(5500 8500)

7700plusmn 400(7000 8500)

23 9382plusmn 551(8750 100)

8862plusmn 834(6667 100)

8877plusmn 686(7500 100)

9061plusmn 901(7333 100)

Ave 8108plusmn 889(6000 9382)

7999plusmn 899(6451 9700)

7976plusmn 1021(6745 9597)

8603plusmn 869(6948 9800)

Value meanplusmn standard deviation (SD) of decoding accuracy ()-e range of decoding accuracies was in parenthesis Abbreviation TF time frequency Aveaverage values over all subjects

Computational Intelligence and Neuroscience 7

4 Discussion

We showed that it is possible to decode the intentions toanswer ldquoyesrdquo and ldquonordquo with high accuracy from single-trialEEGs e best decoding accuracy averaged over 23 subjectswas as high as 8603 when useful features in multipletime-frequency subwindows were all combined edecoding accuracy was above 70 for most of the subjects

(22 out of 23 subjects) which is considered as a reasonableaccuracy for the binary classication [32] We decoded theldquoyesrdquo and ldquonordquo answers directly from the brain activitiesrepresenting the two dicenterent answers which implies thatthe ldquomind readingrdquo in a true sense is feasible e exper-imental paradigm of our study is based on a natural taskwhich required the subjects to answer self-referentialquestions as in conversation with others without any self-

6ndash8 Hz200ndash400 ms

10ndash12 Hz800ndash1000 ms

10ndash12 Hz 1000ndash1200 ms

1

ndash1

0

Figure 5 Dicenterence between the most important common spatial patterns for ldquonordquo and ldquoyesrdquo answers averaged over all subjects within 3time-frequency subwindows e topography was obtained from the dicenterence between the last (ldquonordquo answer) and rst (ldquoyesrdquo answer)columns of the inverse of the matrix W for each subject and then averaged over all subjects

4ndash8 Hz 0ndash1200 ms

eta power

4

ndash4

8ndash13 Hz 0ndash1200 ms

Alpha power

t val

ue

(a)

0 600400200 1200800Time (ms)

10

15

20

25

30

Freq

uenc

y (H

z)

1000

Right frontal

5

Nondashyes

p lt 005

0 600400200 1200800Time (ms)

ndash5

5

t val

ue10

15

20

25

30

Freq

uenc

y (H

z)

1000

Nondashyes

Right parietal p lt 005

5

(b)

Figure 6 Statistical comparisons in spectral power between ldquoyesrdquo and ldquonordquo (a) Topographical distributions of t values (left theta band(4ndash8Hz) 0ndash1200ms right alpha band (8ndash13Hz) 0ndash1200ms) Black dots (FC2 FC6 and C4 in the left panel and CP2 Pz and P4 in the rightpanel) denote high t values above a predetermined threshold corresponding to the highest 10 (b) e clusters in the time-frequencydomain showing signicant dicenterences between ldquoyesrdquo and ldquonordquo in right frontal (left panel) and right parietal regions (right panel) (denotedby black contours)

8 Computational Intelligence and Neuroscience

regulation of the brain signals or high cognitive efforts Nounpleasant stimuli and volition or high cognitive efforts arerequired since our approach is based on a direct decodingof ldquoyesrdquo and ldquonordquo without any self-regulation of the brainsignals Birbaumerrsquos group has suggested a new alternativeapproach based on classical conditioning to solve theproblem of conventional BCI in the CLIS patients [16ndash18]For the training two distinct unconditioned stimuli arepresented to the subjects immediately after the simple ldquoyesnordquo questions (corresponding to the conditioned stimuli)so that the cortical responses can be conditioned differentlyfor yes and no -e unconditioned stimuli include auditorypink noise and white noise [16 18] and weak electricalstimulation to the thumb [17] -e main idea of this ap-proach is to modulate the usersrsquo brain activities indirectlythrough the unconditioned stimuli so that ldquoyesrdquo and ldquonordquocan be easily discriminated from neural signals respondingto the sensory stimuli rather than to read the usersrsquo an-swers from neural signals -is approach may provide analternative to the conventional BCI approaches in thatvolition or high cognitive efforts are not required How-ever it remains unclear how long the conditioned corticalresponse can be maintained considering the extinctioneffect of classical conditioning [33] Moreover un-conditioned stimuli such as auditory noise or electricalstimulation can evoke significant displeasure

Recently a more natural approach for the ldquoyesnordquodecoding was demonstrated based on functional near-in-frared spectroscopy (fNIRS) in the CLIS patients [34] -eyachieved ldquoyesnordquo decoding accuracy over 70 based onfNIRS signals which were recorded while the patientsanswered ldquoyesrdquo or ldquonordquo to personal and open questions inminds repeatedly Interestingly for the same experimentalprotocol they reported that EEG-based decoding yieldedaccuracy below the chance level -is study employed anatural questionanswer task which does not require highcognitive efforts or volition just as ours But due to the slownature of hemodynamics the duration of each trial for thedecoding was quite long (gt10 sec) Here we showed thepossibility of ldquoyesnordquo decoding from considerably shortersignal recording which is more beneficial for a practicalBCI communication tool

We took a systematic approach of finding features ofbrain activities reflecting ldquoyesnordquo answers in minds andthen developing the decoding algorithm by utilizing thesefeatures Further studies may be necessary to investigatewhether the patients who would potentially benefit fromthe BCI can hold the intentions to answer in minds for ashort time and to validate our method on the patientsrsquo data

In this study the intentions regarding self-referentialquestions based on the autobiographic facts were in-vestigated It is important to further try decoding the in-tentions to answer various types of questions includingdesire feeling and preference In addition our questionswere presented only in visual stimuli Neurological patientsmay have an abnormal visual function such as disability tofix their gaze on specific visual stimuli [35] Different sensorymodality such as auditory stimuli has been tried for the BCIcommunication tools [34 36] It would be beneficial if our

approach can be validated with auditory stimuli such asvoice considering that a high decoding accuracy above 80was obtained even when the brain activities during theperiod of retaining the decision in minds (10ndash12Hz1000ndash1200ms) used for the decoding-us we expect that itis possible to decode the ldquoyesrdquo and ldquonordquo intentions in asimilar way even if other types of questions andor theauditory stimuli are employed in the further studies Inaddition here we did not try to optimize the detailed pa-rameters of the SVM including the RBF kernel parameterand regularization penalty -e use of the best parameters ofthe SVM for example by using the ldquogrid-searchrdquo method[37] may be obviously helpful for better results

We found two time-frequency regions containing usefulinformation for the ldquoyesnordquo decoding in early theta and latealpha bands-e useful features for the ldquoyesnordquo decoding inthe alpha band were found to be concentrated in the parietalregion at 800ndash1200ms from the CSP algorithm Recently weshowed that the alpha rhythms in the right parietal regionare differentiated between the intentions to answers eitherldquoyesrdquo or ldquonordquo in minds presumably due to the difference incognitive loads for the WM retention [2] Several previousstudies showed that the higher parietal alpha power reflectsincreased memory load [38 39] or attentional demand[40 41] during WM retention -e higher alpha power isattributed to active inhibitory control to block incomingstimuli during WM retention for efficient cortical in-formation processing [38 39 42 43] Our results showedhigher parietal alpha power for ldquonordquo compared to ldquoyesrdquowhich may imply higher cognitive load during retainingldquonordquo in minds compared to ldquoyesrdquo [2] -e greater increase inalpha-band activity for ldquonordquo may reflect the increased WMload during the intention retention In Korean languageldquoyesrdquo is the one-character word ldquo네rdquo and ldquonordquo is three-character word ldquo아니오rdquo It is plausible that the higherWMload is required to represent intention to respond ldquonordquo thanldquoyesrdquo due to the length of the Korean words resulting in thehigher alpha rhythm -is assumption is supported by anERP study which reported that greater alpha-band powerwas induced for retaining longer word [44]

It can also be interpreted that the significantly higheralpha-band activity in the centroparietal region for ldquonordquo isdue to the higher attentional demand [40 45] and thiscontributed to the high decoding accuracy -is is also inagreement with a recent study [46] which reported that ahigher alpha rhythm was identified in the right parietalcortex for a higher internal attention condition during adivergent thinking task Our result of greater alpha powerfor ldquonordquo than for ldquoyesrdquo may imply a stronger inhibition of theouter stimuli by the bottom-up attention network for ldquonordquoinduced by higher internal attentional demand -is issupported by psychophysical which showed that saying ldquonordquorequires more effortful reconsideration after comprehendinga sentence and a longer response time for saying ldquonordquo thanldquoyesrdquo [47 48]

-e theta-band activity in the frontal region in 200ndash500ms was another major feature for ldquoyesnordquo decoding-e theta ERS showed topography focused on midlinefrontal and lateral temporal regions -e difference

Computational Intelligence and Neuroscience 9

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

4 Discussion

We showed that it is possible to decode the intentions toanswer ldquoyesrdquo and ldquonordquo with high accuracy from single-trialEEGs e best decoding accuracy averaged over 23 subjectswas as high as 8603 when useful features in multipletime-frequency subwindows were all combined edecoding accuracy was above 70 for most of the subjects

(22 out of 23 subjects) which is considered as a reasonableaccuracy for the binary classication [32] We decoded theldquoyesrdquo and ldquonordquo answers directly from the brain activitiesrepresenting the two dicenterent answers which implies thatthe ldquomind readingrdquo in a true sense is feasible e exper-imental paradigm of our study is based on a natural taskwhich required the subjects to answer self-referentialquestions as in conversation with others without any self-

6ndash8 Hz200ndash400 ms

10ndash12 Hz800ndash1000 ms

10ndash12 Hz 1000ndash1200 ms

1

ndash1

0

Figure 5 Dicenterence between the most important common spatial patterns for ldquonordquo and ldquoyesrdquo answers averaged over all subjects within 3time-frequency subwindows e topography was obtained from the dicenterence between the last (ldquonordquo answer) and rst (ldquoyesrdquo answer)columns of the inverse of the matrix W for each subject and then averaged over all subjects

4ndash8 Hz 0ndash1200 ms

eta power

4

ndash4

8ndash13 Hz 0ndash1200 ms

Alpha power

t val

ue

(a)

0 600400200 1200800Time (ms)

10

15

20

25

30

Freq

uenc

y (H

z)

1000

Right frontal

5

Nondashyes

p lt 005

0 600400200 1200800Time (ms)

ndash5

5

t val

ue10

15

20

25

30

Freq

uenc

y (H

z)

1000

Nondashyes

Right parietal p lt 005

5

(b)

Figure 6 Statistical comparisons in spectral power between ldquoyesrdquo and ldquonordquo (a) Topographical distributions of t values (left theta band(4ndash8Hz) 0ndash1200ms right alpha band (8ndash13Hz) 0ndash1200ms) Black dots (FC2 FC6 and C4 in the left panel and CP2 Pz and P4 in the rightpanel) denote high t values above a predetermined threshold corresponding to the highest 10 (b) e clusters in the time-frequencydomain showing signicant dicenterences between ldquoyesrdquo and ldquonordquo in right frontal (left panel) and right parietal regions (right panel) (denotedby black contours)

8 Computational Intelligence and Neuroscience

regulation of the brain signals or high cognitive efforts Nounpleasant stimuli and volition or high cognitive efforts arerequired since our approach is based on a direct decodingof ldquoyesrdquo and ldquonordquo without any self-regulation of the brainsignals Birbaumerrsquos group has suggested a new alternativeapproach based on classical conditioning to solve theproblem of conventional BCI in the CLIS patients [16ndash18]For the training two distinct unconditioned stimuli arepresented to the subjects immediately after the simple ldquoyesnordquo questions (corresponding to the conditioned stimuli)so that the cortical responses can be conditioned differentlyfor yes and no -e unconditioned stimuli include auditorypink noise and white noise [16 18] and weak electricalstimulation to the thumb [17] -e main idea of this ap-proach is to modulate the usersrsquo brain activities indirectlythrough the unconditioned stimuli so that ldquoyesrdquo and ldquonordquocan be easily discriminated from neural signals respondingto the sensory stimuli rather than to read the usersrsquo an-swers from neural signals -is approach may provide analternative to the conventional BCI approaches in thatvolition or high cognitive efforts are not required How-ever it remains unclear how long the conditioned corticalresponse can be maintained considering the extinctioneffect of classical conditioning [33] Moreover un-conditioned stimuli such as auditory noise or electricalstimulation can evoke significant displeasure

Recently a more natural approach for the ldquoyesnordquodecoding was demonstrated based on functional near-in-frared spectroscopy (fNIRS) in the CLIS patients [34] -eyachieved ldquoyesnordquo decoding accuracy over 70 based onfNIRS signals which were recorded while the patientsanswered ldquoyesrdquo or ldquonordquo to personal and open questions inminds repeatedly Interestingly for the same experimentalprotocol they reported that EEG-based decoding yieldedaccuracy below the chance level -is study employed anatural questionanswer task which does not require highcognitive efforts or volition just as ours But due to the slownature of hemodynamics the duration of each trial for thedecoding was quite long (gt10 sec) Here we showed thepossibility of ldquoyesnordquo decoding from considerably shortersignal recording which is more beneficial for a practicalBCI communication tool

We took a systematic approach of finding features ofbrain activities reflecting ldquoyesnordquo answers in minds andthen developing the decoding algorithm by utilizing thesefeatures Further studies may be necessary to investigatewhether the patients who would potentially benefit fromthe BCI can hold the intentions to answer in minds for ashort time and to validate our method on the patientsrsquo data

In this study the intentions regarding self-referentialquestions based on the autobiographic facts were in-vestigated It is important to further try decoding the in-tentions to answer various types of questions includingdesire feeling and preference In addition our questionswere presented only in visual stimuli Neurological patientsmay have an abnormal visual function such as disability tofix their gaze on specific visual stimuli [35] Different sensorymodality such as auditory stimuli has been tried for the BCIcommunication tools [34 36] It would be beneficial if our

approach can be validated with auditory stimuli such asvoice considering that a high decoding accuracy above 80was obtained even when the brain activities during theperiod of retaining the decision in minds (10ndash12Hz1000ndash1200ms) used for the decoding-us we expect that itis possible to decode the ldquoyesrdquo and ldquonordquo intentions in asimilar way even if other types of questions andor theauditory stimuli are employed in the further studies Inaddition here we did not try to optimize the detailed pa-rameters of the SVM including the RBF kernel parameterand regularization penalty -e use of the best parameters ofthe SVM for example by using the ldquogrid-searchrdquo method[37] may be obviously helpful for better results

We found two time-frequency regions containing usefulinformation for the ldquoyesnordquo decoding in early theta and latealpha bands-e useful features for the ldquoyesnordquo decoding inthe alpha band were found to be concentrated in the parietalregion at 800ndash1200ms from the CSP algorithm Recently weshowed that the alpha rhythms in the right parietal regionare differentiated between the intentions to answers eitherldquoyesrdquo or ldquonordquo in minds presumably due to the difference incognitive loads for the WM retention [2] Several previousstudies showed that the higher parietal alpha power reflectsincreased memory load [38 39] or attentional demand[40 41] during WM retention -e higher alpha power isattributed to active inhibitory control to block incomingstimuli during WM retention for efficient cortical in-formation processing [38 39 42 43] Our results showedhigher parietal alpha power for ldquonordquo compared to ldquoyesrdquowhich may imply higher cognitive load during retainingldquonordquo in minds compared to ldquoyesrdquo [2] -e greater increase inalpha-band activity for ldquonordquo may reflect the increased WMload during the intention retention In Korean languageldquoyesrdquo is the one-character word ldquo네rdquo and ldquonordquo is three-character word ldquo아니오rdquo It is plausible that the higherWMload is required to represent intention to respond ldquonordquo thanldquoyesrdquo due to the length of the Korean words resulting in thehigher alpha rhythm -is assumption is supported by anERP study which reported that greater alpha-band powerwas induced for retaining longer word [44]

It can also be interpreted that the significantly higheralpha-band activity in the centroparietal region for ldquonordquo isdue to the higher attentional demand [40 45] and thiscontributed to the high decoding accuracy -is is also inagreement with a recent study [46] which reported that ahigher alpha rhythm was identified in the right parietalcortex for a higher internal attention condition during adivergent thinking task Our result of greater alpha powerfor ldquonordquo than for ldquoyesrdquo may imply a stronger inhibition of theouter stimuli by the bottom-up attention network for ldquonordquoinduced by higher internal attentional demand -is issupported by psychophysical which showed that saying ldquonordquorequires more effortful reconsideration after comprehendinga sentence and a longer response time for saying ldquonordquo thanldquoyesrdquo [47 48]

-e theta-band activity in the frontal region in 200ndash500ms was another major feature for ldquoyesnordquo decoding-e theta ERS showed topography focused on midlinefrontal and lateral temporal regions -e difference

Computational Intelligence and Neuroscience 9

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

regulation of the brain signals or high cognitive efforts Nounpleasant stimuli and volition or high cognitive efforts arerequired since our approach is based on a direct decodingof ldquoyesrdquo and ldquonordquo without any self-regulation of the brainsignals Birbaumerrsquos group has suggested a new alternativeapproach based on classical conditioning to solve theproblem of conventional BCI in the CLIS patients [16ndash18]For the training two distinct unconditioned stimuli arepresented to the subjects immediately after the simple ldquoyesnordquo questions (corresponding to the conditioned stimuli)so that the cortical responses can be conditioned differentlyfor yes and no -e unconditioned stimuli include auditorypink noise and white noise [16 18] and weak electricalstimulation to the thumb [17] -e main idea of this ap-proach is to modulate the usersrsquo brain activities indirectlythrough the unconditioned stimuli so that ldquoyesrdquo and ldquonordquocan be easily discriminated from neural signals respondingto the sensory stimuli rather than to read the usersrsquo an-swers from neural signals -is approach may provide analternative to the conventional BCI approaches in thatvolition or high cognitive efforts are not required How-ever it remains unclear how long the conditioned corticalresponse can be maintained considering the extinctioneffect of classical conditioning [33] Moreover un-conditioned stimuli such as auditory noise or electricalstimulation can evoke significant displeasure

Recently a more natural approach for the ldquoyesnordquodecoding was demonstrated based on functional near-in-frared spectroscopy (fNIRS) in the CLIS patients [34] -eyachieved ldquoyesnordquo decoding accuracy over 70 based onfNIRS signals which were recorded while the patientsanswered ldquoyesrdquo or ldquonordquo to personal and open questions inminds repeatedly Interestingly for the same experimentalprotocol they reported that EEG-based decoding yieldedaccuracy below the chance level -is study employed anatural questionanswer task which does not require highcognitive efforts or volition just as ours But due to the slownature of hemodynamics the duration of each trial for thedecoding was quite long (gt10 sec) Here we showed thepossibility of ldquoyesnordquo decoding from considerably shortersignal recording which is more beneficial for a practicalBCI communication tool

We took a systematic approach of finding features ofbrain activities reflecting ldquoyesnordquo answers in minds andthen developing the decoding algorithm by utilizing thesefeatures Further studies may be necessary to investigatewhether the patients who would potentially benefit fromthe BCI can hold the intentions to answer in minds for ashort time and to validate our method on the patientsrsquo data

In this study the intentions regarding self-referentialquestions based on the autobiographic facts were in-vestigated It is important to further try decoding the in-tentions to answer various types of questions includingdesire feeling and preference In addition our questionswere presented only in visual stimuli Neurological patientsmay have an abnormal visual function such as disability tofix their gaze on specific visual stimuli [35] Different sensorymodality such as auditory stimuli has been tried for the BCIcommunication tools [34 36] It would be beneficial if our

approach can be validated with auditory stimuli such asvoice considering that a high decoding accuracy above 80was obtained even when the brain activities during theperiod of retaining the decision in minds (10ndash12Hz1000ndash1200ms) used for the decoding-us we expect that itis possible to decode the ldquoyesrdquo and ldquonordquo intentions in asimilar way even if other types of questions andor theauditory stimuli are employed in the further studies Inaddition here we did not try to optimize the detailed pa-rameters of the SVM including the RBF kernel parameterand regularization penalty -e use of the best parameters ofthe SVM for example by using the ldquogrid-searchrdquo method[37] may be obviously helpful for better results

We found two time-frequency regions containing usefulinformation for the ldquoyesnordquo decoding in early theta and latealpha bands-e useful features for the ldquoyesnordquo decoding inthe alpha band were found to be concentrated in the parietalregion at 800ndash1200ms from the CSP algorithm Recently weshowed that the alpha rhythms in the right parietal regionare differentiated between the intentions to answers eitherldquoyesrdquo or ldquonordquo in minds presumably due to the difference incognitive loads for the WM retention [2] Several previousstudies showed that the higher parietal alpha power reflectsincreased memory load [38 39] or attentional demand[40 41] during WM retention -e higher alpha power isattributed to active inhibitory control to block incomingstimuli during WM retention for efficient cortical in-formation processing [38 39 42 43] Our results showedhigher parietal alpha power for ldquonordquo compared to ldquoyesrdquowhich may imply higher cognitive load during retainingldquonordquo in minds compared to ldquoyesrdquo [2] -e greater increase inalpha-band activity for ldquonordquo may reflect the increased WMload during the intention retention In Korean languageldquoyesrdquo is the one-character word ldquo네rdquo and ldquonordquo is three-character word ldquo아니오rdquo It is plausible that the higherWMload is required to represent intention to respond ldquonordquo thanldquoyesrdquo due to the length of the Korean words resulting in thehigher alpha rhythm -is assumption is supported by anERP study which reported that greater alpha-band powerwas induced for retaining longer word [44]

It can also be interpreted that the significantly higheralpha-band activity in the centroparietal region for ldquonordquo isdue to the higher attentional demand [40 45] and thiscontributed to the high decoding accuracy -is is also inagreement with a recent study [46] which reported that ahigher alpha rhythm was identified in the right parietalcortex for a higher internal attention condition during adivergent thinking task Our result of greater alpha powerfor ldquonordquo than for ldquoyesrdquo may imply a stronger inhibition of theouter stimuli by the bottom-up attention network for ldquonordquoinduced by higher internal attentional demand -is issupported by psychophysical which showed that saying ldquonordquorequires more effortful reconsideration after comprehendinga sentence and a longer response time for saying ldquonordquo thanldquoyesrdquo [47 48]

-e theta-band activity in the frontal region in 200ndash500ms was another major feature for ldquoyesnordquo decoding-e theta ERS showed topography focused on midlinefrontal and lateral temporal regions -e difference

Computational Intelligence and Neuroscience 9

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

between ldquoyesrdquo and ldquonordquo was also most prominent in theseregions Hald et al reported that temporal and frontaltheta-band activity in 300ndash800ms was significantly higherin semantically incongruent compared to congruentsentences [49] -is is commensurate with our result inthat ldquonordquo stimuli are incongruent with autobiographicfacts -e increase of theta-band activity for semanticincongruence was interpreted to reflect the general errordetection mechanism which is associated with error-re-lated negativity (ERN) [50] Interestingly Luu and Tuckershowed that frequency domain analysis of the ERN yieldstheta-band activity in the midfrontal region [50] A relatedstudy reported higher theta oscillation for syntactic vio-lation as well [51] We observed that frontal theta power in200ndash500ms contributed to high decoding accuracyConsidering the location and frequency band our resulton the usefulness of frontal theta power in 200ndash500ms canbe interpreted as another evidence suggesting that error-related frontal theta oscillation is a general phenomenonunderlying processing of incoming stimuli containingviolation with internal information

Data Availability

-e data used to support the findings of this study have notbeen made available because some participants of this studydid not agree to distribute their physiological signals

Conflicts of Interest

-e authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

-is research was supported by the Brain Research Programthrough the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (no2017M3C7A1029485)

Supplementary Materials

-e supplementary figures demonstrate the sensitivity andspecificity of ldquoyesnordquo decoding for each time-frequencysubwindow (Supplementary Figure 1) and the differencebetween the most important spatial patterns discriminatingldquoyesrdquo and ldquonordquo for the individual subjects at the three time-frequency subwindows (Supplementary Figure 2) (Supple-mentary Materials)

References

[1] J W Choi K S Cha K Y Jung and K H Kim ldquoGamma-band neural synchrony due to autobiographical fact violationin a self-referential questionrdquo Brain Research vol 1662pp 39ndash45 2017

[2] J W Choi K S Cha and K H Kim ldquoCovert intention toanswer to self-referential questions is represented in alpha-band local and interregional neural synchroniesrdquo Computa-tional Intelligence and Neuroscience vol 2019 Article ID7084186 11 pages 2019

[3] U Chaudhary N Birbaumer and A Ramos-MurguialdayldquoBrain-computer interfaces for communication and re-habilitationrdquo Nature Reviews Neurology vol 12 no 9pp 513ndash525 2016

[4] G Gallegos-Ayala A Furdea K Takano C A Ruf H Florand N Birbaumer ldquoBrain communication in a completelylocked-in patient using bedside near-infrared spectroscopyrdquoNeurology vol 82 no 21 pp 1930ndash1932 2014

[5] N Birbaumer N Ghanayim T Hinterberger et al ldquoAspelling device for the paralysedrdquo Nature vol 398 no 6725pp 297-298 1999

[6] D Beukelman S Fager and A Nordness ldquoCommunicationsupport for people with ALSrdquo Neurology Research In-ternational vol 2011 Article ID 714693 6 pages 2011

[7] N Birbaumer T Elbert A G Canavan and B RockstrohldquoSlow potentials of the cerebral cortex and behaviorrdquo Phys-iological Reviews vol 70 no 1 pp 1ndash41 1990

[8] G Pfurtscheller and A Aranibar ldquoEvaluation of event-relateddesynchronization (ERD) preceding and following voluntaryself-paced movementrdquo Electroencephalography and ClinicalNeurophysiology vol 46 no 2 pp 138ndash146 1979

[9] C Guger G Edlinger W Harkam I G Niedermayer andG Pfurtscheller ldquoHow many people are able to operate aneeg-based brain-computer interface (bci)rdquo IEEE Trans-actions on Neural Systems and Rehabilitation Engineeringvol 11 no 2 pp 145ndash147 2003

[10] C Vidaurre and B Blankertz ldquoTowards a cure for BCI il-literacyrdquo Brain Topography vol 23 no 2 pp 194ndash198 2010

[11] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface(BCI)rdquo Neuroscience Letters vol 462 no 1 pp 94ndash98 2009

[12] F Nijboer E W Sellers J Mellinger et al ldquoA P300-basedbrain-computer interface for people with amyotrophic lateralsclerosisrdquo Clinical Neurophysiology vol 119 no 8pp 1909ndash1916 2008

[13] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in Neurosciencevol 6 p 169 2012

[14] G R Muller-Putz R Scherer C Brauneis andG Pfurtscheller ldquoSteady-state visual evoked potential(SSVEP)-based communication impact of harmonic fre-quency componentsrdquo Journal of Neural Engineering vol 2no 4 pp 123ndash130 2005

[15] A Kubler and N Birbaumer ldquoBrain-computer interfaces andcommunication in paralysis extinction of goal directedthinking in completely paralysed patientsrdquo Clinical Neuro-physiology vol 119 no 11 pp 2658ndash2666 2008

[16] A Furdea C A Ruf S Halder et al ldquoA new (semantic)reflexive brain-computer interface in search for a suitableclassifierrdquo Journal of Neuroscience Methods vol 203 no 1pp 233ndash240 2012

[17] D De Massari T Matuz A Furdea C A Ruf S Halder andN Birbaumer ldquoBrain-computer interface and semanticclassical conditioning of communication in paralysisrdquo Bi-ological Psychology vol 92 no 2 pp 267ndash274 2013

[18] C A Ruf D De Massari A Furdea et al ldquoSemantic classicalconditioning and brain-computer interface control encodingof affirmative and negative thinkingrdquo Frontiers in Neurosci-ence vol 7 p 23 2013

[19] A Delorme and S Makeig ldquoEEGLAB an open source toolboxfor analysis of single-trial EEG dynamics including in-dependent component analysisrdquo Journal of NeuroscienceMethods vol 134 no 1 pp 9ndash21 2004

10 Computational Intelligence and Neuroscience

[20] T-P Jung S Makeig C Humphries et al ldquoRemovingelectroencephalographic artifacts by blind source separationrdquoPsychophysiology vol 37 no 2 pp 163ndash178 2000

[21] G Huang G Liu J Meng D Zhang and X Zhu ldquoModelbased generalization analysis of common spatial pattern inbrain computer interfacesrdquo Cognitive Neurodynamics vol 4no 3 pp 217ndash223 2010

[22] H Ramoser J G Muller-Gerking and G PfurtschellerldquoOptimal spatial filtering of single trial EEG during imaginedhand movementrdquo IEEE Transactions on Rehabilitation En-gineering vol 8 no 4 pp 441ndash446 2000

[23] C Tallon-Baudry O Bertrand C J Delpuech and J PernierldquoStimulus specificity of phase-locked and non-phase-locked40 Hz visual responses in humanrdquo Journal of Neurosciencevol 16 no 13 pp 4240ndash4249 1996

[24] C S Herrmann M H J Munk and A K Engel ldquoCognitivefunctions of gamma-band activity memory match and uti-lizationrdquo Trends in Cognitive Sciences vol 8 no 8 pp 347ndash355 2004

[25] E Maris and R Oostenveld ldquoNonparametric statistical testingof EEG- and MEG-datardquo Journal of Neuroscience Methodsvol 164 no 1 pp 177ndash190 2007

[26] B Blankertz R Tomioka S Lemm M Kawanabe andK-R Muller ldquoOptimizing spatial filters for robust EEGsingle-trial analysisrdquo IEEE Signal ProcessingMagazine vol 25no 1 pp 41ndash56 2008

[27] J R Sato A Fujita C E -omaz et al ldquoEvaluating SVM andMLDA in the extraction of discriminant regions for mentalstate predictionrdquo Neuroimage vol 46 no 1 pp 105ndash1142009

[28] F Lotte L Bougrain A Cichocki et al ldquoA review of clas-sification algorithms for EEG-based brain-computer in-terfaces a 10 year updaterdquo Journal of Neural Engineeringvol 15 no 3 article 031005 2018

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 pp 1ndash43 1998

[30] V N Vapnik ldquoAn overview of statistical learning theoryrdquoIEEE Transactions on Neural Networks vol 10 no 5pp 988ndash999 1999

[31] G R Muller-Putz R Scherer C Brunner R Leeb andG Pfurtscheller ldquoBetter than random A closer look on BCIresultsrdquo International Journal of Bioelectromagnetism vol 10pp 52ndash5 2008

[32] J Perelmouter and N Birbaumer ldquoA binary spelling interfacewith random errorsrdquo IEEE Transactions on RehabilitationEngineering vol 8 no 2 pp 227ndash232 2000

[33] S J Shettleworth Cognition Evolution and behavior OxfordUniversity Press Oxford UK 1998

[34] U Chaudhary B Xia S Silvoni L G Cohen andN Birbaumer ldquoBrain-computer interface-based communi-cation in the completely locked-in staterdquo PLos Biology vol 15no 1 article e1002593 2017

[35] R Sharma S Hicks C M Berna C Kennard K Talbot andM R Turner ldquoOculomotor dysfunction in amyotrophiclateral sclerosisrdquo Archives of Neurology vol 68 no 7pp 857ndash861 2011

[36] N J Hill E Ricci S Haider et al ldquoA practical intuitive brain-computer interface for communicating ldquoyesrdquo or ldquonordquo by lis-teningrdquo Journal of Neural Engineering vol 11 no 3 article035003 2014

[37] C W Hsu C C Chang Lin et al ldquoA practical guide tosupport vector classificationrdquo Technical Report pp 1ndash12

Department of Computer Science and Information Engi-neering University of National Taiwan Taipei Taiwan 2003

[38] O Jensen J Gelfand J Kounios and E Lisman John ldquoOs-cillations in the alpha band (9ndash12 Hz) increase with memoryload during retention in a short-term memory taskrdquo CerebralCortex vol 12 no 8 pp 877ndash882 2002

[39] W Klimesch P Sauseng and S Hanslmayr ldquoEEG alphaoscillations the inhibition-timing hypothesisrdquo Brain ResearchReviews vol 53 no 1 pp 63ndash88 2007

[40] N R Cooper R J Croft S J J Dominey A P Burgess andJ H Gruzelier ldquoParadox lost exploring the role of alphaoscillations during externally vs internally directed attentionand the implications for idling and inhibition hypothesesrdquoInternational Journal of Psychophysiology vol 47 no 1pp 65ndash74 2003

[41] M Benedek R J Schickel E Jauk A Fink andA C Neubauer ldquoAlpha power increases in right parietalcortex reflects focused internal attentionrdquo Neuropsychologiavol 56 pp 393ndash400 2014

[42] C S Herrmann D Senkowski and S Rottger ldquoPhase-lockingand amplitude modulations of EEG alpha two measuresreflect different cognitive processes in a working memorytaskrdquo Experimental Psychology vol 51 no 4 pp 311ndash3182004

[43] P Sauseng W Klimesch M Doppelmayr T PecherstorferR Freunberger and S Hanslmayr ldquoEEG alpha synchroni-zation and functional coupling during top-down processingin a working memory taskrdquo Human Brain Mapping vol 26no 2 pp 148ndash155 2005

[44] L Michels M Moazami-Goudarzi D Jeanmonod andJ Sarnthein ldquoEEG alpha distinguishes between cuneal andprecuneal activation in working memoryrdquo Neuroimagevol 40 no 3 pp 1296ndash1310 2008

[45] W Ray and H Cole ldquoEEG alpha activity reflects attentionaldemands and beta activity reflects emotional and cognitiveprocessesrdquo Science vol 228 no 4700 pp 750ndash752 1985

[46] M Benedek E Jauk A Fink et al ldquoTo create or to recallNeural mechanisms underlying the generation of creative newideasrdquo NeuroImage vol 88 pp 125ndash133 2014

[47] E S Knowles and C A Condon ldquoWhy people say ldquoyesrdquo adual-process theory of acquiescencerdquo Journal of Personalityand Social Psychology vol 77 no 2 pp 379ndash386 1999

[48] M R Sheridan and K A Flowers ldquoReaction times and de-ception - the lying constantrdquo International Journal of Psy-chological Studies vol 2 no 2 pp 41ndash51 2010

[49] L A Hald M C M Bastiaansen and P Hagoort ldquoEEG thetaand gamma responses to semantic violations in online sen-tence processingrdquo Brain and Language vol 96 no 1pp 90ndash105 2006

[50] P Luu and D M Tucker ldquoRegulating action alternatingactivation of midline frontal and motor cortical networksrdquoClinical Neurophysiology vol 112 no 7 pp 1295ndash1306 2001

[51] M C M Bastiaansen J J A van Berkum and P HagoortldquoSyntactic processing modulates the θ rhythm of the humanEEGrdquo Neuroimage vol 17 no 3 pp 1479ndash1492 2002

Computational Intelligence and Neuroscience 11

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom