1

1

BarcelonaMataró

Signal processing GroupEUPMt Tecnocampus

2

Team

Director:– Prof. Marcos Faundez-Zanuy

Members:– Dr. Josep Roure-Alcobé– Dr. Enric Sesa-Nogueras– Dr. Joan Fabregas– Dr. Antonio Satue-Villar– Dra. Virginia Espinosa– Dr. Xavier Font-Aragones– M.Sc. Carles Paul– M.Sc. Andreu Comajuncosas

External members:– Juan Manuel Pascual Gaspar, Carlos Vivaracho (UVA)– Jordi Sole (UVIC)

2

3

Research lines

1. On-line signature recognition2. On-line handwritten biometric

recognition3. thermal imaging4. Face recognition5. Hand biometrics6. Biometrics applied to health7. Watermarking8. Databases9. Gender recognition10. Emotion recognition

4

Research lines

On-line signature recognition1. M. Faundez-Zanuy “On-line signature recognition based on VQ-DTW”. Pattern

Recognition Vol. 40 (2007) pp.981-992. Elsevier.2. C. Vivaracho, M. Faundez-Zanuy and J. M. Gaspar “An efficient low cost approach for On-

Line signature recognition based on length normalization and fractional distances” Pattern recognition Vol. 42 (2009), Issue 1, pp. 183-193. Elsevier

3. J. Fabregas and M. Faundez-Zanuy “On-line signature verification system with failure to enrol management” Pattern Recognition, Volume 42, Issue 9, pp. 2117-2126 Elsevier

4. M. Faundez-Zanuy and J. M. Pascual-Gaspar “Efficient On-line signature recognition based on Multi-section VQ” Pattern Analysis and Applications. Volume 14, Number 1 pp. 37-45, February 2011.

5. J. M. Pascual Gaspar, M. Faundez-Zanuy, C. Vivaracho “Fast On-line signature recognition based on VQ with time modelling”. Engineering Applications of Artificial Intelligence. Elsevier. Vol. 24 (2011) 368–377, March 2011

6. N. Houmani, A. Mayoue, S. Garcia-Salicetti, B. Dorizzi, M.I. Khalil, M. Mostafa, H. Abbas, Z.T. Kardkovàcs, D. Muramatsu, B. Yanikoglu, A. Kholmatov, M. Martinez-Diaz, J. Fierrez, J. Ortega-Garcia, J. Roure Alcobé, J. Fabregas, M. Faundez-Zany, J. M. Pascual-Gaspar, V. Cardeñoso-Payo, C. Vivaracho-Pascual “BioSecure Signature Evaluation Campaign (BSEC’2009): Evaluating Online Signature Algorithms Depending on the Quality of Signatures”. Pattern Recognition. Elsevier. March 2012, Vol 45(2012), pp.993–1003

3

5

Research lines

On-line handwritten biometric recognition1. E. Sesa Nogueras and M. Faundez Zanuy “Biometric recognition using online

uppercase handwritten text” Pattern Recognition Vol. 45 (2012) pp 128–144. 2. E. Sesa Nogueras, M. Faundez-Zanuy & J. Mekyska “An information analysis of in-

air and on-surface trajectories in online handwriting” Cognitive Computation. Cognitive Computation. Volume 4, Number 2 (2012), pp. 195-205

3. E. Sesa Nogueras, M. Faundez-Zanuy “Writer recognition enhancement by means of synthetically generated handwritten text”. Engineering applications of artificial intelligence. Volume 26, Issue 1, January 2013, Pages 609–624

6

Research lines

Thermal imaging1. Marcos Faundez-Zanuy, Jiri Mekyska and Virginia Espinosa “On the focusing of

thermal images” Pattern Recognition letters. Elsevier. Vol. 32 (2011) pp. 1548–1557, August 2011.

2. Radek Benes, Pavel Dvorak, Marcos Faundez Zanuy, Virginia Espinosa-Duro, JiriMekyska “Multi-focus thermal image fusión”, Pattern Recognition Letters 34 Issue 5, 1 April (2013) 536–544.

4

Research lines

Face recognition1. Marcos Faúndez-Zanuy, Josep Roure-Alcobé, Virginia Espinosa-Duró, Juan Antonio

Ortega “An efficient face verification method in a transformed domain” Pattern recognition letters. Vol.28/7 May 2007 pp.854-858 Elsevier.

2. Joan Fabregas and Marcos Faundez-Zanuy “Biometric recognition performing in a bioinspired system” Cognitive Computation. Springer. pp.257-267. Vol.1 issue 3 Septiembre 2009

3. Virginia Espinosa, Marcos Faundez-Zanuy, Jiri Mekyskya and Enric Monte, “A criterion for analysis of diferent sensor combinations with an application to face biometrics” Cognitive Computation. Volume 2, Issue 3 (2010), Page 135-141

4. Virginia Espinosa, Marcos Faundez-Zanuy and Jiri Mekyska “Beyond cognitive signals” Cognitive Computation. Springer. Vol. 3 pp.374–381, June 2011

7

8

Research lines

Hand biometrics1. Marcos Faundez-Zanuy, David Elizondo, Miguel Angel Ferrer-Ballester and Carlos

M. Travieso-González “Authentication of Individuals using Hand Geometry Biometrics: a Neural Network Approach”. Neural Processing letters. volume 26, issue 3, December 2007, pp 201-216 Springer.

2. Joan Fabregas y Marcos Faúndez-Zanuy “Biometric dispersión Matcher”. Pattern Recognition. Elsevier. Pattern Recognition Vol. 41 (2008), Issue 11, pp. 3412-3426. Elsevier.

3. Joan Fabregas and Marcos Faundez-Zanuy “Biometric Dispersion Matcher versus LDA” Pattern Recognition, Volume 42, Issue 9, pp. 1816-1823

4. Xavier Font-Aragones, Marcos Faúndez Zanuy, Jiri Mekyska “Thermal hand image segmentation for biometric recognition”. IEEE Aerospace and Electronic Systems Magazine. June 2013

5

9

Research lines

1. Biometrics applied to health1. M- Faundez-Zanuy, Amir Hussain, Jiri Mekyska, Enric Sesa-Nogueras, Enric Monte-

Moreno, Anna Esposito, Mohamed Chetouani, Josep Garre-Olmo, Andrew Abel, Zdenek Smekal, Karmele Lopez-de-Ipiña “Biometric Applications Related to Human Beings: There Is Life beyond Security”. Cognitive Computation. Vol. 5, pp. 136-151

2. Karmele López-de-Ipiña, Jesus-Bernardino Alonso, Carlos-Manuel Travieso, JordiSole-Casals, Harkaitz Egiraun, Aitzol Ezeiza, Nora Barroso, Marcos Faundez-Zanuy, Miriam Ecay-Torres, Pablo Martinez-Lage, Unai Martinez-de-Lizardui “On the selection of non-invasive methods based on speech analysis oriented to automatic Alzheimer Disease diagnosis”. Sensors 2013, 13(5), 6730-6745

3. Diana Gallego, Jan Mikulka, Ondrej Smirg, Francisco Espin, Victor Gil, Marcos Faundez-Zanuy, Marcel Jimenez and Pere Clavé “In vitro motor patterns and electrophysiological changes in patients with colonic diverticular disesase”. IntJournal Colorectal Disease. 2013 May 24. PMID: 23702821

Security – health implications

10

6

Privacy is not only about your identity.It’s also about your health status

11

…

Signals

Bio

met

ric

appl

icat

ions

writespeech Gait eye face finger hand

…

…

…

Peo

ple

Pla

nts/

ani

mal

s

Security

…

…

Health

Amb. Int.

Behavioral Morphological

12

Research lines

Watermarking1. Marcos Faúndez-Zanuy, Martin Hagmüller, Gernot Kubin “Speaker verification

security improvement by means of speech watermarking”. Speech Communication Vol. 48 pp.1608-1619 issue 12 December 2006

2. Marcos Faúndez-Zanuy, Martin Hagmüller, Gernot Kubin “Speaker identification security improvement by means of speech watermarking”. Pattern Recognition. Vol. 40 (2007), Issue 11, pp. 3027-3034

3. Marcos Faúndez-Zanuy, Jose Juan Lucena-Molina, Martin Hagmüller “Speech watermarking: an approach for authentication of forensic audio digital recordings” Journal of Forensic Sciences. July 2010, Vol. 55 No 4. Pp.1080-1087

7

13

Research lines

Databases1. J. Ortega-Garcia, J. Fierrez, D. Simon, J. Gonzalez, M. Faúndez-Zanuy, V.

Espinosa, A. Satue, I. Hernaez, J.-J. Igarza, C. Vivaracho, D. Escudero, Q.-I. Moro “MCYT Baseline Corpus: A Multimodal Biometric Database” IEE Proceedings -Vision, Image and Signal Processing Volumen 150, pp.395-401, ISSN 1350-245X. December 2003

2. J. Fierrez, J. Galbally, J. Ortega-Garcia, M. R. Freire, F. Alonso-Fernandez, D. Ramos, D. T. Toledano, J. Gonzalez-Rodriguez, J. A. Siguenza, J. Garrido-Salas, E. Anguiano-Rey, G. Gonzalez-de-Rivera, R. Ribalda, M. Faundez-Zanuy, J. A. Ortega, V. Cardeñoso-Payo, A. Viloria, C. E. Vivaracho, Q. I. Moro. J. J. Igarza, J. Sanchez, I. Hernaez, C. Orrite-Uruñuela, F. Martinez-Contreras, J. J. Gracia-Roche “BiosecurID: A Multimodal Biometric Database”. Pattern Analysis and Applications. ISSN 1433-7541 Springer. Vol. 13 Issue 2. Pp. 235-246 , May 2010

3. Virginia Espinosa-Duró, Marcos Faundez-Zanuy, Jiri Mekyska “A new face database simultaneously acquired in visible, near infrared and termal spectrum” Cognitive computation. Pp 119-135, March 2013

14

Research Projects

1. BIOMETRICS FOR HEALTH AND SECURITY (MCINN TEC2012-38630-C04-03) 2013/2015

2. Effect of afferental oropharyngeal pharmacological and electrical stimulation on swallow response and activation of human cortex in stroke patients with oropharyngeal dysphagia (OD). A randomized controlled trial (La Marato TV·3) 2012/2014

3. AZTI: Sistema de Apoyo al Diagnóstico precoz del Alzheimer basado en Técnicas de Detección Inteligente no invasivas Saiotek AZTI S-PR11UN006 (Gobierno Vasco) 2012/2013

8

A proposal forCOST IC-1206

Handwriting: genderrecognition

Male or female?

9

Feature transformation

A transformation function is applied to the biometric information and on-ly the transformed template is stored in the database.

In salting is invertible. Thus, if a hacker knows the key and the trans-formed template, he can recover the original biometric template, and the security isbased on the secrecy of the key or password. This is the unique approach that requiresa secret information (key). This is not necessary in the other categories. The secondgroup is based on noninvertible transformation systems. They apply a one-way func-tion on the template and it is computationally hard to invert a transformed templateeven if the key (transform function) is known.

Y=f(x)

x1 x3 x4 x5 x6 x7 x8 x x2

Short term missionproposal

From Univ degli Studi di Roma La Sapienza, to Mataro (ES)

– Modification of X, Y scale in order to distortthe gender appearance (invertible and non-invertible functions).

– Check the relevance on genderidentification/ user identification

18

12/07/2013

1

Marcos Faundez‐Zanuy, Enric Sesa –Nogueras, Josep Roure‐Alcobé

Escola Universitaria Politecnica de Mataro ‐Tecnocampus

Outline Introduction

Human classification accuracy

Proposed system

12/07/2013

2

Gender classification Introduction

State‐of‐the‐art

Human classification accuracy vs. Proposed system

Future applications

Author(s) and reference 

Year  Trait  Approaches  # of users  BestreportedAccuracies 

Rojas et al. [1]  2011  face  SVMs, Neural Networks, ADABOOST, PCA+LDA, SIFT (BOW, EvidenceRandomTrees, NBNN and VotedNearest‐Neighbor) 

411 frontal imagesfrom gray FERET (304 for training + 107 fortesting) 

EER= 9.7% ± 2.7% 

Ramesha et al. [2] 

2009  face  Posteriori classprobabilityclassifier  40 male + 18 female (¿? For training + ¿¿ fortesting) 

95% 

Mäkinen et al. [3] 

2008  face  Multilayer neural network, SVM, adaboost  450 males + 450 femalesfrom FERET (80% for training + 20% fortesting) 

90% 

Hyun‐Chulet al. [4] 

2006  face  Gaussianprocessclassifiers (variant of Bayesiankernelclassifiers) 

53 males + 50 women, 17 images per person (PF01 database) and 70 males + 56 females (4000 imagesfrom AR database) 

Error > 5% 

Alexandre [5]  2010  face  Shape, texture and plainintensityfeatures gathered at differentscales 

Same as [3] and 487imagesfrom UND database (130 images of eachgenderfor training + 56 female and 171 maleimagesfortesting 

90% 

Tolba [6]  2001  face  LVQ, RBF  171 imagesfrom 13 females and 36 males (training: 69 faceimages (27 imagesfrom 9females and 42 imagesfrom 13 males) + testing: 102 images (28 imagesfrom 13 females + 74 imagesfrom 36 males) 

100% 

Guo [7]  2009  face  Local binarypattern (LBP)and histograms of orientedgradients (HOG) 

YGA database (4000 males + 4000 females)  92.25% 

Shan[8]  2012  face  Local BinaryPatterns + Adaboost, SVM  Labeled Faces in the Wilddatabase, 7,443 faceimages (2,943 females and4,500 males)5‐fold cross‐validation 

94.81% 

Castrillón et al. [9] 

2010  face  PCA, LBP + SVM  5847 heterogeneousfaceimages(3380 correspondingtomale and 2467 tofemale) takenfromInternet and personal archives 

87.5% 

Bekios et al. [10] 

2011  face  SVM, boosting  Severaldatabases, includingsameconditions as [3] 

93.57% 

Duan at al. [11]  2010  face  block‐based color andedgefeatures + Adaboost  469 testing faces (210 male + 259 female)  87.63% 

Fellous [12]  1997  face  Fiducialpoints + discriminantfunctions  109 imagestraining: 26 males + 26 females. Testing: 26 females + 31 males. 

90% 

Lapedriza et al. [13] 

2006  face  Adaboost, Jointboosting  FRGC database3440 controlledimagesand1886clutteredimages.10‐fold crossvalidation test 

96.77% controlled 91.72% uncontrolled 

Li [14]  2010  Face + fingerprint 

DiscriminativeLatentDirichletAllocation  197 females and 201 males. Testing: 50 males + 50 females 

80% fingerprint 92% face 95% combined 

Li [15]  2012  Clothing +Hair +Face 

Local binatrypatterns + SVM  FERET: 227 training + 114 testing BCMI: 821 training + 274 testing 

73% Clothing 80.6% Hair 88.6% Face 95.8%combined 

Zhang et al. [16]  2008  Face + gait  PCA + SVM  32 male+ 28 female. Leave‐one‐out. Onepersonchosen as probe data in turn and alltheotheras gallery data 

90% face 90% gait 90% combined 

Kos et  al. [17]  

2011  Speech  Average MFCC+GMM  36 hours of speech of labeledspeech  91.76%

Nguyen et  al. [18] 

2011  Speech  MFCC+F0+ZCR+E+HNR+SVM‐RBF  54 male 54 female 10‐fold crossvalidation 

100% 

Yingle et  al. [19] 

  Speech  3D features +Backpropagation neural network  20 male 20 female 

85.2% forisolatedwords 90.9%forcontinousspeech 

Ting et  al. [20]    Speech  MFCC+pitch+GMM  20 male 20 female 

96.7% 

12/07/2013

3

Ichino et  al. [21] 

  Speech  MFCC+pitch+Adaboost  40 speakers  98.6% 

[22]    Speech  A combination of acoustic parameters, including MFCC, pitch, formants, harmonic structure 

472 speakers, 32527 utterancesfor training 300 speakers, 20549 utterancesforvalidation 17332 utterancesfortesting  

TBA 

[23]    Speech  A combination of acousticparameters, including HFCC, ACW SVM wasused as a classifier 

TBA  TBA 

Davis et al. [24] 

2004 Gait  Threemode PCA  40 people (20 male + 20 female). Leaveone‐ outcross‐validation 

90% 

Livne et al. [25] 

2012 Gait  Modifiedversion of anAnnealedParticleFilter (APF) 

46 mocapsequences(2 walks/subject), and 86 pose trajectoriesfrom video tracking (2 tracking trials per sequence), 24 test subjects 

93% 

Amayeh et al. [26] 

2008 Hand  region and boundaryfeaturesbasedonZernikemoments and Fourier descriptors + LDA 

20 males + 20 females. Leave‐one‐outcross‐validation.  

98% 

Wang et al. [x] 

2010 Hand  33 features (25 fingerwidthsamples, 2 palmmeasurements, 3 fingerlength ratios), normalizedsize of images, SVM‐RBF 

85 males + 90 females. (125 training + 50 validation) 10 round cross‐validation 

72% 

Font et al. [x2] 

2012 Hand  39 anthropometricfeatures of hand, BiometricDispersionMethod 

104 people (68% male, 32% female), 1040 images (10 forperson), training – 36 male(284  images) +19 female (132 images) 

97.8% 

Liwicki et al. [27] 

2012 Online text  Gaussian mixture models  Training: 40 male + 40 female; validation: 10 male + 10 female; testing: 25 male+ 25 female 

67.57% 

Yuan et al. [28] 

2010 footwear  Histogram of orientedgradient (HOG), PCA + SVM 

100 male + 100 female (50% training + 50% testing) 

85.49% 

Collins et al. [29] 

2009 Full body  HOG, Spatialpyramid bag of words + SVM  600 male + 288 female, 5 cross‐folddivisions 

80.62% 

Zura et al. [30] 

2010 Bodyradiation  Chakrapointsmeasurements  26 (14 male + 12 female)  statisticallysignificantdifferencebetweenmales and femalesoncombinedchakrasradiation 

 

State‐of‐the‐artAuthors Accuracy Online/o

ff‐line

Classification and 

experimental conditions

Population

[6] 73.2% Off‐line Single neural network; 

CEDAR database, cursive 

letters

training set =800, testing 

set=400

[7] 67.06% On‐line GMM, IAM‐OnDB 

database, cursive letters

Training set =100

Testing set=50

[8] 64.25% On‐line GMM, IAM‐OnDB 

database, cursive letters

Training set =100

Testing set=50

Our approach 76% On‐line SOM, BIOSECURID 

database

Training set =

Testing set=

12/07/2013

4

Gender classification:human performance

Success ratemales

Success ratefemales

Success rateaverage

Figure of merit

Range [0,100%] [0,100%] [0,100%] [‐25, 25]

Expert 1 71,62% 61,02% 66.92% 3.72

Expert 2 71,62% 61,02% 66.92% 3.87

Amateur 1 52.70% 84.75% 66.92% 3.50

Amateur 2 67.58% 61.02% 64.66% 4.44

Amateur 3 85.14% 54.24% 71.43% 4.85

Ground truth: score for males =5, for females = ‐5Manual score: [‐5, 5]Figure of merit: Ground truth x manual score

Handwriting: gender recognition Male or female?

12/07/2013

5

Handwriting: gender recognition Male or female?

Handwriting: gender recognition Male or female?

12/07/2013

6

Clasificacion automatica vs manualCursive letters Capital letters

Identification rates Identification rates

classifier FM ρ mean male female FM ρ mean male female

machine 4,04 0,5033 76,00% 86,11% 62,26%

expert 1 4 0,3543 68,80% 72,22% 64,15%

expert 2 4,2 0,3683 68,80% 72,22% 64,15%

amateur 

1 3,48 0,3969 68,00% 52,78% 88,68% 4,12 0,3792 66,40% 63,89% 69,81%

amateur 

2 4,44 0,3100 64,80% 65,28% 64,15% 3,92 0,3316 60,00% 72,22% 43,40%

Amateur 

3 5,28 0,3961 73,60% 84,72% 58,49% 6,12 0,3845 68,80% 77,78% 56,60%

Figure of merit for humanclassification

-25 -20 -15 -10 -5 0 5 10 15 20 250

2040

Expert 1 Score=3.7218

-25 -20 -15 -10 -5 0 5 10 15 20 250

2040

Expert 2 Score=3.8722

-25 -20 -15 -10 -5 0 5 10 15 20 250

2040

Amateur 1 Score=3.4962

-25 -20 -15 -10 -5 0 5 10 15 20 250

2040

Amateur 2 Score=4.4361

-25 -20 -15 -10 -5 0 5 10 15 20 250

2040

Amateur 3 Score=4.8496

12/07/2013

7

-6 -4 -2 0 2 4-5

0

5Expert 1 vs expert 2

Expert 1

Exp

ert 2

-6 -4 -2 0 2 4-5

0

5Expert 1 vs amateur 3

Expert 1

Am

ateu

r 3

COST IC1206 De‐identification for privacy protection in multimedia content  De‐identification in multimedia content can be defined as the 

process of concealing the identities of individuals captured in a given set of data (images, video, audio, text), for the purpose of protecting their privacy. This will provide an effective means for supporting the EU’s Data Protection Directive (95/46/EC), which is concerned with the introduction of appropriate measures for the protection of personal data. The fact that a person can be identified by such features as face, voice, silhouette and gait, indicates the de‐identification process as an interdisciplinary challenge, involving such scientific areas as image processing, speech analysis, video tracking and biometrics. This Action aims to facilitate coordinated interdisciplinary efforts (related to scientific, legal, ethical and societal aspects) in the introduction of person de‐identification and reversible de‐identification in multimedia content by networking relevant European experts and organizations.

12/07/2013

8

Security solutions Standard encryption techniques are not useful for securing biometric templates: While it is possible to decrypt the template and perform matching between the query and decrypted template, such an approach is not secure because it leaves the template exposed during every authentication attempt.

The solutions proposed in the literature can be split into two categories : 

Feature transformation.

Biometric Cryptosystems.

COST2102 Prague october 2008

Feature transformationA transformation function is applied to the biometric information and

only the transformed template is stored in the database.

In salting is invertible. Thus, if a hacker knows the key and thetransformed template, he can recover the original biometric template, and the securityis based on the secrecy of the key or password. This is the unique approach thatrequires a secret information (key). This is not necessary in the other categories. Thesecond group is based on noninvertible transformation systems. They apply a one-way function on the template and it is computationally hard to invert a transformedtemplate even if the key (transform function) is known.

Y=f(x)

x1 x3 x4 x5 x6 x7 x8 x x2

12/07/2013

9

De‐identification proposal for handwritten texts Transformation of X, Y coordinates, probably modifying the gender style.

Reversible de‐identification: invertible function

Non reversible de‐identification: non‐invertible function