cubscenter for unified biometrics and sensors, ny, usa 1 er 2 : an intuitive similarity measure for...

CUBS Center for Unified Biometrics and Sensors, NY, USA

1

ER2: An Intuitive Similarity Measure for On-Line Signature Verification

Hansheng Lei, Srinivas Palla, Venu Govindaraju

CUBS, Center for Unified Biometrics and SensorsUniv. at Buffalo, the State Univ. of New York

Amherst, NY USA 14260{hlei, spalla2, govind}@cse.buffalo.edu


2


1. Introduction- On-line signature verification

2. ER2: Intuitive Similarity Measure3. Experimental Results4. Demo – CUBS signature verification system

5. Conclusion6. References


3

Introduction Handwritten signatures are commonly used for financial

transactions and documents. Verification is usually done by visual inspection. Unlike iris, retina, fingerprint, face, signature does not

require any expensive hardware, thus it is already widely accepted by general public.

Two kinds of signatures: off-line and on-line.

Fig.1 An on-line signature sensor. The X-Y coordinates and Pressure of signing are captured. With more sophisticated devices, Altitude and Azimuth are also recorded.


4

Introduction Ideal Goals of On-line Verification

1. High accuracy (current accuracy is about 97% depending on test datasets)

2. Eliminating fraud. 3. Cheap implementation.4. Substituting PIN or password.

On-line signature verification is attracting increasing interests, academic and industrial.


5

Introduction Challenges

1). Intra-class variation We are unaware of whether an individual’s signature is unique.

The variation of a person’s signature can be large. 2). ForgeryEasier to be forged than other biometric attributes such as fingerprint, iris, etc.3). Very limited signatures for training Usually we can not expect more than 6 genuine signatures for training for each individual. This is unlike handwriting recognition.4). Decide the consistent featuresThere are possibly over 100 features for signature[2], such as Width, Height, Duration, Orientation, X positions, Y positions, Speed, Curvature, Pressure, so on. Which of them are reliable?


6

IntroductionBasic Procedure for Signature Verification

Raw data Preprocessing Make signature invariant to scaling, translation & rotation. Template generation from given signature The generated template include: 1)what kinds of feature are

chosen, 2)the features,3) distance measures, 4) the threshold for decision.

Verification according to the template 1). Preprocess the raw data of the given signature.

2). Extract features and compare distances with the those in the template.

3). Make decision according to the threshold specified in the template.


7

Introduction - Raw data Preprocessing

Invariant to scaling and translation

Suppose Sig=[X Y], both X and Y are sequences. To make it invariant to scaling and translation by mean-standard deviation normalization:

)(X

XXX

)(Y

YYY

Invariant to rotation Method A. Represent sig=[X Y] in polar space. (xi, yi) => (ri,

θi). Method B. Determine the orientation of the mass of

signature and rotate it.


8


Arc-length NormalizationGiven signature is considered as a 2D curve. It is believed that it is necessary to normalize its length and resample the points by equal arc-length.

Smoothing the curveSmoothing is to discard the noises. Basically two choices:1). Gaussian filters. Convolute the curve with a Gaussian mask.2). FFT transform. FFT makes energy concentrated on the first few coefficients. We can extract these coefficients and reversely FFT back to reconstruct the sequences.


9


mean-std norm.

-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5-3

-2

-1

0

1

2

3

4

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-3

-2

-1

0

1

2

3

20 40 60 80 100 120 140 160-170

-165

-160

-155

-150

-145

-140

-135

-130

-125

-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5-3

-2

-1

0

1

2

3

4

Resam

pling

Smoothing


10

Introduction -Template generation

Feature Extraction/SelectionBecause of limited training samples of signatures (say, 6) and no forgeries, features can not be extracted statistically. We think statistics-based methods are quite difficult.

Distance Measures Distance measures are associated with features. For

scalar features, Euclidean norm is a proper measure; for sequential features, Dynamic Time Warping (DTW) is good measure.


11


Features Global features:

#Width, Height, #Duration, #Orientation Local features:

#X-coordinates, #Y-coordinates , #Curvature Dynamic features:

#Velocity, #Acceleration, #Pressure, #Pressure changing Other features:

# Number of segments, #Critical points, etc.


12



13


Coordinate sequences

X, Y , [X,Y ] are the most straightforward features. They are featureless features.

Speed sequences.Speed V, speed of X-coordinate Vx and speed of Y- coordinate Vy can be derived from sequence [X,Y ] directly by subtracting neighboring points. From the speed, acceleration Va can be further derived.


14


Pressure, Altitude, AzimuthPressure is one of the most common dynamicinformation of on-line signature. Some devices can capture additional information, such as Azimuth (the clockwise rotation of cursor about the z-axis) and altitude (the angle upward toward the positive z-axis).


15


Center of Mass, Torque, Curvature-ellipse S1 and S2

The five features were defined by Vishvjit S. Nalwa [6].

Torque measures the area swept by the vector of pen position. S1 and S2 measure the curvature ellipse based on moments. The distance measure used here is cross-correlation (Pearson's r) weighted by the consistency of points.


16


Average, average positive speed on X-axis ,average positive speed on Y-axis, total signing duration. Lee et al. [3] lists two sets of scalar features (over 100 features). These four features have the highest preference in the first set. The distance measure is Euclidean norm.


17


Cos(a), sin(a), Curvature a is the angle between the speed vector and the X-axis. The three features are proposed by Jain et al. [10]. It also proposes coordinate sequence differences.


18


Features (examples)

0 20 40 60 80 100 120 140 160 180-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

0 20 40 60 80 100 120 140 160 180-3

-2

-1

0

1

2

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-3

-2

-1

0

1

2

3

A Signature sample

X coordinates Y coordinates


19


Features (examples)

0 20 40 60 80 100 120 140 160 180-3

-2

-1

0

1

2

3

4

5

Torques S1 of Curvature ellipse

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-3

-2

-1

0

1

2

3

A Signature sample

0 20 40 60 80 100 120 140 160 180-1

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0


20


Feature comparison

0 20 40 60 80 100 120 140 160 180-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

0 20 40 60 80 100 120 140 160 180-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-3

-2

-1

0

1

2

3

X-coordinates (genuine) X-coordinates (forgery)

Genuine sig. Forgery sig.

Only X-coordinates can not distinguish them!


21


We have following experience: 1). One of the most reliable features is the shape of the

signature. Shape is described by the combination of X, Y-coordinates [X,Y]. 2). The second reliable feature is the speed of writing.

To represent shape and speed, each signature is a 3-D sequence: Sigi=[Xi, Yi, Vi], where V is the sequence of speed magnitude. Then we use ER-Squared to match two signatures and return a Confidence of similarity (0%-100%). The details will be given later in section 2..


22

Introduction -Distance measures

Most commonly-used measures Euclidean norm

Weighted Correlation

Where f(l), h(l) are functions of two signatures and w(l) is the consistency function.

Dynamic Time Warping (DTW)Elastic sequence matching. Very good for on-line signatures.

N

i

xxXXD1

22121 )(),(

dllhlwdllflw

dllhlflwr

)()()()(

)()()(2222

2


23


DTW S1

S2

S1

S2

One-One alignment Dynamic alignment

Both Euclidean norm and correlation assume one-one alignment. Easy but brittle!

Elastic alignment is more robust for sequences, at the cost of computational resources.


24


DTW

)}1,(),1,(),,1(min{)(),( 2 jiDjiDjiDxxjiD ji

Current cost Recursive cumulative cost

The calculation of matrix D. The DTW warping path in the matrix D is the path which has minimum average cumulative cost. The unmarked area is the constraint imposed by |i-j|<w (w is the width of the allowed margin).

Subject to optional constrain: |i-j|<w


25

Introduction - some remarks Remarks on some research directions in on-line signature

verification Segmentation?

Signature is an art of drawing, not limited to some kind language. A Segmentation method by Perceptually Important Points was proposed by Jean-Jules Brault et al [7]. Many works have been done to apply segmentation to signature verification. Problems: 1)The consistency of segmentations? 2)If DTW is used as measure, Segmentation is of little necessity, because those Perceptually Important Points can be aligned accurately by DTW.

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2


26

Introduction - some remarks

User-dependent distance threshold? Distance (Euclidean, DTW, etc.) for dissimilarity measure is not intuitive. In real applications, users tends to ask: how similar is the two signatures? Or, what is the confidence that this signature is genuine? It is intuitive to answer: their similarity confidence is 90%! (instead of saying their distance of dissimilarity is 5.8). It is hard to obtain a user-dependent threshold, because of limited genuine samples. Though it is a choice to use the genuine samples from other users as forgeries, it won’t help much on determining the threshold.


27

Introduction - some remarks

Statistics based methods?Again because we can not expect many signature samples, statistics based methods, such as Markov Model, is hard to achieve high performance.

Artificially generate genuine signatures? Using random forgeries or use the signatures from other users? Possible ways.


28


1. Introduction- On-line signature verification √2. ER2: Intuitive Similarity Measure3. Experimental Results4. Demo – CUBS signature verification system



29

ER2: Intuitive Similarity Measure

Similarity measures must satisfy: The similarity of intra-class is very high. (so that

we can accept genuine signature) The similarity of inter-class is very low. (so that

we can reject forgery). An intuitive score range, like 0 - 1.


30


Traditional Linear Regression

-2 0 0 0

-1 5 0 0

-1 0 0 0

-5 0 0

0

5 0 0

1 0 0 0

1 5 0 0

2 0 0 0

0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0-2 0 0 0

-1 0 0 0

0

1 0 0 0

2 0 0 0

3 0 0 0

4 0 0 0

0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0-2 0 0 0

-1 0 0 0

0

1 0 0 0

2 0 0 0

3 0 0 0

4 0 0 0

-2 0 00 -1 5 00 -1 0 00 -5 0 0 0 5 00 1 00 0 1 50 0 2 00 0

-5 0 0

0

5 0 0

1 0 0 0

1 5 0 0

2 0 0 0

2 5 0 0

3 0 0 0

3 5 0 0

4 0 0 0

0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0-5 0 0

0

5 0 0

1 0 0 0

1 5 0 0

2 0 0 0

2 5 0 0

3 0 0 0

3 5 0 0

0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0-5 0 0

0

5 0 0

1 0 0 0

1 5 0 0

2 0 0 0

2 5 0 0

3 0 0 0

3 5 0 0

4 0 0 0

-500 0 500 1000 1500 2000 2500 3000 3500

Similarity: 91%

Similarity: 31%


31


Linear RegressionGiven two sequences X=(x1,x2, …, xn), Y=(y1,y2, …, yn), then the similarity by R2 of X and Y is:

n

ii

n

ii

n

iii

YyXx

YyXxR

1

2

1

2

1

2

2

)()(

)])(([

R2 named R-squared because R2 = (r)2, where r is Pearson’s correlation r.


32

ER2: Intuitive Similarity MeasureExtended Regression Traditional regression handles two 1-dimentional

sequences. We extend it to multi-dimensional sequences as follows:

M

j

n

ijij

M

j

n

ijij

M

j

n

ijijjij

YyXx

YyXx

ER

1 1

2

1 1

2

1 1

2

2

)()(

))])((([

We name it ER2 since is an extension from 1-D to multi-D


33


The intuition of ER2


34

ER2: Intuitive Similarity MeasureRemarks on Linear Regression

Advantages: Invariant to scale and translation; Similarity (Goodness-of-fit) makes sense.Disadvantages: One-one alignment, brittle.

S1

S2

S1

S2

One-One alignment Dynamic alignment


35

ER2: Intuitive Similarity MeasureWe couple ER2 with DTW-based Curve Matching

Dynamic Alignment by DTW. However, we found direct DTW on two signatures is not very robust.

We use Curve Matching, which is to calculate the total cost of changing one curve to fit another curve. The dynamic programming of DTW is used to realize the calculation.


36


DTW-based Curve Matching

Suppose we have two curves C and C’. Curve matching is actually:

))'(),(()',(_ CspeedCspeedDTWCCMatchC Where speed(C)i= Ci+1-Ci.


37


ER2 coupled with Curve Matching

The DTW warping path in the matrix is the path which has minimum average cumulative cost. The unmarked area is the constraint that path is allowed to go.

],...,,[ 221 myyyyY

],...,,[ 321 nxxxxX

),(2 XYERSimilarity

( y2 is matched x2, x3, so we extend it to be two points in Y sequence.)


38

Experimental Results

1. Introduction- On-line signature verification √2. ER2: Intuitive Similarity Measure √3. Experimental Results4. Demo – CUBS signature verification system



39

Experimental Results Signature database The released signature datasets by SVC( First

International Signature Verification Competition). SVC released the signatures of 80 individuals, 20 genuine and 20 skilled forgeries each.

Methods comparisonER2 coupled with Curve Matching

Vs.Curve Matching without ER2


40

Experimental Results Enrollment

Enroll 6 genuine signatures from each individual.

PreprocessingOnly X,Y-coordinates are used. Other information, such as Pressure, Altitude, Azimuth are not used in the experiments. 1) Smooth the raw sequence by Gaussian

filter. 2) Rotate if necessary. 3) Normalizeeach signature by:

)(X

XXX

)(Y

YYY


41


a) FRR and FAR of ER2 (coupled with Curve Matching). b) FRR and FAR of Curve Matching (without ER2). Both using universal threshold.

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 00

1 02 03 04 05 06 07 08 09 0

1 0 0

E E R = 2 0 . 9 %

F R R F A R

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 00

1 02 03 04 05 06 07 08 09 0

1 0 0

E E R = 7 . 2 %

F R R F A R

a ) b )

T h r e sh o ld ( % ) T h r e sh o ld ( % )

Err

or R

ate

(%)

Err

or R

ate

(%)


42


Table 2. EERs with universal or user-dependentthreshold. Skilled forgeries are provided by the dataset, while random forgery means the forgeries are selected from the signatures of different individuals.

*The results of SVC are available at http://www.cs.ust.hk/svc2004/results.html. We are team 14.


43

Experimental Results A project regarding on-line signature

Recently, we have a Multimodal Biometrics project supported by US Army Laboratory. It requires to test signatures from 1000 individuals, each 2 as enrollment and 3 as queries. We collected 330 individuals so far. The preliminary ROC based on ER2 is:


44


1. Introduction -On-line signature verification √2. ER2: Intuitive Similarity Measure √3. Experimental Results √4. Demo – CUBS signature verification system



45


46

Demo –CUBS Sign. System


47

Conclusion

We propose ER2 as a similarity measure for multi-dimensional sequence matching. Signature verification system can use ER2 coupled with curve matching for intuitive similarity output and higher performance as well. The experimental results are encouraging, although we have to notice that further evaluation on large and real databases is necessary.

Our future work will explore the feasibility of ER2 ondynamic features like pressure, speed, etc.


48

References[1] Rejean Plamondon, Guy Lorette. Automatic Signature Verification and Writer

identification-the state of the art. Pattern Recognition, Vol.22, No.2, pp.107-131, 1989.

[2] F. Leclerc and R. Plamondon. Automatic signature verification: the state of the art 1989-1993. International Journal of Pattern Recognition and Artificial Intelligence, 8(3):643-660, 1994.

[3] Luan L. Lee, Toby Berger, Erez Aviczer. Reliable On-line Human Signature Verifications Systems. IEEE trans. On Pattern Analysis and Machine Intelligence, Vol. 18, No.6, June 1996.

[4] R. Plamondon. The Design of On-line Signature Verification System: From Theory to Practice. Int’l J. Pattern Recognition and Artificial Intelligence, vol. 8, no. 3, pp. 795-811, 1994.

[5] Mario E. Munich, Pietro Perona. Visual Identification by Signature Tracking. IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol. 25, No. 2, pp. 200-216, February 2003.


49

References[6] Vishvjit S. Nalwa. Automatic On-line Signature Verification. Proceedings of

the IEEE, Vol. 85, No. 2, pp. 215-239, February 1997.[7] Jean-Jules Brault and Rejean Plamondon. Segmenting Hanwritten Signat

ures at Their Perceptually Important Points. IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol, 15, No. 9, pp. 953-957, September 1993.

[8] Taik H. Rhee, Sung J. Cho, Jin H. Kim. On-line Signature Verification Using Model-Guided Segmentation and Discriminative Feature Selection for Skilled Forgeries. Sixth International Conference on Document Analysis andRecognition (ICDAR '01), September, Seattle, Washington, 2001.

[9] Thomas B. Sebastian, Philip N. Klein, Bejamin B. Kimia. On Aligning Curves. IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol. 25, No. 1, January 2003.


50

References[10] A.K. Jain, Friederike D. Griess and Scott D. Connell. On-line Signature Veri

fication. Pattern Recognition, vol. 35, no. 12, pp. 2963--2972, Dec 2002. [11] K. Huang and H. Yan, “On-Line Signature Verification Based on Dynamic

segmentation and Global and Local Matching,” Optical Eng., vol. 34, no. 12, pp. 3480-3487, 1995.

[12] G. Lorette and R. Plamondon, “Dynamic Approaches to Hand-written Signature Verification,” Computer Processing of Hand-writing, pp. 21-47, 1990.

[13] R. Martens and L. Claesen, “On-Line Signature Verification by Dynamic Time-Warping,” Proc. 13th Int’l Conf. Pattern Recognition, pp. 38-42, 1996.

[14] B. Wirtz, “Stroke-Based Time Warping for Signature Verification,” Proc. Int’l Conf. Document Analysis and Recognition, pp. 179-182, 1995.

cubscenter for unified biometrics and sensors, ny, usa 1 er 2 : an intuitive similarity measure for...

Documents

signature invariant

unified biometrics

given signature

individuals signature

persons signature

mass of signature

signature sensor

ny usa