Non-Iris Occlusions Detection

Michal Haindl Mikulas KrupickaInstitute of Information Theory and Automation of the ASCR

182 08 Prague, Czech Republic

Faculty of Information Technology, Czech Technical University160 00 Prague, Czech Republic

haindl@utia.cas.cz, krupimik@fit.cvut.cz

Abstract

The prerequisite for the accurate iris recognition is todetect all iris occlusions which would otherwise confuse arecognition method and impair its recognition rate. Thispaper presents a fast multispectral eyelid, eyelash, and re-flection detection method based on the underlying three-dimensional spatial probabilistic textural model. The modelfirst adaptively learns its parameters on the flawless iris tex-ture part and subsequently checks for non iris occlusionsusing the recursive prediction analysis. We provide colouriris occlusion detection results that indicate the advantagesof the proposed method and compare it with 97 recent NoisyIris Challenge Evaluation algorithms.

1. IntroductionBiometrics based human identification systems have evergrowing importance in recent trends towards more securemodern information society. Biometrics recognition sys-tems are not only widespread in various security applica-tions such bank access, airport entry points, or criminal ev-idence gathering but also for smart homes or cars control orhandicapped help systems. Various biometric data can beexploited for these applications. It can be human voice, fin-gerprint, eye, face, gait, veins, handwriting and many more.Various biometric data differ in ways how to acquire them,their durability, reliability, safety, and necessary technologyfor their acquisition and evaluation. In this work we fo-cus on iris recognition because of its stability over man’slife, ease of acquisition (can be acquired remotely from dis-tance of up to several meters), and accuracy. The possibilityfor the eye-based human identification was originally sug-gested by [1] and later estimated that the probability of twosimilar iris is 1 in 1072 [24]. For a survey of the iris recog-nition results see [2].

The iris identification is complex task containing several

sub-tasks (see the processing schema on Fig.1) that have tobe solved. The whole process starts with image acquisitionwhich hardly produces ideal noise-free, focused, and homo-geneously illuminated images, thus the corresponding pre-processing steps for data normalization, denoising, or ge-ometric corrections are inevitable before the iris segmen-tation can be performed. The iris segmentation results aretypically coordinates of two circles, inner and outer borderof iris. Additionally, a normalization step has been intro-duced to simplify the subsequent processing steps. Normal-ization is usually done transforming the iris into a fixed sizerectangle. The selected features are then computed from thenormalized rectangle and used in a classifier to recognize acorresponding human.

Unconstrained iris measurements contain numerous oc-clusion defects such as eyelid, eyelash, and reflectionswhich have to be detected in the preprocessing step ofany iris recognition algorithm. Otherwise such occlusionswould confuse the recognition method and impair its recog-nition rate. Although such unconstrained visible wave-length iris image acquisition impose minimal constraintson the iris verification and identification process as wellas on the subject [3], hence allowing wider range of possi-ble iris recognition applications, it obviously requires moredemanding iris processing methods to achieve comparablerecognition rate with the rigid optimal acquisition condi-tions (homogeneous fixed illumination, direct, close, andfixed distance look to the sensor).

1.1. Non-Iris Overlaps Detection

Due to the various iris imperfections which are oftenfound on unconstrained iris measurements, such as reflec-tions, upper and lower eyelids, eyelid shadows, or eye-lashes, iris classifiers performance have insufficient relia-bility. Thus it is necessary to remove such areas from theiris texture prior to the classification process what consti-tutes on of the most challenging problem in the iris recog-

Figure 1. Iris recognition processing pipeline .

nition research [24]. Some detection methods are spe-cialized to single imperfection category only, while others[3, 7, 14, 17, 18, 19, 27, 28, 31] can detect several types ofimperfections.

Methods focused only on reflections are based on (adap-tive) thresholding (eg. [29]). Thresholding is very fast ap-proach with decent results but typical problem are glasses orcontact lenses (with large reflectors). Typical eyelid detec-tors are based on edge detection followed with polynomialfitting (eg. [8, 32]). Chen et al. [4] proposed method ofeyelash detection based on simple thresholding but claimsthat result quality highly depends on chosen image capturedevice. He et al. [13] proposed similar method (applica-ble also on eyelid shadow) with threshold value based on astatistical prediction model. One of the first general imper-fection detection methods was presented by Proenca [27].This method is based on training classifier on manually de-tected irises. The eye representation is based on texturalGLCM [12] features and the detector uses a neural networkclassifier.

The conventional approach for defect detection [5] is tocompute a texture features in a local sub-window and tocompare them with the reference values representing a per-fect pattern. The method [20] preprocesses a gray level tex-tile texture with histogram modification and median filter-ing. The image is subsequently thresholded using the adap-tive filter and finally smoothed with another median filterrun. Another approach for detection of gray level textureddefects using linear FIR filters with optimized energy sepa-ration was proposed in [16]. Similarly the defect detection[30] is based on a set of optimized filters applied to waveletsub-bands and tuned for a defect type. Method [9] usestranslation invariant 2D RI-Spline wavelets for textile sur-face inspection. The gray level texture is removed using thewavelet shrinkage approach and defects are subsequentlydetected by simple thresholding. Contrary to above ap-

proaches the presented method uses the visible wavelengthmultispectral information.

Recent state-of-the-art non-iris occlusions detectorswere mostly competing in the 2008 NICE.I (Noisy IrisChallenge Evaluation) focusing especially on detection ac-curacy. Nearly hundred various methods from 22 countrieswere submitted to this challenge and the best-ranked algo-rithms were published in [23]. The presented method usesthese best-ranked algorithms for comparison. Anyhow, con-trary to our method none of these NICE methods use truemultispectral information. The source images (which arein RGB colour space) are typically either converted to grey-scale before any analytical steps or only one spectrum chan-nel is used. The best method by Tan et al. [31] uses clus-tering for iris localization followed with prediction and cur-vature models for eyelid and eyelash detection. The secondbest method by Sankowski et al.[28] consists of three steps- threshold based reflections detection, iris boundaries de-tection based on modified integro-differential operator, andeyelids detection based on parametric modelling. The thirdmethod by Pedro Almeida [7] is based on an expert sys-tem with set of decision rules which mainly present noveliris boundary detection and upper and lower eyelid arc fit-ting. Finally, the fourth method by Jeong et al. [14] usesK-Means clustering in the co-occurrence histogram for irisboundaries detection followed with upper and lower eyeliddetection based on the fitting model of parabolic integro-differential operator.

The organization of this paper is as follows. First, a con-cise description of the underlying 3D multispectral texturemodel as well as the model selection criterion is given. Thethird section summarizes the core part of the detection algo-rithm followed by the experimental results and conclusionssections.

2. Multispectral Iris Texture Model

We assume that the multispectral iris texture can berepresented by an adaptive 3D causal simultaneous auto-regressive model [10, 11]:

Xr =∑s∈Icr

AsXr−s + εr , (1)

where εr is a white Gaussian noise vector with zero mean,and a constant but unknown covariance matrix Σ and r ={r1, r2}, s = {s1, s2} are multiindices with the row andcolumn indices, respectively. The noise vector is uncorre-lated with data from a causal neighbourhood Icr ,

As1,s2 =

as1,s21,1 , . . . , as1,s21,d

...,. . . ,

...as1,s2d,1 , . . . , as1,s2d,d

(2)

are d×d parameter matrices where d is the number of spec-tral bands. r, r − 1, . . . is a chosen direction of movementon the image lattice (e.g. scanning lines rightward and topto down). This model can be analytically estimated usingnumerically robust recursive statistics hence it is exception-ally well suited for possibly real-time texture defect detec-tion applications. The model adaptivity is introduced usingthe standard exponential forgetting factor technique in theparameter learning part of the algorithm. The model can bealternatively written in the matrix form

Xr = γZr + εr , (3)

where γ = [A1, . . . , Aη] , η = card(Icr) is a d×dη pa-rameter matrix and Zr is a corresponding vector of Xr−s.To evaluate the conditional mean values E{Xr |X(r−1)} ,where X(r−1) is the past process history, the one-step-ahead prediction posterior density p(Xr |X(r−1)) isneeded. If we assume the normal-gamma parameter priorfor parameters in (1) this posterior density has the form ofStudent’s probability density with β(r) − dη + 2 degreesof freedom, where the following notation is used:

β(r) = β(0) + r − 1 , (4)γTr−1 = V −1zz(r−1) Vzx(r−1) , (5)

Vr−1 =

(Vxx(r−1) V Tzx(r−1)Vzx(r−1) Vzz(r−1)

)+ I , (6)

Vuw(r−1) =

r−1∑k=1

UkWTk , (7)

λ(r) = Vx(r) − V Tzx(r) V−1z(r) Vzx(r) , (8)

eyelid (a) reflection (b) eyelash (c)

Figure 2. Detected iris regions containing all three (a,b,c) occlu-sion types.

where β(0) > 1 and U,W denote either X or Z vector,respectively. If β(r − 1) > η then the conditional meanvalue is

E{Xr|X(r−1)} = γr−1Zr (9)

and it can be efficiently computed using the following re-cursion

γTr = γTr−1 +V −1z(r−1) Zr(Xr − γr−1Zr)T

1 + ZTr V−1z(r−1)Zr

. (10)

The selection of an appropriate model support (Icr) is im-portant to obtain good iris representation. If the contex-tual neighbourhood is too small it can not capture all de-tails of the random field iris model. Inclusion of the unnec-essary neighbours on the other hand adds to the computa-tional burden and can potentially degrade the performanceof the model as an additional source of noise. The optimalBayesian decision rule for minimizing the average probabil-ity of decision error chooses the maximum posterior proba-bility model, i.e., a model Mi corresponding to

maxj{p(Mj |X(r−1))}

can be found analytically [10, 11].

3. Defect DetectionThe eye area is found using the integro-differential

Daugman operator [6]. Single multispectral pixels are clas-sified as belonging to the defective (non-iris) area based ontheir corresponding prediction errors. If the prediction erroris larger than the adaptive threshold:

|E{Xr |X(r−1)} −Xr| > (11)

α

l

l∑i=1

∣∣∣E{Xr−i |X(r−i−1)} −Xr−i

∣∣∣ ,then the pixel r is classified as a detected defect pixel.The parameter l in (12) is a process history length of theadaptive threshold and the constant α = 2.7 was foundexperimentally.

The one-step-ahead predictor

E{Xr |X(r−1)} = γs Zr (12)

differs from the corresponding predictor (9) in using pa-rameters γs which were learned only in the flawless tex-ture area (s < r). The small learning flawless texturecutout is found automatically inside reflection-less iris area.The whole algorithm is extremely fast because the adaptivethreshold is updated recursively:

|εr+1| >α

l

[l−1∑i=0

|εr−i|

], (13)

where εr is the prediction error

εr = E{Xr |X(r−1)} −Xr ,

and γs is the parametric matrix which is not changing.Hence the algorithm can be easily applied in real time irisdefect detection.

detected detectedeye defects eye defects

Figure 3. Eye images and the corresponding detected occlusionsmasks (even columns).

4. Experimental ResultsThe presented method was tested on the eye UBIRIS v1

and v2 databases [26] and compared with the best resultsachieved during the Noisy Iris Challenge Evaluation con-test [23]. These databases provide eye images with or with-out different occlusion types (Fig.2), and thus are an usefulresource for the evaluation iris recognition methods. TheUBIRIS.v1 database contains 1877 images collected from241 persons in two distinct sessions. The RGB 800 × 600,24 bit images were captured with the Nikon E5700 cameraand saved in the JPEG format. For the first image capturesession, the enrollment, they tried to minimize noise fac-tors, specially those relative to reflections, luminosity, and

Table 1. Performance criteria (a eyelid, b reflection, c eyelash).

UBIRIS [26] type FP [%] FN [%] TP+TN [%]Img 1 1 1 bc 1.00 10.56 88.44Img 1 2 1 abc 8.13 5.25 86.62Img 2 1 1 abc 8.83 2.81 88.36Img 2 2 1 abc 14.40 3.49 82.11Img 3 1 1 abc 5.04 2.93 92.04Img 3 2 1 abc 3.16 3.60 93.23Img 4 1 1 abc 31.79 2.20 66.01Img 4 2 1 abc 15.51 1.77 82.72Img 5 1 1 abc 3.08 8.22 88.70Img 5 2 1 bc 5.94 1.09 92.97Img 6 1 1 abc 2.83 3.07 94.10Img 6 2 1 abc 12.67 7.39 79.95Img 7 1 1 abc 7.41 4.66 87.93Img 7 2 1 abc 32.75 2.35 64.89Img 8 1 1 ab 0.44 4.36 95.20Img 8 2 1 b 1.52 3.33 95.15Img 9 1 1 abc 11.17 0.91 87.93Img 10 1 1 abc 4.40 1.96 93.64Img 11 1 1 abc 4.05 4.91 91.03Img 11 2 1 abc 6.42 2.24 91.34Img 12 1 1 abc 1.11 8.57 90.32Img 12 2 1 abc 0.81 6.46 92.73Img 13 1 1 abc 11.01 2.46 86.53Img 13 2 1 abc 30.93 1.16 67.91Img 14 1 1 abc 8.41 5.93 85.66Img 15 1 1 abc 0.18 6.08 93.74Img 15 2 1 abc 13.05 3.90 83.05Img 16 1 1 ab 12.85 5.74 81.40Img 16 2 1 bc 13.45 2.37 84.19Img 17 1 1 ab 9.84 3.68 86.48Img 18 1 1 abc 4.46 5.00 90.54Img 18 2 1 abc 24.19 3.70 72.11Img 19 1 1 ab 0.11 2.53 97.36Img 19 2 1 bc 12.81 4.47 82.72Img 20 1 1 abc 9.27 2.48 88.25Img 20 2 1 abc 18.94 0.18 80.87overall perf. 9.78 3,94 86.38

contrast, having installed the framework inside a dark room.In the second session they changed the capture location inorder to introduce natural luminosity factor. This enabledthe appearance of heterogeneous images with respect to re-flections, contrast, luminosity, and focus problems.

Single iris occlusions (Fig.2) were recognized on the se-lected subset of twenty persons the UBIRIS.v1 database(Tab.1) containing one or a combination of several iris oc-clusions (a - eyelid, b - reflection, c - eyelash). Single Tab.1columns contain false positive (FP), false negative (FN),and correct recognition (TP+TN) for single test images. The

ground truth masks for single occlusion types were handmade. All combined results presented (Fig.3 even columns)are presented with basic majority filtering only to demon-strate the basic method’s performance. Fig.3 exhibits reli-able defects detection which is clearly visible on the corre-sponding resulted thematic maps. All defects were detectedusing simple models with a causal neighbourhood contain-ing 5 sites (η = 3). Fig.3 bottom left indicates a type ofhighly defective iris texture. This example illustrates cor-rect detection and localization of the most frequented irisocclusion by the presented method.

The UBIRIS.v2 database [25] contains 11102 imagescollected from 261 persons. The RGB 400 × 300, 24 bitimages were captured with the Canon EOS 5D camera andsaved in the TIFF format.

The presented method was also compared with the topeight results (from 97 participants) [31, 28, 7, 18, 14, 3,17, 19] from the Noisy Iris Challenge Evaluation Contest(NICE.I) [23]. The contest was run on the UBIRIS.v2database which contains highly noisy eye images. The par-ticipants had 500 training images and a disjoint test set of500 images was used to measure the pixel-by-pixel agree-ment between the binary maps made by each participant andthe ground-truth data, manually built by the NICE.I orga-nizers.

Our method ranked third (Tab.2) closely behind the sec-ond method [28]. However, the winning algorithm [31] isvery complex, time consuming and suffers with numerousexperimentally set control parameters. Similarly the secondranked method [28] based on the reflections localization, re-flections filling in, iris boundaries localization and eyelidsboundaries localization steps, relies on several experimen-tally found parameters.

Table 2. Iris defect detection Noisy Iris Challenge Evaluation Con-test [23] top eight results comparison.

Rank Method No. par. Error1 Tan et al. [31] 9 0.01312 Sankowski et al. [28] 6 0.01623 presented method 2 0.01684 Almeida [7] 5 0.01805 Li et al. [18] 4 0.02246 Jeong et al. [14] 3 0.02827 Chen et al. [3] 5 0.02978 Scotti & Labbati [17] 12 0.03019 Luengo-Oroz et al. [19] 7 0.0305

5. ConclusionsThe majority of published iris defect detection methods

do not use any multispectral information, while our methodtakes advantage of exploiting both multispectral as well as

the spatial information simultaneously. The method is sim-ple, extremely fast and robust in comparison with these top-ranking alternative methods. The presented method resultsare promising, most iris occlusions on the UBIRIS iris tex-tures were correctly localized. Our method ranked thirdwhen evaluated on the the Noisy Iris Challenge EvaluationContest from the 97 competing algorithms. The presentedmethod can be easily generalized for gradually changing(e.g., illumination, colour, etc.) iris texture defect detectionby exploiting its adaptive learning capabilities.

AcknowledgmentsThis research was supported by grant GACR 102/08/0593and partially by the grant GACR 103/11/0335.

References[1] F. H. Adler. Physiology of the eye: clinical applica-

tion. The C.V. Mosby Co., St.Louis, 1959.[2] K. W. Bowyer, K. Hollingsworth, and P. J. Flynn. Im-

age understanding for iris biometrics: A survey. Com-puter vision and image understanding, 110(2):281–307, 2008.

[3] Y. Chen, M. Adjouadi, C. Han, J. Wang, A. Bar-reto, N. Rishe, and J. Andrian. A highly accu-rate and computationally efficient approach for uncon-strained iris segmentation. Image and Vision Comput-ing, 28(2):261 – 269, 2010.

[4] Y. Chen, S. C. Dass, and A. K. Jain. Localized irisimage quality using 2-D wavelets. Advances in Bio-metrics, pages 0–6, 2005.

[5] D. Chetverikov. Detecting defects in texture. In In-ternational Conference on Pattern Recognition, pages61–63, 1988.

[6] J. Daugman. High confidence visual recognition ofpersons by a test of statistical independence. Pat-tern Analysis and Machine Intelligence, IEEE Trans-actions on, 15(11):1148 –1161, Nov. 1993.

[7] P. de Almeida. A knowledge-based approach to the irissegmentation problem. Image and Vision Computing,28(2):238 – 245, 2010.

[8] Y. Du, R. Ives, and D. Etter. A new approach to irispattern recognition. Proceedings of SPIE, pages 1–12,2004.

[9] H. Fujiwara, Z. Zhang, H. Toda, and H. Kawabata.Textile surface inspection by using translation invari-ant wavelet transform. In IEEE Int. Symp. on Compu-tational Intelligence in Robotics and Automation, vol-ume 3, pages 1427 – 1432. IEEE, IEEE, 2003.

[10] M. Haindl. Visual data recognition and model-ing based on local markovian models. In L. Flo-rack, R. Duits, G. Jongbloed, M.-C. Lieshout, and

L. Davies, editors, Mathematical Methods for Signaland Image Analysis and Representation, volume 41 ofComputational Imaging and Vision, chapter 14, pages241–259. Springer London, 2012. 10.1007/978-1-4471-2353-8 14.

[11] M. Haindl and S. Simberova. Theory & Applicationsof Image Analysis, chapter A Multispectral ImageLine Reconstruction Method, pages 306–315. WorldScientific Publishing Co., Singapore, 1992.

[12] R. Haralick, K. Shanmugam, and I. Dinstein. Tex-tural features for image classification. Systems, Manand Cybernetics, IEEE Transactions on, 3(6):610–621, 1973.

[13] Z. He, T. Tan, Z. Sun, and X. Qiu. Toward Accurateand Fast Iris Segmentation for Iris Biometrics. Pat-tern Analysis and Machine Intelligence, IEEE Trans-actions on, 31(9):1670–1684, Sept. 2009.

[14] D. S. Jeong, J. W. Hwang, B. J. Kang, K. R. Park, C. S.Won, D.-K. Park, and J. Kim. A new iris segmentationmethod for non-ideal iris images. Image and VisionComputing, 28(2):254 – 260, 2010.

[15] J. V. Kittler, R. Marik, M. Mirmehdi, M. Petrou, andJ. Song. Detection of defects in colour texture sur-faces. In IAPR Workshop on Machine Vision Applica-tions, pages 558–567, 1994.

[16] A. Kumar. Inspection of surface defects using opti-mal fir filters. In ICASSP, volume II, pages 241–244.IEEE, 2003.

[17] R. D. Labati and F. Scotti. Noisy iris segmentationwith boundary regularization and reflections removal.Image and Vision Computing, 28(2):270 – 277, 2010.

[18] P. Li, X. Liu, L. Xiao, and Q. Song. Robust and accu-rate iris segmentation in very noisy iris images. Imageand Vision Computing, 28(2):246 – 253, 2010.

[19] M. A. Luengo-Oroz, E. Faure, and J. Angulo. Robustiris segmentation on uncalibrated noisy images usingmathematical morphology. Image and Vision Comput-ing, 28(2):278 – 284, 2010.

[20] R. Meylani, A. Ertuzun, and A. Ercil. A comparativestudy on the adaptive lattice filter structures in the con-text of texture defect detection. In ICECS ’96, pagesVol II: 976 – 979, 1996.

[21] M. Mirmehdi, R. Marik, M. Petrou, and J. V. Kittler.Iterative morphology for fault detection in stochastictextures. Electronic Letters, 32:443–444, 1996.

[22] P. Mitra, C. Murthy, and S. Pal. Unsupervised fea-ture selection using feature similarity. IEEE Trans-actions on Pattern Analysis and Machine Intelligence,24(3):301–312, March 2002.

[23] H. Proena and L. A. Alexandre. Introduction to thespecial issue on the segmentation of visible wave-length iris images captured at-a-distance and on-the-move. Image and Vision Computing, 28(2):213 – 214,2010.

[24] H. Proenca and L. Alexandre. Iris recognition: Mea-suring feature’s quality for the feature selection in un-constrained image capture environments. In Compu-tational Intelligence for Homeland Security and Per-sonal Safety, Proceedings of the 2006 IEEE Interna-tional Conference on, pages 35–40. IEEE.

[25] H. Proenca, S. Filipe, R. Santos, J. Oliveira, andL. Alexandre. The ubiris.v2: A database of visiblewavelength iris images captured on-the-move and at-a-distance. Pattern Analysis and Machine Intelligence,IEEE Transactions on, 32(8):1529 –1535, 2010.

[26] H. Proenca and L. A. Alexandre. {UBIRIS}: A noisyiris image database. In 13th International Confer-ence on Image Analysis and Processing - ICIAP 2005,volume LNCS 3617, pages 970–977, Cagliari, Italy,2005. Springer.

[27] H. Proenca and L. A. Alexandre. A Method for theIdentification of Noisy Regions in Normalized IrisImages. 18th International Conference on PatternRecognition (ICPR’06), 4:405–408, 2006.

[28] W. Sankowski, K. Grabowski, M. Napieralska, M. Zu-bert, and A. Napieralski. Reliable algorithm for irissegmentation in eye image. Image and Vision Com-puting, 28(2):231 – 237, 2010.

[29] W. Sankowski, K. Grabowski, M. Zubert, andM. Napieralska. Iris Finder–Program For Reliable IrisLocalization In Images Taken Under Visible Light.Journal of Medical Informatics & Technologies Se-lected full texts, 10:125–132, 2006.

[30] J. L. Sobral. Optimised filters for texture defect detec-tion. In International Conference on Image Process-ing, volume III, pages 565–568, 2005.

[31] T. Tan, Z. He, and Z. Sun. Efficient and robust seg-mentation of noisy iris images for non-cooperative irisrecognition. Image and Vision Computing, 28(2):223– 230, 2010.

[32] R. P. Wildes. Iris recognition: an emerging biomet-ric technology. Proceedings of the IEEE, 85(9):1348–1363, 1997.