Correlation Based Image Defect Detection

Toshiyuki AmanoGraduate School of Engineering,Nagoya Institute of Technology

Gokiso, Showa, Nagoya, 466-8555, Japanamano@nitech.ac.jp

Abstract

The defect inspection that used image sensing such asautomated pattern inspection is a useful solution to autom-atize the visual check, not limit to factory automation field.Mostly such defect inspection is using the models of defectthat described by primitive features. This paper proposesa new defect detection method that is the non-model basedapproach. In this approach, the method extracts the imagedescription rule from local regions.It is useful for the de-fect inspection problems that cannot prepare a defect modelsuch as scratch or superimpose detection, texture imageanalysis, etc. In the experiment, I tried the defect detectionto the landscape picture which several types of superim-pose were added. From these results, it was confirmed thatthe proposed method has high ability to detect the defectedregions independently with the texture type.Furthermore, Iattempted the application to a scene image.Therefrom, thepossibility to apply the figure-ground separation of the im-age understanding basic problem was confirmed.

1. Introduction

From the early 1980s, the defect inspection which basedon image measurement was developed to automatize visualcheck of factory laborer in the FA field. The applicationfield of defective inspection includes defective inspectionof the print circuit board, the engine valve [1], the internalcrack of the casting part which a X-rays image was used,appearance of tablet or capsules [2]. This defect inspectionconsists of the pattern matching of edge shape, area dimen-sion, perimeter length, roundness, etc. with image acquisi-tion under the optimal lighting environment, and packagedproducts were launched on the market by various manufac-turers at present. Moreover, it isn’t confined to the FA field,defection inspection by image measurement is applying tothe various fields such as digital film restoration [3], ap-ple quality sorting [4], detection of lung cancer [5]. The

algorithms of these defect inspections are different in ev-ery problem, because these methodologies are dependingon recognition objects. However, these methodologies areable to prepare rules to inspect defects, and it is possiblewith these applications that a defect is given to it as a modelbeforehand. From the perspective of this, in this paperI call the method which is able to give defect model be-forehand is model based defect detection. On the contrary,Kurita et al. [6] proposed neural based occlusion detectionthat is not given a defect model. Serdaroglu et al. [7] pro-posed defect detection method with wavelet transform andICA. These methodologies were given no kind of defectmodel. So these are able to classified non-model based de-fect detection. However, these methodologies use undam-aged patterns or features that learned beforehand. This pa-per proposes a defect detection method that not uses learn-ing sample or prior knowledge. The proposed method as-sumes ”The image has some kind of image description ruleat almost local regions” and assumes ”The defect is a smallnumber of local regions which don’t comply with this rule.”The non-model-based approach is not needed in the defectinspection of the FA field, but it is a useful tool for sceneunderstanding and texture analysis.

2. Defect Detection based on Correlation

2.1. An Image Description Rule

In this paper assumes the image has the autocorrelationproperty because of image fractality. If this assumption isapproved, we can extract the essence of image descriptionfrom finding the principal components [9]. At the first, inorder to extract this essence from an image (W × H), wesampled many image regions by small window (w × h :w << W,h << H) and express local region as imagevector

X = [x1, x2, ..., xR], xi = [ξ1, ξ2, ..., ξN ]T (1)

Input Image

H

W

w

h

Acquired Learning Samples

Window

Figure 1. Local region clipping by window.

e1

e2

ξ1

ξ2

ξN

Image-vector Space

Eigen Space

Learning Sample

Figure 2. The distribution of the learning sam-ples on the eigenspace.

by raster scan of image (Fig 1), where R is the numberof regions sampled by the small window. In the N dimen-sional Euclidean space, the distribution of projection pointsby local region images not becomes random distribution, aslong as the image has some visual meaning. If the lower di-mensional subspace that expressed this cluster efficiently isfound, we can get the rule about projection point of the localarea that is sampled from one image by the window. That isthe local region image projected on this subspace in the Ndimensional Euclidean space. Next, we generate variancematrix from these image vectors and compute eigenvectors

E = [e1, e2, ..., edim] (2)

by the orthogonalization such as KL-expansion. Where D(D << R) is eigenspace dimension. The cumulative pro-portion indicates image reproducibility and also if cumula-tive proportion of the eigenspace is high, we can put theseeigenvectors as a kind of the rule of image description.

2.2. Eigen Space Defect Detector

If the strong correlations caused by image fractality orperiodicity are found in an image, we can express the im-age by the eigen vector combination of the far small numberthan the number of acquired local regions. When we thinkabout this condition geometrically, learning samples are

ξ1

ξ2

ξN

Image-vector Space

Eigen Space

Learning Sample x

Reconstructed Learning Sample x'

p

|∆x|

Figure 3. The distance between an image vec-tor that acquired from the local region andeigen space.

distributed in the neighborhood of the hyper-plane whichpasses the origin that spanned by dim numbers of eigenvectors. Additionally, if a projection point of one local re-gion sample is close to the eigen space in the image vectorspace, we can assume the local region sample obeys the im-age description rule described by the eigen vectors. There-fore, this paper defines

|∆x| = |x − x′| =√

(x − Ep)T (x − Ep)

=√

xT x − pT p

(3)

that is a measure of heterogeneity at the local region andproposed method detects the defect by this measure shownin the figure 3.Where p is the projection point of the lo-cal region image on the eigen space and x′ is the recon-structed image vector by the linear combination of eigenvectors and I call this measure the eigen space defect de-tector (ESDD). Ideally, the eigen space should be generatedby only the learning sample which doesn’t contain a defectfor pure image description rule extraction, but to know thelocation of defect is impossible in advance. Therefore, atproposed method reconstructs eigen space after exclusionof outlier evaluated by ESDD and repeats this reconstruc-tion with each one defect detection like a statistical outlierdetection [8].

3. Experimental Results

3.1 Defect Detection Results for Superim-posed Image.

I attempted the defect detection to superimposed land-scape image shown in figure 4 for confirmation of the be-havior of proposed method. The size of this image is480×359 pixels, and I set the size of window to 48×35pixels. Additionally, captured 841 local regions by the uni-form step of the window from the image in the proposed

Figure 4. Superimposed landscape picture

e1 e2 e3 e4 e5

e6 e7

e24 e25

e26 e27 e28 e29 e30

...

...

Figure 5. Eigen vectors for local regions ofthe land scape picture.

method. The visualized eigen vectors that are shown in fig-ure 5 were computed from captured 841 samples, and thecumulative contribution ratio was 82.2% for the 30 dimen-sional eigen space. Figure 6 shows defect detection resultsthat used the ESDD and detected one outlier by each detec-tion. The calculation time was 55 seconds, and 41 secondsby the first and 120th detection by the Linux based com-puter (Intel Xeon 2.80GHz) because the number of localregion sample is declining with each detection. The eigenspace of 30 dimensions was generated by each detection,and the cumulative contributions were 82.35%，82.49%，82.48%，82.61%. From this detection result, most super-imposed regions are detected at the 120th detection evenif a few landscape regions are detected since 100th detec-tion. At the landscape picture, most area is occupied by thebare rock texture, and the area which a character occupiesis small. Therefore, the local region samples that containscharacters are depart from the major group of bare rock tex-tures and the areas that contain characters were detected. Itis not the purpose of the proposed technology to detect acharacter because what is defect is dependent on each ap-plication. However, it made good detection result for theassumption of ”The defect is a small number of local re-gions which don’t comply with this rule.”

(a) 10th detection (b) 50th detection

(c) 100th detection (d) 120th detection

Figure 6. Defect detection results.

(a) checked pattern (b) complex texture

Figure 7. Superimposed images by texturedcharacters.

3.2 Textured Characters Detection andApplication to the Scene Image

If the superimposed characters are always white, we candetect these characters from the color phase and the conti-nuity of the surrounding pixels. However, we need somealgorithm for individual case that the color of characteris unknown, or it has not solid color and has a complextexture pattern. Contrary to this algorithm, the proposedmethod never uses any knowledge of color or texture pat-tern of the superimposed area. Proposed method assumesonly the existence of image description rule, so we can ex-pect the defect detection of the complex texture is possiblewithout any information of the detection area. Therefore, Iattempted the defect detection of the superimposed charac-ter that has pattern or complex texture shown in figure 7, andI got the detection results for these images shown in figure 8.These have shown results of the 120th detection. Parameterof window size and sampling-steps, eigenspece dimensionswere configured by the same values as the section 3.1. From

(a) checked pattern (b) complex texture

Figure 8. Results of textured characters.

(a) input image (b) 5th detection

(c) 30th detection (d) 100th detection

Figure 9. Application result to the scene im-age.

the result of figure 8(b), we can confirm that the detectionresult of the proposed method does not depend on the tex-ture pattern of the characters, even if the color and feelingof the material of the added texture pattern are very similarto bare rock texture. As a trial of the scene understanding,figure 9 was shown the detection result that attempted to thecomplex scene image. The proposed method was not aimedto detect humans and windows, but these regions were in aminority and the　 definition of defect were suitable to de-tect these regions. In the figure-ground separation problem,it knows the possibility that is recognized with the figure ishigh, at the smaller and closed regions by the gestalt psy-chology. Therefore, it is able to expect that the proposedmethod has an ability to separate the figure and ground.

4 Conclusion

This paper proposed a defect detection method that is thenon-model based. The proposed method assumed the exis-tence of image description rule, and defined the defect asa few areas that not complied to this rule. From the frac-

tality or periodicity, we can expect the correlation at the allaround of local region of the image, and it is a kind of de-scription rule of the image. In the experiment, the robust-ness for character detection of the superimposed image wasconfirmed even if it has complex texture or pattern. Fromthe section 3.2,it was confirmed that the definition of defectof the proposed method has ability to apply to the scene un-derstanding.

References

[1] W.A. Perkins,“Computer vision inspection of valvespring assemblies on engine heads,” General MotorsResearch Laboratories, GMR-3369, 1980.

[2] K. Nakamura, K.Edamatsu and Y.Sano,“Auto-matedpattern inspection based on boundary length compari-son method,” Proc. International Joint Conference onPattern Recognition, pp.955-957, 1978.

[3] Buisson O., Bessere B., Boukir S., HeltF.,“Deterioration detection for digital film restora-tion”, Proc. of Int. Conf. on CVPR, Vol.1, pp.78-84,1997.

[4] D. Unay, B. Gosselin, “An Approach for Recogniz-ing Stem-end/Calyx regions in Apple Quality Sort-ing,” Proc. of ACIVS 2004, 2004.

[5] Tsubamoto M, Kuriyama K, Kido S, et al.:Detectionof lung cancer on chest radiographs: analysis on thebasis of size and extent of ground-glass opacity at thin-section CT.Radiology, pp.139-144. 2002.

[6] T. Kurita, M. Pic, and T. TAKAHASHI,“Recognitionand detection of occluded faces by a neural networkclassifier with recursive data reconstruction,” In Proc.of IEEE Conf. on Advanced Video and Signal BasedSurveillance, pp.53-58, 2003.

[7] A. Serdaroglu, A. Ertuzun, A. Ercil, “Defect Detec-tion In Textile Fabric Images Using Wavelet Trans-forms And Independent Component Analysis,” Proc.of PRIA ’7, PP.890-893, 2004.

[8] R. Gnanadesikan and J. R. Ketternring,“Robust esti-mates, and outlier detection with multiresponse data,”Biometrics, No. 28, pp. 81-124, 1972.

[9] Toshiyuki Amano. ”Image Interpolation by High Di-mensional Projection based on Subspace Method,”icpr, pp. 665-668, 17th International Conference onPattern Recognition (ICPR’04) - Volume 4, pp.665-668, 2004.