identification and classification of coating defects using...

IDENTIFICATION AND CLASSIFICATION OF

COATING DEFECTS USING MACHINE VISION

A THESIS

submitted by

RIBY ABRAHAM BOBY

for the award of the degree

of

MASTER OF SCIENCE

(By Research)

MANUFACTURING ENGINEERING DEPARTMENT OF MECHANICAL ENGINEERING INDIAN INSTITUTE OF TECHNOLOGY MADRAS

OCTOBER 2010

ii

And God said, "Let there be LIGHT," and there was LIGHT.” Genesis 1:3

iii

Dedicated to the one, who loved me and endured the cross even when I was an enemy to him.

iv

THESIS CERTIFICATE

This is to certify that the thesis entitled “IDENTIFICATION AND

CLASSIFICATION OF COATING DEFECTS USING MACHINE VISION”

submitted by RIBY ABRAHAM BOBY to the Indian Institute of Technology

Madras for the award of the degree of Master of Science (By Research) is a bonafide

record of research work carried out by him under our supervision. The contents of this

thesis, in full or in parts, have not been submitted to any other Institute or University

for award of any degree or diploma.

Chennai-600 036 Research Guides

Date:

Prof. B. RAMAMOORTHY

Prof. M. SINGAPERUMAL

v

ACKNOWLEDGEMENT

Life and research in particular had never been a bed of roses and surviving this ordeal

is due to the bountiful grace of the Lord and Savior Jesus Christ who is closer to me

than a friend and a brother. The times spent reading the scripture had helped to

refocus and reinstate the fact that ‘hope is a good thing, perhaps the best of things’.

The guidance and faith of Prof. B Ramamoorthy had kept the ship sailing even in the

toughest of storms. He has given me ample liberty thus motivating me to give the best

in all situations. In spite of his varied responsibilities, it was never impossible to break

in and stir up a discussion with him. Prof. M. Singaperumal could even use a few

moments at the parking lot to give a great idea that is worth hours of arduous

searching of literature. Here I could see excellence and experience personified. I am

truly indebted to them for the guidance and mentoring.

The laboratory was a family away from home and especially Srikanth and

Rahamatullah always had a friendly air about them. Srinivas, Arunachalam, Vinoth

Jose, Mohan, Denis Ashok, Murugarajan, Giridhar, Periasamay, Bala, Kanthababu

were not just research colleagues but fellow comrades of a great ‘team’ who even

ventured outdoor with some bouts of cricket on weekends. Srikanth, Sekhar Reddy

and Janardhan have given me significant suggestions regarding research and career.

Thippesamy and Rajsekhar, have been a source of a lot of activity and life and gave a

breather during our monotonous laboratory schedule. Santhosh, Uppu Srinivas Rao,

Deepak Lawrence and Binu helped me with their invaluable suggestions regarding

research and life at IIT.I also thank Praveen Kumar and Prashant S. Sonakar had

helped me significantly during the early days of my research.

IITCF was another big family I had in IIT and we spent a lot of good times singing

songs and worshipping God. It supplemented as a place to learn about helping one

vi

another and thus living loving God and people. Srikar, Revanth, Srinivas, Praveen

Kumar, Eldhose, Balamurugan, Oliver has all been source of so much warmth. The

members of IITCF including Chethan, Srinath, Ebenezer, Christopher, Ronald are

people whom one will like to stay close to till the very end. Prof. David Koilpillai,

Prof. G.L.Samuel, Prof. Koti, Prof. George, Kokila akka, Anu akka and other elders in

IITCF had been real mentors who showed the way to live, bearing fruits which will be

useful to society as a whole. I also feel indebted to the friends and pastor at NLAG

Chennai for the great time we had.

The experience in Munich was as varied as the -200C of Bavarian winter and the 420C

of Chennai summer. It was divine grace that I could meet Thorbjörn Buck, Ana Peréz,

Mathias Müller, Thomas Bodendorfer, Daniel Dorigo, Florian Hirth, Benjamin and

Markin Jakobi. Their company always inspired me to get back to work after the

slumber during Bavarian weekends. I had really nice time singing Deutsche Gospel

hymns with Xi, Daniel Baumann, Korina, Sedoni, Jana and Prisca and would like to

relive those moments.

It was during the last year that a beautiful damsel made entry into my life and life was

never the same after that. Life itself has found a new course of sailing once she has

come into my life. Also I am indebted to my father mother and brother who were so

close even understanding my thoughts even before I spoke about them. They were

always there in all situations and were a constant source of encouragement and

comfort. I convey my gratitude to the former HOD Prof. M. S. Shanmugam and

current HOD Prof. S. P. Venkateshan for their inspiration and assistance. I thank Mr P

Sreenivasulu, Mr Sridhar, Mrs J E Malarvizhi Alice, Mrs A Vasantha Kumari and

Mrs B. Sundari for their assistance in all administration related issues. I extend my

warmest thanks to Mrs. Thamara Dineshan, Mrs. Saraswathi, Mr. Muthiyal, Mr.

Paranjyothi, Mr. Radhakrishnan, and Mrs. Raniamma for helping in all the

vii

proceedings in the laboratory. I also thank Prof. K. Balakrishnan of Machine design,

Prof. Chandrasekhar of Computer science, Prof. Kothiyal of Applied optics, Prof.

Baburaj of Applied Mechanics, Prof. N Ramesh Babu, Prof. Shanmugam and Prof. L

Vijayaraghavan for their assistance at various stages of my work. I also express

special gratitude to Lakshmi Machine Works Coimbatore India for supplying the ring

samples for inspection and for their hospitality. Though space, time constraint renders

personal expression of gratitude impossible, there have been innumerable people who

have contributed to my life during stay at IIT Madras and TU München and I am

really happy that I could meet them all and I thank each of them for leaving a mark in

my life.

Riby A. B.

viii

ABSTRACT

Keywords: machine vision, inspection, coating defects, classification, dark field, Gray

Level Co-occurence Matrix, image series

Textile spinning uses spindle ring machines in which large number of chrome coated

ring components is used. The rings are made of Bearing Steel and then Chrome

coating is done to improve the life of the rings while in use. After coating, the rings

are to be inspected for coating defects which are visually done in the current set up.

The automation of the process has been attempted but with no significant success.

Given the reflective nature of the surface and the inner intricate profiles, it is really

challenging to build a set up that can successfully detect all the coating defects. To

solve the inspection problem, first an attempt was made using bright field illumination

set up based on which two different algorithms for defect detection and region

segmentation have been attempted. One was based on Fast Fourier Transform (FFT)

filtering, which was previously used for defect detection on textured surfaces. To

augment the performance, an auto-median operation based on morphology was

appended to the FFT filtering algorithm. The new algorithm was more accurate than

the one which is currently used in the industry. From the segmented defective region

in the image, many geometrical parameters can be identified. The geometrical

parameters of defect that are useful for the classification have been identified. But, as

many non defective samples were triggering false alarms, it was necessary to image

the surface with greater contrast.

Dark field illumination is a technique of getting very high contrast images by using

illumination with light falling at grazing angles to the surface under inspection. This

method of illumination helped in imaging the defective area with higher contrast. The

histogram had a bimodal distribution, which implies that lesser computation will be

ix

sufficient for segmenting out the defects. To detect the presence of defects, a single

step thresholding method was used. Later, a better thresholding was done using the

Fractional Change in Derivative (FCD) method which is proposed in this work. The

dark field illumination was found to be more accurate and faster than the bright field

illumination based approaches. It is thus possible to detect even smaller sized defects

on the ring surface using higher magnification.

The dark field approach which is based on scattering of light might lead to loss of

some information about the defective regions in the image. As a result, a new

approach based on different illumination sources was used. This helped in avoiding

the loss of information through the scattering of light at the defective surface and has

been used to highlight discontinuities. A series of images have been captured and

were used to find the Gray Level Co-occurrence Matrix (GLCM) parameters for each

of the pixels under consideration but across the series. A window was transposed over

the pixel and the GLCM matrix for the same was obtained. Later the thresholding

operation was done using the GLCM parameters and the thresholded regions were

used to train a Bayesian classifier. All these steps used an information fusion

approach, where results from different approaches and sources are combined together

to obtain the final conclusion for classification of defects. The pattern recognition

machine developed in this manner can classify each of the pixels of the input image

series as a particular defect class. Thus it is possible to detect the presence of defects

and then to classify the rings according to the type of defects. Therefore the proposed

inspection set up described in this work consists of a machine vision system using

dark field illumination technique coupled with multiple illumination sources. The

associated algorithms developed are aimed at detecting the presence of defects and the

subsequent classification of the rings with the speed and accuracy required by the

industry.

x

TABLE OF CONTENTS

Title Page No. ACKNOWLEDGEMENTS……………………………………………...... v

ABSTRACT……………………………………………………………....... viii

LIST OF TABLES…………………………………………………….…... xiii

LIST OF FIGURES…………………………………………………….…. xiv

ABBREVIATIONS…………………………………………………….….. xviii

MATHEMATICAL NOTATIONS............................................................. xix

CHAPTER 1 INTRODUCTION

1.1 Quality control……………………………………………….…. 1

1.2 Automated visual inspection…………………………………… 2

1.2.1 Basic steps to build a machine vision set up…………………… 3

1.3 Pattern recognition using machine vision………......................... 4

1.3.1 Sensing…………………………………………………………. 4

1.3.2 Segmentation and grouping…………………………………….. 5

1.3.3 Feature extraction………………………………………………. 5

1.3.4 Classification…………………………………………………… 6

1.3.5 Post processing…………………………………………………. 6

1.4 Objectives and scope of the present work……………………… 7

1.5 Organization of the thesis…………………………………....…. 7

CHAPTER 2 LITERATURE REVIEW

2.1 Introduction................……………………………….................. 9

2.1.1 Inspection of reflective and curved surfaces……….................... 9

2.1.2 Set up for inspection of reflective and curved surfaces................ 11

xi

TABLE OF CONTENTS (Contd...) Page No. 2.1.3 Illumination set up…………………………………................... 13

2.1.4 Defect classification…………………………………................ 19

2.2 Summary……………………………………….……................ 20

CHAPTER 3 PROBLEM DEFINITION

3.1 Introduction………………………………………..................... 22

3.2 Ring spinning.............................................................................. 22

3.3 Ring specifications………………………………….................. 24

3.4 Ring inspection system………………………………................ 25

3.5 Motivation for the image processing algorithms………............. 28

3.6 Summary .........................…………………………................... 30

CHAPTER 4 IMAGING AND ILLUMINATION SYSTEM

4.1 Introduction................................................................................. 32

4.2 Bright field illumination………………………..................…… 33

4.2.1 Axial illumination system…………………………................... 36

4.3 Dark field illumination………………………………................ 41

4.3.1 Implementation of dark field illumination.................................. 50

4.4 Summary…………………………………………..................... 52

CHAPTER 5 DEFECT IDENTIFICATION

5.1 Introduction................................................................................. 54

5.2 Bright field illumination…………………………….................. 54

5.2.1 FFT filtering based on critical radius……………….................. 55

5.2.2 Auto-median based approach in image segmentation................. 58

5.3 Dark field illumination………………………………................ 59

5.3.1 Fractional change in derivative (FCD) method for finding optimum threshold value.............................................................

62

5.4 Defect classifier using bright field illumination images............. 63

xii

TABLE OF CONTENTS (Contd..) Page No.

5.4.1 Classification parameter selection…………………................... 64

5.5 Results and discussions……………………………................... 65

5.5.1 Image processing algorithms for bright field images.................. 65

5.5.2 Image processing algorithm for dark field images….................. 70

5.6 Summary…………………………………………..................... 78

CHAPTER 6 DEFECT CLASSIFICATION USING MULTIPLE IMAGES

6.1 Introduction 80

6.2 Multiple imaging set up…………………………................…... 82

6.3 Gray Level Co-occurrence Matrix (GLCM) approach for multiple images...........................................................................

84

6.4 Image masks for each region of the ring…………................…. 92

6.5 Image thresholding operation…………………................…….. 93

6.6 Defect classifier………………………………................……... 98

6.6.1 Bayesian classifier…………………………................………... 100

6.7 Results and discussions……………………................………... 107

6.8 Summary…………………………………................………..... 109

CHAPTER 7 CONCLUSION AND SCOPE FOR FUTURE WORK

7.1 Conclusion………….………………………................……….. 111

7.2 Scope for future work………………………..………................ 112

REFERENCES.……………………………………...…………………… 117

LIST OF PUBLICATIONS……………………………………………… 121

xiii

LIST OF TABLES

Table No. Table caption Page No.

5.1 Performance comparison of algorithms.......................................... 72

5.2 Comparative performance with respect to FFT approach on bright field images.................................................................................... 78

5.3 Time of inspection per ring............................................................ 78

6.1 Classification based on discriminant (g(x)) values.......................... 109

6.2 Decision making by scoring for different parameters corresponding to pitting defect..............................................................................

105

xiv

LIST OF FIGURES

Fig. No. Title Page No.

1.1 Standard steps in the pattern recognition system............................. 4

1.2 Steps in classifier design.................................................................. 6

2.1 Variation in application of illumination techniques according to the surface reflectivity and shape................................................. 18

3.1 Textile ring component............................................................... 23

3.2 Details of ring component........................................................... 26

3.3 Images obtained using a machine vision system representing the various defects on ring components............................................ 26

3.4 Camera positions in automated defect detection set up used in industry........................................................................................ 28

3.5 Non-homogenous texture of Chrome coated surfaces seen at 100x magnification................................................................................... 29

3.6

Steps involved in the adopted methodology................................... 31

4.1 Lambertian surface showing the incident light being scattered in all possible directions....................................................................... 33

4.2 Reflection of light on specular surface............................................ 33

4.3 Experimental setup used for imaging of components using a vision system.................................................................................... 37

4.4 Images showing non defective surfaces under bright field illumination.................................................................................. 37

4.5 Images showing deep line under bright field illumination............. 38

4.6 Images showing yellow stain under bright field illumination......... 38

4.7 Images showing blister under bright field illumination.................. 39

4.8 Images showing built up under bright field illumination............... 39

4.9 Images showing damage under bright field illumination................ 40

xv

List of Figures (Contd...) Page No.

4.10 Images showing peel off under bright field illumination................ 40

4.11 Images showing pitting under bright field illumination.................. 41

4.12 Images showing rough finish under bright field illumination......... 41

4.13 Principle of dark field illumination.............................................. 42

4.14 Setup for inspection of ring outer diameter using dark field illumination...................................................................................... 45

4.15 Images showing non defective surface under dark field illumination................................................................................. 46

4.16 Images showing blister under dark field illumination..................... 46

4.17 Images showing deep line under dark field illumination................. 47

4.18 Images showing damage under dark field illumination................... 47

4.19 Images showing rough finish under dark field illumination............ 47

4.20 Images showing pitting under dark field illumination..................... 48

4.21 Images showing built up under dark field illumination.................. 48

4.22 Images showing peel off under dark field illumination................... 49

4.23 Images showing yellow stain under dark field illumination............ 49

5.1 FFT filtering................................................................................... 57

5.2 Image characteristics for non defective surface............................ 60

5.3 Image characteristics for defective surface..................................... 60

5.4 Effect of using different threshold values.................................... 61

5.5 Conditional probability density functions for blister, built up and damage classes............................................................................. 65

5.6 Image segmentation using FFT approach to highlight the defective region in bright field image........................................... 67

5.7 Failure of the FFT method on bright field images of non defective surface.......................................................................................... 67

5.8 Image segmentation using auto-median approach on bright field image............................................................................................ 68

xvi


5.9 Performance of FFT and Auto-median approach on images of defective surface of ring............................................................... 69

5.10 Performance of FFT and auto-median approach on images of non defective surface of ring............................................................... 70

5.11 Image of blister in different illumination systems........................... 72

5.12 Image of peel off in different illumination systems......................... 73

5.13 Single step thresholding for images of blister............................... 73

5.14 Single step thresholding for images of deep line.......................... 73

5.15 Single step thresholding for images of heavy blister.................... 74

5.16 Single step thresholding for images of built up............................ 74

5.17 Single step thresholding for images of rough finish..................... 74

5.18 Single step thresholding for images of pitting............................. 75

5.19 Single step thresholding for images of damage........................... 75

5.20 Single step thresholding for images of non defective surface...... 75

6.1 Experimental set up for illumination with 3 illumination sources.. 83

6.2 Effect of variation in illumination position..................................... 83

6.3 GLCM computation..................................................................... 84

6.4 Image fusion for calculation of GLCM parameters......................... 90

6.5 Gray scale representations of GLCM parameters............................ 91

6.6 Masks used for different regions of the image of ring surface........ 93

6.7 Thresholding operation.................................................................... 95

6.8 Threshold images considering different parameters........................ 96

6.9 Information fusion for thresholding operation............................. 97

6.10 Plot for Sum average showing the separation of defects using normal distribution values........................................................... 98

6.11 Plots showing the sum of squares against sum average with the clusters of defects well separated out........................................... 99

xvii


6.12 Working of a classifier.................................................................... 101

6.13 Bayesian decision boundaries for defect classes in the scatter plot of sum average against difference variance.................................. 104

6.14 Illustration of information fusion in pixel classification............... 105

6.15 Thresholded images with defective areas colored........................ 106

6.16 Flowchart for the defect classification approach............................. 109

xviii

ABBREVIATIONS

CCD Charge Coupled Device

DFT Discrete Fourier Transform

FFT Fast Fourier Transform

GLCM Gray Level Co-occurence Matrix

ID Inner diameter

LED Light emitting diode

OD Outer diameter

RAM Random Access Memory

xix

MATHEMATICAL NOTATIONS

English Symbols

c Number of classes considered for classification

CΔ(g1,g2) The entry at (g1,g2) of the Gray Level Co-occurrence Matrix with

neighbor condition defined as Δ=( Δx, Δy, Δθ).

d Dimension of x

F(u,v) Fast Fourier Transform of f(x,y)

f(x,y) A function in x and y. Used here as representing pixel values for the

index i=x and j=y.

g Gray scale value of the image

g(x) Discriminant function

g(x,y,θ) Gray scale value at index (x,y) in the image taken with illumination

position θ

h(g) One dimensional array representing the histogram values of the

image

I Two dimensional matrix representing image gray scale values

j √ 1

Li Represents the instance of the ith illumination being switched on

ln(x) Natural logarithm of x

M Two dimensional matrix representing structuring element in image

morphology

N(x) Normal distribution function

nCm Combinations of selecting m out of n samples. nCm!

!

Ng Number of distinct gray scale levels. It can be a maximum of 256.

xx

Nx Number of pixels in horizontal direction

Ny Number of pixels in vertical direction

p(i,j) Entry at (i,j)th address of Gray Level Co-occurence Matrix

P(u,v) Power spectrum of F(u,v)

p(x) Probability density function for x

p(x, ωi) Probability density function in two variables x and ωi

p(x/ωi) Class conditional probability density function of x given class ωi.

Also called likelihood.

P(ωi) Prior probability for event ωi to occur

P(ωi/x) Posterior probability

px(i) Array formed by addition of all the column elements of p(i,j)

py(j) Array formed by addition of all the row elements of p(i,j)

x Parameter vector given as input to the classifier defined as

[x1,x2....xd]T

Greek Symbols

δ Kronecker symbol. Becomes equal to one when a=b else 0

µ Mean

σ Standard deviation

Σ Covariance matrix for x given by Σ(i,j)=cov(xi,xj)=E[(xi- µi)(xj- µj)]

where µi=E(xi) is the expected value of i th element of x.

ωi ith category of defect

CHAPTER 1

INTRODUCTION

1.1 QUALITY CONTROL

Quality is pivotal to the successful functioning of an enterprise as it can be explained

as the ability of the products or services to satisfy the requirement of the customers,

rather than just conforming to the standards. Quality planning, quality control and

quality improvement are suggested by Juran et al. (1998) as the trilogy which ensures

quality in the modern day manufacturing centres. For an effective decision at all these

stages (in line with Garbage In Garbage Out-GIGO- concept), there needs to be a

measure of the current status of the quality in an accurate and quick way. A viable

option is to use sensors for such measurements of the critical parameters of the

finished product. There is another important decision to be made as to the degree of

inspection that is required for any product. Given the situation of progressive

improvement of the manufacturing process, the inspection has to be flexible enough

to accommodate newer quality standards.

When the process capability is to be improved due to sudden changes in the quality

specification, there is a requirement for 100% inspection. 100% inspection has its

disadvantages, since the cost involved is higher and additionally the human operators

also have to be educated to make the right decision. But the availability of

inexpensive sensors, higher computer memory storage capabilities and calculation

speeds makes it economical to implement automated inspection methodology in the

current production environment.

2

Though sensors are available for automating many of the inspection tasks, there are

still many situations where manual inspection is the best possible solution. When it

comes to visual inspection of varied kinds of defects in complex surfaces which are

sometimes hidden to the view completely, manual inspection still may be considered

as a suitable option.

Accounting for all shortcomings, it has been found that manual inspection has only

80% accuracy in making correct inspections (Juran et al., 1994). As a result there is

always an attempt to augment the senses of the inspector using instruments like sound

amplifier, optical magnifier etc. The development of an application specific inspection

system is the ultimate form of this. Manual inspection can be replaced by automatic

inspection to reduce labour fatigue, and save money and valuable man hours wherever

possible. This helps in quality improvement, cost reduction, increased volume, and

shorter cycle time.

1.2 AUTOMATED VISUAL INSPECTION

Currently in industries, inspection of components is automated to a very large extent.

In this context, optical inspection always has been a tricky problem to solve since

each situation needs a unique approach. Machine vision is a form of non-contact

inspection, which can be used for a wide range of electro optical sensing techniques

from triangulation, profiling, 3D object recognition to bin picking based on image

processing routines. In addition, it is being used for simple detection and measuring

tasks to robotic control. The most common applications are measuring critical

dimensions, detecting flaws, counting/sorting, assembly verification, position

3

analysis, character or bar-code reading/verification, and determination of

presence/absence of features on small parts. The major advantage is avoidance of

human error. The process involves capturing an image of the object to be inspected

using vision sensors and then processing it using image processing algorithms to

measure the features of interest. One very important advantage of machine vision

systems is that 100% inspection can be achieved.

1.2.1 Basic Steps to Build a Machine Vision Set Up

The machine vision approach to solve optical inspection problems can be done in

three stages. The first is to set up a camera and proper illumination to capture images

with very good quality required for any specific application. Generally the usage of

vision inspection is dictated by the application and it should be always aimed to make

it simple. Complex image processing algorithms that are used in many automatic

inspection systems could create a burden on the machine memory and delay the

detection process. By careful selection of the light source, illumination intensity and

its positioning, the effectiveness of inspection can be ensured. It is more cost effective

and better engineering practice to capture a good image at the source, rather than to

pour resources into cleaning it up or simplifying it later. The second stage is the pre

processing of images to gather the necessary quality for the application desired and to

obtain the image data. Later, the data has to be processed and then the decision

regarding the inspection status has to be made. The available information has to be

used to make a decision regarding the type of defects, since no approach aimed at

defect detection will be complete without quantifying the decision. The final step is an

4

elaborate process in itself because decision making is a case of pattern recognition

and involves a lot of preliminary steps to ensure correct decisions.

1.3 PATTERN RECOGNITION USING MACHINE VISION

The defects identification and classification problem using machine vision application

can be modelled as a pattern recognition problem, since the underlying aim is to make

classification based on the input image. The steps involved are shown in Figure 1.1

(Duda et al., 2001).

Figure 1.1 Standard steps in the pattern recognition system (Duda et al., 2001)

1.3.1 Sensing

The inputs to the pattern recognition system are the images of the component to be

inspected. For this purpose, a Charge Coupled Device (CCD) camera coupled to an

image grabbing system is generally used. A suitable lighting system and fixtures to

hold the ring is also a part of the set up. The orientation of the camera and lighting

system are made in an optimum fashion such that the images are clear, better quality

and of high contrast. Also the optics should have sufficient magnification and

Sensing Segmentation Feature extraction Classification Post processing

Input

Analysis & decision

5

working distance. If intrinsic surfaces are to be imaged, then the orientation should be

proper to enable easy imaging of each surface.

1.3.2 Segmentation and Grouping

The output from the sensor is digital images, which consist of small picture elements

or pixels having a particular range of gray scale values. This value is proportional to

the amount of light that is incident on the physical pixel element on the machine

vision sensor after reflection of the incident light from the surface or object under

inspection. The final image is a matrix having indices of the pixels and their gray

scale values. If the image is in colour, the matrix will have the third dimension as the

different gray scale values corresponding to Red, Green and Blue (RGB). But the

feature to be inspected is among other information and cannot be easily separated out.

For this purpose, the region of interest has to be separated out from the rest of the

region where measurements are not made and this process is image segmentation. The

algorithms for image segmentation are highly dependent on the type of images taken.

It varies from one situation to another but can be a set of independent image

processing operations which are done one after another to get the required results. In

most of the defect detection set ups, the algorithm should reveal the defective area of

the surface imaged, if they exist.

1.3.3 Feature Extraction

The results from segmentation will have the gray scale values and indices of the

defective area of the image and the details are supplied to the feature extraction

6

algorithm. The features may be many, depending upon either the gray scale values or

a combination of both or even just the index values. Many statistical and geometrical

features can be defined for the area that is under consideration. The best choices are

the ones which will be useful in making a decision at the next step, which is the

classification. While designing the pattern recognition system, there should always be

a feedback from the higher levels to the lower levels of the system.

1.3.4 Classification

The feature vectors should initially be used to find the presence of defects, which can

also be done at the segmentation stage. The defect classes where the probable

defective component may belong to, have to be clearly defined. Also other

information regarding the peculiarities or statistics of occurrence of defects has to be

procured. The probability of the defect belonging to a particular defect class is found

out and the one with the greatest probability is considered as the final defect type.

Figure 1.2 presents an idea about the typical classifier design.

Figure 1.2 Steps in classifier design (Duda et al., 2001).

1.3.5 Post Processing

The ultimate aim is to separate out the defective products into respective bins. The

output from the classifier can be supplied to mechanical actuators, for example, to

Collect data Choose features Choose model Train classifier Evaluate classifier

Start End

7

make the relevant sorting. Also at this stage, the success of the whole system can be

evaluated and used for subsequent improvements in the classifier.

1.4 OBJECTIVES AND SCOPE OF THE PRESENT WORK

To identify and to classify the various coating defects present in textile ring

components using a machine vision system. The methodology followed to achieve

this is as follows

1. To use multiple cameras at various locations and at specified angles.

2. To use different methodologies of illumination namely bright field, axial and

dark field. Then to choose the best one for the specific application.

3. To develop image processing algorithms which are fast and accurate for every

kind of illumination set up used and compare their performances.

1.5 ORGANIZATION OF THE THESIS

The thesis is organized as follows. The need for automated visual inspection and the

methodology for solving pattern recognition problem using machine vision has

already been presented in Chapter 1. Chapter 2 sketches a brief literature survey

describing the different applications where machine vision has been used for defect

detection on curved and reflective surfaces. Different methodology for classification

of defects using images also has been discussed in this chapter. The description and

basic information regarding the ring inspection problem is given in Chapter 3. In

Chapter 4, different kinds of illumination systems that are used for capturing the

images in the present work have been explained. Different algorithms to enable fast

8

and accurate defect region segmentation on the images captured using the imaging set

up is discussed in Chapter 5. In Chapter 6 an approach to classify defects based on

information from multiple images is presented. Also explained in this chapter are the

GLCM approach of parameter calculation for image series and the thresholding

operation using series of images. The summary of work done and the conclusion and

scope for future work are presented in the final Chapter.

CHAPTER 2

LITERATURE REVIEW

2.1 INTRODUCTION

There have been many attempts to inspect manufactured components for surface

defects using machine vision. Most of these works have been on the area of

development of image processing algorithms since the decision about the component

quality is made on the basis of the image and not directly based on the real object.

Many of them were based on FFT based filtering since it is an easy and fool proof

method of separating out defective regions from images of the component to be

inspected. Illumination has been studied by industrial users of machine vision and the

knowledge has been put to good use in quick and accurate inspection. Pattern

recognition algorithms are useful in the final classification of defects and form the

final stage of the solution for defect identification and classification problems.

2.1.1 Inspection of Reflective and Curved Surfaces

Fourier domain operations are a useful way of visualizing frequency variation in

signals. Two dimensional Discrete Fourier Transform (2D DFT) has been

successfully applied in the separation of lay patterns of surface textures in many

applications. Bayerer and León (1997, 1998) have used a similar method for the

separation of background and lay pattern of surface images. León (2002) has

successfully applied it in industry for the detection of honing angle in cylinder

engines. The approach involved the separation of background and lay pattern. The lay

10

pattern is visible as straight lines in the intensity plot of the 2D DFT of the image. The

angle between these lines which is the actual angle of honing in the cylinders can be

found out by Hough transform. The problem is similar in the sense that the inner

surface of the cylinder is reflective.

Tsai and Huang (1999, 2003) used Fourier image reconstruction for the detection of

defects on engineered surfaces. They had dealt with statistical textures like that of

sandpaper and have clearly demonstrated the selection of cut-off radius for such

surfaces. Later the same algorithm was used on flat sputtered surfaces which are non

homogenous and non-statistical texture. The inspection problem has a prominent

resemblance to the one discussed in this thesis, albeit curvature of the inspection

surface.

Zhang et al. (2006) attempted defect detection on ground and polished surfaces. The

approach involved extraction of the image features which were then used to classify

the defects into the respective classes. The defect classes were well defined and the

accuracy of the system to classify these defects has been discussed.

Wiltschi et al. (2000) devised a method of finding steel quality from the microscopic

images of etched and polished steel specimens. The process was automated with a

very significant accuracy. The steel carbide distribution images were taken and then

the carbide regions were thresholded based on the second derivative of the histogram

values. The shape and size of the segmented surface area was determined and the

most valid parameters were selected as input to the classifier. Finally classification

was done using a minimum Euclidean distance classifier. The process has been

11

implemented in the steel manufacturing industry and is currently used for routine

inspection.

2.1.2 Set Up for Inspection of Reflective and Curved Surfaces

Rosati et al. (2009) have attempted real time defect detection on curved reflective

surfaces using a set of mirrors. A special mirror was designed to direct the

illumination at specific direction and the presence of defects could be observed by

analysing the images using standard image processing software. Bright field

illumination approach was used, where the machine vision sensor was in the path of

specularly reflected light rays. A similar set up can be used for inspection of

automobile parts with highly reflective surfaces. But the high intensity reflections

might saturate the machine vision sensor resulting in loss of information and this was

not discussed in detail. To avoid saturation, reduction in the intensity of incident light

might cause lesser contrast in the defects that are imaged. This is a particular anomaly

while using bright field imaging systems.

Sun et al. (2005) applied X-ray imaging for real time detection of welding defects in

steel tubes. Though X-ray was used, the signals were captured in digital form just like

visible light and the image was used for subsequent analysis. The use of X-rays and

image processing paradigms could easily automate the inspection process which was

undertaken by human inspectors till then. The feed back after the implementation of

the set up was that it is more efficient than manual inspection of video streams. This is

an example of capturing higher contrast image information by use of X-rays.

12

Moreover, in this particular case, choices for inspection set ups were limited since the

defects were subsurface.

Luo and Liou (1998) used multiple cameras to measure the surface conditions of a

cylinder and used the images to measure deviations. The methodology was based on

binocular vision. The images captured by the two imaging devices have been

correlated with each other to determine the diameter of the sample to be inspected.

Both the imaging devices were used to capture the same surface under inspection. Use

of two cameras helped in the evaluation of depth information, though there were

anomalies due to the curvature of the surface. The explanation is particularly useful

for determining the diameter of cylindrical components, rather than surface

inspection.

Microstructure surfaces were reconstructed from SEM images by Samak et al. (2007).

Unlike the previously discussed approach, the sample to be inspected was tilted and

the change in position of the point under observation was converted into height

information using the standard triangulation algorithm used for most of the depth

detection problems. Later De launey’s triangulation has been done using this

information. The images were microscopic and the CAD model of many broken

materials was successfully created such that fracture analysis can be done using this

information.

Abrahamovich et al. (2005) described the calibration and use of multiple cameras for

inspection. Quite interestingly, the cameras used were commercially available

webcams, array of cameras such that imaging of a large area can be covered. Many of

13

the inspection scenarios like wooden sheet inspection, sheet metal inspection demand

multiple cameras and the performance, as stated by the author, is really convincing for

an industrial user since the web camera costs 1/50th of an industrial camera. The

methodology of imaging different areas of the same surface and then stitching all of it

together will enhance the speed of inspection provided the illumination source for all

the surfaces also remains the same. But the approach is really beneficial when the

surface for inspection is planar (which the author has specifically stated) and can be

imaged at once.

2.1.3 Illumination Set Up

Microscopic defect detection on mirror surfaces using laser beams has been

investigated by Porteus et al. (1986). The mirror surfaces that get damaged due to

laser illumination were inspected for defects based on the scatter from the surface of a

beam of laser. The system was developed for analyzing a very small surface area.

Though the technique of illumination was for detecting presence of laser initiated

defects, a similar scattering technique can be used with a machine vision sensor for

inspection of a larger surface area of highly reflective surfaces. Additionally, the use

of polarized beams can reveal the presence of defects and enable easier separation of

background light from saturating the machine vision sensor. The approach reveals that

optical methods also will aid in the detection of defective surfaces and this

observation was used again in the detection of laser damage in mirrors by Marrs and

Porteus (1985). In this case, a source of pulsed laser was used to illuminate the

surface to be inspected and the scattered light was received into a video microscope

system. The illumination direction was normal to the surface to be inspected and a

14

polarizer was used in front of the video microscope to enable reception of more

details regarding the light scattered by defective surfaces. The set up was used to

obtain the physical images of the surfaces with defects shown at higher contrast.

Khalili and Webb (2007) used multiple wavelength illumination technique for defect

detection on hemispherical surfaces. The surface to be reflected was rivet surface

which is spherical and metallic, thus reflective. Such surfaces always make a standard

illumination difficult since the light is received at varying illumination angles at the

different surfaces of the spherical rivet. Added to it, the problem was an online

inspection system, further compounding the difficulty in deciding upon a single

illumination system. The solution was use of multiple illumination sources, but that

meant, usage of a particular image for a given illumination source. Later two lights of

different wavelength were used to illuminate the surface in unison while using a

colour camera to process different intensities of the corresponding imaged surface

from single image, which enabled quicker inspection. They used comparison of the

two images to get the final image, free from specular reflection and shadows. Later a

correction was done to accommodate for linear variation of light because of the off

centric position of the imaging source. Image of a rivet can be captured using a single

image and the defects can be identified to enable automatic inspection. This is a very

good example where illumination variation in position has been used to successfully

detect the presence of defects with higher speed in an online inspection environment.

Nayar et al. (1991) have discussed, in detail, the geometrical reflectance model which

does not use the electromagnetic theory of propagation of light and the physical

reflectance models which consider the electromagnetic theory of propagation of light.

15

When the physical features under observation have dimensions comparable to the

wavelength of light, physical optics are valid. Geometrical optics is a generalized case

of physical reflectance model when the wavelength is much lesser in comparison to

the surface dimensional features. On the basis of Beckmann-Spizzochino (Beckmann

et al., 1963) and Torrence-Sparrow (Torrance et al., 1967) models, any illuminated

surface will produce a specular lobe, specular peak and a diffuse lobe. The incident

luminous intensity on the sensor, thus, can be written as a sum of specular reflection

lobe component, diffuse reflection lobe component and specular reflection peak

component. Prior knowledge about the surface under inspection will help in finalizing

the reflectance model to be implemented, because in any given application one of the

reflecting natures is supposed to dominate. According to the surface characteristics of

reflection, the best approach can be selected and used to solve the machine vision

problem.

Later, a similar approach was used to image dielectric substances, but the method

used was polarisation and colour separation (Nayar et al., 1997). The highlights due to

specular reflection were masked out using polarizing filters. Colour intensities of the

image were also considered to capture images with better details. The approach

worked well for many applications involving non metallic surfaces. In the case of

non-metallic surfaces, all the reflections from the inspected object will be completely

masked out by a polarising filter, thus leading to loss of information.

Geometrical measurements and object verification can be done very efficiently if

images are of very high contrast. This was studied by Yi et al. (1995) by varying the

position of the sensor and the illumination source. The best contrast images will have

16

the least variance across the edges in the image and have the maximum possibility of

being detected. It is obvious that the capture of high contrast images will avoid the

requirement of complex image processing routines, which are an imposition on the

speed, accuracy and development cost of the machine vision system. Software has

been developed to find the optimum position of sensor and the illumination source,

given the geometry of the object, photometric property of the object, object edges

involved in inspection, measurement type and light source. Torrence Sparrow model

of reflection has been used to develop the aforesaid approach. Though the approach

sheds significant amount of light into the positioning of the sensors and illumination

source in the pursuit of images of high quality, it was suitable for simple edge

detection problems where the edge could be easily represented by simple geometrical

curves. The experimental data on dimensional measurement of cubic shaped

specimens also reinstates the feasibility of the approach in getting more accurate

information about the imaged region.

For practical applications, there are a wide variety of illumination systems that are

dependent on the condition of surface under inspection and feature to be inspected.

Since most of the vision inspection systems attempt to undertake the inspection

process in minimum time and with maximum accuracy, the best type of illumination

and the best position for illumination has to be determined. Connolly (2002) has given

a description of the application specific illumination systems and explained the

concept of dark field illumination, which is used to highlight scratches and other

surface irregularities in metals. The light source is kept at an angle in such a way that

there will be a low angle of incidence to the object surface to be inspected. If the

object is flat, the light emerges in the dark field angular region and thus misses the

17

sensor giving a dark appearance to the object surface. This is not the case if there are

some scratches. The specular light reflects back into the sensor region from the angled

side of the scratch, thus appearing bright. Dark field illumination can be used behind

and around small transparent objects to highlight their edges against a bright

background and this is used for inspection of syringes and in microscopy to examine

cells in medical applications. In short, the dark field illumination enhances the

contrast between the specular and non specular areas of the object, and this can be

exploited to a very good extent in the inspection of metallic surfaces.

Dark field illumination is predominantly used in many applications, especially

microscopy. In a perfectly clear, transparent medium, the field of vision will remain

black, but in the presence of refractive boundary or scattering surface or particles,

light is reflected into the optical axis and is visible, just as dust is visible in sunbeam.

The key advantage is the avoidance of flooding of the focal plane by specular

reflections. It is also used for macro specimen photography like revealing the fine

structure in the cataract of the eyes, lenses, jellyfish, amoeba etc. (David, 2010).

For machine vision applications the lighting condition has been the subject of

considerable importance as it has wide and varied inspection applications. The

lighting systems have a major influence in achieving the final result/goal for machine

vision applications. Martin (2007) explained the commercially available and more

popular lighting systems in detail. The illustration regarding the application of

different kinds of illumination is given in Figure 2.1. An interesting observation is

that when the surface becomes reflective and curved, the inspection problem becomes

more complex. Dome illumination or diffuse illumination, which is the recommended

18

methodology, is possible for easy to illuminate surfaces like spherical surfaces and

not for internal features.

Figure 2.1 Variation in application of illumination techniques according to the surface reflectivity and shape (Martin, 2007)

Images with dark field features have been used in many applications and one of it is in

the process of lace cutting (Bamforth et al., 2007). The illumination using ring light

gave a higher contrast image of the transparent lace boundary. The approach could

reveal more details about the lace boundary and thus enabled precision laser cutting of

the lace pattern. Miyoshia et al. (2001) has used dark field illumination systems in

microscopy of semiconductor surfaces. Noda et al. (2008) has boasted of sub-

Matte

Mixed

Mirror Specular

Flat Uneven Topography

Curved

Bright Field

Dark Field

Diffuse Dome

Geometry independent area

19

nanometre measurements on biological specimens, using dark field imaging set up

utilizing axicons. Piper (2009) has attempted to create dark field images using

scattered light but directed normally rather than oblique illumination. The aim was to

increase the contrast of the specimen images in microscopy. Metallography is another

application of oblique illumination using laser source but in this case with the

production of speckles. The investigation by Povolo (1997) compared the results with

that from partial dark field illumination. All these applications reiterate the fact that

the contrast enhancement by oblique illumination can be used for inspection of

metallic surfaces.

2.1.4 Defect Classification

The development of efficient defect detection systems will not be successful without

an accompanying classification system. The classification problem, which is the final

stage of defect detection, involves the selection of the most relevant parameters as

well as the associated classifying algorithm.

Iivarinen et al. (1998) explained the selection of features for classification from the

segmented images of defects and used Gray Level Co-occurrence Matrix (GLCM)

and the related parameters from it for giving input to the classifier.

Guise et al. (2002) discussed the defect classification for semiconductor surfaces. The

classification was done using the shape of the defects using the shape based

parameters or geometrical parameters. The shape description parameter, especially

perimeter by area ratio (P/A), has been used in cartography by Salas et al. (2003).

20

Most of the development in the area of textural studies was promoted by the need for

remote sensing and cartography. Pioneering work was done by Haralick et al. (1973),

in which the Gray Level Co-occurrence Matrix was introduced and fourteen texture

parameters based on it were used for quantification. These parameters were used in

many applications involving surface characterization, though the primary intent

behind the introduction of them was to use them for analyzing remote sensing images.

These parameters have been used in the classification of directional textures

(Gadelmawla, 2004).

2.2 SUMMARY

There have been several attempts previously to detect defects on metallic surfaces.

But the approach of each researcher was unique, given the situation. Most of the

applications involved microscopy, which is not necessarily 100% inspection and thus

slower. Also, the components under inspection were either specially prepared surfaces

or else planar with minimum reflection. 100% inspection on metallic surfaces with

curvatures demanded special set ups, which are normally custom made according to

the applications. Most of them cannot be generalized to other applications but the

approach used gives a faint idea of how to solve a similar inspection problem. In this

work, machine vision system has been custom made according to the inspection

requirement which is to sense the ring defects according to the need at industry.

The lighting system, which is an inherent and important part of the machine vision

system, has been given the least significance for many years. This casts the entire

21

burden on the algorithms for defect detection. But many studies have proved beyond

doubt, that an optimum positioning of the sensor and the illumination system will

enable capture of very high contrast images and thus simplify the image processing

procedures by quickening it and minimizing the storage requirements.

Any defect detection algorithm has to incorporate a defect classification system.

There are many parameters based on geometry and texture that can be used but the

selection of the best one will enable accurate and quicker defect classification. The

steps involved have been briefly illustrated in Figure 1.2. The information regarding

the defect classes has to be collected. Defect class features to be used for

classification needs to be selected. After this, most suitable model of classifier has to

be chosen. Later the classifier is trained and then used for testing the data at the later

stages in the industry. Any shortfall in performance requires the redesigning of

intermediate steps until the industrial requirement is satisfied.

Based on these observations the following methodology has been adopted in this

work. Initially the maximum contrast image is captured by utilizing different

illumination systems. The relevant algorithms to process the images captured using

each of these illuminations have been developed. The results of the algorithms were

analyzed for accuracy and speed. Later the classification algorithm was developed

using image series based on an image fusion approach. A GLCM based approach has

been used to process the images for defect classification.

CHAPTER 3

PROBLEM DEFINITION

3.1 INTRODUCTION

The most important parts of a textile spinning machine are the spindles and rings

which are used in large numbers in each of the machines. As a result, thousands of

rings are manufactured at the industry to cater to the needs of the newer machines and

for the maintenance of older ones. The set up for ring spinning consist of a traveller

running on a ring and it is described in the following section.

3.2 RING SPINNING

There are four main methods of textile spinning, namely, ring spinning, rotor

spinning, air-jet spinning and friction spinning (Jayavarthanavelu, 2010). Of these,

ring spinning has the advantage of providing higher quality yarn that can be used in

many types of textile end products. Though the other approaches are new

developments which allow higher rates of production, the yarn that is produced

through these methodologies is of lower quality. This restricts their use and is the

reason behind the widespread use of ring spinning. The only technological limitation

for ring spinning lies in the quality of the rings that are used. This makes the ring as

the most vital part of the machinery.

The spinning process involves converting the un-spun fibres to yarn. In ring spinning,

the fibres are passed through rollers to compact them into a single yarn. Later, it

passes through the traveller and then, it is wound to the cob. The cobs are attached to

23

spindles which run at a maximum speed of 20,000 rpm. The traveller revolves around

the ring slower than this, but still the speed is considerable enough to cause significant

wear in the traveller. The traveller wears out more frequently than the ring and the

amount of wear is dependent on the overall quality and in particular the surface finish

of the ring surface. As a result, the rings are made with high carbon high chromium

ball bearing steel and are given a low friction coating. The low friction coating allows

the traveller to run at 3 m/s higher speed than the uncoated rings. This enhances the

productivity and at the same time, reduces the expenditure due to the wear in the

travellers. The spinning machine is called ring frame and has multiple (at least

600/machine) spindle ring combination system on both sides. It is clear from the name

itself that the ring traveller is a principal component of the whole machinery. The ring

assembly is shown in the Figure 3.1b.

(a) (b)

Figure 3.1 Textile ring component (a) CAD drawing for textile ring component, (b) Ring and traveller assembly in ring frame

24

These rings are produced at a very large scale owing to the fact that the ring is

subjected to constant wear and needs replacement frequently. Also, the functional

region of the ring has to be maintained at a high level of quality.

3.3 RING SPECIFICATIONS

A typical photograph of a ring component and its cross section details are shown in

Figure 3.2. Due to their critical role, they are subjected to inspection for various

possible defects in order to maintain a high quality, as explained later in this section.

The rings are manufactured in a CNC machine and then the chrome coating is done.

Hard chromium coating is highly reflective. It is an example of a non homogenous

texture. The thickness of coating varies from 5 μm to 10 μm. The outside diameter of

ring is around 45 mm and the inside diameter 38 mm (Figure 3.2c). Later, the rings

are subjected to an inspection process normally using conventional equipments, such

as profile projector, tool makers microscope etc., to identify the coating defects. The

frictional wear is dependent on the surface finish of the ring. For non defective

surfaces, the surface finish value should be approximately 0.1 μm for minimum

friction and frictional wear. It has been observed that the defective surfaces will have

higher surface finish values of the order of 0.5 μm. Such regions are visible under the

microscope on careful observation. Defects can be considered as those kinds of

features which deem the component to be unusable by the customers. The types of

possible defects (Figure 3.3) on the component surface are given below.

1. Pitting - Small pits in the coating leading to corrosion.

2. Damage - Dents of different shapes and sizes.

25

3. Deep line - Marks similar to feed mark which continue around the whole ring.

4. Built up - Bright elevated deposits. This is a more common defect. It can be

reprocessed before use to salvage the work piece.

5. Peel off - Small patches or spots showing small discontinuities, which are

normally visible only on careful manual inspection and are likely to be present

both on inner and outer surfaces.

6. Rough finish - Elevated features occurring as clusters, which are similar to

peel off, usually found at the inner and outer edges.

7. Blisters - Small porous surfaces on the coating, usually larger in spread as

compared to other defects.

3.4 RING INSPECTION SYSTEM

There have been efforts in the industry to attain lesser or even nil inspection

requirement which is not generally practical. Normally, for high volume applications,

inspection methodology consists of a sampling plan. This involves selective

inspection and is suitable for applications where the process is really capable of

producing components having consistent quality. This is however used at an advanced

stage of quality improvement process by which the manufacturing process has been

improved. This is normally done based on careful evaluation of process capability

based on the output from the inspection process over a long period. So, the quality

control department interacts with the production departments in the industry, making

a closed cycle of interaction and improvement of the process. But, any change in the

26

product specification will warrant a full repetition of the whole process from the

starting point.

Figure 3.2 Details of ring component: (a) Ring (b) Cross section of the ring showing defect prone area (c) Cross section details of the ring with dimensions

(d) (e) (f)

Figure 3.3 Images obtained using a machine vision system representing the various defects on ring components: (a) Pitting (b) Damages and deep lines (c) Built up (d) Peel off (e) Blister (f) Rough finish

In the present inspection problem, inspection criteria have been strengthened for

increasing the quality of the end product supplied to customers. This is a normal

development in any given industry due to a host of parameters like competitor pricing,

27

competitor quality, customer feedback etc. As a result, initially for a long duration,

the inspection was carried out manually by inspectors using magnification 10X

microscopes. Since the volume of production is very high (of the order of a few

hundreds of thousands of components every month), it presents a huge challenge and

a considerable amount of strain on the quality control department. There is also much

incompetence arising from using human inspectors for a long duration as discussed in

Chapter 1. The immediate solution was to switch over to an automated inspection

system.

An automated machine vision inspection system exists in the industry for this

purpose. The set up included five different cameras oriented at five different

directions to image and to analyze each of the defect prone areas of the ring (Figure

3.4). A gripper was used to bring the ring to be inspected into the inspection area and

images were taken after rotating the ring. The images were analyzed using LabView

software and based on the results; the rings were sorted into respective bins. The type

of illumination used was bright field and it was implemented using two fluorescent

ring lamps inside the enclosure of the machine vision system. The inspection time

required for one ring was 13 seconds. After switching over to an automated vision

inspection, it was observed that the accuracy of detection was a maximum of 60%

(i.e. If 100 defective samples classified as defective by the human inspector was

supplied to the system, only 60 were correctly detected as defective) and was

inconsistent too (i.e. accuracy used to be slightly higher if all non defective rings were

supplied).

In the light of the present situation, the industry wanted to completely automate the

system making it still faster and foolproof, in addition to avoiding human involvement

28

in the whole process of inspection. The rings have to be imaged using a machine

vision system and the corresponding algorithms need to be developed, tested and

analyzed for all the types of defects mentioned in the earlier section.

Figure 3.4 Camera positions in automated defect detection set up used in industry

3.5 MOTIVATION FOR THE IMAGE PROCESSING ALGORITHMS

The coating defects are really difficult to detect using a microscope. Only an

experienced technician can correctly identify the defects, which is really strenuous in

the long run. The major challenges in the inspection of these components are the

highly reflective nature of Chrome coated surfaces (Rosati et al., 2009; Khalili and

Webb, 2007; Pfeifer and Wiegers, 1998), the curvature of the surface to be inspected

(Rosati et al., 2009; Lee et al., 2000; Aluze et al., 2002) and also the inaccessibility of

some inner surfaces. Unlike directional textured surfaces, homogenous statistical

features pose considerable challenge when it comes to the detection of defects. If

there is a presence of a definite lay pattern, there can be a standard procedure to

segment out the particular pattern and then analyze the continuity of it. Discontinuity

will deem the sample to be defective. Most of the machining processes like turning

Camera 1

Camera 2

Camera 3

Camera 4

Camera 5

29

and milling produce such patterns of definite lay pattern. In a homogenous statistical

texture like that of sandpaper, the statistical features are more consistent all through

and this characteristic can be utilized to successfully ascertain the quality of the

surface (though the approach may not be fully foolproof using the digital images). It

is not so in the case of non homogenous features. Most of the coating processes

produce non homogenous texture (Figure 3.5).

(a) (b) (c)

Figure 3.5 Non-homogenous texture of Chrome coated surfaces seen at 100x magnification

The curvature of the surface might give bands of illumination which might interfere

with the image details and often gives an impression of false defects. This problem is

compounded by the fact that the surface is highly reflective owing to the low friction

surface coating. Inner fillets with varying direction of curvature make it really

difficult to image larger surface area, avoiding the specular reflections to the machine

vision sensor. The requirement to inspect the inner diameter surface makes it

necessary to overcome the space constraint for imaging and illumination source. As a

result, it is very difficult to image the surface with the defects in higher contrast. It

was impossible to use a single illumination source for the whole ring since the

reflective and curved surfaces used to reflect the light in different directions, thus

making inter-reflections on the surface. Though many standard software modules are

30

available to inspect defects on flat surfaces and non-reflective surfaces, none of them

are useful for inspecting internal, curved and highly reflective surfaces. Also, a less

computationally intensive algorithm is required to speed up the entire inspection

process.

3.6 SUMMARY

The inspection problem is difficult to solve, owing to the condition of surfaces to be

inspected particularly that of the ring components considered in this study. Special

nature of curved and reflective surfaces warrants a newer approach for achieving

accurate and speedy inspection. The adopted approach in this work is briefly given

below.

The first effort was made to image the surface using a standard illumination source,

which was a ring fluorescent lamp. The resulting images are called bright field

images. The images were then segmented using different algorithms. A Fourier

transform method has been implemented in an attempt to achieve this (Tsai et al.,

1999). To increase the accuracy of defect detection, a method combining a Fourier

filtering method and an auto-median operation has been proposed in the second

algorithm. Since the algorithms were slow and not consistently accurate, a different

illumination methodology was attempted.

The illumination source was used to direct illumination at a glazing angle to the

surface under inspection to achieve the dark field illumination. Another approach

which involves single step thresholding of the dark field image helps in increasing the

speed and accuracy of detection. This approach is based on an optimum positioning of

31

camera and illumination source (Pfeifer and Weigers, 1999). All the algorithms were

designed so that they will work for images taken from any of the four camera

directions, given the kind of illumination. Another major deciding factor is that the

inspection of a single ring has to be completed in 8 seconds with accuracy as per the

conditions set by the company. So, the algorithms have to be optimized for speed. All

the defects stated earlier in Section 3.3 have to be detected using the algorithms.

The classification of rings into categories of defects was attempted using multiple

illumination sources. Gray Level Co-occurrence Matrix (GLCM) was used to find the

parameters relevant for accurate classification and then an algorithm for thresholding

and classifying the defective region in the image was developed. It was finally

possible to detect the presence of defects and then to classify them, enabling sorting

of the defective rings with considerable accuracy and speed as required by the

industry. To achieve this result, various steps and procedures followed in this work

are shown in Figure 3.6. More details and explanations of the same are presented in

the following chapter.

Figure 3.6 Steps involved in the adopted methodology

MACHINE VISION SET

UP Camera set up

Illumination set up

•Bright Field

•Axial

•Dark Field

•Comparison

Optics

Frame grabber

Software

IMAGE PROCESSING

ALG

ORITHMS Bright Field

•FFT approach

Automedian

Dark Field

•Single step thresholding

•FCD method

Comparison

DEFECT CLA

SSIFICATION Image fusion

GLCM for image series

Thresholding algorithm

Bayes classifier

CHAPTER 4

IMAGING AND ILLUMINATION SYSTEM

4.1 INTRODUCTION

In a machine vision system, the sensor receives light rays reflected off the surface

being imaged using the camera system. The primary objective is to obtain maximum

details of all the defects that are to be inspected. If defective areas are visible in the

image with very high contrast a better analysis of the inspected surface is possible.

This is dependent on the machine vision set up including the camera, illumination,

lenses etc. There are different kinds of illumination that are possible and the decision

about illumination is mostly governed by the nature of the surface being inspected.

The usually considered model for illuminating surfaces is the Lambertian surface,

which causes diffuse reflection and where the brightness of the illuminated surface is

unchanged depending on the direction of observation. In day to day life, this is the

most widely encountered type of surface and many reflection models can be

developed on the basis of this model. But in the case of metallic objects, the reflection

conditions are based on the specular lobe of reflection. All the rays of incident light

are scattered or reflected predominantly in a single direction, which is usually referred

to as specular lobe of reflection (León, 2002). The reflection characteristics of the

different surfaces are shown in Figures 4.1 and 4.2. The ring surface under

consideration in this work is an example of specular object and produces directional

reflection on being illuminated.

Figu

Figu

4.2

This

ima

obje

with

ure 4.1

ure 4.2 R

BRIGH

s type of ill

age that reve

ect boundar

h regards to

Lambertianpossible dir

Reflection o

HT FIELD

lumination i

eals a lot o

ries are visib

o different ty

n surface shrections

of light on s

ILLUMIN

is well suite

of informatio

ble in the im

ypes of illum

33

howing the

specular sur

NATION

ed for inspe

on about it.

mage. A det

mination is

incident li

rface

ecting diffus

. More info

tailed explan

presented l

ight being

se objects a

ormation im

nation of th

later. The so

scattered in

and produce

mplies that m

his phenome

ource of lig

n all

es an

more

enon

ght is

34

a ring shaped fluorescent lamp. The light is incident as shown in Figure 4.3a. The

characteristic name ‘bright field illumination’ comes from the nature of the image,

which is the whole image area looking very bright. This is one of the most commonly

used illumination systems.

The basic set up for the inspection remains the same for different illumination systems

except the change in position of illumination. The camera used is PULNIX TMC 6

analog camera. The camera is connected to Matrox Corona frame grabber which

facilitate conversion of analog signal from camera into digital images in the PC to be

used for further processing. Matrox Imaging Library is used to capture and to store

the image. Later the images have been processed using Labview or Matlab software.

Optics which is a convex lens attachment is used to form the image in the machine

vision sensor with the required magnification. An aperture can be used to control the

amount of light flooding the machine vision sensor axis. Here 4X objective is used in

microscope and it is attached using a custom made adapter for C-mount lens.

According to the requirement, higher magnification objectives can be used. The

lighting for bright field illumination was made using a circular fluorescent bulb. For

other schemes of illumination, halogen or LED based illumination sources can be

used. Also the direction of illumination changes according to the type of illumination.

For bright field illumination set up the circular fluorescent bulb is concentric with the

camera axis. For using axial illumination which is a special case, a beam splitter was

used. Beam splitter reflects the incident light into the direction of the axis of the

camera and at the same time it transmits the reflected light from the object under

inspection into the camera through the lens system. The ring to be inspected is kept on

an indexing mechanism which can be used to change its position to enable 360°

inspection, step by step.

35

Four camera positions are required to image the critical regions of the ring. Camera

position-4 is used to image two surfaces that are at 103° angle. So the point of

intersection and a small part of both the surfaces can be imaged with camera aligned

bisecting the angle thus giving 52° orientation with respect to vertical for camera-4.

Camera position-1 is used to image the surfaces that are at 112° angle, the bisector of

which is 56°. It has been observed that the imaging is not much affected with ±4°

variation in camera angular position. Angular position of 52° may give an option to

image the ring surface corresponding to camera position-4, in case one camera is to be

avoided (using one camera for both camera position-1 and camera position-4, by

imaging the current surface and the surface corresponding to camera position-4 by

suitably translating the ring or camera since the surfaces are diametrically opposite).

Camera position-2 is for imaging the top surface which is at 6° and thus camera

position can be 6±4° and an angular position of 8° have been decided and used for the

current application according to convenience in installation. Camera position-3 is

used to image the curved surface. If a tangent is drawn to the curved surface, the

normal to the tangent is at 52° with respect to vertical and hence the angular position

value of 52° for that camera.

The calibration of the set up has been done by capturing image of a physical object of

known dimension. The dimension divided by the number of pixels between the edges

in the image will give the calibration value. This has been done for both x direction

and y direction.

The set up discussed in Chapter 6 was developed at Messsystem und Sensor Technik,

Technical Universität München, Munich, Germany. The imaging system consists of a

36

Firewire (IEEE 1394) camera. This is a digital camera which means the image can be

directly captured and stored in the computer system in the form of a digital image

without using an intermediate frame grabber. The Apple Macintosh system can be

directly connected to the imaging system through the IEEE firewire port which is

available in all Apple systems. The images are captured using the Macintosh based

image capturing software and then processed using the standard software as discussed

before. The optics consisted of a convex lens of focal length 6cm and a variable

aperture to control the amount of incident light on the machine vision sensor.

OSRAM LED light sources were connected to a 2.4 volt 700mA DC power source

through a junction box facilitating selective lighting of each of them. The whole set

up was enclosed in a dark light isolating case to prevent disturbance from external

light sources.

4.2.1 Axial Illumination System

Axial illumination system is a special case of bright field illumination. The axial

illumination set up has the direction of illumination and the axis of the camera

coinciding. This is attained by using a beam splitter at the camera axis which allows

illumination of the specimen from a direction perpendicular to the axis of the camera.

The reflected beam from the specimen reaches back through the beam splitter to the

camera. Figure 4.3b shows the system which can use either the axial illumination or

the circular illumination using two separate electrical switches. The images of the ring

taken under bright field illumination are presented in Figures 4.4 -4.12.

37

Figure 4.3 Experimental setup used for imaging of components using a vision

system: (a) Schematic representation of the setup used, showing only camera-2 position, as in Figure 4.3c (b) Photograph of the setup (c) The camera positions for imaging different surfaces.

(a) (b) (c)

(d) (e) (f)

(g) (h) Figure 4.4 Images showing non defective surfaces under bright field illumination

Figu

Figu

(a)

(d

ure 4.5 I

ure 4.6 I

d)

Images sho

(a)

Images sho

wing deep l

wing yellow

38

(b)

(e)

line under b

(b)

(d)

w stain unde

bright field i

er bright fie

(c)

illumination

(c)

eld illumina

n

ation

Figu

Figu

(a)

(d

ure 4.7 I

(a)

(d) ure 4.8 I

)

d)

Images sho

)

Images sho

wing blister

wing built u

39

(b)

(e)

r under brig

(b)

(e) up under br

ght field illu

right field ill

(c)

(f)

umination

(

lumination

)

(c)

Figu

Figu

(a

ure 4.9 I

(a)

(d)ure 4.10 I

a)

(d)

Images sho

)

) Images sho

wing damag

wing peel o

40

(b)

(e)

ge under br

(b)

(e) off under br

right field il

ight field ill

(c

lumination

(

(f)lumination

)

(c)

f)

Figu

Figu 4.3

In m

ima

chal

ima

insp

stud

refle

(a)

ure 4.11 I

(a)

(ure 4.12 I

DARK

microscopy

age and fo

llenging. Th

ages by usin

pected surfa

died using

ection and

)

(d) Images sho

(d) Images sho

FIELD IL

application

or highly

he dark fie

ng an illum

ace (Figure

this strateg

thus receiv

wing pitting

wing rough

LLUMINAT

ns, attempts

reflective

eld approach

mination so

4.13). Typi

gy. The cam

ves only li

41

(b)

(e)g under brig

(b)

(e) h finish unde

TION

s are always

or transpa

h is a meth

ource that s

cally, non-L

mera is ke

ight rays th

ght field illu

er bright fie

s made to im

arent objec

hod of imp

sends rays

Lambertian

ept away fr

hat have be

(c

(f) umination

(c

eld illumina

mprove the

cts, this b

proving the

at a grazin

surfaces ca

from the sp

een scattere

c)

c)

(f) ation

e contrast of

becomes re

contrast of

ng angle to

an be image

pecular lob

ed by the n

f the

eally

f the

o the

ed or

e of

non-

unif

met

Figu

Dar

prep

and

be p

the

back

obje

dark

cont

In t

syst

illum

form defect

thod of illum

ure 4.13

rk field illum

pared surfac

reducing im

predominan

name. Th

kground. T

ects and in

k field illum

trast and thu

the case of

tems are us

mination w

ive surface.

mination.

Principle odefective sa

mination is

ces. It has th

mage proce

ntly dark, if

e presence

This is a co

the inspecti

mination w

us required

f most of th

sed (Figure

would result

. Chrome co

of dark fielample.

applied in

he advantag

essing steps

f there are n

of abnorm

ommonly us

ion of highl

were free fro

less proces

he machine

2.1). Howe

t in very b

42

oated ring su

ld illumina

microscopy

ge of highlig

required fo

no abnormal

malities wi

sed procedu

ly reflective

om the effe

ssing steps f

e vision ap

ever, if the

bright regio

urface is we

ation; (a) d

y for transp

ghting the f

or certain ap

lities in the

ill figure a

ure in the m

e surfaces. T

ect of spec

for segment

plications,

surface is

ns in the i

ell suited fo

defective sa

parent, plana

features wit

pplications.

imaged sur

as bright re

microscopy

The images

ular reflect

tation of def

axial surfa

reflective a

image getti

or this partic

ample; (b)

ar and spec

th good con

The image

rface and he

egions in

y of transpa

s captured u

tions, had g

fects.

ace illumina

and curved,

ng highligh

cular

non

ially

ntrast

will

ence

dark

arent

using

good

ation

this

hted,

43

which suppress the surface details and get detected as boundaries when standard

image processing algorithms are used. Whereas in the case of dark field illumination,

the light rays strike the component at an angle (Figure 4.13) and as a result, flooding

of the machine vision sensor from the on-axis light from the main source of

illumination is reduced (Martin, 2007; Connolly, 2002). The presence of some foreign

particles will reflect the light back into the optical axis (Figure 4.13a). The non

defective surface sends rays out of the machine vision sensor (Figure 4.13b). The

schematic and laboratory setup for dark field imaging is shown in Figure 4.14a and

Figure 4.14b respectively.

Images of defective and non defective regions of different ring specimens have been

captured using bright field and dark field illumination and are shown in Figures 4.4 –

4.12 and Figures 4.15 – 4.23 respectively. Figure 4.15 shows a non defective ring

surface. The whole image is dark and this implies that there are no defective regions.

Non defective surfaces imaged under bright field illumination have been shown in

Figure 4.4 for comparison. Figure 4.16 shows a surface image with blister defect. It is

visible as conglomeration of bright regions and is clearly visible with high contrast in

the dark field image. This implies that the defective region shown in the image can be

segmented out with minimum image processing steps. The corresponding defective

region under bright field illumination is shown in Figure 4.7. The bright streaks

present in bright field image are absent in the dark field image. Figure 4.17 shows

deep line defect which is visible as straight bright line in the dark field image. This

defect is due to the metal cutting process and not just due to the coating process. The

defect occurs around the ring surface as a feed mark. The dark field image (Figure

4.17) reveals the defect area with more details as compared with that shown in Figure

44

4.5, which will aid in quicker and accurate segmentation of defective region. Damage

is a similar defect which occurs due to manufacturing process prior to the coating

process. The coating in this case is without defect but the surface on which coating is

done is defective. Figure 4.18 shows damage region under dark field illumination.

The defect is seen as a very small bright region in this image. Just like deep line,

damage defect also is hard to segment out from bright field image but not so in the

case of dark field since it is comparatively more prominent. Figure 4.19 shows rough

finish which occurs as a defect which continues around the ring surface. Since the

spread is more, it can be easily detected and is visible clearly in both dark field and

bright field illumination (Figure 4.12). It is a defect which is more clearly visible

under bright field illumination. Dark field illumination is suitable for detecting small

sized defects. Figure 4.20 shows the images of pitting defect which appears as small

bright spots and as small clusters. It is distinctly visible both in bright field and dark

field images (Figure 4.11). Built up is visible as a bright spot without much spread

and is visible in both the illuminations (Figure 4.21 and Figure 4.8). Peel off is due to

the absence of coating over a large area and is visible at the boundaries as bright

regions in dark field image (Figure 4.22). This defect is distinctly visible in bright

field image shown in Figure 4.10 also. Figure 4.23 shows the image of the surface

with yellow stain which is visible as yellow colored regions in the inner diameter of

the ring (Figure 4.6). Though it is difficult to detect the defect, there is a unique dark

pattern in the dark field image which implies that it can be processed to segment out

the defective region. It can be concluded that most of the defects can be imaged with

greater contrast using dark field illumination.

Figuillum

ure 4.14 mination; (a

Set up foa) Schemati

or inspectioic; (b) Labo

45

(a)

(b) on of ringoratory set u

g outer diup

iameter usiing dark f

field

Figu

Figu

(a)

(d)

ure 4.15 I

(a

ure 4.16 I

Images sho

a)

(d) Images sho

wing non d

wing blister

46

(b)

(e)

defective sur

(b)

(e)r under dark

rface under

k field illum

(c

(

dark field i

(c

(f)mination

)

(f)

illumination

c)

f)

n

47

(a) (b) (c)

(d) (e) (f) Figure 4.17 Images showing deep line under dark field illumination

(a) (b) (c)

Figure 4.18 Images showing damage under dark field illumination

(a) (b) (c)

(d) (e) (f)

Figure 4.19 Images showing rough finish under dark field illumination

48

(a) (b) (d)

(e)

Figure 4.20 Images showing pitting under dark field illumination

(a) (b) (c)

(d) (e) (f)

Figure 4.21 Images showing built up under dark field illumination

49

(a) (b) (c)

(d) (e) (f)

Figure 4.22 Images showing peel off under dark field illumination

(a) (b) (c)

(d)

Figure 4.23 Images showing yellow stain under dark field illumination

50

4.3.1 Implementation of Dark Field Illumination

The camera, oriented with optical axis normal to the surface of inspection along with

the ring indexing mechanism has been used to create images of all the defect prone

surfaces of the ring which are to be inspected. Four different camera positions are

required to create images of all the required defect prone surfaces and they are shown

in Figure 4.3c (The same positions are used for both the bright field and dark field

illumination set ups). The geometry of the ring surfaces to be inspected has been used

to decide the location of these cameras and this was discussed in Section 4.2.

The proposed system in the industry will have four cameras oriented in these

positions. In the present study, a single camera has been used to take images for

development of algorithms for defect inspection. Figure 4.3a and Figure 4.3b show

the camera being oriented to inspect a specific area on the top of the ring. The

resolution of images captured by the camera is 816 x 612 pixels, where each pixel

represents 5.7 μm x 5.7 μm ring surface area. This explains the smallest defect size

that can be detected which is 10.14 μm i.e. the presence of two defect pixels. The

algorithms were implemented on Intel Core 2 Duo, 2.8 GHz PC with 2GB RAM

using National Instruments Labview software, such that it will be compatible with the

proposed set up in the industry. Equal number of non defective sample images and

defective sample images were used for testing the algorithm. The performance of the

four approaches has been evaluated based on 210 images of ring samples obtained

from the manufacturers.

The expression for accuracy as given by Juran et al. (1994) is,

51

Accuracy of inspector = Percentage of defects correctly identified= (4.1)

Where,

d = defects reported by the inspector

k = number of defects reported by the inspector but determined by the check inspector

not to be defects

(d – k) = true defects found by the inspector = correct detections by the algorithm

b = defects missed by the inspector, as determined by check inspection

(d - k + b) = true defects originally in the product = the total number of samples of

known status that is supplied to the algorithm

The expression is defined especially for correct detection of defective surface

(without considering correct detection of non defective surface). Therefore, in this

study, the accuracy of the algorithm was found out using the expression,

(4.2)

Number of defective sample images, classified as defective by algorithm

Number of non defective sample images, classified as non defective by

algorithm

N Total number of images

Therefore the equation 4.2 can be rewritten as,

Accuracy = N

T (4.3)

For example, if 100 defective image samples are input to the algorithm and the

52

algorithm correctly detects say, 80 of them; accuracy will be 80%. The expression for

accuracy holds true for 100% visual inspection. In the automation of visual inspection

the accuracy of the method is determined by first getting the samples tested by the

inspection system and then by an experienced inspector. A similar methodology has

been followed in this case also. The quality control team from the industry had

supplied chrome coated rings with the defective areas carefully marked out. The

images of these defective regions of the samples were used to test all the algorithms.

This was done by running the algorithm on the images that were predetermined to be

defective and then evaluating the results in terms of number of sample images

detected as defective. The same procedure was repeated for non defective regions of

the ring.

4.4 SUMMARY

A machine vision experimental set up for inspecting the chrome coated ring surface

has been made. All the critical regions of the ring component can be imaged and

processed for presence of defects. Classification can be done if defects are present.

The complete system is a result of a proper organization of camera, frame grabber,

computer system, software, optics, illumination and object locating mechanisms. It

has been observed that the image quality is highly dependent on the type of

illumination to be used. Different illumination methodologies are possible by

variation in the position of illumination. The images of the coated ring surface were

taken under different illuminations namely dark field and bright field. The basic

principle behind each illumination set up has been explained in detail. Each of the

defects was imaged using both the illumination systems and the resultant image has

been studied. The use of suitable magnification lenses and camera resolution will

53

enable detection of smaller sized details. Cameras at different angular positions will

enable inspection of all the defect prone areas of the ring. The indexing mechanism

allows for inspection of entire 360° of the ring surface. As a result all the defect prone

areas of the whole ring can be successfully inspected. The method of evaluation of the

accuracy of the defect detection system has also been discussed.

The types of illuminations used are bright field, axial and dark field. The aim of the

imaging set up is to capture images of the surface to reveal complete details with

suitable magnification. The details should have good contrast, such that they can be

identified and separated out very easily by a simple thresholding algorithm. Bright

field illumination reveals more details about the surface under inspection. The

defective regions are shown as darker areas in the image of the ring surface. There are

significant specular reflections, which might complicate the image processing

algorithm that is to be developed at the next stage. Though axial illumination, which

is a special case of bright field illumination, has similar characteristics, it gives lesser

specular reflection from the surfaces that are near the inspected area. But specular

reflections are still prevalent and this might lead to the loss of some information in the

final image. In dark field illumination, the illumination hits the surface of inspection

at oblique angles. As a result, there is very less specular reflection back into the vision

sensor. Those surfaces that are with abnormalities will be specially highlighted in the

image. This might make the image thresholding operation very simple and fast as

compared to that when the bright field images are used. Though the scattered light

from the surface under consideration is important and influences the image quality,

there is a possibility of losing some light reflected from the inspected surface. With

the experimental set up to capture the images of defective surface images at various

angles and under various illuminations, experiments and analysis have been carried

54

out and the details are presented in the following chapter.

CHAPTER 5

DEFECT IDENTIFICATION

5.1 INTRODUCTION

Image processing methodology to be adopted generally is based on the type of

illumination, the characteristics of object under inspection, the data to be extracted

and type of machine vision system. Though some of the image processing procedures

are adaptable, most of the new applications require newer and different techniques of

pre-processing of the images. The complete vision system has to be designed based on

the inspection requirement which vary for different applications. In many of the

industrial applications the size, dimension, pattern or shape of the objects are

assessed. The application discussed here in this work is principally to detect the

presence of defects and classify them. Different methodologies tried out with regards

to the kind of illumination will be discussed in this chapter. Since speed and accuracy

are the major requirements, each of the methods will be analyzed for suitability,

according to the speed and the accuracy at which the inspection process can be

completed.

5.2 BRIGHT FIELD ILLUMINATION

The presence of more details in the image than what is actually required makes it

difficult to separate out the defective region effectively from the rest of the details. As

a result, image processing is made more complex and error prone. The initial images

of coating defects received from the industry were taken using bright field

illumination technique and as a result, the initial approach was to segment out the

55

defects using those bright field images. The Fast Fourier Transform filter was tried out

for this purpose and the procedure is explained below.

5.2.1 FFT Filtering Based on Critical Radius

This approach is based on the Fourier image reconstruction proposed by Tsai and Kuo

(2007) which they had used for analyzing sputtered surfaces. The defects are known

to be normally the high frequency details in the image. Hence a high pass filter in

frequency domain was used for defect detection.

Conversion from spatial domain to frequency domain was done using 2D FFT given

by,

11

0 0

( , ) ( , ) exp 2yx

NN

x y x y

ux vyF u v f x y j

N N

(5.1)

for frequency variables u = 0, 1,2…Nx-1

and v = 0, 1,2…Ny-1

The power spectrum was calculated from the real and imaginary components of the

FFT. That is

2 2 2( , ) ( , ) , ,P u v F u v R u v I u v (5.2)

Where, R(u,v) and I(u,v) are the real and imaginary parts of F(u,v), respectively

56

The logarithmic value of the amplitude of the FFT of the image is shown in Figure

5.1a. Logarithmic value is useful when it comes to visualizing sudden variations in

value. A radial line from the center of the image to the rightmost end of the image has

been considered and the intensity of the logarithmic plot along this line has been

studied. It was observed that after an initial surge (due to contribution from metallic

streaks) in the value of intensities, the plot begins to stabilize at a particular intensity

value (Figure 5.1b). This point is of significance, since it is the point at which all the

lower frequency components are eliminated and the higher frequency components

start to appear. The radial cut-off distance has to be close to this point. To find the

exact cut-off radius, a disk of increasing radius was used to mask all the intensity

values proceeding from the center outward. The mask size is the radial distance along

the radial line from the center of logarithmic FFT image. Figure 5.1c shows

logarithmic value of the amplitude of the FFT of gray scale values of the image and

the overlapping disk of optimum diameter. Corresponding plot of intensity along the

radial line is shown in Figure 5.1d. Original image before Fourier filtering is shown in

Figure 5.1e. The image is reconstructed using 2D inverse FFT after making zero all

the intensity values within the disk of optimum size (Figure 5.1f). This reconstructed

image predominantly gives the defective area segmented out from the statistical

texture. After running it on many images, the frequency corresponding to the optimum

size of the disk has been found as 60% of maximum frequency. In the algorithm for

the FFT approach discussed, this observation was used to filter out the FFT at single

step with a high pass cut-off frequency of 60% to save processing time and to reduce

the complexity of the approach. This process of training the model and then using it to

predict future results is a methodology followed in classifier design for pattern

recognition algorithms. The filtered frequency image was again converted to spatial

domain, using 2D inverse FFT. The reconstructed image showed the high frequency

57

details (defects), which were then segmented by thresholding.

(a) (b)

(c) (d)

(e) (f) Figure 5.1 FFT filtering(a) FFT image (b) Intensity (logarithmic value of FFT

amplitude) variation along the radial line indicated in Figure 5.1a (c)

Radial line and disk of optimum radius used to mask the intensity

(logarithmic value of FFT amplitude) values within it (d) Intensity

(logarithmic value of FFT amplitude) variation in Figure 5.1c along the

radial line after the filtering operation using the mask (e) Original

image used (f) Reconstructed (Inverse FFT) image of Figure 5.1c.

Intensity

Distance along profile (pixels)

Intensity

Distance along profile (pixels)

58

5.2.2 Auto-Median Based Approach in Image Segmentation

The accuracy of the detection was improved when the reconstructed image was

subjected to a contrast enhancement and then thresholded. Later, the auto-median

morphology operation was used to remove the small objects which used to produce

false detections. The expression for the auto-median operation is given below

· · · (5.3)

Where,

operation

· operation

M – 3 x 3 Structuring element

M = 1 1 11 1 11 1 1

I – Image

Auto-median operation( ) on image(I), using a 3 x 3 square structuring element(M),

is a morphological operation which consists of producing two images by

implementing opening( ) and closing(·) operations in the sequence shown in equation

5.3, and then applying logical AND( ) operation on them (Thomas, 2003). Opening

is erosion followed by dilation whereas closing is dilation followed by erosion. Auto-

median is used to remove noise (which might appear as defect in the FFT approach)

in the analyzed image.

59

5.3 DARK FIELD ILLUMINATION

From the images of defective area under dark field illumination it can be understood

that a simpler thresholding operation may suffice for image segmentation of these

images. Just as bright field illuminated images are processed by auto-median based

algorithm, a new algorithm has been developed to successfully detect defects and then

segment them out while inspecting rings using dark field illumination. Figure 5.2a

shows the histogram of the image of a non defective component under dark field

illumination and Figure 5.2b shows the first derivative plot of the histogram. The first

derivative is given by,

1 (5.4)

Where,

.

It can be observed that, lim 0. Histogram in Figure 5.3a (of defective

sample image) does not show any major spike in the number of pixels but there is

noticeable spike in the plot of at g→255 (Figure 5.3b) unlike in the case of non

defective sample mentioned before. This observation has been used to fix a threshold

value of 230 for all the dark field images, such that details which are corresponding to

the defects are segmented out. The major spike in the plot of h’(g) corresponds to the

boundary between the ring surface and the background (visible at g≈50 in Figure 5.2b

and Figure 5.3b. The ring surface information is visible from g=50 or g=70. The

defective region if present will be visible above the value of g=70. Thus the lower

60

limit of G is defined as g=70. The dark field illuminated images after applying this

threshold is shown in Figure 5.4. After thresholding, the number of particles or the

number of white (g=255) pixels can be counted. Here the number of particles (any

inter connected group of white pixels) can be counted as a single particle. After

testing threshold g=230 for a large sample of images it has been found that for a

threshold value of 230, detection of a single particle deems the component to be

defective. Lower the threshold value, more the particles will be detected but at the

same time non defective areas may also get detected as defective areas. This is

illustrated in Figure 5.4.

(a) (b)

Figure 5.2 Image characteristics for non defective surface; (a) Histogram of image of non defective surface of ring under dark field illumination, (b) Plot of first derivative of h(g) in 5.2a where h’(g)≈0 as g→255

(a) (b) Figure 5.3 Image characteristics for defective surface; (a) Histogram of image of

defective surface (blister) of ring under dark field illumination, (b) Plot of first derivative of h(g) in 5.3a showing the peak in the value when g→255

Gray scale values

First Derivative

Gray scale values

First Derivative

Peak

61

Original dark field image

Thresholded image using FCD approach

Thresholded at g=127

Thresholded at g=230 (Few white pixels present for

defective)

Blister (a) (b) (c) (d)

Yellow stain (e) (f) (g) (h)

Blister (i) (j) (k) (l)

Non defective False detection Few white pixels

detected No white pixels

detected (m) (n) (o) (p)

Figure 5.4 Effect of using different threshold values Note: White pixels represent the defective regions. Images in each row are obtained by thresholding the images shown in the first column. Images were taken from different rings.

62

5.3.1 FRACTIONAL CHANGE IN DERIVATIVE (FCD) METHOD FOR

FINDING OPTIMUM THRESHOLD VALUE

FCD (Zhang et al., 2006) can be used to segment out the defect completely from the

surrounding non defective surface in the image of ring surface. The basics of this

approach are explained in the following paragraph. Details in segmented image can be

used to quantitatively assess the type of defect. The image has to be segmented at the

point of steep increase in the histogram plot of dark field image. This can be found

out using the value of the variation of slope of histogram plot. Figure 5.4b, Figure 5.4f

and Figure 5.4j show the defective region segmented out distinctly by using this

observation. For finding optimum threshold g’, the increase in the value of with

respect to the immediate neighboring gray scale value has been found out. This is

given by,

FCD(g)= 1 / (5.5)

g’: =maximum value of FCD(g) in G, where (5.6)

G={ , 255 70

FCD(g)=Fractional Change in Derivative

g’=optimum threshold value

The value g=70 has been found to be the least observed value for getting the defects

segmented out for all the thresholded images and hence the lower limit for G was

fixed as g=70. In Figure 5.4 c, Figure 5.4g and Figure 5.4k the threshold value was set

close to half of the maximum value of 255 for illustration purpose. This gives a fair

idea about the approximate spread of the defective area, if present in the image. Both

the threshold values obtained by maximum FCD(g) and g=127 as it is in this case,

63

fails for non defective images. Hence a very high threshold value of 230 was fixed

such that none of the non defective areas are segmented out as defects and such

images are shown in Figure 5.4d, Figure 5.4h and Figure 5.4l. The image of non

defective sample in Figure 5.4m has been subjected to thresholding based on all the

three values and the respective resultant images are shown in Figure 5.4(n), Figure

5.4(o) and Figure 5.4p. It can be found that only from Figure 5.4p with threshold

value, 230 as discussed before; resultant image is obtained with no objects being

detected. The conclusions based on these observations are given below.

1. Threshold value of 230 can be used to identify the presence of defects on the

imaged surface.

2. After ascertaining the presence of defects, the maximum FCD(g) method can

be used to correctly segment out the defects for further processing. This may

be used only on images of defective regions after the identification of defect

by using single threshold value of 230, since the non defective images will

give erroneous results when directly operated upon by the algorithm.

3. The images of non defective surfaces will not show any distinct area

representing defects while threshold value of 230 is used.

5.4 DEFECT CLASSIFIER USING BRIGHT FIELD ILLUMINATION

IMAGES

Defect classifier need to be used, only if the image is found to be having a defective

area. The defect classes for classification problem were redefined as,

64

1. Blister – Defect class containing blister defect samples. This was kept as a

separate class because the defect contained larger spread of small objects as

compared to the other defects. This is non-repairable defect.

2. Built-up – Defect class containing built up defect samples. This was also a

separate class because this kind of defective ring can be reworked to salvage

the ring.

3. Damage – Defect class containing all the other kinds of defects for example;

damage, peel off, pitting, deep lines. This is non-repairable defect.

The defective region parameters that have been considered while designing the

classifier are:

1. Area- Scalar which is equal to the actual number of pixels in the region.

2. Perimeter- The length of the boundary of a region.

3. Extent - Scalar that specifies the proportion of the pixels in the bounding box

those are also in the region. Computed as the pixel area of defect divided by

the pixel area of the bounding box.

5.4.1 Classification Parameter Selection

Though, the segmented defect area can be characterized by different parameters, only

in few of them may be able to give an efficient classification of defects (Duda et al.,

2001). The parameters that give clear classification between the classes of defects can

be used for classification. For this purpose conditional probability of different kinds

of d

peri

clas

the

the

can

The

Figu

5.5

5.5.

In a

pixe

defect param

imeter/area

ssification a

curve perta

parameter

be underst

erefore these

ure 5.5

RESUL

.1 Image P

all the algor

els) with an

meters have

and exte

and this is i

aining to bl

clearly sepa

tood that b

e two param

Conditionadamage cextent/area

LTS AND D

Processing

rithms, to re

n area of mo

been plotte

ent/area wi

llustrated in

lister defect

arates out t

uilt up def

meters can b

al probabiliclasses (a) a(Ex/A)

DISCUSSIO

Algorithm

educe the e

ore than 5 b

65

ed. It has be

ill serve t

n Figure 5.5

t class is se

the blister d

fect class ca

be used for t

(a)

(b)

ity density Plot of

ONS

ms for Brigh

effect of noi

by 5 pixels,

een found th

the purpos

5 (Eiden et

eparate from

defects. Sim

an be easily

the classifie

functions perimeter

ht Field Im

ise, the obj

, were set a

hat two para

se of givin

al., 2000).

m the other

milarly from

y separated

er.

for blisterr/area(P/A)

ages

ects (repres

as defects. A

ameters nam

ng very g

In Figure 5

ones, impl

m Figure 5.5

d from the

r, built up (b) Plot

sented by w

Any presenc

mely

good

5.5a,

ying

5b it

rest.

and t of

white

ce of

66

such objects will imply that the work piece sample that has been imaged is defective,

and the pixel count or pixel area of such particles will indicate the intensity of the

defect. The FFT approach for defect detection on bright field images is shown in

Figure 5.6. This approach has been discussed by Tsai and Kuo, (2007) to detect

defects in sputtered surfaces. In this particular case, non defective ring surfaces were

not correctly detected from the corresponding images (Figure 5.7). Figure 5.7b shows

the plot of intensity function after filtering frequencies below 60% in the FFT of

image in Figure 5.7a. Figure 5.7d shows the final reconstructed image after the

filtering operation. There are some frequencies in a non defective component image

that were above the cut-off frequency and appeared as defective areas in the

segmented images. There is no ideal cut-off radius or frequency for defects which

would segregate only the defective region. It could be due to the fact that the defects

in the components are diverse in nature, localized and prominent, unlike the defects

that occur on the sputtered surfaces.

The auto-median operation was appended to the Fourier filtering approach in an

attempt to improve the accuracy. This approach removes the smaller particles left

behind, after the Fourier filtering method and the final resulting image is shown in

Figure 5.8f. Figure 5.8d is obtained after contrast enhancement on the image obtained

from the FFT approach. The resulting image is thresholded and then closing operation

is done (Figure 5.8e). The final image, with defects segmented out, is shown in Figure

5.8f and this is the result of auto-median operation and removal of border objects. The

89% accuracy of the system in detecting and classifying the components as defective

and non defective parts, was at the cost of increased time of inspection to 0.292

seconds from 0.248 seconds for the FFT method. The reason for this is the fact that an

additional auto-median operation was used after the Fourier filtering.

67

Figure 5.6 Image segmentation using FFT approach to highlight the defective region in bright field image: (a) Original image of blister (b) Logarithmic plot of intensity as a function of radius from center of FFT power spectrum (c) High-pass filtered FFT power spectrum, showing radial line considered (d) Logarithmic plot of intensity as a function of radius from center of FFT power spectrum in 5.6c (e) Reconstructed image of 5.6c.

Figure 5.7 Failure of the FFT method on bright field images of non defective surface: (a) Original image of non defective surface (b) Logarithmic plot of intensity as a function of radius from center of FFT power spectrum (c) High-pass filtered FFT power spectrum, showing radial line (d) Reconstructed image of 5.7c

As mentioned earlier, the rings are to be inspected for all the defect-prone areas by

indexing, which necessitate the reduction in run time for an image. Finally, it was

observed that the performance of the auto-median approach is good in terms of

average processing time and also has a fairly high accuracy of detection. Auto-median

68

approach which gave an accuracy of detection of 96% when only defective samples

were tested makes sure that very less defective samples will make it to the customer.

When only non defective samples were tested, auto-median approach gave an

accuracy of 81%.

Figure 5.8 Image segmentation using auto-median approach on bright field image: (a) Original image of blister (b) Value plane extracted from 5.8a (c) Reconstructed image of High-pass filtered FFT (altered by brightness and contrast changes to make details visible to reader) (d) Contrast enhancement of 5.8c (e) Close operation on 5.8d (f) Auto-median operation followed by removal of border objects in 5.8e

Different rings were inspected and the images of the defective as well as non

defective surfaces were captured. The algorithms for bright field image were applied

on these images. Figures 5.9 a-r show the resultant images after applying each of the

two algorithms on the defective sample image (Figures 5.9 a, 5.9 d, 5.9 g, 5.9 j, 5.9 m

and 5.9 p). Figures 5.10 a-i show the resultant images after applying each of the two

algorithms on a non defective sample image (Figures. 5.10 a, 5.10 d and 5.10 g). It

can be observed that FFT approach fails for many of the non defective surface images

unlike auto-median approach. Both the approaches correctly detect and segment the

defective region for images of defective surfaces.

Br

(

Figu

Wh

resu

surf

right field im

(a) Built u

(d) Peel of

(g) Rough fin

(j) Deep lin

(m) Damag

(p) Pittingure 5.9

en used und

ult, while im

face area wi

mage

up

ff

nish

nes

ge

g Performancsurface of rshown in tImages wer

der 4X mag

maging all

ill be in foc

FFT Appro

(b)

(e)

(h)

(k)

(n)

(q) ce of FFT aring. Note: the first colre taken fro

gnifications,

the surface

cus (approx

69

oach A

and auto-meWhite pixe

lumn is usem selected

, the depth

es other tha

imately 17m

Auto-median

(c)

(f)

(i)

(l)

(o)

(r)edian approaels represened to obtaindefective su

of focus is

an the top

mm) and th

n Approach

)

)

)

)

)

) ach on imag

nt defective n other imaurfaces on d

approximat

flat surface

e rest of the

ges of defecregions. Im

ages in the different rin

tely 2 mm. A

e, only a s

e surface im

ctive mage row.

ngs.

As a

mall

mage

will

resu

Suc

algo

was

B

Figu

5.5.

The

pres

5.11

are

l be blurred

ults while pr

ch an effec

orithm and t

s applied.

Bright field im

(a)

(d)

(g) ure 5.10

.2 Image P

e blister defe

sence of str

1d). Thus, t

four majo

d gradually

rocessing im

ct was reall

therefore ve

mage

Performancdefective sregions. Imin the row. different rin

Processing

fect imaged

reaks along

the defect d

or peaks an

y as the su

mages of cu

ly high for

ery low ove

FFT Appr

(b)

(e)

(g)ce of FFT surface of

mage shownImages we

ngs.

Algorithm

using brigh

with the d

detection ma

nd many m

70

urface heigh

urved surfac

r non defe

erall accurac

roach (

and auto-mring. Notein the first

ere taken fro

m for Dark F

ht field illum

defects make

ay not be e

more minor

ht varies. T

ce resulting

ctive samp

cy in result

Auto-media(No white pi

(

(

(median appe: White pcolumn is uom selected

Field Imag

mination is s

es the histo

easy as it w

r peaks eac

This might

in less accu

ples tested

s where the

an Approachixels detected

(c)

(f)

(h) proach on pixels repreused to obtad non defect

ges

shown in Fi

ogram multi

was discusse

ch indicatin

give erron

uracy of res

using the

e FFT algori

h d)

images of esent defec

ain other imtive surface

igure 5.11a.

i-modal (Fi

ed earlier. T

ng a partic

eous

sults.

FFT

ithm

non ctive

mages es on

The

gure

There

cular

71

important feature in the image. This shows that features other than this defect are

being focused upon. And when observed under axial illumination, this defect appears

as presented in Figure 5.11b. The streaks are still present, but the histogram (Figure

5.11e) reveals that the defect detection would be simpler than in the previous case.

Three major peaks are present and show that the number of features visible in the

image is less than that in the case of bright field illumination. But still, the specular

reflection is present and may get detected as a false defect. Further, the same defect

observed under dark field illumination (Figure 5.11c) resulted in uni-modal intensity

distribution (Figure 5.11f). This shows clearly that the features of the defect alone are

focused and the rest are neglected. Hence, the defect segmentation is now a simple

thresholding operation (i.e. simplified image processing). The setup used for such a

dark field imaging on OD of a ring is shown in Figure 4.14. With a suitable

orientation of camera and illumination, it is possible to image ID or any other curved

surface of the ring under dark field imaging conditions. Dark field illumination is

found useful, for inspecting inner and outer fillets of the ring, which was nearly

impossible when bright field or axial illumination systems were used.

Another kind of coating defect (peel off) imaged using axial illumination is shown in

Figure 5.12a. The histogram (Figure 5.12d) again follows the same trend as before.

The dark field illuminated image of the same defect (Figure 5.12c) gives uni-modal

histogram (Figure 5.12f). The mode corresponding to the bright pixel region occurs as

a sudden disturbance in the histogram towards the higher gray scale value. In short,

the dark field illumination gives an image which is uni-modal and thus easy to

threshold to detect the defects. The histograms for all the defects in different areas of

the ring follow the same pattern. All the images clearly highlight the defective areas

leaving out the details of areas that are non defective. The thresholding as discussed in

72

Section 5.3 helps in quick defect detection. The performance of defect detection

algorithms on a Core2Duo 2.8 GHz computer with 2GB RAM, for different imaging

conditions is shown in Table. 5.1. The information was deducted from a sample size

of 210 images for bright field illumination approach and 210 images for dark field

approach. Dark field images for different kinds of defects from different rings and the

corresponding thresholded images are shown in Figures 5.13 - 5.20. Dark field images

of blister (Figure 5.13), deep line (Figure 5.14), heavy blister (Figure 5.15), built up

(5.16), rough finish (5.17), pitting (5.18), damage (5.19) and non defective surface

(5.20) are processed and the resulting binary images are shown.

Table 5.1 Performance comparison of algorithms

S. No

Method/ Algorithm AccuracyRun time per

image 1 FFT on bright field images 57% 0.248 s 2 Auto-median on bright field images 89% 0.292 s

3 Single step thresholding on dark field

images 96.5% 0.047 s

Figure 5.11 Image of blister in different illumination systems; (a) Bright field; (b)

Axial illumination; (c) Dark field illumination; (d) Histogram of 5.11a, (e) Histogram of 5.11b, (f) Histogram of 5.11c.

Figu

Figu

Figu

ure 5.12

(a)ure 5.13

(a)ure 5.14

Image of pilluminationHistogram

) Single stering(showinthresholdinthresholdindetected)

) Single stepring(showinthresholdinthresholdin

peel off in n; (b) Axiaof 5.12a; (e

ep thresholng blister)

ng 5.13a ang 5.13a at

p thresholdng deep lin

ng 5.14a ang 5.14a at g

73

different ilal illuminate) Histogram

(b) lding for under dark

at gray scgray scale

(b) ding for imne) under daat gray scgray scale v

lumination tion; (c) Dam of 5.12b;

images ofk field illucale value e value g=2

mages of ark field illcale value value g=230

systems; (ark field il(f) Histogra

f blister; umination (

g=127 (c230 (Few w

deep line; lumination

g=127 (c0

a) Bright Flumination;am of 5.12c

(c) (a) Image

(b) Image ac) Image awhite pixels

(c) (a) Image

(b) Image ac) Image a

Field ; (d) c

e of after after s are

e of after after

Figu

Figu

Figu

(a)ure 5.15

(a)ure 5.16

(a)

ure 5.17

) Single stepring(showinafter threshthresholdin

) Single stepring(showinthresholdinthresholdindetected)

)

Single stepring(showinafter threshthresholdin

p thresholdng heavy bholding 5.1

ng 5.15a at g

p thresholdng build up

ng 5.16a ang 5.16a at

p thresholdng rough fholding 5.1

ng 5.17a at g

74

(b) ing for im

blister) und15a at graygray scale v

(b) ding for ip) under daat gray sc

gray scale

(b)

ding for imfinish) unde17a at graygray scale v

ages of heder dark fiey scale valuvalue g=230

images of ark field illucale value e value g=2

mages of roer dark fiely scale valuvalue g=230

eavy blistereld illuminaue g=127 (

0

built up; umination (

g=127 (c230 (Few w

ough finish;ld illuminaue g=127 (

0

(c) r; (a) Imagation (b) Im(c) Image a

(c) (a) Image

(b) Image ac) Image awhite pixels

(c)

; (a) Imagation (b) Im(c) Image a

e of mage after

e of after after s are

e of mage after

Figu

Figu

Figu

The

oper

whe

dete

(a)

ure 5.18

(a)ure 5.19

(a)

ure 5.20

e algorithm

ration using

ether the im

ection. This

)

Single stering(showinthresholdinthresholdin

) Single stering(showinthresholdinthresholdin

)

Single stepof ring(shothresholdindetected) (c(No white p

to identify

g a thresho

maged ring

is illustrate

ep thresholng pitting)


p thresholng damage


thresholdinowing damang 5.20a at c) Image afpixels are d

y the presen

ld value of

g is defecti

ed for differ

75

(b)

lding for under dark

at gray scgray scale v

(b) ding for i) under darat gray scgray scale v

(b)

ng for imagage) under d

gray scalefter thresholetected)

nce of defec

f 230. This

ive or not

rent images

images ofk field illu

cale value value g=230

images of rk field illucale value value g=230

es of non ddark field ile value g=1lding 5.20a

ct involves

single step

and is su

in Figures

f pitting; umination (

g=127 (c0

damage; umination (

g=127 (c0

efective surlumination

127 (Few wat gray sca

a single ste

p may be us

fficient for

5.13c, 5.14

(c)

(a) Image(b) Image ac) Image a

(c) (a) Image

(b) Image ac) Image a

(c)

rface; (a) Im(b) Image a

white pixelsale value g=

ep threshol

sed to ascer

r simple de

4c, 5.15c, 5.

e of after after

e of after after

mage after s are =230

ding

rtain

efect

.16c,

76

5.17c, 5.18c, 5.19c and 5.20c. The gray scale value of g=127 is used to ascertain the

spread of defects for illustration purpose in Figures 5.13b, 5.14b, 5.15b, 5.16b, 5.17b,

5.18b, 5.19b and 5.20b. The presence of a single white pixel implies the presence of

defects. If the defects are to be segmented out distinctly, the FCD approach can be

used. This information may be used to find out the actual spread and class of defects.

However, applying dark field illumination to specific areas of ring without mutual

interference of the light from multiple illumination systems will be difficult. Since the

machine vision sensor is away from the specular lobe of reflection from the reflective

surface in the case of dark field illumination (Figure 4.2), the intensity of light

reflected from the defects are always very less and in some cases might cause the

defects not to be detected. But still in the image, the defects are in very high contrast

from the other non defective surfaces and thus the defects are easily detected in most

cases. Finer details regarding the regular textured surface are lost, since it appears as

dark pixels in image and as a result, it cannot be used directly to quantify many of the

surface texture features. There is also possibility of pixels getting saturated due to the

powerful illumination that is being used which might be harmful to machine vision

sensor.

In the past, contrast enhancement using different illumination conditions were

attempted by earlier researchers on many other engineering applications (Piper, 2008;

Bamforth, 2007). Therefore, there is further scope here in this case as well, for using

dark field illumination with similar contrast enhancement for inspection of defects in

these ring components.

77

A single step thresholding algorithm (with a higher bound as threshold value) has

been run on 210 images, taken using dark field technique, and has given an accuracy

of 96.5% with running time of 0.047s. Out of the 105 images of non defective rings

that have been tested, 97% were detected as non defective. The accuracy of detection

for images of defective rings is 96%. The comparative performance is given in Table

5.2. This is a significant improvement over the other approaches. The images taken

using dark field illumination can be easily evaluated for the presence and absence of

defects. On the other hand, there is a significant loss of information, since major

portion of the light is reflected out of the machine vision sensor (Majority of which is

specular reflections but with some from the defective surfaces). As a result,

thresholding may not necessarily give all the details of the defects as in the case of the

other discussed algorithms, which are applied to bright field images. Single step

thresholding operation, using dark field imaging technique, cannot be used to

correctly quantify the extent and spread of defects but can be used as an easy and fast

way to identify the presence of defects. This can be understood from comparing

Figure 5.17 a-c and Figure 5.9 g-i. The spread of defect is not correctly visible in the

dark field image but represented clearly in bright field image. Added to this, is the

practical difficulty in setting up directed illumination systems for each part of the ring

to be inspected. As a result, each area to be inspected will require its own specific

illumination system.

The experimental set up in the laboratory was used only to evaluate the algorithm

developed and thus indexing was done manually. The proposed set up will have the

following specifications. The camera has a field of view of 46mm x 35mm (area).

There needs to be 360/60=6 indexes to cover the whole ring (where 600 is angular

value corresponding to each index). The processing time stated in the table is for a

78

single image captured at a single indexed position of the ring. An indexing mechanism

with 0.5s for a single index can be used. The maximum image processing time for the

auto-median approach on all the four angular positions of the camera and for all the

indexed positions of the ring is given by 0.155 x 4 x 6 = 7s/ring (The indexing time of

2.5s is included in this, since both are done in parallel). The time of inspection for

different approaches is given in Table 5.3. This proves that the entire inspection

process can be completed within 8 seconds, as required by the company, by using any

of these approaches.

Table 5.2 Comparative performance with respect to FFT approach on bright field images (FFT takes 0.248s at an accuracy of detection of 57%)

S. No

Method/ Algorithm Improvement in

accuracy Change in run time

per image

1 Auto-median on bright field

images 56% -17%

2 Single step thresholding on dark

field images 69% 97%

Table 5.3 Time of inspection per ring

S. No

Method/ Algorithm Run time per

image Time/Ring

1 FFT on bright field images 0.248 s 5.95 s 2 Auto-median on bright field images 0.292 s 7 s

3 Single step thresholding on dark field

images 0.047 s

1.1 s

5.6 SUMMARY

The advantages of using dark field illumination in particular have been highlighted in

this case study. Based on the results and analysis the following conclusions are drawn.

It was clearly observed that the dark field illumination was performing better

compared to bright field illumination in the sense that the image processing became

79

simpler because of the improved quality of images obtained. Therefore, the entire

process of defect detection became faster. When using the dark field illumination, the

effect of bands of reflective regions was absent. These bands were occurring due to

specular reflections in axial and bright field illumination. The presence of defect on a

dark field illuminated image can be identified using a single thresholding operation

rather than using other complex image processing algorithms like FFT approach and

auto-median approach which are discussed in Section 5.2. The entire process of

identification of defects by using simple thresholding operation was faster by

approximately 6 times (Table 5.3), when compared with algorithm based on auto-

median approach on bright field images. There is scope for further improvement if the

image processing set up is coupled with higher magnification lenses. In that case, very

small defects can be easily detected.

CHAPTER 6

DEFECT CLASSIFICATION USING MULTIPLE IMAGES

6.1 INTRODUCTION

An accurate and quick way to detect the presence of defects has been presented and

discussed in Chapter 5. Once the presence of defects is detected it has to be classified

into the corresponding defect class which is a normal requirement in many pattern

recognition applications. The basic steps to build a classifier are discussed in Section

1.3.4. The images of ring surfaces containing particular classes of defect have been

captured. For doing this a multiple illumination based method has been proposed. The

images were segmented using a thresholding algorithm and the defective regions were

separated out. The thresholded region has been quantified using different parameters

and the most suitable one was selected. The selected parameters are to be input to the

classifier. The classifier which satisfies the requirements of classification is selected.

Later the classifier is trained on the basis of gathered information. The classifier is

then tested for performance and depending on the performance all the intermediate

steps are fine tuned. This is the procedure used in this work for design of classifier.

The basic methodology to develop pattern recognition algorithm has been discussed

earlier in Chapter 1. In Chapter 5, the use and effectiveness of machine vision sensor

to detect the presence of defects has been discussed. The design and implementation

of machine vision sensor system is also explained. Now there is a requirement of

finding the most suitable parameter to be used in the defect classifier. There can be

different parameters that can be used in the classifier. Some of them are geometrical

and some are statistical. Also, the kind of algorithm to be used to process the

81

parameter needs to be identified. This will indeed help in the easy separation of rings

into their respective lots by the help of mechanical sensors.

Dark field images have the inherent disadvantage that the information from some of

the defective surfaces is lost due to reflection outside the angular field of view of the

sensor. As a result, the images in dark field reveal less information than bright field

images, though they amplify the light reflection from some of the defective regions.

The first approach used for bright field images using geometrical parameters makes

the defect detection more complex and less accurate. Parameters which aid in easier

and more accurate defect detection will help in more efficient defect detection and

classification. As discussed earlier, the modification in the initial manufacturing

process will result in some newer types of defects or some defect classes becoming

desolate. This has been experienced during the execution of the current project for the

ring manufacturer. As a result, the study has been conducted on a selected number of

classes of defects that are more frequent than the rest. The classifier can be modified

depending upon the changes in the inspection requirement in the industry.

The classes of defects that were considered in this particular approach are,

1. Blister

2. Built up

3. Damages

4. Pitting

5. Yellow stain

These were selected based on the requirement of the industry to classify the rings

specifically to avoid defective components being supplied to the customers.

82

6.2 MULTIPLE IMAGING SET UP

For a given component which is to be inspected, its underlying physical

characteristics remain unchanged for any angle of illumination or imaging. But the

specularities and the other illumination dependent aspects, as seen in the image, keep

changing their relative position; with the change in illumination set up. This aspect

can be used to find the defects separated out from the images of the inspected

component with minimum errors. This has been attempted by Khalili and Webb

(2007) using different wavelengths of illumination. In the current approach, three

illumination positions have been used, all of them giving partial dark field images.

Each illumination inclination and the position is similar to the one discussed in the

previous chapters. For discussion purpose, the images can be described as L1 for the

one with illumination at the left, L2 for the one with illumination at the centre and L3

for the one with illumination at the right sides.

The number of lighting combinations possible = 3C1+3C2+

3C3 =7,

Consider that Li is the case of ith illumination being used while image is captured.

3C1 is from using each of the illumination separately which is {{L1},{L2},{L3}}.

3C2 is from using two illumination simultaneously which is {{ L1 L2},{ L1 L3},{ L2

L3}}

3C3 is from using three illumination simultaneously which is { L1 L2 L3}

All the seven images were captured one by one and then given as input to the feature

separation algorithm. The number of images can be any depending upon the

feasibility of implementation. The image of the set up is given in Figure 6.1. In the set

up,

syst

thre

blis

Figu

Figu

an IEEE F

tem. The il

ee different

ter defect, c

ure 6.1

ure 6.2

irewire CC

lumination

illuminatio

captured usi

Experiment

(a)

(d)

Effect of vblister usin

D camera h

sources we

on positions

ing differen

tal set up fo

variation inng three diffe

83

has been att

ere three L

s. The imag

nt illuminati

or illuminati

(b)

(e)

(g) n illuminatiferent illumi

tached to a

LED (OSRA

ges of the s

on sources

ion with 3 i

ion positionination posi

suitable len

AM make) m

same area o

are given in

llumination

n (a-g) shoitions

ns and aper

mounted in

of ring show

n Figure 6.2

n sources

(c)

(f)

ows the de

rture

n the

wing

2.

efect

84

6.3 GRAY LEVEL CO-OCCURRENCE MATRIX (GLCM) APPROACH

FOR MULTIPLE IMAGES

GLCM approach is frequently used in texture measurements. Initially it was

formulated to segment satellite images of the earth for remote sensing purposes. The

basic concept is the correlation between a pixel and its nearby pixel. The neighbor to

the pixel is determined by the relations defined by ∆x, ∆y and ѳdir (Figure 6.3a). Gray

Level Co-occurence Matrix will have address indexes ranging from 1 to 256 in both

axes. Each entry in the matrix will be the number of times of occurrence of pixel

having x axis index value and y axis index value as neighbors, as defined by the

condition for neighbors. This is indicated in Figure 6.3b. This approach is particularly

useful while dealing with textures which have repetitive patterns. A host of statistical

parameters have been defined to quantify the GLCM matrix. Earlier, they were used

to classify based on single image only.

(a) (b)

Figure 6.3 GLCM computation (a) The conditions for being neighbors (b)Formulation of GLCM matrix for condition 00 [0,1].

Lindner et al. (2007) have explained how multiple illumination systems can be used

for the inspection of carbide cutting tool surfaces and the corresponding processing of

the image by using GLCM matrix approach adapted for multiple images. The

mathematical expression used in this approach is given below,

85

C∆ , ∑ ∑ ∑ , , . ∆ , ∆ , ∆ , (6.1)

1 0 (6.2)

The variables g1 and g2 are the gray values of the first and second pixel, respectively,

and the vector ∆ = (∆x, ∆y, ∆ѳ)T describes the considered displacement between two

pixels.

The increment used in this case was ∆ = (0, 0, ∆ѳ) where ∆ѳ had only two possible

increments since only three positions were used. A window of a particular length and

breadth and angular displacement has been considered and the matrix has been found

for each of the index of the pixels considering this window. The bigger window size

will result in the loss of the information from the image. A very small window size

will defeat the purpose behind using the GLCM approach. The optimum window size

has to be determined that would result in the image details getting amplified. For the

discussions which follow, window size of 3 pixels x 3 pixels is considered.

The dark field imaging has the inherent property of saturating the machine vision

sensor pixels thus leading to a loss of information. So, it is necessary to consider the

neighbouring pixels while arriving at any particular decision. This is the major

advantage of Gray Level Co-occurrence Matrix approach, since one cannot make a

decision based on the information given by certain stray pixels; especially when there

is a possibility of pixel saturation due to the enormous amount of light intensity at the

86

bright pixels. All these reasons make it well suited for applications warranting

repetition of pattern or lay.

But in the current approach, the lay can be defined across the images in the third

dimension considering the image series produced using different illumination

positions. There is an inherent repetition of patterns across the series of images since

they are of the same region of interest, but with distinctive variations. This makes the

approach, which is otherwise not well suited to non homogenous textures, applicable

to this particular problem. In this approach, the unique but less pronounced

disadvantage of dark field imaging is circumnavigated in two steps

1. By considering a window across the same image and pixel under

consideration.

2. By considering the same windows in the respective series of images.

The approach under consideration will make up for the lack of certainty, while

deciding the defect class based on geometrical features.

Here a particular window size was considered and the GLCM matrix for that

particular window across the series of images has been obtained for each of the pixels

in the image (Figure 6.4). The GLCM matrix can be used to define 14 statistical

parameters for any given pixel in the image series under consideration. Though the

parameters are not quantized into a particular dynamic range, the display shown in

Figure 6.5 has the parameter matrix values scaled to fit the dynamic range for a gray

scale image. But each of the images shows a representation of the visual contrast

between defective area and the normal non defective adjacent areas. This information

87

can be used to select the parameters giving the most optimum separation for the

defective region from the surrounding non defective region. The mathematical

definitions (Haralick et al. 1973) for these parameters are as follows.

If,

p(i, j) := (i, j)th entry in a normalized gray tone spatial dependence matrix (GLCM)

px (i) := ith entry in the marginal probability matrix obtained by summing the rows of

p(i, j), =∑ , (6.3)

Ng:=Number of distinct gray levels in the quantized image

Σi and Σj are ∑ and ∑ respectively

∑ , (6.4)

∑ ∑ , , 2,3, … ,2 (6.5)

∑ ∑ , , 0,1 … . , 1| |

(6.6)

The parameters are,

1. Angular Second moment or energy

∑ ∑ , (6.7)

2. Contrast

∑ ∑ ∑ ,| |

(6.8)

3. Correlation

∑ ∑ , (6.9)

88

Where,

µx µy σx and σy are the means and standard deviations of px and py.

4. Sum of squares

∑ ∑ , (6.10)

5. Inverse Difference moment

∑ ∑ , (6.11)

6. Sum average

∑ (6.12)

7. Sum variance

∑ (6.13)

8. Sum entropy

∑ log (6.14)

9. Entropy

∑ ∑ , log , (6.15)

10. Difference Variance

(6.16)

89

11. Difference Entropy

∑ log log (6.17)

12 Measure of correlation-1

, (6.18)

13 Measure of correlation -2

1 (6.19)

∑ ∑ , log , (6.20)

Where,

HX and HY are entropies of px and py and

1 ∑ ∑ , log (6.21)

2 ∑ ∑ log (6.22)

14 Maximal Correlation Coefficient

, ∑ , , (6.23)

Figu

From

vari

con

calc

num

requ

find

segm

ure 6.4 I

m Figure 6

iance and

sidering the

culations w

mber of com

uirement to

d the idea

mentation.

Image fusio

.5, the para

Entropy h

e higher con

ere time co

mputations w

consider a

l cut off

on for calcu

ameters Con

have been

ntrast of the

onsuming g

which gets c

a particular

value for

90

ulation of GL

ntrast, Sum

selected f

images und

given the si

compounded

region of in

each par

LCM param

average, S

for implem

der consider

ize of imag

d with the s

nterest to im

ameter for

meters

Sum of squa

menting the

ration. Sinc

ges and the

size of the im

mplement th

r better de

ares, Differe

e classifica

ce the param

e correspon

mage, there

he algorithm

efective re

ence

ation

meter

nding

e is a

m to

gion

91

(a) (b) (c) (d)

(e) (f) (g) (h)

(i) (j) (k) (l)

(m) (n) (o)

Figure 6.5 Gray scale representations of GLCM parameters: (a) Contrast, (b) Sum average, (c) Sum of squares, (d) Difference variance, (e) Entropy, (f) Sum of entropy, (g) Measure of correlation-1, (h) Measure of correlation-2, (i) Energy, (j) Correlation, (k) Homogeneity, (l) Sum variance, (m) Difference entropy, (n) Maximal correlation coefficient, (o) Region of interest from Figure 6.2 under consideration for production of images Figure 6.5 a-n

92

6.4. IMAGE MASKS FOR EACH REGION OF THE RING

Fast and accurate implementation of the algorithm to find the ideal cut off value of the

parameters requires that the regions of the image should be separated out. For this

purpose, three regional masks were made with unique shapes matching the shape of

the region of the ring under inspection. These masks can be generated for any

particular region of the ring that is imaged, provided the illumination conditions are

same throughout the region under consideration. One major problem with the

processing of images was that while camera position is kept constant and more

regions are imaged using the particular position of camera under three different

illumination conditions, there is a major variation of appearance of defects in the ring

surfaces. In some cases, the background is bright and the defect regions are dark,

contrary to what happens when dark field illumination is done. Though this is not the

case in dark field illumination, the reduction in the number of cameras will force a

trade off in this aspect of the image. So, it was necessary that the regions needed to be

masked according to the nature of illumination, dividing them to different regions.

Three masks that have been used for the images are shown in Figure 6.6. They can be

generated using morphological operations on the initial image sample to be used.

First, the values of the gray scale intensities have been increased to a constant value

leaving the lower pixel values and then the regional minimum pixel values were

found out to from a binary image. Later the holes (stray pixels) have been removed,

revealing three different regions which were labelled such that they can be used to

separate out the required pixel regions from the image. Though the masks are

dependent on the position of camera, they can be easily generated for the given

positions of the camera and then used for further processing at the later stages.

93

Figure 6.6 Masks used for different regions of the image of ring surface.

6.5 IMAGE THRESHOLDING OPERATION

The image will have majority of pixels lying in the non defective region of image and

remaining very few, if at all, in the defective region of image. This information has

been used to design a thresholding algorithm. This is based on finding the ideal cut-

off value for the parameters that have been evaluated before. For this purpose, two

parameters were scaled and the scatter plot was made. The values which are forming

the bigger cluster were removed. Every combination of parameters was plotted and

the corresponding threshold values were combined to form the final threshold image

which was mapped back to the original image to find the boundaries of the defective

images. This enables the use of the parameter values from the defect threshold images

for the classification step.

For a better understanding of the thresholding process methodology, a flow chart is

provided in Figure 6.7. The input to the thresholding algorithm is any two among the

selected parameters. Each of the parameters will be used to form the two different

Top region Middle region Bottom region

94

axes of an image matrix and the data is represented as white pixels in a dark

background. Later the number of bright pixels in this image is found out by using the

number of pixel values above one half of the maximum gray scale resolution of the

image. If the number of bright pixels is more than the number of dark pixels, then

each of the axes is inverted. This is preceded by an image closing operation using a

structuring element of square shape and 5 x 5 size. This operation will cluster all the

closely lying pixels together as a single object, the extremity of which is taken as the

image index corresponding to the threshold value of the two parameters considered.

The parameter threshold value corresponding to the index that has been obtained is

used to cut off the parameter values in the input parameter matrix, finally ending up in

a thresholded binary image region. So, output of a single thresholding operation gives

two binary images each based on one each of the two parameters that was initially

given as input. These two binary images are added together using logical AND

operation to obtain a single binary image. Similarly, all such binary images from the

different parameter combinations considered are subjected to logical OR operation

culminating in a single image which has all the necessary details of the defect

segmented out. Element wise multiplication of this resultant binary image and each of

the parameter matrices will yield a matrix having parameters only in the image region

which is defective.

Figu

The

It ca

l) an

ure 6.7

e series of th

an be obser

nd by using

Find the paramthreshold corresponding tindex of image

If Axiinverson done

N

Y

Thresholdinoperation woperation

hreshold im

rved that, al

g only a sing

Input twat a tim

Find theafter remobject in

meter

to

is si

Remthan

Remgreatimag

ng operatio

with graphic

mages obtain

ll the combi

gle combina

wo parameters e

e starting index moval of the n the image

move pixels having parcut off to get two bin

move pixels having parter than cut off to get ges

95

(a

(b) on (a) Scatcal explanat

ned from th

inations wil

ation, one c

Scale the parameters to same range

Morphologicclosing on im

rameter less nary images

rameter two binary

a)

tter image tion (b) Flow

e parameter

ll not yield

can not arriv

Form imagparameter

cal mage

Invert a

Logical opfinal thresh

N

used in thwchart for t

rs are show

good result

ve at all the

ge using as each axis

If more bright pixels

axis of image

peration to get the holded image

Y

he thresholthe threshol

wn in Figure

ts (Figure 6

e correct det

ding ding

6.8.

.8 k-

tails.

96

As a result, the good images are combined to get an optimum threshold image. The

information fusion at the thresholding operation is illustrated in Figure 6.9.

(a) (b) (c) (d)

(e) (f) (g) (h)

(i) (j) (k) (l)

Figure 6.8 Threshold images considering different parameters;(a) Contrast image after thresholding with Difference variance,(b) Difference variance image after thresholding with Contrast, (c) Contrast image after thresholding with Sum average,(d) Sum average image after thresholding with Contrast,(e) Contrast image after thresholding with Sum square, (f) Sum square image after thresholding with contrast, (g) Sum average image after thresholding with Sum square, (h) Sum square image after thresholding with Sum average,(k) Sum average image after thresholding with difference variance, (l) Difference variance image after thresholding with Sum average.

97

(a)

(b) Figure 6.9 Information fusion for thresholding operation (a) Schematic

representation (b) Final thresholded image from sample threshold images

98

6.6 DEFECT CLASSIFIER

The parameters after the thresholding operation have been plotted together to

understand the separability for the defect classes on the basis of the parameters

considered. The parameters can be plotted and depending on the value of the normal

distribution function, the class of defect corresponding to the parameter value can be

found out as shown in Figure 6.10.The defect parameters have been plotted against

each other in a scatter plot as shown in Figure 6.11. Though there is overlap in some

classes, considering all the parameter combinations together is a more effective

method of classification. For each defect class, six different combinations of

parameters can be plotted on the two dimensional graph.

Figure 6.10 Plot for Sum average showing the separation of defects using normal

distribution values

0 50 100 150 200 250 300 350 400 450 5000

0.02

0.04

0.06

0.08

0.1

0.12

Sum average

N(x

)

Blister

Built up

DamagePitting

Yellowstain

Pitting Blister

Yellow stain

Damage Built up

Norm

al distrib

ution, N(x)

Sum average

99

Figure 6.11 Plots showing the sum of squares against sum average with the clusters

of defects well separated out. Plots showing pixels corresponding to: (a) all the defects (b) blister, (c) built up, (d) damage, (e) pitting, (f) yellow stain.

Later, a Bayesian classifier has been applied to find the boundaries for the defects.

The parameters were approximated as normal distribution and the corresponding

graph has been plotted to visualize the boundary. Similar graphs have been plotted for

(a) (b)

(c) (d)

(e) (f)

Sum Average

Sum

of

squa

res

Sum

of

squa

res

Sum

of

squa

res

Sum

of

squa

res

Sum

of

squa

res

Sum

of

squa

res

Sum Average

Sum Average Sum Average

Sum Average Sum Average

100

all the combinations of the parameters and finally information has been fused to

decide the class membership of each pixel of the image under consideration. In fact,

the graphs give the visualization of the final decision process but the original decision

is taken based on the value of the discriminant function for the particular pixel with

respect to all the classes of defects.

6.6.1 Bayesian Classifier

Following are the basics of construction of Bayesian classifier based on the

assumption that the distribution is normal. Multivariate normal density in d

dimensions is written as

| | (6.24)

Where, x is a d-component column vector, µ is the d-component mean vector, Σ is the

d-by d covariance matrix, and | | and Σ-1 are its determinant and inverse,

respectively. If d=2,

Σµ µ µ

µ µ µ

Let x be a continuous random variable whose distribution depends upon the state of

nature and is expressed as p(x|ω) which is the class conditional probability density

function for x given the state of nature is ω. Consider the prior probabilities P(ωj) and

the conditional densities p(x|ωj) for j=1,2 i.e 2 classes. The probability density of

finding a pattern that is in category ωj and has feature value x can be written in two

ways : p(ωj,x)=P(ωj|x)p(x)=p(x|ωj)P(ωj). From rearranging this Bayes formula is

arrived at.

101

, (6.25)

Where in the case of two categories

∑ , 1 (6.26)

The basic rule is: decide that the variable belongs to class ω1 if P(ω1|x) > P(ω2|x);

otherwise decide ω2.

To represent pattern classifier, one of the most useful methods is to define a set of

discriminant functions gi(x), i=1,….,c. The classifier assigns a feature x to class ωi if

gi(x) > gj(x) for all j≠i.

Thus the classifier can be viewed as a network or machine that computes c

discriminant functions and selects the category corresponding to the largest

discriminant. A network representation can be seen in Figure 6.12

Figure 6.12 Working of a classifier (Duda et al., 2002)

X1 X2 Xd X3 ……….

g1(x) g2(x) gc(x) ……….

Action (e.g., classification)

Cost calculation from discriminant functions

input

102

There is no unique discriminant function. Each function can be multiplied by a

constant or scaled by another constant. Also, it can be written as a function of

monotonically increasing function without changing the classification. It is normally

written as

gi(x) = p(x|ωi)P(ωi) (6.27)

To simplify computation it has been written as,

gi(x)=ln p(x| ωi) + ln P(ωi) (6.28)

Though the discriminant functions are written in many forms, the purpose of it is to

divide the feature space into c decision regions, R1, R2…Rc. If gi(x)> gj(x) for all j≠i,

then x is in Ri, and the decision rule calls for us to assign x to ωi. The regions are

separated by decision boundaries, surfaces in feature space where ties occur among

the largest discriminant functions.

After approximating the distribution as normal distribution the following has been

defined. Multivariate normal density in d dimensions is written as

| |, where (6.29)

X is a d-component column vector, µ is the d-component mean vector, Σ is the d-by d

covariance matrix, and |Σ| and Σ-1 are its determinant and inverse, respectively (All

bold faced characters represent vectors). In the general multivariate normal case, the

covariance matrices are different for each category. So the expression can be written

as,

0.5 Σ ln 2 ln|Σ | ln P (6.30)

, (6.31)

103

, (6.32)

(6.33)

and ln| | ln (6.34)

The decision boundary in this case will be hyper quadrics depending upon the spread

of the distributions (Figure. 6.13).

Finally, given an image series of the defective region, the algorithm returns the final

thresholded image with parameter representations for each of the defect pixels. The

information fusion approach used in classification is illustrated in Figure 6.14. The

final image after defect pixel classification is shown in Figure 6.15. The images

shown are without usage of masks. The masks can be used depending upon the

requirement in the real life application. The decision regarding the pixel membership

was taken after considering every combination of parameters as in the thresholding

algorithm. Whichever defect class has a better score, the allotment is made to that

particular class. Table 6.1 represents an example of decision making process based on

discriminant values after giving parameter value as input. The defect class is assigned

to the defect class having the highest value of discriminant. An example of calculation

of scores for a sample defect is shown in Table 6.2. The defect considered is pitting

and the corresponding discriminant values for every defect class and every parameter

is shown. If one of the parameter values points to the defect class as pitting, the score

for pitting is set to one. Finally the pixel is classified as the defect class having

maximum score. In this particular case the score for pitting is maximum and thus the

defect class is finally assigned as pitting.

104

Figure 6.13 Bayesian decision boundaries for defect classes in the scatter plot of

sum average against difference variance. Plots showing: (a) all the defects with under lying boundaries, (b) blister, (c) built up, (d) damage, (e) pitting (f) yellow stain

Difference variance

Difference variance

(a)

(c) (d)

(e) (f)

Sum

Ave

rage

Sum

Ave

rage

Su

m A

vera

ge

Sum

Ave

rage

Su

m A

vera

ge

Sum

Ave

rage

Difference variance

(b)

Difference variance Difference variance

Difference variance

Tab Def

Blis

Bui

Dam

Pit

Yel

Fin

Tab

P Def BlisBuiDamPittYelRes

Figu

ble 6.1 Parameter

fect

ster

lt up

mage

low stain

al Decision

ble 6.2

arameter

fect

ster ilt up mage ting llow stain sult

ure 6.14

Classificatir g(x

for

-3.7

-11

-5.0

-1.2

-7.7

n Pitt

Decision mpitting defeContrast=5

-3.74 -11.03 -5.01 -1.22 -7.73 Pitting

Illustration

on based onx) for sum ar pitting(x=

74

.03

01

22 →Larges

73

ting

making by sect

Sum Average

-6.79 -44.88 -6.19 -3.17 -17.23 Pitting

of informat105

n discriminaaverage

=5)

st

coring for d

e=60 DifVar

-24-76-84-4.3-14Pitt

tion fusion

ant (g(x)) vag(x) for sufor built u

-3.75

-3.29→La

-4.99

-220.02

-13.06

Built up

different pa

fference riance=280

4.36 6.5 4.94 39

4.78 ting

in pixel cla

alues um square up(x=300)

argest

arameters co

0 Sum squar

-4.83-17.2-5.1 -0.7 -9.4 Pittin

assification

orrespondin

re=4 Sco

0 0

0 4 0

ng Pit

ng to

ore

tting

Figu

(a

(c

(e)

(g)

(i)

ure 6.15

a)

c)

)

)

Thresholdeblue is builstain). (a) Icontrast beRepresentaFigure 6.15image usingpitting, (h)thresholdinRepresentaFigure 6.15

ed images wlt up, greenImage of blfore thresho

ation of orig5 c, (e) Img contrast b) Represent

ng for Figation of orig5i

106

with defectivn is damagelister, (b) Rolding for Fginal image mage of dambefore threshtation of o

gure 6.15gginal image

(b)

(d)

(f)

(h)

(j)

ve areas col, yellow is p

RepresentatiFigure 6.15a

using contrmage, (f) Rholding for

original imag, (i) Imag

using contr

lored (wherpitting and on of origin

a, (c) Imagerast before tRepresentatFigure 6.15

age using ge of yelrast before t

re red is bliblack is ye

nal image ue of built upthresholdingtion of orig5e, (g) Imagcontrast be

llow stain, thresholding

ister, ellow using p, (d) g for ginal ge of efore

(j) g for

107

6.7 RESULTS AND DISCUSSIONS

The defect classification requires more information about the defects, if a better

classification is to be achieved. The usage of different illumination systems gives

more information about the area under consideration. From the images in Figure 6.2,

it can be understood that the details are varied in each of the images, though the same

details are being imaged in all of them. Each of the features have varying highlights in

different images. The usage of the window has made sure that each pixel represents

information which is related to the neighbouring pixels, rather than an abnormal value

which is seldom correlated to any other pixel values in the vicinity. The comparison

of the same pixel values in different images will yield a more concrete idea about the

presence of a particular feature in image. The images in Figure 6.5 prove this aspect.

Throughout the implementation of GLCM approach, a fusion of information has been

used to make better decisions. One major disadvantage of this approach is the number

of calculations which in turn increased run time for each algorithm implementation.

As a result, it was attempted to consider a region of interest by creating a mask based

on the camera position which might reduce the computational load and run time by

many times. It also helped to simplify the thresholding operation. The thresholding

operation was done by finding the optimum cut off frequency for the parameters

under consideration. An interesting fact to be noted is that only the spatial coordinates

of the image is the same at all stages of classification operations, since the gray scale

values have been replaced by statistical parameters which have their own unique

range. As a result, they had to be normalized to a unique scale while doing operations

in unison with another pixel. Similar conversion of scale happens in all the

measurement processes in metrology. As a result, the parameters have to be

accompanied with a scaling factor while implementing the final classification process

108

also. The assumption that dark field images have predominantly all the pixel values in

the darker side have been used to make the thresholding algorithm. The images in

Figure 6.8 have the defective area separated out to a very good extent. Here again, the

information fusion has been applied through logical operations to arrive at a final

image with an optimum separation of defective region. The usage of information

fusion will help in making decisions on the basis of a large amount of valid

information, thus decreasing the possibility of error in the decision. This can be done

by avoiding all information, which is obviously erroneous, by implementing a

condition for evaluating the suitability of particular information for use in the

algorithm (For example, in a thresholding operation the whole image cannot be

showing defective area of the ring). The parameters in the index values corresponding

to the thresholded region is now passed to the defect classifier. The Bayesian

classifier based on normal distribution has been considered. All the data points could

not be fitted exactly into the normal distribution. Again, an information fusion

approach was considered where the data from different combinations could be

considered together and then the best possible decision taken. The parameters were

considered two at a time, since the classifier becomes erroneous when the covariance

matrix approaches zero (singularity). Instead of using a pseudo inverse which could

not give consistent results, inversion has been done in a manual fashion and then the

determinant multiplication done at a later stage. The results are good except for the

case of yellow stain, which is a difficult kind of defect compared with the rest.

Nevertheless many of the pixels corresponding to yellow stain were classified

correctly. The flow chart for the proposed approach is given in Figure 6.16

109

Figure 6.16 Flowchart for the defect classification approach

6.8 SUMMARY

The images from the single step thresholding cannot be used to get the exact details of

the defect which will enable classification of the rings into the respective defect

classes. A multiple imaging approach has been successfully implemented to enable

defect classification and thus, the defective ring classification in the proposed set up.

Three illumination sources have been used for imaging and these images were used to

calculate the GLCM parameters defined especially for image series. The defect region

in image was segmented out using these parameters and then the parameter

corresponding to the defective region in image was transferred into the pattern

recognition algorithm. The scatter data shows that the defects form well separated

clusters with minimal overlap. The Bayesian classifier has been used to determine the

defect class of the pixel. All these processes were done basically using an information

fusion approach in the background and this has facilitated an optimum solution to the

problem. An added advantage is that the classifier can be updated for different kinds

of defects, more kinds of images and different types of classifiers depending upon the

requirement. This aspect of defect classifier makes it adaptable for different kinds of

Image Series

Multiplication of mask

Parameter calculation

Thresholding Discriminant evaluation and scoring for each pixel

Highest Score

Mask generation

Training data

Pixel classified as defect with highest score

110

inspection methodologies and has better chances of being implemented successfully

for any given inspection task. This approach coupled with the basic dark field

illumination approach can be used in the ring inspection problem considered in this

work, successfully for classification of all the various defects.

CHAPTER 7

CONCLUSION AND SCOPE FOR FUTURE WORK

7.1 CONCLUSION

An approach to inspect highly reflective Chrome coated textile ring components using

machine vision has been presented in this work. The coated surfaces of the ring were

imaged using an experimental set up with a machine vision system using different

illumination systems. These approaches may be used to find the presence of defects

and successful classification of defects with very high accuracy and reliability.

Based on this work, the following are the specific conclusions:

1. Though bright field illumination reveals a lot of information about the ring surface

under inspection, too bright an image with large amount of light reflected due to

the curvature and high reflectivity of the ring does not suit this particular

application of inspecting ring components. Therefore, dark field illumination is

considered since high contrast images could be obtained revealing only the

information related to the abnormalities in the surface.

2. Images taken using bright field illumination can be processed to segment the

defective regions but at the cost of increased processing time. Auto-median based

image processing algorithms could segment the defective surfaces successfully

with high accuracy.

3. Dark field illuminated samples have been processed using single step thresholding

process, which can be used to find the presence or absence of defects with both

higher speed and accuracy. The segmentation for defective surfaces can be done

112

by Fractional Change in Derivative (FCD) method for dark field image proposed

in this work to get the optimum segmentation of image.

4. Creating series of images by various illumination systems in the dark field

imaging and then, processing them using GLCM approach for image series is

proposed for defect classification. After segmentation of the image, the parameters

under consideration are fed into the classifier, where a Bayesian classifier is used

to make decisions namely the classification of defects in images. An information

fusion approach was used successfully to get the final results.

5. Using the present approaches it was possible to speed up the entire defect

identification and classification of ring components in particular. The accuracy

improved by 56% and the speed decreased by 17% when auto-median based

image processing algorithm on bright field images was used. The improvement in

accuracy and speed was of the order of 69% and 97% respectively when single

step thresholding on dark field illumination was used.

In short, an accurate and quick method to identify and classify the coating defects on

Chrome coated ring components using dark field illumination has been established.

7.1 SCOPE FOR FUTURE WORK

The proposed set up could be practically implemented and fine tuned to meet the

requirement of the industry. Also, instead of the normal halogen light source, other

light sources could be tried out. There can also be quantification of the surface or

geometrical features of the ring by methods such as phase shifting interferometry and

structured lighting sources etc. Such approaches can be as well used to find the

surface finish of the components under consideration.

REFERENCES

1. Abrahamovich G., Barhak J. and Spicer P. (2005) Reconfigurable array for machine vision inspection (RAMVI). Proceedings of the 3rd International CIRP Conference on Reconfigurable Manufacturing. Ann Arbor, USA, R-01, May 10–12, 2005.

2. Aluze D., Merienne F., Dumont C. and Gorria P. (2002) Vision system for defect imaging, detection, and characterization on a specular surface of a 3D object. IVC, 20, 569-580.

3. Bamforth P. E., Jackson M. R. and Williams K. (2007) Transmissive dark-field illumination method for high-accuracy automatic lace scalloping, International Journal of Advanced Manufacturing, 32(5-6), 599-607.

4. Beckmann P. and Spizzochino A. The scattering of electromagnetic waves from Rough Surfaces. New York: Pergamon, 1963.

5. Beyerer J. and León F. P. (1997) Detection of defects in groove textures of honed surfaces, International Journal of Machine Tools & Manufacture. 37(3), 371-389.

6. Beyerer J. and León F. P. (1998) Adaptive separation of random lines and background. Optical Engineering, 37(10), 2733-2741.

7. Connolly C. (2002). Lighting for industrial inspection. Sensor Review. 22(2), 106-112.

8. David M. (2008) Dark field illumination Available from: http://encyclopedia.jrank.org/articles/pages/1126/Darkfield-Illumination.html. Accessed on 8 April, 2010

9. Duda R. O, Hart P. E., and Stork D.G. Pattern Classification. New York: John Wiley & Sons, 2001.

10. Eiden G., Kayadjanian M.and Vidal C. (2000). Capturing landscape structures:Tools.Availablefrom:http://ec.europa.eu/agriculture/publi/landscape/ch1.htm#1. Accessed 25 April, 2009.

11. Gadelmawla E. S., Koura M. M., Maksoud T. M. A, Elewa I. M. and Soliman H. H. (2002) Roughness parameters. Journal of Material Processing Technology, 123, 133-145.

12. Gadelmawla E. S. (2004) A vision system for surface roughness

characterization using the Gray Level Co-occurrence Matrix. NDT&E, 37, 577-588.

13. Guise M., Poe D. K., Stafford J. and Wahba A. (2002) Automated defect

pattern recognition: An approach to defect classification and lot characterization.Availablefrom:http://www.sys.virginia.edu/students/capstone/past/cap2002/2002-01.pdf. Accessed on 25 April 2009.

14. Haralick R. M., Dinstein and Shanmugam K. (1973) Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-3, 610-621.

114

15. Iivarinen J. and Visa A. (1998) An adaptive texture and shape based defect classification. 14th International Conference on Pattern Recognition (ICPR'98), 1, 117.

16. Jayavarthanavelu D, Technical Newsletter Available from: www.lakshmimach.com/tmd/training-newsletter.pdf, accessed online on 23 March 2010.

17. Juran J.M. and Godfrey A.B. Juran's Quality Control Handbook. Fifth Edition, McGraw-Hill. New York, 1998

18. Khalili K. and Webb P. (2007) The development and application of a multiple wavelength illumination technique for the vision-based process monitoring of aero-structure riveting. Machine Vision Application, 18, 73-83.

19. Lee M. F. R., de Silva C. W., Croft E. A. and Wu Q. M. J. (2000) Machine vision system for curved surface inspection. Machine Vision Applications, 12, 177-188.

20. León F. P. (1997). Enhanced imaging by fusion of illumination series, Sensors, Sensor Systems, and Sensor Data Processing. Proceedings of SPIE, 3100, 297-308.

21. León F P. (2002) Evaluation of honed cylinder bores, Annals of CIRP, 51(1),

503-506.

22. Lindner C., Schäffler F. and León F. P. (2007) Texture-based surface segmentation using second-order statistics of illumination series. Lecture notes in informatik, 109, 32-37.

23. Luo P.F. and Liou S.S. (1998) Measurement of curved surfaces by stereo vision and error analysis. Optics and. Lasers Engineering, 30, 471-486.

24. Marrs C.D. and Porteus J.O. (1985) Nondestructive defect detection in laser optical coatings. Journal of Applied Physics, 57(5), 1719-1722.

25. Martin D. (2007). A Practical guide to machine vision lighting. Advanced Illumination.Availablefrom:http://advancedillumination.com/uploads/downloads/A Practical Guide to Machine Vision Lighting.pdf. Accessed on 30 Jan 2009

26. Miyoshia T., Takahashia S., Takayaa Y. and Shimadab S. (2001) High

sensitivity optical detection of oriented micro defects on Silicon wafer surfaces using annular illumination, CIRP Annals - Manufacturing Technology, 50(1), 389-392.

27. Nayar S. K., Fang X. and Boult T. (1997) Separation of reflection components using color and polarization. International Journal of Computer Vision, 21(3), 163-186.

28. Nayar S.K., Ikeuchi K. and Kanade T. (1991) Surface reflection: physical and geometrical perspectives. IEEE Transactions on. Pattern Analysis and Machine Intelligence. 13(7), 611-634.

115

29. Noda N. and Kamimura S. (2008) A new microscope optics for laser dark-field illumination applied to high precision two dimensional measurement of specimen displacement. Review of Scientific Instruments, 79(023704), 1-7.

30. Pfeifer T. and Wiegers L. (1998) Adaptive control for the optimized adjustment of imaging parameters for surface inspection using machine vision. Annals of CIRP, 47(1), 459-462.

31. Piper J. (2008) Luminance contrast, a new illumination technique in light microscopy: Optical basics, practical evaluations, further developments, Optik - International Journal for Light and Electron Optics, 120(18), 963-975.

32. Porteus J. O., Spiker C. J. and Franck J. B. (1986) Correlation between He-Ne scatter and 2.7 μm pulsed laser damage at coating defects. Applied Optics, 25(21), 3871-3879.

33. Povolo F., Canzian A., and Favret E. (1997) Unidirectional laser oblique illumination (ULOI): a comparison with partial dark field illumination (PDFI) in the optical microscopy of metals, Applied Optics, 29(2), 67-73.

34. Rosati G., Boschetti G., Biondi A. and Rossi A. (2009) Real-time defect detection on highly reflective curved surfaces. Optics and Lasers in Engineering, 47(3-4), 379-384.

35. Salas W. A., Boles S. S. H., Frolking S., Xiao X. and Li C. (2003) The perimeter/area ratio as an index of misregistration bias in land cover change estimates. International Journal of Remote Sensing, 24(5), 1165–1170.

36. Samak D, Fischer A and Rittel D. (2007) 3D Reconstruction and visualization of microstructure surfaces from 2D images. Annals CIRP, 56(1), 149-152.

37. Sun Y., Bai P., Sun H.Y. and Zhou P. (2005) Real time automatic defect detection of weld defects in steel pipe. NDT&E International, 38, 522-528

38. Thomas K. Image Processing with LabVIEW™ and IMAQ™ Vision. Prentice, 2003.

39. Torrance K. and Sparrow E. (1967) Theory for off-specular reflection from roughened surfaces. Journal of Optical Society of. America, 57, 1105-1114.

40. Tsai D.M. and Huang T.Y. (1999) Automated surface inspection for directional textures. Image and Vision Computing, 18, 49–62.

41. Tsai D.M. and Huang T.Y. (2003). Automated surface inspection for statistical textures. Image and Vision Computing, 21, 307–323.

42. Tsai D.M. and Kuo C.C. (2007) Defect detection in in-homogenously textured sputtered surfaces using 3D Fourier image reconstruction. Machine Vision and Applications, 18, 383-400.

43. Wiltschi K., Pinz A. and Lindberg T. (2000). An automatic scheme for steel quality inspection. Machine Vision Applications, 12, 113-128.

116

44. Yi S., Haralick R. M. and Saphiro L. G. (1995) Optimal sensor and light source positioning for machine vision. Computer Vision and Image Understanding, 61(1), 122-137.

45. Zhang X., Krewet C. and Kuhlenkötter B. (2006) Automatic classification of defects on the product surface in grinding and polishing. International Journal of. Mach. Tools Manufacturing, 46, 59-69.

117

LIST OF PUBLICATIONS 1. Riby Abraham Boby, Prashant S. Sonakar, B. Ramamoorthy and M.

Singaperumal. Identification of defects on highly reflective ring components and analysis using machine vision. International Journal of Advanced Manufacturing Technology, (available online DOI:10.1007/s00170-010-2730-3).

2. Prashant S. Sonakar, Riby Abraham Boby, B. Ramamoorthy and M. Singaperumal. Identification of defects on highly reflective coated ring components using dark field illumination and image segmentation using simple thresholding technique. International Journal of Automation and Control, (accepted).

3. R. A. Boby, M. Singaperumal, B. Ramamoorthy. Automated inspection and sorting of defects of ring components using Machine Vision. IJP Tech special issue on Precison Metrology (accepted).

identification and classification of coating defects using...

Documents