Segmentation of Three Phase Micrograph – An Automated Approach

Siddhartha Banerjee Dept. of Computer Science

R. K. Mission Residential

College

Narendrapur, 24 Pgs(S), India

+919830637708

sidd_01_02@yahoo.com

Subhabrata Datta Hooghly Engg. &

Technology College

Hooghly, West Bengal

India

+919477485253

shu.datt@gmail.com

Biplab Paul Dept. of Computer Science

&Engg.

Jadavpur University

Kolkata, India

+919433452188

biplabme@gmail.com

Sanjoy Kumar Saha Dept. of Computer Science

& Engg.

Jadavpur University

Kolkata, India

+919433526300

sks_ju@yahoo.co.in

 

ABSTRACT Quantitative description of micro-structure of the materials enables the analysis of material characteristics. Material under study undergoes heat and cool treatment and micro-structures thereby get revealed. Image processing has enormous potential in the analysis of microscopic images of the processed materials. Thus, the extraction of microstructures/phases present in the material forms the fundamental step towards material description. In this work, we present an automated scheme for segmenting the phases namely Ferrite, Martensite and Bainite present in the microscopic image of HSLA steel. First of all, region contours are obtained using topological watershed. Each region is then characterized as Ferrite, Martensite and Bainite based on average intensity and white grain concentration of the region. Finally, a refinement process is carried out to merge the adjacent regions of same type. As no state of the art technique is readily available, the segmentation output has been validated by the domain expert.

Categories and Subject Descriptors I.4.6 [Segmentation]: Pixel classification, Region growing, partitioning

General Terms Design, Experimentation.

Keywords micrograph segmentation, phase extraction, material analysis

1. INTRODUCTION It is known that properties of materials are related to their micro-structure [13, 17]. Thus, it is essential to obtain a quantitative description of the micro-structure of the materials. Such description may obtained following direct and indirect approaches

[18]. X-ray diffraction based measurement [9] is an indirect technique where structural parameters are estimated by measuring lattice parameter. On the contrary, in direct technique, the structural parameters are directly measured. Moreover, indirect approach cannot give correct result in presence of stress and texture. Direct technique relies on the microscopic investigations. Such analysis includes steps like microscopic observation and image collection, image processing and analysis [10]. The images/micro graphs produced by a microscope are easily converted into digital form for subsequent storage, analysis, or processing and subsequent interpretation [19, 16, 15, 7, 11]. Digital Image processing greatly enhances the process of extracting information about the specimen from a micrograph and has become an integral part of microscopy related experimentation in metallurgy and materials engineering [2, 5]. In micrograph analysis, segmentation is the major challenge. Micro-structure denoting various phases are to be extracted first. Phases may be discriminated using the features like gray level intensity, textural pattern, edge orientation. Depending on the material under study, a suitable feature has to be chosen as the discriminator. Presence of noise or the impurities introduced by the image acquisition system or the environmental issues (i.e. contrast, brightness, magnification etc.) may pose as additional challenges. Sometimes revealed phase boundaries with striking similarity with one of the phases makes the task very difficult. A two stage problem has been presented by Gauthier et al. [6] for segmenting WC grains in the Cobalt matrix. They have relied on gray level threshold and morphological gradient filter at first level and in the next stage, they have addressed the issue of phase boundary removal. Classification based approach was tried in [14]. In [8], a scheme has been presented where an image classifier has been integrated with contextvision [3]. It facilitates contextual analysis (i.e. spatial dependencies among the regions) for extracting the areas of interest. But, such analysis incurs computational cost and accuracy heavily depends on proper training. Neural network also have been tried to classify the phases of an alloy [4]. Phase extraction enables their analysis which includes grain size analysis, inclusion rating, volume fraction, porosity, particle size, morphology etc. It is quite obvious that the most crucial and fundamental task is to properly classify different phases in dual-phase or multiphase metallography samples. The accuracy of subsequent measures relies

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CUBE 2012, September 3–5, 2012, Pune, Maharashtra, India. Copyright 2012 ACM 978-1-4503-1185-4/12/09…$10.00.

 

1

on the performance of the underlying segmentation scheme. Past study also indicates that potential of image processing technique in addressing the issue has not been exploited in a structured manner. As a result elegant solutions for various problems are not very common. It has motivated us to focus on problem. In the present work we have concentrated on the extraction of different phase regions present in the micrograph that will act as the foundation for subsequent analysis. Thus, it turns into an image segmentation problem. The paper is organized as follows. The brief introduction presented in this section is followed by the detailed description of the proposed methodology in section 2. Experimental results are presented in section 3 and it is finally concluded in section 4.

2. PROPOSED METHODOLOGY In the present work we have taken up the task of extracting the three phases present in the steel. Micro structures/phases in a steel

Figure 1: A micrograph with three phases.

are formed due to various continuous cooling and isothermal holding heat treatments. During the process, if cooling is done in controlled manner then austenite phase is transformed in to ferrite phase. On the other hand, rapid cooling gives rise to two phases – ferrite and martensite. Bainite phase is generated if the cooling process includes isothermal hold at some intermediate temperature. Bainite is the ferrite phase with retained austenite in it. The phases are observed at high resolution with Single Electron Microscope (SEM) micrograph as shown in Fig. 1.

Phase regions in a micrograph are separated by revealed boundaries. Sample ferrite, bainite and martensite region have been marked as F, B and M respectively in Fig. 1. As it is visible in the figure, a ferrite phase appears as a smooth region with low intensity value. Ideally, a martensite phase is characterized by a smooth region with high intensity value. A bainite region is of hybrid nature as it represents ferrite with retained austenite. Depending on actual cooling rate and material composition martensite regions may deviate from the ideal scenario. But, it has been observed that both martensite and bainite phase regions have white blocks embedded in them. Moreover, concentration of such blocks is relatively higher for martensite and that serves as the distinctive feature. Proposed methodology relies on these observations. The major steps are as follows.

• Extraction of region contours

• Region classification

– Ferrite phase detection

– Martensite and Bainite phase detection

• Refinement of phase regions

2.1 Extraction of Region Contour As the phase regions are mostly enclosed by revealed boundary, our primary focus is on to extract the same. But, simple threshold based cannot serve the purpose as the intensity value of such pixels may overlap with some of those pixels forming the white patches in the martensite/bainite phase regions. Moreover, boundaries may have discontinuities. Thus, obtaining the desired closed contours is a major challenge.

It has been observed that contrast around the desired boundary pixels and those around the white pixels in martensite/bainite phase regions differs significantly. Contrast in the neighborhood of the boundary pixels are relatively higher. On the contrary, contrast in the vicinity of high intensity pixels in the martensite/bainite phase is lower as sharp transition of intensity does not occur; rather a blurring effect results into a relatively smooth variation. Thus, jumping from one region to another adjacent region requires to overcome an intensity barrier. As watershed algorithms relies on similar philosophy in segmenting a gray scale image, we have opted for the same.

Figure 2: Image after Contour Extraction.

Watershed is a popular segmentation scheme used to extract the significant contours present in the image. Among several algorithms, we have followed the topological watershed scheme presented in [1] It extracts the most significant contours by preserving the image contrast. A gray scale image is considered as a topographic relief where intensity value may be thought of as the altitude in the relief. A drop of water falling on a topographic relief flows along a path and finally reaches a local mini ma. Intuitively, the watershed of a relief correspond to the limits of the adjacent catchment basins of the drops of water. Topological watershed focuses on the detection of the contour separating the adjacent basins. The detection relies on a parameter (t) that specifies the minimal altitude (intensity value) to be climbed for traversing from one region to another region.

2

Applying topological watershed algorithm, the closed contours of the regions are obtained. The value of the parameter t is to be chosen carefully. A low value of t over splits a region as it becomes sensitive to smaller intensity variation. On the other hand,

Figure 3: Image after Ferrite Phase Detection.

high value of t will accommodate more variation with in a region and overlook weak boundaries. In our experiment, a moderate value for t is estimated which approximates the contrast between the region pixels (low intensity value) and boundary pixels (high intensity value). t has been taken as 2 × σ where σ is the standard deviation of the intensity values present in the image. The extracted contours for the image in Fig. 1 has been shown in Fig. 2. It may be noted that a single continuous phase, particularly a martensite region is prone for over splitting because of the presence of white regions.

2.2 Region Classification

Once the region contours are extracted, the task of classification is carried out on individual region. A region classified as one of the three phases namely, ferrite, bainite and martensite. At first level ferrite phases are identified and remaining regions are later classified either as bainite or martensite.

It has been already mentioned that ferrite phases are almost smooth regions with low intensity. For a bainite or martensite region average intensity is relatively higher because of the presence of white patches arising out of heating-cooling process on the material. Based on this observation, ferrite and non-ferrite discrimination at the region level is made as follows. Let I1, I2 . . . In denote the average intensity values of the n regions in the image. μ and σ denote average and standard deviation of Iis. A threshold, th1 is chosen as μ−σ. A region with average intensity Ii is considered as ferrite phase if Ii < th1. Otherwise, it is taken as non-Ferrite. The output after ferrite phase identification for the image in 1 has been shown in Fig. 3. Dark regions in the figure denote the ferrite phase and the bright regions correspond to non-ferrite phase.

Classification of non-ferrite regions are carried out considering the region wise white pixel density as the discriminating feature. Martensite phase possesses higher density in comparison to the same in bainite phase as it is an intermediate process of transformation from ferrite to martensite. First of all, binarization of each non-ferrite region is done by thresholding. Based on the intensity histogram of the overall image, a threshold, th2 is chosen. Pixels with intensity higher than th2 are considered as white and black otherwise. As the phases show considerable variation, selection of th2 is important. We have relied on thresholding scheme proposed by Otsu [12]. It chooses the optimal threshold in a way to

minimize the intra-class variance. Once the binarization is done, white pixel density is di computed as Wi/Si where Wi denotes number of white pixel and Si be the number of pixels in the i-th region. A region is classified as martensite phase if it’s white pixel density, di > μd − 0.5 × σd. μd and σd denote average and standard

Figure 4: Image after Region Classification.

Figure 5: Output after Refinement.

deviation of dis. The output after classification has been shown in Fig. 4. Here, the dark, bright and regions with intermediate intensity correspond to ferrite, martensite and bainite phase respectively.

2.3 Region Refinement

The outcome of the automated phase extraction process primarily depends on the effectiveness of the region contour detection described in section 2.1. It has been mentioned that a moderate value for t has been chosen for the purpose. A high value for t may overlook the weak boundaries and as a result adjacent regions of different phases are merged. Hence, high value is avoided. On the other hand, low value for t fragments the region heavily that may lead to misclassification. For example, a martensite/bainite region may be splitted in to number of ferrite and non-ferrite regions. A moderate value for t substantially reduces the over splitting and merging of regions. Still, we carry out post-processing to minimize the effect of over splitting, if any.

Because of considerable variation present in the background, unwanted small regions may be detected. Such small regions may be classified as a phase different from its surroundings. At this stage, we identify such small regions and modify its class by making it similar to its surroundings. Thus, the effect of over splitting, if

3

any, is minimized. Regions with size smaller than 0.1×μs (μs is the average region size) are taken as small regions. It may noted that, for a low value of t, as the regions detected are small enough, it is difficult to carry out the refinement process. Finally, the adjacent regions of same class are merged to generate the final output as shown in Fig. 5. Here also, the dark, bright and regions with intermediate intensity correspond to ferrite, martensite and bainite phase respectively.

3. EXPERIMENTAL RESULTS In order to carry out the experiment, we have considered 10 micrographs. They vary in terms phase size, magnification and illumination level. Such collection has enabled us to judge the robustness of proposed scheme. Sample outputs are shown in Fig. 6. Black denote ferrite phase, white corresponds to martensite phase and intermediate gray values denote bainite. Subjective evaluation of the segmentation output has been made by the domain experts and the proposed methodology has been considered effective.

(a) (b)

Figure 6: Sample output: (a) Original Image and

(b) Output Image.

4. CONCLUSION In In this work, we have presented an automated scheme for extracting the three phases namely, ferrite, martensite and bainite present in a high strength low alloy steel. Region contours are first obtained following topological watershed scheme and then classification of each region is carried out hierarchically. Features like average intensity and white pixel density of the regions are used for such classification. Experimental result indicates the effectiveness of the scheme. The segmented output will enable the

quantitative analysis like volume fraction of the phases present in a material.

5. REFERENCES [1] G. Bertrand. On topological watersheds. J. of Mathematical

Imaging and Vision, 22(2-3):217–230, 2005.

[2] K. R. Castleman. Digital Image Processing. Prentice-Hall, 1996.

[3] Contextvision. Users’s Guide, MicroGOP2000/S Software v. 3.1. 1999.

[4] V. H. C. deAlbuquerque, A. R. Alexandria, P. C. Cortez, and J. M. R. S. Tavares. Evaluation of multilayer perceptron and self-organizing map neural network topologies applied on microstructure segmentation from metallographic images. NDT and E International, 42:644–651, 2009.

[5] A. Diaspro. Confocal and Two-photon Microscopy. Wiley-Liss, 2001.

[6] G. Gauthier, M. Coster, L. Chermant, and J. Chermant. Morphological segmentation of cutting tools. Microsc. Microanal. Microstruct., 7:339–344,1996.

[7] S. Inoue and K. R. Spring. Video Microscopy. Springer, 1997.

[8] J. Komenda. Automatic recognition of complex microstructures using the image classifier. Materials Characterization, 46:87–92, 2001.

[9] C. E. Kril and R. Birringer. Estimating grain-size distributions in nanocrystalline materials from x-ray diffraction profile analysis. Philosophical Magazine A, 77(3):621–640, 2001.

[10] K. J. Kurzydlowski and B. Ralph. The Quantitative Description of the Microstructure of Materials. CRC Press, London, 1995.

[11] D. B. Murphy. Fundamentals of Light Microscopy and Electronic Imaging. Wiley-Liss, 2001.

[12] N. Otsu. A threshold selection method from gray-level histogram. IEEE trans. Sys., Man, Cyber., 9(1):62–66, 1979.

[13] N. J. Petch. The cleavage strength of polycrystals. J. of the Iron and Steel Institute, 174:25–28, 1953.

[14] H. V. Saebo, K. Braten, N. L. Hjort, B. Llewellyn, and E. Mohn. Contextual classification of remotely sensed data: Statistical methods and development of a system. Technical Report, Norwegian Computing Center, 1985.

[15] G. Sluder and D. E. Wolf. Digital Microscopy. Academic Press, UK, 2003.

[16] D. L. Spector and R. D. Goldman. Basic Methods in Microscopy. Cold Spring Harbor Laboratory Press, 2005.

[17] W. L. Spychalski, K. J. Kurzydlowski, and B. Ralph. Computer study of inter- and intraangular surface cracks in brittle polycrystals. Materials Characterization, 49(9):45–53, 2002.

[18] T. Wejrzanowski, W. L. Spychalski, K. Rozniatowski, and K. J. Kurzydlowski. Image based analysis of microstructures of engineering materials. Int. J. Appl. Math. Comp. Sc., 18(1):33–39, 2008.

[19] Q. Wu, F. S. Merchant, and K. R. Catleman. Microscope Image Processing. Academic Press, UK, 2008.

4