medical image segmentation using mean shift algorithm and ... · 3.1 mean shift algorithm the mean...

Medical Image Segmentation Using Mean Shift Algorithm and General Edge Detection

Jinghua Lu, Jie Chen, Juan Zhang, Lihui Zou

*1 School of Automation, Beijing Institute of Technology, Beijing, 100081, China 2 Key Laboratory of Complex System Intelligent Control and Decision, Ministry of Education, Beijing, 100081, China

(Tel: 86-010-68913795; e-mail: [email protected]).

Abstract: The segmentation of the medical image faces the challenges of the existence of large number of diverse structures of human anatomy and inevitable artifacts induced from the imaging procedure. In this paper we treat some structures and artifacts as general edge features, and introduce the edge information into the mean shift segmentation algorithm in both clustering and the fusion steps. Considering the medical images as two dimensional signals, the general edge can be detected and described with its local spatial frequency properties. The segmentation results show the improvement in preserving the completeness of details while sketching the overall structures.

Keywords: image segmentation, signal processing, medical application

1. INTRODUCTION

Medical images such as magnetic resonance (MR) and computed tomography (CT) are digital representation of human internal structures that mapping the 3D anatomy on 2D image sequences. Medical image analysis is critical in numerous biomedical applications such as detection of abnormalities, tissue measurement, surgical planning and simulation, and more. In particular, image segmentation is an essential step, which partitions the medical image into different non-overlapping regions such that each region is nearly homogeneous and ideally corresponds to some anatomical structure or region of interest.

Segmentation algorithms available vary widely depending on the specific application, image modality and other factors as pointed out in (Pham, Xu, & Prince, 2000). Our segmentation algorithm was developed for the need of knee modelling in an arthroscopic surgical simulation system (Lu, Chen, Cakmak, & Maass, 2009), aiming at fully segmenting and labelling all structures simultaneously. MR images are used in our modelling procedure for its ability to provide high resolution, excellent soft tissue contrast, and high signal-to-noise ratio. For the simulation purpose, the demand for accuracy is much lower than clinical applications, while an efficient automatic method is desirable. However, there are particular challenges we had to overcome:

Multi objects with diversity in shape. The knee is a complex joint mainly comprised of bones, cartilages, menisci, ligaments, tendons, muscles, fat, nerves and blood vessels. Each of these structures has their own size and topological shape mapping on the 2D image plane. We aim to simultaneously segment multiple objects, avoiding of over-segmentation of large structures such as bones, while preserving the details of the tiny line-like structures such as nerves and blood vessels. In addition

the existence and appearance of the anatomic structures change from slice to slice in the same MR series over space, which makes the determination of the number of classes to be segmented very hard.

Bias field and partial volume effects artifacts in MR. The MR image suffers from the artifacts introduced by the image acquisition process such as bias field and partial volume effects. Bias field leads the intensity of the same tissue to vary with the location in the image. Partial volume effect where multiple tissues contribute to a single pixel results a blurring of intensity across boundaries. Although acceptable to human observers, such artifacts bring additional difficulties in computer segmentation. Many correction methods have been proposed (Vovk, Pernus, & Likar, 2007)(Ballester, Zisserman, & Brady, 2002), but most of them are based on some strong assumptions, and still can’t separate the adverse phenomena from the true signal.

After above analysis, the mean shift algorithm (Comaniciu & Meer, 2002) turns out to be particularly suitable for handling our multi objects segmentation task. Being in advance to other commonly used clustering method, like K-means and Fuzzy C-means, the mean shift algorithm can decide the number of clusters automatically, and delineate the clusters with arbitrary shape. However, the mean shift algorithm encounters the problem of mis-segmentation as it groups the image pixels only respect to the low-level feature similarities. Given the importance of edge information, the paper addresses this problem by incorporating an extracted edge map in the framework of mean shift algorithm in both clustering and fusion steps.

The paper is organized as follows: section 2 describes the existence of the general edge feature in the MR image and its phase congruency based detection. Section 3 recalls the mean shift segmentation method briefly and then proposes our

Preprints of the 18th IFAC World CongressMilano (Italy) August 28 - September 2, 2011

Copyright by theInternational Federation of Automatic Control (IFAC)

9656

modification in detail. Section 4 shows the effect of our new method with some experiments on the knee MR images. Discussions are presented in Section5.

2. GENERAL EDGE DETECTION

2.1 General Edge in MR Image

Edge is the most common feature in an image. It is often ideally modelled as a step where the intensity changes abruptly. However, in MR images the intensity changes have various profiles from step via ramp to roof and even to more complex forms, as shown in Fig 1. These diverse edges including step, ramp, roof and more others between them are named as ‘general edges’.

Fig. 1. A MR knee image showing actual step (bottom, left), ramp (top, left), and roof (right) edge profiles indicated by the

short line segments shown in the small circle

In the spatial domain, the traditional gradient based detectors, e.g. canny detector (Canny, 1986) taking use of the ideal step model, perform poorly on the detection of general edges. Additionally, characterizing edge strength by the magnitude of the intensity gradient, gradient based detectors are sensitive to variations in image brightness and contrast. On the other hand, considering the spatial frequency domain, phase congruency is a very appealing concept for general edge detection because the dimensionless quantity can provide an absolute measure of the significance of feature points independent of the feature type, and is insensitive to brightness or contrast changes.

2.2 Phase Congruency from Monogenic Signal

Phase based image analysis premises that features are perceived by the human visual system at points of high phase congruency (PC) in an image where the Fourier components are maximally in phase. That is, the phase is congruent at that point over some scales.

The phase congruency was first defined in terms of the Fourier series expansion of a 1D signal at position x . (Morrone & Owens, 1987) Kovesi (1999) then developed a more accurate measure as

n nn

nn

W x A x x TPC x

A x

(1)

cos sinn n nx x x x x (2)

Where nA x and n x represents the local amplitude and

phase of the thn Fourier component at position x , and x is

the amplitude weighted mean local phase angle of all the Fourier terms at the position being considered. W x is the

frequency spread weighting function, T is the estimated noise influence, is added to avoid division by zero.

The extension of phase congruency to 2D can be obtained by more than one ways. Kovesi (1999) employed orientation sampling to estimate the local frequency properties of 1D signal and sum the results over all orientations. Recently, Felsburg and Sommer (2001) extended the phase concept to arbitrarily higher dimensional signals by monogenic signal, which provided us a powerful mathematical framework for the estimation of local spatial frequency properties of signals without the need of orientation sampling.

The monogenic signal of a 2D image is defined as the 3D vector formed by the signal and its Riesz transform:

1 2 1 2 1 1 2 2 1 2, , , , , ,MI x x I x x h I x x h I x x (3)

where denotes convolution. The convolution with the kernel

of Reisz Transform 1 1 2 2 1 2, , ,h x x h x x results in two

filtered versions of I .

In practice, the image signal I is often pre-filtered by a family of band-pass filters to form a scale space

: 1, ,n nI I b n N for the reason that a certain

feature exists only over a certain scale range. Then we can derive the local amplitude nA and local phase n efficiently as

follow:

2 221 2 1 2,n n n nA x x I h I h I (4)

11 2 2 2

1 2

, tan nn

n n

Ix x

h I h I

(5)

Thus the 1 2,PC x x can be obtained by substituting (4) and

(5) in (1) and (2).

2.3 The Description of General Edge

The phase congruency described above is a dimensionless measure of feature significance. It values from a maximum of 1 to 0. This allows one to describe the strength of the general edge at a point 1 2,x x with its phase congruency 1 2,PC x x

instead of the magnitude of the image gradient. A phase congruency of value one means that there is a strong feature,


9657

zero indicates there is no structure. Points whose edge strength is larger than a threshold can be treated as general edge feature. As the phase congruency is independent of the magnitude of the signal, fixed threshold values can be used over wide classed of images.

The difference between the edge detection by image gradient and by phase congruency in our application is shown in Fig 2.

It can be seen that the magnitude of the image gradient map fails to detect the weak edges of menisci in the rectangle, whereas the phase congruency marked them clearly. At the line-like structure in the ellipses, the gradient map has double response at the location of the line, where the phase congruency can localize them with single response. However, the phase congruency seems sensitive to the image texture. We intend to fix it in the next step.

Fig2 Comparison of the magnitude of the image gradient and the phase congruency map for a knee MR image. (a) is the original image, (b) is the magnitude of the gradient, and (c) is the map of phase congruency. The image features we are particularly interested in are enclosed with the rectangle and ellipses

We can describe the type of the significant feature by the mean local phase angle 1 2,x x , who is structural

information in the range , . 0 indicates an upward going

step, indicates a downward going step, 2 indicates a

bright line feature, and 2 indicates a dark line feature

(Kovesi, 2002). Given that it makes no sense to distinguish upward and down going steps, bright and dark lines in our application, the phase data is ‘folded’ twice mapping angles into the range 0, 2 . As the radian is dimensionless, it can

be further normalized into 0,1 . This simplifies the range of

feature types to a scale that varies continuously from ‘line’ through ‘step/line’ to finally ‘step’.

Fig3 the original MR superposed by the edge map detected by the phase congruency. The types of the general edge are coded in colour for demonstration with the bar indicating the correspondence from line with of 0 and step with of 1.

The edge map in Fig 3 is detected with phase congruency based method with additional process of non-maximum

suppression and hysteresis thresholding (Canny, 1986) to reduce the false detection. The wide range of edge detection and the rich information of edge type contribute to the following segmentation task in the framework of mean shift algorithm.

3. MEAN SHIFT SEGMENTATION

3.1 Mean Shift Algorithm

The mean shift algorithm is an unsupervised method commonly used in computer vision problems such as filtering, tracking and segmentation. It is first derived by Fukunaga and Hostetler (1975) from nonparametric density estimation, and developed by Cheng (1995) , Comaniciu and Meer (2002) et al. The characteristic of the mean shift procedure where feature points move toward some significant modes and cluster themselves automatically makes it the ideal computational module for multi-object image segmentation in our application.

Given n data points , 1,i i nx in the d-dimensional

space dR , clustering of the feature points ix set is achieved

by placing a seed point 0iy at each ix ,

1

1

1

1, 2,

M i kkk

ij

M i kk

gh

j

gh

x xx

yx x

(6)

is the weighted mean at ijy and 0

iy is the center of the initial

position of the kernel G with profile : [0, )g R . The

(a) (b) (c)


9658

sequences 1,2{ }ij jy converge to a local maximum (mode) i

cy . Feature points whose corresponding series converge to the same mode are grouped, automatically delineating a cluster of arbitrary shape. For image segmentation, each pixel is assigned a joint-domain feature point ix accounting for the

spatial domain information position 1 2,i ix x and the range

domain information which can be composed grey, colour or other local information. The different nature of location and range vectors has to be compensated by normalization. Thus the multivariate kernel is defined as the product of two kernels with their bandwidth respectively

2 2

, 2 2s r

s r

h hs rs r

CG g g

h hh h

x xx (7)

Where sx and rx stand for the spatial and the range part respectively, sh and rh the employed kernel bandwidths, and

C is the corresponding normalization constant. In this paper

simple joint feature vector , ,i i i ix y Ix with the position

and the intensity information are used primitively.

Unfortunately, the intensity-only mean shift method will lead to misclassification at object boundaries, and suffers from over-segmentation of large homogeneous regions. In order to solve these mis-segmentation problems, we proposed a solution incorporate the mean shift algorithm with the general edge information derived from last section.

3.2 Mean Shift Segmentation with General Edge

The mean shift segmentation procedure is the grouping of image pixels into clusters respect to their similarities measured by the distance of pixels in the spatial domain and the difference of the intensity in the range domain. Given that discontinuity information is as important as similarity information in the image segmentation process, we combine the outputs of the general edge detection into the mean shift segmentation to improve the quality of the segmented image.

The phase congruency 1 2,PC x xas the strength of edge

enhance the feature space by weighting, and the mean local

phase 1 2,x x which indicate the type of general edge to

guide the after process of region merging. As the signal flow in Fig 4 said, the procedure follows the steps as below:

Input: The original image 1 2,I x x .

Step1: To compose the monogenic signal 1 2,MI x x with log

Gabor filtered signal and its Reisz transformation as in (3) at a set of scales.

Step2: To compute the spatial frequency properties 1 2,x x and 1 2,PC x x over all the scales.

1 2,x x

Fig 4 the flow chart of the proposed method

Step3: To enhance the intensity feature space at the point whose edge strength is large and reduce the influence of the weak ones. The enhancement is done by dot production of intensity and phase congruency at each pixel in the image as 1 2 1 2, ,I x x PC x x . The convergence of mean shift was

proven to remain valid when each data point is associated with a nonnegative weight.(Comaniciu & Meer, 2002)

Step4: To analysis the weighted spatial-range joint feature space via the mean shift procedure iteratively until convergence or reaching the end condition. Then the points in the image are automatically grouped into arbitrary shape clusters and present by the intensity of their corresponding modes.

Step 5: To fuse the over-segmented regions using the Region adjacency graphs (RAG) and transitive closure operation as describes in (Christoudias & Georgescu, 2002). Taking the modes of regions as the vertices of the graph whose edge is the average value of normalized 1 2,x x for the pixels on

the boundary shared by two regions. The fusion strategies are taken in a logical ‘or’ way:

A. Region similarity strategy: Fuse two regions if the intensity difference between their modes is under

2rh .

B. Edge discontinuity strategy: Merge two regions with the condition of 1 2, typex x T . Having been

described in section 2.3, the general edge type can be indicated with mean local phase normalized range 0 to 1. That is, 0 indicates a line, 0.5 indicates a ramp edge with the slope of 4 , 1 indicates step

edge. As the localization of a line always in the center of it, the 0 average value of local phase means that pixels sharing this boundary belong to the same line structure, and then merge them; if 1 2,x x is

1 indicating a ideally step edge, no operation. Particularly, we take the value of 1 2,x x under

0.7 as slow variation in homogenous regions, fuse them; otherwise, it’s a blurred step, keep the separation between them. The above analysis


9659

concludes with the threshold typeT equals to 0.7. This

value can be adjusted according to the application.

C. Minimal region strategy: Eliminate regions containing less than areaT pixels.

Output: The final segmented image 1 2,S x x .

4. EXPERIMENT AND DISCUSSION

Experiments on the segmentation of knee MR images with 512×512 resolution are carried out to evaluate the proposed method. To generate the general edge map, the Log-Gabor filter is used in the band-pass filtering with the wavelength of the smallest scale filter to be 6 and the scaling factor between successive filters to be 2.1. For the mean shift clustering procedure, a uniform kernel is used for the reason that the convergence can be reach in finite iterations. We set the kernel’s bandwidth as , 10,5s rh h h empirically to

preserve most useful details in the medical image and the efficiency of the program. For the fusion stage, parameters

, , (5,0.7,100)r type areah T T are needed. The processing

required seconds on a standard PC using a Matlab and C++ mixed implementation.

The proposed solution presented in Section 3.2 has been applied to the gray-level image (fig5 a), the result being shown in Fig. 5b.The comparison with the original mean shift segmentation algorithm is shown in Fig 5c. And for better visualization of the region we are interested in, small windows marked in Fig 2 shows again with ‘zoom in’ to make the detail display. Due to the space limitation, we only show one of our experiment results.

As shown in Fig 5, the mean shift based segmentation algorithm can label the complex structures contained in the whole medical image as non-overlapping regions with different average intensity feature. With the comparisons of the segmented result by the methods the proposed method and the original mean shift under the same set of parameters, we can see the benefit of introducing the general edge information in the detail processing. The region containing line-like structure in the second column with our algorithm in the middle row has a better result as the line-like structure is relative completely preserved. For the region containing structures with varying texture in the third column, our algorithm delineates the real boundary more correctly rather than divides the same region into parts due to the textures. In the third case where the weak boundaries with low intensity are being detected, our algorithm also shows its goodness in

keeping the completion of the structure with less discontinuity.

5. CONCLUSIONS

In this paper we regard the medical image as a two dimensional signal and take use of its local spatial frequency properties to improve the image segmentation based on mean shift algorithm. The proposed algorithm is highly automatic with little parameter to be set, which is desirable in our modelling application.

To handle the multi-object segmentation task, we take the mean shift feature space clustering as the segmentation method. It needs no manually initialization for the number of clusters which seems impossible in the complex MR images containing various tissue and structures.

To reduce the negative influence of the bias field and the partial volume effect from MR imaging technology, we regard the induced artifacts, the smooth intensity variation across the image and the blurred boundaries, along with the line structures as general edges. Not well detectable by gradient based edge detector in the spatial domain, we handle the problem in the spatial frequency domain where a monogenic signal scale space can be established. In the monogenic signal scale space, the general edge can be detected efficiently and precisely. The description of the general edge indicates not only the strength of the feature by the phase congruency PC but also the type of the edge by its structural information mean local phase .

The main contribution of the proposed method is introducing the general edge information derived from monogenic signal scale space into a mean shift segmentation algorithm in a way that has not been attempted previously. The combination is taken both in the stage of the mean shift clustering and the region fusion. The experiments results show its improvement in preserving the completeness of details while sketching the overall structures.

Moreover, the use of phase based properties make the method robust to the image contrast and less sensitive to the artifacts in the image. However, in the monogenic signal scale space, there are other useful local properties not in the scope of this paper. We have been working on mining those properties to get further achievement in the medical analysis field.

ACKNOWLEDGEMENT

This work is supported by National Science Fund for Distinguished Young Scholars，No. 60925011.


9660

Fig 5 The comparison of the segmented results via the proposed method and the original mean shift method. (a) is the original knee MR image with the region of interest enlarged beside. (b) is the segmented image

with our proposed algorithm. (c) is achieved using the original mean shift method.

REFERENCES

Ballester, M. G., A. P. Zisserman, & M. Brady. (2002). Estimation of the partial volume effect in MRI. Medical Image Analysis, 6, 389-405.

Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8(6), 679-698.

Cheng, Y. (1995). Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8), 790-799.

Christoudias, C. M. & B. Georgescu. (2002). Synergism in low level vision. Proceedings of 16th International Conference on Pattern Recognition (pp. 150-155). Quebec.

Comaniciu, D., & P. Meer. (2002). Mean shift: A robust approach toward feature space analysis. Analysis and Machine Intelligence, 24(5), 603-619.

Felsberg, M., & G. Sommer. (2001). The monogenic signal. IEEE Transactions on Signal Processing, 49(12), 3136-3144.

Fukunaga, K., & L. Hostetler. (1975). The estimation of the gradient of a density function, with applications in

pattern recognition. IEEE Transactions on Information Theory, IT-21(1), 32-40.

Kovesi, P. (1999). Image features from phase congruency. Videre: Journal of Computer Vision Research, 1(3), 1-27.

Kovesi, P. (2002). Edges are not just steps. Proceedings of the Fifth Asian Conference on Computer Vision (pp. 822-827). Melbourne.

Lu, J., J. Chen, H. Cakmak, H. Maass, U. Kuehnapfel, & G. Bretthauer. (2009). A knee arthroscopy simulator for partial meniscectomy training. the 7th Asian Control Conference (pp. 763-767). Hong Kong.

Morrone, M. C., & R. A. Owens. (1987). Feature detection from local energy. Pattern Recognition Letters, 6(5), 303-313.

Pham, D. L., C. Xu, & J. L. Prince. (2000). A survey of current methods in medical image segmentation. Annual Review of Biomedical Engineering, 2, 315-338.

Vovk, U., F. Pernus, & B. Likar. (2007). A review of methods for correction of intensity inhomogeneity in MRI. IEEE Transactions on Medical Imaging, 26(3), 405-421.

(a)

(b)

(c)


9661

medical image segmentation using mean shift algorithm and ... · 3.1 mean shift algorithm the mean...

Documents