sik

Lesion segmentation in breast sonography

Joan Massich

Department of Computer Architecture and TechnologyUniversity of Girona

A Thesis Submitted for the Degree ofMSc Erasmus Mundus in Vision and Robotics (VIBOT)

· 2008 ·

Abstract

Breast cancer is one of the leading causes of death for women in developed countries and is mosteffectively traded when detected at an early stage, where imaging techniques and ComputerAided Diagnosis systems play an important role. A hot topic under research is Ultra-Sound (US)image diagnosis because is the cutting edge image modality for a dense breast scenario whereother techniques such as the wide spread Digital Mammography (DM) fail. This work proposesa novel methodology for breast lesion segmentation in B-mode US images taking advantageof both region and boundary information in order to cope with the inherent artifacts of USimaging.

Contents

Acknowledgments v

1 Introduction 1

1.1 Problem definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Background 4

2.1 US imaging in the world . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 US images modalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2.1 A-mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2.2 B-mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2.3 M-mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3 US artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3.1 Speckle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3.2 Shadowing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3.3 Refraction and Multiple pathway . . . . . . . . . . . . . . . . . . . . . . . 8

2.4 Segmentation hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.5 Previous work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

i

3 State of the art 10

3.1 Classical approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.2 Methodologies based on Region or Boundary information for US image segmen-

tation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2.1 Gaussian function constrained segmentation . . . . . . . . . . . . . . . . . 11

3.2.2 Disk expansion segmentation methodology . . . . . . . . . . . . . . . . . . 14

3.2.3 Level-set segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.3 Boundary and region infromation mixture to achieve better performance on US

images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.4 Post-processing boundary refinement methodologies . . . . . . . . . . . . . . . . 20

3.4.1 Methodology steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.4.2 Reported results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.5 Post-processing over-segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.5.1 Watershed transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.5.2 Reported results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4 Proposal 28

4.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.1 Seed placement (initial region) . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.2 Region growing segmentation . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.1.3 Best Fitting Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.1.4 Gaussian constraining segmentation . . . . . . . . . . . . . . . . . . . . . 35

5 Experimental setup 37

5.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.2 Experiments procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6 Conclusion and Future work 43

ii

Bibliography 47

iii

List of Figures

2.1 B-mode Ultra-Sound image example . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 Shadowing artifact in US images . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.1 Preprocessing procedure illustrations. . . . . . . . . . . . . . . . . . . . . . . . . 12

3.2 Thresholding procedure images by Horsch et al. . . . . . . . . . . . . . . . . . . . 13

3.3 Disk expansion methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.4 Madabhushi and Metaxas procedure diagram . . . . . . . . . . . . . . . . . . . . 21

3.5 Image enhancement example proposed by Madabhushi and Metaxas . . . . . . . 22

3.6 Probability planes used by Madabhushi and Metaxas . . . . . . . . . . . . . . . . 22

3.7 Orthogonal lesion boundary founding illustration by Madabhushi and Metaxas . 23

3.8 Radial gradient example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.9 Lesion boundary finding using radial gradient . . . . . . . . . . . . . . . . . . . . 24

3.10 Lesion segmentation illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.11 Huang and Chen procedure diagram . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.12 Topographic view of watershed transformation. . . . . . . . . . . . . . . . . . . . 27

4.1 Block diagram of the proposed method . . . . . . . . . . . . . . . . . . . . . . . 30

4.2 Knowlage extracted from the training step . . . . . . . . . . . . . . . . . . . . . . 31

4.3 Seed placement illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.4 Region growing output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.5 Gaussian distribution fitting a breast lesion. . . . . . . . . . . . . . . . . . . . . . 34

iv

4.6 Several thresholding segmentations. . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.1 Experimental setup diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.2 Sonographic lesions image data base (1) . . . . . . . . . . . . . . . . . . . . . . . 39

5.3 Sonographic lesions image data base (2) . . . . . . . . . . . . . . . . . . . . . . . 40

5.4 Graphical visualization of the results . . . . . . . . . . . . . . . . . . . . . . . . . 41

v

List of Tables

5.1 Segmented areas pixel classification . . . . . . . . . . . . . . . . . . . . . . . . . . 42

vi

Acronyms

ARD Average Radial Derivative

ACWE Active Contour Without Edges

CAD Computer Aided Diagnosis

CAS Computer Aided Surgery

CDF Cumulative Distribution Function

CT Computed Tomography

CV Computer Vision

DM Digital Mammography

FNA Fine Needle Aspiration

GVF Gradient Vector Flow

LBP Local Binary Pattern

MRA Multi Resolution Analysis

MRI Magnetic Resonance Image

PCA Principal Component Analysis

vii

PDF Probability Density Function

ROI Region Of Interest

SNR Signal to Noise Ratio

SOM Self Organizing Map

US Ultra-Sound

viii

Acknowledgments

I would like to express profound gratitude to my advisor, Dr. Joan Mart for his invaluable

support, encouragement, supervision and useful suggestions throughout this research work. (

He would probably notice that the last sentence is copy-pasted from somewhere, because there

are no mistakes to correct but indeed it is true) I am also highly thankful to Dr. Elsa Perez,

expert radiologist on ultrasound image diagnosis of breast cancer from the hospital Josep Trueta

of Girona.

I am as ever, especially indebted to my parents, my brother, my girlfriend and my friends.

ix

Chapter 1

Introduction

1.1 Problem definition

Breast cancer is one of the leading causes of death for women in developed countries. Accordingto Sivaramakrishna et al. [35], breast cancer is most effectively traded when detected at an earlystage, where imaging techniques play an important role.

There are several imaging techniques used on breast analysis, while efforts to improve theresults focus on refining the technologies, enhancing the image quality, developing techniques forcomputer-assisted interpretation of images, enabling long-distance and electronic image trans-mission technology (telemammography/teleradiology) for clinical consultations, and improvingimage-guided techniques to assist with breast biopsies.

Digital Mammography (DM) is a technique for recording X-ray images in computer codeinstead of on X-ray film, as with conventional mammography. The images are displayed ona computer monitor and can be enhanced (lightened or darkened) before they are printedon film. Digital mammography may have some advantages over conventional mammography.The images can be stored and retrieved electronically, which makes long-distance consultationswith other mammography specialists easier. Because the images can be adjusted by the ra-diologist, subtle differences between tissues may be noted. The improved accuracy of DigitalMammography may reduce the number of followup procedures while several reports, such asthe work done by Moskowitz et al. [22], show that DM is the most effective method for theearly detection of breast cancer.

Ultra-Sound (US) imaging, also called sonography, is a technique in which high-frequencysound waves are bounced off tissues and internal organs. Sound waves are pressure variations

1

Chapter 1: Introduction 2

forming a longitudinal wave whose echoes produce a picture called a sonogram. US imaging ofthe breast is used to distinguish between solid tumors and fluid-filled cysts. US images can alsobe used to evaluate lumps that are hard to see on a mammogram. Sometimes, Ultra-Sound isused as part of other diagnostic procedures, such as Fine Needle Aspiration (FNA) (also calledneedle biopsy). FNA is the removal of tissue or fluid with a needle for examination under amicroscope to check for signs of disease.

Ultra-Sound is not used for routine breast cancer screening because it does not consistentlydetect certain early signs of cancer such as microcalcifications, but it is used to evaluate breastabnormalities that are found during screening mammography, diagnostic mammography, or ona physical exam (see the imaging techniques usage and review done by Moore [21]). BreastUltra-Sound is also used for screening women with dense breasts since digital mammographydoesn’t report good results. As Stavros et al. [36] and Tylor et al. [39] point out, sonographycan diagnose cysts with an accuray approaching 100% and, hence, could reduce the potentialnumber of unnecessary biopsies.

Summing up, Drukker et al. [5] state that Ultra-Sound is able to depict small, early-stage,otherwise occult malignancies, similar in size and stage as those detected by mammography, andsmaller and lower in stage than palpable cancers in dense breasts. Therefore, including sono-mammography to the breast cancer Computer Aided Diagnosis (CAD) systems and diagnosticroutines would mean a reduction of the women mortality rate.

Computer Aided Diagnosis (CAD) involves the use of computers to bring suspicious areason a mammogram to the radiologists attention. Giger [9] estimates that approximately two-thirds of the missed cancers are retrospectively detected by the radiologists. Even though CADsystems are usually used after the radiologist has done the initial review of the mammogram,CAD becomes a powerful tool.

Magnetic Resonance Image (MRI) consists on a magnet linked to a computer depictingareas inside the body without the use of radiation. Each MRI produces hundreds of images ofthe breast from side-to-side, top-to-bottom, and front-to-back. The images are then interpretedby a radiologist.

Breast MRI is not used for routine breast cancer screening, but clinical trials (researchstudies with people) are being performed to determine if MRI is valuable for screening certainwomen, such as young women at high risk for breast cancer. MRI cannot always accuratelydistinguish between cancer and benign (noncancerous) breast conditions. Like US, MRI cannotdetect microcalcifications.

MRI is used primarily to evaluate breast implants for leaks or ruptures, and to assessabnormal areas that are seen on a mammogram or are felt after breast surgery or radiation

3 1.2 Objectives

therapy. It can be used after breast cancer is diagnosed to determine the extent of the tumor inthe breast. MRI is also sometimes useful in imaging dense breast tissue, which is often found inyounger women, and in viewing breast abnormalities that can be felt but are not visible withconventional mammography or sonomammography.

1.2 Objectives

Duncan and Ayache [6] spot that medical image analysis broadly covers image acquisition,image formation, image enhancement, image segmentation, image compression and storage,image matching/registration, motion tracking, the measurement of anatomical and physiologicalparameters from images, and image-based visualization. Any of those topics concerning themedical image analysis is an entire life research field itself. Hence it have been lot of work donefor different medical diseases and different medical image modalities. However, breast lesionsegmentation on B-mode US images is still a hot topic under research. This is also mentionedby Huang and Chen [12] when conclude that the future in breast diagnosis rely on derive theprecise shapes of tumors in 3D US images. In such scenario a radiologist cannot manually sketchthe contours of a tumor in a 3D sonomammography of hundreds of 2D US images. Thereforethe development of an automatic contouring or segmentation method is indeed important andits medical application is urgent.

As previous work of other authors such as Noble and Boukerroui [24] or Shrimali et al. [34]among others, this work analyses the segmentation techniques over B-mode images (refer tosection 2.2 where US imaging modalities are reported). Therefore any other kind of image modeor even the raw radio-frequency signal segmentation are out of this scope.

Besides, this work aims to analyze the state of the art in breast lesion segmentation onB-mode US images by achieving the following goals:

• Analyze the segmentation techniques applied to US images.

• To propose a new methodology for an effective and automatic segmentation of hypoechoiccyst lesions of the breast in US images.

1.3 Organization

This document is organized as follows. Chapter 2 gives a background of the fundamental physicsbehind the US image formation. Chapter 3 gives a comprehensive overview of the state of theart on sonography segmentation field. The proposed methodology to segment breast lesions inB-mode US images is described in chapter 4, while the experimental setup and the obtainedresults are shown in chapter 5. Conclusions and future work are given in chapter 6

Chapter 2

Background

2.1 US imaging in the world

Diagnostic using US is noninvasive, it is also safe and it does not have any cumulative biologicalside effects. Mainly strength of this imaging technique are low acquisition time, portability andthe fact that is relatively inexpensive. This last feature allows realistic image-guided diagnosisand makes US imaging suitable as a perception system for Computer Aided Surgery (CAS).Although US imaging has need of experience to achieve useful results, according to Szabo [37]sonography leads the imaging modalities worldwide after the standard X-ray exams. Moreover,the better understanding of the Ultra-Sound interaction with tissues, and, the usage of moresophisticated signal processing algorithms, increase the diagnostic potential using US. Thereforenowadays the sonographic images are markers for the early detection of some breast cancers.Madabhushi and Metaxas [18] state that US imaging has been reported an accuracy between96% and 100% for a breast cysts presence.

2.2 US images modalities

Diagnositc US images are formed using Ultra-Sound longitudinal waves in a range from around1Mhz through to over 20Mhz (refer to the work of Hendee and Ritenour [10] and Ensmingerand Stulen [7] for more physical background of US image formation). Those images are formedas a measurement of the tissue reflection signature or supply focused energy, where its spatiallyvariant resolution depends on the transducer parameters such as active aperture and bandwidth,and also to acquisition parameters like the focal depth and the central frequency. Despite theuse of dynamic focusing techniques, the lateral resolution is poor and the structures are often

4

5 2.2 US images modalities

blurred in a direction perpendicular to ultrasonic propagation. On the other hand, parallelto the ultrasonic propagation, the dynamic range resolution is reduced while the US beampenetrates trough the tissue. Consequently, it is difficult to achieve good contrast at higherdepths due to the attenuation as an energy looseness. It has to be pointed here that thisattenuation phenomenon becomes accentuated as the central frequency increase

A sonograph maps the body structures subject to the acoustic properties of the tissuesdepending on their density and stiffness or elasticity. The term mode is used to describe ordefine the operation and display of the imaging system. Despite the numerous sonographicimaging models most of them are combinations and overlays of the three main sonographmodes: the amplitude mode or A-mode, the brightness mode or B-mode, and the motion modeor M-mode.

2.2.1 A-mode

The A-mode is the most basic imaging modality. Here the received amplitude is plotted as afunction of time. Assuming a constant sound propagation speed, then the amplitude is alsodisplayed as a function of distance, where the time-distance relation is the propagation speeddivided by 2, due to the wavefront has to propagate back and forth. The A-mode is usedto better visualize interfaces. The A-mode is commonly used in nondestructive testing andresearch systems for the study of wave propagation in materials and structures.

2.2.2 B-mode

The B-mode or B-scan is used to display a 2D image. The received amplitude is representedas a brightness of a spot, whose 2D position represents a location in the body. Interfacesbetween tissues with very different impedances are displayed with high brightness. When theimpedances are the same or nearly so, the position appears dark.

Figure 2.1 exemplifies a B-mode sonography of the liver. In this figure different scatterphenomena can be observed: a high back scatter is produced near the liver boundaries due tothe different impedances from the different tissue interface; on an ideal system, any tissue apartfrom the liver boundaries should appear completely dark. A second phenomenon appears asspeckle texture, which covers almost the entire image. This speckle texture of tissue is formedby high concentration of living cells with respect the resolution . This living cells have some kindof interface between them therefore some of those interfaces are bright spotted. The imagedblack blobs correspond to vessels where the concentration of living cells is lower, so their echotexture is more weak.

The present work is focused on this B-mode images because this is the most useful and usedimage modality to depict breast lesions using sonography.

Chapter 2: Background 6

Figure 2.1: B-mode Ultra-Sound image of a liver showing different scattering effects. (imagetaken from Szabo [37] )

7 2.3 US artifacts

2.2.3 M-mode

The M-mode scan depicts moving structures and there are mainly two types. Those thatthe position of each echo-producing interface is presented as a function of time. And, thosethat the frequency of the returned signal is measured, also called Doppler-shift images. Themost frequent application of the first ones is echocardiography where the motion of the heartinterfaces are chart recorded. Doppler-shift images are mainly used to to visualize blood flowwithin the heart and arteries. The M-mode data are generally plotted in color over a grayscaleB-scan, which allows the clinician to evaluate the blood flow in and out of a structure of interest.

2.3 US artifacts

Although other medical imaging modalities like MRI depict specific fundamental properties ofthe tissue, or like, X-ray or Computed Tomography (CT) that their values are associated withlinear attenuation coefficients, this is not the case of sonography. US images are generatedby fundamental properties such as acoustic impedance and physical density. However, due itsstrongly nonlinear behavior the brightness of each blob not only depends on physical properties(density and acoustic impedance) of the imaged blob, but also on the physical properties of thesurrounding tissue. Thus, such complex image formation avoids to unfold it quantitatively.

Due to its process of formation, B-mode images have some artifacts. Some of them are usefuland desirable while some others are a bother. Among some other artifacts speckle, shadowing,and refraction and multiple pathway could be mentioned according to their importance in theB-mode images.

2.3.1 Speckle

Speckle is a common phenomenon in Ultra-Sound imaging systems. It comes from scattererslocated throughout the tissue, so that even in the same tissue, it appears as a granular structuresuper imposed on the image. Speckle is detrimental because it reduces both image contrast (theability to see desired structure against a background) and the distinction of subtle gradationsand boundaries in tissue structure. Although speckle seems to form an acoustic texture pattern,speckle is intrinsically dependent on the imaging system and it does not represent any physicalcharacteristic of the tissue.

2.3.2 Shadowing

The shadowing effect occurs when the whole ultrasonic beam is attenuated by a structure withinthe B-mode scan range. This effect can be observed on US B-mode image as a dark shadow

Chapter 2: Background 8

(any dark zone is called hypo-echoic zone) beyond which there is no speckle. Note that ifthere is an hypo-echoic zone surrounded by hyper-echogenity (that’s the name of the brightzones), then the shadow is likely a liquid-filled cyst that does not have structural componentsto create speckles. Figure 2.2 shows the difference between a shadowed US B-mode image an anon-shadowed image with a liquid cyst.

Also, note that as long as the Ultra-Sound beam is propagated through the media it becomesattenuated, and therefore the B-mode image becomes darker without meaning that there is ashadowing presence.

(a) (b)

Figure 2.2: Shadowing artifact in Ultra-Sound (US) image examples. (a) Presence of shadowsdue to the attenuation of the media. (b) Cyst without shadow artifact presence.

2.3.3 Refraction and Multiple pathway

Refraction and multiple pathway artifacts affects the spatial accuracy of a B-mode image.Refraction can cause displacement of the sound beam as it crosses tissue boundaries. Because

during the B-mode image formation is assumed that the Ultra-Sound travels in straight line,those refractions generate displacement errors positioning reflective interfaces on the B-modescan. With the same nature and because of the same straight line traveling assumption, multiplepathways artifacts can displace a target to a greater depth in the image due the time of flightdifferences from different paths.

2.4 Segmentation hints

Image segmentation plays an essential role in both qualitative and quantitative image analysis.The goal is to divide an image into meaningful regions that are distinct. Although this dis-tinction is mainly visual, its interpretation refers to some kind of distinct and uniform physical

9 2.5 Previous work

property of the imaged tissue.Despite this US image segmentation into anatomic or pathological meaningful regions has

been addressed using various approaches, the problem still remains unsolved for an automaticmethodology (or even for a semi-automatic with minimal operator interaction). Since theimages have poor quality due to the possible artifacts that can show up during the US imageformation procedure (some of them are described in section 2.3), Noble and Boukerroui [24]put forward the inadequacy of applying general image processing methods in the case of Ultra-Sound data. Application driven or customized methods, which attempt to model the imagedphysical properties of the tissue in some way, have proven to be more successful for US data.However, apart from those task-specific constrains and priors, all segmentation algorithms aredriven using either intensity gradient, texture or both.

If intensity gradient is used to perform the segmentation then the goal becomes findingacoustic impedance discontinuities, interfaces and boundaries. However, the inherent noisynature of the US images makes virtually impossible to use such technique to achieve the men-tioned goal unless the topology of the region to be segmented is known or at least very restricted.Therefore, those methods use to be really sensitive to its initialization and the quality of theacquisition because speckle gives a strong intensity gradient response.

On the other hand, texture driven methodologies aim to characterize patterns correspondingto microstructure of the imaged tissues. But those characterized patterns not only depend onthe tissue, but also on the imaging system and the acquisition conditions. Speckle generatestexture with weak edge definition leading to an inappropriate segmentation, and also generatesfalse edges causing holes in a segmented region. Moreover, speckle artifacts change from anacquisition to another, thus affecting the repetitiveness.

2.5 Previous work

Although many segmentation surveys can be found in the literature and most of them includemedical imaging among their applications, there is an important lack when linking segmentationalgorithms to US images, while focusing the applications to breast lesion segmentations in B-mode US images is hardly cited. Therefore, an important effort to find comprehensive papersclose to the scope of this work, i.e. the segmentation of US breast images, has been made duringthe first part of this research work.

Chapter 3

State of the art

Image segmentation has been, and still is, a relevant research area in Computer Vision (CV).In the last decades, hundreds of segmentation algorithms have been proposed trying to solvethe segmentation problem by means of different approaches. However, it is well known that ele-mental segmentation techniques should use rather boundary or region information. Boundary-based methods refers to those methodologies based on some discontinuity property of the pixels,whereas region-based methods refers to the exploitation of some similarity property of the pix-els.

3.1 Classical approaches

Boundary based methods are often used to look for explicit or implicit boundaries betweenregions. If we think on the image as a topographic surface, a boundary based method triesto outline the shape of a crater from the crater’s rim. This task can be driven from bothfollowing the image peaks or from a gradient plane. More elaborated techniques such as snakesproposed by Kass et al. [13] use the gradient information to deform a shape model initializedclose to the object boundary in order to cope with edge discontinuities. From the snake derivedmethodologies it has to be highlighted the Gradient Vector Flow (GVF) aproach describedby Xu and Price [41, 42] that incorporate a new external force computed as a diffusion of thegradient vectors of a gray-level or binary edge map derived from the image, allowing a lessaccurate initialization.

However, those methodologies for images with low Signal to Noise Ratio (SNR) such assonograms, may lead to incorrect assignment of pixels unless significant manual intervention isused.

10

11 3.2 Methodologies based on Region or Boundary information for US image segmentation

Region-based techniques are usually the first option in order to segment an image. Thebasic idea of most region-based approaches is to connect adjacent pixels with similar charac-teristics according to some user-specified criteria. Specifically, the basic region-based approachis composed of two essential components: the first is a mechanism to gather adjacent pixels toform regions, while the second is a criterion to determine the set of pixels that form a region.Region growing, split-and-merge, morphological watershed transformation, fuzzy connectednessor some active contour techniques (such as Active Contour Without Edges (ACWE) or levelset methodologies) are under the region-based segmentation techniques scope. From the abovementioned methodologies, region growing and split-and-merge are the most artless within thisclassification. From those two, the region growing technique (see the work of Zucker and Cen-ter [45]) is the most popular, reporting good results when referred to US image segmentationapplications.

Despite region-based approaches have been well studied and successfully applied, the resultsare subject to the quality of the image acquisition. In a poor image quality scenario like USimaging, those classical segmentation techniques can easily fail due to the inherent US imageproblems such as: weak edges produced by shadowing and speckle artifacts, false edges producedby speckle, refraction and multiple pathway; and finally, the superposed US texture derived fromspeckle noise.

3.2 Methodologies based on Region or Boundary infor-

mation for US image segmentation

Segmenting an US image using just region information or boundary information is a tedioustask. Even so, it has been reported some approaches [11,31–33,43,44] that perform the segmen-tation using just one kind of information rather boundary or region information. There are twobasic approaches to achieve good results: the first ones try to find an elaborated derived imageplane where is easier to perform a segmentation rather than applying it only to the intensityor a naive texture plane of the image, and the others use an iterative methodology where asegmentation is done and used afterwards as a constrain to perform another segmentation andso on and so forth. In this section 3.2, some of those methodologies are reviewed.

3.2.1 Gaussian function constrained segmentation

Horsch et al. [11] propose a computationally efficient boundary-based segmentation algorithmfor breast masses on sonography that is based on maximizing a utility function over partitionmargins defined through gray-value thresholding of a preprocessed image.

Chapter 3: State of the art 12

The proposed algorithm for automatic lesion segmentation, involves:

• Preprocessing by cropping and median filtering.

• Multiplication with a Gaussian constraint function.

• Determination of potential lesion margins through gray-value thresholding.

• Maximimization of a utility function on the potential lesion margins.

Preprocessing

By cropping the top of the image by 35 pixels, the authors remove the subcutaneous fat thatappears on the images, which in fact is completely dependent of the used database. Afterremoving the subcutaneous fat, a 10 by 10 median filter is used to suppress the ultrasoundspeckle, as is shown in figure 3.1.

(a) (b) (c)

Figure 3.1: Preprocessing procedure illustrations. (a) Original image. (b) The preprocessedimage (cropped and median filtered). (c) The inverted preprocessed image.

Gaussian constraint function

The next step proposed by the authors involves multiplying by a constraint function centeredon the lesion center. The authors previously used this method for lesions in mammography [14]with the effect of suppressing distant pixel values and encouraging potential lesions marginsto be more lesion-like. A similar technique may be applied to ultrasound images by invertingthe gray scale of the preprocessed image before multiplying by a constraint function. Thisconstraint function should have higher gray values in the region of the lesion and gray valuesnear zero far from the lesion. In this work, a Gaussian was used as the constraint function,wich was centered at the manually defined lesion center.


In order to study the sensitivity of the segmentation algorithm on the choice of variance, bothmanual and automatic width and height estimation were performed. In the former, manuallydelineated lesion margin were performed, while in the latter estimations of the lesion widthand height are determined through Sobel edge detection. The variances in the width and depthdirections for the Gaussian constraint function are varied adaptively and automatically for eachimage.

Determination of potential lesion margins

After applying the Gaussian constraint function to the inverted preprocessed image, gray-valuethresholding defines partitions whose margins are potential lesion margins, as it can be seen infigure 3.2a The potential margin that maximizes the utility function on the preprocessed imagethen defines the lesion margin.

(a) (b) (c)

Figure 3.2: Thresholding procedure images. (a) The inverted preprocessed image multipliedby a Gaussian. (b) Several partitions resulting from gray-value thresholding. (c) The AverageRadial Derivative (ARD) as a function of partition number.

Maximization of a utility function

The utility function used in the proposed segmentation algorithm is the ARD, which givesthe average directional derivative in the radial direction along the margin accordingly to equa-tion 3.1.

ARD(Γ) =1N

∑p∈Γ

∇I(P ) · r(P ) (3.1)

where Γ is the discretized potential lesion margin, N is the number of points in Γ, r(P ) isthe unit vector in the radial direction from the geometric center of the partition to the pointP = (x, y), and · is the dot product between vectors. Figure 3.2b shows an example of potential


lesion margins resulting from gray-value thresholding and figure 3.2c illustrates the ARD as afunction of the partition number.

Results

The performance of the segmentation algorithm has been assessed by comparing the computer-delineated outlines against the outlines drawn by human observers. It is observed that, onaverage, both the partially automatic and the fully automatic segmentation algorithms tend toresult in smaller lesions than those defined manually. One reason for this may be that radiol-ogists seem to overdraw lesion margins. This remark is based on observing many radiologistsoutline lesions. In addition, the lesions segmented by the fully automatic algorithm tend to besmaller than those segmented by the partially automatic algorithms. This is in part becausethe lesion height and width estimations for fully automatic segmentation tend to be less thanthose for partially automatic segmentation. The fully automatic width estimation is twice theminimun of the lenfths from the lesion center to the left and right lesion edges, while the par-tially automatic width estimation is the maximum horizontal length in the manually outlinedmargin.

3.2.2 Disk expansion segmentation methodology

Yeh et al. [43,44] propose an iterative disk expansion methodology to segment breast lesions inultrasonic images. The authors stand that contour extraction of breast lesions in US images canbe equivalent to the removal of speckle noise once thresholded the image. Thus, this methodcomes from disk expansion method for closure noise removal described by Chen and Lin [16]used to recover degraded paint characters.

The proposed methodology can be divided into three main parts: an adaptive thresholdingto convert a sonography into a useful binary image; a disk expansion to extract the significantobjects, and finally, a refinement of the the extracted object to obtain the more accurate lesionboundary.

Adaptive Thresholding

According to the study presented by Chen and Chen [4], the uneven-brightness problem isan inherent phenomenon on speckle image, and extracting useful information with a planethresholding is tough. Hence, an adaptive thresholding method is proposed to convert a grayspeckle-type ultrasound image into a useful binary image.

This adaptive thresholding consists in a sliding window acting as a Region Of Interest (ROI)of the central pixel which has to be thresholded. This ROI is used to determine a weighting


factor used to adjust the binary thresholding depending on the ratio between the mean intensityvalue of the whole US image and the mean intensity value within the ROI. Since the desiredlesions are hypo-echoic zones, if the ratio is greater than 1 means that the characteristic of theROI tends to be a lesion and thus, the binary thresholding value should be assigned to a highervalue.

Figure 3.3a shows three ROIs examples, representing the speckle (ROI 1), lesion edge (ROI2), and lesion (ROI 3) regions. Figure 3.3b illustrates the Cumulative Distribution Function(CDF) of the three ROIs from figure 3.3a and the already weighted threshold value for eachROI denoted by a circle symbol. Figure 3.3c shows the result after the adaptive thresholdingis applied to the whole image.

Disk Expansion and Object extraction

On the binary image B(xi, yj), for each black pixel we assign a starting disk with radius r = 1and check whether or not the outside neighbors of the disk have white pixels. The disk expandsby increasing its radius r by 1 until a white pixel is found. Every pixel B(xi, yj) has therespective radius R(xi, yj). After checking all black pixels, we can obtain a radius matrix.

Up to here, this step basically consists on a Hilbert transform (introduced by Hilbert in1892). Therefore, what to this work refers, an implementation of a quasi-Euclidean distancetransform described by Rosenfeld and Pfaltz [30] is used for better computational performance.

The Hilbert space of the image (represented in figure 3.3d) is used to determine which regionbelongs to the lesions as an initial estimation, by applying the following steps:

1. Obtain the CDF of the radius assigned.

2. Determine the thresholding radius. In order to find it, a user defined threshold is used tothreshold the CDF and obtain the thresholding radius.

3. Label all the pixels with a radius value higher than the thresholding radius as 1.

4. Label the maximum radius pixel as 2. Grow region labeled as 2 over the pixels labeledas 1 until the region stops, afterwards all the pixels non-labeled as 2 must be labeled as0. Figure 3.3e shows the intensity values of those pixels labeled as 2.

5. Perform some closing operation if needed.

Boundary refinement

From the initial estimation obtained applying the disk expansion procedure, a a refinement isdone by the following steps:


1. Perform a morphological dilation in order to cover the boundary zone close to the regionlabeled as 2. Afterwards label this aggregated zone as 3. Figure 3.3f shows the intensityvalues of those pixels labeled as 3.

To perform this operation the authors propose to apply two times a morphological dilateoperation using a 3x3 structure element. However this can change depending on theapplication.

2. Calculate the intensity mean value of the pixels belonging to the region labeled as 2, aswell as, the intensity mean value of the pixels labeled as 3.

3. Obtain a new label plane by labeling as 1 those pixels whose intensity is closer to region2 mean value and label as 0 those pixels whose intensity is closer to the region 3 meanvalue.

4. Use the labeled image obtained from the previous step as the thresholded image ob-tained from the adaptive thresholding and perform again the disk expansion procedure.Figure 3.3g illustrates the intensity values of the image supplied to the disk expansionprocedure, and, figure 3.3h shows the output after this second iteration.

Reported results

The authors have tried the proposed disk expansion segmentation with simulated images, invitro phantom images and clinical images. The simulation results show that the mean normal-ized true positive area overlap between simulated contour and contour obtained by the proposedalgorithm is over 85% and the mean shortest distance is below 1.5 pixels. There are high corre-lations between 10 experienced physicians’ manual contours and the contours obtained by theproposed method in clinical breast images. The proposed approach has also been verified to beable to simultaneously contour multiple lesions in a single image by applying it multiple timesand removing the already found lesions.

Moreover, the authors also compare the obtained results with a manually initialized GVFsnake algorithm showing that the disk expansion methodology performance is better for bothsimulated and real images.

3.2.3 Level-set segmentation

Savelonas et al. [31–33] propose an algorithm for unsupervised texture segmentation by findingan image plane derived from Local Binary Pattern (LBP) texture descriptor and use a level-setframework such as ACWE to perform the segmentation in such a plane. The authors use such


(a) (b)

(c) (d) (e)

(f) (g) (h)

Figure 3.3: Disk expansion methodology. (a) A simulated ultrasound image G(x, y). Thethree Region Of Interest (ROI) represent speckle (ROI 1), lesion edge (ROI 2), and lesion(ROI 3) regions respectively. (b) The Cumulative Distribution Function (CDF) rising rate ofthe lesion regions (solid line) is larger than that of either lesion edge (dashed line) or speckleregion (dashed-dot line). (c) The obtained binary image B(x, y) after applying the adaptivethresholding process. (d) Hilbert Transform R(x, y). (e) The corresponding gray informationunder the region A2 of the extracted significant object. (f) The outer band region A3. (g) Thecorresponding gray information under the region A2 after the refinement procedure. (h) Thefinally found region.


approach to segment the thyroid glands on US images. Where the thyroid glad appears to be anhypo-echoic region not much different that segmenting a breast lesion from a breast sonography.

As stated, the proposed method is a two steps methodology: first step finds a plane where iseasier to perform the segmentation, and, the second step segments the lesion within the foundplane. In this work we briefly review the LBP texture descriptor, how to obtain the derivedplane form the LBP representation, and, the ACWE methodology also briefly described.

Local Binary Pattern (LBP) descriptor

The LBP operator proposed by Ojala et al. [25, 26] offers an alternative approach for texturerepresentation. The LBP descriptor, as originally defined, utilizes a binary representation of thelocal texture pattern constructed by comparing the neighbor with the current pixel achievingluminance invariance to any monotonic intensity change by construction. The LBP descriptoris obtained as follows: lets assume a local neighborhood P where p = 0 is the central pixel;then the LBP value associated with the central pixel corresponds to equation 3.2 where H is aHeaviside function defined as equation 3.3 and gp means the gray level of the pixel.

LBPp0 =P∑p=1

H(gp − g0) · 2p−1 (3.2)

H(z) =

{1 if z ≥ 00 if z < 0

(3.3)

LBP driven plane

The main idea consists on unfold the LBP descriptor as a binary images obtained from com-paring the LBP with all the LBP possible values. Combine some of those binary images byadding them and smooth the result.

According to Maenpaa [19], a subset of LBP bins may perform even better than the completehistogram for texture discrimination. Therefore instead of generate a binary image of eachpossible value only the b most frequently LBP values are used to generate the binary imagesset.

From those b binary images a set of cumulative images is generated. The cumulative imageset is generated by applying the logical or operation to a permutation of the binary imagestaken in sets from 1 to b.

The desired plane is one of the generated cumulative images. In order to select the bestimage, a contrast index ξ(i) of the image i is defined as equation 3.4 where Pw(i, j) is thenumber of white pixels that contain the image i and the properly sized block j. Hence the

19 3.3 Boundary and region infromation mixture to achieve better performance on US images

cumulative image i that maximizes the contrast accordingly to equation 3.5 will be selectedand smoothed for a further segmentation with ACWE.

ξ(i) = maxj

(Pw(i, j))−minj

(Pw(i, j)) (3.4)

i = argi(max(ξ(i))) (3.5)

Active Contour Without Edges (ACWE)

The ACWE model proposed by Chan et al. [2] performs region-based segmentation as aMumford-Shah minimization problem (see Mumford and Shah [23]) using the level set methodproposed by Osher and Sethian [27, 28] which provides an efficient way for moving curves andsurfaces on a fixed regular grid, allowing for automatic topology changes such as merging andsplitting.

3.3 Boundary and region infromation mixture to achieve

better performance on US images

Producing accurate segmentation results when performing segmentation on sonography oftenfail when using either boundary-based or region-based methodologies. Boundary-based method-ologies tries to segment the image into regions finding the acoustic impedance discontinuitiesproduced by different tissue interfaces or boundaries. On the other hand region-based method-ologies aim to characterize the tissue area by differencing its depicted region pattern.

Hence this segmentation inaccuracy caused by follow one clear technique either using pixeldiscontinuities or using pixel similarities, has been tried to overcome towards algorithms whichtake advantage of the complementary nature of such information. Although this informationmerging has been applied on different image segmentation fields, not only applied to US images,it perfectly suits the Ultra-Sound (US) imaging scenario. However, as pointed by Madabhushiand Metaxas [18] due to the relatively low quality of clinical Ultra-Sound images the combinationof both approaches is not enough to achieve good Ultra-Sound image segmentation. Therefore,an US image segmentation methodology needs to make use of all task-specific constraints orpriors given by its application.

This section also classifies those methodologies accordingly to the six highlighted strate-gies or methodologies to fuse region and gradient information pointed by Freixenet et al. [8].In such work they first distinguish between embedded and post-processing methods. Withinthe embedded methods they differentiate between those using boundary information for seed


placement purposes, and those which use this information to establish an appropriate decisioncriterion. Within the post-processing methods, they differentiate three different approaches:over-segmentation, boundary refinement (using Multi Resolution Analysis (MRA) or snakes),and selection evaluation

3.4 Post-processing boundary refinement methodologies

A very interesting approach for automated segmentation of ultrasonic breast lesions is proposedby Madabhushi and Metaxas [17, 18, 20]. It takes advantage of merge boundary and regioninformation towards a post-processing boundary refinement using a snake-like methodology.The authors propose a novel technique to automatically find lesions margins in ultrasoundimages, making use of the balloon method to deform a model initialized to the boundary points.This initialization is given by the contour obtained from a naive lesion segmenting using a regiongrowing procedure. The region growing initialization is also automatic, by combining intensityand texture with empirical domain specific knowledge.

Figure 3.4 reproduces the procedure of the proposed method.

3.4.1 Methodology steps

Image Enhancement

As stated before, speckle is strongly present on US images, making them more complex anddropping down their SNR. In order to increase the image quality, the methodology also proposesan image enhancing step. This enhancing is composed by a filtering step and a histogramequalization. Originally [17], this filtering step is a variation of the second order 2D Butterworthfilter proposed by Tang et al. [38]. However, the authors spot that a median filter can also beused [18,20].

Figure 3.5 illustrates the image enhancement procedure. Figure 3.5a shows the originalimage. Figure 3.5b shows the result of Butterworth filtering.We can see that the speckle noisein 3.5a has been reduced in 3.5b. Figure 3.5c shows the result of scaling and equalization toproduce the contrast-enhanced image. The tumor which is in the upper middle portion of theimage is barely visible in 3.5a but is clearly apparent in figure 3.5c. We have drawn an arrow infigure 3.5c to show its position. As can be seen, the margins of the tumor have been enhanced.

Obtaining the joint probability plane

The region-based methodology part of the hybrid segmentation methodology proposed in thismethod neither takes place in the intensity nor in the texture plane, but takes into account the

21 3.4 Post-processing boundary refinement methodologies

Figure 3.4: Madabhushi and Metaxas procedure diagram


(a) (b) (c)

Figure 3.5: Image enhancement. (a) Original image. (b) Butterworth filtered. (c) Finalenhanced image.

malignant probability of each pixel. This implies two steps procedure: firstly a training step inorder to extract the Probability Density Functions (PDFs) of the intensity and texture planesfrom a manually segmented sonography set; secondly the generation of the lesion probabilityplane is performed. This plane is the joint probability that a pixel intensity and its texturebelong to a lesion with respect the training step. It has to be noted that the authors, performthe training step with a reduced image set. This training step is performed only once.

Figure 3.6a shows a sonogram with a large circular cyst in the lower portion of the image.The scaled probability images based on intensity and texture are shown in figure 3.6b andfigure 3.6c. As can be seen in the joint probability image figure 3.6d, most of the pixels havinga high probability of belonging to a lesion (brighter pixels) have been identified correctly withinthe cyst.

(a) Enhanced image (b) intensity probability (c) texture probability (d) joint probability

Figure 3.6: Probability planes of an imaged cyst. (a) Enhanced image. (b) Intensity probability.(c) Texture probability. (d) Joint probability.

23 3.4 Post-processing boundary refinement methodologies

Determining the seed point

For automatically determining the seed point within the lesion, a mathematical formulation,popularly known as the Stavros Criteria [36], is used. The Stavros formulation consists on anempirical rule commonly used by radiologists to detect ultrasonic breast lesions. This criterionis used in conjunction with the image lesion probability plane to pronounce the seed pointcandidate.

Once obtained, the seed is used in both region-based and boundary-based steps of themethod. The region-based step uses the seed as a initial condition for the region growing, whilethe boundary-based step uses it to obtain a concentric gradient plane to constrain the snake-likeprocedure.

Region Of Interest (ROI) growing

A region growing algorithm is performed in the joint probability image in order to obtain anaive segmentation. Pixels in the vicinity of the previously obtained seed point are groupedtogether based on their probability and connectivity. This segmentation is used to guess thefirst approximation of the lesion boundary set as a ROI, as depicted in image 3.7.

(a) (b) (c) (d)

Figure 3.7: Obtaining the orthogonal lesion boundary. (a) Original image, (b) Joint probabilityplane, (c) Binarized result of the region growing, (d) Boundary points

Directional gradient

The directional gradient is obtained by means of the computation of the directional derivative ofthe contrast-enhanced image based on the seed point. Tis is subsequently used for determiningboundary points on the lesion margin in order to refine and complete the initial lesion boundary(figure 3.7d) to achieve a more accurate initialization for the deformable model procedure.Figure 3.8 compares the magnitude gradient (see fig. 3.8b) with the circular (or, also called,directional) gradient (see fig. 3.8c) form the original image shown in figure 3.8a. On figure 3.8can be observed that tumor margin is more pronounced for the directional gradient rather thanfor the magnitude gradient that has much less structure by comparison.


(a) (b) (c)

Figure 3.8: Radial gradient example. (a) Original image. (b) Gradient magnitude of (a). (c)Directional gradient of (a).

Finding boundary points to initialize the deformable model

Once obtained the directional gradient image, it is scanned obtaining the line profiles passingtowards the seed point. This circular gradient profiles are used to find the boundary seeds ornodes to initialize the deformable model. In order to find this nodes, the boundary obtainedfrom the region growing is used to constrain where is the boundary. This procedure is doneiteratively until is minimized the error between the new boundary found and the initial boundaryobtained from the region growing step. Figure 3.9a illustrates the scanned lines trough the seedpoint in order to determine the boundary and figure 3.9b are some of those profiles highlightingthe seed point and the boundary position.

(a) (b)

Figure 3.9: Lesion boundary finding using radial gradient. (a) Scanned lines towards the seedpoint (b)Some of line profiles.

Boundary refinement

The previously obtained boundary nodes are supplied as an initial estimation to the deformablemodel. Instead of using just the gradient magnitude as Kass et al. [13]or Chen and Metaxas [3],the approach here analyzed proposes to use the directional gradient as the stop criterion. Thisapproach is more robust to noise and complex backgrounds. Incorporating empirical domain

25 3.5 Post-processing over-segmentation

specific knowledge along with low and high-level knowledge makes it possible to avoid shadowingartifacts and lowers the chance of confusing similar tumor like structures for the lesion.

Figure 3.10 shows the system results for a tumor with poorly defined margins.

(a) (b) (c) (d)

Figure 3.10: Lesion segmentation illustrations. (a) Original image. (b) Contrast-enhancedversion. (c) Detected boundary points. (d) Output of deformable model.

3.4.2 Reported results

The system was validated on a database of breast sonograms for 42 patients. The average meanboundary error between manual and automated segmentation was 6.6 pixels and the normalizedtrue positive area overlap was 75.1%. The algorithm was found to be robust to 1) variations insystem parameters, 2) number of training samples used, and 3) the position of the seed pointwithin the tumor.

3.5 Post-processing over-segmentation

This boundary and region information merging approach is useful to improve the results ofthose segmentation methods in which false boundaries are generated, for instance watershedtransformation. Pavlidis and Liow [29] argued that such segmentation results can be signifi-cantly improved if all region boundaries qualified as edges are checked rather than attemptingto fine-tune the uniformity criteria. So, this approach starts with the obtention of an overseg-mented result segmentation, which is achieved by properly setting the parameters of the naivesegmentation algorithm. Afterwards, this resulting segmentation is improved as the result froma dual approach: each boundary is checked to see if it is coherent in both results. When thiscorrespondence does not exist, the boundary is considered false and is removed. At the end,only real boundaries are preserved.

A good example to illustrate this kind of information merging is proposed by Huang andChen [12]. The proposed method performs a watershed transformation over a preprocessedimage and afterwards the found boundaries are contrasted by means of markers extracted from


the preprocessed image. Figure 3.11 illustrates the methodology procedure, where can easily beseen the two mainly steps. The first step consists on a texture-based pre-processing applying aSelf Organizing Map (SOM) model to classify the input images. Several preprocessing filters areespecially designed for each textural class. Afterwards, a watershed transformation algorithmautomatically produces the lesion contour. The preprocessing phase prevents over-segmentationand improves the watershed segmentation.

Figure 3.11: Huang and Chen procedure diagram

3.5.1 Watershed transformation

The purpose of the watershed transformation is to determine the watershed lines on a topo-graphic surface. To find those watershed boundaries the intensity of a pixel is interpreted as theelevation of the corresponding position. When the water floods from each regional minimumsome dams are built in order to prevent merging the water that rises through two adjacent

27 3.5 Post-processing over-segmentation

minima. Eventually, each regional minimum is surrounded by dams. Figure 3.12 shows the wa-tershed transformation to be applied to a topographic surface, which exemplifies an ultrasonicimage.

Watershed segmentation yields to over-segmentation when lots of local minima appear inthe image, as in sonographs due to speckle noise. Therefore, some markers are generated fromthe preprocessing step in order to double check the dams found as a lesion b oundary.

Figure 3.12: Topographic view of watershed transformation.

3.5.2 Reported results

For this study the authors use a randomly selected set of 60 images. The experimental resultsreveal that the proposed method can practically determine the contours of a breast tumor fromUS images if compared with the manual segmentation. This study is not to emphasize that theautomatic contouring technique is superior to the one undertaken manually.

Chapter 4

Proposal

This chapter presents a novel approach in order to segment anechoic and hypo-echoic regions ona sonography in an automatic way assisting the radiologists to perform their diagnosis. It takesadvantage from the fact that dark zones are lesion suspicious areas, since Stavros et al. [36]characterized malignant breast lesions as markedly hypo-echoic regions of a breast sonogram.Similarly, Leucth [15] stated that anechoic is the hallmark of a cyst. Hence, both cysts andmalignant lesion appear darker than glandular tissue or fat which are usually either isoechoicor hyperechoic. To segment a possible lesion zone, the proposed segmentation methodologyconsists on a region-based and boundary-based methodology mixture.

The proposed segmentation methodology can be mainly assumed as a two steps procedure:

1. The obtention of a preliminary segmentation from information extracted both from a setof sonograms used to train the system, and from information driven from the US imageacquisition procedure knowledge.

2. A refinement step to achieve a region-like segmentation.

Our approach has been inspired in some of the methodologies analyzed in chapter 3 (Stateof the Art). For instance, the proposed method contains a region growing and an automaticseed placement for the region growing recalling the methodology proposed by Madabhushi andMetaxas [17,18,20]. Besides, our proposal uses relevant information from the initial segmenta-tion in order to feedback the segmentation, as in the work of Yeh et al. [43,44]. The output ofthe region-growing algorithm is used as input for thte later boundary refinement of the lesion,as in the work of Horsch et al. [11], where boundary information is used to achieve a morelesion-like segmentation.

28

29 4.1 Methodology

4.1 Methodology

The proposed methodology is illustrated in figure 4.1. The whole segmentation methodology isdriven from the intensity and texture plane of the sonography, as well as from some knowledgeextracted from a training step and the observation of the radiologists during the US imagesacquisition.

There are two mainly steps. The first step is used to determine the gaussian function thatbest adjust to the lesion, while the second step performs the segmentation by constraining andthresholding the previous gaussian function.

The determination of the gaussian function is composed by the seed determination, a re-gion growing operation and an analysis to determine the best fitting gaussian function. Thisprocedure is done twice in order to refine the result using the given gaussian function as theprobability seed placement constraint. The seed determination procedure is performed over theintensity and texture plane of the image. As the seed is selected as the most probable placewhere the lesion is, the PDF of the intensity and texture of the pixels that belong to a lesion areneeded. This information is provided by a training step procedure using a manually segmentedsonogram dataset. Moreover, this seed placement needs a probability distribution of the lesionplacement. With all this data the best candidate region is selected to be grown using a regiongrowing procedure. The output of this region growing is simplified with some closure operationand used determine which parameters (mean and covariance) for a gaussian function describesbetter the obtained ROI.

The gaussian constraining segmentation consists on a pre-processing step that transformsthe intensity plane on a more useful simplified plane that is weighted accordingly to the gaus-sian distribution obtained on the previous step. To perform the segmentation a threshold isdetermined and used to threshold the image obtaining the desired segmentation.

4.1.1 Seed placement (initial region)

As in [36] we make use of the fact that echogenicity feature driven from the intensity of thesonogram pixels is a major criteria to find breast lesions in US images. However, due to theinherent image artifacts of the sonograms, the internal echo pattern (referring to texture orecho arrangements) within a focal sonographic lesion has to be taken into account. Hence,both intensity and texture have a high specificity for characterizing breast masses. Simillaryto [17, 18, 20], the proposed texture to use is a distance measure between the pixel intensityI(x, y) and its N neighbors intensity mean, following equation 4.1. In that sense, the localvariance texture seems to be able to appropriately capture the internal echo pattern of a lesion.

Chapter 4: Proposal 30

Figure 4.1: Block diagram of the proposed method

31 4.1 Methodology

T (x, y) = I(x, y)− 1N

N∑δ=1

Wδ(x, y) (4.1)

This texture plane T (x, y) is used in conjunction to the intensity plane I(x, y) to comeup with a Joint probability plane Γ(x, y) providing the methodology with the probability ofa pixel to be a lesion. Such a plane is calculated as illustrates equation 4.2 that depends onthe probability of a pixel to be a lesion regarding its intensity ΓI = P (Lesion|I(x, y)) and itstexture ΓT = P (Lesion|T (x, y)). Although those terms are a priory unknown, they can beassumed equal to P (I(x, y)|Lesion) and P (T (x, y)|Lesion) respectively. This equality comesfrom two assumptions: on one hand all the available sonograms used in this work contain alesion; and on the other hand, the probability of each possible value (either intensity or texture)has been assumed equiprobable. Thus, obtaining the probability of a pixel to belong to a lesionhas been steered to compute the P (I(x, y)|Lesion) and P (T (x, y)|Lesion) that can be derivedfrom a manual segmentation done by an expert. Those probability priors stand for the PDF ofthe pixels belonging to a lesion in the images supplied for the training step. Figure 4.2 showsthe PDFs output provided by a training step.

(a) (b)

Figure 4.2: PDFs extracted from a training procedure. (a) PDF of the Intensity values thatbelong to lesion regions. (b) PDF of the texture values inside the lesion.

Γ(x, y) = ΓI(x, y) · ΓT (x, y) (4.2)

The benefit of this joint probability plane Γ is to find the best candidate for initializethe region growing procedure. However, the obtained plane is not enough due to the factthat dark zones such shadows and other inherent sonography artifacts can be mistaken as themost probable pixel inside the lesion when it really lies outside. To cope this situation weuse a simplified version of the Stavros rule by weighting the probability plane with a distancefunction. The used distance function consists on a 2D gaussian with its mean value equal to the


image centroid and a covariance value proportional to the image aspect ratio. Such approachtakes advantage of the knowledge that when a radiologist take a sonogram, the lesion is placedin the central part of the image as a way to facilitate the lesion observation.

Since the constrain we use is less restrictive than the original Stavros rule, the pixel withhighest probability might does not lie inside the lesion. Therefore, the seed selection consistson picking out the widest area over a probability threshold. Afterwards, this area is suppliedto the region growing procedure.

For the second iteration towards the region-based operations, the distribution of the seedplacement is updated with the gaussian approximation of the lesion. Figure 4.3 illustrates theseed region determination for a second iteration procedure. As commented before, for the firstiteration the seed probability distribution (fig. 4.6b) should be a gaussian function centered onthe central pixel of the image and its covariance value proportional to the image aspect ratio.

(a) (b)

(c) (d)

Figure 4.3: Illustration of the seed placement for a second iteration. (a) Joint probability planeΓ(x, y). (b) Seed placement probability. (c) Seed probability. (d) Iinitial Region R0

33 4.1 Methodology

4.1.2 Region growing segmentation

Since the success of an image segmentation algorithm is directly associated with the choice ofthe measures and a suitable threshold, we have selected for this work an adaptive similaritycriterion that uses the mean and the standard deviation values as control parameters to be usedfor region growing.

The proposed algorithm starts with an initial region of one or more pixels that lie inside thedesired ROI. The gathering function for the region growing procedure spreads this initial regionby adding its adjacent pixels that have similar properties. The spreading behavior depends ona user defined structural element of any shape or dimension used to expand the region as amorphological dilation. If a pixel of the spread zone is similar enough to the region, thenit is aggregated. Otherwise, the pixel is rejected. This similarity or homogeneity criteriondepends on the mean and standard deviation statistical properties of the region accordinglyto equation 4.3. For a given iteration i, V ′(x, y) is a matrix containing the value V (x, y) ofthe spread zone pixels on this instant, V (x, y) is the image plane used to perform the regiongrowing, α is a weighting factor defined by the user, µR is the mean of the region and σR itsstandard deviation. Figure 4.4 provides an example of the preliminary segmentation obtainedusing the region growing operation.

Ri(x, y) =

{1 if |V (x, y)− µR|α · σR or Ri−1(x, y) = 10 otherwise

(4.3)

µR = µ(V (x, y)) | Ri−1(x, y) = 1 (4.4)

σR = σ(V (x, y)) | Ri−1(x, y) = 1 (4.5)

V ′(x, y) =

{V (x, y)1 (x, y) ∈ Spread regionundefined otherwise

(4.6)

4.1.3 Best Fitting Gaussian

The idea behind this step consist in find a gaussian function that best fits the breast lesion.The proposed step improve the Horsch et al. [11] proposal. Other authors propose to use a userdefined centroid and an a methodology to obtain an automatic diagonal covariance matrix. Toimprove the results we propose a blob analysis using Principal Component Analysis (PCA) todetermine the gaussian function covariance as well as its mean value. Figure 4.6 illustrates theconstraining gaussian distribution and the lesion fitting.


Figure 4.4: Region growing output

(a) (b)

(c) (d)

Figure 4.5: This image illustrates how is fitted the breast lesion using a gaussian distribution.(a) Original image. (b) Gaussian distribution Gµ,σ(x, y). (c) Gaussian and image overlaping.(d) Comparation between the ground truth and the gaussian distribution Gµ,σ(x, y).

35 4.1 Methodology

4.1.4 Gaussian constraining segmentation

The segmentation method begins with preprocessing the image I(x, y) using a 5x5 median filterin order to suppress the ultrasound speckle.

The Gaussian constraining method was prosed by Kupinski and Giger [14] that use thismethod for lesions in Digital Mammography with the effect of suppressing distant pixel valuesand encouraging potential lesion margins to be more lesion-like simply multiplying by a gaussianfunction, yet it can not be used straight forward for US images. The main issue is that for DMimaging the lesions appear as high values, while for US imaging, the lesions are basically dark.To cope this trouble the constrained function should be calculate as illustrates equation 4.7where Gµ,σ(x, y) is the constraining function and I ′(x, y) is the pre-processed image. Theproposed approach essentially inverts the image lesions from dark to light assuming high valuesas a potential lesion.

G′(x, y) = Gµ,σ(x, y) ·(

1− I ′(x, y)max(I ′(x, y))

)(4.7)

Once determined the gaussian constrained function of the lesion (G′(x, y)), the final seg-mentation S(x, y) is performed by thresholding such function (see eq. 4.8). Accordingly toother authors the threshold th should be found using boundary maximizing a cost functionthat depend on the threshold.

S(x, y) =

{1 if G′(x, y) > th

0 otherwise(4.8)


(a) (b)

(c) (d)

Figure 4.6: Segmentation of some examples using different thresholding values. From outside toinside the used threshold values are: 0.3 0.5 0.74 0.8 0.9 0.95. The boundary colors represent thethreshold value from cold to hot colors. (a) image 20090510002 (b) 20090510028 (c) 20090315008(d) 20090330001

Chapter 5

Experimental setup

5.1 Dataset

One of the common problems when talking about performing a study about segmentationtechniques on mammography sonograms is the lack of a public database that could be used torun the experiments and compare the results with other authors. Although many authors havespotted this absence (among others Noble and Boukerroui [24], and Shrimali et al. [34] shouldbe cited), this issue still uncovered. However, there exist references to some public database,such as Tian et al. [40], but despite the authors claim it is available, the servers were missing.On top of that, and not happy with the vanished dataset, we tried to contact the authorswhereas they gave no answer.

In order to perform the experiments for this work, it has been collected a set of fifteensonograms. Figures ?? and 5.3 show the twenty one sonograms that composes our dataset. Ascan be observed on the figure, the dataset is formed by three simulated US images and eighteenreal cases acquired in the hospital Josep Trueta of Girona. All the images forming the datasethave a ground truth corresponding to a manual segmentation performed by an expert.

In order to facilitate the manual segmentation and the management of the sonograms andits segmentations, it has been implemented an small user interface with basic operations. Thoseoperations include:

• Load a sonographic dataset with its manual segmentation.

• Change the manual segmentation associated with a desired sonogram.

• Visualize both the manual and the automatic segmentation as well as their overlap.

• Save the dataset and its associated segmentations.

37

Chapter 5: Experimental setup 38

Figure 5.1: Experimental setup diagram

5.2 Experiments procedure

Due to the fact of having a reduced dataset and the fact that the proposed methodologyrequires a training step, the experiments have been designed accordingly to those restrictions.Figure 5.1 illustrates the proposed experiment procedure. The proposed test consists in performthe automatic segmentation over all the sonograms one by one using a leave one out technique.Such cross-validation technique consists in use all the dataset but the selected sonography toperform the training step. Hence, the statistics obtained from the rest of the dataset are usedperform the seed placement for the selected image.

Once all the images are automatically segmented a confusion matrix between the manualand the automatic segmentation is created. This confusion matrices allows the comparison ofboth segmentations and aids to grade the proposed approach performance.

39 5.2 Experiments procedure

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

(j) (k) (l)

Figure 5.2: Sonographic lesions image data base (1)


(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

Figure 5.3: Sonographic lesions image data base (2)

41 5.3 Results

5.3 Results

To evaluate the accuracy of our segmentation scheme, we quantitatively compared the auto-mated results with the manual delineations produced by one expert. Although we would likedto compare the results of our automated segmentation technique with the delineations of mul-tiple experts, for the present work we used the delineations of the single expert as the goldstandard. Figure 5.4 shows a visual interpretation of the achieved segmentation results usingour proposal method. On those images, the blue zone corresponds to the false negative zoneswhere pixels manually labeled as a lesion but automatically segmented as no lesion. The redregion is the false positive zone where the segmentation algorithm suggest that there is lesionwhen it is not really present. The overlapping zones correspond to the green zones for thoseregions with a lesion presence and rest is not colored.

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

Figure 5.4: Graphical representation for the comparison between the ground truth and theachieved results by the method.


Table 5.1: Segmented areas pixel classification

FP% FN% TP% TN%1.35 1.09 8.35 89.2112.53 10.12 77.35 ∼

Although the result compression can be visually done, it becomes a vast effort when thedataset size grows up and valorize how good is the performed segmentation on an automaticway is not solved. Chalana and Kim [1] discussed he lack of a well-defined metric to comparethe computer-generated segmentation results to the segmentation results produced by expertobservers. In this work the metric to compare the automated an manual delineations is based onthe overlapping regions. Table 5.1 provides comparison by using the confusion matrix represen-tation. This table reports the percentage of pixels correctly classified and miss-classified. Theobtained results show that the system is classifying correctly a pixel in a 97.56% of cases. Thishigh value is manly due to the number of true negative elements. If we take only into accountthe classification on the conjunction area the information is more reliable. Table 5.1 also showsthe classification percentage of the pixels within the overlapping area. Such table shows that thetrue positive classification is over 75% which is a good result for the preliminary methodologyproposed for this work. Here has to be said that Madabushi and Metaxas [17,18,20] reporteda true positive ratio around 74%. This is attributed to the fact that our images are mainlysimilar one each other and not much of them differ.

To analyze the automatic segmentation, it has also been used the overlap ratio (O) calculatedas equation 5.1. Where Θ is the set of points in the manually segmented lesion and Π the setof points in the computer-segmented lesion. The mean value of the overlapping coefficient forour experiments have been an overlapping area of 0.78

O =Area(Θ ∩Π)Area(Θ ∪Π)

(5.1)

Chapter 6

Conclusion and Future work

The use of sonogrpahy by itself as a screening tool is still controversial. However it is usefulunder a dense breast scenario where Digital Mammography techniques fail. But US images haveinherent problems such speckle noise and the tissue-related texture making difficult to identifythe contours of the anatomical structures accurately. In this work some of those artifactsformation has been reviewed as well as their consequences. It has been also reviewed thecurrent tends in segmenting breast lesions in such challenging image modality.

More over it has been posed a methodology to cope with automatic segmentation. Andfinally the performance of the proposed methodology has been reviewed.

This work has also pointed out the absence of a public data base of B-mode US breastimages. Although we have already started, the data collection and its segmentations associatedas well as its publication is something that remains to do as further work. This data collectionis a task enclosed on a breast lesion segmentation benchmark (as a Matlab toolbox ). Such abenchmark also requires a deeper research in segmentation metrics in order to rate the testingmethods. And, to compare the testing methods the toolbox should incorporate a collection ofclassical segmentation methodologies from the bibliography.

43

Bibliography

[1] V Chalana and Y Kim. A methodology for evaluation of boundary detection algorithmsonmedical images. IEEE Transactions on medical imaging, Jan 1997.

[2] T Chan, B Sandberg, and L Vese. Active contours without edges for vector-valued images.Journal of Visual Communication and Image Representation, Jan 2000.

[3] T Chen and D Metaxas. Image segmentation based on the integration of markov randomfields and deformable models. LECTURE NOTES IN COMPUTER SCIENCE, Jan 2000.

[4] Yung-Sheng Chen and Bor-Tow Chen. Measuring of a three-dimensional surface by use ofa spatial distance computation. Applied optics, 42(11):1958–72, Apr 2003.

[5] K Drukker, M Giger, K Horsch, and M Kupinski. Computerized lesion detection on breastultrasound. Medical Physics, Jan 2002.

[6] J Duncan and N Ayache. Medical image analysis: progress over two decades and thechal-lenges ahead. IEEE Transactions on Pattern Analysis and Machine . . . , Jan 2000.

[7] Dale Ensminger and Foster B. Stulen. Ultrasonics: Data, equations, and their practicaluses. page 520, Jan 2008.

[8] J Freixenet, X Munoz, D Raba, J Martı, and X Cufı. Yet another survey on image seg-mentation: Region and boundary information integration. LECTURE NOTES IN COM-PUTER SCIENCE, Jan 2002.

[9] M Giger. Computer-aided diagnosis in medical imaging–a new era in image interpretation.Medical Imaging-Ultrasound.

[10] William R. Hendee and E Russell Ritenour. Medical imaging physics. page 512, Jan 2002.

[11] K Horsch, M Giger, L Venta, and C Vyborny. Automatic segmentation of breast lesionson ultrasound. Medical Physics, Jan 2001.

44

45 BIBLIOGRAPHY

[12] YL Huang and DR Chen. Watershed segmentation for breast tumor in 2-d sonography.Ultrasound in Medicine & Biology, 30(5):625–632, 2004.

[13] M Kass, A Witkin, and D Terzopoulos. Snakes: Active contour models. InternationalJournal of Computer Vision, Jan 1988.

[14] M Kupinski and M Giger. Automated seeded lesion segmentation on digital mammograms.IEEE Transactions on medical imaging, Jan 1998.

[15] Dagmar Leucht and H Madjar. Teaching atlas of breast ultrasound. page 250, Jan 1996.

[16] Y Tsay-Der Lin. An iterative approach to removing the” closure noise” using disk expansionmethod. IEEE Signal Processing Letters, Jan 1995.

[17] A Madabhushi and D Metaxas. Automatic boundary extraction of ultrasonic breast lesions.Biomedical Imaging, 2002. Proceedings. 2002 IEEE International Symposium on, pages 601– 604, Jun 2002.

[18] A Madabhushi and D Metaxas. Combining low-, high-level and empirical domain knowl-edge for automated segmentation of ultrasonic breast lesions. IEEE Transactions on med-ical imaging, Jan 2003.

[19] Topi Maenpaa. The local binary pattern approach to texture analysis: Extenxions and ....Jan 2003.

[20] D Metaxas and A Madabhushi. Method and apparatus for automatically detecting breastlesions and tumors in images. US Patent App. 10/736,455, Jan 2005.

[21] S Moore. Better breast cancer detection. IEEE Spectrum, Jan 2001.

[22] M Moskowitz, S Feig, C Cole-Beuglet, and S Fox. Evaluation of new imaging proceduresfor breast cancer: proper process. American Journal of Roentgenology, Jan 1983.

[23] D Mumford and J Shah. Optimal approximations by piecewise smooth functions andassociated variational problems. Comm. Pure Appl. Math, Jan 1989.

[24] JA Noble and D Boukerroui. Ultrasound image segmentation: A survey. IEEE Transac-tions on medical imaging, Jan 2006.

[25] T Ojala, M Pietikainen, and T Maenpaa. Multiresolution gray-scale and rotation invarianttextureclassification with local binary patterns. IEEE Transactions on Pattern Analysisand Machine . . . , Jan 2002.

BIBLIOGRAPHY 46

[26] T Ojala, M Pietikainen, T Maenpaa, and J Viertola. Outex-new framework for empiricalevaluation of texture analysis algorithms. International Conference on Pattern Recognition,Jan 2002.

[27] S Osher and RP Fedkiw. Level set methods and dynamic implicit surfaces. 2003.

[28] S Osher and J Sethian. Fronts propagating with curvature dependent speed: algorithmsbased on hamilton-jacobi formulations. Journal of computational physics, Jan 1988.

[29] T Pavlidis and Y Liow. Integrating region growing and edge detection. IEEE Transactionson Pattern Analysis and Machine . . . , Jan 1990.

[30] A Rosenfeld and J Pfaltz. Sequential operations in digital picture processing. Journal ofthe ACM (JACM), Jan 1966.

[31] M Savelonas, D Iakovidis, I Legakis, and D Maroulis. Active contours guided by echogenic-ity and texture for delineation of thyroid nodules in ultrasound . . . . IEEE transactions oninformation technology in biomedicine: . . . .

[32] M Savelonas, D Iakovidis, and D Maroulis. An lbp-based active contour algorithm forunsupervised texture segmentation. Proceedings of the 18th International Conference onPattern . . . , Jan 2006.

[33] M Savelonas, D Iakovidis, and D Maroulis. Lbp-guided active contours. Pattern RecognitionLetters, Jan 2008.

[34] V Shrimali, R Anand, and V Kumar. Current trends in segmentation of medical ultrasoundb-mode images: A review. . . . Trends in Segmentation of Medical Ultrasound B-modeImages: . . . , Jan 2009.

[35] Radhika Sivaramakrishna, Kimerly A Powell, Michael L Lieber, William A Chilcote, andRaj Shekhar. Texture analysis of lesions in breast ultrasound images. Computerized medicalimaging and graphics : the official journal of the Computerized Medical Imaging Society,26(5):303–7, Jan 2002.

[36] A Thomas Stavros, Cynthia L. Rapp, and Steve H. Parker. Breast ultrasound. page 1015,Jan 2004.

[37] Thomas L Szabo. Diagnostic ultrasound imaging: Inside out. pages 1–572, Jun 2007.

[38] H Tang, T Zhuang, and E Wu. Realizations of fast 2-d/3-d image filtering and enhance-ment. IEEE Transactions on medical imaging, Jan 2001.

47 BIBLIOGRAPHY

[39] K Taylor, C Merritt, C Piccoli, and R Schmidt. Ultrasound as a complement to mammog-raphy and breast examination to characterize breast masses. Ultrasound in Medicine &Biology, Jan 2002.

[40] J Tian, Y Wang, J Huang, C Ning, H Wang, and Y Liu. title: The digital database forbreast ultrasound image publication. atlantis-press.com.

[41] C Xu, A Yezzi Jr, and J Prince. On the relationship between parametric and geometricactivecontours. Signals, Jan 2000.

[42] C Xu and J Prince. Snakes, shapes, and gradient vector flow. IEEE Transactions on imageProcessing, Jan 1998.

[43] C Yeh, Y Chen, W Fan, and W Chen. A novel method for automatic contour extractionof ultrasonic breast lesions. 2005 IEEE Ultrasonics Symposium, Jan 2005.

[44] C Yeh, Y Chen, W Fan, and Y Liao. A disk expansion segmentation method for ultrasonicbreast lesions. Pattern Recognition, Jan 2008.

[45] S Zucker and C Science Center. [book] region growing: Childhood and adolescence. Uni-versity of Maryland, Jan 1975.

sik

Documents

breast lesion segmentation

segmentation hints

bmode us images

orthogonal lesion boundary

boundary information

us imaging

us artifacts

metaxas procedure diagram