Detection of Pigment Network in Dermoscopy Images

Ana Catarina Fidalgo Barata(Degree in Biomedical Engineering)

Master Thesis in

Biomedical Engineering

JuryPresident: Patrıcia Margarida Piedade FigueiredoSupervisor: Jorge dos Santos Salvador MarquesCo-Supervisor: Armando Jorge de Mariz RozeiraMember: Teresa Maria de Gouveia Torres Feio MendoncaMember: Maria Margarida Campos da Silveira

Friday 2nd December, 2011

Research is to see what everybody else has seen, and to think what nobody else has thought.Albert Szent-Gyorgi (1893-1986) U. S. biochemist

Acknowledgments

This work was supported by FCT in the scope of project PTDC/SAU-BEB/103471/2008.

I would like to thank to Prof. Jorge Salvador Marques for all the supportive guidance, ideas and

help during this thesis, regarding its engineering aspects. For all the time invested, I am also thankful

to him. I also would like to thank to Dr. Jorge Rozeira for all the support, help with the thesis’ medical

aspects, for the tutoring and for providing the access to the dermoscopy database of Hospital Pedro

Hispano. Without their collaboration and guidance, this thesis would not have been possible.

Ms. Barbara Amorim, who selected and organized the database used in this work, thanks to her

for this. I would also like to express my thanks to Profs. Teresa Mendonca, Andre Marcal from the

University of Porto, Drs. Ofelia Morais, Marta Pereira, Marta Teixeira and nurse Lina Santos from

Hospital Pedro Hispano, for their support during this work.

To my parents, who always encouraged and supported me and that gave me the conditions to

study and to pursuit the future that I would like to have, and to my sister and my grandfather, for all

their support and comprehension, I thank to them.

Because life is more than just work and I needed to rest at some points during the conception of

this thesis, to all my friends thank you for the enjoyable moments! I would like to address a special

thanks to Jo, for keeping me sane and for all the patience that he showed during my most stressed

and grumpy moments.

i

Abstract

Melanoma is one of the deadliest forms of cancer. However, an early diagnosis significantly im-

proves the probability of survival. Several medical procedures have been proposed to examine and

diagnose skin lesions. Dermoscopy is an inpection method used by dermatologists, which magnifies

the size of a lesion up to 100x, allowing a better visualization of several dermoscopic structures that

are invisible to the naked eye. One of the dermoscopic features that becomes visible is Pigment Net-

work, which is used in several medical procedures (ABCD rule, 7-point checklist) not only to classify

skin lesions but also to diagnose a melanoma. The automatic detection of pigment network and the

extraction of its shape can help dermatologists performing a diagnosis. Despite its importance, pig-

ment network is not used in automatic systems which try to classify skin lesions. A few works which

address the detection of this structure can be found in literature. However, most of them rely only

on detecting its presence, without extracting its skeleton, which is important for dermatologists. This

thesis proposes a new method for detecting and extracting pigment network in dermoscopy images

based on its color and geometry. The detection and extraction of pigment network will be accom-

plished by using a bank of directional filters and a connected component analysis. A methodology for

classifying lesions regarding the presence of pigment network was also developed. The algorithm for

network detection achieves a Sensitivity (SE) of 78% and a Specificity SP of 77% in a database of 57

clinical images acquired at Hospital Pedro Hispano.

Keywords

Skin lesion classification, Dermoscopy, Melanoma, Pigment Network detection, Directional Filters,

Connected Component Analysis, Features Extraction

iii

Resumo

O melanoma e uma das formas de cancro mais letais. Contudo, um diagnostico precoce aumenta

consideravelmente as hipoteses de sobrevivencia. Os dermatologistas utilizam diferentes procedi-

mentos para observar e diagnosticar uma lesao cutanea. A dermoscopia e um desses metodos e

permite a ampliacao de uma lesao ate 100x, possibilitando a observacao de estruturas que sao in-

detectaveis a olho nu. Uma das estruturas que se torna visıvel e a rede pigmentada, que e utilizada

em varios metodos medicos (regra ABCD e 7-point checklist) para classificar lesoes e diagnosticar

melanomas. A deteccao e extraccao automaticas desta estrutura em imagens dermoscopicas pode

ajudar os dermatologistas. Todavia, os sistemas automaticos que classificam lesoes cutaneas igno-

ram esta estrutura. Na literatura sao poucos os trabalhos que dizem respeito a deteccao automatica

de rede pigmentada e a maior parte deles foca-se apenas na deteccao de rede, sem proceder a

sua extraccao, que pode ser importante para os dermatologistas. Esta tese propoe um novo metodo

para deteccao e extraccao de rede pigmentada, utilizando duas das suas propriedades: cor e ge-

ometria. O algoritmo automatico utiliza filtros direccionais e uma analise de componentes conexas

para detecter a rede pigmentada. Um algoritmo que classifica lesoes relativamente a presenca de

rede pigmentada tambem foi desenvolvido. Este algoritmo apresenta uma SE = 78% e uma SP=77%

numa base de dados de 57 imagens, obtidas no servico de dermoscopia do Hospital Pedro Hispano.

Palavras Chave

Classificacao de lesoes cutaneas, Dermoscopia, Melanoma, Deteccao de rede pigmentada, Fil-

tros direccionais, Analise de componentes conexas, Extraccao de features.

v

Contents

1 Introduction 1

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Skin Lesions Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Pigment Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4 Thesis Objectives and Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Dermoscopy Analysis 11

2.1 Dermoscopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2 Dermoscopic Criteria and Pattern Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3 Medical procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3.1 ABCD Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3.2 7 Point-checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.4 Automatic Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.4.1 Automatic Systems for Melanoma detection . . . . . . . . . . . . . . . . . . . . . 15

2.4.2 Automatic Systems for Dermoscopic Features Extraction . . . . . . . . . . . . . 16

3 Directional Filters 19

3.1 Directional Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2 Directional Filter Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4 Pre-Processing 25

4.1 Gray Scale Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.2 Reflection Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.3 Hair Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.4 Inpainting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.4.1 Inpainting Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5 Network Detection 35

5.1 Previous works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.2 Network Enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.2.1 Directional Filters Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.2.2 Thresholding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

vii

5.3 Network Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6 Region Validation and Lesion Classification 43

6.1 Region and Lesion Ground Truths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

6.2 Background Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6.3 Region Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6.4 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6.4.1 Image Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6.4.2 Topological Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

6.5 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6.6 Lesion Classification algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

7 Results and Discussion 53

7.1 Database of dermoscopy images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

7.2 Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

7.2.1 Assessment of Region Detection Algorithm . . . . . . . . . . . . . . . . . . . . . 55

7.2.1.A Pixel based statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

7.2.1.B Region-based statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

7.2.2 Assessment of the validation step . . . . . . . . . . . . . . . . . . . . . . . . . . 57

7.2.3 Assessment of Lesion Classification . . . . . . . . . . . . . . . . . . . . . . . . . 57

7.3 Comparison between 4 and 8 pixels-neighborhood . . . . . . . . . . . . . . . . . . . . . 57

7.4 AdaBoost parameters selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

7.4.1 α weight parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

7.4.2 Features Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

7.4.3 Number of iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

7.5 Comparison between gray-level conversion methods . . . . . . . . . . . . . . . . . . . . 63

7.6 Region Detection System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

7.6.1 Detection examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

7.6.2 Statistical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

7.6.2.A Pixel-based statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

7.6.2.B Region-based statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

7.7 Lesion Classification Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

8 Conclusions and Future Work 71

8.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

8.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Bibliography 75

Appendix A Inpaiting Algorithm A-1

viii

List of Figures

1.1 Melanocytic Nevi [2] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Skin Lesions’ Classification Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Melanoma that mimics Clark Nevus [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4 Melanoma that mimics Blue Nevus [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.5 Melanoma that mimics Seborrheic Keratosis [3] . . . . . . . . . . . . . . . . . . . . . . . 5

1.6 Melanoma that mimics BCC [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.7 Examples of Pigment Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.8 Pigment Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.9 Thesis objectives (examples): Pigment network detection and lesion classification. . . . 8

1.10 Artifacts usually found in dermoscopy images . . . . . . . . . . . . . . . . . . . . . . . . 9

1.11 Block Diagram of the Detection System . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 Directional Filters Bank. The inpute I and all the outputs Ii are images . . . . . . . . . . 22

3.2 Impulse responses of 19 directional filters used in the pigment network’s enhancement

block. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.3 Final output of the directional filters bank. . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.1 Block Diagram of the Pre-Processing step . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.2 Intensity studies in Reflection regions. Figures 4.2(a) and 4.2(b) are the original images

with a green line showing the direction along which the intensity profile study and the

deviation study were performed. Figures 4.2(c) and 4.2(d) are the intensity profiles

corresponding to the figures that are above them. Figures 4.2(e) and 4.2(f) are the

same as the previous two, but represent the deviation from average intensity. . . . . . . 28

4.3 Reflection detection in dermoscopy images . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.4 Presence of Hair artifacts in dermoscopy images . . . . . . . . . . . . . . . . . . . . . . 29

4.5 Block Diagram of the Hair detection system . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.6 Hair detection in dermoscopy images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.7 Gray Scale images after artifacts removal . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.8 Propagation direction as the normal to the signed distance to the boundary of the region

to be inpainted [9] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.9 Gray Scale images after inpaiting algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 33

ix

5.1 Network Enhancement and Detection blocks . . . . . . . . . . . . . . . . . . . . . . . . 36

5.2 Output for the directional filters bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.3 Output of the thresholding step. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.4 Output of the network detection block. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.5 Detection of Pigment Network: original images(left) and pigmented network (right) . . . 41

5.6 Detection errors: original image(left) and pigmented network (right) . . . . . . . . . . . . 42

6.1 Ground truth segmentation of the pigmented network regions. . . . . . . . . . . . . . . . 44

6.2 Steps of the background extraction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6.3 Region labeling. Overlapping: A(blue) = 74% (label 1), B(green) = 0% (label 0), C(pink)

= 0% (label 0), D(yellow) = 0% (label 0), E(orange) = 76% (label 1), F(red) = 24% (label

0) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6.4 Lesion Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

7.1 Output of the network detection block for different connectivities. . . . . . . . . . . . . . 58

7.2 ROC curve obtained for the three different features’ sets by varying the α ∈ 1, 3, 5, 10, 15

weight parameter and T = 2000. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

7.3 Histogram of the average rate of selection of each topological feature for α = 15 and

T = 2000. Each one of the ”Area Holes i” feature stands for the i − th bin of the

histogram of the area of holes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

7.4 Histogram of the average rate of selection of all the features for α = 15 and T = 500.

Each one of the ”H i” , ”HoG i” feature and ”H i” feature stands for the i − th bin of,

respectively, the histogram of intensities, HoG and histogram of the area of holes. . . . . 61

7.5 Correlation Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

7.6 Output of the automatic detection system for lesions with Pigment Network: original

image (left), output of the network detection block (center) and final output after the

region validation block (right) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

7.7 Output of the automatic detection system for lesions without Pigment Network: original

image (left), output of the network detection block (center) and final output after the

region validation block (right) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

x

List of Tables

2.1 ABCD Rule of Dermoscopy [50] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2 7 Point-Checklist: Criterion and Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3 Summary of recent CAD systems. Mel. stands for Melanoma. . . . . . . . . . . . . . . . 16

7.1 Percentage of melanocytic and non-melanocytic lesions in the dataset. . . . . . . . . . . 54

7.2 Percentage of . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

7.3 Statistical Results for the different α values and T = 2000. FP-rate stands for the

percentage of not pigment network regions that are wrongly classified ; TP-rate is the

percentage of correctly classified pigment network regions. . . . . . . . . . . . . . . . . 59

7.4 Statistical results for different numbers of iterations for α = 15 and topological features.

FP-rate stands for the percentage of non-network regions that are wrongly classified

as so while TP-rate is the percentage of correctly classified pigment network regions. . 62

7.5 Statistical results for AdaBoost performance using the highest entropy channel (see

(4.2)) as gray level conversion method for α = 15, T = 100 and topological features. . . 63

7.6 Statistical results for AdaBoost performance using the linear combination of RGB val-

ues (see (4.1)) a gray level conversion method for α = 15, T = 100 and topological

features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

7.7 Statistical results at a pixel level for the Network detection system before the validation

step, for α = 15, T = 100 and topological features. . . . . . . . . . . . . . . . . . . . . . 67

7.8 Statistical results at a pixel level for the Network detection system after the validation

step, for α = 15, T = 100 and topological features. . . . . . . . . . . . . . . . . . . . . . 67

7.9 Statistical results at region level for the network detection system before and after the

validation step. The validation is performed with α = 15, T = 100 and topological

features. CD - correct detection; FA - false alarm; DF - detection failure. . . . . . . . . . 68

7.10 Statistical results for lesion classification after validation, with α = 15, T = 100 and

topological features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

7.11 Statistical results for lesion classification before validation, with α = 15, T = 100 and

topological features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

xi

Acronyms

ANN Artificial Neural Network

BCC Basal Cell Carcinoma

CAD Computer Aided Diagnosis

CD Correct Detection

DF Detection Failure

FA False Alarme

FN False Negatives

FP False Positives

HoG Histogram of Gradient

kNN k-Nearest Neighbor

LoG Laplacian of Gaussian

PDE Partial Differential Equation

ROC Receiver Operator Characteristics

SE Sensitivity

SP Specificity

SVM Support Vector Machines

TDS Total Dermoscopy Score

TN True Negatives

TP True Positives

xiii

1Introduction

Contents1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Skin Lesions Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Pigment Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4 Thesis Objectives and Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1

1.1 Motivation

Melanoma is one of the deadliest forms of cancer, being responsible for most of the deaths related

to skin cancer. However, when detected in an early stage, it can be cured by recurring to a simple

excision of the lesion. Therefore, it is important do develop methodologies that allow a premature and

correct detection of melanomas.

Dermoscopy is a non-invasive medical technique for in vivo observation of pigmented skin lesions,

allowing their magnification and a better visualization of their surface and subsurface structures. This

medical tool has improved the accuracy of melanoma detection as well as an increasing of early de-

tections. Nevertheless it requires a trained physician to make a competent use of this tool, otherwise

the medical diagnosis can be compromised. Based on this technique there are, nowadays, a set

of medical procedures that simplify the classification of skin lesions and, as a result, the detection of

melanomas. These procedures use the structures that were enhanced by dermoscopy to help derma-

tologists in their diagnosis. The most common are ABCD rule and 7 point checklist, just to cite some

of them. These rules use a set of dermoscopic criteria as the color of the lesion and the presence

or absence of certain kind of structures to classify the lesions (see Section 2.2). One of the major

structures analyzed by these rules is the pigment network. [3]

Pigment network has an essential role in skin lesion classification since it allows the distinction

between different classes of skin lesions as well as an identification of the lesions as melanoma or

not, since the presence of an atypical network is usually a sign of a malignant lesion. With that in

mind, the detection of this structure poses as an important task due to its great significance in medical

diagnosis. [3,7]

Recently, several works have been developed in order to create automatic algorithms for melanoma

detection. Some of them extract a large set of features and use them to train a classifier so that it

will be able to distinguish between melanoma and other skin lesions. Other works choose a differ-

ent approach and try to detect structures used by dermatologists to classify the lesions. The work

developed and described in this master thesis belongs to this second group.

Despite the importance of pigment network, only a few works try to detect this feature [5, 10, 13,

20,26,42], most of which do not focus on extracting its shape which is usually used by dermatologists

when performing a diagnosis. Therefore, new approaches for Pigment Network detection and extrac-

tion are required and this work presents a new, different and functional algorithm that can be used to

detect Pigment Network and extract its characteristic structure.

1.2 Skin Lesions Classification

Pigmented skin lesions classification is done in several steps. First, a distinction between melanocytic

and non-melanocytic lesions must be performed. This distinction is made based on a set of dermo-

scopic features, some of which are described in Section 2.2.

There are four main types of non-melanocytic lesions, each one of them with specific charecter-

istics. Three of the four are benign neoplasms that can appear in most parts of the human body.

2

Their names are: Seborrheic Keratosis, Vascular Lesions and Dermatofibroma. The malignant type

of non-melanocytic lesions is called Basal Cell Carcinoma (BCC). This is the most common primary

malignant neoplasm in humans. However, it grows very slowly, so usually a BCC is considered in-

nocuous. Nevertheless it should be treated when detected, because it can still cause an extensive

tissue destruction. [3]

Melanocytic lesions originate from melanocytes, which are a type of cells responsible for the pro-

duction of a protein pigment called melanin. This kind of lesions can be divided in two separate groups

(benign lesion, which include regular and atypical lesions, and malignant lesions or melanomas) con-

sidering their characteristics and if they are benign or malignant. Benign melanocytic lesions are

usually called moles or nevi and can be classified as acquired or congenital, when they are present

at the age of birth or appearing by the age of two. Another classification can be performed based on

the origin of these nevi within the skin. Melanocytic nevi are formed by organized clusters of nevus

cells (which have as precursors the melanocytes) arranged in various levels in the skin. The nest of

these clusters can be localized in the dermis (deepest skin layer), in the dermoepidermal junction or

even in both of them. Based on this a mole can be classified as intradermal, junctional or compound

since each one presents a set of specific features [1]. In Figure 1.1 examples of each type of nevi

can be seen. It is also possible to distinguish between nevi that have the same origin within skin,

considering their prevailing color (for example Blue Nevus, that can be seen in Figure 1.4(a) presents

a blue coloration) and other dermoscopic features such as Pigment Network, dots and streaks (see

Section 2.2) allowing a more specific classification of a skin lesion.

(a) Intradermal (b) Junctional (c) Compound

Figure 1.1: Melanocytic Nevi [2]

Some melanocytic nevi are irregular. Usually they are called Atypical Nevi and are histologically

distinctive from other melanocytic lesions. They can occur as solitary or multiple lesions and are

considered as an intermediate state between benign and malignant skin neoplasms. Atypical nevi

can be distinguished from regular melanocytic ones since they are usually larger, have an irregular

and indistinct border, exhibit a mixture of colors and an irregular surface, that usually shows a central

or eccentric papule, surrounded by a proeminent macular component. Atypical nevi can also be called

Clark Nevi. [1]

Melanoma is a common malignant form of melanocytic lesions that has the potential to metasta-

size. The term ”Melanoma in situ” refers to the early stage of the neoplasm, when it is situated within

the epidermis. Since it is not present in the dermis it has no contact with the vascular plexus and can

3

not metastasize. Therefore a simple excision of the neoplasm should be enough to cure the disease.

Nowadays, melanoma detection can be achieved by a combined use of the dermoscopy technique

described in Section 2.1 and of the medical procedures described in Section 2.3. Usually melanomas

in an early stage show an irregular shape and several colors as well as outlined macules or slightly

elevated plaques. Invasive melanomas can be papular or nodular, ulcerated and with a coloration that

ranges from brown to black, exhibiting also regions of red, white or blue. [3]

The several steps for lesion classification can be resumed in a classification tree as the one in Fig-

ure 1.2. This tree allows a better understanding of the lesion classification described in the previous

paragraphs.

Figure 1.2: Skin Lesions’ Classification Tree

One of the major pitfalls in lesion classification is the distinction between melanoma and other

types of skin lesions (melanocytic or not). Sometimes, a melanoma is underdiagnosed as being a

different kind of lesion, which results in a dangerous false negative, because melanoma mimics its

appearance. The existence of these kind of errors requires that in case of uncertainty in the diagnosis,

an histological exam must be performed. [3]

In the following figures some examples of the most common misleading melanocytic lesions are

presented. In Figures 1.3 and 1.4 it is shown how melanoma can mimic melanocytic lesions as

Atypical Nevi (Clark Nevi) and Blue Nevus.

(a) Clark Nevus (b) Melanoma

Figure 1.3: Melanoma that mimics Clark Nevus [3]

4

(a) Blue Nevus (b) Melanoma

Figure 1.4: Melanoma that mimics Blue Nevus [3]

Melanomas can also be mistakenly classified as non-melanocytic lesions. Figures 1.5 and 1.6 are

examples of that.

(a) Seborrheic Keratosis (b) Melanoma

Figure 1.5: Melanoma that mimics Seborrheic Keratosis [3]

(a) BCC (b) Melanoma

Figure 1.6: Melanoma that mimics BCC [3]

These images are quite elucidative of how difficult it is for dermatologists to correctly identify

melanomas sometimes. This is the main reason for the necessity of creating new automatic systems

for melanoma detection that can help dermatologists in lesion classification.

5

1.3 Pigment Network

Pigment network is one of the most important structures in dermoscopy [7]. It appears as a grid of

thin brown lines over a diffuse light brown background, having a shape very similar to a honeycomb [1].

In Figure 1.7 several examples of Pigment Network can be seen.

(a) (b)

(c) (d)

Figure 1.7: Examples of Pigment Network

Pigment network can cover most parts of a lesion (as illustrated in the two top images of Figure

1.7) or only small areas (see the two bottom images of Figure 1.7). However, in both cases, it forms

a pattern called reticular pattern, which is the most common feature in melanocytic lesions. It is

considered the dermoscopic hallmark of benign acquired melanocytic nevi and of thin melanomas,

although a pigment network can also be found in other kind of skin lesions as seborrheic keratosis

and dermatofibromas [3].

In order to have a better understanding of how the pigment network can help in the detection of

melanomas and other kinds of melanocytic lesions one must make use of histophatology. Skin has

two different layers: epidermis and dermis. Epidermis has a basal layer that possesses a type of cells

called melanocytes and the melanin produced by them is mostly distributed to keratinocytes localized

in the basal layer at the dermoepidermal junction.

If one does a vertical cut on the epidermis, it is possible to see that its structure is wave-like with

high and low points, this kind of structure is usually called rete ridge pattern. Usually there is a higher

concentration of melanin in the lower rete than in the upper rete near the skin surface.

If a horizontal slice of epidermis is taken, it is possible to see that the rete make a continuous

6

structure, having round walls with a hole in the middle. The walls of the rete are thick, project down

into the dermis and contain melanin. When viewed from above these round walls look like dark circular

lines and since they are connected with adjacent rete they are responsible for the dark grid pattern

characteristic of pigment network. The holes inside the walls contain a smaller amount of melanin and

because of this, when viewed from above, they look like a light circle [1]. Based on this explanation it

is possible to conclude that the grids in a pigment network are actually regions with a high amount of

melanin and that the holes are regions of a low concentration of this pigment.

The presence of pigment network represents an important feature for dermoscopy image analysis.

The assessment of its alterations is also very important, since it allows the distinction between benign

and malignant melanocytic lesions. Pigment network can be typical or atypical. As the classification

suggests, a network is considered typical when it is light to dark-brown pigmented, regularly meshed,

narrowly spaced and its distribution is more or less regular (see Figure 1.8(a)). An atypical pigment

network is characterized by a black, brown or gray, irregular network (with thickened lines), distributed

more or less irregularly throughout the lesion and usually ending abruptly at the periphery (see Figure

1.8(b)). [3]

(a) Typical Network (b) Atypical Network

Figure 1.8: Pigment Network

A melanoma is caused by malignant melanocytes that vary in size and degree of pigmentation.

They can move through the epidermis in all directions, distorting its normal structure and causing vari-

ations in pigmentation patterns. Malignant cells are also capable of multiplying themselves, causing a

thickening of the rete, which is responsible for the thickening of the grid and being also responsible for

the darkening of the network [1].This analysis suggests that an atypical network is an unmistakable

sign of melanoma.

Achieving an automatic detection of pigment network and posterior assessment of its structure

can be an important tool in the future for the detection of melanomas.

7

1.4 Thesis Objectives and Structure

This master thesis has two main objectives. The first one is to develop an automatic algorithm

for the detection of pigment network regions in dermoscopy images and the second one is to classify

skin lesions regarding the presence or absence of pigment network. Figure 1.9 illustrates these two

problems applied to two different kinds of melanocytic skin lesions. The left image is a melanocytic

lesion labeled as with pigment network and the pigment network regions that must be detected are

highlighted. The right image is also a melanocytic lesion, but labeled as without pigment network. For

this lesion the algorithm should not detect pigment network.

(a) Lesion with pigment network - Networkregions are highlighted

(b) Lesion without pigment network

Figure 1.9: Thesis objectives (examples): Pigment network detection and lesion classification.

As was explained in Section 1.3, pigment network is part of a reticular pattern that can cover the

entire lesion or only some areas of it. Figure 1.7 shows that this pattern may present itself with different

visual characteristics such as color and width. However, the main properties of pigment network (dark

grid lines over a lighter diffuse background and the organization of these connected lines that lead to

the existence of holes) are always present and can be used for its detection. Chapters 3 and 5 explain

how these two properties are used is this work for network detection. Chapter 3 gives an introduction

to one of the main procedures used in network detection while Chapter 5 describes step by step how

the detection is performed. The second main objective is to perform a binary labeling of lesions, i.e.,

to be able to classify lesions as with or without pigment network. The work developed regarding this

second objective is be discussed in Chapter 6.

The work was developed using dermoscopy images from a database of Hospital Pedro Hispano.

As it is common with dermoscopy images, these images present a set of artifacts (hair and reflections)

that hamper the detection of pigment network. These artifacts can be seen in Figure 1.10 and needed

to be removed before a detection could be performed. The removal of these artifacts is performed in

a pre-processing step described in Chapter 4.

The results obtained with the network detection block are improved in a post-processing step.This

block checks if the regions detected have a pigmented network structure. This is done using a binary

classifier (AdaBoost [54]) trained from the data. Chapter 6 describes in detail the region validation

block.

8

(a) Hair and air bubbles (b) Reflection

Figure 1.10: Artifacts usually found in dermoscopy images

Figure 1.11 summarizes the detection system briefly described above. It consists of five succes-

sive blocks: a pre-processing block where the hair and reflections are removed from the dermoscopy

image, a network enhancement block, a network detection block, a block of post-processing, where

the regions are validated and a final block where a binary classification of the whole lesion is per-

formed. The chapters where each one of the blocks is discussed in detail are marked on the figure.

Figure 1.11: Block Diagram of the Detection System

Before the chapters which explain the work developed in this thesis, a review of the state of the art

is made in Chapter 2. In this chapter the dermoscopy technique as well as medical and computational

algorithms for melanoma diagnosis will be presented.

Chapter 7 presents an experimental evaluation of the proposed system using a database of 57

images. This chapter also addresses some of the decisions made in the design of the system, such as

using the best entropy channel for gray-scale conversion and AdaBoost parameters will be discussed.

The final Chapter 8 addresses the conclusions of the work performed and some proposals of work

to be done in the future.

9

1.5 Contributions

In this master thesis a new algorithm for automatic detection and classification of pigment network

in dermoscopy images (see Figure 1.11) is proposed. This algorithm includes the following original

contributions:

Pre-Processing: A method for pre-processing dermoscopy images is proposed in which the de-

tection of reflection and hair artifacts is accomplished by using, respectively, thresholding methods

and a bank of directional filters.

Network Enhancement: This work proposes a new procedure for enhancing the lines of pigment

network. The enhancement is performed by recurring to a bank of directional filters followed by a

threshold step.

Network Detection: The new methodology proposed for network detection makes use of the geo-

metrical properties of pigment network. The detection algorithm uses connected component analyses

to detect and extract the skeleton (the mesh) of pigment network.

Post-Processing/Region Validation: The proposed method for region validation can be seen as

a post-processing step. A binary classification of the pigment network regions detected is performed

by training a classifier (AdaBoost) with a set of features (which discriminate intensity, texture and

geometry) extracted from the detected regions.

Lesion Labeling: A method for a binary lesion classification based on a statistical approach is

also proposed.

Most of this work was published in the main conference on IEEE Biomedical Engineering Society:

C. Barata, J.S. Marques and J. Rozeira [8] - ”Detecting the pigment network in dermoscopy im-

ages: a directional approach” - in proceedings of the 33rd Annual International Conference of the

IEEE Engineering in Medicine and Biology Society (August 30th- September 3rd) at Boston (USA) .

10

2Dermoscopy Analysis

Contents2.1 Dermoscopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.2 Dermoscopic Criteria and Pattern Analysis . . . . . . . . . . . . . . . . . . . . . . 122.3 Medical procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.4 Automatic Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

11

2.1 Dermoscopy

Dermoscopy is a non-invasive diagnostic technique used to see a variety of patterns and struc-

tures in skin lesions that are not discernible to the naked eye and that can be very useful in the

characterization of skin lesions. It has several other names such as epiluminescence microscopy

(ELM), dermatoscopy or skin surface microscopy.

The technique consists in placing mineral oil, alcohol or even water on the surface of the lesion.

These fluids make epidermis more transparent to light and eliminate part of the surface reflection [1].

After placing the fluid, the lesion is inspected using a dermatoscope, a stereomicroscope, a camera

or a digital imaging system. Depending on the instrument used, the magnification of a given lesion

ranges from 6x to 40x and even up to 100x [3]. With the magnification, pigmented structures within the

epidermis, dermoepidermal junction and superficial dermis become visible. In Section 2.2 a reference

to the different structures detected is done.

Several studies have demonstrated that dermoscopy analysis improves the accuracy of melanoma

detection by dermatologists by 10-27%, when compared with a naked eye analysis [35]. However,

this technique’s performance is highly dependent on the expertise of the dermatologist as has been

shown by [11], and even trained users can be lured in some particular cases of melanomas that have

no characteristic dermoscopy findings [1].

2.2 Dermoscopic Criteria and Pattern Analysis

As was described in Section 2.1, dermoscopy magnifies a pigmented skin lesion, highlighting a set

of dermoscopic structures. The structures enhanced can be used as an input in several procedures

that are used by dermatologists to distinguish between different kinds of skins lesions (melanocytic

and non-melanocytic) as well as in the detection of melanomas. Some examples of these algorithms

are: ABCD rule [50], 7 poins-checklist [6] (both described in Section 2.3), Menzies method [36] and

pattern analysis [39].

The dermoscopic features detected can be divided in two separate groups: global and local fea-

tures. Global features allow a quick, albeit preliminary categorization of a given pigmented skin lesion.

They are considered as patterns that may or may not be present in a given lesion. For example, the

Reticular Pattern (Section 1.3) is considered a global feature. Other cases of global features are:

Globular Pattern (presence of numerous variously sized, round to oval structures with different col-

ors), Starburst Pattern (characterized by the existence of pigmented streaks in a radial arrangement

at an edge of a given pigmented lesion), and Cobblestone Pattern, which is very similar to globular but

is composed of closely aggregated, angulated globules. Local features are the ones that are usually

used when performing a diagnosis of a malignant lesion, since it is the analysis of these features

that will be considered in medical algorithms. Pigment network, dots and globules, pigmentation,

blue-whitish veil, streaks, regression and vascular structures are all considered local features. [3]

Pattern Analysis is the classic dermoscopic method for diagnosing skin lesions [1]. It evolves in

two steps. In the first one, the lesion is classified as being melanocytic or non-melanocytic. This

12

classification is performed based on the dermoscopic features described previously, namely based

on global features. A reticular pattern, a globular or cobblestone patterns and a starburst pattern are

usually identifiers of melanocytic lesions. The second step of the algorithm is to distinguish between

malignant and benign lesions, considering only the lesions that were classified as melanocytic in the

first step. To do so, an analysis of the local features and global features is performed. If a lesion

presents atypical features it is considerer as malignant. However, if the features are typical, the lesion

is benign. [39]

2.3 Medical procedures

The Pattern Analysis described previously in Section 2.2 proved itself a very useful tool for im-

proving the clinical diagnosis of melanoma, increasing the accuracy of correct diagnosis by 10% to

30% [3]. However, it relies on subjective basis, since the assessment of dermoscopic criteria and

its classification as typical or atypical may change between dermatologists. In order to try to solve

these problems, additional diagnostic methods have been presented: ABCD rule and 7 point checklist

are two notable examples. Both of these algorithms require a preceding step, in which the lesion is

classified as melanocytic or not. This step is similar to the first step of Pattern Analysis, since it makes

use of the assessment of dermoscopic global and local features to perform the classification. In the

following subsections both these procedures will be explained.

2.3.1 ABCD Rule

For performing ABCD rule four parameters (Asymmetry, Border, Color and Differential structure)

must be assessed. Each of these characteristics is analyzed separately and each one is scored

based on how atypical they are. For instance, in asymmetry the score can be 0, 1 or 2 based in how

asymmetric the lesion is (2 is for asymmetry along two orthogonal axis and 0 is given when the lesion

is fully symmetric). After giving a score to each one of the 4 criterion (usually they are called Ascore,

Bscore, Cscore and Dscore), the Total Dermoscopy Score (TDS) is computed as a liner combination of

these parameters (see (2.1). Table 2.1 and expression (2.1) summarize the ABCD rule.

Table 2.1: ABCD Rule of Dermoscopy [50]

Criterion Description Score WeightAsymmetry In 0,1 or 2 axes; assess not only contour, but

also colors and structures.0-2 1.3

Border Abrupt ending of pigment pattern at the pe-riphery, in 0-8 segments.

0-8 0.1

Color Presence of up to 6 colors 1-6 (white,red, light-brown, dark-brown, blue-gray andblack).

1-6 0.5

DifferentialStructures

Presence of network, structureless or homo-geneous areas,streaks, dots and globules.

1-5 0.5

TDS = 1.3Ascore + 0.1Bscore + 0.5Cscore + 0.5Dscore (2.1)

13

After computing the TDS score, the diagnosis is made based on its value. If TDS for a given lesion

is higher than 5.45, there is a high probability that it might be a melanoma. For TDS values below

4.75, the lesion is considered as being benign. The lesions that have TDS values between the two

referred previously, are considered suspicious and a close follow-up is advised.

2.3.2 7 Point-checklist

7 point-checklist is an alternative medical algorithm for detecting melanomas. It requires only the

identification of 7 atypical local features to perform a diagnosis. The 7 dermoscopic criteria assessed

are divided in two classes: major and minor criteria, based on the capacity of each criterion to increase

the probability of a melanoma diagnosis. Each feature present in a lesion receives a score. If it is a

major feature it receives a score of 2 and if it is a minor criteria a score of 1. In the end, a sum of the

individual scores is computed. A lesion is classified as being a melanoma if the final score is higher

than 3 [6]. Table 2.2 shows the 7 features that can be detected and their score.

Table 2.2: 7 Point-Checklist: Criterion and Scores

Local Features ScoreMajor Criteria

1.Atypical Pigment Network 22.Blue-Whitish Veil 23.Atypical Vascular Pattern 2

Minor Criteria4.Irregular Streaks 15.Irregular pigmentation 16.Irregular dots/globules 17.Regression structures 1

2.4 Automatic Systems

The diagnosis of a skin lesion by one of the previous medical methodologies is a subjective one,

since it depends on human vision and on the clinical experience of the dermatologist, lacking accuracy

and reproducibility. Moreover, its accuracy is highly dependent on the expertise of the dermatologist

[11].

Computerized dermoscopy image analyses systems do not have the limitation of subjectivity.

These systems consider that a computer can be used as a second independent diagnostic method,

which can be used by non-experienced operators to perform a preliminary evaluation of a clinical

case and that can also be used to help dermatologists. They can also be used to improve biopsy

decision-making and follow-up. [28]

Several systems for lesion classification, and ultimately for melanoma detection, have been pro-

posed by different groups [12,15,23,30,38,41,43,45]. They will be described briefly in the following

subsection. There are other systems which try to incorporate detailed medical knowledge and de-

tect the structures which characterize a melanocytic lesion (see Section 2.2). These systems will be

14

described in Subsection 2.4.2.

2.4.1 Automatic Systems for Melanoma detection

CAD systems can be found in literature as early as 1987 [14]. These methods usually start by

performing an automatic segmentation of the lesion, which by itself is already a challenge, because

of the great variety of lesion shapes, sizes and colors, along with different skin types and textures. A

good lesion segmentation is required, since the accuracy of the subsequent steps of the CAD system

are affected by it [44].

The following step is feature extraction. The features extracted are selected based on the ABCD

rule. Therefore, the CAD systems usually uses features that describe shape (asymmetry and border),

color and texture.

The final step is to perform a binary lesion classification (malignant or not). This is done by

recurring to learning algorithms (artificial neural networks (ANN), support vector machines (SVM),

boosting). Some systems have an intermediary step between feature extraction and classification, in

which they reduce the size of the feature space by eliminating redundant, irrelevant or noisy features.

In recent years very promising results have been obtained by some of these systems. Ganster

et al. [23] achieved a sensitivity (SE) (melanoma detection accuracy) of 73% and a specificity (SP)

(benign detection accuracy) of 89%. They used a large dataset of more than 5000 images of which

only 2% were melanomas and their results were obtained by training a k nearest neighbor (kNN)

classifier. Rubegni et al. [41] proposed a classification system using an ANN. Their results were quite

good, achieving a SE of 94.3% and a SP of 93.8% on a dataset of 550 images, 200 of which were

cases of melanoma. The work developed by Blum et al. [12] reported a SE of 83.3% and a SP of

86.9% on 753 cases of nevi and 84 cases of melanoma, with a logistic regression model. By using

SVM and a dataset of 564 images, containing 88 cases of melanoma, Celebi et al. [15] achieved a

SE of 93.3% and a SP of 92.3%. In 2008 Iyatomi et al. [30] improved their Internet-based melanoma

screening system that was developed in 2004 [38], increasing their former SE and SP values, so there

is currently one automatic system available on the net. Table 2.3 summarizes the information related

with the different CAD systems described in this paragraph.

Despite the promising results, the approaches resumed above have some issues. First of all they

do not try to incorporate the medical knowledge. They are based on a machine learning paradigm:

if features are rich enough and if there is a (very) large database of lesions classified by experts,

inference methods will be able to learn the classification rule [18]. This can affect the results obtained

by the different CAD systems. Another problem is related with comparison between different methods,

since each group uses a different dataset it is not possible to determine which system actually gives

the best results.

15

Table 2.3: Summary of recent CAD systems. Mel. stands for Melanoma.

Authors Classifier Images Mel. SE (%) SP (%)Ganster et al. [23] kNN 5363 107 73.0 89.0Rubegni et al. [41] ANN 550 200 94.3 93.8Blum et al. [12] Logistic Regression 837 84 83.3 86.9Oka et al. [38] ANN 247 59 87.0 93.1Celebi et al. [15] SVM 564 90 93.3 92.3Iyatomii et al. [30] ANN 1258 198 85.9 86.0

2.4.2 Automatic Systems for Dermoscopic Features Extraction

The dermoscopic or medical features detection and extraction is a relatively unexplored area. Nev-

ertheless there are already some remarkable works in which the detection of dermoscopic features

used by dermatologists such as asymmetry and border, color and differential structures (dots, pig-

ment network and blue-white veil) is accomplished [5, 10, 13, 16, 19, 20, 26, 34, 40, 42, 47–49, 55]. By

detecting these kind of features it is possible to incorporate detailed medical knowledge in the CAD

systems.

Blotches are irregular spots or discolorations in a skin lesion. Two interesting studies can be found

on literature that focus on the detection of this medical feature, one proposed by Pellacani et al. [40]

where they try to automatically extract dark areas in melanocytic lesions, and another one proposed

by Stoeker et al. [47], which is more focused on the detection of asymmetric blotches. Recently, the

detection of the presence of specific color regions such as blue-white veil [16,19] or regression struc-

tures (white areas, where loss of pigmentation has occurred) [19] have been developed. Interesting

works have also been proposed for the detection of asymmetry [48] and extraction of the degree of

border irregularity [34].

Differential structures have also received some attention from different groups. By using a mor-

phological closing operation Yoshino et al. [55] developed an automatic system for dots extraction.

Their automatic system had some problems since it could also detect the outline of pigment network.

Nevertheless their results are very promising. An algorithm for globules analysis had already been

proposed by Fleming et al. [20]. More recently, a work which aimed to detect granularity in der-

moscopy images of malignant melanomas was proposed. They detected this pattern by using color

and texture features to train a back-propagation neural network [49]. The detection of streaks has

also received some attention and an algorithm for the automatic detection of irregular streaks has

been proposed by Fabbrocini et al. [19]. This algorithm achieved a SE of 86% and a SP of 88%.

The main aim of this thesis is to develop an automatic algorithm for pigment network detection.

This problem was ignored for a long time in the literature. However, several works have been recently

published [5, 10, 13, 20, 26, 42]. Each one of these detection systems proposes a different approach

for performing the detection of pigment network in dermoscopy images. Their final result is usually a

binary classification of a lesion as with or without pigment network. This is very useful for performing

a classification of a lesion as melanocytic or not. However, for dermatologists the structure of pigment

network (the mesh of dark lines) is also very important, since they use it to classify the pigment

16

network as typical or atypical and, consequently, to diagnose a lesion as benign or malignant. Most of

the works found in literature do not extract the skeleton of pigment network. Two notable exceptions

can be found in the works of Fleming et al. [20] and Grana et al. [26]. This thesis tries to overcome

this issue and proposes a new method for detecting the pigment network in dermoscopy images

based on color and geometry, which is also capable of extracting the dark lines of pigment network.

The system proposed can still perform a lesion classification regarding the presence or absence of

pigment network. The detection algorithm proposed is then a new approach to the problem of pigment

network detection.

17

18

3Directional Filters

Contents3.1 Directional Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.2 Directional Filter Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

19

One of the main objectives of this thesis is to detect and extract pigment network. Another objective

is the removal of hair artifacts which can occlude important regions of the lesions to be studied. Both

these structures contain linear strokes, with highly directional shapes. Therefore, filters which are able

to detect features in a specific orientation can be used to detect and enhance them. The directional

filters proposed in this thesis belong to this group of filters so they have the correct properties and are

capable of enhancing pigment network and hairs.

In this chapter the bank of directional filters used to sharp both pigment networks and hairs will be

presented. An introduction to the basis of directional filtering will also be made, in order to provide a

better understanding of the main reasons to choose these kind of filters.

3.1 Directional Filters

Directional filters have been proposed in different areas . One of them is a biology related area

and the other one is the signal processing area, namely processing signals with noise.

The biology behind directional filters is related with the human perception of vision. Visual percep-

tion can be defined as the psychological manifestation of visual information. This information comes

from visible light that is detected by the photoreceptors of retina and the processing of that informa-

tion occurs in the visual cortex. The transmission of information between the eyes and the brain is

performed first by the optic nerve and then by the optic tract, which ends at the thalamus. It is the

thalamus that is responsible for relaying the visual image to the visual cortex for processing. The path

described previously forms the sensory system of the human visual system. Each one of the compo-

nents of the sensory system possesses a receptive field, which can be defined as the region of space

to which a receptor organ and receptor cells respond. For instance, the receptive field of an eye is the

part of the world that it can see, i.e., the light that each photoreceptor receives. The receptive fields

in visual cortex of mammals have been studied by Hubel and Wiesel [29]. They found and classified

three distinct receptive fields of cells in the visual cortex: simple, complex and hypercomplex cells.

Their initial studies also concluded that the majority of neurons in the primary visual cortex respond

to a line or edge of a certain orientation in a given position of the visual field, they are directional. Be-

tween the three different kinds of neurons, two of them were defined as orientation-selective neurons,

one that was sensitive to the contrast polarity of lines and edges, the simple cells, and other that was

not, the complex cells. [29]

Several computational models have been proposed to simulate the functioning of these neurons.

One of the more interesting works was developed by Jones and Palmer [31]. They proposed that the

functioning of simple cells could be modeled by a family of 2D Gabor Filters. Their work proved that

simple cells in mammal visual cortex perform a linear and directional filtering. 2D Gabor filters have

been proposed by Daugman [17] as a 2D generalization of the family of linear filters described by

Gabor [22], and each filter has the following impulse response

gλ,φ,γ,θi(x, y) = C exp x′2 + γ2y′2

2σ2 cos (2π

x′

λ+ φ) (3.1)

where C is a normalization constant, λ is the wavelength and 1λ the spatial frequency of the cosine

20

factor and γ is the spacial aspect ratio, that determines the ellipticity of the mask. The parameter

φ ∈ [−π, π] is a phase offset that determines the symmetry of gλ,φ,γ,θi(x, y) with respect to the origin

[17]. The values of (x′, y′) are related with (x, y) as follows

x′ = x cos θi + y sin θi (3.2a)

y′ = y cos θi − x sin θi (3.2b)

where θi defines the direction of the filter.

These filters have been used with good results in texture discriminator [21,52] and edge detection

algorithms [27]. Usually a bank of Gabor filters with different orientations θi is used and for each

direction the original image I is convoluted twice, with a pair of filters with a phase difference of π2

rλ,0,γ(x, y) = gλ,0,γ,θi(x, y) ∗ I (3.3a)

rλ,−π2 ,γ(x, y) = gλ,−π2 ,γ,θi(x, y) ∗ I (3.3b)

The final response for each direction is a combination of the information provided by the pair of filters

Eλ,γ,θi =√r2λ,0,γ(x, y) + r2λ,−π2 ,γ

(x, y) (3.4)

The use of Gabor filters as models for simple cells which perform a directional filtering suggested that

this family of filters could be used to enhance highly directional structures such as pigment network

lines and hairs. As will be shown in the next section the filters used have an impulse response

which has the same gaussian base as Gabor filters and the bank is also composed of filters with

different orientations. However, the cosine term is not used and the impulse response is a difference

of gaussians.

Detection theory [53] can also be used to explain the use of directional filters. Based on the

methods used to detect signals in noise it is possible to determine the ideal detector.

The general detection problem consists in checking if a pattern IN (x, y), that in this case can be

pigment network or hair artifacts, is present in a given signal I(x, y) (the dermoscopy image). Two

hypotheses are formulated regarding this problem

H0 : I(x, y) = w(x, y) x ∈ X, y ∈ Y (3.5a)

H1 : I(x, y) = IN (x, y) + w(x, y) x ∈ X, y ∈ Y (3.5b)

where w(x, y) is additive noise.

The optimal solution to this problem consists of filtering the image I(x, y)

J(x, y) = I(x, y) ∗ h(x, y) (3.6)

with a linear filter for which the impulse response is

h(x, y) = IN (−x,−y) (3.7)

where h(x, y) is designated by matched filter and has the shape of the signal that one wants to detect

(apart from a spatial inversion) [51]. This theory suggested that in order to detect the skeleton of

21

pigment network and hairs, directional filters should be used. Therefore, it was decided that the best

approach should be to design a filter by recurring to gaussians equations, as will be shown in the next

section.

3.2 Directional Filter Bank

In this section it will be shown how the properties of an ideal detector discussed previously were

used in order to develop a bank of directional filters capable of detecting both pigment network and

hairs. Figure 3.1 summarizes the functioning of the directional filters bank.

Figure 3.1: Directional Filters Bank. The inpute I and all the outputs Ii are images

The analysis bank is composed of several directional filters each of them tuned to a specific

orientation θi ∈ [0, π], i = 0...N . The filters were designed this way because the linear structures that

compose the artifacts and pigment network’s dark lines not only have an unknown direction but may

appear in a dermoscopy image with more than one orientation as well.

Each one of the filters from the analysis block has the impulse response

hθi(x, y) = G1(x, y)−G2(x, y) (3.8)

where Gk is a Gaussian filter:

Gk(x, y) = Ck exp − x′2

2σ2xk

− y′2

2σ2yk

, k = 1, 2 (3.9)

In (3.9) Ck is a normalization constant and the values of (x′, y′) are related with (x, y) by a rotation

of amplitude θi

x′ = x cos θi + y sin θi (3.10a)

y′ = y cos θi − x sin θi (3.10b)

22

The values for the parameters σxk and σyk are chosen in such a way that the second filter is highly

directional and the first one is less directional or even isotropic. A difference of gaussians was chosen

since this design allows a better enhancement of directional structures while removing the effect of

the background. The image I is decomposed in N + 1 filtered images Ii by using all the directional

filters

Ii = hθi ∗ I (3.11)

Figure 3.2 shows the filter bank designed for pigment network enhancement. The orientation of

the filters ranges between 0 and π and in this image the differences between each impulse response

are easily detected. It is possible to see that, as expected, all the impulse responses have a linear

shape. This makes them good matched filters for the detection of pigment network and hairs.

θ = 0º θ = 10º θ = 20º θ = 30º θ = 40º θ = 50º

θ = 60º θ = 70º θ = 80º θ = 90º θ = 100º θ = 110º

θ = 120º θ = 130º θ = 140º θ = 150º θ = 160º θ = 170º

θ = 180º

Figure 3.2: Impulse responses of 19 directional filters used in the pigment network’s enhancement block.

The output of the filters bank is composed of N + 1 filtered images Ii. To synthesize the final

image J a simple maximization is performed at each pixel

J(x, y) = maxiIi(x, y) (3.12)

An example of the final output can be seen in Figure 3.3.

23

Figure 3.3: Final output of the directional filters bank.

When compared with the original image from Figure 1.7(a) it is possible to see that this bank of

directional filters performs a good pigment network and hair enhancement.

24

4Pre-Processing

Contents4.1 Gray Scale Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.2 Reflection Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.3 Hair Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.4 Inpainting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

25

The dermoscopy images used in computer-aided diagnosis often display artifacts (e.g., hair, re-

flections) produced during the acquisition process. These artifacts can interfere with many analytic

procedures required for accurate diagnosis such as border detection and dermoscopic criterion ex-

traction (see Figure 1.10). Therefore, a pre-processing step in which these artifacts are detected and

removed is required in order to ensure that the results obtained by the network detection algorithm

will not be compromised by them.

The scheme of Figure 4.1 presents in detail the operation of the pre-processing block of the de-

tection system.

Figure 4.1: Block Diagram of the Pre-Processing step

In the following section each block of the previous figure will be analyzed and explained in detail.

4.1 Gray Scale Conversion

The dermoscopy images used in this work are RGB images stored in bitmap and jpeg formats. In

order to perform the network detection these images are converted into 8-bit gray scale images.

There are two ways for performing this conversion. The first is done by applying the linear combi-

nation of RGB values

Y = 0.299R+ 0.578G+ 0.114B (4.1)

where Y is the 8-bit gray scale image and R, G and B are the red, green and blue color components,

respectively. [24]

Another way is by selecting the color channel with the highest value of entropy, since usually

this channel is the one which allows the best discrimination in most dermoscopy images [44]. The

selection of the color channel is an automatic process. First the entropy of each one of the color

components i is computed

S(i) = −L−1∑k=0

hi(k) log [hi(k)] (4.2)

where hi(k) is the histogram of the color component i. This histogram is computed using L = 256

bins, since each color channel ranges from 0, ..., 255. After computing the entropy values, the color

plane with the highest one is selected:

i∗ = argmaxi

Si (4.3)

Section 7.5 shows the results for region classification after the post-processing step described in

Chapter 6 for the two methods for gray-scale conversion.

26

4.2 Reflection Detection

Reflections appear in dermoscopy images as a result of the presence of air bubbles in the immer-

sion oil at the interface between the skin and the face plate of the dermatoscope or other acquisition

system. This kind of artifacts may appear around a lesion as in Figure 1.10(b) but they can also be

seen on top of the lesion, as is the case of image 1.10(a). The presence of reflections may influence

a correct detection of the border of the lesion as well as interfere with the main objective of this work,

since sometimes air bubbles can be on top of pigment network and so it will compromise its detection.

An algorithm for air bubbles removal has been proposed by Fleming et al.. Their algorithm was

based on two distinctive characteristics of air bubbles: the presence of speckles in their interiors and

the presence of a strong, bright edge in their boundaries. For performing the air bubbles detection they

used top hat filters for speckles segmentation and a radial search (having as starting point the centroid

of the speckle) combined with an outlier rejection procedure in order to improve the segmentation. In

the end the contours detected were filled and a air bubble mask was obtained [20].

In this thesis a simpler approach is proposed. After a brief study, it was concluded that pixels that

belong to regions classified as reflection have higher values of intensity than other pixels in the same

image, and that this value is usually above a certain threshold. Figures 4.2(c) and 4.2(d) show the

intensity profile along the green line draw in the two above pictures. It is possible to see that in the

positions that correspond to the edges of the reflection regions the pixels’ intensity values are very

high and that inside those regions the values are high as well. Another study showed that the pixels

that belong to reflection regions also have intensity values that diverge substantially from the average

intensity Iavg computed in the neighborhood of the pixel, as can be seen in Figures 4.2(e) and 4.2(f),

where it is shown the deviation from the average intensity along the two green lines. Both the intensity

profiles were determined for gray-scale images.

These examples suggest that reflection can be detected by comparing if the pixel intensity is high

and well above the average intensity in its neighborhood, i.e.,

I(x, y) > TR1 ∧ I(x, y)− Iavg(x, y) > TR2 (4.4)

where I is the gray scale image, Iavg is the average intensity in a local neighborhood of the pixel and

Tr1, Tr2 are both threshold values.

Experimentally it was determined that the values of TR1, TR2 that gave the best results were:

• TR1 = 180

• TR2 = 25

these values allow a good relationship between the correctly and wrongly detected regions.

27

(a) (b)

0 100 200 300 400 500 600 700 80050

100

150

200

250

Inte

nsity

(c)

0 100 200 300 400 500 600 700 80040

60

80

100

120

140

160

180

200

220

Inte

nsity

(d)

0 100 200 300 400 500 600 700 8000

10

20

30

40

50

60

70

80

90

Dev

iatio

n fr

om a

vera

ge in

tens

ity

(e)

0 100 200 300 400 500 600 700 8000

10

20

30

40

50

60

70

80

90

Dev

iatio

n fr

om a

vera

ge in

tens

ity

(f)

Figure 4.2: Intensity studies in Reflection regions. Figures 4.2(a) and 4.2(b) are the original images with a greenline showing the direction along which the intensity profile study and the deviation study were performed. Figures4.2(c) and 4.2(d) are the intensity profiles corresponding to the figures that are above them. Figures 4.2(e) and4.2(f) are the same as the previous two, but represent the deviation from average intensity.

Figure 4.3 exemplifies the application of this algorithm to images 4.2(a) and 4.2(b).

28

(a) (b) (c)

Figure 4.3: Reflection detection in dermoscopy images

4.3 Hair Detection

Hair artifacts are long, dark and linear structures. Their presence in dermoscopy images usually

occludes some information of the lesion such as its border and texture (dermoscopic features), as

can be seen in Figures 1.10 and 4.4. Therefore, the removal of this kind of artifacts is an important

step before a lesion segmentation or a feature extraction process can take place.

(a) (b)

Figure 4.4: Presence of Hair artifacts in dermoscopy images

A few methods for hair detection and removal have already been developed and are resumed in a

paper written by Abbas et al. [4]. They describe that most of the hair detection methods developed in

the past were based on the fact that hair pixels are much darker than skin or tumor area. However,

this principle is not always true, as can be seen in Figure 4.2(a) where the hair artifacts have a color

quite similar to the one of the lesion. The proposed methods used detection techniques such as

morphological closing operation, which is limited because the masks applied have only a discrete

number of orientations, thresholding and 2-D Derivative of Gaussian. This last technique was used

by Abbas [4] to propose a new hair detection method and brings some differences to the previous

methods since it also considers the thickness of the hair, their length and magnitude and increases

the number of orientations.

Hair artifacts are highly directional structures with a linear shape. The algorithm proposed in this

thesis for hair detection is different from the ones found in literature. First to enhance this structure

a bank of directional filters is used. The filter impulse responses are defined by (3.8) and (3.9) (see

29

Section 3.2). The algorithm uses not only the color information but also the characteristic shape of

hair (thin and linear structures). These informations can be used to define the best parameters for

(3.8) and (3.9), in order do provide good matched filters. Also, by using a difference of gaussians

as impulse response, hairs that have colors similar to lesions can be detected more easily, without

compromising the lesion.

The hair detection block runs in three different steps, as can be seen in Figure 4.5.

Figure 4.5: Block Diagram of the Hair detection system

First, the original image is filtered with a bank of directional filters. Hairs are long and thin struc-

tures, therefore it was decided that the impulse responses should be highly directional, as can be

seen by the parameters used for (3.8) and (3.9)

• σx1= 20 and σy1 = 6

• σx2= 20 and σy2 = 0.5

The filters’ mask size is 41x41. The step between θi and θi+1 should be small, since hair orientation

does not change sharply. Therefore, it was determined experimentally that a good value for N , the

number of rotations, should be N = 64. These filters enhance linear structures (hair artifacts) present

in the image.

The next step is to enforce a thresholding condition in order to perform a binary classification of

each one of the pixels from the filters response J

J(x, y) > TH (4.5)

Each one of the pixels (x, y) that is above the threshold is classified as hair. The threshold value

is TH = 0.06. This parameter and the ones from directional filters were determined empirically, and

are the ones which allow a better global hair detection, without detecting pigment network as well.

The morphological operation applied to the output of the thresholding algorithm is a dilation. The

gray scale dilatation of A(x, y) by a structuring element B(x, y) is defined by the following expression:

(A⊕B)(x, y) = maxA(x− x′, y − y′) +B(x′, y′) | (x′, y′) ∈ DB (4.6)

where A(x, y) is the output of the thresholding and DB is the domain of the structuring element

B(x, y) [25], that in this case was the following matrix:

B =

1 1 1

1 1 1

1 1 1

Using a morphological operation such as dilation in the last step allowed a simple dilation of hair

lines. This dilation is performed in order to secure that hair lines are entirely removed. This morpho-

logical operation will increase the performance of the artifacts removal and inpaiting step that will be

described in the next section.

30

Figure4.6 shows the result of applying the hair detection algorithm to the 3 images of Figures

1.10 and 4.4. It is possible to see that this algorithm performs reasonably well, achieving a good hair

detection. Nevertheless it still has some wrongly detected regions as can be seen in image 4.6(c) and

also misses some hair artifacts (see image 4.6(b)).

(a) (b) (c)

Figure 4.6: Hair detection in dermoscopy images

4.4 Inpainting

After applying the detection algorithms for the two kinds of artifacts described previously, the

masks obtained are used to remove both of the artifacts from the original gray scale image, as can

be seen in the scheme of Figure 4.1, and replacing them by zeros (see Figure 4.7). This removal

resulted in the appearance of dark regions without information, as can be seen in Figure 4.7.

(a) (b) (c)

Figure 4.7: Gray Scale images after artifacts removal

These regions must be filled, since they are darker than their surroundings and sometimes have

a linear shape, which will interfere with the directional filters used in the network enhancement step.

Abbas et al. [4] report in their paper that there are three main alternatives to perform hair removal

after the detection. Since the main principle is the same, each one of these alternatives can also

be applied in this work to remove the artifacts detected. The first one is performed by replacing the

hair artifacts by bilinear interpolation, followed by a smoothing of the final result using a median filter.

However, this operation often leads to undesirable blurring and color bleeding, so it can disturb the

tumor’s texture and so should not be considered . An example of this kind of hair removal method is

the DullRazor software developed by Lee et al. [33].

31

The second case is performing a hair removal by non-linear PDE-based diffusion methods. These

methods are also called image inpainting [9] and have some advantages over the bilinear interpola-

tion methods because they use the neighboring information in a natural manner through non-linear

diffusion, filling large gaps in an image while maintaining a sharp boundary [4]. However, image in-

painting algorithms can introduce some blurring and have some problems when they try to reproduce

large textured regions because the inpainting algorithm does not synthesis textures. Therefore these

methods should not be used if large areas of a lesion need to be reconstructed, since it will disrupt

the lesion texture.

The third alternative is based in an inpaintg method as well, but in this case it combines the PDE-

diffusion with texture synthesis techniques. This kind of inpainting is called exemplar-based and is

more difficult to use since the parameters required by this algorithm are hard to determine in general.

This algorithms also requires a larger number of iterations than the previous one, which makes it more

slow and with a higher computational complexity. Therefore, the exemplar-based inpainting method

does not provide, in practice, an effective solution for hair and other artifacts removal. [4]

The technique chosen to replace the hair and reflection artifacts was the PDE-based inpainting.

In this method a region Ω that has to be inpainted will be fulfilled progressively by the propagation

of lines, called isotopes, that arrive at its boundary ∂Ω. Each time the lines are propagated the

different regions inside Ω that were created by them are then filled with color, matching those of ∂Ω.

By performing this two steps the region Ω will progressively shrink in a smooth way. In Figure 4.8

it is possible to see how the isotopes are progressively prolongated (the dashed lines represent the

successive prolongations) and the direction of drawing (represented by the vectors ~N ). [9]

Figure 4.8: Propagation direction as the normal to the signed distance to the boundary of the region to beinpainted [9]

In appendix A a mathematical description of the inpainting algorithm developed by [9] and used in

this thesis will be made.

4.4.1 Inpainting Results

Figure 4.9 shows some examples of applying the inpainting algorithm to some images of the

dataset. These images are reconstructions of the ones from Figure 4.7 and if a comparison is per-

formed it can be seen that the inpaiting algorithm performs reasonably well. Nevertheless this algo-

32

(a) (b) (c)

Figure 4.9: Gray Scale images after inpaiting algorithm

rithm does not perform a perfect removal of artifacts and does not propagate textures, only intensities

as can be seen in the three images, so in the future other algorithms should be tested and the results

compared with the ones obtained in this work.

33

34

5Network Detection

Contents5.1 Previous works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365.2 Network Enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385.3 Network Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

35

The following step after the pre-processing block is to enhance and detect pigment network. This

Chapter will be divided in two main sections: one in which it will be described how the network dark

lines were sharpened and a second section where the actual network detection will be performed. In

the scheme of Figure 5.4 the main operations of the network enhancement and detection blocks are

presented in detail.

Figure 5.1: Network Enhancement and Detection blocks

5.1 Previous works

The detection of pigment network was ignored for a long time in literature. However, several works

have been recently published.

Fleming et al. [20] proposed a set of techniques for extracting and measuring pigment network’s

characteristics such as its thickness and the variability of thickness of network lines and the size and

variability of network holes. The method used for network detection was based on an algorithm for

detecting lines (curvilinear structures) proposed by Steger [46]. In this algorithm the lines are regarded

as ridges. The preliminary step of the algorithm is to find the ridge points, i.e., the line centers and

those are identified by recurring to the first and second derivatives values in the direction orthogonal

to the line. A vanishing first derivative and a high second derivative value are the identifiers. After

the detection of the ridge points, the second step is to link them and form lines. This is done by

recurring to information obtained in the previous step such as second derivative values, orientation

and proximity. Grana et al. [26] proposed a similar algorithm for network detection in their of network’s

texture characterization. However, their proposal for line linking is to use a set of morphological

masks to find the line ending points, followed by a extension that begins at each termination found.

An interesting aspect of their work is that they try to distinguish between different kinds of lesions

with pigment network. They classify a lesion as complete network and partial network regarding the

spatial distribution of pigment network within the lesion. Their algorithm achieves and overall network

detection performance of 88.3%.

Two algorithms for network detection were proposed by Anantha et al. [5]. The first one uses a

statistics over neighboring gray-level dependence matrices and the second method involves filtering

with Laws energy masks. The masks used have different dimensions and the output obtained by

applying each one of the masks is squared. In the end a color filtering block is applied to perform the

network detection. Their algorithm achieves an accuracy of 80%.

36

Betta et al. [10] work’s main objective was to the detect the presence of atypical pigment network

in a lesion. To achieve that aim they developed an algorithm which combines structural and spectral

information. The structural technique uses a median filter followed by a close-opening operation. The

combination of these two processes allows the detection of lines and/or points. The spectral method

makes use of a Fourier analysis (Fast Fourier Transform, high-pass filtering, Inverse Fast Fourier

Transform and thresholding) to determine the spacial period of the texture and, as a consequence,

the identification of regions where pigment network exists. In the end the two masks obtained are

combined with a segmentation of the lesion, resulting in one image where areas that belong to the

lesion and posses pigment network are highlighted. In the detection process they appear to achieve a

SE = 50% and a SP = 100%. Those areas are then screened for the presence or absence of atypical

pigment network.

Sadeghi et al. [42] algorithm for network detection is based on two main features of pigment

network: its color (dark lines over a diffuse background) and spatial characteristics. To detect the

sudden transitions in intensity they used a LoG filter. This allows the enhancement of the potential

meshes of the pigment network. After, they perform an analysis based on the spatial characteristics,

transforming the filtered image in a set of graphs, using a 8-connected neighboring. They consider

that the presence of round or cyclic structures in a lesion represent the presence of pigment network

so, after obtaining the set of graphs, they apply the Iterative Loop Counting Algorithm [32] which

allows the detection of all the cyclic subgraphs by transforming the network into a tree and performing

a dept first search on the tree for loops. In the end the subgraphs are filtered and noise, artifacts and

wrongly detected structures (such as dots and globules) are removed and a graph of pigment network

is presented. Their algorithm has an accuracy of 94.3%.

Pigment network is considered an hallmark of melanocytic lesions and is one of the features

used by dermatologists for performing a diagnosis (see Sections 2.2 and 2.3). When inspecting a

lesion which possesses pigment network, experts usually assess the structure of the network, i.e.,

the thickness of the lines and the size of the ”holes”, and classify it as typical or atypical. Therefore,

the development of an automatic algorithm which is capable of detecting and extracting the skeleton

of pigment network could be very useful for dermatologists. Most of the network detection systems

described previously do not try to extract the skeleton of pigment network (the dark lines), they only

intend to determine if pigment network is present or absent and the regions of the lesion where it

can be found. Nevertheless there are two remarkable exceptions [20, 26]. The detection system

proposed is capable not only of detecting the presence of pigment network but also of extracting its

skeleton. A different approach for detecting pigment network is also proposed in this thesis. The main

difference regarding previous works is that color and shape (dark lines over a diffuse background)

are not the only characteristics of pigment network that will be used to perform its detection. The

spacial organization of pigment network is also an important feature that has only been exploited by

Sadeghi et al. [42] and that will be used in this work to extract the skeleton of pigment network, by

recurring to a connected component analysis. Prior to the detection and extraction of pigment network

an enhancement step will be performed. This step uses the bank of directional filters described in

37

Chapter 3, which is also a new proposal for network enhancement.

5.2 Network Enhancement

The dark lines that compose the mesh of pigment network are ones of its most common charac-

teristics. Therefore, these lines can be used to perform a preliminary separation between pigment

network and other structures of the skin.

The network enhancement step occurs after the pre-processing block where hair, reflections and

air bubbles have been removed. The main operations that occur in the network enhancement block

will be presented in the next two subsections.

5.2.1 Directional Filters Bank

The dark lines of pigment network are linear structures. To enhance these linear structures, the

bank of directional filters described in Chapter 3 is used (see (3.8) and (3.9)) with the following pa-

rameters

• σx1= 40 and σy1 = 40

• σx2= 3 and σy2 = 0.5

and each filter mask has the dimensions 11x11. The number of rotations for θi is N = 18, which

means that there are 19 directional filters in the analysis bank. These parameters were all determined

experimentally and are the ones which gave the best global results for the dataset. When comparing

these values with the ones used in the hair detection block (Section 4.3) it is possible to conclude that

they are very different. This happens because the length of each line stroke in the pigment network

is much smaller than the case of the hair artifacts.

Figure 5.2 show a gray scale dermoscopy image (left) and the output of the directional filters bank

(right)

(a) Original Gray-Scale Image (b) Filter Response

Figure 5.2: Output for the directional filters bank

38

5.2.2 Thresholding

As can be seen in image 5.2(b) the output of the directional filters bank still presents some in-

formation without interest such as noise and artifacts. By applying a simple thresholding step it is

possible to reduce the amount of information, keeping only the importante one. Being If the filter

response, the thresholding step operates in the following way:

If (x, y) > TN (5.1)

where TN is a threshold value. Only the pixels (x, y) above the threshold value are validated as

belonging to the pigment network.

It was experimentally determined that the best value for TN is 0.0185. By applying this threshold

to the image 5.2(b) was possible to obtain the binary Figure 5.3, where a great amount of noise was

removed.

Figure 5.3: Output of the thresholding step.

5.3 Network Detection

The previous step independently classifies each pixel, without taking neighbor pixels into account,

and does not consider the topological relationships among pixels of the pigmented network. In this

block the network geometry will be considered. Now, it will be assumed that the pigment network is

a set of connected regions, so by performing a connected component analysis over the binary image

obtained in the previous block, the connected dark lines can be extracted.

Connected component analysis works by scanning an image, pixel-by-pixel, in order to identify the

connected pixel regions. A region A (a group of pixels) is said to be connected if for any two pixels

p1, p2 ∈ A, there is a path that links these two pixels and that belongs to A. The path is composed of

a sequence of neighbor pixels which starts in p1 and ends in p2. The detection of connected regions

depends on the connectivity or neighborhood used. The two most common neighborhoods are 4-

connectivity (4 adjacent pixels) and 8-connectivity (8 adjacent pixels). In this work a 8-connectivity

was used because it considers not only vertical and horizontal links but also the ones along the

diagonals. This allows the detection of larger regions and the linking of some dark lines that were not

39

going to be linked otherwise. The qualitative comparison between these two ways of assessing the

connectivity of a pixel can be seen in Section 7.3. The 8-connectivity or 8-neighbors of a pixel p(x, y)

is defined as:

N8(p) = (x+1, y), (x−1, y), (x, y+1), (x, y−1), (x+1, y+1), (x+1, y−1), (x−1, y+1), (x−1, y−1)

(5.2)

Of all the connected regions detected by the connected component analysis only the ones which

have areas bigger than a threshold are selected as being part of the pigment network. Let Ri be the

i− th connected region. The area criteria that the region must fulfill is

A(Ri) > Amin (5.3)

where A(Ri) denotes the area of the region Ri and Amin is the threshold. This threshold was deter-

mined experimentally and it is Amin = 700 pixels. By enforcing this conditions it is possible to exclude

connected components with small areas.

The final pigment network region R is computed as the union of all the connected components

which met condition (5.3)

R = ∪i:A(Ri)>Amin

Ri (5.4)

The final result for the network detection block can be seen in Figure 5.4. This is the output for

image 5.2(a) and if this image is compared with Figures 5.3 and 5.2(b) it is possible to see that these

two final steps allow the removal of the incorrect data that remained after the thresholding step.

Figure 5.4: Output of the network detection block.

40

5.4 Examples

For a better visualization of the final output of the network enhancement and detection blocks

some examples will be presented this section. Figure 5.5 shows two examples for which the detection

algorithm performed well and Figure 5.6 shows two detection errors. In this last figure the pattern

present in the lesion is the so called cobblestone pattern, which is composed of dark circular struc-

tures (see Section 2.2). The algorithm detects the boundaries of the circular structures which form a

connected component.

(a) (b)

(c) (d)

Figure 5.5: Detection of Pigment Network: original images(left) and pigmented network (right)

The detection algorithm tends to detect regions in excess. This is a good characteristic since the

number of detection failures (pigment network regions which are not detected) is very small. However,

there are many false alarms as well.

41

(a) (b)

(c) (d)

Figure 5.6: Detection errors: original image(left) and pigmented network (right)

42

6Region Validation and Lesion

Classification

Contents6.1 Region and Lesion Ground Truths . . . . . . . . . . . . . . . . . . . . . . . . . . . 446.2 Background Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.3 Region Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.4 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.5 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496.6 Lesion Classification algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

43

Chapter 6 addresses two problems:

(i) How to improve the detection of pigmented network regions?

(ii) How to classify the whole lesion?

The algorithm proposed in Chapter 5 detects pigment network regions in excess. This means that

it detects regions in all the images of the database used in this thesis which contain pigment network,

but has many false alarms too (see Figure 5.6). The algorithm used in the network detection block

(see Section 5.3) is simple and does not explore all pigment network properties. Therefore, some

region post-processing can be done to improve the detection results. To do so, a pattern recognition

approach will be adopted. First, features will be extracted from each of regions detected. These

features characterize each one of the regions regarding their intensity, texture and topology and will be

then used to train a classifier (AdaBoost [54]) to separate pigment network regions from not pigment

network regions. The procedures used to perform the features extraction will be addressed in Section

6.4 and the classification algorithm used will be explained in Section 6.5.

To assign a binary label to the whole lesion, a classifier based in area ratio will be used. By

using this procedure it is possible to classify a lesion as with or without pigment network. The lesion

classification method will be discussed in Section 6.6.

6.1 Region and Lesion Ground Truths

In order to train and test the classifiers, each dermoscopy image used in this thesis was classified

by the author under the guidance of an expert.

The expert classified each lesion regarding the presence or absence of pigment network. This

classification will be denoted as lesion ground truth in this thesis. For the cases where the expert de-

cided that pigment network was present, he manually segmented the regions where pigment network

existed (see Figure 6.1). This manual segmentation will be used as the binary region ground truth GT

in this thesis.

(a) Original image with pigment network re-gions highlighted

(b) Binary Ground Truth

Figure 6.1: Ground truth segmentation of the pigmented network regions.

44

6.2 Background Extraction

The output of the network detection block (see Section 5.3) contains only the dark lines which

form the mesh (the skeleton) of pigment network. The top left image of figure 6.2 shows the binary

output of the network detection block. To perform the region validation step several features which

characterize the intensity, texture and topology of the regions will be extracted. To extract the intensity

and texture features it is necessary to provide the whole region, i.e., the lines and the background.

Therefore, it is necessary to add the background, the ”holes”, to the output of the network detection

block in order to obtain the complete region from which the features will be extracted.

To obtain the background, the regions Ri obtained in the detection block are processed in the

following way:

1. A morphological dilation, such as the one described in (4.6), is applied using a structuring

element equal to the one used in section 4.3. This operations allows the linking of some of the

lines from pigment network which are not completely closed and that, therefore, decreased the

performance of the next morphological operation.

2. By using a morphological filling it is possible to fill the holes in the mesh and create a binary

mask N of the possible network regions Rfi in an image.

Figure 6.2 displays the steps of region processing described above. The results showed are binary

masks. The final regions Rfi are obtained by multiplying the gray scale image by the respective binary

mask N .

The regions Rfi are used for the region labeling process described in Section 6.3 and, the ones

which were not discarded, for the extraction of image features, which will be described in Section 6.4.

6.3 Region Labeling

A classification algorithm is used to assign a binary label to all the detected regions. This step is

called region validation.

In order to train the classifier, a database of regions and the desired label for each region (training

data): 1 for pigmented network and 0 otherwise is required. Therefore, it is necessary to label the

detected regions. This is done by overlapping the detected regions Rfi with the ground truth GT . The

algorithm used to perform region labeling will be explained in this section.

For each image I there is a set of detected regions Rfi such that Rf1 , Rf2 , . . . , Rfn ∈ I. Each

one of the regions is analyzed separately. First, the intersection between the region Rfi and the

Ground Truth mask GT of image I is computed

R∗fi = Rfi ∩GT (6.1)

The amount of overlap λ is obtained as follows

λ =A(Rfi ∩GT )

A(Rfi)(6.2)

45

(a) Original (b) Output of the network detection block

(c) Morphological Dilation (d) Morphological Reconstruction

Figure 6.2: Steps of the background extraction.

where A(Rfi ∩GT ) is the area of the overlapping and A(Rfi) is the area of the region Rfi .

A region Rfi is classified as pigment network (1), not pigment network (0), or discarded, according

to

Rfi =

1 if λ ≥ 0.7

discarded if 0.3 < λ < 0.7

0 if λ ≤ 0.3

(6.3)

By establishing a labeling condition based on an area ratio, λ, it is possible to classify the regions

with a high level of confidence. The regions discarded are the ones that could induce a bad learning

in the training group so they are excluded and not used for training. The regions that have an area

equal or larger than 70% of pigment network receive the label 1 and are used for training. The same

happens with the regions that have less than 30% of pigment network, which receive the label 0. By

enforcing this last condition, it is possible to ensure that all the detection errors will receive the label 0

and will be used as negative examples.

Figure 6.3 exemplifies the labeling process for the regions detected in the original left image of

Figure 6.1. The left image shows the regions detected in the lesion (there are 6 detected regions:

A-blue, B-green, C- pink, D-yellow, E-orange and F-red) and the right image shows the overlapping

between the regions detected and the region ground truth (see Section 6.1).

46

(a) Detected regions (b) Overlapping between the detected regions andthe ground truth

Figure 6.3: Region labeling. Overlapping: A(blue) = 74% (label 1), B(green) = 0% (label 0), C(pink) = 0% (label0), D(yellow) = 0% (label 0), E(orange) = 76% (label 1), F(red) = 24% (label 0)

6.4 Feature Extraction

The vector of features extracted from each region is composed of features which describe texture,

intensity and geometry or shape (area of the network, number of holes, etc). The total number of

features is 51 and they can be divided in two different groups: image features and topological features.

In this section the procedures used to extract these two group of features will be presented.

6.4.1 Image Features

The following image features are used to characterize each region obtained with the processing

described in Section 6.2. They comprise information about intensity (histogram of intensities, vari-

ance) and texture (histogram of gradient, entropy), leading to a total of 38 features.

Next, each set of features is defined:

• Histogram of Intensities (26 features) - The histogram of pixel intensities inside the region is

computed by splitting the intensity range into L=26 bins. The histogram is normalized by the

number of pixels in the region Rfi .

• Variance (1 feature) - The variance σ2 for each region is computed using the following equation

[24,25]

σ2 =

(1

n

n∑i=1

(Ii − I)2

)2

(6.4)

where Ii is the intensity of the pixel i, I is the average intensity of the region and n is the number

of pixels in the region Rfi .

• HoG (10 features) - The Histogram of gradient is computed by recurring to the gradient vector

of an image. An image gradient is a directional change in the intensity in an image and is given

by the expression

∇I(x, y) =

[∂I(x, y)

∂x,∂I(x, y)

∂y

](6.5)

47

where I(x, y) is the intensity of region Rfi and the two partial derivatives are computed by using

a Sobel Operator S [24]

S =

−1 0 1

−2 0 2

−1 0 1

such that [24]

∂I(x, y)

∂x' S ∗ I(x, y) (6.6a)

∂I(x, y)

∂y' ST ∗ I(x, y) (6.6b)

The two partial derivatives are actually two images which at each point contain the horizontal

and vertical derivative. The gradient vector can also be characterized by the amplitude and

orientation (angle - θ) defined as follows [24]

G =

√(∂I(x, y)

∂x

)2

+

(∂I(x, y)

∂y

)2

(6.7)

The direction of the gradient θ is also important for computing the HoG [24]

θ = tan−1

(∂I(x,y)∂y

∂I(x,y)∂x

)(6.8)

The histogram of gradient (HOG) is defined as follows

hogi =∑

(x,y)∈I(x,y)

G(x, y) · bi(θ(x, y)) (6.9)

where

bi(θ(x, y)) =

1 if θ ∈ i-th bin0 if otherwise

(6.10)

10 bins, with an angular interval of 18, are used so a total of 10 features is computed (10 bins

x 1 region). A normalization is performed by using the number of pixels in the region Rfi .

• Entropy (1 feature) - The entropy S characterizes the texture of the region and is computed in

the following way [24,25]

S = −L∑k=1

h(k)log[h(k)] (6.11)

where h(k) is the histogram of intensities extracted previously and L = 10 bins.

6.4.2 Topological Features

These features are used in an attempt to separately characterize the structure of the network and

holes. 13 topological features, that can be divided in 4 different sets, are extracted

• Network Area (1 feature) - This feature characterizes the dark lines of pigment network and is

computed by calculating the area value of the region Ri obtained after the network detection

block. The final value is normalized by the area of the region Rfi .

48

• Network Area after erosion (1 feature) - This feature characterizes the mesh after applying

a morphological erosion to region Ri. The erosion of region Ri by a structuring element B is

defined as

(Ri B)(x, y) = minRi(x+ x′, y + y′)−B(x′, y′) | (x′, y′) ∈ DB (6.12)

where DB is the domain of the structuring element B, which is the same element as the one

used in Section 4.3 for the morphological dilation [25]. In the end the area value is computed for

the region obtained and this value is normalized by the area of the region Rfi .

• Number of Holes (1 feature) - The number of holes characterizes the background. The back-

ground RBi is extracted from region Rfi

RBi = Rfi \R∗i (6.13)

where R∗i is the result of applying the morphological dilation described in Section 6.2 to the

output of the network detection block Ri. To determine the number of holes a connected com-

ponent analysis is performed to extract all the connected regions from RBi . The number of

holes is also normalized by the area of the region Rfi .

• Histogram of the Area of Holes (10 features) - These 10 features also characterize the back-

ground. The connected component analyses used to compute previous feature can also be

used to determine the area of each connected region (hole) present in RBi . The area of the

holes is normalized in the same way as the previous features. The histogram is composed of

10 bins, which result in a total of 10 features.

Since these features describe the lines and the background separately, they can be used in the

future to discriminate between atypical and typical pigment network.

6.5 Classification

The algorithm used to learn the classification function was the AdaBoost [54] which classifies the

data using an ensemble of simple classifiers denoted as weak classifiers. This algorithm was chosen

not only because it gives good results in challenging object recognition problems,e.g. face detection,

but also because this algorithm is able to select a good subset of informative features for a given

problem (feature selection). This best subset is selected by assuming that each classification function

depends on a single feature.

The AdaBoost procedure can be seen as a greedy feature selection process. At the beginning

it receives as an input the feature vectors xk = x1k , x2k , . . . , xNk which characterize the regions

Rfkand the corresponding label yk = 0, 1. The weights wk are associated to each region in such way

that the positive examples receive weights with higher values, since a correct classification of these

examples is more important

wk =

αm if yk = 11l if yk = 0

(6.14)

49

where m is the number of positive examples, α is a parameter and l is the number of negative exam-

ples.

The final classification function is learned in T iterations. In each iteration the best weak classifier

is trained, being each classifier restricted to a single feature. They are called weak classifiers because

each of them is too simple to provide a good classification of the data. The combination of all weak

classifiers outputs may achieve a good classification performance (strong classifier). After the weak

learning, the best weak classifier, and as a result the best feature, is selected with respect to the

weighted error

εt = minxik ,p,θ∑k

wk | h(xik , p, θ)− yk | (6.15)

where t is the iteration number and h(xik , p, θ) is a learned weak classifier, restricted to feature xik

h(xik , p, θ) =

1 if pxik < pθ

0 otherwise(6.16)

where p ∈ +1,−1 is a polarity and θ is the optimal threshold for which the minimal number of

examples are misclassified.

After selecting the best weak classifier hb(xbk , pb, θb) the weights are updated

wt+1,k = wt,kβ(1−ek)t (6.17)

where ek = 0 if Rfk is classified correctly and 1 otherwise and βt = εt1−εt . This way of updating the

weights makes possible to emphasize the examples that were incorrectly classified by the best weak

classifier.

Finally, a strong classifier is defined by combining the decisions of the weak classifiers. This final

classifier is a combination of weighted best weak classifiers and a threshold

Ct(Rfk) =

1 if

∑tj=1 ηjhbj ≥

12

∑tj=1 ηj

0 otherwise(6.18)

where ηj = 1βj

.

After T iterations a final classification CT (Rfk) is obtained using the data belonging to the training

set. The parameters (α, T and the best feature vector) used in this work were selected empirically.

The reasons for choosing this parameters will be discussed in Section 7.4.

6.6 Lesion Classification algorithm

The second main objective of this work is to perform a binary a classification of skin lesions as

with or without pigment network (1 or 0, respectively, just to consider a simple labeling). Classifying a

lesion regarding the presence or absence of pigment network is not a trivial task. In a first approach

a simple classification based on the regions detected and validated as pigment network should be

enough, i.e., a lesion would be classified as with pigment network if it possessed at least one region

detected and validated as network. Despite being simple, this proposal is not a good one since it

would not consider the size of the lesion, the size of the region or regions detected and the distribu-

tion of pigment network throughout the lesion. Therefore it could induce the appearance of several

50

classification errors (lesions classified as with pigment network without being so), since even after the

post-processing block there are still some regions wrongly classified as pigment network (see Section

7.6.2.B).

It was decided that a different approach should be used to perform the lesion classification. Pig-

ment network is usually a structure regularly meshed with narrow spaces, which is distributed more

or less regularly throughout the lesion [3]. This suggests that pigment network is a dense struc-

ture. Therefore a good method for performing a lesion classification could be based on the area ratio

between the regions detected and the area of the lesion. This was the approach used.

To perform the area ratio it was necessary to segment the lesion. Since the segmentation is not

the main focus of this thesis, this step was not automated. The dermoscopic images were manually

segmented and each segmentation is actually a binary mask SGT , as can be seen in Figure 6.4.

(a) Original image (b) Binary Segmentation Mask

Figure 6.4: Lesion Segmentation

The statistical procedure used will be described in detail now. First the total area of the lesions

is computed by using the binary mask SGT . After that, the total network area is determined. This is

done by computing the area of the total pigment network region R obtained as follows

R = [ ∪i:A(Ri)>Amin∧CT (Ri=1)

Ri] ∩ SGT (6.19)

R is then defined as the union of all the regions Ri detected by the network detection block and

validated as pigment network by the region validation block, that can be found inside the lesion. The

regionR is only composed of the potential skeleton of pigment network, the ”holes” are not considered

to perform lesion classification.

After determining the area of the pigment network region the classification of a lesion L is per-

formed in the following way

L =

with pigment network if λ ≥ TAwithout pigment network if λ < TA

(6.20)

where TA is a threshold and

λ =A(R)

A(SGT )(6.21)

A(R) is the area of R and A(SGT ) is the area of SGT . It was determined that the best value for

TA = 0.1.

51

52

7Results and Discussion

Contents7.1 Database of dermoscopy images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547.2 Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547.3 Comparison between 4 and 8 pixels-neighborhood . . . . . . . . . . . . . . . . . 577.4 AdaBoost parameters selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587.5 Comparison between gray-level conversion methods . . . . . . . . . . . . . . . . 637.6 Region Detection System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 647.7 Lesion Classification Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

53

This chapter presents and discusses the application of the proposed algorithm to a database

of dermoscopy images. Some parameters choices will also be discussed in this chapter, namely

decisions regarding the number of neighbors used in the connected component analysis (Section

7.3), the Adaboost parameters (Section 7.4) and the gray-level conversion method (Section 7.5).

7.1 Database of dermoscopy images

The algorithm described in the previous chapters was tested on a set of 57 dermoscopy images,

extracted from the database of Hospital Pedro Hispano, Matosinhos. This set contains RGB images

of size around 576x767, stored in bitmap and jpeg formats. Each color component is in the range 0,

. . . , 255. The images were acquired during clinical exams using a dermatoscope and a magnification

of 20x.

The images were selected in such a way that there were examples of melanocytic and non-

melanocytic lesions in the dataset used, but more melanocytic examples. The percentages can be

seen in Table 7.1.

Table 7.1: Percentage of melanocytic and non-melanocytic lesions in the dataset.

Melanocytic Non-melanocytic91% 9%

Pigment network is present in 13 of the melanocytic lesions, this means that around 23% of all

images in the dataset were classified by an expert as ”with pigment network”.

Table 7.2: Percentage of .

Pigment Network Non-pigment Network23% 77%

Both classifications were performed by an expert dermatologist with training in the analysis of

dermoscopy images.

7.2 Performance Measures

In this section, the procedures used to evaluate the performance of the detection algorithm are

addressed.

Since the database of dermoscopy images is small and it is not possible to use the same image for

training and testing the classifiers, a K fold cross validation procedure (K = 5) was adopted to perform

the evaluation. The advantage of this method is that all observations are used for training and for

test but in different experiments. Each one is used for test exactly once. The K fold cross validation

method requires the dataset to be sampled in K subsets with approximately the same size. Therefore,

54

the dataset of 57 dermoscopy images was divided into 5 subsets with approximately the same number

of positive and negative examples (regions with and without pigment network, respectively).

One of the subsets is used to test the data while the other four are used for training. In the test

phase, the output of the algorithm for the test data is compared with the respective ground truth and a

statistics is computed. The process is repeated five times, each one with a different subset as the test

data. The final statistics, which evaluates the performance of the algorithm, is computed by averaging

the statistical values obtained for the five different subsets. All the statistical values presented in this

chapter were obtained using the 5 fold cross validation method described.

This thesis addresses two main problems: the detection of regions containing pigment network

and the binary labeling of lesions with respect to the presence of pigment network. These two prob-

lems were evaluated with different statistics that will be presented in the following subsections. The

statistic used to evaluate the performance of the AdaBoost classifier will also be discussed, since

an evaluation process is required in order to be able to select the best parameters for training the

classifier.

7.2.1 Assessment of Region Detection Algorithm

The output of the network detection system consists of a set of regions that must be compared

with the binary region ground truth (see Section 6.1). This comparison can be done using two types

of statistics: pixel based and region based statistics. These two alternative will be presented next.

7.2.1.A Pixel based statistics

This evaluation method is done by comparing the (binary) segmented image BN to the ground

truth segmentation GT ) (see Section 6.1). A pixel p(x, y) can be classified as TP, TN,FN or FP as

follows

p(x, y) =

TP if BN (x, y) = 1 ∧GT (x, y) = 1

FP if BN (x, y) = 1 ∧GT (x, y) = 0

TN if BN (x, y) = 0 ∧GT (x, y) = 0

FN if BN (x, y) = 0 ∧GT (x, y) = 1

(7.1)

where

TP: Correctly detected as pigment network pixel.

FP: Wrongly detected as pigment network pixel.

TN: Correctly undetected pixel (not pigment network).

FN: Wrongly undetected pixel (not pigment network).

The performance of the detection system is assessed by determining the number of TP, TN,FN

and FP in the test images and by computing the SE and SP values

55

SE =#TP

#TP + #FN(7.2a)

SP =#TN

#TN + #FP(7.2b)

7.2.1.B Region-based statistics

To evaluate the performance of the network detection algorithm at the region level the binary

mask BN of each image in the test set is compared with the respective binary ground truth GT

(see Section 6.1) and classified in one of the the following classes: correct detection (CD), false

alarm (FA) and detection failure (DF). This procedure is called region matching [37] and establishes a

correspondence between the detected regions and the ground truth. The three cases considered are

the following

CD: The detected region matches one or more regions in the GT .

FA: The detected region has no correspondence.

DF: One GT region has no correspondence.

Cases regarding the splitting or merging of regions [37] are not differentiated from CD.

The region matching algorithm is performed by computing a correspondence matrix C which de-

fines the correspondence between the active regions of a pair of images [37]. Assuming that GT is

composed of P regions and that BN contains M detected regions, C will be a P ×M matrix, defined

as follows

C(i, j) =

1 if A(BNj ∩GTi) ≥ 0.3A(BNj )

∀ i ∈ 1, . . . ,P, j ∈ 1, . . . ,M0 if otherwise

(7.3)

where A(BNj ∩ GTi) is the area of the region BNj ∩ GTi and A(BNj ) is the area of BNj . It is also

useful to add the number of ones in each line or column, defining two auxiliary vectors [37]

L(i) =

P∑j=1

C(i, j) i ∈ 1, . . . ,M (7.4)

K(j) =

M∑i=1

C(i, j) j ∈ 1, . . . , P (7.5)

A region BNj is classified as CD if K(j) ≥ 1. By imposing this condition it is possible to ensure that

detected regions which are actually the merge of two regions of the ground truth are only accounted

once as CD. Detected regions which result from a split of a ground truth region are independently

classified as CD. FA and DF are determined by computing the number of empty columns or lines in

C, respectively [37].

The final statistics is performed by computing the total values of CD, FA and DF for the test set.

56

7.2.2 Assessment of the validation step

The output of the AdaBoost classifier consists of a binary classification of a region as 1 (pigment

network) or 0 (not-network). The classifier can be evaluated by computing the SE and SP values. To

do so, the output of the classifier is compared the ground truth label (see Sections 6.1 and 6.3). By

performing this comparison is possible to assess the values of

#TP: Number of regions correctly classified as pigment network.

#FP: Number of regions wrongly classified as pigment network.

#TN: Number of regions correctly classified as not pigment network.

#FN: Number of regions wrongly classified as not pigment network.

After computing the previous values, it is possible to determine the SE and the SP of the classifier

for the test set.

7.2.3 Assessment of Lesion Classification

To evaluate the performance of the lesion classification algorithm (see Section 6.6) the binary

output of the classifier is compared with the binary ground truth (see Section 6.1) for each one of the

images of the test set. The output of the classifier can be regarded as

TP: The lesion is correctly classified as with pigment network.

FP: The lesion is incorrectly classified as with pigment network.

TN: The lesion is correctly classified as without pigment network.

FN: The lesion is incorrectly classified as without pigment network.

The numbers of TP, TN,FN and FP are determined for the test set and the sensitivity (SE) and

specificity (SP) are computed.

7.3 Comparison between 4 and 8 pixels-neighborhood

Connected component analysis (see Section 5.3) depends on the neighborhood system chosen

by the user. The two most usual neighborhood systems are 4 and 8 connectivity. The main difference

between the two connectivities is that by using the 8-connectivity one uses the information of the

8 adjacent pixels (vertical, horizontal and the two diagonals), while in the 4-connectivity only four

neighbors are used (vertical and horizontal). In theory, 8-connectivity allows the detection of larger

linked areas since it also uses the information from the upper diagonal neighbors. Both connectivities

were tested and a qualitative result can be seen in Figure 7.1. Both images are outputs of the network

detection block.

By examining these two images it is possible to conclude that, although the outputs are quite

similar, the one obtained by using a neighborhood of 8 pixels is slightly better, since more network

57

(a) Original (b) 4-connectivity (c) 8-connectivity

Figure 7.1: Output of the network detection block for different connectivities.

regions are detected. Therefore the final decision was to use 8-connectivity. However, it is important

to say that for most examples of lesions with pigment network the output of the detection block was

exactly the same for the two connectivities. This suggests that a 4-connectivity could also be used

and the detection block would perform well too.

7.4 AdaBoost parameters selection

The parameters used to train the classifier were selected in order to be the ones which give the

best final results, i.e., the best values for SE and SP (see Section 7.2.2). It was decided that wrongly

classifying positive network regions should be seen as a more problematic error, since pigment net-

work is a very important structure in dermoscopy image analyses. To enforce the correct classification

of this kind of regions different weights were assigned to the positive and negative regions (see Sec-

tion 6.5). Different values for the α parameter were tested and will be discussed next. Simultaneously,

a study was performed in order to determine which features’ combination could discriminate better

the positive from negative regions. Finally, the number of iterations of the classifier were also cho-

sen in order to achieve the best classification. After computing the best α value different tests were

performed in order to determine the best number T of iterations.

7.4.1 α weight parameter

The classifier was trained using different α ∈ 1, 3, 5, 10, 15 values (see (6.14)) and 5-fold cross

validation.

A large number of iterations was considered (T = 2000). This value guarantees that there are no

classification errors in the training set.

The final results for the different α values can be seen in Table 7.3. This table shows the percent-

age of correctly classified network regions (SE or TP rate) and the percentage of wrongly classified

not-pigment network regions (100% − SP or FP rate) for three different sets of features composed

of: only Image Features, only Topological Features, and both. These statistics are displayed in the

receiver operating characteristic (ROC) shown in Figure 7.2.

By inspecting the previous table it is possible to conclude that when α increases the percentage

58

Table 7.3: Statistical Results for the different α values and T = 2000. FP-rate stands for the percentage of notpigment network regions that are wrongly classified ; TP-rate is the percentage of correctly classified pigmentnetwork regions.

Image Features Topological Features Bothα FP-rate TP-rate FP-rate TP-rate FP-rate TP-rate1 22% 30% 20% 44% 18% 38%3 26% 35% 21% 45% 22% 42%5 27% 36% 24% 45% 23% 42%

10 29% 37% 24% 46% 24% 45%15 30% 39% 25% 47% 26% 45%

of correctly classified regions increases for the three sets of features. However, the number of false

positives increases as well. Nevertheless, the correct identification of pigment network regions is con-

sidered more important than rejecting detection errors. Therefore, and based on the results obtained,

the value chosen for parameter α was 15. The low percentage of TP might be explained by the large

number of iterations used, since it could have caused an overfitting of the classifier to the training data

used.

Higher values for α could be tested and a better ROC curve could be obtained (see Figure 7.2.

However, an increase in the value of this parameter resulted in an increase of the number of iterations

of the learning algorithm, which is time consuming, and in a decrease of the correctly classified

negative examples.Therefore, the decision was made to use 15 as the highest value for parameter α.

This is not the better approach to select a parameter so, in the future, different solutions for computing

this weight index should be considered, e.g. using the bayesian probability.

7.4.2 Features Comparison

The previous study was performed using three different sets of features. The results from Table

7.3 were used to create a ROC curve for each one of the features’ sets. The various ROC curves can

be seen in Figure 7.2.

The receiver operator charecteristics (ROC) curve is a good performance measure. It is a plot of

the TP-rate (percentage of correctly classified positive samples) versus the FP-rate (percentage of

wrongly classified negative samples). The curve illustrates the behavior of a classifier without regard

to class distributions or error costs, and thus decouples the classification performance of these factors

[15]. The area under the curve for the peak value gives information about the classifier performance.

The higher the value the better the test is.

The ROC curves obtained show the difficulty of the problem. The TP rate is far from 100%. Never-

theless, they provide some information regarding the set of features that better discriminates between

positive and negative examples. By analyzing the results it is possible to conclude that the best set of

features is the one composed of only topological network features. This is the set that simultaneously

has the lowest percentage of wrongly classified negative examples (25%) the highest TP-rate of 47%,

when compared with the other two sets for α = 15 (see Table 7.3). Another comparison can be done

59

Figure 7.2: ROC curve obtained for the three different features’ sets by varying the α ∈ 1, 3, 5, 10, 15 weightparameter and T = 2000.

by inspecting Figure 7.2, where it is clear that the ROC curve for the topological features set is the

one which have a higher area under the curve value, therefore, this is the best set of features.

An interesting study is to determine which features, among all the topological ones, play a major

role in the learning procedure. To perform this study a statistical analysis was made considering all

the features selected by the AdaBoost algorithm (see Section 6.5), i.e., the features used to define

the strong classifier. The results of this study can be seen in the histogram of Figure 7.3.

Figure 7.3: Histogram of the average rate of selection of each topological feature for α = 15 and T = 2000.Each one of the ”Area Holes i” feature stands for the i− th bin of the histogram of the area of holes.

The previous histogram plots the number of times each topological feature is chosen during the

Adaboost training with five fold cross validation, α = 15 and T = 2000. Five different trainings groups

60

are used, each one using a different combination of the five existing subsets. Therefore the results

shown in Figure 7.3 are an average of the results of the five training groups. By inspecting the results

it is possible to conclude that there are four topological features that have a more preponderant role

in the final classification rule. Those are, by rate of selection, ”Number of Holes”, ”Network Area”,

”Network Area after erosion” and the 10 − th bin of the ”Histogram of the Area of Holes”. The most

important feature is the ”Number of Holes”. This may be due to the fact that ”holes” are characteristic

of a mesh structure such as the pigment network; other structures that are not meshed do not have

holes (or at least not as many). Therefore the ”Number of Holes” can be a feature that performs a

good discrimination between mesh-like structures and others.

Despite the fact that topological features were the ones selected to obtain the final results, it was

considered that a similar study should be performed using the set composed of both kinds of features,

since it could be interesting to verify if the same behavior was observed. Figure 7.4 presents the

results obtained.

Figure 7.4: Histogram of the average rate of selection of all the features for α = 15 and T = 500. Each one ofthe ”H i” , ”HoG i” feature and ”H i” feature stands for the i− th bin of, respectively, the histogram of intensities,HoG and histogram of the area of holes.

As previously, the ”Number of Holes” is the most used feature. The ”Network Area” is also used

several times. These findings and the previous ones suggest that ”Number of holes” and ”Network

Area” might be cue features in distinguishing between true and false pigment network regions.

The correlation between different features was also studied. By plotting one feature against the

other in scatter plots it is possible to extract information regarding how well the two features can

discriminate positive from negative samples. In Figure 7.5 four scatter plots between pairs of the three

most used topological features can be seen. By observing these graphics it is possible to conclude

that the positive and negative examples, labeled respectively as ”network” and ”no network”, are

not linearly separable and show a strong overlap. This makes the discrimination between pigment

and non-pigment network regions very difficult. Since some positive and negative regions have very

61

similar values for the same features this explains why the classification scores shown in 7.3 and 7.4

are far from perfect since it is not possible to simultaneously obtain high SP and SE scores.

(a) ”Number of Holes” vs ”Network Area” (b) ”Number of Holes” vs ”Network Area afterErosion”

(c) ”Network Area” vs ”Network Area afterErosion

Figure 7.5: Correlation Graphics

7.4.3 Number of iterations

The number of iterations determines the number of best weak classifiers that will be selected

to compute the final strong classifier. By keeping constant α = 15 and the input of features used

(topological features) several tests were performed in order to determine which is the best number of

iterations. The cross-validation results for the several tests are resumed in Table 7.4.

Table 7.4: Statistical results for different numbers of iterations for α = 15 and topological features. FP-rate standsfor the percentage of non-network regions that are wrongly classified as so while TP-rate is the percentage ofcorrectly classified pigment network regions.

T FP-rate TP-rate100 49% 86%150 46% 77%200 45% 75%300 40% 74%2000 25% 47%

These results show unequivocally that an increasing in the number of weak classifiers results in

a dramatically decrease of the percentage of true positives. Despite the decrease of the FP-rate,

62

those are not good results since pigment network is a very important dermoscopic structure which

should not be missed. This reinforces the idea that increasing the number of iterations will lead to

an overfitting of the classifier. Therefore, it was decided that to prevent a high percentage of false

negatives (pigment network regions wrongly classified) and an overfitting, the number of iterations

should be equal to T = 100. The hypothesis of using T = 150 was also tested. However, the results

for lesion classification (which will be presented in Section 7.7) regarding lesions with pigment network

were worse.

7.5 Comparison between gray-level conversion methods

In order to select the best gray-scale conversion method (see Section 4.1) the performance of

the AdaBoost was evaluated for both methods. Tables 7.5 and 7.6 show the results using (4.2) and

(4.1), respectively. The results were obtained by using the best parameters for AdaBoost determined

previously (α = 15, T = 100 and topological features).

Table 7.5: Statistical results for AdaBoost performance using the highest entropy channel (see (4.2)) as graylevel conversion method for α = 15, T = 100 and topological features.

DetectedNot pigment network Pigment Network

Truth Not pigment network 51% 49%Pigment Network 14% 86%

Table 7.6: Statistical results for AdaBoost performance using the linear combination of RGB values (see (4.1)) agray level conversion method for α = 15, T = 100 and topological features.

DetectedNot Pigment Network Pigment Network

Truth Not Pigment Network 33% 67%Pigment Network 27% 73%

By inspecting both tables it is possible to notice that the classifier achieves a SE = 86% and a

SP = 51% when using the highest entropy channel and a SE = 73% and a SP = 33% when using the

linear combination of RGB values. Since both SE and SP have higher values for (4.2) it is possible

to conclude that the conversion method which gives the best discrimination and, therefore, the best

results for the detection algorithm is the highest entropy channel. This was the method used to obtain

the results presented in the following sections.

63

7.6 Region Detection System

7.6.1 Detection examples

Figure 7.6 shows some examples of the output of the detection system for lesions which contain

pigment network. In these figures, the left column corresponds to the original image, the middle

column is the output of the detection block before the validation steps described, respectively, in

Sections 6.2 and 6.3 and the left column corresponds to the final output of the region detection

system.

By observing the examples of Figure 7.6 it is possible to conclude that the region validation block

does not affect significantly the extraction of pigment network areas. In most cases, some of which

are exemplified, the output of the two blocks is exactly the same. There are other cases for which the

output is slightly different and this is due to some misclassified pigment network region. Nevertheless,

most of the pigment network skeleton is still extracted, as can be seen in the last set of three images

from Figure 7.6.

The main objective of the post-processing block is to eliminate detection errors or false alarms.

Figure 7.7 shows some examples of applying the detection algorithm to lesions without pigment net-

work. The results demonstrate that the post-processing block removes some of the wrongly detected

regions but not all of them. Actually, in the last two rows of Figure 7.7 it is possible to see that the

post-processing block is almost ineffective, since most part of the false alarms remain. These results

suggest that in the future new approaches for removing some misleading structures such as dots

(last row of the figure) or the cobblestone pattern (7.7(g)) should be tested. The post-processing

block must also be altered and different combinations of features should be tried, like combining topo-

logical features with only HoG or variance, in order to provide the classifier with some features that

discriminate intensity or texture.

64

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

(j) (k) (l)

(m) (n) (o)

Figure 7.6: Output of the automatic detection system for lesions with Pigment Network: original image (left),output of the network detection block (center) and final output after the region validation block (right)

65

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

(j) (k) (l)

Figure 7.7: Output of the automatic detection system for lesions without Pigment Network: original image (left),output of the network detection block (center) and final output after the region validation block (right)

66

7.6.2 Statistical Results

The examples presented in the previous section are very interesting. However, a quantitative

evaluation, in which the output of the detection system developed is compared with the region ground

truth (see Section 6.1), is also required. In this section the assessment of the performance of the

detection algorithm, with 5 fold cross validation, at the pixel and region levels, will be presented. The

methods used are described in Section 7.2.1.

7.6.2.A Pixel-based statistics

By assessing the performance of the detection algorithm on a pixel level it is possible to establish a

comparison between the area of the detected regions and the area of the segmentation performed by

the expert. The results presented compare the output of detection block before and after the validation

block. The statistics can be seen in Tables 7.7 and 7.8, respectively.

Table 7.7: Statistical results at a pixel level for the Network detection system before the validation step, forα = 15, T = 100 and topological features.

DetectedNot Pigment Network Pigment Network

Truth Not Pigment Network 95% 5%Pigment Network 43% 57%

Table 7.8: Statistical results at a pixel level for the Network detection system after the validation step, for α = 15,T = 100 and topological features.

DetectedNot Pigment Network Pigment Network

Truth Not Pigment Network 96% 4%Pigment Network 48% 52%

The results show that the algorithm has a SE = 57% and a SP = 95% before the validation and

a SE = 52% and a SP = 96% after the validation step. It is possible to see that the validation step

has a minor effect on the number of correctly classified not-network pixels. However, it reduces the

number of correctly classified pigment-network pixels around 5%, which means that the validation

step wrongly rejects some pigment network regions. By inspecting the values obtained it is possible

to see that the regions detected by the detection algorithm are smaller (smaller number of pixels) than

the ones from the ground truth, since has can be seen in the next subsection the algorithm does not

have many DF and, therefore, the low SE value is not mainly related with undetected regions. This

is not an unexpected result, since it is not possible to segment perfectly a pigment network region

(see example in Section 6.1). The segmentation will always include more pixels than the ones which

actually belong to pigment network.

67

7.6.2.B Region-based statistics

Another interesting quantitative evaluation is computing the number of regions which are correctly

detected, the ones which are false alarms and the regions which the algorithm misses. This is done

by comparing the output of the detection system with the lesion ground truth as described in Section

7.2.1 for region level statistics.

Tables 7.9 shows the results for the 5 fold cross validation before and after the validation step.

Table 7.9: Statistical results at region level for the network detection system before and after the validation step.The validation is performed with α = 15, T = 100 and topological features. CD - correct detection; FA - falsealarm; DF - detection failure.

Before Validation After ValidationCD 15 13FA 53 27DF 1 3

By inspecting the previous table it is possible to conclude that the algorithm presents a high num-

ber of CD and FA before the validation. This confirms the observations made in Chapter 5 (see

Section 5.4) in which was observed that the network detection block detects regions in excess, which

results in a small number of DF but also in a high number of FA. After the validation step it is possible

to observe that the number of FA decreases considerably. The number of CD decreases slightly and

the number of DF increases in the same proportion, which was expected based on the results of

both the AdaBoost (see table 7.5) and pixel validation(see table 7.8). By analyzing this results it is

possible to conclude that the validation step brings benefits to the final detection algorithm, in terms

of region statistics, since the number of wrongly rejected pigment network regions is very small and

it removes a considerable part of the FA (detection errors). In a larger scale it is possible to prove

that the detection algorithm proposed performs well in the dataset provided, achieving good network

detection scores.

By comparing this validation method with the pixel-based one it is possible to conclude that this

method is better to evaluate the performance of the region detection system, since it allows a more

viable comparison between the output of the detection system and the ground truth.

68

7.7 Lesion Classification Results

Table 7.10 shows the results of the lesion classification algorithm, using 5 fold cross validation

and the statistical method described in section 7.2.3. The algorithm obtains a final SP = 77% and

a SE = 78% in this dataset of clinical examples. These statistics prove that the detection system

developed achieves good detection scores and performs reasonably well in the dataset provided.

Table 7.10: Statistical results for lesion classification after validation, with α = 15, T = 100 and topologicalfeatures.

DetectedWithout Network With Network

Truth Without network 77% 23%With Network 22% 78%

In order to perform a final assessment of the benefits provided by the region validation (post-

processing) block a statistical study similar to the one described previously was applied to the output

of the network detection block. In this study, the classification is performed as described previously.

However, since there was no post-processing, all the regions detected inside a lesion that fulfilled

the area thresholding condition were considered. The results, obtained by the 5 fold cross validation

method, can be seen in Table 7.11.

Table 7.11: Statistical results for lesion classification before validation, with α = 15, T = 100 and topologicalfeatures.

DetectedWithout Network With Network

Truth Without network 63% 37%With Network 15% 85%

These results show that if no region validation block were applied, the algorithm would achieve a

SP = 63% and a SE = 85%. By comparing these results with the ones obtained previously (see table

7.10) it is possible to conclude that by applying the post-processing block it is possible to increase the

SP by 14%. The SE decreases around 7%. The decrease in the SE value maybe due to misclassified

pigment network regions that will be excluded from the final region R and, therefore, reduce the area

ratio. Another explanation can be the fact that this statistic was performed without performing a pre-

classification of regions as the one described in section 6.3 and, therefore, without discarding regions.

Since this pre-classification is performed before the post-processing block, this might as well explain

the reduction in the area ratio of a given lesion and consequent misclassification. Despite reducing the

number of correctly classified lesions with pigment network, it is considered that the post-processing

block brings some benefits to the final automatic system, since a relevant reduction of the SP value is

achieved and a comparison with a ground truth is performed, which suggests that the results obtained

for lesions with pigment network have a high degree of confidence.

69

70

8Conclusions and Future Work

Contents8.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 728.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

71

8.1 Conclusions

This thesis proposes an automatic algorithm for pigment network detection in dermoscopy images.

The algorithm involves several processing steps: pre-processing, network enhancement and detec-

tion, region validation and lesion classification. In each one of these blocks new approaches were

developed, trying to improve the performance of the whole system.

The pre-processing block works by detecting and removing reflection and hair artifacts from der-

moscopy images. The results obtained for this block were quite promising and suggest that this block

can be used in other automatic algorithms which try to extract dermoscopic features, different from

pigment network, from dermoscopy images or even in automatic algorithms which perform lesion

segmentation or lesion classification.

The network enhancement and detection blocks explore line color and geometry using a bank

of directional filters and a connected component analysis. By combining these two blocks with the

region validation block it is possible to achieve good detection scores both for region detection (CD =

13, FA = 27 and DF = 3) and lesion classification (SE = 78% and SP = 77%). This algorithm

is also capable of extracting the mesh of pigment network, which can help dermatologists when

performing a diagnosis. These results prove that the automatic algorithm proposed is an useful tool

in a dermoscopy analysis system.

8.2 Future work

In this section future work suggestions will be discussed.

Improve the region validation step results - The results obtained in this thesis for the validation

demonstrate that selecting the best features and training a classifier to perform a binary dis-

tinction between pigment network and not pigment network regions is a challenging problem.

The results obtained were far from perfect, which suggests that future work should regard the

improvement of these results. This can be done in two different ways:

• New combination of features: Try to combine the topological features already used, with

only one or two image features like HoG or variance, in order to provide some information

regarding the intensity of the region. Another approach could be the use of features which

characterize the color of the region instead of only the intensity, i.e., extracting features

from the three color channels R,G and B. By working only with the gray scale image it is

possible that some information regarding pigment network regions is being lost. Therefore,

by using color features this problem could be overcome.

• Different classifier: AdaBoost is a good classifier, since it is capable of performing a

feature selecting process. However, different classifiers like ANN or SVM could be tested

as well, and their results compared with the ones from AdaBoost,

Detecting other structures - Directional filters are matched filters. Therefore, their impulse re-

sponse can be designed to adapt to a specific shape and so, a bank of directional filters with

72

an impulse response which matches the shape of dots or globules can be used in the future

to try to detect these dermoscopic features in dermoscopy images. These filters could also be

integrated in the pre-processing step of the automatic system developed in this thesis to help

distinguishing the dots pattern from pigment network, since one of the detection errors is related

with the presence of this pattern.

Larger database - The algorithm proposed achieves good detection scores in the dataset of clinical

cases used. However, this dataset is small (only 57 images). In order to improve the results

obtained an to analyze their reproducibility this algorithm must be tested in a larger dataset,

with more lesions with pigment network, since a better learning rule could be achieved if more

positive examples are present.

Comparison with other works - This thesis proposes a new approach for pigment network detec-

tion, different from the ones found in literature. In the future, this algorithm should be compared

with other algorithms for pigment network detection, since a comparison between the several

methods will allow a better realization of the value of the algorithm proposed, its assets and

weakest points. The comparison should be performed with the same dataset that was used in

this thesis or with a larger dataset.

Distinguishing between typical and atypical pigment network - The detection of pigment network

is important for two reasons. First, since it is an hallmark of melanocytic lesions, its detection

can help in a preliminary classification of lesions as melanocytic or not. Secondly, an atypical

pigment network is an unmistakable sign of melanoma. The algorithm proposed does not dis-

tinguish between typical and atypical pigment network. Therefore, future work should also focus

on the distinction between typical and atypical since this will increase the medical value of the

algorithm.

73

74

Bibliography

[1] Dermnet. http://www.dermnet.com/.

[2] Dermoscopy atlas. http://www.dermoscopyatlas.com.

[3] Dermoscopy tutorial. http://www.dermoscopy.org/atlas/base.htm.

[4] Q. Abbas, M. Celebi, and I. Garcıa. Hair removal methods: A comparative study for dermoscopy

images. Biomedical Signal Processing and Control, 2001.

[5] M. Anantha, RH. Moss, and WV.. Stoecker. Detection of pigment network in dermatoscopy

images using texture analysis. Computarized Medical Imaging and Graphics, 28(5):225–234,

2004.

[6] G. Argenziano, G. Fabbrocini, P. Carli, V. De Giorgi, E. Sammarco, and M. Delfino. Epilumi-

nescence microscopy for the diagnosis of doubtful melanocytic skin lesions. comparison of the

ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis. Archives of

Dermatology, (134):1563–1570, 1998.

[7] G. Argenziano, H.P. Soyer, and S. Chimenti. Dermoscopy of pigmented skin lesions: results of a

consensus meeting via the internet. Journal of American Academy of Dermatology, 148(5):375–

389, 2003.

[8] C. Barata, J.S. Marques, and J. Rozeira. Detecting the pigment network in dermoscopy images:

a directional approach. In IEEE, editor, 33rd Annual International Conference on the IEEE EMBS,

pages 5120–5123, September 2011.

[9] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester. Image inpainting. SIGGRAPH, pages

417–424, 2000.

[10] G. Betta, G. Di Leo, G. Fabbrocini, A. Paolillo, and P. Sommella. Dermoscopic image-analysis

system: estimation of atypical pigment network and atypical vascular pattern. In International

Workshop on Medical Measurement and Applications, April 2006.

[11] M. Binder, M. Puespoeck-Schwartz, A. Steiner, H. Kittler, K. Wolff, M. Muellner, and H. Pe-

hamberger. Epiluminescence microscopy of small pigmented skin lesions : short-term formal

training improves the diagnosis performance of dermatologists. Journal American Academy of

Dermatology, pages 197–202, 1997.

75

[12] A. Blum, H. Luedtke, U. Ellwanger, R. Schwabe, G. Rassner, and C. Garbe. Digital image analy-

sis for diagnosis of cutaneous melanoma. development of a highly effective computer algorithm

based on analysis of 837 melanocytic lesions. British Journal of Dermatology, 151:1029–1038,

2004.

[13] B. Caputo, V. Panichelli, and G.E Gigante. Towards a quantitative analysis of skin lesions. Studies

in health technology and informatics, 90:509–513, 2002.

[14] N. Cascinelli, M. Ferrario, T. Tonelli, and E. Leo. A possible new tool for clinical diagnosis of

melanoma: The computer. Journal American Academy of Dermatology, 16(2):361–367, 1987.

[15] M.E. Celebi, Kingravim H., B. Uddin, H. Iyatomi, Y. Aslandogan, W. Stoecker, and R. Moss. A

methodological approach to the classification of dermoscopy images. Computerized Medical

Imaging and Graphics, 31:362–373, 2007.

[16] M.E. Celebi, H. Iyatomi, W. Stoecker, R.H. Moss, H. Rabinovitz, G. , and H.P. Soyer. Automatic

detection of blue-white veil and related structures in dermoscopy images. Computerized Medical

Imaging and Graphics, 32:670–677, 2008.

[17] G. Daugman. Uncertainty relation for resolution in space, spatial frequency, and orientation

optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America,

(7), 1985.

[18] R. Duda, P. Hart, and D. Stork. Pattern Classification. Wiley, 2001.

[19] G. Fabbrocini, G. Betta, G. Di Leo, C. Liguori, and et al. Epilluminescence image processing for

melanocytic skin lesion diagnosis based on 7-point check list: a preliminary discussion on three

parameters. The open dermatology journal, 4:110–115, 2010.

[20] M. Fleming, C. Steger, J. Zhang, J. Gao, A. Cognetta, I. Pollak, and C. Dyer. Techniques for

a structural analysis of dermatoscopic imagery. Computarized Medical Imaging and Graphics,

(5):375–389, 1998.

[21] I. Fogel and D. Sagi. Gabor filters as texture discriminator. Biological Cybernetics, (61):103–113,

1989.

[22] D. Gabor. Theory of communication. J. IEE Lond., 93:429–457, 1946.

[23] H. Ganster, A. Pinz, E. Wildling, M. Binder, and H. Kittler. Automated melanoma recognition.

IEEE transaction on Medical Imaging, 20(3):233–239, 2001.

[24] R. Gonzalez and R. Woods. Digital Image Processing. Prentice Hall, 2 edition, 2002.

[25] R. Gonzalez, R. Woods, and S. Eddins. Digital Image Processing using MATLAB. Gatesmark

Publishing, 2009.

76

[26] C. Grana, R. Cucchiara, G. Pellacani, and S. Seidenari. Line detection and texture characteri-

zation of network patterns. In In ICPR’06: Proceedings of the 18th International Conference on

pattern Recognition, 2006.

[27] C. Grigorescu. Contour detection based on nonclassical receptive field inhibition. IEEE

transactions on image processing, 7, 2003.

[28] K. Hoffman, T. Gambichler, A. Rick, M. Kreutz, M. Anschuetz, and T. Grunendick. Diagnostic

and neural analysis of skin cancer (DANAOS, a multicentre study for collection and computer-

aided analysis of data from pigment skin lesions using digital dermoscopy. Britain Journal of

Dermatology, 149(4):801–809, 2003.

[29] D.H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture

in the cat’s visual cortex. Journal of Physiology, 160:106–154, 1962.

[30] H. Iyatomi, H. Oka, M.E. Celebi, M. Hashimoto, M. Hagiwara, M. Tanaka, and K. Ogawa. An im-

proved internet-based melanoma screening system with dermatologist-like tumor area extraction

algorithm. Computerized Medical Imaging and Graphics, 32:566–579, 2008.

[31] J. P. Jone and L. A. Palmer. An evaluation of the two-dimensional gabor filter model of simple

receptive fields in cat striate cortex. Journal of Neurophysiology, 58(6):1233–1258, 1987.

[32] J. Kirk. Count loops in a nework. http://www.mathworks.com/matlabcentral/fxfiles/10722/

1/content/html/runloopshtml.html, 2006.

[33] T. Lee, V. Ng, R. Gallagher, A. Coldman, and D. McLean. Dullrazor:a software approach to hair

removal from images. Comp. Biol. Med., 27(6):533–543, 1997.

[34] TK. Lee, D. McLean, and MS. Atkins. Irregularity index: a new border irregularity measure for

cutaneous melanocytic lesions. Medical Image Analysis, 7(1):47–64, 2003.

[35] J. Mayer. Systematic review of the diagnostic accuracy of dermatoscopy in detecting malignant

melanoma. Medical Journal Australia, (167):206–210, 1997.

[36] S.W. Menzies, C. Ingvar, K.A. Crotty, and W.H. McCarthy. Frequency and morphologic char-

acteristics of invasive melanomas lacking specific surface microscopic features. Archives of

Dermatology, (132):1178–1182, 1996.

[37] J. C. Nascimento and J. S. Marques. Novel metrics for performance evaluation of object detection

algorithms. In IEEE, editor, Sixth IEEE International Workshop on Performance Evaluation of

Tracking and Surveillance, May 2004.

[38] H. Oka, M. Hashimoto, and M. Tanaka. Internet based program for automatic discrimination of

dermoscopic images between melanoma and clark nevi. British Journal of Dermatology, 150(5),

2004.

77

[39] H. Pehamberger, A. Steiner, and W. Wolff. In vivo epiluminescence microscopy of pigmented

skin lesions. I. pattern analysis of pigmented skin lesions. Journal American Academy of

Dermatology, (17):571–583, 1987.

[40] G. Pellacani, C. Grana, R. Cucchiara, and S. Seidenari. Automated extraction of dark areas in

surface microscopy melanocytic lesion images. Dermatology, 208(1):21–26, 2003.

[41] P. Rubegni, G. Cevenini, M. Burroni, R. Perotti, G. Dell’Eva, P. Sbano, and C. Miracco. Automated

diagnosis of pigment skin lesions. International Journal of Cancer, 101:576–580, 2002.

[42] M. Sadeghi, M. Razmara, T. Lee, and M. Stella Atkins. A novel method for detection of pigment

network in dermoscopic imagesusing graphs. Computerized Medical Imaging and Graphics,

(35):137–143, 2011.

[43] P. Schmid-Saugeon, J. Guillod, and J. Thiran. Towards a computer-aided diagnosis system for

pigmented skin lesions. Computerized Medical Imaging Processing and Graphics, 27:65–78,

2003.

[44] M. Silveira, J. Nascimento, J. Marques, A. Marcal, T. Mendonca, S. Yamauchi, J. Maeda, and

J. Rozeira. Comparison of segmentation methods for melanoma diagnosis in dermoscopy im-

ages. IEEE Journal of selected topics in signal processing, (1), 2009.

[45] N. Situ, X. Yuan, J. Chen, and G. Zouridakis. Malignant melanoma detection by bag-of-features

classification. In EMBC, 2008.

[46] C. Steger. An unbiased detector of curvilinear structures. IEEE Trans Pattern Analysis Machine

Intell, 20(2), 1998.

[47] W.V. Stoecker, K. Gupta, R.J. Stanley, and et al. Detection of asymmetric blotches in dermoscopy

images of malignant melanomas using relative color. Skin Research and Technology, 11(3):179–

184, 2005.

[48] W.V. Stoecker, W.V. Lee, and R.H. Moss. Automatic detection of asymmetry in in skin tumors.

Computerized Medical Imaging and Graphics, 16(3):191–197, 1992.

[49] W.V. Stoecker, M. Wronkiewicz, R. Chowdhury, and et al. Detection of granularity in dermoscopy

images of malignant melanoma using color and texture features. Computerized Medical Imaging

and Graphics, 35:144–147, 2011.

[50] W. Stolz, A. Riemann, and A. B. Cognetta. ABCD rule of dermatoscopy: a new practical method

for early recognition of malignant melanoma. European Journal Dermatology, (4):521–527,

1994.

[51] G. Turin. An introduction to matched filters. IRE TRANSACTIONS ON INFORMATION THEORY,

pages 311–329, 1960.

[52] R. Turner. Texture discrimination by gabor functions. Biological Cybernetics, (55):71–82, 1986.

78

[53] H.L. Van Trees. Detection, Estimation and Modulation Theory. John Wiley & Sons, Inc., 2001.

[54] P. Viola and M. Jones. Robust real-time face detection. International Journal of Computer Vision,

57(2):137–154, 2004.

[55] S. Yoshino, T. Tanaka, M. Tanaka, and H. Oka. Application of morphology for the detection of

dots in tumors. In SICE Annual conference in Saporo, 2004.

1

AInpaiting Algorithm

A-1

Consider I0(x, y) as the original gray image that has a region Ω to be inpainted. The inpainting

algorithm will create a family of In(x, y) images, such that I0(x, y) = I0(x, y) and limn→∞ In(x, y) =

IR(x, y), where IR is the final output of the algorithm. This algorithm can then be written as

In+1(x, y) = In(x, y) +4tInt (x, y), ∀(x, y) ∈ Ω (A.1)

where n denotes the inpainting ”time”,(x, y) are the pixel coordinates, 4t is the rate of improvement

and Int (x, y) stands for the update of the image In. With this equation, the image In+1 is an improved

version of In, with the ”improvement” given by Int (x, y)

Int (x, y) = ∇Ln(x, y) · ~Nn(x, y) (A.2)

where, considering that ~Ln(x, y) is the information that has to be propagated from outside Ω into Ω,

∇Ln(x, y) is a measure of the change in that information and ~Nn(x, y) is the propagation direction.

The algorithm will converge when∇Ln(x, y)· ~Nn(x, y) = 0, i.e., when the information has been exactly

propagated in the direction ~N .

The propagation must be a smooth process, so ~Ln(x, y) must be an image smoothness estimator.

Based on that, ~Ln(x, y) will be described by a simple Laplacian

~Ln(x, y) =∂2In(x, y)

∂x2+∂2In(x, y)

∂y2(A.3)

The propagation direction ~Nn(x, y) changes with each iteration and it is given by

~Nn(x, y) = ∇⊥In(x, y) (A.4)

~Nn(x, y) is then defined as a time varying estimation of the isotopes direction field.

A-2