ms thesis-final-defense-presentation

Computer Assisted Screening of Microcalcifications in Digitized Mammogram for Early Detection of Breast CancerThesis Presentation

Nashid AlamRegistration No: 2012321028annanya_cse@yahoo.co.uk

Supervisor: Prof. Dr. Mohammed Jahirul Islam

Department of Computer Science and EngineeringShahjalal University of Science and Technology

Driving research for better breast cancer treatment “The best protection is early detection”

40-0.1

5; 0.5; 0.7854

Thursday, December 23, 2015

Introduction

Breast cancer:The most devastating and deadly diseases for women.

o Computer aided detection (CADe) o Computer aided diagnosis (CADx) systems

We will emphasis on :

Background Interest

Interest comes from two primary backgrounds

1. Improvement of pictorial information- - For Human Perception

How can an image/video be made more aesthetically pleasing

How can an image/video be enhanced to facilitate:extraction of useful information

Background Interest

Interest comes from two primary backgrounds

2. Processing of data for:Autonomous machine perception- Machine Vision

Micro-calcification

Mammography

Mammogram

Micro-calcification

Background knowledge

Micro-calcification

Micro-calcifications :- Tiny deposits of calcium- May be benign or malignant- A first cue of cancer.

Position:1. Can be scattered throughout the mammary gland, or 2. Occur in clusters.(diameters from some µm up to approximately 200 µm.)3. Considered regions of high frequency.

Micro-calcification

They are caused by a number of reasons:

1. Aging –The majority of diagnoses are made in women over 50

2. Genetic –Involving the BRCA1 (breast cancer 1, early onset) and

BRCA2 (breast cancer 2, early onset) genes

Micro-calcifications Pattern Determines :The future course of the action-

I. Whether it be further investigatory techniques (as part of the triple assessment), or

II. More regular screening

Mammography

Mammography Machine

Mammography

USE:I. Viewing x-ray imageII. Manipulate X-ray image on a computer screen

Mammography :

Process of using low-energyx-rays to examine the human breast

Used as a diagnostic and a screening tool.

The goal of mammography :The early detection of breast cancer

Mammography Machine

Mammogram

mdb226.jpg

Mammogram

Mammogram:An x-ray picture of the breast

Use:To look for changes that are not normal.

Result Archive:The results are recorded:

1. On x-ray film or 2.Directly into a computer

mdb226.jpg

Literature Review

To detect micro-calcifications:-A number of methods have been proposed

These include:

Global and local thresholding

Statistical approaches

Neural networks

Fuzzy logic

Thresholding of wavelet coefficients and related techniques.

Literature Review

Focus:Preprocessing Techniques of Mammogram:

Goals:- Pectoral mussel identification- Noise removal- Image enhancement

-No method gives full satisfaction andclinically acceptable results.

Drawbacks:

Literature Review

local range modification algorithmIntegrated wavelets form MC modelStein's thresholding [7]for denoisingUse Contourlets

Method used:Watershed transformationBoundary based methodHybrid techniquesThresholding techniques

Heinlein et.al(2003) [1]Zhibo et.al.(2007) [2]Papadopoulus et al. (2008) [3]

Razzi et al.(2009) [4]Pronoj et al.(2011) [5]Camilus et al.(2011) [6]

Focus:Local feature extraction of Mammogram:

Pal et al.(2008) [8]Yu et al. (2010) [9]

Oliver et al.(2010) [10]

Goals:

- Detect microcalcification and MC cluster

- Only deals with MC morphology

-Position of microcalcifications(Take into account)

-To segment mammogram:Only salient fracture are computed

Drawbacks:

Literature Review

Method used:Inspect local neighborhood of each MCWeighted density functionFuzzy Shell Clustering

Training stage: Pixel-based boosting classifierMulti-layered perception networkBack propagated neural network

Balakumaran et.al.(2010) [11] Oliver et al.(2012) [12]Zhang et.al.(2013)[13]

Literature Review

Focus:Wavelet based Techniques

Wang et.al.(1989) [14]Daubechies I.(1992) [15]Strickland et.at (1996) [16]Papadopoulus et al. (2008) [3]

Goals:

Method used:two-stage decomposition wavelet filteringdiscrete wavelet linear stretching and shrinkage algorithm.low-frequency subbands are discardedbiorthogonal filter bank used

Drawbacks:

-Cluster was considered:if more then 3 microcalcifications

were detected in a 1cm2 area

- Detect microcalcification and MC cluster

Razzi et al.(2009) [4]Yu et al.(2010) [9]Balakumaran et.al.(2010) [11]Zhang et.al.(2013) [13]

Literature Review

Focus:Analysis of large massesinstead of microcalcifications

Zhibo et.al.(2007)[2]Lu et.al.(2013) [17]

Goals:Drawbacks:

Method used:

Mass Detection

Multiscale regularized reconstructionHybrid Image Filtering MethodNoise regularization in DBT reconstructionUse Contourlets

- Detecting subtle mass lesionsin Digital breast tomosynthesis (DBT)

- Only detect large mass

Digital BreastTomosynthesis captures

PHOTO COURTESY :http://www.itnonline.com/article/trends-breast-imaginghttp://www.hoag.org/Specialty/Breast-Program/Pages/breast-screening/screening-types/Tomosynthesis.aspx

Literature Review

Focus:Detect /Classify mammograms

Fatemeh et.al.(2007) [18]

Goals:

Drawbacks:

Method used:

Automatic mass classification

Contourlets Transform

Does not give full satisfaction andclinically acceptable results.

PHOTO COURTESY :https://www.youtube.com/watch?v=kRwKO5k6pi

Mammogram

Literature Review

Focus:

Template matching algorithm

Leeuw et.al.(2014) [7]

Goals: Drawbacks:

Method used:

Detect microcalcifications in breast specimens

Phase derivative to detect microcalcifications

Used MRI instead of mammogram

Breast MRIBreast MRI Machine

PHOTO COURTESY :http://www.leememorial.org/mainlanding/Breast_mri.asp

Literature Review

Focus:

Goals:Insertion of simulated microcalcification clusters:

- In a software breast phantom

PHOTO COURTESY :http://www.math.umaine.edu/~compumaine/index.html

Left: Cluster microcalcification in breast tissue. Right: Simulated cluster microcalcification.

-Algorithm developed as part ofa virtual clinical trial (VCT) :

-Simulation of breast anatomy, - Mechanical compression- Image acquisition- Image processing- Image displaying and interpretation.

Shankla et.al.(2014)[19]

Problem Statement

Burdensome Task Of Radiologist : Eye fatigue:

-Huge volume of images-Detection accuracy rate tends to decrease

Non-systematic search patterns of humansPerformance gap between :

Specialized breast imagers andgeneral radiologists

Interpretational Errors:Similar characteristics:

Abnormal and normal microcalcification

Problem Statement

Reason behind the problem( In real life):

The signs of breast cancer are:

Masses CalcificationsTumorLesionLump

Individual Research Areas

Problem Statement

Motivation to the Research

Motivation to the research: Goal

Better Cancer Survival Rates(Facilitate Early Detection ).

Provide “second opinion” : Computerized decisionsupport systems

Fast,Reliable, andCost-effective

Overcome:The development of breast cancer

Challenges

Develop a logistic model:

Feature extraction Challenge:

-To determine the likelihood of CANCEROUS AREA -- From the image values of mammograms

Challenge:Occur in clusters

The clusters may vary in size from 0.05mm to 1mm in diameter.

Variation in signal intensity and contrast.May located in dense tissue

Difficult to detect.

Challenges

Materials and Tools

Matlab 2014

Database: mini-MIAS

Database: mini-MIAS databasehttp://peipa.essex.ac.uk/pix/mias/

Class of Abnormality

Severity of Abnormality

The Location of The

Center of The

Abnormality and It’s

Diameter.

1 Calcification(25)

1.Benign(Calc-12)

2 Circumscribed Masses

3 Speculated Masses

4 Ill-defined Masses

5 Architectural Distortion

2.Malignant(Cancerous)

(Calc-13)

6 Asymmetry

7Normal

mdb223.jpg mdb226.jpg

mdb239.jpg mdb249.jpg

Figure01:X-ray image form mini-MIAS database

Database: Mini-MIAS Databasehttp://peipa.essex.ac.uk/pix/mias/

Mammography Image Analysis Society (MIAS) -An organization of UK research groups

• Consists of 322 images-- Contains left and right breast images for 161 patients

• Every image is 1024 X 1024 pixels in size

• Represents each pixel with an 8-bit word

• Reduced in resolution(Is not good enough for MC to be detectable)

•Very Poor Quality with .jpg compression effects(Original MIAS doesn’t have such artifacts)

Mini-MIAS Database

Mammography Image Analysis Society (MIAS) -An organization of UK research groups

Database: http://peipa.essex.ac.uk/pix/mias/

http://see.xidian.edu.cn/vipsl/database_Mammo.html

Plan of Action

Where Are We? Our Current Research Stage

Thesis SemesterM-3

Chart 01: Gantt Chart of this M.Sc thesis Showing the duration of task against the progression of time

Where Are We? Our Current Research Stage

Thesis SemesterM-3

Schematic representation of the system

Removing Pectoral MuscleAnd

X-ray Label

X-ray Label Removing Finding The Big BLOB

The types X-ray Label:High Intensity Rectangular LabelLow Intensity LabelTape Artifacts

X-ray Label Removing

1. Histogram equalization of the original X-ray image

2. Adjust image contrast

3. Apply Otsu's Thresholding Method [20] and

find bi-level the image which has several blobs in it.

4. Finding the Largest blob (Bwlargest.bolb)

5. Hole filling within the blob region

6. Keep the true pixel value covering only the area of largest blob and discard other features from the original image

7. X-ray label is successfully removed

Plan of Action

[20] Otsu, N., "A Threshold Selection Method from Gray-Level Histograms," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 9, No. 1, 1979, pp. 62-66.

To Achieve The Desired Final Result:Apply:

A Range Of Techniques on original image

1.Original image

2.HistogramEqualization

3.Contrast Image

4.Binary Image

mdb239.jpg

Combining Range of techniques

J = histeq(I); %histogram equalization

contrast_image = imadjust(J, stretchlim(J), [0 1]); %high contrast image

%Apply Thresholding to the Image level = graythresh(contrast_image);

%GRAYTHRESH Global image threshold using %Otsu's methodbw_image = im2bw(contrast_image, level);%getting binary image

5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

Combining Range of techniquesX-ray Label Removing

Result image(Label Removed)

Original image

Compare the original and final image

Experimental results

X-ray Label Removing1.Original image

2.HistogramEqualization 3.Contrast Image 4.Binary Image 5.Finding biggest blob

mdb212.jpg

mdb214.jpg

mdb218.jpg

mdb219.jpg

Benign

X-ray Label Removing Benign1.Original image

mdb222.jpg

mdb223.jpg

mdb226jpg

mdb227jpg

X-ray Label Removing Benign1.Original image

mdb226.jpg

mdb240.jpg

mdb248.jpg

mdb252.jpg

X-ray Label Removing Malignant1.Original image

mdb209.jpg

mdb211.jpg

mdb213.jpg

mdb216.jpg

mdb231.jpg

mdb233.jpg

mdb238.jpg

mdb239.jpg

mdb241.jpg

mdb245.jpg

mdb249.jpg

mdb253.jpg

mdb256.jpg

Successful

Finally!

Removing pectoral muscleKeeping fatty tissues and ligaments

mdb212.jpg(a)Main Image (b)Result Image

mdb213.jpg(a)Main Image (b)Pectoral Muscle

mdb214.jpg

Main Image

Result Image

o Fat t y t i s s ue are ao Duc to Lobul e so Si nuso l i gam e nt s

Extraction of ROIRemoving pectoral muscle

Why removing pectoral muscle?

o Pectoral muscle will never contain micro-calcification

o Less Computational Time And Cost-Operation on small image area

Existence of micro-calcification:

Edge Detection of pectoral muscleRemoving pectoral muscle

Points to be noted :

-Pectoral muscle a Triangular areamdb212.jpg

mdb214.jpg

Based on this point: Moving on towards solution

mdb209.jpg

(2)Binary Image(1)Original Image

Triangle Detection of pectoral muscleRemoving pectoral muscle

1. Find the triangular area of the pectoral muscle region

I. Finding white seeding pointII. Finding the 1st black point of 1st row after getting a white seeding pointIII. Draw a horizontal line in these two points.IV. finding the 1st black point of 1st column after getting a white seeding pointV. Draw a vertical line and angular line.

2. Making all the pixels black(zero)resides in the pectoral muscle area

Triangle Detection of pectoral muscle

Visualization in next slide

Approach-03(Triangle Detection of pectoral muscle):

mdb212.jpg1.Original image

2.Contrast stretching

3.Binary of contrast image

stratching_in_range=uint8(imadjust(I,[0.01 0.7],[1 0]));

BW=~stratching_in_range;

Approach-03(Triangle Detection of pectoral muscle):

4.Triangle

5.Triangle Filled

6.muscle removed

Experimental results

Removing pectoral muscleApproach-03(Triangle Detection of pectoral muscle):

Triangle Detection of pectoral muscle

mdb212.jpg

mdb214.jpg

1.Original image 2.Contrast stretching 3.Binary of contrast image 4.Triangle

mdb240.jpg

mdb248.jpg

5.Triangle Filled 6.muscle removed

Class: Benign

mdb222.jpg

mdb226.jpg

mdb227.jpg

2.Contrast stretching1.Original image 3.Binary of contrast image 4.Triangle

Problems faced

The triangle does not always indicates the proper pectoral muscle area.Reason:

Class: Benign

Artifacts in mammogram

2.Contrast stretching1.Original image

3.Binary of contrast image 4.Triangle

Problems faced:

Defects in mammogram (Vertical Stripe Missing)

mdb227.jpg

Class: Benign

mdb223.jpg

2.Contrast stretching1.Original image 3.Binary of contrast image 4.Triangle 5.Triangle Filled 6.muscle removed

Problems faced:

Defects in mammogram (Horizontal Stripe Missing)Solution: Replicate the 2nd and 3rd row)

Class: Benign

1.Original image

2.Contrast stretching

3.Binary of contrast image

4.Triangle

5.Triangle Filled

6.muscle removed

Class: Malignant

mdb256.pg

mdb212.jpg

mdb214.jpg

1.Original image 2.Contrast stretching 3.Binary of contrast image 4.Triangle

mdb240.jpg

mdb248.jpg

Class: Benign

Successful

Pectoral Muscle Removing

Finally!

Improved Computer Assisted Screening

Enhancement of digitized mammogram

MAIN NOVELTY

Input image

BandpassDirectionalsubbands

Based on the classical approach used in transform methods for image processing.

1. Input mammogram

2. Forward CT

3. Subband Processing

4. Inverse CT

5. Enhanced Mammogram

Schematic representation of the system

Contourlet transformation

Implementation Based On :

• A Laplacian Pyramid decomposition followed by -

• Directional filter banks applied on each band pass sub-band.

The Result Extracts:-Geometric information of images.

Details in upcoming slides

Main Novelty

Input image

Why Contourlet?

•Decompose the mammographic image:-Into directional components:

To easily capture the geometry of the image features.

Target

Enhancement of the Directional Subbands

The Contourlet TransformLaplacian Pyramid: 3 level

DecompositionFrequency partitioning of a directional filter bank

Decomposition level l=3

The real wedge-shape frequency band is 23=8.

horizontal directions are corresponded by sub-bands 0-3

Vertical directions are represented by sub-bands 4-7

Decomposition

Laplacian Pyramid Level-1

8 Direction

4 Direction

(mdb252.jpg)

Decomposition

Wedge-shape frequency band is 23=8.

Horizontal directions are corresponded by sub-bands 0-3

(1) sub-band 0

(2) sub-band 1

(3) sub-band 2(4) sub-band 3

Contourlet coefficient at level 4

DecompositionContourlet coefficient at level 4

Wedge-shape frequency band is 23=8.

(5) sub-band 4

(6) sub-band 5

(7) sub-band 6

(8) sub-band 7

Decomposition

(a) Main Image(mdb252.jpg)

(b) Enhanced Image(Average in all 8 direction)

(a) Main image(Toy Image)

Contourlet Transform Example

(b) Horizontal Direction

(c) Vertical Direction

Directional filter banks: Horizontal and Vertical

Contourlet Transform ExampleDirectional filter banks

Horizontal directions are corresponded by sub-bands 0-3

(1) sub-band 0

(2) sub-band 1

(3) sub-band 2

(4) sub-band 3

Contourlet Transform ExampleDirectional filter banks

(5) sub-band 4

(6) sub-band 5

(7) sub-band 6

(8) sub-band 7

Input image

Plan-of-Action

For microcalcifications enhancement :

We use-The Contourlet Transform(CT) [21]

The Prewitt Filter.

21. Da Cunha A. L., Zhou J. and Do M. N,: The Nonsubsampled Contourlet Transform: Theory, Design, and Applications, IEEE Transactions on Image Processing,vol. 15, (2006) pp. 3089-3101

Art-of-Action

An edge Prewitt filter to enhance the directional structures

in the image.

Contourlet transform allows decomposing the image in

multidirectional and multiscale subbands[22].

22. Laine A.F., Schuler S., Fan J., Huda W.: Mammographic feature enhancement by multiscale analysis, IEEE Transactions on Medical Imaging, 1994, vol. 13, no. 4,(1994) pp. 7250-7260

This allows finding • A better set of edges,• Recovering an enhanced mammogram with better visual characteristics.

Microcalcifications have a very small size a denoising stage is not implemented

in order to preserve the integrity of the injuries.

Decompose the digital mammogram

Using Contourlet transform

(b) Enhanced image(mdb238.jpg)

(a) Original image (mdb238.jpg)

The Contourlet Transform

The CT is implemented by:Laplacian pyramid followed by directional filter banks (Fig-01)

Input image

Figure 01: Structure of the Laplacian pyramid together with the directional filter bank

The concept of wavelet:University of Heidelburg

The CASCADE STRUCTURE allows:- The multiscale and

directional decomposition to be independent

- Makes possible to:Decompose each scale into

any arbitrary power of two's number of directions(4,8,16…)

Figure 01

Details ………….

Decomposes The Image Into Several Directional Subbands And Multiple Scales

Figure 02: (a)Structure of the Laplacian pyramid together with the directional filter bank(b) frequency partitioning by the contourlet transform(c) Decomposition levels and directions.

(a) (b)

Input image

Details….

DenoteEach subband by yi,j

Wherei =decomposition level and J=direction

The Contourlet TransformDecomposes The Image Into Several Directional Subbands And Multiple Scales

The processing of an image consists on:-Applying a function to enhance the regions of

interest.

In multiscale analysis:

Calculating function f for each subband :-To emphasize the features of interest-In order to get a new set y' of enhanced subbands:

Each of the resulting enhanced subbands can be expressed using equation 1.

)(', , jiyfjiy = ………………..(1)

-After the enhanced subbands are obtained, the inverse transform is performed to obtain an enhanced image.

Denote

Each subband by yi,jWherei =decomposition level and J=direction Details….

Details….

The directional subbands are enhanced using equation 2.

=)( , jiyf)2,1(

,1 nnWjiy

)2,1(,2 nnWjiy

If bi,j(n1,n2)=0

If bi,j(n1,n2)=1………..(2)

Denote

Each subband by yi,jWherei =decomposition level and J=direction

W1= weight factors for detecting the surrounding tissueW2= weight factors for detecting microcalcifications

(n1,n2) are the spatial coordinates.

bi;j = a binary image containing the edges of the subband

Weight and threshold selection techniques are presented on upcoming slides

The directional subbands are enhanced using equation 2.

=)( , jiyf)2,1(

,1 nnWjiy

)2,1(,2 nnWjiy

If bi,j(n1,n2)=0

If bi,j(n1,n2)=1………..(2)

Binary edge image bi,j is obtained :-by applying : Prewitt edge detector

-To detect edges on each directional subband.

In order to obtain a binary image:A threshold Ti,j for each subband is calculated.

Details….

Weight and threshold selection techniques are presented on upcoming slides

Threshold Selection

Details….

The microcalcifications appear :

On each subband Over a very

homogeneous background.

Most of the transform coefficients:

-The coefficients corresponding to theinjuries are far from background value.

A conservative threshold of 3σi;j is selected:where σi;j is the standard deviation of the corresponding subband y I,j .

Weight Selection

Exhaustive tests:-Consist on evaluating subjectively a set of 322 different mammograms

-With Different combinations of values,

The weights W1, and W2 are determined:- as W1 = 3 σi;j and W2 = 4 σi;j

These weights are chosen to:keep the relationship W1 < W2:

-Because the W factor is a gain -More gain at the edges are wanted.

Experimental Results

Applying Contourlet Transformation Benign

Original image Enhanced image

Goal: Microcalcification Enhancement

mdb222.jpg

mdb223.jpg

mdb248.jpg

mdb252.jpg

mdb226.jpg

mdb227.jpg

mdb236.jpg

mdb240.jpg

Original image Enhanced image Original image Enhanced image

mdb218.jpgmdb219.jpg

Applying Contourlet Transformation MalignantGoal: Microcalcification Enhancement

mdb209.jpg

mdb211.jpg

mdb213.jpg

mdb231.jpg

mdb238.jpg

mdb239.jpg

mdb241.jpg

mdb249.jpg

mdb253.jpg

mdb256.jpg

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

mdb003.jpg

mdb004.jpg

mdb006.jpg

mdb007.jpg

mdb009.jpg

mdb018.jpg

mdb027.jpg

mdb033.jpg

mdb046.jpg

mdb056.jpg

mdb060.jpg

mdb066.jpg

mdb070.jpg

mdb073.jpg

mdb074.jpg

mdb076.jpg

mdb093.jpg

mdb096.jpg

mdb101.jpg

mdb012.jpg

mdb128.jpg

mdb137.jpg

mdb146.jpg

mdb154.jpg

mdb166.jpg

mdb169.jpg

mdb224.jpg

mdb225.jpg

mdb263.jpg

mdb294.jpg

mdb316.jpg

mdb320.jpg

Wavelet Transformation

Use Separable Transform

2D Wavelet Transform

Visualization

Label ofapproximation

HorizontalDetails

VerticalDetails

DiagonalDetails

VerticalDetails

DiagonalDetails

Decomposition at Label 4

Original image(with diagonal details areas indicated)

Diagonal Details

Vertical Details

Decomposition at Label 4

Original image(with Vertical details areas indicated)

1.Original Image(Malignent_mdb238) 2.Decomposition at Label 4

2.Decomposition at Label 1 3.Decomposition at Label 2 3.Decomposition at Label 3

1.Original Image(Malignent_mdb238) 2.Decomposition at Label 4

1.Original Image(Benign_mdb252) 2.Decomposition at Label 4

1.Original Image(Malignent_mdb253.jpg) 2.Decomposition at Label 4

Metrics: Quantitive Measurement

Metrics

To compare the ability of :Enhancement achieved by the proposed method

1. Measurement of distributed separation (MDS)2. Contrast enhancement of background against target (CEBT) and3. Entropy-based contrast enhancement of background against target (ECEBT) [23].

Measures used to compare:

23. Sameer S. and Keit B.: An Evaluation on Contrast Enhancement Techniques for Mammographic Breast Masses, IEEETransactions on Information Technology in Biomedicine, vol. 9, (2005) pp. 109-119

Metrics

1. Measurement of Distributed Separation (MDS)

Measures used to compare:

The MDS represents :How separated are the distributions of each mammogram

…………………………(3)MDS = |µucalcE -µtissueE |- |µucalc0 -µtissue0 |

µucalcE = Mean of the microcalcification region of the enhanced imageµucalc0 = Mean of the microcalcification region of the original image

µtissueE = Mean of the surrounding tissue of the enhanced imageµtissue0 = Mean of the surrounding tissue of the enhanced image

Defined by:

Where:

Metrics

2. Contrast enhancement of background against target (CEBT) Measures used to compare:

The CEBT Quantifies :The improvement in difference between the background and the target(MC).

…………………………(4)

0µucalcEµucalc

0µtissue0µucalc

EµtissueEµucalc

Defined by:

Where:

Eµucalcσ

0µucalcσ

= Standard deviations of the microcalcifications region in the enhanced image

= Standard deviations of the microcalcifications region in the original image

Metrics

3. Entropy-based contrast enhancement of background against target (ECEBT)Measures used to compare:

The ECEBT Measures :- An extension of the TBC metric- Based on the entropy of the regions rather

than in the standard deviations

Defined by:

Where:

…………………………(5)

0µucalcEµucalc

0µtissue0µucalc

EµtissueEµucalc

= Entropy of the microcalcifications region in the enhanced image

= Entropy of the microcalcifications region in the original image

Eµucalcζ

0µucalcε

MDS, CEBT and ECEBT metrics on the enhanced mammograms

CT Method DWT Method

MDS CEBT ECEBT MDS CEBT ECEBT0.853 0.477 0.852 0.153 0.078 0.555

0.818 0.330 0.810 0.094 0.052 0.382

1.000 1.000 1.000 0.210 0.092 0.512

0.905 0.322 0.920 1.000 0.077 1.000

0.936 0.380 0.935 0.038 0.074 0.473

0.948 0.293 0.947 0.469 0.075 0.847

0.665 0.410 0.639 0.369 0.082 0.823

0.740 0.352 0.730 0.340 0.074 0.726

0.944 0.469 0.494 0.479 0.095 0.834

0.931 0.691 0.936 0.479 0.000 0.000

0.693 0.500 0.718 0.258 0.081 0.682

0.916 0.395 0.914 0.796 0.079 0.900

Table 1. Decomposition levels and directions.

Mammogram

MDS Matrix

CT DWT

The proposed method gives higher results than the wavelet-based method.

Experimental Results Analysis

Mammogram

CEBT Matrix

CT DWT

Mammogram

ECEBT Matrix

CT DWT

Experimental Results AnalysisMesh plot of a ROI containing microcalcifications

(a)The original mammogram

(mdb252.bmp)

(b) The enhanced mammogram

using CT

(mdb238.bmp)

using CT

(mdb253.bmp)

using CT

More peaks corresponding to microcalcifications are enhanced

The background has a less magnitude with respect to the peaks:

-The microcalcifications are more visible.

Observation:

(a)Original image (b)CT method (c)The DWT Method

These regions contain :• Clusters of microcalcifications (target)• surrounding tissue (background).

For visualization purposes :The ROI in the original mammogram are marked with a square.

ACHIEVEMENT

Improved Computer Assisted screen of mammogram

Achievements!

Enhancement of MC in digitized mammogramfor diagnostic support system

Figure: Diagnostic support system

Suspected

Digital mammography systems :Presents images to the Radiologist with properly image processing applied.

Achievements!

(b) Enhanced image(mdb238.jpg)

(a) Original imageROI

(mdb238.jpg)

(a) Original imageWHOLE IMAGE (mdb238.jpg)

Digital mammography systems :Presents images to the Radiologist with properly image processing applied.

Hard to find MC Easy to find MC

Whilephysicians

interact with

The information in an image During interpretation process

Achievements!!

Enhancement of MC in digitized mammogram

With improved visual understanding, we can develop :

ways to further improve :o Decision making ando Provide better patient care

Improved Computer Assisted Screening

Goal Accomplished

Another Step Ahead..how about training a machine?

Dealing with Features

Why Feature Extraction?

Finding a feature:That has the most

discriminative information

The objective of feature selection:

Differs from its immediate surroundings by texture colorintensity

Fig: MC features (Extracted Using Human Visual Perception)

Finding a feature:That has the most

discriminative information

The objective of feature selection:

Differs from its immediate surroundings by texture colorintensity

Fig: MC (Irregular in shape and size)(Extracted Using Human Visual Perception)

MoreFeatures: Shape Size

Problems With MC Features:Irregular in shape and sizeNo definite patternLow Contrast -

Located in dense tissueHardly any color intensity variation

MC Feature

Fig: MC (Irregular in shape and size)(Extracted Using Human Visual Perception)

Why Feature Extraction? MC Feature

How radiologist deals with feature Detection/Recognition issue ?

Using Human Visual Perception

How Radiologist (Using Human Eye) deals with feature detection/Recognition issue ?

Using Human Visual Perception

Humans are equipped with sense organs e.g. eye-Eye receives sensory inputs and -Transmits sensory information to the brain

http://www.simplypsychology.org/perception-theories.html

Teach the machine to see like just we doObjective:

Irregular in shape and sizeNo definite patternLow Contrast -

Located in dense tissueHardly any color intensity variation

Machine Vision Challenges:-To make sense of what it sees

In Real:MC is Extracted Using Human Visual Perception

SURF Point Algorithm

Speeded-Up Robust Features (SURF) Algorithm

Point feature algorithm (SURF)Approach:

Improving the prediction performance of CAD Providing a faster, reliable and cost-effective prediction

Features will facilitate:

Fig: MC Point features (Extracted Using SURF point feature algorithm)

Point feature algorithm (SURF)Approach:

SURF point algorithm

Detect a specific object

Speeded-Up Robust Features (SURF) algorithm to find blob features.

Objective

based on Finding point correspondences

between . The reference and the target image

Reference Image Target Image

Context in using the features:

Feature ExtractionSURF point algorithmSpeeded-Up Robust Features (SURF) algorithm to find blob features.

I. Finding Key pointsII. Matching key pointsIII. Classification

Fig. Putatively Matched Points (Including Outliers )

Estimate Geometric Transformation and Eliminate Outliers

Moving Towards MC Feature DetectionUsing

SURF Point Algorithm

Local feature

Details In Next slide

To keep in mind

Local Feature Detection and Extraction

Local features :

A pattern or structure :Point, edge, or small image patch.

- A pattern or structure found in an image,

Differs from its immediate surroundings bytexture colorintensity

- Associated with an image patch that:

Fig.1 : Some Image Patch We used for Feature Point Detection Purpose

Applications: Image registration Object detection and classification TrackingMotion estimation

Using local features facilitates: handle scale changes rotation occlusion

Detectors /Methods :• FAST• Harris• Shi & Tomasi• MSER

• SURF

Feature Descriptors:SURFFREAKBRISKHOG descriptors

Detecting corner features

detecting blob/point features.

Detector Feature Type Scale IndependentFAST [24] Corner No

Minimum eigen value algorithm[25]

Corner No

Corner detector [26] Corner NoSURF [27] Blob/ Point YesBRISK [28] Corner YesMSER [29] Region with uniform

intensityYes

Why Using SURF Feature?Trying to identify MC cluster Blob

detectSURFFeatures(boxImage);

selectStrongest(boxPoints, 100)

extractFeatures(boxImage, boxPoints)

matchFeatures(boxFeatures, sceneFeatures);

Read the reference image containing the object of interest

Read the target image containing a cluttered scene.

Detect feature points in both images.

Select the strongest feature points found in the reference image.

Select the strongest feature points found in the target image.

Extract feature descriptors at the interest points in both images.

Find Putative Point Matches using their descriptors

Display putatively matched features.

Locate the Object in the Scene Using Putative Matches

SURF Point Detection

1.Read the reference image

containing MC cluster

2.Target image containing MC.

2.Strongest feature point

in MC cluster

2. Strongest Feature point in Target Image

3. No match point Found

Are we getting less feature points?

Figure: No match point Found

No. of SURF feature points: 2 No. of SURF feature points: 47

Image Size256*256

Image Size 549*623

Image mdb238.jpg

More features from the image extracted(most points are mismatched)

To extract relevant feature point from the image

Case 1: Consider Big Reference Image

To get more feature points

Case 2: Consider A bigger Reference Image andWhole mammogram as Target Image

1. Image of MC Cluster(mdb238.jpg) (256*256)

2. Main mammogram (mdb238.jpg) 1024*1024

3. 100 strongest point of ROI) (256*256) 4. 300 strongest point of Main mammogram (mdb238.jpg) 1024*1024

To get more feature points

What we finally have? No putative match Point

Case 2: Consider A bigger Reference Image andWhole mammogram as Target Image To get more feature points

1. Image of an Microcalcification Cluster

Too small ROI will cause less feature points to match

2. 23 strongest pointsAmong 100 Strongest Feature Points

from reference image

Reference image: mdb248.jpgImage size: 256 *256

detectSURFFeatures(mc_cluster);

Problem 1: less number of feature points to matchSURF Feature Point

4. Only 1 strongest pointsAmong 300 Strongest Feature Points

from Scene Image

Too small ROI will cause less feature points to match

3. Image of a Cluttered Scene

Scene image: mdb248.jpgImage size: 427*588

detectSURFFeatures(sceneImage)

Result of small ROI (256*256):No Putative Point Matches

[mcFeatures, mc_Points] = extractFeatures(mc_cluster, mc_Points);[sceneFeatures, scenePoints] = extractFeatures(sceneImage, scenePoints);mcPairs = matchFeatures(mcFeatures, sceneFeatures);matchedmcPoints = mc_Points(mcPairs(:, 1), :);matchedScenePoints = scenePoints(mcPairs(:, 2), :);showMatchedFeatures(mc_cluster, sceneImage, matchedmcPoints, ... matchedScenePoints, 'montage');

Image Image Size Number of feature points

1190*589 15

588*427 23

256*256 1

541*520 86

Varying image size to see the effect to get SURF feature points

Approach-01 to solve: Considering the Whole image(Label and Pectoral Muscle)

Image size No. of SURF feature points

1024*1024 63

Target:To acquire more feature

2. Irrelevant Feature Points

1024*1024 63

1. Less Feature points

Approach-01 to solve: Considering the Whole image(Label and Pectoral Muscle)

Result:

255*256 2

Approach-02 : Detect feature from the cropped image

256*256 2

2. Relevant Feature Points

1. Less Feature pointsResult:

Approach-02 : Detect feature from the cropped image

Observation from approach 1 and 2

1. Image Size does not affectThe number of Feature Points

2. Zooming an image mayhelp to extract relevant featuresfrom the image(very few points to match)

mdb238.jpgImage Size: 1024*1024

mdb238.jpgImage Size: 256*256

Observation:Varying image size is not helping to get feature points

Image of an Microcalcification Cluster

23 strongest pointsAmong 100 Strongest Feature Points

from reference image

Reference image: mdb248.jpgImage size: 256 *256

Only 1 strongest pointsAmong 300 Strongest Feature Points

from Scene Image

Scene image: mdb248.jpgImage size: 427*588

Observing SURF Drawback

This method works best for :-- Detecting a specific object

(for example, the elephant in the reference image,rather than any elephant.)

-- Non-repeating texture patterns-- Unique feature

This technique is not likely to work well for:-- Uniformly-colored objects-- Objects containing repeating patterns.

detecting blob /point features. AIM Failed

Image Correlation Technique

Alternate ApproachImage Correlation Technique

Correlation

∑∑ ++=⊗k l

kjkihlkfhf ))((),(=f Image

=h Kernel/Mask

f1 f2 f3

f4 f5 f6

f7 f8 f9

h1 h2 h3

h4 h5 h6

h7 h8 h9

f1h1 f2h2 f3h3

f4h4 f5h5 f6h6

f7h7 f8h8 f9h9

=⊗ hf⊗

Experimental ResultsImage Correlation Technique

Image no: Benign mdb218.jpg

1. Original image

2. Kernel/ Mask/Template

3. Correlation Output

4. Identified MC(High value of sum.)

(Fixed Template Problem)..

Image no: Benign mdb222.jpg(Fixed Template Problem) Cont….

(Fixed Template Problem)..

Using Gabor Filter

• Make Gabor patch:

2; 2; 0.7854

2; 0.5; 0.7854

2; 2; 1.5708

5; 0.5; 1.5708

5; 2; 0.7854

2; 0.5; 1.5708

5; 0.5; 0.7854

5; 2; 1.5708

• Correlate the patch with image-To extract features of MC

0 10 20 30 40 50 60 70 80 90 100-0.5

Creating Gabor Mask

1. Linear RAMP

2. Linear RAMP values across: Columns Xm (left) and Rows Ym (Right)

3. Linear RAMP values across - Columns(Xm)

The result in the spatial domain

0.5Xm (Across Columns) Ym- (Across rows)

4. Across Columns, Xm :a) Increase frequencyb )Use gray color map

6. Adding Xm and Ymtogether in

different proportions5. Across Rows, Ym :

a) Increase frequencyb )Use gray color map

Creating Gabor Mask

7. Create Gaussian Mask

8. Multiply Grating and Gaussian

Grating Gaussian Mask

Creating Gabor Mask

7. GABOR Mask

Creating Gabor Mask

Alternate ApproachUsing Gabor Filter Gabor kernel

2; 2; 0.7854

2; 0.5; 0.7854

2; 2; 1.5708

5; 0.5; 1.5708

5; 2; 0.7854

2; 0.5; 1.5708

5; 0.5; 0.7854

Scale , frequency, orientation

5; 2; 1.5708

MatrixSize = 26; %always scalar!

Scales = [2, 5];Orientations = [pi/4, pi/2];Frequencies = [0.5, 2];

CenterPoints = [13 13]; %int type (eg. [5 5; 13 13])

CreateMethod = FilterBank.CREATE_CROSSPRODUCT;

40-0.5

2; 2; 0.7854

40-0.2

2; 0.5; 0.7854

40-0.2

2; 0.5; 1.5708

40-0.2

5; 2; 0.7854

40-0.2

5; 2; 1.5708

40-0.1

5; 0.5; 0.7854

40-0.1

5; 0.5; 1.5708

40-0.5

2; 2; 1.5708

Using Gabor Filter Gabor kernel

; 0 5; 5 08

Using Gabor Filter

Image In Spatial DomainUsing Gabor Filter Final Scenario

mini-MIAS drawbacksExperimental Realization

mini-MIAS drawbacksBenign mdb218

Original Enhanced

Gabor Effects

Original

Enhanced

Gabor Effects

Observation 1

Original Enhanced

- NO definite Feature found

Gabor Effects

OBSERVATION-1:

More Evaluation (Gabor)mdb222.jpgBenign

OBSERVATION-1:

-NO definite feature of MC

Original Enhanced

Are these really enhanced?

-There is more detail, but could be noise.

Question Arise?

Gabor Effects

mini-MIAS drawbacks

Enhanced version can contain Noise

Experimental Realization

1.Very Poor Quality with .jpg compression effects

a) Original image b) Enhanced image b) Enhanced imagea) Original image

mdb209

mdb213

mdb219

mdb249

mini-MIAS drawbacks

Not good enough for MC to be detectable

2. Reduced in resolution

Benign mdb218

Original Enhanced

Observation 2

mini-MIAS drawbacks

Not good enough for MC to be detectable

2. Reduced in resolution

Benign mdb218

Original

Enhanced

Where is MC?

OBSERVATION-2:

-There is more detail, but could be noise.

-Enhanced versionseems to contain compression artifacts.

OBSERVATION-2:

- Bad resolution- Noise dominant- No definite feature of MC

OBSERVATION-2:

- Bad resolution/Poor quality image

- No definite feature of MC

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

OBSERVATION-2:

More Evaluation (Gabor)mdb209.jpgMalignant

OBSERVATION-2:

More Evaluation (Gabor)Malignant mdb231.jpg

OBSERVATION-2:

Observation 3

OBSERVATION-3:

-Image Smoothing to remove edge will

Vanish the existenceof MC

-No definite feature of MC- Noise dominant

OBSERVATION-3:

Observation 4,5,6

OBSERVATION-4:

-NO definite feature of MCFalse contour

OBSERVATION-5:

No feature

OBSERVATION-6:

No feature

Several similar area false positive o/p

Observation 7

OBSERVATION-7:

-feature of MC-But MC has different

orientationin different image

OBSERVATION-7:

-feature of MC-But MC has different

orientationin different image

Observation &

Drawing Conclusion

Future detection

Observation & Drawing Conclusion Feature Detection

• Very Poor Quality with .jpeg compression effects(Original MIAS doesn’t have such artifacts)

Limitations of mini-MIAS:

What can be done using mini-MIAS ?

• Can be used for big object detection(Pectoral Muscle, X-ray Label, Tumor, Mass detection)

Conclusion: mini-MIAS is not a good choice for:MC feature extraction

Any alternative to mini-MIAS?

Database Name Authority

MIAS ( Mammographic Image Analysis Society Digital Mammogram Database)

Mammography Image Analysis Society- an

organization of UK research groups

DDSM (Digital Database for Screening Mammogram) University Of South Florida, USA

NDM (National Mammography Database) American College Of Radiology, USA

LLNL/UCSF Database

Lawrence Livermore National Laboratories

(LLNL), University of California at San Fransisco (UCSF)

Radiology Dept.

Database Name Authority

Washington University Digital Mammography Database Department of Radiology at the

University of Washington, USA

Nijmegen Database Department of Radiology at the

University of Nijmegen, the

Netherlands

Málaga mammographic database University of Malaga Central Research

Service (SCAI) ,Spain

BancoWeb LAPIMO Database Electrical Engineering Department at

Universidad de São Paulo, Brazil

These databases are NOT FREE

Research Findings

5; 0.5; 0.7854

Research FindingsImproved computer assisted

screening of mammogram

Detection and removal of big objects:- Pectoral Muscle - X-ray level

Suspected

Observation & Drawing Conclusion On

Feature Detection

• Very Poor Quality with .jpeg compression effects(Original MIAS doesn’t have such artifacts)

Limitations of mini-MIAS:

What can be done using mini-MIAS ?

• Can be used for big object detection(Pectoral Muscle, X-ray Label, Tumor, Mass detection)

Conclusion: mini-MIAS is not a good choice for:MC feature extraction

BesideResearch Findings…

Published PaperAvailable Online:

http://cennser.org/IJCVSP/paper.html

Submitted Paperhttp://www.journals.elsevier.com/image-and-vision-computing/

Further Research ScopeThere is always more to work on..In Research:

Future Plan

1. Segment the image

2. Find out the feature from the segmented image

3. Train the machine with features:

-ANN (Artificial Neural Network)-SVM (Support Vector Machine)

- GentleBoost Classifier [30]

4. Identify the MC5. Classify the MC

Available options

[1]Heinlein P., Drexl J. and Schneider Wilfried: Integrated Wavelets for Enhancement ofMicrocalcifications in Digital Mammography, IEEE Transactions on Medical Imaging, Vol.22, (2003) pp. 402-413

[2]. Zhibo Lu, Tianzi Jiang, Guoen Hu, Xin Wang: Contourlet based mammographicimage enhancement, Proc. of SPIE, vol. 6534, (2007) pp. 65340M-1 - 65340M-8

[3]A.Papadopoulos, D.I . Fotiadis, L.Costrrido,” Improvement of microcalcification clusterdetection in mammogaphy utilizing image enhancement techniques”.Comput.Bio.Med.10,Vol 38,Issue 38,pp.1045-1055,2008

[4]M.Rizzi, M.D’Aloia, B.Castagnolo,” Computer aided detection of microcalcification in digitalMammograms adopting a wavelet decomposition ”,Integr.Comput.-Aided Eng.,Vol 16,Issue 2,pp.91-103,2009

[5]D.Narain Ponraj, M.Evangelin Jenifer, P. Poongodi, J.Samuel Manoharan “A Survey on thePreprocessing Techniques of Mammogram for the Detection of Breast Cancer”, Journal ofEmerging Trends in Computing and Information Sciences, Volume 2, Issue 12, pp. 656-664,2011

Reference

[6]K. Santle Camilus , V. K. Govindan, P.S. Sathidevi,” Pectoral muscle identification inmammograms”, Journal of Applied Clinical Medical Physics , Vol. 12 , Issue No. 3 , 2011

[7] Leeuw H.D., Stehouwer BL, Bakker CJ, Klomp DW, Diest PV, Luijten PR, Seevinck PR,Bosch MA, Viergever MA, Veldhuis WB:Detecting breast microcalcifications with high-field MRI, NMR in Biomedicine,Vol 27, Issue 5, pages 539–546,2014

[8]N.R.Pal,B.Bhowmik, S.K.Patel, S.Pal, J.Das,”A multi-stage nural network aided system fordetection of microcalcification in digitized mammogeams”,Neurocomputing, Vol 11,pp.2625-2634,2008

[9]S.N.Yu, Y.K. Huang,” Detection of microcalcifications on digital mammograms using combined Model-based and statistical textural features”, Expert Syst.Appl. , Vol 37,Issue 7,pp.5461-5469, 2010

[10]Arnau Oliver, Albert Torrent, Meritxell Tortajada, Xavier Llad´o,Marta Peracaula,Lidia Tortajada, Melcior Sent´ıs, and Jordi Freixenet,” Automatic microcalcification andcluster detection for digital and digitised mammograms”, Springer-Verlag BerlinHeidelberg, 36, pp. 251–258, 2010

Reference

[11] Balakumaran T., Vennila ILA, Shankar C.G: Detection of Microcalcification in Mammograms Using Wavelet Transform and Fuzzy Shell Clustering, International Journal of Computer Science and Information Security, Vol 7,Issue 1,pp.121-125,2010

[12] Arnau Olivera, Albert Torrenta , Xavier Lladóa , Meritxell Tortajada, LidiaTortajadab, Melcior Sentísb, Jordi Freixeneta, Reyer Zwiggelaarc,” Automaticmicrocalcification and cluster detection for digital and digitisedmammograms”, Elsevier:Knowledge-Based Systems, Volume 28, pp. 68–75, April 2012.

[13] Zhang X., Homma N., Goto S.,Kawasumi Y., Ihibashi T.,Abe M.,Sugita N.,Yoshizawa M:A Hybrid Image Filtering Method for Computer-Aided Detection of Microcalcification Clusters in Mammograms, Journal of Medical Engineering, Vol 3,Issue 1,pp.111-119,2013

[14]Wang T. C and Karayiannis N. B.: Detection of Microcalci¯cations in Digital Mammograms Using Wavelets, IEEE Transaction on Medical Imaging, vol. 17, no. 4,(1989) pp. 498-509

[15]. Daubechies I.: Ten Lectures on Wavelets, Philadelphia, PA, SIAM, (1992)

Reference

[16] Strickland R.N. and Hahn H.: Wavelet transforms for detecting microcalcificationsin mammograms, IEEE Transactions on Medical Imaging, vol. 15, (1996) pp. 218-229

[17] Lu J., Ikehara T., Zhang Y,Mihara T., Itoh T.,Maeda R:High quality factor silicon cantilever driven by piezoelectric thin film actuator for resonant based mass detection, Micro system Technologies , Vol 15, Issue 8, pp:1163-1169., 2009

[18]Fatemeh Moayedi, Zohreh Azimifar, Reza Boostani, and Serajodin Katebi: Contourlet-based mammography mass classification, ICIAR 2007, LNCS 4633,(2007) pp. 923-934

[19] Shankla V, David D. P, Susan P. Weinstein; Michael D., Tuite C, Roth R., Emily F:Automatic insertion of simulated microcalcification clusters in a software breast phantom, , Proc. SPIE 9033, Medical Imaging 2014: Physics of Medical Imaging, 2014

[20] Otsu, N., "A Threshold Selection Method from Gray-Level Histograms," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 9, No. 1, 1979, pp. 62-66.

Reference[21] Da Cunha A. L., Zhou J. and Do M. N,: The Nonsubsampled Contourlet Transform:Theory, Design, and Applications, IEEE Transactions on Image Processing,vol. 15, (2006)pp. 3089-3101

[22]Laine A.F., Schuler S., Fan J., Huda W.: Mammographic feature enhancement bymultiscale analysis, IEEE Transactions on Medical Imaging, 1994, vol. 13, no. 4,(1994)pp. 7250-7260

[23] Sameer S. and Keit B.: An Evaluation on Contrast Enhancement Techniques forMammographic Breast Masses, IEEE Transactions on Information Technology inBiomedicine, vol. 9, (2005) pp. 109-119

[24] Rosten, E., and T. Drummond. "Machine Learning for High-Speed Corner Detection." 9th European Conference on Computer Vision. Vol. 1, 2006, pp. 430–443.

[25] Shi, J., and C. Tomasi. "Good Features to Track." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. June 1994, pp. 593–600.

[26] Harris, C., and M. J. Stephens. "A Combined Corner and Edge Detector." Proceedings ofthe 4th Alvey Vision Conference. August 1988, pp. 147–152.

[27] Bay, H., A. Ess, T. Tuytelaars, and L. Van Gool. "SURF: Speeded Up RobustFeatures." Computer Vision and Image Understanding (CVIU). Vol. 110, No. 3, 2008, pp.346–359.

[28] Leutenegger, S., M. Chli, and R. Siegwart. "BRISK: Binary Robust Invariant Scalable

[29] Matas, J., O. Chum, M. Urba, and T. Pajdla. "Robust wide-baseline stereo frommaximally stable extremal regions."Proceedings of British Machine Vision Conference.2002, pp. 384–396.

[30] Oliver A.; Torrent A. , Tortajada M, Liado X, R., Preacaula M , Tortajada L., Srntis M.,Ferixenet J: A Boosting based approach for automatic Microcalcification Detection,Springer-Verlag Berlin Heldelberg,Lecture notes on Computer Science (LNCS 6136), (2010)pp. 251- 258

Reference

Thank you foryour time and attention

ms thesis-final-defense-presentation

Education

ms thesis defense - ku ittc · master’s thesis defense...

jr holmes ms thesis defense

defense thesis

holmes ms thesis defense

ms thesis defense, aug 2012 - visualizing digital...

ms thesis defense

thesis defense

graduate study manual - northwestern university ·...

background trace element concentrations in the franciscan...

ms thesis defense “improving gpu performance by regrouping...

ms thesis defense dynamic fault tolerant grid workflow in...

j raphael holmes ms thesis defense

ms thesis defense presentation

gupta roy ms thesis defense

ms thesis defense - ittc · master’s thesis defense...

eden neill-hastings thesis oral defense ms. sauve –...