a pipeline for computer aided polyp detection

A Pipeline for Computer Aided Polyp Detection

A Pipeline for Computer Aided Polyp Detection

Wei Hong, Feng Qiu, and Arie KuafmanWei Hong, Feng Qiu, and Arie Kuafman

Center for Visual Computing (CVC) andCenter for Visual Computing (CVC) and

Department of Computer ScienceDepartment of Computer Science

Stony Brook UniversityStony Brook University

Related WorkRelated Work

Shape based polyp detection methodShape based polyp detection method• Vining et al. ’99: colon wall thicknessVining et al. ’99: colon wall thickness• Tomasi et al. ’00: sphere fittingTomasi et al. ’00: sphere fitting• Summers et al. ’01: local curvature variationsSummers et al. ’01: local curvature variations• Yoshida et al. ’01: shape index and curvednessYoshida et al. ’01: shape index and curvedness• Paik et al. ’04: Paik et al. ’04: intersecting normal vectorsintersecting normal vectors• Wang et al. ’06: global curvatureWang et al. ’06: global curvature

Sensitive to the irregularity of the Sensitive to the irregularity of the

colon wallcolon wall Relatively high false positive rateRelatively high false positive rate

2

Electronic BiopsyElectronic Biopsy

3

Opaque transfer Opaque transfer functionfunction

Transparent transfer Transparent transfer functionfunction

Polyps have a slightly higher density and Polyps have a slightly higher density and different texture.different texture.

Overview of Our CAD PipelineOverview of Our CAD Pipeline

4

Segmentation and Digital Segmentation and Digital CleansingCleansing

Colon Surface ExtractionColon Surface Extraction

Conformal Colon FlatteningConformal Colon Flattening

Electronic Biopsy Image Electronic Biopsy Image GenerationGeneration

Polyp Detection by Polyp Detection by ClusteringClustering

False Positive ReductionFalse Positive Reduction

Integration with Virtual Integration with Virtual ColonoscopyColonoscopy

Contrast-enhanced CT

Step 1Step 1

5








Segmentation & Digital CleansingSegmentation & Digital Cleansing

Input: contrast-enhanced CT scan of patient’s abdomenInput: contrast-enhanced CT scan of patient’s abdomen Goal: segment and cleanse colon lumenGoal: segment and cleanse colon lumen Challenges for digital cleansing:Challenges for digital cleansing:

1.1. Remove the interface layer between air and tagged fluidRemove the interface layer between air and tagged fluid

2.2. Restore the CT densities in the enhanced mucosa layerRestore the CT densities in the enhanced mucosa layer

6

interfinterfaceace aiai

rr

tagged tagged fluidfluid

mucosamucosa

Partial Volume SegmentationPartial Volume Segmentation

Assumptions:Assumptions:

1.1. Four material classes within each voxel Four material classes within each voxel i i (air, soft (air, soft tissue, muscle, and bone)tissue, muscle, and bone)

2.2. Each material follows a Gaussian distributionEach material follows a Gaussian distribution

: the observed density value at voxel : the observed density value at voxel ii : Gaussian noise with zero mean: Gaussian noise with zero mean : fraction of classes : fraction of classes kk Using expectation-maximization algorithm to estimateUsing expectation-maximization algorithm to estimate

7

4

1i ik i i

k

y m μ ε=

= +∑

2{ , , }k k ikmμ σ

2{ , }k kμ σ

iεikm

iy

4

1

1, 0 1ik ikk

m m=

= ≤ ≤∑

Segmentation ResultsSegmentation Results

8

airbone

air & fluid partial volume effecttissue & fluid partial volume effect

Digital Cleansing ResultsDigital Cleansing Results

Original CT Original CT sliceslice

Cleansed sliceCleansed slice

Zoomed Zoomed viewview

new old fluid fluid fluid airy y m mμ μ= − +

9

Cleansing equation:Cleansing equation:

Step 2Step 2

10








11

Genus zero Colon Surface ExtractionGenus zero Colon Surface Extraction

Topological noise (i.e., tiny handles) Topological noise (i.e., tiny handles) • makes our flattening algorithm complexmakes our flattening algorithm complex• introduces distortionintroduces distortion

Simple point: Simple point: A point is simple if its A point is simple if its

addition to and removal from objects addition to and removal from objects

does not change object topologydoes not change object topology.. Our 3D region growing based methodOur 3D region growing based method

• Computing a distance fieldComputing a distance field• Computing colon centerlineComputing colon centerline• Region growing all simple pointsRegion growing all simple points

Simple Simple pointpointCritical Critical pointpoint

Step 3Step 3

12








Virtual Colon FlatteningVirtual Colon Flattening

Bartroli et al. ’01: area preservingBartroli et al. ’01: area preserving Haker et al. ’00: angle preservingHaker et al. ’00: angle preserving

• Genus 0 surfacesGenus 0 surfaces• Mapping to a planar parallelogramMapping to a planar parallelogram

Our Method: angle preservingOur Method: angle preserving• Surfaces with arbitrary topologySurfaces with arbitrary topology• Mapping to a 2D rectangleMapping to a 2D rectangle

13


14

1.1. Computing a gradient field of the conformal mapComputing a gradient field of the conformal map

2.2. Computing the conformal map by integrationComputing the conformal map by integration

3.3. Tracing a horizontal lineTracing a horizontal line

4.4. Cutting the colon surface along the horizontal lineCutting the colon surface along the horizontal line

15

Angle PreservingAngle Preserving

3D3D

2D2D

cutting cutting lineline

Step 4Step 4

16








Electronic Biopsy Image GenerationElectronic Biopsy Image Generation

17

1.1. Pre-defined translucent transfer functionPre-defined translucent transfer function2.2. Surface normals used as ray directionsSurface normals used as ray directions3.3. Rays are allowed to traverse up to 40 Rays are allowed to traverse up to 40

steps (0.5mm/step)steps (0.5mm/step)4.4. Rays cannot enter colon lumenRays cannot enter colon lumen5.5. GPU acceleration (300ms for 4000X196 GPU acceleration (300ms for 4000X196

image)image)

Step 5Step 5

18








Polyp Detection by ClusteringPolyp Detection by Clustering

For each pixel, we use the color information in its small neighborhood as the feature vector

Method:

1. PCA: reduce the dimension of the feature vectors

2. Clustering algorithm: classify each pixel

3. A labeling algorithm: extract the connected components

We only consider polyps with a diameter > 5mm, small components are removed.

19

Step 6Step 6

20








Reduction of False PositivesReduction of False Positives

Shape features are exploited for polyp detection• Volumetric shape index & curvedness

Yoshida et al.

The computation of these volumetric shape features is time consuming

We only do it at suspicious areas for FP reduction

21

Results of Clustering and False Positive ReductionResults of Clustering and False Positive Reduction

22

Electronic biopsy image

The result of our clustering algorithm

The result of FP reduction

Step 7Step 7

23








Integration with Virtual ColonoscopyIntegration with Virtual Colonoscopy

The extracted colon mesh is used to accelerate volumetric The extracted colon mesh is used to accelerate volumetric ray-castingray-casting• Colon mesh is projected onto the image planeColon mesh is projected onto the image plane• Empty space between image plane and colon wall is Empty space between image plane and colon wall is

skippedskipped• Frame rate: 17-20/sec for image size of 512X512 Frame rate: 17-20/sec for image size of 512X512

Suspicious polyp candidates are highlighted in the Suspicious polyp candidates are highlighted in the endoscopic view to attract the attention of the radiologistsendoscopic view to attract the attention of the radiologists

A flattened colon image is also providedA flattened colon image is also provided• Suspicious polyp locationsSuspicious polyp locations• Bookmarks Bookmarks

24

Enhanced Endoscopic ViewEnhanced Endoscopic View

25

The User Interface of Our CAD SystemThe User Interface of Our CAD System

26

27

DatasetsDatasets

52 CT datasets from National Institute of Health (NIH)52 CT datasets from National Institute of Health (NIH)• 400~500 Raw DICOM images (512X512)400~500 Raw DICOM images (512X512)• VC reports and videosVC reports and videos• OC reports and videosOC reports and videos• Pathology reportsPathology reports

46 CT datasets from Stony Brook University Hospital (SB)46 CT datasets from Stony Brook University Hospital (SB)• 400~500 Raw DICOM images (512X512)400~500 Raw DICOM images (512X512)• VC reports and videosVC reports and videos• OC reportsOC reports• Pathology reportsPathology reports

Experimental ResultsExperimental Results

28

SourceSource Total PolypsTotal Polyps SensitivitySensitivity FP per DatasetFP per Dataset FP ReductionFP Reduction

NIHNIH 5858 100%100% 3.13.1 96.3%96.3%

SBSB 6565 100%100% 2.92.9 97.1%97.1%

Stage of our CAD PipelineStage of our CAD Pipeline TimingTiming

Segmentation and Digital CleansingSegmentation and Digital Cleansing 3 mins3 mins

Colon Surface ExtractionColon Surface Extraction < 1 min< 1 min

Conformal Colon FlatteningConformal Colon Flattening 7 mins7 mins

Biopsy Image GenerationBiopsy Image Generation 300 ms300 ms

Polyp Detection by ClusteringPolyp Detection by Clustering < 1 min< 1 min

False Positive ReductionFalse Positive Reduction < 1 min< 1 min

3.6GHz Pentium IV, 3G Ram, Quadro 3.6GHz Pentium IV, 3G Ram, Quadro FX4500, 512^FX4500, 512^33

ConclusionsConclusions

A novel method for automatic polyp detection by A novel method for automatic polyp detection by integrating direct volume rendering with conformal integrating direct volume rendering with conformal colon flatteningcolon flattening• 100% sensitive to polyps with a low FP rate100% sensitive to polyps with a low FP rate• Highlighting the polyp locationsHighlighting the polyp locations• Enhancing the user interface of VCEnhancing the user interface of VC• Improve the efficiency and accuracy of VCImprove the efficiency and accuracy of VC

29

Future WorkFuture Work

Improving detection algorithm to further reduce Improving detection algorithm to further reduce FPsFPs

Porting our CAD pipeline to a clinical VC systemPorting our CAD pipeline to a clinical VC system Supine and prone registrationSupine and prone registration Applying our methods to other human organsApplying our methods to other human organs

• Blood vesselBlood vessel• Bladder Bladder

30

Questions?Questions?

Thank You!Thank You!

31

a pipeline for computer aided polyp detection

Documents

cleanse colon lumenchallenges

colon wall thicknesstomasi

segmentation results

polyp detection methodvining

cleansing equation

ct densities

enhanced mucosa layer

contrastenhanced ctstep