Imaging and Visualizing Micro-Vascular Architecture

Michael GleicherGaret Lahvisand the UW Graphics GroupUniversity of Wisconsin- Madisonwww.cs.wisc.edu/~gleicherwww.cs.wisc.edu/graphics

Imaging and Visualizing Micro-Vascular Architecture

Michael Gleicher Assistant Prof

Elizabeth Osten Research Assistant

Adam HuppCIBM Research Intern

Brian RiesUndergraduate Assistant

Chris Olsen Undergraduate Assistant

and the UW Graphics and Vision Gang

Garet LahvisAssistant Prof

Matt McElweeAssociate Scientist

Adam GepnerUndergrad Assistant

Summer Hanson Medical Student

Synopsis

This talk is about toolsUltimately want to understand vascular systemNeed to see vessels to understand itEarly stages: no biology results yet

Need new tools for a unique problemBiology and CS techniques

New histology techniques to get imagesNew reconstruction techniquesNew analysis and visualization techniques

Lots of problems, fewer solutions (so far…)

Outline

Motivation: the biological questions

Why is this problem unique?

Imaging and Visualization Pipeline

Each stage:Why is it hard?Initial ExperimentsFuture Directions

HistologyHistology

ImageAnalysisImage

Analysis

Reconstruction(Registration/Modeling)

Reconstruction(Registration/Modeling)

VisualizationVisualization AnalysisAnalysis

Imaging(Microscopy)

Imaging(Microscopy)

Garet

Switch to Garet

What if…You could see every capillary?

That’s a lot of vessels!

And this is just one slice!

Why is this hard

Massive data setsSmall, discrete structures

Hard to reduce without losing featuresInterested in patterns of small things

Details are not the sameBetween slicesBetween brains

Noisy, invasive imaging

What’s Similar?Retinal Fundus Imaging

Also looks at networks of capillaries2D structures on a 2D surfaceKnown branching patternNo non-rigid deformation

Virtual AngiographyDetails small numbers of large vessels

Neuron TracingMore structureNot done at this scale?

Image Processing and Analysis

Segmentation: indentify what is vessel (and what is not vessel)

Background Finding

Not so Easy!

Semi-Automatic for now (better safe than sorry)

Current Status

Adaptive Thresholding“bright” varies across image

Edge enhancementSome vessels are dim

Semi-automatic background and bubble elimination

Some pictures

Short term utility

Current Practice:Relative density measurements in 2D

Easy to createHelp get results in short termProvide validation

Future

Develop better models of vessel shape and appearanceLess ad-hoc/more reliable methodsMore automationInfer depth from observation

Hard: bad diffusion appears as defocus/dimming

Validation!

Geometric Model Building

Represent vessels as geometric elementsNOT spatial samples / pixels / voxels

Easier to analyzeConnectivityAbstraction

Easier to drawPolygons, not volumes

Easier to visualizeStylized renderingConnect Visualization and Analysis

Represent Uncertainty

Geometric Model

Vessels are generalized cylindersTubes of varying radii

Piecewise linear approximationsStored as graph structure

2D models

Current idea:Build geometry per-sliceConnect slices together

Medial-Axis Transform-like processingLargest circle that covers a pointFinds “spine” or skelletonSimplified algorithms to provide guaranteesSacrifice optimality for lack of artifacts

The Tracer Algorithm

Find “staircases”Connected Horizontal and Vertical LinesAll pixels guaranteed to be seenEntire region connectedSmooth staircases to medial points

Tracer Algorithm (2)

Smooth traces to medial points

Medial point finding tells us thickness of vessel

Tracer Pictures

Future

Less ad-hoc modelingIntegration with segmenter

Better geometryOptimal codingCurves

Stochastic Geometry

Registration

Putting pieces back into a whole

Putting multiple images into a common coordinate system

RegistrationWhy our problem is hard

Small details to line upNeed precisionCan’t work coarse to fine

Brains SquishNon-linear deformations

Slices are differentCan’t rely on image matching

Feature Based Methods

Find correspondences between discrete features

Point to point (standard)Point to region (future)

Two parts:Finding correspondencesFinding deformation (interpolation)

Deformation Modeling

As smooth as possibleNeed speed

Fast solution (interactive placement)Fast drawing

Need robustness

Hierarchical B-Splines

Hierarchical B-Splines

Sets of uniform B-SplinesEach captures different frequency

Sequential SolutionSolve as much as possible in coarse levelEach level is a linear least squares problem

= + +

Advantages of H-B-SplinesFast!

Sparse linear least squaresEasy to draw by sampling into affine grid

Well-behaved for interactionFirst points get overall pictureLater points refine

Very smoothLinear sub-problems afford robustnessTransform geometry – avoid resampling

Need for robustness

Robust norms (not true least-squares)Damped Least Squares (penalize movements of variables)Damped Lagrange MultipliersM-Estimators

Built into solver (BiCG, LSQR)

schematicA little noise makes

a big mess

Registration

Drag points to corresponding locationsDon’t need too many pointsFast – interactive dragging rates

Registration User Interface

Stacks

All slices into common coordinate systemTransforms do not really composeApproximate by transforming points

1 2

F(2->1)

21 3

F(3->1)

user userF(2->1)

Align PairsBuild Stacks

The Stack

Different Brains

Rough manual alignmentsEasy, even if brains are quite different

Caveat: we are introducing distortions

unaligned aligned

Different Brains

Quick

Visual (Ad Hoc)

Comparisons

Comparison is important – how do we do it?

Issues / Future

AutomationIterated Closest Point MethodsDual-Bootstrap ICP

Point to region solving

Error modelingUncertainty

Absolute positoning with fiducial markers

How do we knowwe are right?

Connection Finding

Easy once registration is done

Some catchesEnds don’t always connect to endsVertical vesselsT-junctionsNoise and mis-registration

Noise filtering AFTER connection finding

Depth Inferencing

Slices are thick relative to vessel sizeDifficult to infer depth from imagesUse connections to give sparse informationUse diffusion to interpolateRemember uncertainty and “fiction”

X

?

Have a Model – Now What?

AnalysisMeasurements, statisticsComparisons

VisualizationWhy? Gain insight, look for patterns, …

“Because its cool” is NOT an OK answer

Visualization Challenges

What are we trying to see?

Coping with massive complexityEfficiency in drawingComprehensibilityNavigationFocusCommunication / Collaboration

Why not Volume Visualization

Sampling issuesNon-uniform

Need too much resolutionUse structure to enhanceLeverage Commodity Hardware

Computer Game Technology to the rescue!

A Tiny Example

Olfactory bulb from a neo-natal mouseSmall piece of a small brain

Initial Results: Tubes

Stylized Rendering andOther Visualization Ideas

Illustration methods (Gooch)

Kinetic methods

Challenge: Navigation

How do you move around?How do you not get lost?How do you give directions?

Maps? (but in 3D)Landmarks / Breadcrumbs?Good “flying” controls?

The Real Challenge:What are we looking for?

Couple Analysis and VisualizationAnalysis:Computer sifts through lots of dataVisualization:Human sees patterns and trends

Analysis guides visualizationVisualization directs analysis

More Directions in Visualization

Specific tools for specific tasksComparisonsTrends

Tie with meta-data managementKeep all informationSpatially situated notesLarge databases

Uncertainty and Error

SummaryInteresting biological questionsRequire understanding brain vasculature

Need novel solutions in all phases:Image vasculature through histologyInterpret/reconstruct imageryVisualization and AnalysisAsk questions that our tools can help answer

Thanks!To the UW graphics gang.

The UW graphics group is sponsored by the National Science Foundation, Microsoft, Intel, and the Wisconsin University and Industrial Relations program.

Adam Hupp was supported as a research intern by the CIBM program, and as research scientist by WIMIC