Stephen PizerMedical Image Display & Analysis Group

University of North Carolina, USA

with credit to

T. Fletcher, A. Thall, S. Joshi, P. Yushkevich, G. Gerig

Tutorial:Tutorial: Statistics of Object GeometryStatistics of Object Geometry

10 October 2002

Uses of Statistical Geometric Characterization

Uses of Statistical Geometric Characterization

Medical science: determine geometric ways in which pathological and normal classes differ

Diagnostic: determine if particular patient’s geometry is in pathological or normal class

Educational: communicate anatomic variability in atlases Priors for segmentation Monte Carlo generation of images

Object RepresentationObject RepresentationObjectivesObjectives

Relation to other instances of the shape classRepresenting the real worldDeformation while staying in shape classDiscrimination by shape classLocality

Relation to Euclidean space/projective Euclidean spaceMatching image data

Geometric aspects Geometric aspects Invariants and correspondenceInvariants and correspondence

Desire: An image space geometric representation that is at multiple levels of scale (locality) at one level of scale is based on the object and at lower levels based on object’s figures at each level recognizes invariances associated with

shape provides positional and orientational and metric

correspondence across various instances of the shape class

Object RepresentationsObject Representations

Atlas voxels with a displacement at each voxel : x(x)

Set of distinguished points {xi} with a displacement at each Landmarks Boundary points in a mesh

With normal b = (x,n)

Loci of medial atoms: m = (x,F,r,) or end atom (x,F,r,)

u

v

t

Continuous M-reps: B-splines in Continuous M-reps: B-splines in (x,y,z,r) [Yushkevich](x,y,z,r) [Yushkevich]

Building an Object Representation Building an Object Representation from Atoms from Atoms aa

Sampled aij

can have inter-atom mesh (active surface) Parametrized

a(u,v) e.g., spherical harmonics, where coefficients become

representation e.g., quadric or superquadric surfaces some atom components are derivatives of others

Object representation: Object representation: Parametrized BoundariesParametrized Boundaries

Parametrized boundaries x(u,v)n(u,v) is normalized x/u x/v

Coefficients of decompositionsx(u,v) = i ci f i(u,v)

Spherical harmonics: (u,v) = latitude, longitudeSampled point positions are linear in

coefficients: Ax=c

Object representation: Object representation: Parametrized Medial LociParametrized Medial Loci

Parametrized medial loci m(u,v) = [x,r](u,v)n(u,v) is normalized x/u x/vxr(u,v) = -cos()b

gradient per distance on x(u,v) b

x n

Sampled medial shape representation: Sampled medial shape representation: Discrete M-rep slabs (bars)Discrete M-rep slabs (bars)

Meshes of medial atoms Objects connected as host,

subfigures Multiple such objects,

interrelated

t=+1

p

x

s

br

n

t=-1

t=0

p

x

s

b

n

o

o

o o o

o

o o

o

o o

u

v

t

Interpolating Medial Atoms in a Figure

Interpolate x, r via B-splines [Yushkevich] Trimming curve via r<0 at outside control points

Avoids corner problems of quadmesh Yields continuous boundary

Via modified subdivision surface [Thall] Approximate orthogonality at sail ends Interpolated atoms via boundary and distance

At ends elongation needs also to be interpolated

Need to use synthetic medial geometry [Damon]

Medial sheet

Implied boundary

End Atoms: (x,F,r,)

Medial atom with one more parameter: elongation

Extremely rounded Extremely rounded end atom end atom

in cross-sectionin cross-section

Corner atomCorner atomin cross-sectionin cross-section

=1=1/cos()

Rounded Rounded end atom end atom

in cross-sectionin cross-section

=1.4

Sampled medial shape representation: M-rep tube figures

Same atoms as for slabs r is radius of tube sails are rotated about b Chain rather than mesh

b

x n

x+

rRb,n()bx+rRb,n(-)b

For correspondence: Object-intrinsic coordinatesGeometric coordinates from m-reps

Single figure Medial sheet: (u,v)

[(u) in 2D] t: medial side : signed r-prop’l

dist from implied boundary

3-space: (u,v,t, ) Implied boundary:

(u,v,t)

u

v

t

t=+1

p

x

s

br

n

t=-1

t=0

p

x

s

b

n

Sampled medial shape representation: Linked m-rep slabs

Linked figures Hinge atoms known in

figural coordinates (u,v,t) of parent figure

Other atoms known in the medial coordinates of their neighbors

o

o

o o o

o

o o

o

o o

x+

rRb,n()bb

x nx+rRb,n(-)b

Blend in hinge regions w=(d1/r1 - d2 /r2 )/T Blended d/r when |w| <1 and u-u0 < T Implicit boundary: (u,w, t) Or blend by subdivision surface

Figural Coordinates for Object Made From Multiple Attached Figures

Figural Coordinates for Object Made From Multiple Attached Figures

w

Inside objects or on boundary Per object In neighbor object’s

coordinates Interobject space

In neighbor object’s coordinates

Far outside boundary: (u[,v],t, ) via distance (scale) related figural convexification ??

??

Figural Coordinates for Multiple Objects

Figural Coordinates for Multiple Objects

Heuristic Medial Correspondence

Radius

Original (Spline Parameter) Arclength

0.2 0.4 0.6 0.8 1

0.2

0.4

0.6

0.8

1

Coordinate Mapping

Continuous Analytical Features

Can be sampled arbitrarily.

Allow functional shape analysis

Possible at many scales: medial, bdry, other object

Medial Curvature Boundary texture scale

Feature-Based Correspondence on Medial Locus by Statistical Registration of Features

curvature

dr/ds dr/ds

Also works in 3D

What is Statistical Geometric Characterization

What is Statistical Geometric Characterization

Given a population of instances of an object class e.g., subcortical regions of normal males of age 30

Given a geometric representation z of a given instance e.g., a set of positions on the boundary of the object

and thus the description zi of the ith instance

A statistical characterization of the class is the probability density p(z)

which is estimated from the instances zi

Benefits of Probabilistically Describing Anatomic Region Geometry

Benefits of Probabilistically Describing Anatomic Region Geometry

Discrimination among geometric classes, Ck Compare probabilities p(z | Ck)

Comprehension of asymmetries or distinctions of classes Differences between means Difference between variabilities

Segmentation by deformable models Probability of geometry p(z) provides prior

Provides object-intrinsic coord’s in which multiscale image probabilities p(I|z) can be described

Educational atlas with variability Monte Carlo generation of shapes, of diffeo- morphisms,

to produce pseudo-patient test images

Necessary Analysis Provisions To Achieve Locality & Training Feasibility

Necessary Analysis Provisions To Achieve Locality & Training Feasibility

Multiple scales Allows few random

variables per scale

At each scale, a level of locality (spatial extent) associated with its random variable

Positional correspondence Across instances Between scales

Large scale Smaller scale

Discussion of ScaleDiscussion of Scale

Spatial aspects of a geometric feature Its position Its spatial extent

Region summarized Level of detail captured

Residues from larger scales Distances to neighbors with

which it has a statistical relationship

Markov random field Cf PDM, spherical harmonics,

dense Euclidean positions, landmarks, m-reps

Large scale Smaller scale

Location

Leve

l of

Det

ail

Coarse

Fine

Scale Situations in Various Statistical Geometric Analysis Approaches

Scale Situations in Various Statistical Geometric Analysis Approaches

Location Location

Global coef for Multidetail feature, Detail residues

each level of detail,

E.g., spher. harm. E.g., boundary pt. E.g., object hierarchy

Coordinates at one scale must relate to parent coordinates at next larger scale

Coordinates at one scale must be writable in neighbor’s coordinate system

Statistically stable features at all scales must be relatable at various scale levels

Principles of Object-Intrinsic Coordinates at a Scale LevelPrinciples of Object-Intrinsic Coordinates at a Scale Level

Figurally Relevant Spatial Scale Levels: Primitives and Neighbors

Multi-object complex Individual object

= multiple figures in geom. rel’n to neighbors in relation to complex

Individual figure = mesh of medial atoms subfigs in relation to neighbors in relation to object

Figural section = multiple figural sections

each centered at medial atom medial atoms in relation to neigbhors in relation to figure

Figural section residue, more finely spaced, .. => multiple boundary sections (vertices)

Boundary section vertices in relation to vertex neighbors in relation to figural section

Boundary section more finely spaced, ...

If the total geometric representation z is at all scales or smallest scale, it is not stably trainable with attainable numbers of training cases, so multiscale Let zk be the geometric representation at scale level k Let zk

i be the ith geometric primitive at scale level k

Let N(zki) be the neighbors of zk

i (at level k) Let P(zk

i) be the parent of zki (at level k-1)

Probability via Markov random fields p(zk

i | P(zki), N(zk

i) ) Many trainable probabilities

If p(zki rel. to P(zk

i), zki rel. to N(zk

i) ) Requires parametrized probabilities

Multiscale Probability Leads to Trainable Probabilities

Multiscale Probability Leads to Trainable Probabilities

Multi-Scale-Level Image AnalysisGeometry + Probability

Multiscale critical for effectiveness with efficiency O(number of smallest scale primitives) Markov random field probabilistic basis Vs. methods working at small scale only or at

global scale + small scale only

Multi-Scale-Level Image Analysisvia M-reps

Thesis: multi-scale-level image analysis is particularly well supported by representation built around m-reps Intuitive, medically relevant scale levels Object-based positional and orientational correspondence Geometrically well suited to deformation

Geometric Typicality Metrics

Statistical Metrics

Statistics/Probability Aspects : Principal component analysis Any shape, x, can be written as

x = xmean + Pb + r

log p(x) = f(b1, … bt,|r|2)

x1

x2

p1

xi

xmean

b1

Visualizing & Measuring Global Deformation

c = cmean + z11p1 c = cmean + z22p2

Shape MeasurementModes of shape variation across patients Measurement = z amount of each mode

Statistics/Probability Aspects : Markov random fields

Suppose zT= (z1 … zn) p(zi | {zj, ji}) =

p(zi | {zk : k a neighbour of i})(i. e., assume sparse covariance matrix)

Need only evaluate O(n) terms to optimize p(z) or p(z | image)

Can only evaluate p(zi), i.e., locally Interscale; within scale by locality

If z is at all scales or smallest scale, it is not stably trainable, so multiscale Let zk be the geometric rep’n at scale k Let zk

i be the ith geometric primitive at scale k

Let N(zki) be the neighbors of zk

i

Let P(zki) be the parent of zk

i

Let C(zki) be the children of zk

i

Probability via Markov random fields p(zk

i | P(zki), N(zk

i), C(zki) )

Many trainable probabilities Requires parametrized probabilities for training

Multiscale Geometry and ProbabilityMultiscale Geometry and Probability

z1 (necessarily global): similarity transform for body section z2

i: similarity transform for the ith object Neighbors are adjacent (perhaps abutting) objects

z3i: “similarity” transform for the ith figure of its object in its parent’s figural coordinates

Neighbors are adjacent (perhaps abutting) figures z4

i: medial atom transform for the ith medial atom Neighbors are adjacent medial atoms

z5i: medial atom transform for the ith medial atom residue at finer scale (see next slide)

z6i: boundary offset along medially implied normal for the ith boundary vertex

Neighbors are adjacent vertices

Probability via Markov random fields p(zk

i | P(zki), N(zk

i), C(zki) )

Many trainable probabilities Requires parametrized probabilities for training

Examples with m-reps components p(zk

i | P(zki), N(zk

i), C(zki) )

Geometrically smaller scale Interpolate (1st order) finer spacing of

atoms Residual atom change, i.e., local

Probability At any scale, relates figurally

homologous points Markov random field relating medial

atom with its immediate neighbors at that scale its parent atom at the next larger scale and

the corresponding position its children atoms

Multiscale Geometry and Probability for a Figure

Multiscale Geometry and Probability for a Figure

coarse, global

coarse resampled

fine, local

Published Methods of Global Statistical Geometric Characterization in MedicinePublished Methods of Global Statistical Geometric Characterization in Medicine

Global variability via principal component analysis on

features globally, e.g., boundary points or landmarks, or

global features, e.g., spherical harmonic coefficients for boundary

Global difference via linear (or other) discriminant on features

globally or on global features Globally based diagnosis

via linear (or other) discriminant on features globally or on global features

Example authors: [Bookstein][Golland] [Gerig] [Joshi] [Thompson & Toga][Taylor]

Published Methods of Local Statistical Geometric Characterization

Published Methods of Local Statistical Geometric Characterization

Local variability via principal component analysis on features

globally or on global features, plus display of local properties of principal component

Local difference via linear (or other) discriminant on global

geometric primitives, plus display of local properties of discriminant direction

On displacement vectors: signed, unsigned re inside/outside

Example authors: [Gerig] [Golland] [Joshi] [Taylor] [Thompson & Toga]

A

Outward, p < 0.05

Inward, p < 0.05

p > 0.05

R L

Displacement significance: Schizophrenic vs. control hippocampus

Shortcomings of Published Methods of Statistical Geometric CharacterizationShortcomings of Published Methods of Statistical Geometric Characterization

Unintuitive Would like terms like bent, twisted, pimpled, constricted,

elongated, extra figure Frequently nonlocal or local wrt global template

Depends on getting correspondence to template correct Need where the differences are in object coordinates

Which object, which figure, where in figure, where on boundary surface

Requires infeasible number of training cases Due to too many random variables (features)

Overcoming Shortcomings of Methods of Statistical Geometric Characterization

Overcoming Shortcomings of Methods of Statistical Geometric Characterization

Intuitive Figural (medial) representation provides terms like

bent, twisted, pimpled, constricted, elongated, extra figure

Local Hierarchy by scale level provides appropriate level of

locality Object & figure based hierarchy yields intuitive locality

and good positional correspondences Which object, which figure, where in figure, where on

boundary surface Positional correspondences across training cases & scale levels

Trainable by feasible number of cases Few features in residue between scale levels

Relative to description at next larger scale level Relative to neighbors at same scale level

Conclusions re Object Based Image Analysis

Work at multiple levels of scaleWork at multiple levels of scale At each scale use representation appropriate for that scaleAt each scale use representation appropriate for that scale

At intermediate scalesAt intermediate scalesRepresent mediallyRepresent mediallySense at (implied) boundarySense at (implied) boundary

Papers at midag.cs.unc.edu/pubs/papers

Extensions

Variable topologyjump diffusion (local shape)level set?

Active Appearance Modelsshape and intensity‘explaining’ the imageiterative matching algorithm

Recommended Readings

For deformable sampled boundary models: T Cootes, A Hill, CJ Taylor (1994). Use of active shape models for locating structures in medical images. Image & Vision Computing 12: 355-366.

For deformable parametrized boundary models: Kelemen, Gerig, et al

For m-rep based shape: Pizer, Fritsch, et al, IEEE TMI, Oct. 1999

For 3D deformable m-reps: Joshi, Pizer, et al, IPMI 2001 (Springer LNCS 2082); Pizer, Joshi, et al, MICCAI 2001 (Springer LNCS 2208)

Recommended Readings

For Procrustes, landmark based deformation (Bookstein), shape space (Kendall): especially understandable in Dryden & Mardia, Statistical Shape Analysis

For iterative conditional posterior, pixel primitive based shape: Grenander & Miller; Blake; Christensen et al