![Page 1: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/1.jpg)
Shaping up! Introduction into Shape Analysis
Guido Gerig
University of Utah
SCI Institute
Monreale Cathedral: Pavement
![Page 2: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/2.jpg)
Shape
The word “shape” is very commonly used in everyday language, usually referring to the geometry of an object.
School performance test
![Page 3: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/3.jpg)
Concept of shape is not new
![Page 4: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/4.jpg)
Shape and Human Vision “Our Visual world contains a vast arrangement of objects, yet we are amazingly robust in recognizing them. This includes objects projected from novel viewpoints, or partially occluded objects. We are even able to describe totally unfamiliar objects, or to recognize unexpected ones out of context.”
What aspect of the geometry should be computed to allow robust recognition? Formal Definition? Theory of Shape?
Kimia, Tannenbaum, Zucker, IJCV 1995
![Page 5: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/5.jpg)
Shape Classification
![Page 6: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/6.jpg)
Application Domains
![Page 8: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/8.jpg)
Shape Statistics: Average? Variability?
![Page 9: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/9.jpg)
Shape Metamorphosis
http://www.artnews.com/2013/02/21/chocolate-self-portraits-by-janine-antoni-and-dieter-rot/
Janine Antoni: Two self portrait busts, 1993 (SFMOMA)
![Page 10: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/10.jpg)
Shape Metamorphosis
http://www.artnews.com/2013/02/21/chocolate-self-portraits-by-janine-antoni-and-dieter-rot/
Janine Antoni, Lick and Lather, 1993-1994 (SFMOMA)
Two self-portrait busts: one chocolate and one soap. Defacing: Washing soap head in bathtub -> erosion, fetal features, like MCF Licking chocolate head -> altering features
![Page 11: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/11.jpg)
Shape Metamorphosis
http://www.artnews.com/2013/02/21/chocolate-self-portraits-by-janine-antoni-and-dieter-rot/
Janine Antoni, Lick and Lather, 1993-1994 (SFMOMA);
Two self-portrait busts: one chocolate and one soap. Defacing: Washing soap head in bathtub -> erosion, fetal features Licking chocolate head -> altering features
![Page 12: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/12.jpg)
Shape Space
A shape is a point in a high-dimensional, nonlinear shape space.
![Page 13: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/13.jpg)
Pedagogy: Goals
• Terms – “Shape Representations”, “Shape Analysis”, “Shape Space” – “Kendal Shape Space” – “SSM”, “PCA”, “PGA”, “ASM”, “AAM” – “Diffeomorphisms”, “Ambient Space”
• Concepts – Correspondences/Landmarks in 2-D and 3-D – Generation of Statistical Shape Models – Use of SSMs for Deformable Model Segmentation – Correspondence-Free Shape Analysis – Statistics of Deformations of Ambient Space: Deformetrics
![Page 14: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/14.jpg)
Contents
• What is Shape? • Geometry Representations • Kendall Shape Space
– Statistical Shape Modeling (SSM) – Correspondences – Active Shape & Appearance Models (ASM, AAM)
• Shape Statistics via Deformations – Correspondence-free Mapping & Stats via “currents” – Ambient Space Deformations via Diffeomorphisms – Statistics of Deformations of Ambient Space
![Page 15: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/15.jpg)
Contents
• What is Shape? • Geometry Representations • Kendall Shape Space
– Statistical Shape Modeling (SSM) – Correspondences – Active Shape & Appearance Models (ASM, AAM)
• Shape Statistics via Deformations – Correspondence-free Mapping & Stats via “currents” – Ambient Space Deformations via Diffeomorphisms – Statistics of Deformations of Ambient Space
![Page 16: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/16.jpg)
What is Shape?
Shape is the geometry of an object modulo position, orientation, and size.
Kendall ‘77, Dryden, Mardia From: Stegmann and Gomez, [Kendall, 1977]
![Page 17: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/17.jpg)
Shape Definition
Dryden/Mardia, (Kendall 1977): Shape is all the geometrical information that remains when location, scale and rotational effects are filtered out from an object.
Image: Sebastian and Kimia 2005
![Page 18: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/18.jpg)
Shape Equivalences
Two geometry representations, 𝑥1, 𝑥2, are equivalent if they are just a translation, rotation, scaling of each other:
where λ is a scaling, R is a rotation, and ν is a translation.
In notation: 𝑥1 ~ 𝑥2
![Page 19: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/19.jpg)
Equivalence Classes
The relationship 𝑥1 ~ 𝑥2 is an equivalence relationship:
• Reflexive: 𝑥1 ~ 𝑥1
• Symmetric: 𝑥1 ~ 𝑥2 implies 𝑥2 ~ 𝑥1
• Transitive: 𝑥1 ~ 𝑥2 and 𝑥2 ~ 𝑥3 imply 𝑥1 ~ 𝑥3
![Page 20: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/20.jpg)
Equivalence Classes
The relationship 𝑥1 ~ 𝑥2 is an equivalence relationship:
• Reflexive: 𝑥1 ~ 𝑥1
• Symmetric: 𝑥1 ~ 𝑥2 implies 𝑥2 ~ 𝑥1
• Transitive: 𝑥1 ~ 𝑥2 and 𝑥2 ~ 𝑥3 imply 𝑥1 ~ 𝑥3
• We call the set of all equivalent geometries to x the equivalence class of 𝑥:
𝑥 = {𝑦 ∶ 𝑦 ~ 𝑥}
![Page 21: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/21.jpg)
Equivalence Classes
The relationship 𝑥1 ~ 𝑥2 is an equivalence relationship:
• Reflexive: 𝑥1 ~ 𝑥1
• Symmetric: 𝑥1 ~ 𝑥2 implies 𝑥2 ~ 𝑥1
• Transitive: 𝑥1 ~ 𝑥2 and 𝑥2 ~ 𝑥3 imply 𝑥1 ~ 𝑥3
• We call the set of all equivalent geometries to 𝑥 the equivalence class of 𝑥:
𝑥 = {𝑦 ∶ 𝑦 ~ 𝑥}
• The set of all equivalence classes is our shape space.
![Page 22: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/22.jpg)
Kendall’s Shape Space
• Define object with 𝑘 points.
• Represent as a vector in ℝ2𝑘.
• Remove translation, rotation, and scale.
• End up with complex projective space, ℂℙ𝑘−2.
![Page 23: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/23.jpg)
Constructing Kendall’s Shape Space
• Consider planar landmarks to be points in the complex plane.
• An object is then a point 𝑧1, 𝑧2 , ⋯ , 𝑧𝑘 ∈ ℂ𝑘.
• Removing translation leaves us with ℂ𝑘−1.
• How to remove scaling and rotation?
![Page 24: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/24.jpg)
Scaling and Rotation in the Complex Plane
Recall a complex number can be
written as 𝑧 = 𝑟𝑒𝑖𝜙, with modulus 𝑟 and argument 𝜙.
Complex Multiplication:
𝑠𝑒𝑖𝜃 ∗ 𝑟𝑒𝑖𝜙 = (𝑠𝑟)𝑒𝑖(𝜃+𝜙)
Multiplication of 𝑧 by a complex number 𝑠𝑒𝑖𝜃 is equivalent to scaling by 𝑠 and rotation by 𝜃.
![Page 25: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/25.jpg)
Removing Scale and Rotation
Multiplying a centered point set, 𝒛 = 𝑧1, 𝑧2, ⋯ , 𝑧𝑘−1 , by a constant 𝑤 ∈ ℂ, just rotates and scales it.
Thus the shape of 𝒛 is an equivalence class:
[𝒛] = {(𝑤𝑧1, 𝑤𝑧2, ⋯ ,𝑤𝑧𝑘−1) ∶ ∀𝑤 ∈ ℂ}
This gives complex projective space ℂℙ𝑘−2.
(Note: centering 1DOF, rotation 2DOF (1 in complex space) → ℂℙ𝑘−2)
![Page 26: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/26.jpg)
Non-Euclidean Shape Space
• Shape Space = complex projective space ℂℙ𝑘−2.
• Shape distance between two objects 𝑧, 𝑤:
![Page 27: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/27.jpg)
Shape Distances …
Ian Dryden: Dryden-Stat-Shape-analysis-2000.pdf
![Page 28: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/28.jpg)
The Problem of Size and Shape
Dryden/Mardia (Kendall 1977): (Sometimes we are also interested in retaining scale information as well as shape) → Size-and-Shape is all the geometrical information that remains when location and rotational effects are filtered out from an object.
![Page 29: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/29.jpg)
Shape Analysis
A shape is a point in a high-dimensional, nonlinear shape space.
![Page 30: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/30.jpg)
Shape Analysis
A metric space structure provides a comparison between two shapes.
![Page 31: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/31.jpg)
Shape Space for Object Class • Linear methods are nice,
but shape space is curved surface: Hyper sphere.
• Standard statistics (𝜇, Σ) not build for hyperspheres.
• Tangent-space projection: Modify shape vectors to form hyper plane.
• Use Euclidean distance in this plane rather than true geodesic distance.
• k landmarks in n Euclidean dimensions: 𝑘𝑛-dim space
• Procrustes alignment: Shape vectors of length dimensionality 𝑘𝑛 normalized for size → 𝐿2 norm
• Vectors lie on subpart of a 𝒌𝒏-dimensional hyper sphere
Stegmann and Gomez, 2002
![Page 32: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/32.jpg)
Structure of Shape Space
Kendal Shape Space: Part of Hypersphere, curved manifold
Shape Space ℝ2𝑚
Assumption SSM: Multivariate Gaussian distribution, linear stats
![Page 33: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/33.jpg)
Shape Space: Tangent-Space Projection
Stegmann and Gomez, 2002
• Project shape vectors to tangent space.
• Apply standard statistics (𝜇, Σ).
• Shown to be good approximation (not much difference) in case of small shape variability.
![Page 34: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/34.jpg)
Shape Space: Tangent-Space Projection
Calculate tangent space, projection to tangent space, linear statistics.
![Page 35: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/35.jpg)
Where to Learn More • Pioneers: Fred Bookstein and David Kendall.
• Bookstein (1991, Cambridge).
• Kendall, Barden and Carne, Shape and Shape Theory, Wiley, 1999.
• Dryden and Mardia, Statistical Shape Analysis, Wiley, 1998.
• Small, The Statistical Theory of Shape, Springer-Verlag, 1996.
• Grenander, HISTORY AS POINTS AND LINES, 1998-2003
• Lele and Richstmeier (2001, Chapman and Hall).
• Krim and Yezzi, Statistics and Analysis of Shapes, Birkhauser, 2006.
![Page 36: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/36.jpg)
Given two points on the hypersphere, we can draw the plane containing these points and the origin.
DF
r
DP
r
Procrustes Distances is r.
DP = 2 sin ( r/2)
DF = sin r.
• These are all monotonic in r. So the same choice of rotation minimizes all three.
• DF is easy to compute, others are easy to compute from DF.
![Page 37: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/37.jpg)
Why Procrustes Distance?
• Procrustes distance is most natural. Our intuition is that given two objects, we can produce a sequence of intermediate objects on a ‘straight line’ between them, so the distance between the two objects is the sum of the distances between intermediate objects. This requires a geodesic.
![Page 38: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/38.jpg)
Tangent Space
• Can compute a hyperplane tangent to the hypersphere at a point in preshape space.
• Project all points onto that plane. • All distances Euclidean. Average shape easy
to find. • This is reasonable when all shapes similar. • In this case, all distances are similar too.
– Note that when r is small, r, 2sin(r /2), sin(r) are all similar.
![Page 39: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/39.jpg)
Contents
• What is Shape? • Geometry Representations • Kendall Shape Space
– Statistical Shape Modeling (SSM) – Correspondences – Active Shape & Appearance Models (ASM, AAM)
• Shape Statistics via Deformations – Correspondence-free Mapping & Stats via “currents” – Ambient Space Deformations via Diffeomorphisms – Statistics of Deformations of Ambient Space
![Page 40: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/40.jpg)
Geometry Representations
![Page 41: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/41.jpg)
Geometry Representations
• Landmarks (key identifiable points)
• Boundary models (points, curves, surfaces, level sets)
• Interior models (medial, solid mesh)
• Transformation models (splines, diffeomorphisms)
![Page 42: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/42.jpg)
Boundary via Landmarks
Stegmann and Gomez, 2002
![Page 43: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/43.jpg)
Boundary versus Skeleton
Shape Representation:
– Contour / Boundary / Surface
– Skeleton (medial model)
radius
skeleton shape
![Page 44: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/44.jpg)
Skeleton Shape Representation
Sensitivity of curve matching to spatial arrangement and how shock graph matching avoids the problem.
Sebastian and Kimia, Signal Processing, 2005
Shock Grammar, Symmetry Maps and Transforms For Perceptual Grouping and Object Recognition, Benjamin B. Kimia, Brown
![Page 45: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/45.jpg)
Shock Graph: Shape Transformation
Matching dog to cat via shock graph editing
Sebastian and Kimia, Signal Processing, 2005
Invariance of shock graph to flexibly deformable objects.
![Page 46: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/46.jpg)
Spherical Harmonics (SPHARM)
),(
),(
),(
),(
z
y
x
r
),(),(0
fqfq å å= -=
=K
k
k
km
m
k
m
k Ycr÷÷÷
ø
ö
ççç
è
æ
=m
zk
m
yk
m
xk
m
k
c
c
c
c
1
10
3
6
1. Extract voxel surface 2. Area preserving parameterization 3. First order ellipsoid alignment 4. Fit SPHARM to coordinates 5. Sample parameterization and
reconstruct object
),(
),(
),(
),(
z
y
x
r
Szekely, Kelemen, Brechbuehler, Gerig, MedIA 1996
![Page 47: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/47.jpg)
Medial Axis / Skeletal Representation: Intrinsic Shape Model
S-rep: Prostate s-rep and implied boundary: Pizer et al. (discrete)
CM-rep: Yushkevich (continuous, parametric) Yushkevich et al., TMI 2006
Gorczowski , Pizer, Gerig et al., T-PAMI 2010, Stats on deformations vs. thickness
![Page 48: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/48.jpg)
3D Shape Representations
SPHARM
Boundary, fine
scale, parametric
PDM
Boundary, fine
scale, sampled
),(),(0
k
k
km
m
k
m
k Ycr
Skeleton
Medial, fine
scale, sampled
Skeleton from
boundary points
M-rep
Medial, coarse
scale, sampled
Implied Surface
,,, Frxm
![Page 49: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/49.jpg)
CM-rep
u
v
ξ u
v
ξ
Sun et al. 2007, IEEE TMI
![Page 50: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/50.jpg)
Level-Set Formulation: Shapes as signed distance functions
• Embed shape contour as 0-level set
• Calculate Euclidean distance transform.
• Contour represented as image with embedded set of signed distance functions.
Leventon, Grimson, Faugeras, CVPR 2000
![Page 51: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/51.jpg)
Volumetric Laplace Spectrum
• “Shape DNA”: Fingerprint, Signature
• Laplace-Beltrami Spectrum
• Global Shape Descriptor
• Voxel object: no registration, no mapping, no re-meshing
Reuter, Niethammer, Bouix, MICCAI 2007
![Page 52: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/52.jpg)
Transformation Models
![Page 53: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/53.jpg)
Contents
• What is Shape? • Geometry Representations • Kendall Shape Space
– Statistical Shape Modeling (SSM) – Correspondences – Active Shape & Appearance Models (ASM, AAM)
• Shape Statistics via Deformations – Correspondence-free Mapping & Stats via “currents” – Ambient Space Deformations via Diffeomorphisms – Statistics of Deformations of Ambient Space
![Page 54: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/54.jpg)
Statistical Shape Analysis
• What is the mean of these shapes?
• Quantify variability
• Quantify individuals relative to population
• Hypothesis testing
• Regression
![Page 55: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/55.jpg)
Modelling Shape
• Define each example using points
• Each (aligned) example is a vector
1
2
3
4
5
6
xi = {xi1 , yi1 , xi2 , yi2 …xin , yin}
Cootes, Taylor 1993
![Page 56: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/56.jpg)
SSM: Point Distribution Model
Example of shape configuration (left) and the configuration matrix (right) for a set of hand shapes.
Cootes, Taylor 1993
![Page 57: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/57.jpg)
Modelling Shape Variability
Observation/Assumption: Points in shape population tend to move in correlated ways.
x1
x2
b1
xi
Cootes, Taylor 1993
𝒙
![Page 58: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/58.jpg)
Shape Alignment
Stegmann and Gomez, 2002
![Page 59: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/59.jpg)
SSM and Shape Space
iS
Shape Space ℝ2𝑚
![Page 60: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/60.jpg)
SSM and Shape Space: Correlation
iS
Shape Space ℝ2𝑚
![Page 61: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/61.jpg)
Capturing the statistics of a set of aligned shapes
• Find mean shape
• Find deviations from the mean shape
• Find covariance matrix
• Find eigenvalues/vectors of S
• Modes of variation defined by eigenvectors
1
1
1
1
1
k
T
k
kkk
N
i
T
ii
ii
N
i
i
pp
pSp
dxdxN
S
xxdx
xN
x
![Page 62: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/62.jpg)
Hand Model
Modes of shape variation
b1b 2bb1 b2 b
Courtesy of Chris Taylor, Cootes & Taylor 1992
![Page 63: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/63.jpg)
Landmark Variability & Correlation Matrix
Stegmann and Gomez, 2002
![Page 64: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/64.jpg)
Description in the Shape Space
• The modes of variation of the points of the shape are described by the eigenvectors of S:
• Each shape is described by its weight vector b.
• The eigenvectors corresponding to the largest eigenvalue describe the most significant modes of variation in the training data.
)( xxPb
Pbxx
Pbxx
T
![Page 65: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/65.jpg)
Shape Eigenbasis
![Page 66: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/66.jpg)
Synthetic Shapes: Translation
8*8 Correlation Matrix [(x1,y1),(x2,y2),…,x4,y4)]
Input: Scaling/Translation
PC 1
PC 2
eigenvalues
![Page 67: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/67.jpg)
Synthetic Shapes: Translation
8*8 Correlation Matrix [(x1,y1),(x2,y2),…,x4,y4)]
Input: Scaling/Translation
eigenvalues
PC 1
PC 2
![Page 68: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/68.jpg)
Synthetic Shape: Rotation?
8*8 Correlation Matrix [(x1,y1),(x2,y2),…,x4,y4)].
Input: Rotation of square by 80 deg.
eigenvalues
PC 1
PC 2
![Page 69: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/69.jpg)
Synthetic Shape: Rotation (80deg)
PC 1
PC 2
PC 1
PC 2
PCA in ℝ𝑛generates linear subspaces 𝑉𝑘 that maximize the variance of the projected data.
![Page 70: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/70.jpg)
Statistics in Shape Space
-3σ +3σ mean
1
2
3 m
ode
PCA modes visualization
• Manual/automatic correspondences
• Gaussian models
• PCA for dimensionality in shape space
Shape Space ℝ2𝑚 PCA 1
![Page 71: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/71.jpg)
Summary Concept
Compression/Feature selection: Project high dimensional measures into low-dimensional space of largest variability, few features → Statistics
![Page 72: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/72.jpg)
Example: Corpus Callosum Study
![Page 73: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/73.jpg)
Boundary PCA
Eigenvalues: 95% of deformation energy is in the first 10 principal eigenmodes, and the first 2 represent 65% of the variation.
PCA 1
PCA 2
PCA 3
![Page 74: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/74.jpg)
PCA Shape Space: Corpus Callosum Study PC 1,2
PC 3,4
Mean CC shapes: 6mo, 12mo, 24mo
1040 infant CC shapes
ACE-IBIS Autism Study, UNC J. Piven
![Page 75: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/75.jpg)
Generalizing PCA: Principal Geodesic Analysis
Fletcher et al., IEEE TMI 2004
![Page 76: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/76.jpg)
The Exponential Map • We represent shapes as points on a
manifold, rather than as points in Euclidean space.
• Log map: Function that computes a geodesic from two points on the manifold, representing the shortest path on the manifold between two points: 𝑑 𝑥, 𝑦 = 𝐿𝑜𝑔𝑥 𝑦 =log (𝑥−1𝑦) .
• Exponential map: Function that computes points on the manifold from a base point and a vector in the tangent space: 𝐸𝑥𝑝𝑥 𝑣 =𝑥𝑒𝑥𝑝 𝑦 .
![Page 77: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/77.jpg)
Intrinsic Means (Fréchet)
The intrinsic mean of a collection of points 𝑥1⋯𝑥𝑁 in a metric space 𝑀 is
where 𝑑(. , . ) denotes distance in 𝑀.
![Page 78: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/78.jpg)
PGA
• PGA is the natural generalization of PCA to a manifold space.
• Covariance matrix is constructed with the tangent vectors at the Fréchet mean (vectors 𝑣𝑖).
• Fréchet mean: No closed form solution in this space, iterative procedure:
![Page 79: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/79.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 80: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/80.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 81: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/81.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 82: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/82.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 83: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/83.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 84: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/84.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 85: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/85.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 86: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/86.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 87: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/87.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 88: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/88.jpg)
Computing Means
Fletcher et al., IEEE TMI 2004
![Page 89: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/89.jpg)
Comparison PCA-PGA
![Page 90: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/90.jpg)
Comparison PCA-PGA
PCA with Procrustes
PGA
Discussion: • Qualitatively slightly different but no obvious major differences. • Details are in the math: PGA guarantees by definition rigid invariance (rotation, scale),
PCA after Procrustes shows slight amount of scale differences but none for rotation.
![Page 91: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/91.jpg)
Non Unimodal Shape Space: Gaussian Mixture Model
A Mixture Model for Representing Shape Variation, Cootes et al., IVC 1999
![Page 92: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/92.jpg)
Towards Robust Statistics on Shapes
Input Data: 18 Hand Outlines (Cootes & Taylor)
Outliers: random ellipses
Sarang Joshi, Utah
Example: Complex Projective Kendal Shape Space
![Page 93: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/93.jpg)
Towards Robust Statistics on Shapes
0 outliers 2 outliers 6 outliers 12 outliers
Mean
Median
Sarang Joshi, Utah
![Page 94: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/94.jpg)
Landmarks / Homologous Points
• A landmark is an identifiable point on an object that corresponds to matching points on similar objects.
• This may be chosen based on the application (e.g., by anatomy) or mathematically (e.g., by curvature).
![Page 95: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/95.jpg)
Landmarks ctd.
• Anatomical landmarks are points assigned by an expert that corresponds between objects of study in a way meaningful in the context of the disciplinary context.
• Mathematical landmarks are points located on an object according some mathematical or geometrical property, i.e. high curvature or an extremum point.
• Pseudo-landmarks Constructed points on an object either on the outline or between landmarks.
[Dryden & Mardia]
![Page 96: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/96.jpg)
Landmark Correspondence
Homology: Corresponding (homologous) features on skull images.
![Page 97: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/97.jpg)
Correspondences and Shape
• The choice matters – Defines the shape space
• Manual landmarks – Not practical
– 3D, not clear
– User error
• Need: automatic 2D/3D correspondence placement – Computational concept?
![Page 98: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/98.jpg)
“Good” and “Bad” Correspondence
“Good” placement: • Reduced variability. • May lead to better,
more compact statistical shape models.
Left: Arc-length parametrization
Right: Manual placement of corresponding landmarks
From: PhD thesis Rhodri Davies
![Page 99: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/99.jpg)
Spherical Harmonics: Correspondence via Parametrization
Szekely, Kelemen, Brechbuehler, Gerig, MedIA 1996
![Page 100: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/100.jpg)
Correspondence: SPHARM
• Correspondence by same parameterization – Area ratio preserving through optimization
– Location of meridian and equator ill-defined
• Poles and Axis of first order ellipsoid
• Object specific, independent, but sensitive to objects with rotational symmetry/ambiguity
Surface
Parametrization
SPHARM
![Page 101: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/101.jpg)
Correspondence and quality of shape model
Manual placement Arc-length parametrization
From: PhD thesis Rhodri Davies
![Page 102: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/102.jpg)
Optimization of Correspondence: Reparametrization
Rhodri Davies, 2000
![Page 103: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/103.jpg)
Optimization of Correspondence: Reparametrization
Rhodri Davies, 2000, Springer 2008
![Page 104: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/104.jpg)
Correspondence Depends on the Population
• Image warping based on local/nearest differences
• Alternative: take into account the trends in the ensemble
– Davies et al. 2000 (MDL)
– Particle entropy (Whitaker, Cates, 2011,12)
– Unbiased atlas building (Joshi, Davis, 2004)
![Page 105: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/105.jpg)
Group-wise Approaches
• Use whole set of objects to determine correspondence via optimal group stats
– Can be applied both to parametric & non-parametric descriptions
• Advantages:
– No template bias
– Represent all objects in a population, not just those close to the mean
– Expect higher reliability, lower variance
– Expect higher statistical sensitivity
![Page 106: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/106.jpg)
Correspondence as Optimization • Pairwise mapping of curves
• Search space: all feasible correspondences
• Objective function on quality of correspondence
• Use trend of ensemble: Optimize over population in shape space.
• Re-parameterisation function for each shape. – valid correspondences
diffeomorphic mapping
Rhodri Davies, Chris Taylor, MDL, PMI 2003 Hemant Tagare, IPMI 1997
![Page 107: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/107.jpg)
MDL: The Objective Function
• Simplest Model has minimum stochastic complexity → Information Theory
• Minimum Description Length (MDL) • Transmit training set as encoded message
– parameters of model, encoded data
• Model complexity vs. fit to data
Rhodri Davies, Chris Taylor, MDL, IPMI 2003
![Page 108: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/108.jpg)
Ensemble Correspondence: Evaluation Criteria • Generalization: Ability to describe instances outside of the
training set – leave-one-out – approximation error
• Specificity: Ability to represent only valid instances of the
object – generate new sample – distance to nearest training member
• Compactness: Ability to use a minimal set of parameters
– cumulative variance
i
ii
s
Mn
MG 2/ |)(|1
)( xx
N
j
jj MN
MS 2/ |)(|1
)( xx
M
m
mMC )(
![Page 109: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/109.jpg)
Evaluation: MDL vs. SPHARM
Hippocampal Shapes 82 samples
Generalization: Leave one out
Specificity: Generate new samples, distance to nearest member
Compactness: Cumulative Variance Rhodri Davies, Chris Taylor, MDL, PMI 2003 Styner.,.., Davies, IPMI 2003
![Page 110: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/110.jpg)
• Minimize shape entropy (simple, compact description)
• Linear model -> minimize log of determinant of – Equivalence of MDL – Davies et al. 2002, Thodberg 2003
• Issues – Numerical regularization
• Small modes dominate
– Degenerate solutions • Parameterization favors points where shape overlap
Shape space Compact/simple
shape space
Geometrically accurate on
surfaces
R. Whitaker, J. Cates, Uah
Modeling a Shape Ensemble: Strategy for Landmark Placement
![Page 111: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/111.jpg)
Particle-Based Shape Correspondences
• Shapes as a set of interacting particle systems • Compact models, but balanced against geometric
accuracy (good, adaptive samplings) • Optimize particle positions by minimizing an entropy
cost function
Entropy of the shape ensemble
Entropy of each individual shape
sampling
R. Whitaker, J. Cates, M. Datar, IPMI 2007, MICCAI 2013
Low entropy High entropy
![Page 112: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/112.jpg)
Entropy-based Particle Systems
• Surfaces are discrete point sets, no parameterization
• Dynamic particles, positions optimize the information of the system: ensemble entropy, surface entropy
Low surface entropy
High surface entropy
Images: Oguz, 2009 low is better high is better
![Page 113: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/113.jpg)
Particle Correspondence Model C
onfig
urat
ion
Spa
ce
Sha
pe S
pace
Accurate Representation (in Configuration Space)
vs.
Compact Model (in Shape Space)
Surface Entropy Ensemble Entropy
Cates et al. IPMI 2007, Datar MICCAI 2011
![Page 114: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/114.jpg)
Modeling Head Shape Change
Link to Movie
Changes in head size with age
Changes in head shape with age
Datar, Cates, Fletcher, Gouttard, Gerig, Whitaker, Particle-based Shape Regression, MICCAI 2009
![Page 115: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/115.jpg)
Box-Bump Comparison with MDL
– 24 shapes
– MDL: 128 nodes, mode 2, parameters at default*
– Particle: 100 particles per shape
* See Thodberg, IPMI 2003 for details
Results
Single major mode of variation
MDL: 0.34% “leakage” of total variation to minor modes
Particle: 0.0015% leakage
Cates & Whitaker, IPMI 2007
![Page 116: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/116.jpg)
Graph Spectra/Laplacian
Courtesy Hervé Lombaert, Jon Sporring, Kaleem Siddiqi, Mc Gill, IPMI 2013
![Page 117: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/117.jpg)
Graph Spectra/Laplacian
Courtesy Hervé Lombaert, Jon Sporring, Kaleem Siddiqi, Mc Gill, IPMI 2013
![Page 118: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/118.jpg)
Considering Appearance: Eigenfaces
Sirovich & Kirby 87, Turk & Pentland 91 Source: Iasonas Kokkinos, IPAM-UCLA Course 2013
![Page 119: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/119.jpg)
Eigenfaces
Sirovich & Kirby 87, Turk & Pentland 91 Source: Iasonas Kokkinos, IPAM-UCLA Course 2013
![Page 120: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/120.jpg)
Eigenfaces
Sirovich & Kirby 87, Turk & Pentland 91 Source: Iasonas Kokkinos, IPAM-UCLA Course 2013
![Page 121: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/121.jpg)
Eigenfaces
Sirovich & Kirby 87, Turk & Pentland 91 Source: Iasonas Kokkinos, IPAM-UCLA Course 2013
![Page 122: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/122.jpg)
Eigenfaces
Sirovich & Kirby 87, Turk & Pentland 91 Source: Iasonas Kokkinos, IPAM-UCLA Course 2013
![Page 123: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/123.jpg)
Active Shape and Appearance Models
• Statistical models of shape and texture
• Generative models
– general
– specific
– compact (~100 params)
Courtesy of Chris Taylor, 1995
![Page 124: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/124.jpg)
Building an Appearance Model
• Labelled training images
– landmarks represent correspondences
Courtesy of Chris Taylor, 1995
![Page 125: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/125.jpg)
Building an Appearance Model
• For each example
Shape: x = (x1,y1, … , xn, yn)T
Texture: g Warp to mean shape
Raster Scan
Courtesy of Chris Taylor, 1995
![Page 126: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/126.jpg)
Building an Appearance Model
• Principal component analysis
• Columns of Pr form shape and texture bases
• Parameters br control modes of variation Courtesy of Chris Taylor, 1995
![Page 127: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/127.jpg)
Shape and Texture Modes
Shape variation (texture fixed)
Texture variation (shape fixed) Courtesy of Chris Taylor, 1995
![Page 128: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/128.jpg)
Combined Appearance Model
• Shape and texture may be correlated
– PCA of s
g
b
b
Varying appearance vector c Courtesy of Chris Taylor, 1995
![Page 129: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/129.jpg)
Colour Appearance Model
c1 c2 c3
Courtesy of Chris Taylor, 1995
![Page 130: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/130.jpg)
AAM Search – Deformable Automatic Segmentation
Slide Credit: G. Lang Source: Iasonas Kokkinos, IPAM-UCLA Course 2013
![Page 131: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/131.jpg)
Active Shape Model Search
Method: Cootes et al., 1995 Slide: T. Heimann: - Shape Symposium 2014, Delémont
![Page 132: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/132.jpg)
3D Hippocampus: ASM & AAM Modeling for Deformable Segmentation
Profiles normal to surface
capture local image intensity
function (hedgehog)
Szekely, Kelemen, Brechbuehler, Gerig, MedIA 1996 Styner & Gerig, MedIA 2004
![Page 133: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/133.jpg)
Appearance Profiles across 10 training images
Styner & Gerig, UNC
![Page 134: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/134.jpg)
Inside Outside Inside Outside
Appearance Profiles
Styner & Gerig, UNC
![Page 135: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/135.jpg)
Dual shape representations: PDM/SPHARM
Styner & Gerig, UNC
![Page 136: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/136.jpg)
Computing the fit
Szekely, Kelemen, Styner & Gerig, ETH, UNC
![Page 137: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/137.jpg)
Deformation Forces and Constraints Driving deformation force at boundary points 𝑥𝑖:
• Start with mean shape.
• Correlate local boundary appearance with statistical model.
• Find suggested “shift” for each point 𝑥𝑖 → 𝑑𝑥𝑖.
• Convert 𝑑𝑥 into shift in shape space 𝑑𝑏 → shape change.
Shape constraints:
• Ensure that 𝑑 + 𝑑𝑏 stays within predefined Mahalanobis distance of shape space.
![Page 138: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/138.jpg)
Deformable Model Segmentation
Segmentation of corpus callosum via deformable model segmentation, max order 10 (40 coeffs)
Fourier Descriptors of Shape Contours Paper-Kuhl-Giardina-1982, Paper-Staib-Duncan-1992
Fig. 1: Visualization of 3 MRI mid-hemispheric slices and the final positions (in red) of the automatic corpus callosum segmentation algorithm using deformable shape models.
Styner & Gerig, UNC
![Page 140: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/140.jpg)
• Comparison of constrained and unconstrained AAM search
• Conclusions: Cannot directly handle cases well outside of the training set (e.g. occlusions, extremely deformable objects)
Courtesy of Chris Taylor
Constrained AAMs
![Page 141: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/141.jpg)
Non Unimodal Shape Space: Gaussian Mixture Model
A Mixture Model for Representing Shape Variation, Cootes et al., IVC 1999
![Page 142: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/142.jpg)
Variations of SSM for Segmentation Geodesically Damped Shape Models (Christoph Jud, Thomas Vetter, 2014) • SSM training … too restrictive …, new method for model bias reduction…,
achieved by damping the empirical correlations between points on the surface which are geodesically wide apart.
• Yields locally more flexibility of the model and a better overall segmentation performance.
Jud et al., Proceedings, Shape Symposium 2014, Delémont
![Page 143: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/143.jpg)
Advanced AAMs close to the Clinic
Slide: Yoshinobu Sato: - Shape Symposium 2014, Delémont
![Page 144: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/144.jpg)
Advanced AAMs close to the Clinic
Slide: Yoshinobu Sato: - Shape Symposium 2014, Delémont
![Page 145: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/145.jpg)
Alternative to PCA: Multi-affine
Slide: Christian Lorenzen: - Shape Symposium 2014, Delémont
![Page 146: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/146.jpg)
Alternative to PCA: Multi-affine
Slide: Christian Lorenzen: - Shape Symposium 2014, Delémont
![Page 147: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/147.jpg)
EM-based AAM learning
Source: Iasonas Kokkinos, IPAM-UCLA Course 2013
![Page 148: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/148.jpg)
Contents
• What is Shape? • Geometry Representations • Kendall Shape Space
– Statistical Shape Modeling (SSM) – Correspondences – Active Shape & Appearance Models (ASM, AAM)
• Shape Statistics via Deformations – Correspondence-free Mapping & Stats via “currents” – Ambient Space Deformations via Diffeomorphisms – Statistics of Deformations of Ambient Space
![Page 149: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/149.jpg)
Correspondence-free Shape Analysis
Brain ventricles for infants 6mo to 2yrs DTI Fiber Tracts from two subjects
movie
Durrleman, Pennec, Trouve, Ayache et al., IJCV 2013
Problems: • Correspondence depends on shape parameterization • Shapes with variable topology: correspondence undefined
![Page 150: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/150.jpg)
Correspondence-free similarity measures
• Depends on the kind of objects: • Images: sum of squared differences • Landmarks: sum of squared differences • Surface mesh and curves:
• Currents [Glaunès’05]
• No point correspondence needed • Efficient numerical schemes (FFT) • Robust to changes in topology
• Robust to differences in • mesh sampling • mesh imperfections…
Usable routinely on large data sets
: tangents of curves/normals of surfaces
![Page 151: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/151.jpg)
“Correspondence-free” Registration: Currents
Topology and shape differences and noise can make point-to-point correspondence hard:
• Currents: Objects that integrate vector fields
• Shape: Oriented points = Set of normals (tangents)
• Distance between curves:
[Glaunes2004] Glaunes, J., Trouve, A., Younes, L. Diffeomorphic matching of distributions: a new approach, … CVPR 2004.
[Durrleman2008] S. Durrleman, X. Pennec, A. Trouvé, P. Thompson, N. Ayache, Inferring Brain Variability from Diffeomorphic Deformations of Currents: an integrative approach, Medical Image Analysis 2008
![Page 152: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/152.jpg)
Currents integrate vector field: •W: test space of vector fields (Hilbert space) •W*: the space of continuous maps W->R •W* includes smooth curves, polygonal lines, surfaces, meshes.
[Vaillant and Glaunès IPMI’05, Glaunès PhD’06]
![Page 153: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/153.jpg)
Durrleman, PhD thesis, 2010
![Page 154: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/154.jpg)
Durrleman, PhD thesis, 2010
![Page 155: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/155.jpg)
Durrleman, PhD thesis, 2010
![Page 156: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/156.jpg)
Durrleman, PhD thesis, 2010
![Page 157: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/157.jpg)
The space of currents: a vector space
Distance between shapes: • No point correspondence
• No individual line correspondence
• Robust to line interruption
• Need consistent orientation of lines/surfaces
• Is a norm
• Addition = union • Scaling = weighting different structures • Sign = orientation
Durrleman, PhD thesis, 2010, JCV 2013
![Page 158: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/158.jpg)
Limitations of Kendall Shape Space
• Shape Space depends on correspondence & parametrization.
• Correspondence still an issue, not defined for shapes with varying topology/resolution etc.
• Statistics on “precise” high-dim (often oversampled) descriptions of shape rather than deformations.
• (PCA Problem: Cannot handle cases well outside of the training set (e.g. occlusions, highly deformable objects).)
![Page 159: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/159.jpg)
Critical Assessment
Stegmann and Gomez, 2002
![Page 160: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/160.jpg)
Highly Recommended Reading
D’Arcy Wentworth Thompson, On Growth and Form (1917, mathematics and biology)
http://archive.org/download/ongrowthform00thom/ongrowthform00thom.pdf http://ia700301.us.archive.org/10/items/ongrowthform00thom/ongrowthform00thom.pdf
![Page 161: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/161.jpg)
Shape Spaces: Kendall vs. Deformations
Kendall Shape Space:
• We are interested in the way the points of a shape move (or displace), but there is no general concept of a deformation -- analysis is based on the parameterization of the shape.
• Shape Space forms a complex projective space ℂℙ𝑘−2.
![Page 162: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/162.jpg)
Shape Spaces: Kendall vs. Deformations
Kendall Shape Space:
• We are interested in the way the points of a shape move (or displace), but there is no general concept of a deformation -- analysis is based on the parameterization of the shape.
• Shape Space forms a complex projective space ℂℙ𝑘−2.
D‘Arcy Thompson inspired deformation based analysis:
• Interested in the way the ambient space deforms.
• Statistical analysis is centered on the deformations of space, not movement & displacement of points on shapes.
• What is the Shape Space? Information via deformations.
![Page 163: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/163.jpg)
Shape Analysis via Transformations
D’Arcy Wentworth Thompson, On Growth and Form (1917, mathematics and biology)
D'Arcy Thompson introduced the Method of Coordinates to accomplish the process of comparison.
![Page 164: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/164.jpg)
Biological variation through mathematical transforms D'Arcy Thompson laid out his vision in his treatise “On Growth and Form“. In 1917 he wrote:
In a very large part of morphology, our essential task lies in the comparison of related forms rather than in the precise definition of each; and the deformation of a complicated figure may be a phenomenon easy of comprehension, though the figure itself may be left unanalyzed and undefined."
![Page 165: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/165.jpg)
Even earlier…
http://commons.wikimedia.org/wiki/File:Durer_face_transforms.jpg
Albrecht Dürer (1471-1528): German painter, printmaker, engraver and mathematician. Studies of human proportions.
Face transformations by Albrecht Dürer
http://commons.wikimedia.org/wiki/Albrecht_Durer
![Page 166: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/166.jpg)
Ambient Space Deformation
Change in geometric entities in images represented as transformations of the underlying coordinate grid.
Nikhil Singh, PhD thesis Utah 2013
![Page 167: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/167.jpg)
Ambient Space Deformation
Initial velocity* as a smooth vector field and the corresponding diffeomorphic flow that transforms the shape “plus" to “flower". *velocity: momenta after convolution with kernel
Nikhil Singh, PhD thesis Utah 2013
![Page 168: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/168.jpg)
Concept of Diffeomorphism
Diffeomorphisms: • one-to-one onto (invertible) and differential
transformations • preserves topology
Slide courtesy Sarang Joshi
![Page 169: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/169.jpg)
Large Deformation Diffeomorphic Metric Mapping (LDDMM)
• Space of all Diffeomorphisms forms a group under composition:
• Space of diffeomorphisms not a vector space.
• Small deformations, or “Linear Elastic” registration approaches, ignore these two properties.
)(Diff
)(:)(, 2121 DiffhhhDiffhh
)(:)(, 2121 DiffhhhDiffhh
Slide courtesy Sarang Joshi
![Page 170: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/170.jpg)
Large deformation diffeomorphisms.
• infinite dimensional “Lie Group”. • Tangent space: The space of smooth vector
valued velocity fields on . • Construct deformations by integrating flows of
velocity fields. • Induce a metric via a differential norm on
velocity fields.
)(Diff
d d t
h ( x ; t ) = v ( h ( x ; t ) ; t ) h ( x ; 0 ) = x :
![Page 171: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/171.jpg)
Construction of Diffeomorphisms Diffeomorphisms: • Construct deformations by integrating
flows of velocity fields. • Induce a metric via a differential norm
on velocity fields. • Distance btw. two diffeomorphisms: 𝐷 ℎ1, ℎ2 = 𝐷 𝑒, ℎ1
−1°ℎ2 (metric).
Miller, Christensen, Joshi / Joshi et al., Neuroimage 2004
![Page 172: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/172.jpg)
Ambient Space Deformation
Momenta and the corresponding diffeomorphic flow that transforms the shape “plus" to “flower".
Nikhil Singh, PhD thesis Utah 2013 James Fishbaugh, GSI conference, Springer 20014
![Page 173: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/173.jpg)
Momenta and Statistics
• The momenta field plays the role of the tangent vector in the Riemannian sense → Momenta exist in a linear space.
• Analysis of geometrical variability: PCA on the feature vectors of deformations → PCA* by computing mean and covariance matrix of momenta.
(*kernel PCA for currents)
Miller, Trouve, Younes, On the metrics and euler-lagrange equations of computational anatomy. Annu Rev Biomed Eng.2002
Durrleman et al., NeuroImage 2010
![Page 174: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/174.jpg)
Geodesic Flow: Initial Momenta
momenta velocity Fishbaugh, Durrleman, Gerig, MICCAI 2011, 2012, 2013
diffeomorphic registration
![Page 175: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/175.jpg)
Flows of diffeomorphisms are geodesic → initial momenta parameterize deformation.
Group A Group B
Reference Atlas
ti
ti
ti
ti
ti
ti ti
ti
ti
Statistics on Deformations
Fishbaugh, Durrleman, Gerig, MICCAI 2012
![Page 176: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/176.jpg)
Flows of diffeomorphisms are geodesic → initial momenta parameterize deformation. Geodesics from atlas to each subject share the same tangent space, so we can perform linear operations on the momenta, such as computing the mean and variance.
Statistics on Deformations
Fishbaugh, Durrleman, Gerig, MICCAI 2012
Group A Group B
Reference Atlas
ti
ti
ti
ti
ti
ti ti
ti
ti
![Page 177: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/177.jpg)
First mode of deformation from PCA per age group, explaining the variability of each group w.r.t. the normative reference atlas.
. Hypothesis testing → no significant differences in magnitude of initial momenta
Clinical Application: Autism
![Page 178: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/178.jpg)
Momenta and Statistics
Durrleman et al., NeuroImage 2010
![Page 179: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/179.jpg)
Momenta and Statistics
Durrleman et al., NeuroImage 2010
![Page 180: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/180.jpg)
Momenta and Statistics
Durrleman et al., NeuroImage 2010
![Page 181: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/181.jpg)
Statistics on Deformations
Most discriminative deformation axis between Down’s and Controls.
Durrleman et al, Neuroimage 2014
Geodesics from atlas to each subject share the same tangent space. Momentum vectors of DS subjects (red) and controls (blue) in atlas coordinate space.
![Page 182: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/182.jpg)
Statistics on Deformations
Most discriminative deformation axis between Down’s Syndrome and Controls. Durrleman et al, Neuroimage 2014
![Page 183: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/183.jpg)
Deformetrics with Sparsity: Tackling fundamental problem of high-dim features & low-dim sample size (HDLSS)
Image evolution described by considerably fewer parameters, Concentrated in areas undergoing most dynamic changes
Durrleman, 2013 / Fishbaugh IPMI 2013, GSI 2013
Initialization Atlas
![Page 184: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/184.jpg)
Statistics: Ambient Space Deform.
Common template for 8 Down’s syndrome patients + 8 Ctrls
Most discriminative axis
Classification (leave-2-out) with 105 control points:
specificity sensitivity
Max Likelihood 100% (64/64) 100% (64/64)
LDA 98% (63/64) 100% (64/64)
Importance of optimization in control points positions!
Classification (leave-2-out) with 8 control points:
specificity sensitivity
Max Likelihood 97% (62/64) 100% (64/64)
LDA 94% (60/64) 89% (57/64)
![Page 185: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/185.jpg)
Deformation of Ambient Space
Main advantages:
• Shape space independent on shape representation.
• Natural way to handle multiple shapes, topology variations, combinations of points, lines, contours, image intensity etc.
• Statistics on low #features rather than high-dimensional oversampled shape representation.
![Page 186: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/186.jpg)
Mean and Variability
High-dimensional space: • Variances and covariances • Non-Euclidean geometry • Statistics on tangent spaces
Singh, Fletcher, Joshi et al., ISBI 2013, best paper award
![Page 187: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/187.jpg)
Normative Atlas of 4D Trajectories: Work in Progress
Group differences in rates of longitudinal change/atrophy in AD, MCI and Normal control.
Nikhil Singh et al., ISBI 2013, best paper award
![Page 188: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/188.jpg)
Quotation of the Day
“The perfection of mathematical beauty is such ….. that whatsoever is most beautiful and regular is also found to be the most useful and excellent.”
D’Arcy Wentworth Thompson
![Page 189: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/189.jpg)
Software Resources
http://www.shapesymposium.org/
http://statismo.github.io/statismo/
Keynotes by M. Styner, T. Heimann, Y. Sato, Ch. Lorenz, X. Pennec, G. Gerig
![Page 190: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/190.jpg)
Software Resources
http://www.deformetrica.org/ Paper: Durrleman et al., Neuroimage 2014
![Page 191: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/191.jpg)
Software Resources
http://www.isbe.man.ac.uk/~bim/software/index.html
Tim Cootes, Modeling and Search Software (C++ and VXL)
A set of tools to build and play with Appearance Models and AAMs.
A basic program to experiment with Active Contour Models (snakes).
![Page 192: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/192.jpg)
Conclusions
• “Shape” is a fundamental concept of human perception.
• “Shape Analysis” is still a very active research topic.
• “Shape” is an essential concept for Medical Image Analysis.
• Many methods (SSMs in medicine, face & fingerprint recognition, face indexing,…) have found applications in daily routine.
• Serious mathematical & statistical concepts help to make the field much more mature, but:
• Need to bridge the gap between the “Beauty of Math” and “Biological Shape”.
![Page 193: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/193.jpg)
Acknowledgements • All Research Colleagues & Collaborators contributing to Shape Analysis • NIH-NINDS: 1 U01 NS082086-01: 4D Shape Analysis • NIH-NIBIB: 2U54EB005149-06 , NA-MIC: National Alliance for MIC • NIH (NICHD) 2 R01 HD055741-06: ACE-IBIS (Autism Center) • NIH NIBIB 1R01EB014346-01: ITK-SNAP • NIH NINDS R01 HD067731-01A1: Down’s Syndrome • NIH P01 DA022446-011: Neurobiological Consequences of Cocaine Use
• USTAR: The Utah Science Technology and Research initiative at the Univ. of Utah
• UofU SCI Institute: Imaging Research
• Insight Toolkit ITK
![Page 194: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/194.jpg)
Acknowledgements All research colleagues & collaborators contributing to these collection of slides:
• James Fishbaugh, Utah
• Stanley Durrleman, INRIA, Paris
• Xavier Pennec, INRIA Sophia Antipolis
• Tom Fletcher, Utah
• Sarang Joshi, Utah
• Martin Styner, UNC
• Ross Whitaker, Utah
• and many others ….. (see citations)
![Page 195: Shaping up! Introduction into Shape Analysis · – ^Diffeomorphisms _, ^Ambient Space _ • Concepts –Correspondences/Landmarks in 2-D and 3-D ... • Krim and Yezzi, Statistics](https://reader035.vdocuments.us/reader035/viewer/2022070905/5f759b2645654e08325f5b71/html5/thumbnails/195.jpg)
Are you still in “Good Shape”?
Bob Dylan & the Band: The Shape I'm In