using the inner-distance for classification of articulated shapes
DESCRIPTION
Using the Inner-Distance for Classification of Articulated Shapes. Haibin Ling and David Jacobs Center for Automation Research University of Maryland College Park. Problems: Three toys. Schedule. Related work The inner-distance and its properties - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/1.jpg)
Using the Inner-Distance for Using the Inner-Distance for Classification of Articulated ShapesClassification of Articulated Shapes
Haibin Ling and David JacobsHaibin Ling and David Jacobs
Center for Automation ResearchUniversity of MarylandUniversity of Maryland
College ParkCollege Park
![Page 2: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/2.jpg)
Problems: Three toysProblems: Three toys
![Page 3: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/3.jpg)
ScheduleSchedule
1.1. Related workRelated work
2.2. The inner-distance and its propertiesThe inner-distance and its properties
3.3. Extension of shape context with the Extension of shape context with the inner-distanceinner-distance
4.4. Silhouette matching using dynamic Silhouette matching using dynamic programming based on the new programming based on the new descriporsdescripors
5.5. Experiment resultsExperiment results
![Page 4: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/4.jpg)
Related WorksRelated Works Bending invariant signature for 3D surfaces, Bending invariant signature for 3D surfaces,
[Elad & Kimmel 2003]: geodesic distances + [Elad & Kimmel 2003]: geodesic distances + MDSMDS
Shape context, [Belongie et al. 2002]Shape context, [Belongie et al. 2002] Two categories of methods for handling part Two categories of methods for handling part
structures:structures: Supervised methods explicitly build models for part Supervised methods explicitly build models for part
structures through training. Grimson 1990, structures through training. Grimson 1990, Felzenszwalb and Huttenlocher 2003, Schneiderman Felzenszwalb and Huttenlocher 2003, Schneiderman and Kanade 2004, etc. and Kanade 2004, etc.
Unsupervised methods: do not depend on explicit part Unsupervised methods: do not depend on explicit part models. Basri et al., Siddiqi&etal, Sebastian et al. 04, models. Basri et al., Siddiqi&etal, Sebastian et al. 04, Gorelick et al., etc.Gorelick et al., etc.
![Page 5: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/5.jpg)
The Inner DistanceThe Inner Distance
Given two points Given two points x, yx, y in a shape in a shape OO ( (O O is a is a connected and closed subset of Rconnected and closed subset of R22), the ), the iinner-distancenner-distance between between x,yx,y, denoted as , denoted as d(x,y;O)d(x,y;O), is defined as the length of the , is defined as the length of the shortest path connecting x and y shortest path connecting x and y withinwithin O. O.
Reduced to the Euclidean distance for Reduced to the Euclidean distance for convex object.convex object.
Affected by concavity of shapes – a hint of Affected by concavity of shapes – a hint of part structure.part structure.
![Page 6: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/6.jpg)
Properties of the Inner-DistanceProperties of the Inner-Distance
Articulation InsensitivityArticulation Insensitivity Articulation invariant for ideal articulated shapesArticulation invariant for ideal articulated shapes For most pairs of points in an articulated shape, the For most pairs of points in an articulated shape, the
relative changes of their inner-distances are very relative changes of their inner-distances are very smallsmall
Capturing part structuresCapturing part structures Difficult to prove: the definition of part structure Difficult to prove: the definition of part structure
remains unclearremains unclear We show this with examples and analysis of We show this with examples and analysis of
experimental resultsexperimental results
![Page 7: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/7.jpg)
A Model of Articulated ShapesA Model of Articulated Shapes
Oi is a part; Jij is a junction between Oi, Oj
Oi and Oj has no common points
diameter(Jij) < ε, The diameter is in the sense of the inner-distanceε is very small compared to the parts. When ε =0, all junctions degenerate to single
points, O is called an ideal articulated object.
![Page 8: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/8.jpg)
Articulations between ShapesArticulations between Shapes
The articulation of shape O is a one-to-one The articulation of shape O is a one-to-one mapping mapping ff from O to O‘=f(O) from O to O‘=f(O)O' is also an articulated object, and O' is also an articulated object, and
decomposed to parts O’decomposed to parts O’I I and junctions J’and junctions J’ ijij
where O'where O'ii=f(O=f(Oii), J'), J'ijij=f(J=f(Jijij) . This preserves the ) . This preserves the
topology between the articulated parts.topology between the articulated parts. f is rigid (rotation and translation only) when f is rigid (rotation and translation only) when
limited on each part. This means inner-limited on each part. This means inner-distances distances within each partwithin each part will not change. will not change.
![Page 9: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/9.jpg)
Examples of Articulated ShapesExamples of Articulated Shapes
![Page 10: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/10.jpg)
NotationsNotations
f(P) f(P) denotes {f(x): x in P}denotes {f(x): x in P}C(xC(x11,x,x22;P);P) denotes a shortest path from x denotes a shortest path from x1 1
in P to xin P to x2 2 in P. (P is a closed and in P. (P is a closed and
connected subset of Rconnected subset of R22.. ‘‘ indicates the image of a point or a point indicates the image of a point or a point
set under articulation f. set under articulation f. [[ and and ]] denote the concatenation of paths. denote the concatenation of paths.
![Page 11: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/11.jpg)
Changes of Inner-Distance within Changes of Inner-Distance within Parts and JunctionsParts and Junctions
Fact 1: the inner-distance within any part is Fact 1: the inner-distance within any part is invariant to articulationinvariant to articulation
Fact 2: for two points in a same junction, the change of Fact 2: for two points in a same junction, the change of the inner-distance under articulation is bounded by the inner-distance under articulation is bounded by εε
![Page 12: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/12.jpg)
Articulation Insensitivity of the Articulation Insensitivity of the Inner-DistanceInner-Distance
Theorem: Let O be an articulated object and Theorem: Let O be an articulated object and f be an articulation of O as defined above. f be an articulation of O as defined above. Let x,y be two arbitrary points in O. Let x,y be two arbitrary points in O. Suppose the shortest path C(x,y;O) goes Suppose the shortest path C(x,y;O) goes through m different junctions in O and through m different junctions in O and C(x',y';O') goes through m' different C(x',y';O') goes through m' different junctions in O', thenjunctions in O', then
|d(x,y;O) - d(x',y';O')| < max{m,m'} |d(x,y;O) - d(x',y';O')| < max{m,m'} εε
![Page 13: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/13.jpg)
Proof of the Articulation Insensitivity Proof of the Articulation Insensitivity of the Inner-Distanceof the Inner-Distance
1.1. Decompose the shortest path C(x,y;O) Decompose the shortest path C(x,y;O) into segments. Each segment is either into segments. Each segment is either within a part, or start and end in a same within a part, or start and end in a same junction.junction.
2.2. Construct a relaxed path in O’ according Construct a relaxed path in O’ according to the decomposition.to the decomposition.
3.3. Apply the two facts mentioned above.Apply the two facts mentioned above.
![Page 14: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/14.jpg)
Illustration of the ProofIllustration of the Proof
(a) Decomposition of C(x,y;O) with (a) Decomposition of C(x,y;O) with x=px=p00,p,p11,p,p22,p,p33=y. Note that a segment can go =y. Note that a segment can go
through a junction more than once (e.g. pthrough a junction more than once (e.g. p11,p,p22).). (b) Construction of C”(x',y';O') in O'. Note that (b) Construction of C”(x',y';O') in O'. Note that
C”(x',y';O') is not the shortest path.C”(x',y';O') is not the shortest path.
![Page 15: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/15.jpg)
The Inner-Distance and Part StructureThe Inner-Distance and Part Structure
The inner-distance captures part The inner-distance captures part structuresstructures
With the same sample points, the distributions of With the same sample points, the distributions of Euclidean distances between all pair of points are Euclidean distances between all pair of points are virtually indistinguishable for the four shapes, while the virtually indistinguishable for the four shapes, while the distributions of the inner-distances are quite differentdistributions of the inner-distances are quite different
![Page 16: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/16.jpg)
Another Interesting CaseAnother Interesting Case
With about the same number of sample points, the four With about the same number of sample points, the four shapes are virtually indistinguishable using Euclidean shapes are virtually indistinguishable using Euclidean distances, while their distributions of the inner-distances distances, while their distributions of the inner-distances are quite different except for the first two shapes.are quite different except for the first two shapes.
1) None of the shapes has (explicit) parts. 1) None of the shapes has (explicit) parts. 2) More sample points will not affect much to the above 2) More sample points will not affect much to the above
statement.statement.
![Page 17: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/17.jpg)
Computing the Inner-DistanceComputing the Inner-Distance
Using shortest path algorithmsUsing shortest path algorithmsBuild a graph on the sample points. For each Build a graph on the sample points. For each
pair of sample points x andy, if the line pair of sample points x andy, if the line segment connecting x and y falls entirely segment connecting x and y falls entirely within the object, then build an edge between within the object, then build an edge between them with the weight equal to the Euclidean them with the weight equal to the Euclidean distance |x-y|.distance |x-y|.
Apply a shortest path algorithm to the graph.Apply a shortest path algorithm to the graph.
![Page 18: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/18.jpg)
Application of the Inner-distanceApplication of the Inner-distance
Extend the Extend the shape contextshape context [Belongie 2002] [Belongie 2002] for shape matching and comparisonfor shape matching and comparison
Dynamic programmingDynamic programming for silhouette for silhouette matchingmatching
Other ways:Other ways:Multidimensional ScalingMultidimensional Scaling (MDS), (MDS),
Elad&Kimmel 2003Elad&Kimmel 2003Shape distribution [Shape distribution [Osada et al. 2002]Osada et al. 2002]
![Page 19: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/19.jpg)
Previous Work on Shape ContextPrevious Work on Shape Context
Given n sample points xGiven n sample points x11,x,x22,...,x,...,xnn on a shape, the on a shape, the
shape context at point xshape context at point x ii is defined as a histogram is defined as a histogram
hhii of the relative coordinates of the remaining n- of the relative coordinates of the remaining n-
1points [Belongie et al. 2002]:1points [Belongie et al. 2002]:
[Belongie et al. 2002] proposed combining the [Belongie et al. 2002] proposed combining the shape context and thin-plate-spline (SC+TPS) for shape context and thin-plate-spline (SC+TPS) for shape matching.shape matching.
![Page 20: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/20.jpg)
Extension to Shape Context --Extension to Shape Context --Inner-distance Shape Context (IDSC)Inner-distance Shape Context (IDSC)
The Euclidean distance is directly replaced The Euclidean distance is directly replaced by the inner-distance.by the inner-distance.
The orientation between points is replaced The orientation between points is replaced by the by the inner-angleinner-angle, which is defined as the , which is defined as the angle between the tangential direction of angle between the tangential direction of the shortest path and the local tangential the shortest path and the local tangential on the shape boundary.on the shape boundary.
![Page 21: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/21.jpg)
The Inner-AngleThe Inner-Angle
The inner-angle is insensitative to The inner-angle is insensitative to articulation.articulation.
![Page 22: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/22.jpg)
Extension to Shape Context --Extension to Shape Context --ExamplesExamples
![Page 23: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/23.jpg)
Silhouette Matching through Silhouette Matching through Dynamic ProgrammingDynamic Programming
Utilize the ordering provided by the Utilize the ordering provided by the contour.contour.
Fast and accurateFast and accurateOther works using dynamic programming: Other works using dynamic programming:
[Basri et al. 1998, Petrakis et al. 2002].[Basri et al. 1998, Petrakis et al. 2002].Bipartite matching: more general, less Bipartite matching: more general, less
constraint, slow.constraint, slow.
![Page 24: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/24.jpg)
Experiment: Articulate DatasetExperiment: Articulate Dataset
![Page 25: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/25.jpg)
Experiment: MPEG7 Shape-CE-1Experiment: MPEG7 Shape-CE-1
![Page 26: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/26.jpg)
Analysis of Experiment: MPEG7Analysis of Experiment: MPEG7
Two retrieval examples for comparing SC and IDSC on the MPEG7 data set. The left column show two shapes to be retrieved: a beetle and an octopus. The four right rows show the top 1 to 9 matches, from top to bottom: SC and IDSC for the beetle, SC and IDSC for the octopus.
![Page 27: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/27.jpg)
The Inner-Distance and Part StructureThe Inner-Distance and Part Structure
We observed this data set is difficult for We observed this data set is difficult for retrieval mainly due to the complex part retrieval mainly due to the complex part structures in the shapes, though they have structures in the shapes, though they have little articulation. This shows that the inner-little articulation. This shows that the inner-distance is effective at capturing part distance is effective at capturing part structures.structures.
The following experiments also The following experiments also demonstrate similar effect.demonstrate similar effect.
![Page 28: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/28.jpg)
Experiment: Kimia Silhouette 1Experiment: Kimia Silhouette 1
![Page 29: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/29.jpg)
Experiment: Kimia Silhouette 2Experiment: Kimia Silhouette 2
![Page 30: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/30.jpg)
Experiment: Swedish Leaf DatasetExperiment: Swedish Leaf Dataset
Using 25 training samples and 50 testing samples per Using 25 training samples and 50 testing samples per species. Average correct ratios are:species. Average correct ratios are:
Combination of simple features: 82% [Soderkvist 2001]Combination of simple features: 82% [Soderkvist 2001] Fourier descriptors: 89.60%Fourier descriptors: 89.60% SC+DP: 88.12%SC+DP: 88.12% IDSC+DP: 94.13%IDSC+DP: 94.13%
![Page 31: Using the Inner-Distance for Classification of Articulated Shapes](https://reader036.vdocuments.us/reader036/viewer/2022062322/56814d18550346895dba5144/html5/thumbnails/31.jpg)
Experiment: HExperiment: Huuman Body Matchingman Body Matching
Left: between adjacent frames. Right: silhouettes Left: between adjacent frames. Right: silhouettes separated by 20 frames. Only half of the matched pairs are separated by 20 frames. Only half of the matched pairs are shown for illustration.shown for illustration.