Download - Applications of
Applications of
Shape Similarity
ASR: Applications in Computer Vision
Robotics: Shape Screening(Movie: Robot2.avi)
• Straightforward Training Phase
• Recognition of Rough Differences
• Recognition of Differences in Detail
• Recognition of Parts
ASR: Applications in Computer Vision
Application 2:
View Invariant Human Activity Recognition
(Dr. Cen Rao and Mubarak Shah, School of Electrical Engineering and Computer Science, University of Central Florida)
Application: Human Activity Recognition
Human Action Defined by TrajectoryAction Recognition by Comparison of Trajectories
(Movie: Trajectories)
• Rao / Shah:• Extraction of ‘Dynamic Instants’ by Analysis of Spatiotemporal Curvature• Comparison of ‘Dynamic Instants’ (Sets of unconnected points !)
• ASR: •Simplification of Trajectories by Curve Evolution• Comparison of Trajectories
Application: Human Activity Recognition
Trajectory Simplification
Activity Recognition: Typical Set of Trajectories
Trajectories in Tangent Space
Trajectory Comparison by ASR: Results
Recognition of 3D Objects by Projection
Background: MPEG 7 uses fixed view anglesImprovement: Automatic Detection of Key Views
Automatic Detection of Key Views
(Pairwise) Comparison of Adjacent Views•Detects Appearance of Hidden Parts
Automatic Detection of Key Views
Result (work in progress):
Application: ASR
The Database Implementation
The Main Application: Back to ISS
Task:Create Image Database
Problem:Response TimeComparison of 2 Shapes: 23ms on Pentium1Ghz
ISS contains 15,000 images:Response Time about 6 min.
Clustering not possible:ASR failed on measuring dissimilarities !
Solution:
Full search on entire database using a simplercomparison
Vantage Objects (Vleugels / Veltkamp, 2000) provide a simple comparison of n- dimensional vectors (n typically < 100)
Vantage Objects
The Idea:Compare the query-shape q to a predefined
subset S of the shapes in the database D
The result is an n-dimensional Vantage Vector V,n = |S|
Vantage Objects
q
s1
s2
s3
sn
…
v1
v2
v3
vn
-- Each shape can be represented by a single Vantage Vector
-- The computation of the Vantage Vector calls theASR – comparison only n times
-- ISS uses 54 Vantage Objects, reducing the comparison time (needed to create the Vantage Vector) to < 1.5s
-- How to compare the query object to the database ?
Vantage Objects
-- Create the Vantage Vector vi for every shape di in the database D
-- Create the Vantage Vector vq for the query-shape q
-- compute the (euclidean) distance between vq and vi
-- best response is minimum distance
-Note: computing the Vantage Vectors for the database objects is an offline process !
Vantage Objects
-How to define the set S of Vantage Objects ?
Vantage Objects
-Algorithm 1 (Vleugels / Veltkamp 2000):
-Predefine the number n of Vantage Objects-S0 = { }-Iteratively add shapes di D\Si-1 to Si-1 such that
-Si = Si-1 di
-andk=1..i-1 e(di , sk) maximal. (e = eucl. dist.)
Stop if i = n.
Vantage Objects
-Result:
-Did not work for ISS.
Vantage Objects
-Algorithm 2 (Latecki / Henning / Lakaemper):
Def.: • A(s1,s2): ASR distance of shapes s1,s2
• q: query shape• ‘Vantage Query’ : determining the result r by minimizing e(vq , vi ) vi = Vantage Vector to si
• ‘ASR Query’: determining the result r by minimizing A(q,di )
Vantage Query has certain loss of retrieval quality compared to ASR query.
-Define a loss function l to model the extent of retrieval performance
Vantage Objects
Given a Database D and a set V of Vantage Vectors, the loss of retrieval performance for a single query by shape q is given by:
lV,D (q) = A(q,r),
Where r denotes the resulting shape of the vantage query to D using q.
Property:lV,D (q) is minimal if r is the result of the ASR-Query.
Vantage Objects
Now define retrieval error function L(S) of set S={s1 ,…, sn } D of Vantage Vectors of Database D:
L(S) = 1/n lS,D\{si} (si)
Task:Find subset S D such that L(S) is minimal.
Vantage Objects
Algorithm:
V0={ }iteratively determine sj in D\Sj-1 such that Sj =Sj-1 sj and L(Vj) minimal.
Stop if improvement is low
Vantage Objects
Result:Worked fine for ISS, though handpicked objects still performed better.
Vantage Objects
HandpickedAlgorithm 2
Number of Vantage Objects
L(S)
…some of the Vantage Objects used in ISS:
Vantage Objects
The Vantage Objects are used in the ASR in the first (handdrawn) query.
The query is compared to 54 Objects, then a vector comparison is computed with the whole database.
The first result, also called ‘first guess’, is the result of the vantage vector search.
Searching for a ‘grabbed’ a shape on the user interface leads to direct comparison with the ASR, these results are precomputed, since the query is a known shape !
Vantage Objects and ISS
A: the handdrawn sketchB: the result of the Vantage searchC: the result of the exact match
Vantage Objects and ISS