Download - Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition
![Page 1: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/1.jpg)
Scale Saliency: Applications in Visual Matching, Tracking and
View-Based Object Recognition
Jonathon S. Hare and Paul H. Lewis
Intelligence, Agents, Multimedia GroupDepartment of Electronics and Computer Science
University of Southampton{jsh02r, phl}@ecs.soton.ac.uk
![Page 2: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/2.jpg)
Topics of Discussion
• Introduction
• An overview of the scale-saliency algorithm
• Image Matching and View-Based Recognition
• Tracking
• Future Work
![Page 3: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/3.jpg)
• Salient points or ‘interest-points’ have often been used for various vision tasks
• We have modified and extended an existing saliency algorithm and investigated its application to the tasks of matching, recognition and tracking
Introduction
![Page 4: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/4.jpg)
Scale-Saliency
• Algorithm developed by Kadir & Brady:
• Based on previous work by Gilles
• Defines saliency in terms of local signal complexity or unpredictability, weighted by local self-similarity
• Keypoints are selected from peaks in entropy scale-space, thus the algorithm picks regions in (x, y, scale)
![Page 5: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/5.jpg)
Scale-SaliencyLena R^3 Space
0 100
200 300
400 500
x location 0 50
100 150
200 250
300 350
400 450
y location
0
5
10
15
20
25
30
scale
Salient regions in the Lena picture The corresponding ℜ3 space
![Page 6: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/6.jpg)
Scale-Saliency
![Page 7: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/7.jpg)
• The original Thesis showed that the algorithm could be used for efficient tracking and matching using intensity pixel values as a descriptor
• We built on this using higher-order descriptors
• Histograms (intensity, RGB, HSI, etc)
• Possibilities for other feature invariants
Matching and View-Based Recognition
![Page 8: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/8.jpg)
• Simple matching algorithm:
• Feature vectors compared using euclidean distance
• Distance(s) of closest features summed for each image to get overall distance
• Image with lowest overall distance is selected as the winning match
Matching and View-Based Recognition
![Page 9: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/9.jpg)
Matching and View-Based Recognition
![Page 10: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/10.jpg)
Matching and View-Based Recognition
Matching Example
![Page 11: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/11.jpg)
• The tracking problem can be posed as a sub-image matching problem
• i.e. we want to track single salient regions across multiple frames
• This can be accomplished using the matching algorithm across consecutive frames
Tracking
![Page 12: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/12.jpg)
• Tracking Examples
• Advantage of this method is that it directly captures changes in scale
Tracking
![Page 13: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/13.jpg)
Tracking
![Page 14: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/14.jpg)
• We have demonstrated a number of practical uses for the scale-saliency algorithm
• Using the scale-saliency algorithm we show how sparse representations of the image can be created and represented by features that are invariant to translation, rotation and scale change
Conclusions
![Page 15: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/15.jpg)
• We have shown that this sparse representation can be used for image matching tasks
• The matching approach can be extended to enable view-based recognition and Content-Based Retrieval
• It is also possible to use the matching technique for tracking by applying it across consecutive video frames
Conclusions
![Page 16: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/16.jpg)
• Future Work
• We are currently investigating how this work can be extended into a facility for recognition of art objects
• We are also researching methods for ensuring spatial consistency of salient points, without the use of graph-matching
Conclusions
![Page 17: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/17.jpg)
Any Questions?
![Page 18: Scale Saliency: Applications in Visual Matching,Tracking and View-Based Object Recognition](https://reader034.vdocuments.us/reader034/viewer/2022042818/55b53692bb61eb3a258b47df/html5/thumbnails/18.jpg)