![Page 1: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/1.jpg)
Viola 1999MIT AI Laboratory
Progress on: Variable Viewpoint Reality Image Database
Paul Viola & Eric GrimsonJeremy DeBonet, Aparna Lakshmiratan, William Wells, Kinh
Tieu,Dan Snow, Owen Ozier, John Winn, Mike Ross
MIT Artificial Intelligence Lab
![Page 2: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/2.jpg)
Viola 1999MIT AI Laboratory
Overview of Presentation
• Variable Viewpoint Reality– Overview
– Progress at MIT
• Image Database Retrieval – Overview
– Progress
• http://www.ai.mit.edu/projects/NTTCollaboration
![Page 3: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/3.jpg)
Viola 1999MIT AI Laboratory
VVR: Motivating Scenario
• Construct a system that will allow each/every user to observe any viewpoint of a sporting event.
• Provide high level commentary/statistics– Analyze plays
![Page 4: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/4.jpg)
Viola 1999MIT AI Laboratory
For example …
Computed using a single view…
some steps by hand
![Page 5: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/5.jpg)
Viola 1999MIT AI Laboratory
VVR Spectator Environment
• Build an exciting, fun, high-profile system– Sports: Soccer, Hockey, Tennis, Basketball
– Drama, Dance, Ballet
• Leverage MIT technology in:– Vision/Video Analysis
• Tracking, Calibration, Action Recognition
• Image/Video Databases
– Graphics
• Build a system that provides data available nowhere else…– Record/Study Human movements and actions
– Motion Capture / Motion Generation
![Page 6: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/6.jpg)
Viola 1999MIT AI Laboratory
Window of Opportunity
• 20-50 cameras in a stadium– Soon there will be many more
• US HDTV is digital – Flexible, very high bandwidth digital transmissions
• Future Televisions will be Computers– Plenty of extra computation available
– 3D Graphics hardware will be integrated
• Economics of sports– Dollar investments by broadcasters is huge (Billions)
• Computation is getting cheaper
![Page 7: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/7.jpg)
Viola 1999MIT AI Laboratory
Progress at MIT
• Simple intersection of silhouettes (Visual Hull)– Efficient but limited
• Tomographic reconstruction– Based on medical reconstruction
• Probabilistic Voxel Analysis (Poxels)– Handles occlusion & transparency
• Parametric Human Forms
![Page 8: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/8.jpg)
Viola 1999MIT AI Laboratory
Visual Hull in 2D
![Page 9: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/9.jpg)
Viola 1999MIT AI Laboratory
Visual Hull: Segment
![Page 10: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/10.jpg)
Viola 1999MIT AI Laboratory
Visual Hull: Segment
![Page 11: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/11.jpg)
Viola 1999MIT AI Laboratory
Visual Hull: Segment
![Page 12: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/12.jpg)
Viola 1999MIT AI Laboratory
Visual Hull: Intersection
![Page 13: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/13.jpg)
Viola 1999MIT AI Laboratory
Idea in 2D: Visual Hull
![Page 14: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/14.jpg)
Viola 1999MIT AI Laboratory
Real Data: Tweety
• Data acquired on a turntable– 180 views are available… not all are used.
![Page 15: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/15.jpg)
Viola 1999MIT AI Laboratory
Intersection of Frusta
• Intersection of 18 frusta– Computations are very fast
• perhaps real-time
![Page 16: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/16.jpg)
Viola 1999MIT AI Laboratory
New Apparatus
Twelve cameras, computers, digitizers
Parallel software for acquisition/processing
![Page 17: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/17.jpg)
Viola 1999MIT AI Laboratory
Current System
• Real-time image acquisition
• Silhouettes computed in parallel
• Silhouettes sent to a central machine– 15 per second
• Real-time Intersection and Visual Hull– In progress
![Page 18: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/18.jpg)
Viola 1999MIT AI Laboratory
Visual Hull is very coarse …
Agreement provides
additional information
Agreement provides
additional information
![Page 19: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/19.jpg)
Viola 1999MIT AI Laboratory
Tomographic Reconstruction
• Motivated by medical imaging– CT - Computed Tomography
– Measurements are line integrals in a volume
– Reconstruction is by back-projection & deconvolution
![Page 20: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/20.jpg)
Viola 1999MIT AI Laboratory
Back-projection of image intensities
![Page 21: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/21.jpg)
Viola 1999MIT AI Laboratory
Volume Render...
• Captures shape very well• Intensities are not perfect
![Page 22: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/22.jpg)
Viola 1999MIT AI Laboratory
Poxels: An improvement to tomography
• Tomography confuses color with transparency– Does not model occlusion...
• The Probabilistic Voxel Approach: Poxel– Estimates both color and transparency
– Models occlusion
– Much better results
• Though slower
– Work submitted to ICCV 99
![Page 23: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/23.jpg)
Viola 1999MIT AI Laboratory
Occlusion causes disagreement
![Page 24: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/24.jpg)
Viola 1999MIT AI Laboratory
Initial agreement is not enough…
Agreement
is high
“Opaque”
Agreement
is poor
![Page 25: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/25.jpg)
Viola 1999MIT AI Laboratory
Second pass uses information about occlusion
Unoccluded
Cameras
Occluded
camera is
ignoredAgreement
becomes good
“Opaque”
![Page 26: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/26.jpg)
Viola 1999MIT AI Laboratory
Poxels Algorithm: Definitions
kimagecastingrayzyxdvul
imagesvuI
transcolorvoxelszyxv
cytransparanzyx
colorszyxc
k
k
),,(),,(
),(
)(),,(
),,(
),,(
Images
Ik(u,v)
Ray Casting
Grid of color
& transparency
v(x,y,z)
![Page 27: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/27.jpg)
Viola 1999MIT AI Laboratory
Poxels: Model of Transparency
)),,(())2,,(())1,,((),(ˆ dvulvvulvvulvvuI kkkk
BACKccc 2211color
Eye
),( 11 c ),( 22 c
Voxels
BACKc
![Page 28: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/28.jpg)
Viola 1999MIT AI Laboratory
Poxels Algorithm: Agreement (Step 1)
Point of highest
agreement
Point of highest
agreement
![Page 29: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/29.jpg)
Viola 1999MIT AI Laboratory
Results…
Rendering of reconstructed shape.
![Page 30: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/30.jpg)
Viola 1999MIT AI Laboratory
From ICCV paper...
Input Image Reconstructed
Volume
![Page 31: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/31.jpg)
Viola 1999MIT AI Laboratory
… additional results
![Page 32: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/32.jpg)
Viola 1999MIT AI Laboratory
Image Databases: Motivating Scenario
• Image Databases are proliferating– The Web
– Commercial Image Databases
– Video Databases
– Catalog Databases
• “Find me a bag that looks like a Gucci.”
– Virtual Museums
• “Find me impressionist portraits.”
– Travel Information
• “Find me towns with Gothic architecture.”
– Real-estate
• “Find me a home that is sunny and open.”
![Page 33: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/33.jpg)
Viola 1999MIT AI Laboratory
But, the problem is very hard…
There are a very wide variety of images...
![Page 34: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/34.jpg)
Viola 1999MIT AI Laboratory
We have made good progress...
Query: “Waterfall Images”
![Page 35: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/35.jpg)
Viola 1999MIT AI Laboratory
Search for cars?
Trained by
example
![Page 36: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/36.jpg)
Viola 1999MIT AI Laboratory
Complex Feature Representation
• Motivated by the Human brain…– Infero-temporal cortex computes many thousand
selective features
– Features are selective yet insensitive to unimportant variations
– Every object/image has some but not all of these features
• Retrieval involves matching the most salient features
![Page 37: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/37.jpg)
Viola 1999MIT AI Laboratory
Image Database RetrievalNTT: Visit
1/7/99
![Page 38: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/38.jpg)
Viola 1999MIT AI Laboratory
Overview of IDB Meeting
• Motivation from MIT ...• Discuss current and related work
– Flexible Templates
– Complex Features
– Demonstrations
• Related NTT Efforts• Discussion of collaboration• Future work• Dinner
![Page 39: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/39.jpg)
Viola 1999MIT AI Laboratory
Motivating Scenario
• Image Databases are proliferating– The Web
– Commercial Image Databases
– Video Databases
– Catalog Databases• “Find me a bag that looks like a Gucci.”
– Virtual Museums• “Find me impressionist portraits.”
– Travel Information• “Find me towns with Gothic architecture.”
– Real-estate• “Find me a home that is sunny and open.”
![Page 40: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/40.jpg)
Viola 1999MIT AI Laboratory
There is a very wide variety of images...
![Page 41: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/41.jpg)
Viola 1999MIT AI Laboratory
Search for images containing waterfalls?
![Page 42: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/42.jpg)
Viola 1999MIT AI Laboratory
Search for cars?
![Page 43: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/43.jpg)
Viola 1999MIT AI Laboratory
What makes IDB hard?
• Finding the right features– Insensitive to movement of components
– Sensitive to critical properties
• Focussing attention– Not everything matters
• Generalization based on class– Given two images
• Small black dog & Large white dog
• (Don’t have much in common…)
– Return other dogs
![Page 44: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/44.jpg)
Viola 1999MIT AI Laboratory
Overview of IDB Meeting
• Motivation from MIT ...• Discuss current and related work
– Flexible Templates
– Complex Features
– Demonstrations
• Related NTT Efforts• Discussion of collaboration• Future work• Dinner
![Page 45: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/45.jpg)
Viola 1999MIT AI Laboratory
Complex Feature Representation
• Motivated by the Human brain…– Infero-temporal cortex computes many thousand
selective features
– Features are selective yet insensitive to unimportant variations
– Every object/image has some but not all of these features
• Retrieval involves matching the most salient features
![Page 46: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/46.jpg)
Viola 1999MIT AI Laboratory
Features are extracted withFeatures are extracted withmany Convolution Filtersmany Convolution Filters
source image
x2
x2
x2
x2
Filters
Vertica
l
Horizontal
![Page 47: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/47.jpg)
Viola 1999MIT AI Laboratory
* x2
x2
x2
*
*
x2
*
x2
*
x2
*
x2
*
* x2
x2
x2
*
*
x2
*
x2
*
x2
*
x2
*
Cha
ract
eris
tic
sign
atur
e
![Page 48: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/48.jpg)
Viola 1999MIT AI Laboratory
pixels
Feature Value
Resolution is reduced at each step…
![Page 49: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/49.jpg)
Viola 1999MIT AI Laboratory
projection 1 projection 2 projection 2
Query 1 (variation of object location)
Query 2 (variation of lighting)*
Not every feature is useful for a query
Poor FeaturesPoor Featuresfor query 1for query 1
Good FeaturesGood Featuresfor query 1for query 1
Poor FeaturesPoor Featuresfor query 2for query 2
Features: A,BFeatures: A,B Features: C,DFeatures: C,D Features: C,DFeatures: C,D
![Page 50: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/50.jpg)
Viola 1999MIT AI Laboratory
* * normalization
averagesignature
Que
ry im
ages
•Normalization brings some image closer to the mean
Normalization of Signature Space
![Page 51: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/51.jpg)
Viola 1999MIT AI Laboratory
q te c e c
qe cc r g be
, ,
,{ , , }
2
0
253
q t 2
q
Distance/Similarity Measure
Diagonal Mahalanobis Distance
![Page 52: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/52.jpg)
Viola 1999MIT AI Laboratory
Image Database Progress at MIT
• Better learning algorithms to select features• Developed a very compact feature representation
– Fewer features required
– 2-3 bits per feature
• Pre-segmentation of images– Better learning
– More selective queries
• Construction of object models:– Faces, people, cars, etc. (ICCV 99)
factor of 72
![Page 53: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/53.jpg)
Viola 1999MIT AI Laboratory
![Page 54: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/54.jpg)
Viola 1999MIT AI Laboratory
![Page 55: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/55.jpg)
Viola 1999MIT AI Laboratory
![Page 56: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/56.jpg)
Viola 1999MIT AI Laboratory
![Page 57: Viola 1999 MIT AI Laboratory Progress on: Variable Viewpoint Reality Image Database Paul Viola & Eric Grimson Jeremy DeBonet, Aparna Lakshmiratan, William](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649e3b5503460f94b2ded0/html5/thumbnails/57.jpg)
Viola 1999MIT AI Laboratory
Conclusions
• Variable Viewpoint Reality– Prototypes constructed
– New approaches
• Image Database Retrieval– New more efficient representations
– Improved performance