mit6870_orsu_lecture6: scenes and objects
Post on 06-Apr-2018
214 Views
Preview:
TRANSCRIPT
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 1/104
Lecture 6Scenes and objects
6.870 Object Recognition and Scene Understandinghttp://people.csail.mit.edu/torralba/courses/6.870/6.870.recognition.htm
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 2/104
Class business
Next Wednesday«
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 3/104
Week 2: Objects without scenes
Week 5: Scenes without objects
Week 6: Scenes and objects
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 4/104
Why is detection hard?
x
y
1,000,000 images/dayPlus, we want to do this for ~ 1000 objects
10,000 patches/object/image
time
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 5/104
Standard approach to scene analysis
1) Object representation based on intrinsic features:
Local
featuresno car
Classifier
p( car | VL )
2) Detection strategy:
Sky
Mountain
Buildings
cars
3) The scene representation
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 6/104
Is local information enough?
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 7/104
With hundreds of categories
roadtablechair
keyboardtablecar
road
If we have 1000 categories (detectors), and each detector produces 1 fa every 10
images, we will have 100 false alarms per image« pretty much garbage«
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 8/104
Is local information even enough?
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 9/104
Is local information even enough?
Distance
Information
Local featuresContextual features
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 10/104
We know there is a keyboard present in this scene even if we cannot see it clearly.
We know there is no keyboard present in this scene
« even if there is one indeed.
The system does not care about the
scene, but we do«
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 11/104
The multiple personalities of a blob
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 12/104
The multiple personalities of a blob
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 13/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 14/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 15/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 16/104
Look-Alikes by Joan Steiner
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 17/104
Look-Alikes by Joan Steiner
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 18/104
Look-Alikes by Joan Steiner
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 19/104
Why is context important?
Changes the interpretation of an object (or its function)
Context defines what an unexpected event is
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 20/104
The influence of an object extends beyond its physical boundaries
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 21/104
The context challenge
How far can you go withoutusing an object detector?
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 22/104
21
What are the hidden objects?
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 23/104
What are the hidden objects?
Chance ~ 1/30000
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 24/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 25/104
The importance of context
Cognitive psychology ± Palmer 1975
± Biederman 1981
± «
Computer vision ± Noton and Stark (1971)
± Hanson and Riseman (1978)
± Barrow & Tenenbaum (1978) ± Ohta, kanade, Skai (1978)
± Haralick (1983)
± Strat and Fischler (1991)
± Bobick and Pinhanez (1995)
± Campbell et al (1997)
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 26/104
Biederman 1972
Arrow appeared before or after picture.
Selected object from 4 pictures.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 27/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 28/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 29/104
Biederman 1972
Better accuracy with normal scene and
with pre-cue.
C
oherence of surroundings affected objectperception.
But, jumbled pictures had unnatural edge
artifacts.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 30/104
Palmer 1975
Scene preceded object to identify.
Better identification when preceded by a
semantically consistent scene.
Objects seen for 20, 40, 60 or 120 ms.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 31/104
Palmer
Scenes shown ahead of time for 2 s.
More accurate recognition of consistent
objects than inconsistent objects. Similar looking objects were misnamed,
showing a bias effect.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 32/104
Loftus & Mackworth
Inconsistent objects
fixated earlier and
longer.
Suggested additional
processing of objects
out of context. Similar results found
by Friedman (1979).
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 33/104
De Graef et al. 1990
Prior results due to memory task?
Measured eye movements during non-
object search task.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 34/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 35/104
Object Detection
Biederman et al. 1982, relational violations
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 36/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 37/104
Biederman 1982
Pictures shown for 150
ms.
Objects in appropriate
context were detected
more accurately than
objects in aninappropriate context.
Scene consistency
affects object detection.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 38/104
Objects and Scenes
Biederman¶s violations (1981):
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 39/104
Support
[Golconde Rene Magritte]
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 40/104
Interposition
[Blank Check Rene Magritte]
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 41/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 42/104
Position, Probability
[P ersonal Values Rene Magritte]
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 43/104
Object Consistencies
Biederman et al (1982), DeGraef(1990).
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 44/104
Object Consistencies
Examples of inconsistencies
Biederman et al (1982), DeGraef(1990).
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 45/104
Contextual cueing
Chun & Jiang, 1998
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 46/104
Object priming
Torralba, Sinha, Oliva, VSS 2001
Increasing contextual information
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 47/104
Object priming
Torralba, Sinha, Oliva, VSS 2001
bj i i
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 48/104
?Car, pedestrian, mailbox, «
Object priming
p(object | scene)
Torralba, Sinha, Oliva, VSS 2001
Obj i i
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 49/104
Object priming
Torralba, Sinha, Oliva, VSS 2001
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 50/104
Ex amples of consistent scenes (a), inconsistent scenes (b), and isolated objects and
backgrounds (c); from Davenport & P otter, 2004
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 51/104
But do we really need context?
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 52/104
Hollingworth & Henderson
Concerns with object detection studies
± Object label could bias results.
± Location cue selectively helpful for consistent
objects.
Controlled for false alarm biases with post-
cue and 2AFC.
Failed to find consistency effects.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 53/104
Hollingworth & Henderson
Post-cue
2AFC with object
labels
± Both consistent or
inconsistent.
2AFC with tokendiscrimination.
± E.g. sports car or
sedan.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 54/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 55/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 56/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 57/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 58/104
CONDOR systemStrat and Fischler (1991)
Guzman (S EE ), 1968
Noton and Stark 1971
Hansen & Riseman (VI S ION S ), 1978
Barrow & Tenenbaum 1978
Brooks ( ACRONYM ), 1979
Marr, 1982
Ohta & Kanade, 1978
Yakimovsky & Feldman, 1973
A A f S U d di
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 59/104
An Age of Scene Understanding
Guzman (S EE ), 1968 Noton and Stark 1971
Hansen & Riseman(VI S ION S ), 1978
Barrow & Tenenbaum 1978
Brooks ( ACRONYM ), 1979
Marr, 1982
Ohta & Kanade, 1978
Yakimovsky & Feldman, 1973
[Ohta & Kanade 1978]
C
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 60/104
Current approaches
1) Scene to object dependencies
2) Object to object dependencies
L l f
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 61/104
Levels of context
Context in low-level vision
Part-based models
Objects relations Long-range connections
Weak constraints
Multimodal
Fix graph structurescan be useful
approximations
C t h
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 62/104
Current approaches
1) Scene to object dependencies
2) Object to object dependencies
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 63/104
b t this co occ rrence has a hidden
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 64/104
« but this co-occurrence has a hidden
common ³cause´: the scene
streetsoffices
It is easier to first recognize the scene, then predict object presence, than
running local object classifiers
Th l d t t f
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 65/104
The layered structure of scenes
In a display with multiple targets present, the location of one target constraints the µy¶
coordinate of the remaining targets, but not the µx¶ coordinate.
Assuming a human observer standing on the ground
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 66/104
D t ti f ith t f d t t
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 67/104
Detecting faces without a face detector
Torralba & Sinha, 01; Torralba, 03
Context based vision system for place
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 68/104
Context-based vision system for place
and object recognition
Hidden states = location (63 values) Observations = vG
t (80 dimensions)
Transition matrix encodes topology of
environment Observation model is a mixture of
Gaussians centered on prototypes (100
views per place)
Office 610 Corridor 6b Corridor 6c Office 617
We use 17 annotated sequences for training
Torralba, Murphy, Freeman and Rubin. ICCV 2003
O bil i
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 69/104
Our mobile rig
Torralba, Murphy, Freeman, Rubin. 2003
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 70/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 71/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 72/104
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 73/104
An integrated model of Scenes
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 74/104
An integrated model of Scenes,
Objects, and Parts
Ncar
S
g
Scene
Scene
gist
features
0
0
1
1
5
5
N
P(Ncar | S = street)
P(Ncar | S = park)
0 5 10 1 50
0 .0 5
0. 1
0 .1 5
0. 2
0 5 1 0 1 50
0. 2
0. 4
0. 6
0. 8
N
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 75/104
Global to local
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 76/104
Global to local
Use global context to predict objects but there is nomodeling of spatial relationships between objects.
Op1,c1
vp1,c1
OpN,c1
vpN,c1. . .
Op1,c2
vp1,c2
OpN,c2
vpN,c2. . .
Class 1 Class 2
E1 E2
S
c2maxVc1
maxV
X1X2
vg
Keyboards
Murphy, Torralba & Freeman (03)
3d Scene Context
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 77/104
3d Scene Context
Image World
Hoiem, Efros, Hebert ICCV 2005
3d Scene Context
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 78/104
3d Scene Context
meters
m e t e
r s
Ped
Ped
Car
Hoiem, Efros, Hebert ICCV 2005
3D City Modeling using Cognitive Loops
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 79/104
3D City Modeling using Cognitive Loops
N.C
ornelis, B. Leibe, K.C
ornelis, L.V
an Gool.CV
PR'06
Current approaches
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 80/104
Current approaches
1) Scene to object dependencies
2)O
bject to object dependencies
Where should I put the silverware?
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 81/104
Where should I put the silverware?
Sampling from the labels
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 82/104
Sampling from the labels
Sampling from the labels
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 83/104
Sampling from the labels
Cf. Hoiem et al; Hays, Efros. Siggraph 2007
Contextual object relationships
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 84/104
Contextual object relationshipsCarbonetto, de Freitas & Barnard (2004) Kumar, Hebert (2005)
Torralba Murphy Freeman (2004)
Fink & Perona (2003)E. Sudderth et al (2005)
Object-Object Relationships
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 85/104
Fink & Perona (NIPS 03)Use output of boosting from other objects at previous
iterations as input into boosting for this iteration
Object-Object Relationships
Pixel labeling using MRFs
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 86/104
Pixel labeling using MRFs
Enforce consistency between neighboring
labels, and between labels and pixels
Carbonetto, de Freitas & Barnard, ECCV¶04
Beyond nearest-neighbor grids
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 87/104
Beyond nearest-neighbor grids
Most MRF/C
RF models assume nearest-neighbor graph topology
This cannot capture long-distance
correlations
Dynamically structured trees
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 88/104
Dynamically structured trees
Each node pick its parents(Storkey& Williams, PAMI¶03)
2D SCFGs(Pollak, Siskind, Harper & Bouman IC ASSP¶03)
Object-Object Relationships
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 89/104
Object Object Relationships
Use latent variables to induce long distance correlations
between labels in a Conditional Random Field (CRF)
He, Zemel & Carreira-Perpinan (04)
Object-Object Relationships
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 90/104
Object-Object Relationships
[Kumar Hebert 2005]
Hierarchical Sharing and Context
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 91/104
Hierarchical Sharing and Context
Scenes share objects
Objects share parts
Parts share features
E. Sudderth, A. Torralba, W. T. Freeman, and A. Wilsky.
3d Scene Context
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 92/104
3d Scene Context
Image Support Vertical Sky
V-Left V-Center V-Right V-Porous V-Solid
[Hoiem, Efros, Hebert ICCV 2005]
Object
Surface?
Support?
Detecting difficult objects
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 93/104
Detecting difficult objects
Office Maybethere is
a mouse
Start recognizing the scene
Torralba, Murphy, Freeman. NIPS 2004.
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 94/104
Detecting difficult objects
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 95/104
Detecting difficult objects
Detect first simple objects (reliable detectors) that provide strong
contextual constraints to the target (screen -> keyboard -> mouse)
Torralba, Murphy, Freeman. NIPS 2004.
BRF for screen/keyboard/mouse
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 96/104
BRF for screen/keyboard/mouse
Iteration
BRF for screen/keyboard/mouse
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 97/104
BRF for screen/keyboard/mouse
Iteration
BRF for screen/keyboard/mouse
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 98/104
BRF for screen/keyboard/mouse
Iteration
BRF for screen/keyboard/mouse
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 99/104
BRF for screen/keyboard/mouse
Iteration
BRF for screen/keyboard/mouse
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 100/104
BRF for screen/keyboard/mouse
Iteration
BRF for car detection: topology
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 101/104
BRF for car detection: topology
BRF for car detection: results
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 102/104
BRF for car detection: results
A ³car´ out of context is less of a car
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 103/104
A car out of context is less of a car
Car Building Road
b F G b F G b F G
From image
From detectors
Thresholded beliefs
8/3/2019 MIT6870_ORSU_lecture6: Scenes and objects
http://slidepdf.com/reader/full/mit6870orsulecture6-scenes-and-objects 104/104
Contextor no context
top related