visual detection of lintel-occluded doors from a single image
DESCRIPTION
Visual Detection of Lintel-Occluded Doors from a Single Image. Zhichao Chen and Stan Birchfield Dept. of Electrical and Computer Engineering Clemson University Clemson, South Carolina USA. Types of Maps. Metric map. Topological map. Either way, doors are semantically meaningful landmarks. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/1.jpg)
Visual Detection of Lintel-Occluded Doors from a Single Image
Zhichao Chen and Stan Birchfield
Dept. of Electrical and Computer EngineeringClemson University
Clemson, South Carolina USA
![Page 2: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/2.jpg)
Types of MapsTopological mapMetric map
Either way, doors are semantically meaningful
landmarks
![Page 3: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/3.jpg)
Previous Approaches to Detecting Doors
• Range-based approaches– sonar [Stoeter et al.1995] – stereo [Kim et al. 1994]– laser [Anguelov et al. 2004]
• Vision-only (uncalibrated)– low cost– low power– non-contact (passive) measurement– rich capturing ability
![Page 4: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/4.jpg)
Vision-Based Door Detection
fuzzy logic [Munoz-Salinas et al. 2004]
color segmentation [Rous et al. 2005]
neural network [Cicirelli et al 2003]
Limitations: • require different colors for doors and walls• simplified environment (untextured floor, no reflections) • limited viewing angle• high computational load • assume lintel visible
![Page 5: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/5.jpg)
Problem Statement Detect doors in complex environments: • textured and untextured floors• walls and doors with similar colors• specular reflections• variable lighting conditions• wide range of robot poses
![Page 6: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/6.jpg)
lintel
post
Another ChallengeLintel-occluded post-and-lintel architecture camera is low to ground cannot point upward b/c obstacles
![Page 7: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/7.jpg)
Video
![Page 8: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/8.jpg)
Our Approach
color
texture
vertical lines
concavity
door gap
vanishing point
kick plate
Adaboost
Standard features
Novel features
Door detected
Assumptions:• Both door posts are visible• Posts appear nearly vertical• The door is at least a certain width
,)(sign)(N
1nn
xhx n
![Page 9: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/9.jpg)
Pairs of Vertical Lines
1. Edges detected by Canny2. Line segments detected by modified Douglas-Peucker algorithm3. Clean up (merge lines across small gaps, discard short lines)4. Separate vertical and non-vertical lines5. Door candidates given by all the vertical line pairs whose spacing
is within a given range
Canny edges detected lines
vertical lines
non-vertical lines
![Page 10: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/10.jpg)
Cue #1: Color
N
i door
N
i doorwall
i
ii
1
1
][
][ ],[min
1. Threshold the histogram intersection between the wall color model φwall and the color histogram φdoor computed between two vertical line segments.
2. Wall color model can be built either by hand, or automatically.
positive
negative
![Page 11: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/11.jpg)
• The bottom part of the door is usually untextured.
• Texture energy is computed by summing the magnitude of the gradient in the lower region of the door.
Cue #2: Texture
Ap
pIA
)(||
1
positive negative
![Page 12: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/12.jpg)
Cue #3: Gap Below the DoorIntensity along the line
darker (light off)
positive
negative
brighter (light on)
no gap
![Page 13: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/13.jpg)
Cue #4: Kick Plate
A region R in the segmented image is considered as kick plate if:
1. the region R is located between two vertical lines2. the bottom of R is near the bottom of the two vertical lines3. the width and height of R are within a specified range
R
• Kick plates occur in 30% of images• Segmentation algorithm of Felzenszwalb et al., IJCV 2004 is
used (15 fps on 160x120 downsampled image)
positive negative
![Page 14: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/14.jpg)
Cue #5: Vanishing Point• The vanishing point is computed as the mean of the intersection
of pairs of non-vertical lines.
The bottom line of a door should intersect the vanishing point
A distracting line caused by shadowsdoes not intersect the vanishing point
j
j
j
jii
i
i
y
x
cba
cba
wwvwv
,
• If the bottom door edge passes near the vanishing point, then the test succeeds.
![Page 15: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/15.jpg)
Cue #6: Concavity
wall walldoor
Slim “U”
intersection line of wall and floor
extension of intersection line
bottom door edge
vertical door lines
ε
Lleft
LRight
floor
![Page 16: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/16.jpg)
Detect Concavity
Concavity is declared if at least two of the three tests (Hrec(L), Hrec(R), and HU) succeed.
Slim “U”
Extended line
positive negative
U"" slim Find
),( ),( maxmin
U
rec
HRLH
![Page 17: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/17.jpg)
Douglas-Peucker Algorithm
Canny edges Edge labeling Line segmenting
dallowed
![Page 18: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/18.jpg)
Modified Douglas-Peucker Algorithm
Note: important for concavity cue
original algorithm: dallowed is constantmodified algorithm: dallowed is given by half-sigmoid function
dallowed
original modified
![Page 19: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/19.jpg)
• The strong classifier is given by a weighted sum of the weak classifiers:
where weight , error
Adaboost• Bayes decision rule: declare a door if the a posteriori
probability of the predicate is greater than that of its complement:
where )|()|(ψ
1
IDhIDn
iii
,1)|(ψexp)|(
)|(
ID
IDPIDP
)(sign)(ψ1
xhxN
nnn
n
nn
1ln21
iin
niyxhiniinDrn iDyxhP
)(:~ )()(
![Page 20: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/20.jpg)
Experimental Results: Similar or Different Door/Wall Color
![Page 21: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/21.jpg)
Experimental Results: High Reflection / Textured Floors
![Page 22: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/22.jpg)
Experimental Results: Different Viewpoints
![Page 23: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/23.jpg)
Experimental Results: Cluttered Environments
![Page 24: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/24.jpg)
Results
20 different buildings 309 images:• 100 training• 209 testing
90% accuracy with 0.05 FP per image
Speed: 5 fps (unoptimized)
![Page 25: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/25.jpg)
False Negatives and Positives
distracting reflectionconcavity and bottom gap tests fail
strong reflection
concavity erroneously detected
two vertical lines unavailable
distracting reflection
![Page 26: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/26.jpg)
Navigation in a Corridor• Doors were detected and tracked from frame to frame.• Fasle positives are discarded if doors were not
repeatedly detected.
![Page 27: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/27.jpg)
Video
![Page 28: Visual Detection of Lintel-Occluded Doors from a Single Image](https://reader036.vdocuments.us/reader036/viewer/2022070421/568160e5550346895dd01839/html5/thumbnails/28.jpg)
Conclusion• Conclusion
– Detect doors using a single uncalibrated camera in a variety of environments
– Augment standard features (color, texture, and vertical edges) with novel features (concavity, door gap, kick plate, and vanishing point)
– The features are combined in an Adaboost framework– Suitable for real-time mobile robot applications using an off-
the-shelf camera• Future work
– on-line learning of hall color– building of a geometric map– detecting open doors