november 10, 2004
DESCRIPTION
Prof. Christopher Rasmussen [email protected] Lab web page: vision.cis.udel.edu. November 10, 2004. Research in the DV lab. Tracking, segmentation Model-building, mapping, and learning Cue combination and selection Auto-calibration of sensors Current projects: - PowerPoint PPT PresentationTRANSCRIPT
![Page 2: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/2.jpg)
Research in the DV lab
• Tracking, segmentation• Model-building,
mapping, and learning• Cue combination and
selection• Auto-calibration of
sensors• Current projects:
– Road following, architectural modeling
![Page 3: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/3.jpg)
Road Following: Background
• Edge-based methods: Fit curves to lane lines or road borders – [Taylor et al., 1996; Southall & Taylor, 2001; Apostoloff &
Zelinsky, 2003]
• Region-based methods: Segment image based on discriminating charac- teristic such as color or texture
– [Crisman & Thorpe, 1991; Zhang & Nagel, 1994; Rasmussen, 2002; Apostoloff & Zelinsky, 2003]
from Apostoloff& Zelinsky, 2003
![Page 4: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/4.jpg)
Problematic Scenes for Standard Approaches
No good contrast or edges, but organizing feature is vanishing point, which indicates road direction
Grand Challenge sample terrain Antarctic “ice highway”
![Page 5: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/5.jpg)
Results: Curve Tracking
Integrate vanishing point directions to get points along curves parallel to (but not necessarily on) road
![Page 6: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/6.jpg)
Panoramic camera v2.0a
~1.5 inches
![Page 7: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/7.jpg)
Correspondence-based Mosaicing
• Minimum of 4 corresponding points in two images sufficient to define transformation warping one into other
• Can be done manually or automatically
![Page 8: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/8.jpg)
Correspondence-based Mosaicing
Translation only
![Page 9: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/9.jpg)
Road Shape Estimation (3 cameras)
• Road edge tracking– Estimate quadratic curvature via
Kalman filter with Sobel edge measurements
![Page 10: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/10.jpg)
Motion-based Mosaicing
• It’s possible to make mosaics of cameras with non-overlapping fields of view provided we have sequences from them (Irani et al., 2001)– Overlapping pixels are wasted pixels
• We’re working on approaches for n cameras > 2
![Page 11: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/11.jpg)
Motivation: DARPA Grand Challenge
• Organized by DARPA (the U. S. Defense Advanced Research Projects Agency)
• A robot road race through the desert from Barstow, CA to Las Vegas, NV on March 13, 2004
• Prize for the winning team: $1 million (nobody won)
• Running again next October with $2 million prize
![Page 12: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/12.jpg)
Caltech’s 2004 DGC entry “Bob”
![Page 13: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/13.jpg)
Problem: How to Use Roads as Cues?
Bob’s track relative to course corridors
(No road following)
We’re working on integrating camera views from vehicle with aerial photos
![Page 14: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/14.jpg)
Tracing Roads in Aerial Photos
![Page 15: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/15.jpg)
Structure-based Obstacle Avoidance with a LADAR
![Page 16: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/16.jpg)
Merging Structure into Local Map
• Integrate raw depth measurements from several successive frames using vehicle inertial estimates
• Combine with camera information• We’re working on calibration techniques
courtesy of A. Zelinsky
![Page 17: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/17.jpg)
Laser-Camera Registration
Range image (180 x 32)90° horiz. x 15° vert.
Video frame (360 x 240)
Registeredlaser, camera
![Page 18: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/18.jpg)
3-D Building Models from Images
courtesy of F. van den HeuvelShow VRML model
![Page 19: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/19.jpg)
Robot Platform for Mapping Project
PTZ camera
Wirelessethernet GPS antenna
Onboard computer
Analog video capture card
Not shown: electronic compass, tilt sensor
![Page 20: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/20.jpg)
View Planning
• Where to take the photos from?• Hard constraints: Need overlapping fields of view for stereo
correspondences• Soft constraints: Balance accuracy of estimated 3-D model,
quality of appearance (texture maps) with acquisition, computation time– Based on camera field of view, height of building, placement of
occluding objects like trees and other buildings
![Page 21: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/21.jpg)
Path Planning
• How to get a robot from point A to point B?– Criteria: Distance, difficulty, uncertainty
![Page 22: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/22.jpg)
Path Planning
GPS-referenced CAD map of campus buildings is available
Aerial photos contain information aboutpaths, vegetation as well as buildings
![Page 23: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/23.jpg)
Obstacle Avoidance
How to detect trash cans, people, walls, bushes, trees, etc. and smoothly combine detours around them with global path planned from map and executed with GPS?
![Page 24: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/24.jpg)
Segmentation-Based Path Following
![Page 25: November 10, 2004](https://reader035.vdocuments.us/reader035/viewer/2022062304/56813a06550346895da1cff2/html5/thumbnails/25.jpg)
Segmentation of Road Images Using Different Cues
Texture Color +T+L
Laser C+T+L