detection & identification of people & animals by exploration robots
DESCRIPTION
Craig Hahn April 13, 2009. Detection & Identification of People & Animals by Exploration Robots. Importance. robots must interact safely with humans and animals Image: http://depletedcranium.com/riman. Importance. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/1.jpg)
Detection & Identification of People & Animals by Exploration Robots
Craig Hahn
April 13, 2009
![Page 2: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/2.jpg)
Importance
• robots must interact safely with humans and animals
Image: http://depletedcranium.com/riman
![Page 3: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/3.jpg)
Importance
• many “exploration” applications involve close interaction with humans (such as search-and-rescue operations)
• people and animals are often the most dynamic part of a real-world environment
• humans and animals must be identified quickly in all such situations
![Page 4: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/4.jpg)
Basic Methods
• Stationary face detection using basic pattern recognition techniques
Image: http://www.motorauthority.com/wpcontent/uploads/odds_and_ends/2008/3/2/face_pic.jpg
![Page 5: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/5.jpg)
Facial Recognition
• Easily done but of little use to mobile robots:
– robot and target likely moving
– faces are not always visible
Image: http://www-robotics.jpl.nasa.gov/roboticImages/img840-192-browse.jpg
![Page 6: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/6.jpg)
Basic Methods
• Tracking using stereo vision cameras• problem:
– distinguishing people from their surrounding environment, especially moving people
![Page 7: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/7.jpg)
Basic Methods
• Use laser rangefinders to detect larger moving objects (not just looking at people’s faces)– But again you can’t always distinguish people from their
surrounding environment using only laser rangefinders– possible problems:
» people too close to other objects in environment (walls)» other objects (tables, chairs, etc.) mistaken for people
![Page 8: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/8.jpg)
Better Idea
• Combine data from multiple sources• optical sensors• laser rangfinders• auditory systems• cameras• pattern recognition• sonar
![Page 9: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/9.jpg)
Kyoto University
• Created a robot that combines facial recognition and auditory data
• detect people by looking for faces
• track them using stereo vision and sound source detection
![Page 10: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/10.jpg)
Autonomous Systems Lab Swiss Federal Institute of Technology
Lausanne
• Mobile robot Robox finds (moving) objects by detecting changes in a static environment with a laser rangefinder...
![Page 11: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/11.jpg)
Robox
• …then checks such objects using visual cues to determine if they are people
![Page 12: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/12.jpg)
Department of Neuroinformatics and Cognitive Robotics
Ilmenau Technical University
– HOROS• laser-rangefinder • sonar system • omnidirectional
camera
![Page 13: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/13.jpg)
sensory input from each sensing system
![Page 14: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/14.jpg)
- assigns each set of data a Gaussian probability distribution
(,C)- mean represents the position of the detection- covariance matrix C represents the uncertainty about the
position
![Page 15: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/15.jpg)
reasonably successful in a real-world environment
![Page 16: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/16.jpg)
Advantages
• HOROS doesn’t rely on just one sensing system
• easy to add more sensing systems• just factor in another Gaussian distribution
![Page 17: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/17.jpg)
Centre for Applied Autonomous Sensor Systems Örebro University
• Mobile robot identifies people in an indoor environment – laser rangefinder– camera
• Learns a color model of a person
• hair• skin• clothes• shoes
![Page 18: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/18.jpg)
• laser rangefinder scans the environment
• detects and tracks people
• local minima correspond to the legs of a person
![Page 19: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/19.jpg)
• image from camera is broken down into three parts
![Page 20: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/20.jpg)
• color data is extracted from each part of the image
• hue-saturation-value color model
• color data is put into self-organizing maps (SOMs)
• color model of a person is learned based on the best fit
![Page 21: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/21.jpg)
Assignment• Come up with another robotic system that combines
three or more different sensing methods to identify humans or animals, and tell the difference between a human and a humanoid robot– explain:
• why you chose the methods that you did• how each method can be used to identify humans or animals and
differentiate humans from humanoid robots• how the data from each can be integrated together in a
meaningful way• if applicable, identify specific equipment that could be used to
implement your system
![Page 22: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/22.jpg)
Supply Companies• SICK laser rangefinders:
Image: http://www.pages.drexel.edu/~kws23/tutorials/sick/sickLMS200.jpg
• widely used
![Page 23: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/23.jpg)
Omnidirectional Cameras• Nikon
• Sony
• Remote Reality
Image: http://blog.wired.com/photos/uncategorized/2008/07/01/hal9000_focus_jpg.jpg
![Page 24: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/24.jpg)
Other Important Companies
• JAXY Optical Instrument Company (laser radar systems, sonar systems
• Opnext, Inc. (laser rangefinders)
• Instruments, Inc. (amplifiers, sonar systems, range sensors)
![Page 25: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/25.jpg)
Cutting-Edge Researchers• Ilmenau Technical University
Department of Neuroinformatics and Cognitive Robotics
• Örebro University Dept. of Technology, Centre for Applied Autonomous Sensor Systems
• Kyoto University Department of Intelligence Science and Technology
• Swiss Federal Institute of Technology Lausanne Autonomous Systems Lab
• Albert-Ludwigs-Universität Freiburg Social Robotics Laboratory, Institut für Informatik
![Page 26: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/26.jpg)
Future Improvements
• Improvements are expected in:– identifying more dynamic
objects (people and animals)
– differentiating between multiple objects of the same type
– differentiating between many different kinds of moving people and objects
• being worked on at Albert-Ludwigs-University in Freiburg, Germany
![Page 27: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/27.jpg)
Other Improvements
– identifying and classifying the behaviors of humans and animals
– same accuracy of prediction and detection with:• less data
• smaller periods of observation
• can reach conclusions more quickly (less computational time)
![Page 28: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/28.jpg)
References• Sensor Fusion Using a Probabilistic Aggregation Scheme For
People Detection and Tracking Department of Neuroinformatics and Cognitive Robotics Ilmenau Technical University <http://www.tu-ilmenau.de/fakia/fileadmin/template/startIA/neuroinformatik/publications/conferences_int/Martin-ECMR-05b.pdf>
• Person Identification by Mobile Robots in Indoor Environments Centre for Applied Autonomous Sensor Systems Dept. of Technology, O¨ rebro University <ftp://aass.oru.se/pub/tdt/rose03.pdf>
• Unsupervised Learning and Classification of Dynamic Objects Social Robotics Laboratory Institut für Informatik Albert-Ludwigs-Universität Freiburg <http://srl.informatik.uni-freiburg.de/papers/luberRSS08.pdf>
![Page 29: Detection & Identification of People & Animals by Exploration Robots](https://reader034.vdocuments.us/reader034/viewer/2022051821/568157d6550346895dc55d63/html5/thumbnails/29.jpg)
References• Auditory fovea based speech separation and its application to
dialog system Kyoto University, Department of Intelligence Science and Technology <http://thamakau.usc.edu/Proceedings/ICSLP%202002/ICSLP/PDF/AUTHOR/SL021615.PDF>
• Robox at expo.02: A large scale installation of personal robots Swiss Federal Institute of Technology Lausanne, Autonomous Systems Lab <http://infoscience.epfl.ch/record/97516>