tour guide robot project face detection and face orientation on the mobile robot robotino gökhan...
TRANSCRIPT
Tour Guide Robot Project
Face Detection and Face Orientation on The Mobile Robot Robotino
Gökhan Remzi YavuzAyşenur Bilgin
Outline
IntroductionHardwareProject RequirementsProject ProgressQuestions
Introduction
Human Computer Interaction - HCIDetecting human facesDetecting interested people
Face detection Face orientation and face rotation
Keeping people interested by interacting with them
Getting closer when a certain distance is exceeded
Hardware
Robotino: a mobile robot platform from Festo
To satisfy project's requirements, we are planning some extensions on Robotino: Cylindirical body, total height: 1.60m including camera A laptop computer, attached to body at 1.20m Second color camera on top of the laptop computer dedicated to HCI
9 infrared sensors3 omnidirectional drive unitsBump sensorColor camera
Control Unit: a PC with a 300 Mhz Processor and 128 MB SDRam
Hardware con’t
• Sketch of hardware modifications on the robot base Robotino
• 1 additional laptop computer, 2 additional color cameras, skeleton with steel sticks and not yet sketched cylindirical cover for better appearance
Project Requirements
A tour guide robot should:
be able to navigate autonomously and purposefully inside the building
operate in a dynamic environment
attract visitors' attention by interacting with them
be able to monitor its charge level and move to its charging dock on its own
be able to dynamically arrange tours by combining selected destinations
deliver information in forms of visual, audio and text messages
be able to synchronize with the CMPE web site, in order to get up to date information.
Project Progress
What we did? Architecture Pseudocode Examples
What we did not?What we encountered?What we achieved?
What we did?
Combining Robotino API and OpenCVModifying the open source code to satisfy our needsRunning the modified algorithm on the mobile robotTesting the behavior
Architecture
Robotino
RobotinoAPI
Application OpenCV RobotinoAPI
Watchdog Signal Action Requests Camera Requests Camera Data
Application
Action Requests Camera Data
OpenCV
Image Data Face Information
Wireless
Pseudocode
Connect to Robotino over Wireless
While bumpsensorvalue = false do
Retrieve camera image from RobotinoAPI
Process image
. OpenCV face detection using haarcascades, frontal and profile faces
. Compute approximate distances of faces and their angle to the robot
Decide action; rotate, move forward, attract attention, stop
Act
end While
Examples
Examples cont’d
What we did not?
Pose estimationTrackingFace recognitionCompletion of hardware modifications
What we encountered?
Major challenge : Erroneous detection of faces
Processing power Minor problems :
Relatively low (10 fps) performance when frontal and profile face detection algorithms are combined
Position of the camera
What we achieved?
Experiments held in laboratoryMoving towards the human face until the
distance is reduced to 1 meter approximately
Turning with respect to the weighted average of areas of detected faces
Questions???
Thank you!