robotic sensing using - ucmrl.isr.uc.pt/projects/socialrobot/images/stories/rsw/ros_inf... ·...

David Portugal (davidbsp@isr.uc.pt)

Robotic Sensing

using

27th May 2013

Index

• Robotic Sensing Sensor Types and Equipment

Applications

• ROS Overview ROS Organization

How do Nodes talk?

• ROS Drivers for Sensing Equipment Example and Applications

Robotic Sensing

“Branch of robotics science intended to

give robots sensing capabilities, so that robots

are more human-like.”

“Mainly gives robots the ability to see, touch,

hear and move and uses algorithms that

require environmental feedback.”

http://en.wikipedia.org/wiki/Robotic_sensing

Robotic Sensing

Robotic Sensing

Which capabilities should a robot have in

order to interact with its environment?

Primary Human

Senses

Vision

Touch

Hear

Taste

Smell

Robotic Sensing

Other Human

Senses

Balance & Motion

Thermoception

Pain

Time Perception

…

Robotic Sensing

Robotic Sensing

Vision

Touch

Hear

Smell

Robotic Sensing

e.g., color camera, TOF camera,

3D camera, …

e.g., touch sensor, force sensor, gripper…

e.g., microphone, acoustical arrays, …

e.g., gas sensor, alcohol sensor, …

Other Robotic

Senses

Balance & Motion

Chemical Perception

Distance Sensing

…

Robotic Sensing

e.g., odometer, compass,

accelerometer, GPS, …

e.g., humidity sensor, dust sensor, light

sensor, thermal camera …

e.g., laser range finder, ultrasonic sensors,

infrared sensor…

Robotic Sensing

Robotic Sensing - Example

RGB Camera

• Delivers the three basic color

components (red, green, and

blue) on three different wires.

• Used for accurate color image

acquisitions.

• Using cameras, robots may perceive visual information

through computer vision and image processing

algorithms.

Robotic Sensing - Example

Laser Range Finder

• Scans the environment using laser

beams to determine the distance to

objects.

• Used for distance sensing, obstacle

detection, navigation,…

• Robots are able to build 2D representations

of their environment and/or navigate safely

without hitting obstacles.

Robotic Sensing - Example

Microphone

• Acoustic-to-electric sensor that

converts sound into an electrical

signal.

• Microphones may form an array, in

order to enhance speech recognition,

detecting sound sources, etc.

• Robots may localize objects by their sound, interact

with humans, etc. using digital audio signal processing.

Robotic Sensing – Applications

Vision: Optical Flow

http://www.youtube.com/watch?v=V4r2HXGA8jw

Robotic Sensing – Applications

Touch: Object Manipulation

http://www.youtube.com/watch?v=2rd2r2jPWLU

Robotic Sensing – Applications

Hear: Detecting Sound Sources

Robotic Sensing – Applications

Smell: Searching for Odor Sources

http://www.youtube.com/watch?v=Av2fT8iXnqw

Robotic Sensing – Applications

Motion and Range Sensing: Robot Navigation

Robotic Sensing

Sensors should be chosen according to the application

domain and intended robot’s behavior.

Overview

Overview

A popular robot framework

that provides:

• Hardware abstraction

• Low-level device control

• Implementation of commonly-

used functionalities

• Message-passing between

processes

• Package management

Overview

• Peer-to-peer

• Tools-based

• Multi-lingual

• Free

• Open-Source

Command-line Tools

• rosbag

• rosbash

• roscd

• rosclean

• roscore

• rosdep

• rosed

• rosrun

• roscreate-pkg

• roscreate-stack

• rostopic

• roslaunch

• roslocate

• rosmake

• rosmsg

• rosnode

• rospack

• rosparam

• rossrv

• rosservice

• rosstack

Philosophy: Various tools instead of a runtime environment

• Nodes

• Messages

• Services

Groups of packages

Packages

Stacks

Organization

How do Nodes talk?

Node A Node B

Messages

Topic

How do Nodes talk?

Example:

Talker node (1)

Publisher (string message)

How do Nodes talk?

Example:

Talker node (2)

String to publish

(loopwise)

Publishing the

String Message

How do Nodes talk?

Example:

Listener node

Subscriber

(string message)

Callback Function (called

when a new message arrives)

How do Nodes talk?

Example:

Output

How do Nodes talk?

Example:

Output

Sensing in ROS

• After building and developing a robot (or perhaps

buying a new robot):

How can we integrate it in ROS?

Can we add sensors to our robot and easily

access their information?

Yes we can!

Drivers

for Sensing Equipment

Interacting with different sensors

Microsoft Kinect Hokuyo Laser Sick Laser

3D PMD Camcube

Xsens Mti IMU

Mercury5e RFID Reader Arduino Integration BMA Accelerometer

• Expanding robots’ capabilities may be crucial

depending on the application;

• In multi-robot systems you may want to have

heterogeneous teams, where each agent has a

distinct role;

• How difficult is it to add a new sensor to your

robot?

Interacting with different sensors

Drivers

Sensor

ROS

Driver

Other ROS

Nodes

Access to Raw

Readings from the

Sensor

Serial-USB, I2C,

Bluetooth …

ROS Topics or

ROS Services Interpret sensor

data and make it

available for ROS

through an

adequate message

structure Applications that rely

on sensor information

to run properly

Drivers – Example

Wiimote

(Nintendo)

Drivers – Example

Application: Robot Teleoperation

Drivers – Example

http://www.ros.org/wiki/wiimote

ROS Driver for the wiimote, available at:

The wiimote_node (wiimote package, joystick_drivers

stack) exposes the wiimote controller functionalities to

ROS.

Drivers – Example

http://www.ros.org/wiki/wiimote

ROS Driver for the wiimote, available at:

“The package implements a ROS node that uses Bluetooth

to communicate with the Wiimote device, obtaining

accelerometer and gyro data, the state of LEDs, the IR

camera, rumble (vibrator), buttons, joystick, and

battery state. The node additionally enables ROS nodes

to control the Wiimote's LEDs and vibration for feedback to

the human Wiimote operator.”

Drivers – Example

wiimote_node.py

Drivers – Example

wiimote_node.py

Drivers – Example

Application: Robot Teleoperation

http://www.youtube.com/watch?v=WgcGnOoNrBU

Drivers – Example

Application: Robot Teleoperation

Later on…

Microsoft Kinect

Color VGA video camera

3D Depth sensor

Multi-array microphone

Some Applications:

3D scene reconstruction

Gesture Recognition

Detection of sound sources

Later on…

Arduino Uno Open-source electronics prototyping

platform.

The microcontroller is programmed

using the Arduino language (similar to

C/C++) and the Arduino development

environment.

Arduino projects can be stand-alone

or they can communicate with

software running on a computer.

Some Applications: Sense the environment by receiving input from a variety of

sensors, affecting its surroundings by controlling lights, motors,

and other actuators.”

Later on…

Hokuyo URG-04LX-UG01

2D Laser Range Finder

Wide-range (5.6m × 240° FOV).

Accurate (± 30mm)

Angular resolution: approx. 0.35°

(360° / 1024 steps)

Scan Frequency: 10Hz

Some Applications:

Simultaneous Localization and Mapping (SLAM)

Robot Navigation

Object Detection

Later on…

Xsens Mti

Inertial Measurement Unit (IMU)

Output:

• 3D Orientation (360°)

• 3D acceleration

• 3D rate of turn

• 3D magnetic field

Some Applications:

Tracking object position in 3D

Control and stabilization of cameras and robots

Conclusion

• Several available sensing equipment can be

incorporated in a robot, depending on the application,

to improve the robot’s capabilities.

• ROS enables effortless integration of sensors and

robot hardware;

• ROS drivers may interface with ROS tools for data

processing and analysis of the robotic platform;

• ROS tools and software promote code reuse,

hardware abstraction and decrease the development

time;

• Many popular sensors used in robotic applications

are already integrated in ROS (e.g., kinect, hokuyo

and SICK lasers, wiimote, Xsens IMU, …)

• In order to interact a given sensor in ROS, all you

need to do is to expose its functionalities to ROS

through a ROS driver.

Conclusion

Thank You.

Questions?