_Daniela Doroftei
Royal Military SchoolAv. De la Renaissance 30, B1000 Brussels,
Belgium
Contribution to ViewFinder – WP2
The goal of this research project is to prepare the RobuDem robot for an outdoor crisis management task.
2
To achieve this goal, the robot must be able to: Be tele-operated by a remote user Ensure its own safety by avoiding obstacles
detected by its sensors (sonar, stereo, …) Navigate autonomously in an unknown
environment by mapping the surroundings Detect chemical contamination Navigate to pre-defined goal positions Execute complex tasks like searching for human
victims Here we will focus on the design of the
control architecture for such a robot3
RobuDem-platform: Outdoor all-terrain robot 4 driving / steering wheels allowing for
different drive modes Supports heavy loads (up to 300kg)
4
Ultrasound Sensors for obstacle detection
Joystick for remote control
Chemical sensor with integrated temperature sensor
5
Differential GPS (and wheel encoders) for positioning
6
Quad-cam vision system consisting of a Bumble-bee stereo vision system and 2 digital cameras.
7
SteerSteerRobudemRobudem
SensorsSensors
BehaviorBehaviorProcessorsProcessors
VisualVisualProcessorsProcessors
Positioning Positioning & Mapping& Mapping
8
SteerSteerRobudemRobudem
JoystickJoystick
Transmission of joystick commands to the robot:
FilterFilterCommandsCommands
Transmission of filtered commands to the robot:
9
SteerSteerRobudemRobudem
JoystickJoystick
FilterFilterCommandsCommands
ObeyObeyJoystickJoystick
BehaviorBehaviorControllerController
In semi-autonomous control, there is no longer a direct link between joystick and robot actuators:
Instead, there is a Behavior-Based Controller managing the commands sent to the robot
The “Obey Joystick” behavior is only 1 of the multiple behaviors contributing to the global robot control strategy 10
These commands are fused by a behavior-based controller, taking into account both behaviors to come to a globally optimal and consistent command to be sent to the robot
The joystick returns the user commands, telling the robot where to go.
The “ObeyJoystick” behavior calculates the best action to perform in order to follow the users’ commands.
ObeyObeyJoystickJoystick
The on-board sonars deliver information about obstacles in the robots’ path.
The “AvoidObstaclesUsingSonar” behavior calculates the best action to perform in order not to bump into the detected obstacles.
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar
SonarSonarJoystickJoystick
These commands are filtered and sent to the robot
11
General requirements: The robot needs accurate positioning The robot needs to build up a model (map) of its
environment to reason with this data
Robot Safety: The robot needs to avoid areas with excessive heat The robot needs to detect chemicals and avoid
areas which are too contaminated The robot needs to detect obstacles using its sonar
sensors and its stereo vision system and avoid these obstacles
The robot needs to avoid all previously detected obstacles stored in the environmental model (map)
12
Robot Goals: The robot must be tele-operable The robot needs to detect chemicals and find the
source of contamination The robot should maximize the knowledge about
the environment it is put in. The robot needs to search for human victims on
the disaster site The robot needs to be able to execute a user-
defined trajectory, given through a set of waypoints
In the event of a loss of network connection, the robot should be able to return to the base station13
These 2 positioning estimates are fused a first time to come to a more accurate and robust position estimate
A Visual Simultaneous Localization and Mapping module analyses all this input data. It builds up an environmental model (map) and places the robot accurately on this map.
Data from the rotation of the wheels, measured by wheel encoders, is used to estimate the position
VisualVisualSLAMSLAM
CameraCameraFramegrabberFramegrabber
GISMapGISMapGPSGPSOdometryOdometry
PositionPositionEstimationEstimation
A real-time differential GPS system provides absolute positioning data by acquiring signals from at least 4 space-based satellites
If available, GIS (Geographic Information System) data is used to initialize the maps
The environ-ment is observedby a camera, which detects and tracks features in the environment to estimate the robot motion and to update the map
Environmental model 6-dimensional position estimate
14
A Visual SLAM module delivers robot position and a map
A chemical sensor instantaneously measures contaminant concentrations
A temperature sensor measures the temperature
This data is used to build up 2 maps containing the chemical and heat distribution
Using this map, 2 behaviors “AvoidChemicals” and “AvoidHotZones” try to steer the robot away from danger zones
VisualVisualSLAMSLAM
AvoidAvoidChemicalsChemicals
AvoidAvoidHotZonesHotZones
LocalLocalHeatMapHeatMap
ChemicalChemicalSensorSensor
Tempe-Tempe-raturerature
LocalLocalChemicalMapChemicalMap
15
The robot is controlled using a behavior-based controller which sends steering commands.
To generate these commands, the controller aims to fuse a number of objectives / tasks / requirements:
General requirements: The robot needs accurate positioning The robot needs to build up a model (map) of its
environment to reason with this data
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
16
Robot Safety: The robot needs to avoid areas with excessive
heat The robot needs to detect chemicals and avoid
areas which are too contaminated
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
17
Robot Safety: The robot needs to avoid all previously detected
obstacles stored in the environmental model (map)
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
18
Robot Safety: The robot needs to
detect obstacles using its sonar sensors and avoid these obstacles
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonar
19
Robot Safety: The robot needs
to detect obstacles using its stereo vision system and avoid these obstacles
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
StereoStereoFramegrabberFramegrabber
20
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
StereoStereoFramegrabberFramegrabber
ObeyObeyJoystickJoystick
JoystickJoystick
Robot Goals: The robot must be
tele-operable
21
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
StereoStereoFramegrabberFramegrabber
ObeyObeyJoystickJoystick
JoystickJoystick
GoToGoToChemicalsChemicals
Robot Goals: The robot needs to
detect chemicals and find the source of contamination
22
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
StereoStereoFramegrabberFramegrabber
ObeyObeyJoystickJoystick
JoystickJoystick
GoToGoToChemicalsChemicals
Robot Goals: The robot should
maximize the knowledge about the environment it is put in
MaximizeMaximizeTerrainKnowledgeTerrainKnowledge
23
Humans are searched for in each of the 4 camera images
A “Search Humans” behavior directs the robot in the persons’ direction
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
StereoStereoFramegrabberFramegrabber
ObeyObeyJoystickJoystick
JoystickJoystick
GoToGoToChemicalsChemicals
Robot Goals: The robot needs to
search for human victims
MaximizeMaximizeTerrainKnowledgeTerrainKnowledge
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
CameraCameraFramegrabberFramegrabber
SearchSearchHumansHumans
24
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
StereoStereoFramegrabberFramegrabber
ObeyObeyJoystickJoystick
JoystickJoystick
GoToGoToChemicalsChemicals
MaximizeMaximizeTerrainKnowledgeTerrainKnowledge
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
CameraCameraFramegrabberFramegrabber
SearchSearchHumansHumans
GoToGoToGoalsGoals
GlobalGlobalPathPlannerPathPlanner
GoalGoalAssignerAssigner
25
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
StereoStereoFramegrabberFramegrabber
ObeyObeyJoystickJoystick
JoystickJoystick
GoToGoToChemicalsChemicals
MaximizeMaximizeTerrainKnowledgeTerrainKnowledge
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
CameraCameraFramegrabberFramegrabber
SearchSearchHumansHumans
GoToGoToGoalsGoals
GlobalGlobalPathPlannerPathPlanner
GoalGoalAssignerAssigner
ReturnReturnToBaseToBase
ConnectionConnectionCheckerChecker
26
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
LocalLocalHeatMapHeatMap
LocalLocalChemicalMapChemicalMap
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonar SonarSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
StereoStereoFramegrabberFramegrabber
ObeyObeyJoystickJoystick
JoystickJoystick
GoToGoToChemicalsChemicals
MaximizeMaximizeTerrainKnowledgeTerrainKnowledge
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
CameraCameraFramegrabberFramegrabber
SearchSearchHumansHumans
GoToGoToGoalsGoals
GlobalGlobalPathPlannerPathPlanner
GoalGoalAssignerAssigner
ReturnReturnToBaseToBase
ConnectionConnectionCheckerChecker
TaskTaskAssignerAssigner
27
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
GoToGoToChemicalsChemicals
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
PersonPersonDetectorDetector
SearchSearchHumansHumans
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
GoToGoToGoalsGoals
ReturnReturnToBaseToBase
ObeyObeyJoystickJoystick
GlobalGlobalPathPlannerPathPlannerLocalLocal
HeatMapHeatMap
LocalLocalChemicalMapChemicalMap
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
CameraCameraFramegrabberFramegrabber
StereoStereoFramegrabberFramegrabber
SonarSonar
JoystickJoystick
TaskTaskAssignerAssigner
ConnectionConnectionCheckerChecker
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
GoalGoalAssignerAssigner
MaximizeMaximizeTerrainKnowledgeTerrainKnowledge
28
Using this modular behavior based control framework the robot can: Be tele-operated by a remote user Ensure its own safety by avoiding obstacles
detected by its sensors (sonar, stereo, …) Navigate autonomously in an unknown
environment by mapping the surroundings Detect chemical contamination Navigate to pre-defined goal positions Execute complex tasks like searching for human
victims
29
30
FilterFilterCommandsCommands
SteerSteerRobudemRobudem
FuseFuseBehaviorsBehaviors
AvoidObstaclesAvoidObstaclesUsingSLAMUsingSLAM
GoToGoToChemicalsChemicals
VisualVisualSLAMSLAM
PositionPositionEstimationEstimation
AvoidObstaclesAvoidObstaclesUsingSonarUsingSonarAvoidObstaclesAvoidObstacles
UsingStereoUsingStereo
PersonPersonDetectorDetector
SearchSearchHumansHumans
AvoidAvoidChemicalsChemicals AvoidAvoid
HotZonesHotZones
GoToGoToGoalsGoals
ReturnReturnToBaseToBase
ObeyObeyJoystickJoystick
GlobalGlobalPathPlannerPathPlannerLocalLocal
HeatMapHeatMap
LocalLocalChemicalMapChemicalMap
PersonPersonDetectorDetector
PersonPersonDetectorDetector
PersonPersonDetectorDetector
ChemicalChemicalSensorSensor
CameraCameraFramegrabberFramegrabber
CameraCameraFramegrabberFramegrabber
StereoStereoFramegrabberFramegrabber
SonarSonar
JoystickJoystick
TaskTaskAssignerAssigner
ConnectionConnectionCheckerChecker
GISMapGISMap
GPSGPS
OdometryOdometry
Tempe-Tempe-raturerature
GoalGoalAssignerAssigner
MaximizeMaximizeTerrainKnowledgeTerrainKnowledge
31