Simultaneous Simultaneous Localization and Map Localization and Map Building System for Building System for

Prototype Mars RoverPrototype Mars Rover

CECS 398CECS 398

Capstone Design ICapstone Design I

October 24, 2001October 24, 2001

Project Mentors:Project Mentors:Dr. Jeffrey UhlmannDr. Jeffrey UhlmannDr. Marjorie SkubicDr. Marjorie Skubic

Project Members:Project Members:

Karen CaseyKaren Casey

Kenneth EstesKenneth Estes

Laura HeffernanLaura Heffernan

OverviewOverview

Problem DefinitionProblem Definition BackgroundBackground Goals and ObjectivesGoals and Objectives

OverviewOverview

Requirements AnalysisRequirements Analysis System ComponentsSystem Components ConstraintsConstraints RequirementsRequirements Alternative ApproachesAlternative Approaches Testing MethodsTesting Methods SchedulingScheduling

Background Background Covariance Intersection Research and Covariance Intersection Research and DevelopmentDevelopment

Naval Research LaboratoryNaval Research Laboratory Decentralized data fusion problemsDecentralized data fusion problems

Dynamic map building and localizationDynamic map building and localization

NASA Mars RoverNASA Mars Rover Onboard data fusion systemOnboard data fusion system

Simultaneous Localization and Map Simultaneous Localization and Map Building System (SLAM)Building System (SLAM)

Background Background Covariance Intersection Research and Covariance Intersection Research and DevelopmentDevelopment

Sojourner rover on Mars taken by Sojourner rover on Mars taken by the Mars Pathfinder Landerthe Mars Pathfinder Lander

Background Background Covariance Intersection Research and Covariance Intersection Research and DevelopmentDevelopment

What is data fusion?What is data fusion? Demands a method of combining Demands a method of combining

information from multiple sourcesinformation from multiple sources

Covariance IntersectionCovariance Intersection (CI) (CI) consistently provides a consistently provides a conservatively fused estimate of conservatively fused estimate of the input.the input.

Background Background Kalman Filter vs. CI FilterKalman Filter vs. CI Filter

Kalman FilterKalman Filter Represents information about Represents information about

estimated or measured quantities in estimated or measured quantities in terms of a mean and a covariance terms of a mean and a covariance matrixmatrix

Can combine estimates with a Can combine estimates with a known known degree of independencedegree of independence

Background Background Kalman Filter vs. CI FilterKalman Filter vs. CI Filter

CI FilterCI Filter Does not make any assumptions Does not make any assumptions

about the degree of independence about the degree of independence between information it fusesbetween information it fuses

Exhibits considerably more stable Exhibits considerably more stable behavior than Kalman filtersbehavior than Kalman filters

Background Background Kalman Filter vs. CI FilterKalman Filter vs. CI Filter

Kalman Filter:Kalman Filter: CI Filter:CI Filter:

Equations for combining estimates {a,A} and {b,B}Equations for combining estimates {a,A} and {b,B}

)(

)(11

111

bBaACc

BAC

))1((

))1((11

111

bBaACc

BAC

Equations only differ by the parameter ω.Equations only differ by the parameter ω.

Background Background Kalman Filter vs. CI FilterKalman Filter vs. CI Filter

Principle advantage of CI:Principle advantage of CI: Permits filtering and data fusion Permits filtering and data fusion

without the need to know the degree without the need to know the degree of correlationof correlation between the estimates between the estimates being fusedbeing fused

ApplicationsApplications include simultaneous include simultaneous map building and localization for map building and localization for autonomous vehicles.autonomous vehicles.

GoalsGoals

To prove that CI is a reliable To prove that CI is a reliable solution to the SLAM problemsolution to the SLAM problem

Use a test robot to identify Use a test robot to identify predefined beaconspredefined beacons

Given the information gathered, Given the information gathered, use CI to create a simultaneous use CI to create a simultaneous and localization mapand localization map

ObjectivesObjectives

Research and obtain a robot that will Research and obtain a robot that will be able to move in two dimensions. be able to move in two dimensions. We will also need to be able to get We will also need to be able to get speed information from the robot in speed information from the robot in order to dependably update our map. order to dependably update our map.

Construct a small controlled test Construct a small controlled test bed environment with identifiable bed environment with identifiable beacons. beacons.

ObjectivesObjectives

Develop software that will take the image Develop software that will take the image and extract the relative beacon position we and extract the relative beacon position we will need in order to build a map. will need in order to build a map.

Develop software that will control the robot's Develop software that will control the robot's movements and navigate it around its movements and navigate it around its environment so it can map the beacons in environment so it can map the beacons in the test bed.the test bed.

Use Covariance Intersection to estimate the Use Covariance Intersection to estimate the beacon locations so that the relative map beacon locations so that the relative map can be updated.can be updated.

System ComponentsSystem Components

Palm Pilot RobotPalm Pilot Robot Handspring Visor Prism Handspring Visor Prism Eyemodule2 cameraEyemodule2 camera CI softwareCI software

System Components System Components Palm Pilot RobotPalm Pilot Robot

constructed from constructed from the Palm Pilot the Palm Pilot Robot KitRobot Kit

uses a Palm Pilot uses a Palm Pilot to move around to move around and sense the and sense the nearby nearby environmentenvironment

System Components System Components Handspring Visor PrismHandspring Visor Prism

Used for:Used for: Robot motion Robot motion

controlcontrol Object avoidanceObject avoidance Image capture Image capture

and processing and processing systemsystem

CI programCI program LocalizationLocalization Map buildingMap building

System Components System Components Eyemodule2 CameraEyemodule2 Camera

Serves as Serves as integrated image integrated image capture devicecapture device

Connects directly Connects directly to Visor Prismto Visor Prism

Produces color Produces color imagesimages

System Components System Components CI SoftwareCI Software

Modular Modular program will include:program will include: Robot movement and motor controlRobot movement and motor control Image processingImage processing Coordinate triangulationCoordinate triangulation Covariance matrix calculationsCovariance matrix calculations

ConstraintsConstraints

Vision systemVision system EnvironmentEnvironment

Beacon size and heightBeacon size and height Walls and floorWalls and floor

Color recognitionColor recognition Hardware interface connectionsHardware interface connections MaintenanceMaintenance

Constraints Constraints Vision SystemVision System

Environment:Environment: Beacons of same size and heightBeacons of same size and height White wallsWhite walls White level floorWhite level floor

Color Recognition:Color Recognition: Beacons of different colors for individual beacon Beacons of different colors for individual beacon

identification with largest threshold difference identification with largest threshold difference between thembetween them

Objective is to minimize noise in vision Objective is to minimize noise in vision system.system.

Constraints Constraints Hardware Interface ConnectionsHardware Interface Connections

Need serial Need serial connection connection between Visor and between Visor and robotrobot

Need Need communication communication between between development development software and Visor software and Visor to download codeto download code

Constraints Constraints MaintenanceMaintenance

Minimal maintenance:Minimal maintenance: Connections and wiringConnections and wiring C program for Palm OSC program for Palm OS

OS versionOS version Visor modelVisor model Camera modelCamera model

NASA testing and modificationsNASA testing and modifications

RequirementsRequirements

Cost RequirementsCost Requirements Time: Time:

Research the hardware/software Research the hardware/software components components

Implement the robot and software.Implement the robot and software. Meet with project mentors.Meet with project mentors.

Resources: Resources: Mentors will serve as our basic reference Mentors will serve as our basic reference

tool for background and implementation tool for background and implementation information.information.

RequirementsRequirementsMentors as a ResourceMentors as a Resource

Dr. Marjorie Skubic, Dr. Marjorie Skubic,

Assistant ProfessorAssistant Professor Specialties: Specialties:

Sensory perceptionSensory perception Pattern RecognitionPattern Recognition Intelligent controlIntelligent control RoboticsRobotics

RequirementsRequirementsMentors as a ResourceMentors as a Resource

Dr. Jeffery Uhlmann, Dr. Jeffery Uhlmann,

Assistant ProfessorAssistant Professor Specialties:Specialties:

Kalman FilteringKalman Filtering Statistical AlgorithmsStatistical Algorithms Autonomous Vehicles Autonomous Vehicles

and Roboticsand Robotics Large Scale Large Scale

SimulationSimulation

RequirementsRequirements Cost RequirementsCost Requirements

Resources:Resources: Reference books (CodeWarrior) Reference books (CodeWarrior) Visor developer support for code to access Visor developer support for code to access

the structure that contains pixel the structure that contains pixel information.information.

Covariance Intersection software librariesCovariance Intersection software libraries

Facilities: Facilities: EBW Lab 222 with Artemis.EBW Lab 222 with Artemis.

RequirementsRequirements

Cost RequirementsCost Requirements Money: Money:

Purchase serial connection converter to Purchase serial connection converter to handle communication needs between handle communication needs between the robot and Visor. the robot and Visor.

Materials for the testing environment.Materials for the testing environment.

Performance RequirementsPerformance Requirements No required time limit for system No required time limit for system

response.response.

Alternative Solutions for Alternative Solutions for Finding Beacon DistanceFinding Beacon Distance

Infrared LEDs and Infrared LEDs and sensorssensors ProsPros

InexpensiveInexpensive Have been proven Have been proven

to workto work ConsCons

Need filtersNeed filters Occupies too much Occupies too much

spacespace Adds complexityAdds complexity

Alternative Solutions for Alternative Solutions for Finding Beacon DistanceFinding Beacon Distance

Sonar emitter and Sonar emitter and sensorssensors ProsPros

More information More information about the about the environmentenvironment

ConsCons Too much Too much

informationinformation Adds complexityAdds complexity

TestingTestingNavigation and ControlNavigation and Control

Test limitations of robot controlTest limitations of robot control Successful if the robot is able to Successful if the robot is able to

move about the environment and move about the environment and move within visual range of the move within visual range of the beacons for identificationbeacons for identification

TestingTestingIdentification of BeaconsIdentification of Beacons

Test the ability of our software to Test the ability of our software to distinguish between the distinguish between the background and a beaconbackground and a beacon

Need to be able to identify the size Need to be able to identify the size of the beacon and the position of of the beacon and the position of the center of the beaconthe center of the beacon

TestingTestingIdentification of BeaconsIdentification of Beacons

Use trial and error to create a hash Use trial and error to create a hash table associating the size of the table associating the size of the beacon to the distance to the beaconbeacon to the distance to the beacon

Successful if the software can identify Successful if the software can identify beacons, determine the distance to the beacons, determine the distance to the beacons, and reduce the information to beacons, and reduce the information to a specific XYZ coordinate for a beacona specific XYZ coordinate for a beacon

TestingTestingCI SolutionCI Solution

Hardware is being used in order to Hardware is being used in order to test our CI solutiontest our CI solution

Successful when the robot is able Successful when the robot is able to identify beacons and move to identify beacons and move around while maintaining an around while maintaining an acceptably accurate mapacceptably accurate map

ScheduleSchedule

Test CI Software

10/29Oct 22, 2001 Nov 22, 2001

11/5 11/12 11/19

11/12Nov 8, 2001 Dec 8, 2001

11/19 11/26 12/3

1/21Jan 20, 2002 Feb 11, 2002

1/28 2/4

2/11Feb 6, 2002 Feb 20, 2002

2/18

2/25Feb 18, 2002 Mar 26, 2002

3/4 3/11 3/18

3/25Mar 18, 2002 Apr 1, 2002

4/8Apr 1, 2002 Apr 29, 2002

4/15 4/22

Research and TestRobot

Research Video Camera

Create Image ProcessingSoftware

Test Image ProcessingSoftware

Create CI Software

Test Entire System

ConclusionConclusion

Further refine scope of the projectFurther refine scope of the project Familiarize ourselves with the Familiarize ourselves with the

operation of the system operation of the system componentscomponents

Design the algorithm for image Design the algorithm for image processingprocessing

Continue research into CIContinue research into CI