Decision-Making on Robots Using POMDPs and Answer Set Programming

Introduction

• Robots are an integral part of many sectors such as medicine, disaster rescue and the military.

• Real-world domains are characterized by partial observability, non-deterministic action outcomes and unforeseen dynamic changes.

• Probabilistic planning using partially observable Markov decision processes (POMDPs) models the uncertainty in sensing and actuation on robots in the real-world.

• POMDPs are not well-suited for knowledge representation or common sense reasoning.

• Answer set programming (ASP) is a non-monotonic logic programming paradigm that is appropriate for common sense reasoning.

• This research project investigates the use of a POMDP hierarchy in conjunction with ASP to enable mobile robots to reliably and efficiently localize target objects in indoor domains.

POMDP

• Partially Observable Markov Decision Process.• Defined by the tuple (S, A, T, Z, O, R): States, Actions, Transition Function,

Observations, Observation Function, Reward Function. • The true state cannot be observed. A probability distribution over the state is

maintained (Belief state).• POMDP model solved to obtain a policy that maps belief states to actions.

• Problem formulated as a two-level POMDP.• Low-level POMDP computes the sequence of viewpoints to observe the

target.• High-level POMDP uses information from the low-level POMDP to determine

if the observations correspond to the desired target.

Answer Set Programming

References

1. S. Zhang and M. Sridharan. “Active Visual Sensing and Collaboration on Mobile Robots using Hierarchical POMDPs.” AAMAS 2012.

2. S. Zhang, F. Bao and M. Sridharan. “Combining Probabilistic Planning and Logic Programming on Mobile Robots.” AAAI 2012.

3. S. Zhang, M. Sridharan and X. Li. “To Look or Not to Look: A Hierarchical Representation for Visual Planning on Mobile Robots.” ICRA 2011.

Erratic Robot• Three wheeled• Hokuyo laser range finder (30 meter,

270° range)• Stereo camera• Sonar• On-board processor• Robot Operating System (ROS)

Objectives

• Existing POMDP hierarchy enables the robot to decide where to look, what images to process and how to process these images.

• The current project focuses on robustly identifying objects by observing them from different viewpoints.

• Given an object with multiple sides, create a POMDP to correctly identify the target object.

• Use ASP to model common sense reasoning. • Program robot to recognize initial position.• Algorithms implemented and evaluated in simulation and on wheeled robots

in complex indoor domains.

Future Work

• Combine novel POMDP hierarchy with previous research.• Enable robot to set initial position using environment clues.

* This material is based upon work supported by the National Science Foundation under Grant No. CNS-1005212. Any opinions, findings, and conclusions or recommendations

expressed in this material are those of the author(s) and do not necessarily reflect the reflect the views of the National Science

Foundation.

High-Level POMDP

Low-Level POMDP

Is this the desired target?Do I have enough information

to make that decision?

Which side do I observe, if any?

Should I move clockwise? Counter-clockwise?

Localization

Electronics

Office

Shredder

Printer

Living Room

TV

DVD Player

• Logical reasoning used to infer location of similar objects.

• Hierarchy(pictured right) part of robot’s knowledge base.

• Categories: Electronics, Office, Living Room.

• Objects: Shredder, Printer, TV, DVD Player.

• Rule: If object-X is found in room-Y and object-X is in category-Z, objects of category-Z can be found in room-Y.

• To find targets, robot must know initial position in learned map.

• Previously, initial position was set by clicking location on map.

• Initial position can now be set in code and on map.

• Map with robot position initialized: blue/green represents obstacles, red represents particle cloud.

Integration of POMDP and Answer set programming

Christian WashingtonLouisiana State University

Department of Electrical and Computer Engineering

Shiqi Zhang, Mohan SridharanTexas Tech University

Department of Computer Science