joelle pineau michael montemerlo martha pollack * nicholas roy sebastian thrun

Probabilistic Control of Human Robot Interaction:Probabilistic Control of Human Robot Interaction:

Experiments with a Robotic Assistant for Nursing HomesExperiments with a Robotic Assistant for Nursing Homes

Joelle PineauMichael MontemerloMartha Pollack *

Nicholas RoySebastian Thrun

Carnegie Mellon University*University of Michigan

The Nursebot Project Joelle Pineau

Introducing Pearl – A mobile robotic assistant for elderly people and nurses

cameras

sonars

handle bars

mobile base

carrying tray

LCD mouth

touchscreen

microphone& speakers

laser

ROLE:

Moving thingsaroundMoving thingsaround

Management support of ADLsManagement support of ADLs

Providing physical assistance

Providing physical assistance

Remotehealth servicesRemotehealth services

Supportingcommunication

Supportingcommunication

Calling for helpin emergenciesCalling for helpin emergencies

Monitoring Rxadherence & safety

Monitoring Rxadherence & safety

Providing info(TV, weather)Providing info(TV, weather)

Reminding to eat,drink, take meds

Reminding to eat,drink, take meds

Linking caregiverand resources

Linking caregiverand resources


The Nursebot project in its early days


Architecture

Cognitive supportNavigation Communication

High-level controller


• Localization and map building(Burgard et al., 1999)

• People detection and tracking(Montemerlo et al., 2002)

Architecture




• Autominder system (Pollack et al., 2002)

Architecture




• Speech recognition: Sphinx system(Ravishankar, 1996)

• Speech synthesis: Festival system(Black et al., 1999)

Architecture




The role of the top-level controller

• Établir les priorités parmi les objectifs des différents modules

• Négocier entre plusieurs objectifs ayant des coûts/gains variés

• Négocier entre l’acquisition d’information et la rencontre des objectifs

• Passer d’une tâche à l’autre en partageant l’information sensorielle

• Planifier malgré la présence d’incertitude


ACTION SELECTION - based on the trade-off between:

- goals from different modules;

- goals with varying costs / rewards;

- reducing uncertainty versus accomplishing goals.



Speech recognition with Sphinx


Robot control under uncertainty

Belief StateP(st=weather-today)=0.5

P(st=appointment-today )=0.5

USER

Action={ say-weather,update-appointment,clarify-query}

Speech=“today”

Stateweather-today


Robot control using Partially Observable MarkovPartially Observable Markov Decision Processes Decision Processes (POMDPs)

Belief state

USER + ENVIRONMENT + WORLD

Actions

ObservationsCosts / Rewards

State

Problem: Which action allows the robot to maximize its reward?

P(s1)P(s2)


Methods to solve POMDPs

Objective: Find a policy, (b), which maximizes reward.

Complexity

Performance

QMDP

MDP

FIB

UMDP

AMDP

O(S2A) O(S2AT) O(S2AO )O(S2AO) O(S2AB) T

POMDP

New methods?


New approach: A hierarchy of POMDPs

Idea: Exploit domain knowledge to divide one POMDP into many smaller ones.

Motivation: Complexity of POMDP solving grows exponentially with # of actions.

Assumption: We are given POMDP M = {S,A,,b,T,O,R} and hierarchy H

Act

ExamineHealth Navigate

MoveVerifyPulse

ClarifyGoal

North South East West

VerifyMeds

subtask

abstract action

primitive action


PolCA: Planning with a hierarchy of POMDPs

Step 1: Select the action setNavigate

Move ClarifyGoal

South East WestNorth

AMove = {N,S,E,W}

ACTIONSNorthSouthEastWest

ClarifyGoalVerifyPulseVerifyMeds





Step 1: Select the action set

Step 2: Minimize the state set

STATE FEATURESX-positionY-position

X-goalY-goal

HealthStatus


X-goalY-goal

HealthStatus

Navigate

Move ClarifyGoal


AMove = {N,S,E,W}

SMove = {X,Y}









Step 3: Choose parameters


X-goalY-goal

HealthStatus


X-goalY-goal

HealthStatus

Navigate

Move ClarifyGoal


AMove = {N,S,E,W}

SMove = {X,Y}





PARAMETERS

{bh,Th,Oh,Rh}

PARAMETERS

{bh,Th,Oh,Rh}





Step 3: Choose parameters

Step 4: Plan task h


X-goalY-goal

HealthStatus


X-goalY-goal

HealthStatus

Navigate

Move ClarifyGoal


AMove = {N,S,E,W}

SMove = {X,Y}





PLAN

h

PLAN

h

PARAMETERS

{bh,Th,Oh,Rh}

PARAMETERS

{bh,Th,Oh,Rh}


PolCA in the Nursebot domain

• Goal: A robot is deployed in a nursing home, where it provides reminders to elderly users and accompanies them to appointments.

• Domain: |S|=512, |A|=20, |O|=19 • Hierarchy:


Sample scenario


Results for dialogue system

0.1 0.1

0.18

POMDP policy MDP policy


Summary

• We have developed a first prototype robot able to serve as a mobile nursing assistant for elderly people.

• The top-level controller uses a hierarchical variant of POMDPs to select actions.

• This allows it to acquire necessary information and successfully complete assigned tasks.

• Probabilistic techniques have been found to be very useful to flexibly model and track individuals.


For more details: www.cs.cmu.edu/~nursebot

The Nursebot team

CMU - Robotics:Greg Armstrong

Michael MontemerloJoelle PineauNicholas RoyJamie Schulte

Sebastian Thrun

CMU - HCI/Design:Francine Gemperle

Jennifer GoetzSarah KieslerAaron Powers

U. of Pittsburgh - Nursing:Jacqueline Dunbar-Jacobs

Sandra EngbergJudith Matthews

U. of Pittsburgh - CS:Don Chiarulli

Colleen McCarthy

U. of Freiburg - CS:Maren BennewitzWolfram Burgard

Dirk Schulz

U. of Michigan - CS:Laura BrownDirk ColbryCheryl OroszBart PeintnerMartha PollackSailesh Ramakrishnan

Standard Robotics:Greg Baltus

joelle pineau michael montemerlo martha pollack * nicholas roy sebastian thrun

Documents

hierarchy of pomdpsstep

spe pineaurobot control

accomplishing goals

hierarchy of pomdps

complexity of pomdp

action setstep

mobile robotic assistant

different modules goals