A Two-Layer Model for Behavior and Dialogue

Planning in Conversational Service Robots Mikio Nakano, Naoyuki Kanda, Hiroshi G. Okuno

Proc. IEEE/RSJ Int’l Conf. on Intelligent Robots and Systems, 2005

인지과학협동과정 석사 1학기 윤성재

Overview

• Motivation

• Related works

• Proposed model – Multi-Expert-based Behavior and Dialogue Planning, MEBDP

• Experiment & result

Motivation

• Building a intelligent Conversational robots – Understand human requests by spoken dialogue

– Takes as input human speech and gesture recognition obtained by interpreting sensor output

• Give information as well as perform desired behaviors – Set the goal that satisfies the requests.

– The goal may be achieved by combinations of physical behaviors and spoken dialogue

• Focusing features

– Task-oriented dialogue

– Mixed initiative dialogue

– Interruption handling

Related works

• Kanda et al.’s – Situated modules organized in a network

– Perform more than one hundred behaviors including making short utterances

• Topp et al.’s – State-transition model for the highest-level control module

– Substate for clarification questions

• Asoh et al.’s Jijo-2 – Switch tasks and stop navigation based on human utterances

– Dialogue planning is not integrated with global task planning

• Still, it is unclear how to integrate robot architecture with spoken dialogue system architecture.

Proposed model

• Upper layer: Deliberative layer – Performs global task planning by decomposing requested tasks into

a sequence of subtasks using plan libraries

– Task is what humans request the robot to perform

• The lower layer: Reactive layer – Performs subtasks using experts specific to the types of subtasks

– Expert decides what action to perform next until the subtask is achieved or canceled

MEBDP: A two layer model

Proposed model

Modules in MEBDP

The upper layer

• Three key processes in upper layer 1. Understanding process

2. Action selection process

3. Execution report handling process

– Understanding process

• Monitors input from multi-modal information

• Decides which expert should be in charge next, based on the results of the understanding

• Interruption monitoring from selected experts

– If interruption happens, it sends signal to action-selection process

Proposed model

The upper layer (cont.)

– Action selection process

1. Waits for new input to be understood

2. Sends the understood abstract action to the action realizer

– Action realizer transforms the abstract action into a multi-modal expression, and outputs it.

3. Waits for the success/failure report from the execution report handling process

– If Receives an interruption, Calls the handle-interruption method of the expert in charge, and performs this procedure 3.

4. When all subtask finished, a new task is decomposed by the task planner into subtask sequences, and the expert for the first subtask becomes in charge.

5. If there is no task to be performed, it waits for a new human utterance.

– Execution report handling process

• Receives the reports of the success/failure of action execution from the behavior controller and tells them to the action selection process

Proposed model

Proposed model

Modules in MEBDP

Proposed model

Experts in lower layer

Experts in lower layer

• Each expert has the following four main methods – Understand method

• Updates the information based on the recognition results

– Select-next-action method

• Outputs one abstract action based on the content of the information state

– Detect-interruption method

• Determines if the human utterance is an interruption to the action being performed when this expert is being in charge

– Handle-interruption method

• Returns the action to be performed after an interruption is detected.

Proposed model

Proposed model

• Request understanding experts (RU)

• Information providing experts (IP) – Accesses an external database(e.g. timetable) to retrieve

information

– Decomposes the information to be provided into small pieces

• Physical action planning experts (PAP) – Plan a physical action sequence

• Information obtaining dialogue experts (IOD) – Plan dialogues to obtain information necessary for performing

a task.

Types of experts

Experiments & result

• Honda humanoid robot ASIMO

• Julian, Speech recognition system

• FineVoice, Text-to-speech system

Experiments & result

Conclusion

• Flexible inter expert working is available through Layered structure

• Makes it easy to build and maintain the planner – Each expert is designed separately and different planning algorithms

can be used

• Task-oriented, mixed-initiative dialogues are possible – MEBDP employs experts that perform dialogue management for such

kind of dialogues