interactions in planning systems -...

40
Interactions in Planning Systems Barbara ZÖLLER Supervisor: Humbert FIORINO & Cyrille MARTIN Laboratoire LIG-MAGMA ENSIMAG - Introduction à la Recherche en Laboratoire May 16, 2012

Upload: phambao

Post on 12-Apr-2018

216 views

Category:

Documents


1 download

TRANSCRIPT

Interactions in Planning Systems

Barbara ZÖLLER

Supervisor: Humbert FIORINO & Cyrille MARTINLaboratoire LIG-MAGMA

ENSIMAG - Introduction à la Recherche en Laboratoire

May 16, 2012

2 Interactions in Planning Systems – Barbara ZÖLLER

Contents

Contents 3

List of Figures 5

List of Tables 6

1 Planning in the context of Artificial Intelligence 7

2 Automated planning 8

2.1 Aim of automated planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Blocks World as a planning example . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.3 Approaches for automated planning . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4 Challenges in automated planning . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Interaction in automated planning 12

3.1 Motivation for human interaction in automated planning systems . . . . . . . . . 12

3.2 Criterions to analyse the presented systems . . . . . . . . . . . . . . . . . . . . . . 13

3.3 Existing methods for human interaction in planning systems . . . . . . . . . . . . 14

3.3.1 Supporting combined human and machine planning: an interface forplanning by analogical reasoning . . . . . . . . . . . . . . . . . . . . . . . . 14

3.3.2 TRAINS-95: Towards a mixed-initiative planning assistant . . . . . . . . . 15

3.3.3 Planning as mixed-initiative goal manipulation . . . . . . . . . . . . . . . 16

3.3.4 PASSAT: A user-centric planning framework . . . . . . . . . . . . . . . . . 17

3.3.5 Bringing users and planning technology together. Experiences in SIADEX 19

3.3.6 Author in the loop: Using mixed-initiative planning to improve interac-tive narrative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.3.7 MAPGEN Planner: Mixed-initiative activity planning for the Mars Ex-ploration Rover mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Interactions in Planning Systems – Barbara ZÖLLER 3

3.3.8 COMIREM: An intelligent form for resource management . . . . . . . . . 22

3.4 Limits of existing solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4 Planning in Human-Robot-Interaction 27

4.1 Idea of Planning in HRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.2 Criterions to analyse planning in HRI . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.3 Approaches for Planning in HRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.3.1 Toward human-aware robot task planning . . . . . . . . . . . . . . . . . . 28

4.3.2 An integrated planning and learning framework for human-robot inter-action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.3.3 Planning and plan execution for Human-Robot Cooperative Taskachievement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.3.4 Planning for Human-Robot Teaming (HRT) . . . . . . . . . . . . . . . . . . 32

4.4 Limits of existing approaches for planning in HRI . . . . . . . . . . . . . . . . . . 34

5 Interactive Planning in HRI 37

Bibliography 39

4 Interactions in Planning Systems – Barbara ZÖLLER

List of Figures

2.1 Example of blocks world [2, p. 4] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 Blocks world: action unstack [2, p. 6] . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.3 Blocks world: action stack [2, p. 6] . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.4 Blocks world: action pickup [2, p. 5] . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.5 Blocks world: action putdown [2, p. 5] . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.6 Blocks world: planning problem [1, p. 5] . . . . . . . . . . . . . . . . . . . . . . . . 11

Interactions in Planning Systems – Barbara ZÖLLER 5

List of Tables

3.1 Mixed-initiative planning: quality of communication (part I) . . . . . . . . . . . . 25

3.2 Mixed-initiative planning: quality of communication (part II) . . . . . . . . . . . 26

4.1 Planning in HRI (part I) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.2 Planning in HRI (part II) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

6 Interactions in Planning Systems – Barbara ZÖLLER

Chapter 1

Planning in the context of ArtificialIntelligence

Planning is part of our daily life. We have plans for different kind of tasks. For example if wewant to buy different things in town, we normally (in the best case) create a plan, as a sequenceof the shopping tour. Here, it is also probably, that we have to respect different constraints likeopening hours of the shops, carrying capacity of the person who is doing the shopping tour orspatial distances. I estimate, we also could achieve without creating a special plan, but prob-ably we will save time. Therefore we need the correspondent information, as it does not helpso much, if we want to build a plan for a certain day and we do not have information aboutthe opening hours. Then we would probably nevertheless achieve our tasks, but the efforts forcreating the plan are lost, as e.g. we arrive at a shop during its lunch break and have to waitmore than an hour and it does not make sense to go to another one as it is too far. So you cansee, even if you have a good planning method, it does not help you, if you do not have thehelpful information.

In the research of Artificial Intelligence (AI) automated planning is also of interest. Thegeneral aim of AI is to design intelligent systems with intelligent agents who have the capabil-ities to percept the environment and to optimize their success at the same time. Planning is arational behaviour which is indispensable especially in situations where constraints limit theactions. For instance, if the task is to build a bridge with a limited quantity of building mate-rials, it is very useful to build up a plan before resources runs out. Later, the idea of humaninteraction aroused as it pointed out that the results of planning depend highly from the pro-vided information and to fulfil humans’ personal preferences. So the result can be optimizedwhen humans add additional information or select between different provided suggestions.Furthermore, automated planning is applied to autonomous robotic systems. In this domain,new challenges arise as the environment is very complex, dynamic and non-deterministic. Fi-nally, the question arises if it is useful to combine automated planning with human interactionin the field of autonomous robotics, as it is nowadays more and more common to develop anduse robots which results in a human-robot society.

In the survey, I first introduce to automated planning (see chapter 2), then I present systemswith human interaction in automated planning (see chapter 3) and approaches for planning inthe domain of Human Robot Interaction (HRI). Finally the question “Does HRI benefits frommixed-initiative planning?” is discussed in chapter 5.

Interactions in Planning Systems – Barbara ZÖLLER 7

Chapter 2

Automated planning

Automated planning is a subfield of Artificial Intelligence that creates action plans to reach acertain goal. Therefore many decisions have to be made and constraints have to be taken intoaccount. In this chapter, we peopoe a short overview of automated planning (section 2.1), anda concrete example (section 2.2). Then we introduce th different approaches (section 2.3).

2.1 Aim of automated planning

Automated planning is based on two principles. First, it is about achieving a goal by executinga sequence of actions automatically to have an autonomous system (e.g. robots). Second, itis important for a rational behavior to do the planning in an automated process. Techniquesfor knowledge representation are needed for the planning process and an adequate model forprocess are state-transition systems. It is defined by a quadruple Σ = (S, A, E,γ) [13]: 1

• S = {s1, s2, . . .}: set of states

• A = {a1, a2, . . .}: set of actions (controlled by the executor1)

• E = {e1, e1, . . .}: set of Events (not controlled by the executor)

• γ : S× A× E→ 2S: state-transition functions

In a simple planning task, the specification is given by

• Si ⊆ S: set of initial states

• Sg ⊆ S: set of goal states

2.2 Blocks World as a planning example

To give an idea about the planning process, a typical example for planning processes - blocksworld. It is a simple problem, but still enough complex to present a planning process. So it is

1 An executor is an entity which controlls the action for the execution of a plan.

8 Interactions in Planning Systems – Barbara ZÖLLER

easy to understand without a deep knowledge of a special domain.You can imagine the blocks world as a small world consisting of a surface (e.g. a table),

where you can find some blocks. There is a robot arm, which can take one block at a time, canmove it and then he can stack the blocks on each other, but only one on one (see figure 2.1. Todescribe the state, we have primitives (blocks) and predicates with their interpretations:

Figure 2.1: Example of blocks world [2, p. 4]

• Blocks {A, B,C}

• Predicates:

on(A,B) block A is on block B

onTable(A) block A is on the table

holding(A) the arm holds block A

clear(A) block A has nothing on it

armempty the arm holds nothing

Further, we need to define the possible actions A [2, p.5,6]. They consist of an operation, pre-conditions (which have to be satisfied to realise the action) and effects (as consequence of theaction)

Figure 2.2: Blocks world: action unstack [2, p. 6]

unstack(C,A) pick up clear block C from block A (see figure 2.2)preconditions: clear(C) ∧ onTable(C,A) ∧ armemptyeffects: ¬ clear(C) ∧ ¬ on(C,A) ∧ ¬ armempty ∧ holding(C) ∧ clear(A)

stack(C,B) place block C using the arm onto clear block B (see figure 2.3)preconditions: holding(C) ∧ clear(B)effects: ¬ holding(C) ∧ ¬ clear(B) ∧ clear(C) ∧ on(C,B) ∧ armempty

Interactions in Planning Systems – Barbara ZÖLLER 9

Figure 2.3: Blocks world: action stack [2, p. 6]

Figure 2.4: Blocks world: action pickup [2, p. 5]

pickup(C) lift clear block C with the empty arm (see figure 2.4)preconditions: armempty ∧ clear(C) ∧ onTable(C)effects: ¬ armempty ∧ ¬ clear(C) ∧ ¬ onTable(C) ∧ holding(C)

Figure 2.5: Blocks world: action putdown [2, p. 5]

putdown(C) place the held block C onto a free space on the table (see figure 2.5)preconditions: holding(C)effects: ¬ holding(C) ∧ clear(C) ∧ onTable(C) ∧ armempty

Finally we have all the necessary descriptions for the blocks world. Now we can express aplanning problem by specifying an initial state Si and a goal state Sg, which has to be solvedby an automated planning process.

• Si = clear(B) ∧ onTable(B) ∧ clear(C) ∧ onTable(A) ∧ on(C,A) ∧ armempty

• Sg = clear(A) ∧ on(A,B) ∧ on(B,C) ∧ onTable(C) ∧ armempty

10 Interactions in Planning Systems – Barbara ZÖLLER

Figure 2.6: Blocks world: planning problem [1, p. 5]

2.3 Approaches for automated planning

There exist different basic approaches to find possible solutions for a planning problem. Thetwo main differences are:

State space search Starting from the initial state, the system looks for actions that can be ex-ecuted to reach another state. Based on these states, it continues its search. Finally, youobtain a graph with possible action sequences. To find the goal state you have to check ifthe conditions of the goal state are satisfied (required goal states are reached).

Plan space search This means the search through possible plans. Some partial plans existwhich have to be combined to achieve the goal state. Besides, constraints concerningthe order have to be respected.

Hierarchical Task Network (HTN) The dependency of actions is given in form of networkswhich are ordered in an hierarchy manner. For example it could be composed of primitivetasks (can be executed independantly of other actions), compound tasks (composition ofprimitive tasks with dependencies) and goal tasks (fulfill the conditions of goal state).

2.4 Challenges in automated planning

Automated planning is a challenging problem, as the quality of the created plan depends onthe provided information about the environment. Means, that if the model of the planningenvironment is a very simplified representation of the real world problem, it does not coverall (or in large part) occurring events which could affect the result of the executed plannedplan. For instance in the blockworlds example the planning will be more difficult if somethingor somebody manipulates the world during the execution of the plan (by removing a block,disturbing the order of the blocks). Then the execution of the plan, which was created beforewith another initial state, would not lead to the desired goal state and the planning processfails. As it is difficult to model all the possible changing circumstances and uncertainties, theconcept of integrating human interaction in automated planning came up (see chapter 3).Additionally, spontaneous changes of the desired goal could arise during the planning processor the expectations concerning the plan depend of some additional reasons like personalpreferences of a person in the planning environment.

Interactions in Planning Systems – Barbara ZÖLLER 11

Chapter 3

Interaction in automated planning

As described in section 2.4, the capabilities of Automated Planning are constricted. Also oftenhumans are not sure if they can rely on the provided plans from an automated system. Thusthe idea of combining automated planning with human interaction came up. The followingsections first refer to the motivation of human interaction in automated planning, then presentexisting solutions and finally describe occurring problems and difficulties within the followingmethods.

3.1 Motivation for human interaction in automated planning sys-tems

The objective of combining automated planning with human interaction is to take advantageof the strength of machines and humans in planning scenarios to build plans in a quicker andmore efficient way and with greater reliability (see [7]). Especially in critical domains, reliabil-ity is important and humans are often skeptic, when they have to believe in a machine withoutbeing able to follow the decision making process. Humans have their strength in selectingrelevant information, in handling better with uncertainty and in visual or spatial reasoning.In contrast, machines are beneficial, when it is necessary to manage large amounts of data,probably with an additional large number of interacting constraints. Besides, plans build bymachines depend always of the quality of the provided environment model or rather its capa-bility to “observe” the planning environment. When the machine has only access to incompleteinformation, we can hardly expect a perfect plan. This is even more difficult for a dynamic en-vironment.

But human interaction in automated planning also causes some new difficulties, as the twoentities have to collaborate, which was not necessary in automated planning (see section 3.4).The following aims and challenges arouse:

• A certain level of communication possibility necessary to exchange information – pre-ferred multi-modal, so the user can choose the most natural.

• Human users have a different background and with it a different level of experience andknowledge with such applications. Depending on their knowledge, they prefer a differ-

12 Interactions in Planning Systems – Barbara ZÖLLER

ent level of abstraction and they are interested in different kind of information / explana-tion.

• Visualization of the plan and planning process is difficult. Here can exist complex rela-tionships between different actions of the plan and constraints which have to be respectedin planning.

• Humans think about reaching their planning goal in a different way than machines areused to. While machines search in the space of possible actions or in a database withformer plans, humans refer to their own past experience.

• The system motivates the users to interact during the planning process.

• A flexible planning process that adapts quickly, if the user changes his mind.

• A robust system which does not crash, when unexpected things happen.

• To have an appropriate model of the application environment.

3.2 Criterions to analyse the presented systems

To analyse the different existing approaches that are presented in section 3.3, I use the followingcriterions:

• different user modes: Are there different user modes level, like experienced or noviceuser?

• level of user interaction: In which scale can the user interact?

• available information: Which information is available for the user?

• visualization of data: How is the data visualized for the user?

• abstraction level: Can the user choose between different abstraction levels for the repre-sentation?

• ways of communication: Which possibilities have the user to communicate with the sys-tem ?

• power of the user: Which (manipulating) possibilities do the user has in general whenhe interacts with the system?

• plan monitoring during execution: Can the user follow the plan during its execution?

• motivation for user interaction: Does the system forces the user to interact?

As you can recognize the main focus is based on communication as this plays a central rolein human computer inteaction and is very important to have an appropriate way to exchangedata.

Interactions in Planning Systems – Barbara ZÖLLER 13

3.3 Existing methods for human interaction in planning systems

3.3.1 Supporting combined human and machine planning: an interface for plan-ning by analogical reasoning

As mentioned before (see section3.1) machines and humans collaborates in mixed-initiativeenvironments, so a proper way to communicate is necessary. Therefore Cox and Veloso de-veloped an interface for the PRODIGY planning system [8], which tries to handle some of thealready mentioned problems (see section 3.1).

The user can choose their level of interaction by switching between generative (state spaceplanning) and case-based planning with plan space search (see section 2.3) or even automation,which involves an alteration of the mode-specific buttons in the user interface window. Alsohe can define the level of information display and user-control according to his interest andexperience. All the created plans are stored and are accessible later. Additionally, the justifi-cations of the decisions are stored, in order that they are accessible for case-based planning1,which uses old plans and adapts them to new problems, by changing, skipping or merging ofactions (can be observed in the window below the main window). Besides, there is a graphicalrepresentation of the plan as a goal-tree structure and a head plan as a list of the committedplanning steps. By clicking on the nodes of the tree, you can access the appropriate informa-tion: operator definitions (in the main window), justifications2 for the choice of that operator orgoal (also in case of fail) and property lists of a node. Also he can select the parent and childrennodes.

But to start the planning process, you have to load the specified data (planning domain,problem to solve, past cases). The program creates a search tree and a brief statement withrunning time, the number of search nodes expanded, and number of solutions obtained. Theuser has many different possibilities using the interface:

• control the parameters

• execute the planner

• load the data for the problem

• during execution: step incrementally, break, continue, halt

If the user chooses manual retrieval, another window will be displayed, where the user canpick the previous cases, which are supposed to be the most similar and will be considered touse the current problem. For this purpose he can select the cases by click or with a buttonbefore loading them with another button to the system. The system identifies itself thevariables and substitute them correspondingly.

But it is still necessary to adjust the system for a more naive user. Therefore a contextsensitive help system is available, buttons are used instead of text commands and the plan isgraphically visualised. At least, they developed a novice mode, to provide the output to theuser in a more natural form (which means in natural language).

1 The system retrieves plans from a case library that are most similar to a given new problem2 The reasons for why certain actions are chosen.

14 Interactions in Planning Systems – Barbara ZÖLLER

3.3.2 TRAINS-95: Towards a mixed-initiative planning assistant

G. Ferguson et. al. present TRAINS-95 a mixed-initiative planning system for logistic tasks [10].Planners are advantageous, if it is necessary to handle a large amount of complex data. TheTRAINS-95 system provides human-machine communication both spoken and typed English.The aim of the developers was to create a system with the following properties:

• robust: No matter what happens, the system does not fail. This is especially importantfor the speech recognition which is used for the communication.

• modular: The system can utilize several independent modules like knowledge sources,reasoning agents and display engines, which communicate by exchanging messages.

• multi-modal: Several ways of communication exist to permit a choice to the user. Typed,spoken and mouse inputs are available, and graphical elements, speech and text displayto support the output. Speech recognition is crucial concerning its robustness.

• mixed-initiative: Humans and machines are emancipated and each of them can do whathe can do best

To support a natural way of communication, this system provides speech recognition. This isa crucial module of the system as humans often do not use complete sentences during plan-ning processes. For this purpose, the verbal reasoner determines first possible expressions byidentifying some key words and then refined later by querying various knowledge sources. Ifit contains a constraint, it is propagated to the problem solver, which is solving the problem byusing the provided information. On the other hand, the machine has to maintain the communi-cation with the human and has to “talk” to him to get more detailed information if necessary.Therefore it has to decide, what to “ask” first and remind which information were alreadyexchanged.

Concerning the planning process, the system has to force the communication, too. So theplanner is constrained that it can only provide plans for the next four steps. Because of the in-teraction, it also has to deal with plan recognition, when the user adds a new constraint duringthe planning process to identify the resulting necessary modifications (possibly small).

The difficulty of such a mixed-initiative planning problem is the dynamic environment.This means that there are many changes and uncertainties during the execution and planningprocess, so that it is almost impossible to specify it completely. The TRAINS-95 model consistsof four steps:

1. Focus: Identify the (sub)goal under consideration

2. Gather constraints: Background information, preferences, resources, constraints

3. Instantiate solution: generate an efficient solution as soon as possible

4. Criticize, correct or accept: If the solution is accepted, we can continue with a new prob-lem and go to step 1. Otherwise go to step 2 and adapt the constraints.

The difference of this approach to previous approaches is the early release of a plan, becauseit is easier to continue the planning process, if you have a first idea for discussions than talkabout abstract things. This helps also the human to recognize some points (constraints), he had

Interactions in Planning Systems – Barbara ZÖLLER 15

forgotten to mention for the planning process.A four-layer architecture is used to realize the proposed model:

• Discourse level: It maintains all necessary and relevant information for communication(identification and interpretation of speech).

• Problem solving level: Provides strategies to solve the problem, like decompose goalinto subgoals, resolve conflicts, solve goals. Besides, it supports discussion of the problemsolving strategy.

• Abstract plan representation level: It provides plan representations for the differentmodules of the system and also performs plan recognition.

• Domain reasoners: They manage the data exchange between the mixed-initiative plan-ner and various domain specialists to gain additional relevant information.

3.3.3 Planning as mixed-initiative goal manipulation

Cox (see section 3.3.1) devised an approach and had the idea to treat planning as a goal manip-ulation process [9]. This seems to be more user adapted, as the users are able to express theirintentions and to follow better the planning process. But goals can change for different rea-sons – circumstances changed or the user changed his opinion for instance. So it is necessaryto provide this option by so-called goal transformation. The goal changes its position along aset of dimensions defined by some abstraction hyperspace3 The instances are categorized by astandard type-hierarchy and the predicates are represented in a predicate abstraction hierarchyto assign goals to a certain level of specificity. Summarised planning can be seen as a task ofdiscovering, managing and refining the aims of somebody.

GTrans, an adjusted interface, was implemented. It hides unimportant information -planning algorithms and knowledge structures - from the user and prioritises the goal-manipulation. It is built up of a map (as supposed to be used for logistical planning problems)and drag and drop capabilities. Further, the system provides a multi-user cooperation. Theregular action sequence consists of:

1. Create planning problem

2. Generate a plan with the planner

3. If not satisfied,

• modify the goal or other aspects

• or request another plan

Those possibilities exist for the goal transformation:

• Goal type transformation: move the predicate of the goal along an abstraction hierarchy

3 This hyperspace is associated with two hierarchies. A standard conceptual type-hierarchy in classical planningformalisms within which instances are categorized. They are used to assign constraints to operator variables andto define goal predicates. The second hierarchy consists of predicates for operator definitions. They have to be in aseparate unique hierarchy.

16 Interactions in Planning Systems – Barbara ZÖLLER

• Goal argument transformation: upward movement through an abstraction hierarchy

• Valence transformation: toggling between a positive or negative truth value

The user can manipulate the planning world (goals, constraints, initial state) before and dur-ing the planning process and assist the program by solving the problem. The planning com-ponent PRODIGY is a state-space planner and combines actions to achieve the goal state. Itprovides a user-interface with additional information (as described in section 3.3.1). GTransand PRODIGY have to communicate during the planning process to gain all changes of thedynamic environment (object, state and goal information), so that GTrans obtains the relevantdomain information. Therefore different kind of requests exists:

• obj-request: to get the objects that exist in the domain

• goal-request: responds by sending all possible goals that can be achieved

• tree-goal-request: to get information about the goal hierarchy

• state-request: to ask for all the possible initial states

The authors made some user experiments. The task was to find a solution for the bridge prob-lem. Here, the user has to build a bridge under resource constraints. The observed parametersare the satisfaction ratio depending on users’ expertise level (novice vs. expert), problem com-pelxity (easy, medium and hard) and planning model (goal manipulation vs. search basedmodel ) User experiments attest that the goal manipulation model achieves a better goal satis-faction ratio than the search model for problems of each complexity (especially for hard ones)and for both user levels (but a bit more for novice users than for experts).

3.3.4 PASSAT: A user-centric planning framework

Myers et al. present PASSAT (Plan Authoring System based on Sketches, Advice and Tem-plates), that provides interactive tools for constructing plans and also automated and mixed-initiative capabilities to assist the human user during the planning process [15].The two key principles of PASSAT are:

• flexible, out of the box planning: The user has more possibilities than in traditionalsystems. He can add own solutions to the template, override constraints or drop tasks.

• controllable user-centric automation: Automation is only used, if it is desired by theuser.

Although PASSAT is domain-independent, it fits especially to applications with have a fullcapture of all relevant planning and where a certain level of automation would reduce planningtime and improve plan quality.

Further, user input is necessary to obtain high-quality situation specific plans for partiallyformalizable domains (this are domains for which you can define a model which includes allthe necessary information to create acceptables plans. But there are also parts of the real-worlswhich are not formalizable as they are to complex e.g. strategy decisions). The user interfacedisplays a hierarchical decomposition of the current partial plan (with items that have been

Interactions in Planning Systems – Barbara ZÖLLER 17

expanded, those which can be expanded further and other, which does not match templates).Besides, for the planning relevant information are listed in the information requirements andthe list of outstanding planning steps is shown as an agenda. Based on this humans can createthe plans.The following terms are available for the plan representation:

• Template: Describes how to decompose a task into subtasks with constraints, effects andused variables

• Constraints: Conditions which have to be respected during the planning

• Temporal Representation: Constraints for the scheduling of the tasks

• Domain Definition: Ontology4 for the hierarchical representation of the domain. Addi-tionally, functions, state predicates and task statements are declared.

Two main modes are provided by PASSAT for the user-centric plan development:

• Interactive plan refinement:

– expand task: The user can choose between applying a predefined template, specifyinga set o subtasks, sketching a solution or dropping the task. All the resulting effectsto other information (constraints, variables, agenda) will be applied and the data ismodified.

– instantiate variable: PASSAT proposes to the user possible instantiations for variables.

– resolve constraint: The system checks the constraints automatically and assign

• Plan sketching: As the authors suppose that humans prefer generally a bottom-up plan-ning approach, the user can create a sketch of his plan. These sketches can be partiallydefined. The program is able to process them and recognizes problems. So the user can“repair” it by dropping a (soft) constraint or modifying he task.

PASSAT allows to save a plan to have the opportunity to interrupt the planning process withoutdata loss. Moreover Myers et al. plan to integrate an advice system to suggest recommenda-tions to the user. The process facilitation permits the user to access all important informationfor the planning process. This is supported by:

• agenda: List of steps to do to solve problems. But it can become very large, so prioritiza-tion was necessary. Therefore the following approaches exist in PASSAT:

– Predefined: Fixed assigned priority for each subtask, constraint, variable

– Commitment-based: Priority corresponds to the degree of constraining the planningprocess

4An ontology is used to describe a set of concepts within a domain. It consists of entities and describes relation-ships between these entities. Further it has a hierarchical representation. It is used in many applications where it isnecessary to organize information and to reason about it, like Artificial Intelligence, semantic web.

18 Interactions in Planning Systems – Barbara ZÖLLER

– Experience-based: Based on previous plans, an algorithm learns a preference functionabout the order of execution

• information requirements: To direct the user to relevant plan elements and identify keyinformation. They can be used to determine the applicability of the template, for resolv-ing template’s constraints and variable instantiations.

With this approach they achieved to improve the previous system, particularly concerningeffectiveness, efficiency and interactivity, but still they plan to increase the flexibility and addan advisability module to facilitate automatic validation of high-level constraints.

3.3.5 Bringing users and planning technology together. Experiences in SIADEX

SIADEX is an intelligent planning and scheduling application developed to support decisionmaking on crisis intervention planning (current domain is forest fire fighting) [11]. In crisismanagement the decision making process is a cycle consisting of four steps:

Assess⇒ Plan⇒ Act⇒ Observe �

For this, some particular points have to be kept in mind. A short time for response is requiredand a large amount of fighting means. Plan revision by interaction should be possible as well asmonitoring during execution. A Hierarchical Task Network (HTN) is used to be able to realizeall the necessary tasks like temporal reasoning and certain fighting protocols. Further, a lowprocessing time is important to create several plans to have an opportunity to select the mostsuitable. Finally, an adapted and comprehensible representation for the end users is required.

The main components of SIADEX are the knowledge base (named BACAREX - with allthe useful information for the planning) and the planning module. The process supported bySIADEX contains these steps:

• Problem description: Defined by user-friendly interface. A scenario is composed of sev-eral sectors, which is again divided into different operational targets. Strategies can bedefined and available resources have to be determined.

• Knowledge integration: Scenario is stored in BACAREX and integrated in neededknowledge. An ontology server takes charge of this. The ontology also represents tem-poral knowledge, as time is a critical point in execution of the tasks in risk management.Concerning the interfaces, the ontology server provides different services:

– Possibility to query browse and update the information

– Maintains updated the situation of means and resources during the plan progress

– Ontology is used to obtain the problem and the domain. So it translates the problemdefined by the user to the ontology for the planner.

• Requesting a plan: After integration the planning engine is called. It uses the HTN andtemporal reasoning, which includes the possibility to set deadline constraints. Differentcontrol structures for sequencing, splitting and synchronizing are applied. During plan-ning the data has to be translated in an adequate representation for the end-user.

Interactions in Planning Systems – Barbara ZÖLLER 19

• Displaying the plan: The created plan is delivered in XML and presented to the users byan appropriate system e.g. Microsoft Excel and as web service with a user interface.

• Plan execution and monitoring: Created plan is launched and all relevant informationare displayed for the user. The interface offers real-time supervision of the plan executionand to some extend it is also possible to repair it.

3.3.6 Author in the loop: Using mixed-initiative planning to improve interactivenarrative

Another application domain for mixed-initiative planning is the creation of interactive narra-tives [19]. These are stories in virtual worlds, where humans interact with computer(s) – e.g.computer games, training simulators, virtual environments. The challenge of those systems isto provide “human” computer characters, which act in a natural way and are both believableand automated. Further it should be adapted for general domains as well as for specific prob-lems. The difficulty (which has still to be solved) is to get to know a good formalism, whichcharacterise a good narrative. The proposition is to integrate the user when defining heuris-tic functions to gather his preferences. Another challenge is the lack of predictability of thesequence of actions, as there are mostly many possibilities and you never know how it willfinally finish. Consequently, the narrative has to be very flexible as well.

Thomas and Young developed an approach which uses mixed-initiative and advisable5

planning. It is called Domain Elaboration Framework (DEF) and the user can add more de-tails to classical planning domain to improve the reasoning and define the problem in a moreexpressive manner. The planning domain is characterized by:

• objects: entity in the world with an unique name

• conditions: conjunction of function-free literals

• operators: set of literals in the preconditions, which have to be satisfied to invoke actions,which will change the state of the system

The grammar of DEF uses (types and measurements can be combined with every operator andcondition):

• types: symbolic name of a node in a global hierarchy of author-defined types with anunique root

• dimensions: symbolic name selected from a global list of unique author-defined dimen-sions

• weight: specifies a relative intensity of the dimension normalized on the interval [−1,1],which allows to make qualitative judgements about the plans to evaluate them.

• measurement: consists of a dimension and a weight

Finally a user interface is required to allow also to non-technical users to apply the system. It iscalled Bowmann and you can describe types, objects, operators, conditions and the initial and

5 The system gives advices to the user for preferred actions during the planning process

20 Interactions in Planning Systems – Barbara ZÖLLER

goal states. The plan space is presented as tree, where each node is a partial plan.To support narrative mediation to threat undesired user actions, it exists two types of

agents: User-controlled and system controlled. So the system can invoke the agent who iscapable to call an action (but before it has to be specified, which agent can invoke any particu-lar action). If the planner detects a user action that could threaten the story plan, eventually anintervention is required and a failure mode activated to substitute the intended action. Plan-ning “macros” (sets of related literals and operators) are provided to the plan authors to assistthem to achieve their goals with less work.

But all components of DEF are not still realized in the current version of Bowmann. So theycould not evaluate it. A stronger connection between the planning system and the interfacewould allow highlighting the plans, which will fit the best to the actual situation. Additionally,foreseeing of the users’ intensions would help the planner, as former plans could be applied insimilar situations.

3.3.7 MAPGEN Planner: Mixed-initiative activity planning for the Mars Explo-ration Rover mission

MAPGEN (Mixed-initiative Activity Plan GENeration system) is a tool for NASA’s Mars Ex-ploration Rover mission (MER) [3]. But it can be adapted for other domains, too. The aimof this tool is to assist users to build plans which have to fulfil constraints and are restrictedby resource limits. The system with an advanced constraint-based reasoning and planningframework is built up of two tools - one for activity plan editing and the other for resourcemodelling.

During the mission the rover is controlled from Earth and it is only working during daytime when the sun is up. Data is exchanged, so that the engineers can create plans accordingto it. In doing so, they pursue their needs, but they have also to respect different kind of re-strictions (e.g. resources, safety). The scientists approve the plans before they load it to therobot. Therefore, they always have time during the night on mars, when the robot does notwork. A user interface supports the building and evaluating of the plans. User and automatedreasoning operations are provided.

The automated reasoning is based on a system called EUROPA (this is an advancedconstraint-based planning system). It can manage complex planning constructs with temporalconstraints (expiring states, exogenous events, temporal durations, subgoaling rules). The do-main model consists of a set of predicates (with possible values for each of them) and it definesthe configuration constraints (temporal and parametric constraints, requirements for other ac-tivities). The planning process is based on partial plans, which may be incomplete and whereconditions are potentially not satisfied. They are modified until they fit for the current situa-tion. The used methods are backtracking, completing partial plans, propagation, consistencychecks, random exploration and others.

As user interface, they use APGEN, a plan editing system (well established in spacecraftoperations community). The user can edit and merge plans, while the system is checking ifthe existing rules are not ignored and it can be adapted to different missions. Through theautomated reasoning functionality, the activity plan generation process is upgraded:

• Constraints and rules are actively enforced: this means that the system is attentive thatthe constraints are respected and the tasks are done regarded to them.

Interactions in Planning Systems – Barbara ZÖLLER 21

• Variety of automated search techniques: you have the possibility to fix plans that violateconstraints and to complete partial plans.

• Reasoning and record-keeping can be used to provide informative explanations to theuser: like reasons for schedule choices or resource violations

But there are still open problems, like better control over the reasoning process, additionalexplanatory assistance for the user and add reasoning for the camera rotation (lifetime of thismechanism is limited).

3.3.8 COMIREM: An intelligent form for resource management

In several planning and scheduling technologies the output can only be modified by changingthe input. This is not very intuitive to users and not practical either. So Smith et al. developedComirem (Continuous mixed-initiative resource management) which is more adapted to theusers’ desires [16]. The main principles of the system:

• Decisions and planning are possible on different levels of detail

• Abstract domain model and graphical visualization as support for decision making

• Incremental problem-solving capabilities

Comirem is generated to be able to integrate input of other planning and schedule domainmodels and to be applicable for a broad field of resource management.

Thanks to an ontology, the user can specify the domain model consisting of :

• resources: They can be stationary or mobile, that means they move (those are more com-plex as they have more specifications).

• activities: You can distinguish between moves (from one location to another) and events(at one single location). They include resource allocation constraints: Capability require-ments, duration and manifest. To create a plan the user specifies a set of activities as-sociated with temporal constraints. Comirem can aggregate activities and split them insub-activities. Threads are used to define an environment for a sequence of activitieswhich use the same set of resources.

• constraints: They restrict the execution of activities and the assignment of resources.They are constrained by their location and the mobile resources also by their carryingcapacity. Temporal relationships occur between the activities (before and after, same-start and same-finish, over-laps and contains). Therefore, Comirem provides the user toassign a reference hour to activities, time window and anchoring constraints (to set lowerand upper bounds).

Now Comirem can be used to create plans in an interactive and semi-automated way. The usercan request resource feasibility check and auto-scheduling. The system has to execute certainactions to reach the goals by respecting the resource constraints (capacity, location). In thesuccessful created plan the resources are assigned to the relative activity. To refine and sourcea given plan, Comirem supports this by:

22 Interactions in Planning Systems – Barbara ZÖLLER

• Option generation: feasible allocation options are suggested for unassigned activities

• Visualization of decision impact: committed options are immediately visualized

• Requirements and capabilities editing: to change the constraints, the user can edit them.

• Automated assignment and feasibility: checks the resource feasibility for the plans

• What-if analysis: undo function is enabled to try some new options

Further, there exists a drag-and-drop system to configure resources with automatic control forthe constraints. A good system for the user has to be comprehensible, so that the user will trusthim and appreciate the plans. The system has to provide understandable explanations of thedecisions to the user. Therefore they use Comirem domain ontology knowledge to figure outeverything what might be interesting to the user (constraints, conflicts and possible resolutionactions).

3.4 Limits of existing solutions

As described in section 3.3 there exist already different approaches for mixed-initiative plan-ning systems, but they still have some lacks. Gabriella Cortellessa aim is to create a method toevaluate those systems to investigate them in a common way and to adapt them better to users’requirements. Her statement is to learn more about the requirements before developing moremixed-initiative planning systems and to provide an easier way to compare them.

The received results from user experiments:

• Non-expert users prefer the mixed-initiative approach. They tend to interact more ac-tively in the planning process. This can be justified as they are more sceptical concerningautomated systems.

• Explanations are more frequently used in case of failure

• Expert users have a higher trust in the automated solver

• Users access the explanations to understand the artificial solver

• Success has no influence on the frequency of demanding explanations

• Users who apply automated planning use more often the explanations

• Experts call the explanations more frequently

• Explanations are viewed more times for easy problems

So you can discern that the difficulty of developing a mixed-initiative system is founded by thedifferent types of users, the varying environment and the complexity of planning problems.The presented approaches in section 3.3 are designed partially for different environments likelogistics, resource management, scheduling, Mars Rover and Narrative in computer games.The reasons for integrate humans in the planning process or the other way round to use ma-chines to assist humans vary:

Interactions in Planning Systems – Barbara ZÖLLER 23

• Achieve better plans (higher quality) than humans or machines could be able to createalone (see sections 3.3.1 and 3.3.3). So you can use the benefits of both and deficits canbe compensated.

• Planners can be applied possibly in different environments, but it is difficult to adaptthem in equal measure. So humans input supports the system to gain a better solution inspecific domains (see sections 3.3.4 and 3.3.6).

• Humans want to get to know the inside of a planner to be able to comprehend the plan-ning process and to have a higher confidence in the plan (see section 3.3.4).

• Many dynamic processes with uncertainties exist in the real world. They cannot be mod-elled in an appropriate way or as well expert decisions (see sections 3.3.4 and 3.3.8).

• Heuristics can be used to define an adequate model. But sometimes they are not suffi-cient and hard to determine, especially with the occurrence of many uncertainties and incomplex environments (see section 3.3.6).

• Planning has to deal with resource limits and has to satisfy complex rules. This is difficultfor humans. So a mixed-initiative planner can be used as a support while maintaining thepossibility for the human to participate in the planning process (see section 3.3.7).

• It can be necessary to revise plans, if they do not longer satisfy the requirements due tochanges of the environment or of the requirements over time (see sections 3.3.8).

So you can recognize that mixed-initiative planning aids to compensate deficiencies of humansand machines general capabilities.

As mentioned already above, humans interact with the machines. Hence, a way of commu-nication to exchange data is indispensable. To compare the different approaches concerningtheir communication quality I analysed them (see tables 3.1and 3.2) referring to the criterionsdescribed in section 3.2. In most of the systems the user can choose the level of interaction andhe can see for the planning relevant data (pre-filtered by the system). Further it is common toprovide explanations for the decisions made during the planning. Unfortunately the informa-tion about the used visualization is not so detailed, so it is difficult to arrive at a conclusion forthis criterion. But for the plan representation a goal-tree is used (see sections 3.3.1 and 3.3.6)and “drag and drop” technique is used as well (see sections 3.3.3 and 3.3.8). Only one sys-tem provides oral communication as it includes a speech recognition system (see section 3.3.2).TRAINS95 also forces the user to interact by planning only four steps. Plan monitoring duringplan execution seems not to be common, as it is only used for one system, SIADEX (see sec-tion 3.3.2) which is applied for crisis management. Therefore the power of the user is similar.Besides editing the plan, he usually can modify constraints and the goal state. Further he candefine a strategy (SIADEX), approve the plan before execution (MAPGEN - see section 3.3.7)and interrupt the planning process (PASSAT - see section 3.3.4).

24 Interactions in Planning Systems – Barbara ZÖLLER

crit

erio

ndi

ffer

ent

user

mod

es(e

x-pe

rien

ce)

leve

lof

user

inte

ract

ion

(aut

omat

edvs

.in

tera

c-ti

ve)

avai

labl

ein

form

atio

n/

data

visu

aliz

atio

nof

data

(how

?)

Cox

-pl

an-

ning

inte

r-fa

ce

can

defin

ele

vel

ofin

for-

mat

ion

disp

lay

can

bech

osen

(sta

tesp

ace,

plan

spac

e,au

tom

atio

n)ju

stifi

cati

ons

ofde

cisi

ons,

choi

seof

oper

ator

(s);

save

spr

evio

usca

ses

for

plan

retr

ival

plan

asgo

al-t

ree

TRA

INS-

95n/

aan/

an/

aM

ap,G

UI

Cox

-go

alm

anip

ula-

tion

can

defin

ele

vel

ofin

for-

mat

ion

disp

lay

mul

ti-u

ser

inte

ract

ion

just

ifica

tion

sof

deci

sion

s,ch

oise

ofop

erat

or;

save

spr

evio

usca

ses

for

plan

re-

triv

al

GTr

ans

(=G

UI)

,map

wit

hdr

ap&

drop

PASS

AT

inte

ract

ive

plan

refin

e-m

ent

(top

-dow

n)or

plan

sket

chin

g(b

otto

m-u

p)

leve

lofa

utom

atio

nca

nbe

chos

enby

user

for

the

plan

ning

rele

vant

info

rmat

ion

are

liste

din

the

info

rmat

ion

requ

ire-

men

tsan

dth

elis

tof

out-

stan

ding

plan

ning

step

sis

show

nas

anag

enda

hier

arch

ical

deco

mpo

si-

tion

ofth

ecu

rren

tpa

rtia

lpl

an

SIA

DEX

n/a

n/a

envi

ronm

ent

ason

tolo

gy,

avai

labl

ere

sour

ces

disp

layi

ngof

plan

via

web

serv

ice,

Gan

ttch

art

inte

ract

ive

narr

ativ

en/

a“m

acro

s”(s

ets

ofre

late

dlit

eral

san

dop

erat

ors)

toas

sist

toac

hiev

ego

als

wit

hle

ssw

ork

n/a

GU

I(B

owm

ann)

;pl

ansp

ace

astr

ee,

whe

reea

chno

deis

apa

rtia

lpla

n

MA

PGEN

n/a

inte

ract

ive

and

auto

mat

edde

cisi

onex

plan

atio

nsA

PGEN

,a

plan

edit

ing

syst

emC

OM

IREM

n/a

inte

ract

ive

orse

mi-

auto

mat

edre

quir

emen

ts&

capa

bili-

ties

,m

odel

ason

tolo

gy,

deci

sion

axpl

anat

ions

yes,

drag

&dr

opto

edit

re-

sour

ces

Tabl

e3.

1:M

ixed

-ini

tiat

ive

plan

ning

:qua

lity

ofco

mm

unic

atio

n(p

artI

)

a nota

vaila

ble

Interactions in Planning Systems – Barbara ZÖLLER 25

criterionabstractionlevel

ways

ofcom

munication

power

ofthe

userplan

mon-

itoringduringexecution

forcefor

userinterac-

tion

Cox

-plan-

ninginter-

face

n/agraphical

user-interface(G

UI),Buttons

contributeto

plancreation

n/an/a

TRA

INS-95

n/atyped

&spoken

English,G

UI

contributionto

plancreation,

addingconstraints

n/aplans

onlyfor

4steps,

earlyplan

releaseC

ox-

goalm

anipula-tion

n/adrag

&drop

manipulate

theplanning

world

(goals,constraints,initialstate)before

anddur-

ingthe

planningprocess

n/an/a

PASSA

Tn/a

Userinterface

with

textualinform

ationadd

own

solutions,interrupt

planningprocess

withoutdata

loss,modify

taskn/a

n/a

SIAD

EXn/a

n/aplan

revision,problem

description→

ontologyserver,determ

inestrategy

andresources

yes→

real-time

supervision

n/a

interactivenarrative

n/atextualuser

interfacedefine

problem(describe

types,objects,operators,conditions

andthe

initialandgoalstate),add

information

n/an/a

MA

PGEN

n/atextual

edit,merge

andapprove

plansn/a

n/aC

OM

IREM

two

differentlevels

ofde-

tail

typingspecify

domain

model(resources,activ-

ities,constraints),can

requestresource

feasibilitycheck

andauto-scheduling

n/an/a

Table3.2:M

ixed-initiativeplanning:quality

ofcomm

unication(partII)

26 Interactions in Planning Systems – Barbara ZÖLLER

Chapter 4

Planning in Human-Robot-Interaction

An application area of planning is in the fields of Robotics. Especially in dynamic environ-ments with humans the planning is more complicated and new challenges arise. In the follow-ing, first some points for integrating planners in robots are mentioned and subsequently someapproaches will be presented.

4.1 Idea of Planning in HRI

In contrast to industrial robots which work in a closed-world and are specified for certain tasks,the employment of robots in the real-world with humans is more complex. As the robot hasto recognize the presence of the human (actions, behaviour) and to interact with the human,the planning process for the robot means more effort. It is not possible to create a plan withouttaking into account the human because otherwise dangerous situations for the human couldoccur. So the robot needs a decision making ability and in the same time it should achieveits tasks [5]. Though, predicting humans’ behaviour is a critical point, since it contains lot ofuncertainty. Additionally, it might be necessary that the robot acts differently in the presenceof different persons (according to their needs and preferences) [4]. Application areas can beassisting robots, for instance in household or office, or in search and rescue settings, where it istoo dangerous for humans. This means that the humans are typically no robot experts. Thus,the robot has to be intuitive usable and a simple (natural) way for communication is necessaryto facilitate the interaction [12].

4.2 Criterions to analyse planning in HRI

To compare and analyse the different systems for planning in HRI, the followong criterions areapplied:

• social constraints: the applied social constraints that were regarded for the development

• adaption to individual: Is the robot able to adapt his actions to individuals with e.g.different preferences or capabilities ?

Interactions in Planning Systems – Barbara ZÖLLER 27

• direct HRI: indicates if the proposed methods is suitable for direct HRI, i.e. the robot andthe human really have to execute actions together (the human is not only present in therobot’s environment)

• communication methods: Which ways exist to communicate between the human andthe robot ?

• kind of planning: denotes, which kind of planning is included in the system (descrip-tion): motional versus task planning

• used criterions: Which criterions where used to evaluate the developed system ?

• model update: How / When is the actual model updated ?

4.3 Approaches for Planning in HRI

In the following, I present some developed approaches for Task Planning for Robots in thepresence of humans.

4.3.1 Toward human-aware robot task planning

Alami et al. developed a decisional framework that supports the robot in producing behavioursin an environment with humans and in interpreting human behaviours [4]. The research isdone based on the framework of Cogniron, a cognitive robot companion. The objective isto develop task planners and interaction schemes that allow the robot to consider humans’abilities (as well as resulting constraints, special needs and individual preferences) during itis acting in the world. Therefore the robot has to be capable to manage its interactions. Thus,the robot has to observe the human continuously and it has to be accepted by the human.The control of the robot is regulated by a three layer architecture. InterAction Agents (IAA)are used in the decisional layer. These agents represent humans. Additionally, they definea complete process of establishing a common goal, achieving it and verifying commitmentsof all insolved agents involved. The authors imagine a common space for two agents, whoexchange information in a multi-modal way. When the robot has a goal to satisfy, it needs tocommunicate with the human - establishes, maintains and terminates a connection with him.This includes goal establishment, incremental refinement of the task and execution monitoring.The decisional framework consists of different parts:

• Agenda: It manages the current set of robot goals. There are active, inactive and sus-pended goals. It controls the consistency of goals and causal links and defines their pri-ority. Further it assigns new goals to agents.

• IAA manager: These are representations for the humans. They are created dynamicallyand contain various information about the human (used by the planner): abilities, pref-erences, information provided by perception.

• Task delegates: Active goals are achieved by executing actions. A Task Delegate is anentity that represents a task corresponding to an active or suspended goal. In general, theprogress between the robot and the human is monitored.

28 Interactions in Planning Systems – Barbara ZÖLLER

• Robot supervision kernel: This is a central part for the planning. It is responsible of taskrefinement, selection and execution. It can follow all robot activities and send executionrequests to the functional level.

The challenge of human-aware task planning is that besides to performing their tasks, robotshave to perform in an acceptable, legible, predictable and directable way for humans. Socialconstraints have to be respected. A formalization has been created to represent the robot andthe human. The set of actions with the connected costs is registered for every agent (robot orhuman). The costs are used to tag priorities. Undesirable states, (un)desirable sequences ofactions, costs to denote the difficulty and pleasure for an agent in action realization and as wellsynchronization / protocols for social constraints can be represented.

Another issue is the motion planning. That means for robots that they must not only followsafe robot paths, but they should also be “socially” acceptable for humans. It denotes the hu-man should not be frightened by the robot, when it suddenly appears next to him. Informationabout humans vision field, shared motions and accessibility are relevant. The authors use twocriterions to adapt their robot better to humans:

• Safety: to control the distance between robots and humans in the environment

• Visibility: to manage the path of the robots according to humans’ field of view.

The aim of these criterions is to make the humans feeling safe by keeping the robot in a bigdistance and avoiding that it suddenly appears next to him in his field of view. Other relevantpoints are the velocity and the acceleration of the robot. The authors use a weighted combi-nation of distance, visibility and comfort to analyse the action space. Based on these values,they calculate the satisfactory path and the velocity. You can imagine the estimated values asnumerical potentials in a 2D grid depending on the position of the human. User trials wereused to learn more about human preferences.

Finally, the approach has still to be developed to be applied in situations where they haveto get in contact, e.g. to hand an object. On the other hand, also the robot should execute theactions in a way that they are understandable for the human e.g. the human should take anobject which is handed by the robot. At last, another challenge is to find good formalizationsfor the different constraints of different types: temporal, causal or geometrical.

4.3.2 An integrated planning and learning framework for human-robot interaction

Kirsch et al. provide a framework that supports planning for HRI [14]. Application domainsare assistive applications where the robot interacts with the human in a very dynamic environ-ment. The dynamic derives from the humans who change their mind, interrupt tasks (withoutbeing comprehensible for the robot) and do actions not always in a optimal (but still feasible)way. So robots need a way to formalize humans’ behaviour to have a plan about their own andhumans’ actions. Further, the execution of the plan has to be flexible in regard of social rulesand personal preferences. According to the authors, the main issues of HRI are:

• Models of human abilities and intentions: Besides to the need of an appropriate repre-sentation for models about human’s abilities and intentions, the robot should be able to

Interactions in Planning Systems – Barbara ZÖLLER 29

observe and learn individual preferences and habits on his own (during the interactionwith the human).

• Planning techniques to produce legible behaviour: The robot has to consider the per-son’s preferences and abilities during the planning process. Further, it has to distinguishunusual situations from normal ones.

A framework, which is a combination of TRANER (TRAnsformational planNER) and RoLL(Robot Learning Language) is proposed by the authors. The result is an extremely flexiblesystem for HRI.

• TRANER: It can adapt plans to human preferences based on transformation rules and onReactive Plan Language (RPL). Further it reasons about the plans and eventually modifiesthem, if they are not feasible. Its advantages are that it can give abstract advices (sothey are better understandable for humans) and its plans generate flexible, reliable andefficient robot behaviours. Additionally, declarations are available to signal and catchfailures as well as recover from them. To apply TRANER the following conditions haveto be fulfilled:

– Plan transformation rules have to be available (for redefining the assignment of ac-tions to partners)

– Plan language needs a representation to include human activities– More general mechanisms for evaluating plans needs to be added (as humans have

to be taken into account, but they are difficult to simulate)

• RoLL: It deals with the complete learning process: acquisition of experience, executionof learning algorithms and integration of learning results. The problems solved by RoLLinclude time prediction models of activities and decision functions for parameters of ac-tions. It updates the models with the experience gained during plan execution (overtime). The challenges of learning predictive models are: Predict the time a human needsfor an action, the chance of successful execution of a task, the probability of needing helpand the reliable perception and classification of human activities.

In a given example, the situation describes a household assistant robot that helps loadingthe dishwasher. In this scenario its actions depend on the capabilities and preferences of thedifferent individual persons as the robot should not do actions the elderly person can still doon her own. Otherwise, it should recognize unusual situations to adapt its behaviour. Finally,the robot needs a model for the HRI, which specifies all the necessary information about thehuman (abilities, skills, preferences, social rules). They can be split up in two groups: Thosewhich can be derived from social psychological studies and which can change over time likeindividual preferences and abilities. The plan has to represent the human and the robot, there-fore the gathered knowledge about the human is required. Failures of the human have also tobe monitored by the robot. Then it has to decide, if he is able to handle it alone or if he needshelp.

The steps when using the framework are as follows:

1. Robot chooses an appropriate plan (that achieves its goal) from library

30 Interactions in Planning Systems – Barbara ZÖLLER

2. Usefulness of the plan is examined by predicting its outcomes

3. If the plan is still not optimal, it is transformed by using generic plan transformation rules

4. The sufficient good plan is executed

5. During execution the robot observes the human to make new experiences about his be-haviour

6. The Robot updates his model accordingly

The advantage of this approach is that the robot adapts continuously its behaviour over timein respect of his observations.

4.3.3 Planning and plan execution for Human-Robot Cooperative Task achieve-ment

Planning for Human-Robot interaction is different from planning in other contexts. Alili et al.present an human aware task planner (HTP) which is based on hierarchical task planning andintegrates social behaviour (see [6]). In this respect, the human is handled as an agent withcapabilities, preferences and a state (like other agents). So the plan consists of actions of therobot and the agents and has to be consistent. The HATP is composed of:

• World database: Unique set of entities defined by attributes (static or dynamic, atom orvector).

• Tasks:

– Operators: Atomic tasks which is defined as a tuple of name of the action, a precon-dition, the effect of the action execution, a cost function and duration (the latter twodepend of the agent and the context).

– Methods: Higher level task which have to be decomposed for realization (until thelevel of atomic tasks). They consist of a goal and a set of decompositions.

To be applicable in a human environment some special rules are necessary that you will getsocially acceptable plans. Six types of those so called social rules exist:

• Undesirable states: for unpleasant, undesirable and inadequate world states

• Undesirable sequences: to avoid uncomfortable combinations of robot

• Bad decompositions: to classify the decompositions

• Effort balancing: to balance the effort among partners

• Timeouts: to avoid long waiting time if two actions are done by the same agent

• Intricate synchronisation links between streams: complicate the plans and thus shouldbe avoided

The algorithm consists of two steps: Plan refinement and plan evaluation. Planning starts froma single task or from a tree of tasks where some choices have already been done. Further for

Interactions in Planning Systems – Barbara ZÖLLER 31

parallel tasks (without causal link), it is the aim to get a final plan that is as independent aspossible. The evaluation is done with several criterions and in use of a specific metric. Thedecision making is facilitated by decomposing the problem into a hierarchy of more easilycomprehended sub-problems (which can be independently analyzed). The decision architec-ture consists of the task planner HATP and SHARY. This is a supervision system based onan incremental context-based task refinement in a human context. It supports interactive taskachievement by taking into account also communication and monitoring. Contingencies canbe handled , as the system defines communication policies which are linked with to singletasks or a hierarchy of tasks. The communication scheme exists of states and correspondingtask(s) which are launched. It is a finite state machine where communication acts are expressedthrough dialog or by an expressive motion (executed by the robot). Transitions are inferred ofthe humans’ behaviour or directly expressed by the human.

4.3.4 Planning for Human-Robot Teaming (HRT)

Human-robot teaming is a challenging subject as it has to combine the autonomy of the robotwith the presence of a human, which restricts the robot’s “liberty”. Therefore, changes of theenvironment which influence the planning (like new constraints or modified goals) have tobe considered. The aim of the authors is to facilitate teaming by adapting existing planningtechniques for HRI to teaming scenarios [18]. Hereby, the robot and the human want to achievea goal together. The robot is completely autonomous, that means, it receives a set of goals withsome constraints and then it has its own planner to create an appropriate plan. The maincomponents of the scenario are:

• Scenario: This consists of approximations of the real-world. The necessary domainknowledge can be gathered by humans who are experienced in this environment or outof manuals. The scenario determines which kinds of tasks and features the robot has tosupport.

• Robot: There exist various types with varying capacities. So they determine the actionsthat may be uses.

• Human: He is the driving element, as he often defines the goals. Users can be part of oneof those categories:

– Novice: A person who does not understand the intricacies of the system (e.g. assis-tant robor for elderly)

– Domain Expert: The user is an expert in his environment, but he does not knowthe system. He can define new goals, pass information to the robot and acts as anauthority (e.g. commanders in military).

– System Expert: The person is familiar with the system that manages the robot (e.g.programmers).

• Model Management: The internal representation of the environment is set up of differentcomponents. Their model can vary at different levels of detail.

• Goal Management: This component specifies and updates the goals. The goal speci-fication should be flexible and has to manage goals’ priorities, soft goals (need not beachieved) and those that depend on unknown facts.

32 Interactions in Planning Systems – Barbara ZÖLLER

• Communication: As human and robot have to work as a team, communication is a crit-ical point. This affects everything in the scenario (specification, modification). Thereforethe various representations of the human and the robot have to be translated in the samelevel, so that information loss and processing time are as small as possible.

The authors see particularly two problems for planning in HRI. The use of different models -higher level for actions that support the end goals and lower level to decompose those tasks.

As already mentioned, it is complicated to define models for dynamic environments. Es-pecially, if these changes occur during the agent is acting in the world. First updates have tobe specified and then they need to be integrated in the actual model. This influences on theother hand the reasoning about the changes and the resulting effects on the plan’s validity. Thefact, that those changes often influence only a part of the system, is not already used. Reducedrobustness is a result of incomplete models, as the probability of failure grows with the degreeincompleteness. One solution for this problem are reactive plans (but this does not work fornot modelled parts of the problem as this kind of plan gets blocked). So another approach isto have only incomplete models which will be completed and modified during the planningprocess. Therefore good methods to specify changes are necessary. Often it happens that thosechanges implicate a better solution for the planning problem. The process of model updatingcan be divided in two tasks:

• Model maintenance: Changes are specified in the same format that is used for represen-tation. Post-update consistency is easy as the specification already verifies the permit ofupdates. This method fits when the planner is part of a larger integrated scenario.

• Model revision: Here less formal specified updates are integrated in the system. Thisleads to more complex effort to keep the validity, the model consistency and the guaran-tees about goal achievement.

For HRT it is more useful to have reasonable update capabilities instead of replanning froma scratch when changes occurred, if changes often affect only a (small) part of the model andmaybe even not the actual plan.

The task of HRT is to achieve certain goals. At this point, the specification and formalizationof goals is critical. This is mostly done by humans and they may be incomplete or incorrect.Therefore there is also an update method for goals necessary. Often necessary informationfor the goal are not known in the beginning. The situation is made more difficult, as there isa gap between the real open world and the closed world of the robot (only a cut-out of thereal world). A solution for this problem is the use of Open World Quantified Goals1 (OWQL).Thereby information about objects that may be discovered during execution are combined withpartial satisfaction aspects of the problem. Thus the domain expert can add details of recentlydiscovered objects and the involved goals (which enable eventually the achievement of newgoals).

1 This is a new kind of goals introduced by Talamadupula and Co. It provides for the specification of informationand creation of objects required to take advantage of opportunities that are encountered during the plan execution.It enables to influence the planner’s view of the search space towards finding plans that achieve additional rewardin an open world [17].

Interactions in Planning Systems – Barbara ZÖLLER 33

4.4 Limits of existing approaches for planning in HRI

The approaches for planning in HRI are also human-centred but of different nature. As robotand human are supposed to act in the same environment, the robot has to be more carefullythan it would be in an isolated area. Further, the robot has to move as the human supposesand should not scare him by approaching too close or showing up suddenly next to him. Inthe meantime, the robot has still to perform its tasks. So besides to “classical” constraints (likefixed obstacles or robot kinematics), which exist also in an environment without humans, nowso called social constraints make the actions for robots more difficult. It is necessary to under-stand humans’ behaviour, in order that the robot can use this knowledge for action planningin a human environment. Hence, a new challenge is to have an appropriate formalization forhuman behaviour. Sensors for the robot are necessary to be able to observe the human. Indoing so, the robot has to update his model of the “world” and eventually changes its plan ifthe modifications affect the actual one.

The application domain of planning in HRI is different. It is common to use it for assistantrobots which help humans in different areas of life. It exist household robots to help elderly orhandicapped e.g. loading the dishwasher (cf. section 4.3.2) or to aid at work in the office e.g.delivering required documents. Similarly, robots are employed to guide visitors in exhibitions.Moreover, robots are applied if the situation is too dangerous for humans, like as search andrescue services after an earthquake. As further earthquakes could occur, it is risky for humansto search for survivors and injured in ruins. Instead of that, robots can undertake the task ofsearching humans and initiate further measures.

To compare the different approaches (see section 4.3) I used the criterions described insection 4.2. You can get an overview of the results vie the tables 4.1 and 4.2. In summary,all the examined systems use social constraints to adapt the planning for the human two (cf.sections 4.3.1 & 4.3.2) provide the possibility to learn the individual preference of individualpersons. But only one (cf. sections 4.3.2) mention direct interaction between human and robot,when laoding the dishwasher. The difficulty in communication are the different forms of rep-resentation for humans and machines (cf. section 4.3.4). The system of section 4.3.4 applies afintie state machine to generate communication for different tasks. Whereas task planning toachieve the goals is always described, motion planning is named only in one (cf. section 4.3.1).The most important used criterion for the planning is safety (for the human). Updates of themodel are used to complete the model (cf. section 4.3.4) and to add the preferences of individ-uals (cf. section 4.3.2).

34 Interactions in Planning Systems – Barbara ZÖLLER

crit

erio

nso

cial

cons

trai

nts

adap

tion

toin

divi

dual

dire

ctH

RI

com

mun

icat

ion

met

hods

hum

an-

awar

eta

skpl

anni

ng[4

]

pref

eren

ces,

abili

ties

,sp

ecia

lnee

ds,i

nfor

mat

ion

prov

ided

bype

rcep

tion

yes

nom

ulti

-mod

al(n

ode

tails

)

inte

grat

edpl

anni

ngan

dle

arni

ngfr

amew

ork

[14]

hum

anab

iliti

es&

pref

er-

ence

sro

bot

lear

nsin

divi

dual

pref

eren

ces,

upda

tes

mod

el

yes

(loa

ddi

shw

ashe

rto

-ge

ther

)no

tdet

erm

ined

hum

anro

bot

coop

erat

ive

[6]

pref

eren

ces

(und

esir

able

stat

esan

dse

quen

ces)

n/aa

n/a

polic

ies

linke

dto

sin-

gle

task

sor

hier

arch

ies

ofta

sks

(fini

test

ate

mac

hine

)H

RT

[18]

yes

n/a

n/a

need

ofva

riou

sre

pres

en-

tati

ons,

tran

slat

edfo

rth

eco

mpa

nion

inhi

sle

vel

wit

hm

inim

izat

ion

ofda

talo

ssan

dpr

oces

sing

tim

e

Tabl

e4.

1:Pl

anni

ngin

HR

I(pa

rtI)

a nota

vaila

ble

Interactions in Planning Systems – Barbara ZÖLLER 35

criterionplanning

usedcriterions

modelupdate

human-

aware

taskplanning

[4]

motional

andtask

plan-ning

safetyand

visibilityn/a

integratedplanningand

learningfram

ework

[14]

taskplanning

flexibility,reliability,

effi-ciency

accordingto

robot’sexpe-

rience;inform

ationabout

human

(abilities,skills,

preferences,socialrules)

human

robotcooperative[6]

taskplanning

undesireablestates

orse-

quenciesn/a

HR

T[18]

taskplanning

n/aintegrated

inactual

modelto

complete

model:

maintenance

(changeof

alreadyregistered

values),revision

(newinform

ationis

added)

Table4.2:Planning

inH

RI(partII)

36 Interactions in Planning Systems – Barbara ZÖLLER

Chapter 5

Interactive Planning in HRI

After the analysis and comparison of several systems for mixed-initiative planning and plan-ning for HRI, the question arises if it makes sense to combine the both ideas.

As seen in chapter 3, the aim of mixed-initiative planning is to improve planning in case ofincomplete or / and dynamic models and to adapt it better to individual preferences. Addi-tionally, it is applied for criterions which are hardly to define formally, so that they can be usedfor the automated planning and in cases of big uncertainties which are difficult to define, too.Moreover, humans without a sufficient knowledge about the theory of automated planningdo not trust them. In this case, it is better, if they can have a look at its “inside”. Interactionsupports modification affecting the planning process and adaption of the system for differentenvironments. However, a good interface is necessary to permit an appropriate usability forthe user and enable him to interact in the planning process.

However, planning in HRI requires special requirements and consequently constraints topreserve the safety and comfort for the human (cf. chapter4). Therefore certain kinds of addi-tional information are necessary and the planning process has to be adapted to the preferencesof the users, for instance assistance robots are not supposed to help always in the scope ofservices. In summary, actual information (e.g. possible via perception) is very important andessential in HRI. Also in this field, communication between human and robot is fundamentalas they have to fulfil tasks together and each of them should understand the purpose of theactions of the other one.

I can draw the conclusion that in both situations, it is important to have appropriate suf-ficient information. Otherwise, it is hard to achieve the desired result and in situations withhumans it can even be dangerous. For example, the human could move, but the robot is notequipped with adequate possibilities to percept the movement of the human, than it can cometoo close to human from behind what will frighten him or even hurt him, because the robot’sbig weight or any abrupt move. So mixed-initiative interaction for planning in HRI makessense to enable to provide additional information to the robot and close the gap of informationsupplied by the model and sensors. But it reminds the question, how this kind of interactionshould look like. Is there an additional person in the environment, who controls the planningand interacts in the on-line 1 planning to prevent the robot from mistakes or dangerous sit-uations (for human or robot, which could also fall down) or off-line to adapt the plan better

1 This means planning is done during the execution of the actions. The opposite is off-line where the planning isfinished before starting its execution

Interactions in Planning Systems – Barbara ZÖLLER 37

to individual preferences for the human or a special environment. On the other hand, the in-volved person himself could participate at the planning process, either off-line or on-line.

For on-line planning it is necessary that the planning system is able to create plans in real-time. This signifies that the plan is provided without a long waiting time, as it will be uselesswhen a robot is stopping for an hour its actions for creating a new plan. In critical situations,like the rescue scenario, this can be very critical and also when an assistant robot would inter-rupt his actions too long for planning, it is hard to understand the reason for a human and heprobably things some problems with the robot occurred.

Then there is still the problem how to support the interaction. The presented approaches insection 3.3 mostly use a graphical user interface wherefore you need a display. A speech inter-face is provided rarely, but there is still a display necessary to show relevant information. Foron-line planning the planner has to be somehow connected to the robot to enable the transmis-sion of data. Another idea is to use mixed-initiative planning to recheck the plan by a humanbefore execution. In any case, an intuitive and easy solution for communication has to be pro-vided, to facilitate the interaction for the human and motivate him (instead of scaring him bytoo complicate communication modes).

As already seen in chapter 4 it is important to adapt the systems / machines to the hu-mans. This concerns the behaviour of robots which have to adapt (itself) to humans, but alsothe usability of those systems has to be improved. This implies that the system is intuitive com-prehensible and thus easy usable. Here it is necessary to use expertise of psychology to learnmore about the human perception to adapt user interfaces accordingly. In general it is essentialto request more information of others domains which analyse the behaviour of humans - thismeans psychology and sociology. It supports to achieve a better inclusion of the robot in thehuman world by acting in a way that seems more natural for the human by applying the rulesof the human social behaviour. Finally, it conveys that a new domain “social robotics” needs tobe established which joins social science with computer science to a multidisciplinary researchfield.

In conclusion, it is necessary to optimize the planning techniques, so that they will be rea-sonable applicable in real-time. On the other hand, robots have to adapt better in the humanenvironment and therefore the “technical” research has to be combined with social sciences ina multidisciplinary research domain.

38 Interactions in Planning Systems – Barbara ZÖLLER

Bibliography

[1] Logic programming and artificial intelligence. http://www.pling.org.uk/cs/lpaimg/sussmananomaly.png. Accessed: 20/02/2012.

[2] Planning as search: Blocksworld. http://www-prima.imag.fr/jlc/Courses/2011/ENSI2.SIRR/ENSI2.SIRR.S3.pdf. Accessed: 18/02/2012.

[3] Mitch Ai-Chang, John Bresina, Leonard Charest, Jennifer Hsu, Ari K. Jonsson, Bob Kanef-sky, Pierre Maldague, Paul Morris, Kanna Rajan, and Jeffrey Yglesias. Mapgen planner:Mixed-initiative activity planning for the mars exploration rover mission. In IEEE Intelli-gent Systems, vol. 19, no. 1, pages 8–12, 2004.

[4] Rachid Alami, Aurélie Clodic, Vincent Montreuil, Emrah Akin Sisbot, and Raja Chatila.Toward human-aware robot task planning. In AAAI Spring Symposium: To Boldly Go WhereNo Human-Robot Team Has Gone Before, pages 39–46, 2006.

[5] Rachid Alami, Aure Clodic, Vincent Montreuil, Emrah Akin Sisbot, and Raja Chatila. Taskplanning for human-robot interaction. In In sOc-EUSAI 05: Proceedings of the 2005 jointconference on Smart objects and ambient intelligence, pages 81–85. ACM Press, 2005.

[6] Samir Alili, Matthieu Werner, Muhammad Ali, and Rachid Alami. Planning and plan-execution for human-robot cooperative task achievement. In 4th Workshop on Planning andPlan Execution for Real-World Systems. AAAI, 2009.

[7] Mark H. Burstein, Bolt Beranek, Newman Inc, and Drew V. Mcdermott. Issues in the de-velopment of human-computer mixed-initiative planning. In Cognitive Technology, pages285–303. Elsevier, 1996.

[8] Michael T. Cox and Manuela M. Veloso. Supporting combined human and machine plan-ning: An interface for planning by analogical reasoning. In Case-Based Reasoning Researchand Development, Proceedings of ICCBR-97, the Second International Conference on Case-BasedReasoning, pages 531–540. Springer Verlag, 1997.

[9] Michael T. Cox and Chen Zhang. Planning as mixed-initiative goal manipulation. In InProceedings of the Fifteenth International Conference on Automated Planning and Scheduling,pages 282–291. AAAI Press, 2005.

[10] George Ferguson, James Allen, and Brad Miller. Trains-95: Towards a mixed-initiativeplanning assistant. In Proceedings of the Third Conference on Artificial Intelligence PlanningSystems (AIPS-96), pages 70–77, 1996.

Interactions in Planning Systems – Barbara ZÖLLER 39

[11] Juan Fernez-Olivares, Luis A. Castillo, Oscar Garcia-Perez, and Francisco Palao. Bringingusers and planning technology together. experiences in siadex. In ICAPS, pages 11–20.AAAI, 2005.

[12] Cipriano Galindo, Javier Gonzalez, and Juan-Antonio Fernandez-Madrigal. Interactivetask planning through multiple abstraction: Application to assistant robotics. In ECAI,pages 1015–1016, 2004.

[13] Malik Ghallab and Paolo Traverso Dana Nau. Automated planning. Elsevier Inc., 2004.

[14] Alexandra Kirsch, Thibault Krause, and Lorenz Msenlechner. An integrated planningand learning framework for human-robot interaction. In 4th Workshop on Planning andPlan Execution for Real-World Systems (held in conjuction with ICAPS 09), 2009.

[15] Karen L. Myers, W. Mabry Tyson, W. Mabry, Peter A. Jarvis, Thomas J. Lee, Marie des-Jardins, Tyson Michael, and Michael J. Wolverton. PASSAT: A User-centric PlanningFramework. In In Proceedings of the 3rd International NASA Workshop on Planning andScheduling for, 2002.

[16] Stephen F. Smith, David W. Hildum, and David R. Crimm. Comirem: An intelligent formfor resource management. In IEEE Intelligent Systems vol. 20, 2005.

[17] Kartik Talamadupula, J. Benton, Paul W. Schermerhorn, Subbarao Kambhampati, andMatthias Scheutz. Integrating a closed world planner with an open world robot: A casestudy. In AAAI, 2010.

[18] Kartik Talamadupula, Subbarao Kambhampti, Paul Schermerhorn, J. Benton, andMatthias Scheutz. Planning for Human-Robot Teaming. In 21th International Conferenceon Automated Planning and Scheduling, pages 82–89, 2011.

[19] James M. Thomas and R. Miachael Young. Author in the loop: Using mixed-initiativeplanning to improve interactive narrative. In Working Notes of the International Conferenceon Automated Planning and Scheduling (ICAPS) 2006 Workshop on AI Planning for ComputerGames and Synthetic Characters, Cumbria, UK.

40 Interactions in Planning Systems – Barbara ZÖLLER