TrindiKit

What is TrindiKit?

• a toolkit for – building and experimenting with dialogue move engines and systems, – based on the information state approach

• originally developed in TRINDI, considerably extended and improved in SIRIDUS

Key ideas

• thinking in terms of IS updates– update rules

• functions IS (+moves) IS (+moves)

• generic domain-independent dialogue management– requires modularity

• use of global information state – defined in terms of abstract datatypes– all modules can access all information

module1 module…

Total Information State (TIS)•Information state proper (IS)•Module Interface Variables•Resource Interface Variables

resource1

control

modulei modulej module… modulen

resource… resourcem

DME

TrindiKit

basicdialogue theory

domain & languageresources

basic system

application

information state approach

genre-specific theoryadditions

genre-specific system

• explicit information state datastructure – makes systems more transparent – enable e.g. context sensitive interpretation, distributed decision

making, asynchronous interaction

• update rules – provide an intuitive way of formalising theories in a way which can

be used by a system– represent domain-independent dialogue management strategies

• resources– represent domain-specific knowledge– can be switched dynamically

• e.g. switching language on-line in GoDiS

• modular architecture promotes reuse– basic system -> genre-specific systems– genre-specific system -> applications

TrindiKit features

technical features

• interfaces to OAA (but can also run without it)– see e.g. demo by Zinn et al.

• system modules can run either serially or in parallell

• wrappers for off-the-shelf recognizers and synthesizers

• runs on UNIX, Windows, Linux• currently uses SICStus Prolog

– but considering moving to YAP prolog (freely available Sicstus clone)

– possibly reimplement in other language

availability• version 2.1 is available• version 3.0a coming at end of 2002

– SIRIDUS deliverable D6.4

• TrindiKit website– www.ling.gu.se/projects/trindi/trindikit

• SourceForge project– development versions available– developer community?

• licensed under GPL• more info in

– Larsson & Traum: NLE Special Issue on Best Practice in Dialogue Systems Design, 2000

– TrindiKit manual (available from website)

Flexible issue-based dialogue management

Goals

• explore and implement issue-based dialogue management– adapt Ginzburg’s KOS to dialogue system (GoDiS)

and implement– extend theory (incl. accommodation, action-oriented

dialogue, negotiation, conditional responses)

• separating general and domain-dependent phenomena helps reconfigurability– general theory of dialogue, extended into

subtheories for different dialogue types– minimize effort for adapting to new domains

TrindiKit

GoDiS

GoDiS-I GoDiS-A

TravelAgency

Auto-route

Xeroxmanual

VCRmanager

IBDM

homedevice

manager

ISapproach

genre-specific

application-specific

Basic issue-based dialogue management

• enquiry-oriented dialogue (database search)• starting point:

– Ginzburg’s DGB and – related DGB update protocols

• moves: ask, answer, (greet, quit)• raising and addressing issues

– incl. short answers

• dialogue plans• handling multiple simultaneous issues• information sharing between plans• sample domain: travel agency

basic infostate

PRIVATE : PLAN : OpenStack( Action )

AGENDA : OpenQueue( Action )

SHARED :

BEL : Set( Prop )

COM : Set( Prop )

QUD : OpenStack( Question )

LU: SPEAKER: Speaker MOVES: OpenQueue( Move )

ISSUES : OpenStack( Question )

• QUD:local, questions available for ellipsis resolution• ISSUES: global, questions which have been raised but not yet resolved

Semantics• Proposition: n-ary predicate-argument structure

– e.g. dest-city(paris)

• Question– Y/N-questions: ?P, P is a proposition– wh-questions: ?x.P(x)– alt-questions: {?P1, …, ?Pn}

• ShortAns– individual markers: paris, april, …– yes, no

• Q-A relations (adapted from Ginzburg)– resolves(A,Q): A resolves Q

• dest-city(paris) resolves ?x.dest-city(x)– relevant(A,Q): A is a relevant answer to Q

• not(dest-city(paris)) is relevant to ?x.dest-city(x), but does not resolve it

sample dialogue plan

task: ?x.price(x)<findout(?x.how(x))

findout(?x.dest-city(x))findout(?x.dept-city(x))findout(?x.dept-month(x))raise(?x.dept-day(x))findout(?return)If return then < findout(?x.ret_month)

findout(?x.ret_day) >raise(?x.class(x))consultDB(?x.price(x))>

Dealing with multiple issues

• if user asks Q, push Q on ISSUES and load plan for dealing with Q

• if users asks Q’ while system is dealing with Q, throw out plan for Q but Q remains on ISSUES

• when Q’ resolved, Q topmost on ISSUES will trigger reloading plan for dealing with Q– general rule: if SHARED.COM contains info resolving

Q, don’t ask Q– so any resolved questions in plan will be thrown out

Information sharing across plans

• GoDiS does not keep track of which plan was being executed when propositions were added

• so information sharing is determined by question sharing across plans

• plan for VISA question: ?need_visa<findout(?x.dest-city(x))

findout(?x.dept-city(x))findout(?x.citizenship(x))consultDB(?need_visa) >

– shared 2 questions with plan for ?x.price(x)

Sample dialogue: multiple tasks & info sharing

S> Welcome to the travel agency! U> price information S> (…) Lets see. How do you want to travel? U> by flightS> (…) What city do you want to go to? U> parisS> (…) What city do you want to go from? U> do I need a visa ?S> (…) Lets see. What country are you from? U> swedenS> Okay. Yes, you need a Visa. S> Returning to the issue of price. Lets see. What city do

you want to go from?

Grounding and feedback(Interactive Communication Management)

• feedback types (Allwood, Clark)– action level: contact, perception, understanding,

acceptance– polarity: positive, negative

• update strategies– optimistic

• non-cautious• cautious

– pessimistic

• feedback and grounding for a dialogue system

Some ICM dialogue moves

• Interactive Communication Management– feedback– turntaking– sequencing: reflects dialogue structure

• icm:Level{*Polarity}{:Content}– icm:und*neg – ”I don’t understand”– icm:und*pos:P – ”To Paris.”– icm:acc*neg:Q – ”Sorry, I can’t answer Q”– icm:acc*pos – ”Okay”– icm:reraise:Q – ”Returning to the issue Q”– icm:loadplan – ”Let’s see…”

Question Accommodation

• to deal with – user initiative (other than asking questions)– giving more/less/other information than

requested– simple information revision– reraising issues

• basic idea:– move questions to QUD or ISSUES to

adapt to user utterances

Sample dialogue: accommodation

S: Welcome to the travel agency.U: From London to Paris in April

– not relevant to anything on ISSUES (since ISSUES is empty)– look in domain knowledge for a plan with matching questions– the plan for dealing with ?x.price(x) is found; this question depends on – dependent issue accommodation: push on ISSUES ?x.price(x)– load plan for ?x.price(x) – issue accommodation: push on ISSUES ?x.dest-city(x)– integrate answer (requres matching question on ISSUES)– same with ?x.dept-city(x), ?x.dep_month(x)

S: Alright, you want to know about price. (…)

– proceed to next plan item

S: How do you want to travel?– ISSUES=<?x.how(x), ?x.price(x)>

(NOTE: sys only deals with price & visa issues)

Action-oriented dialogue based on menus

• GoDiS-A: for action-oriented dialogue • each plan now associated with an action or a

question– add SHARED.ACTIONS: Stack(Action)– ACTIONS has a similar role to ISSUES

• New moves: – request(Action) – confirm(Action) [or report(Action, Status)]

• adapt accommodation strategies to AOD• sample domain: menu-based dialogue for VCR• (demo during lunch break)

Converting menus to plans

Menu Plan construct

multi-choice list: <a1, a2, ..., an >

findout({?action(a1), ?action(a2),...,?action(an)})

tick-box:+/-P

findout(?P)

dialogue window:parameter=___

findout(?x.parameter(x))

pop-up message confirming action a as done

confirm(a)

VCR menu structure fragment

• change play status– play, stop etc.

• change channel– channel: _

• timer recording– add program

• channel:_• date:_• start-time:_• end_time:_• display added program

– delete program• display existing programs• delete program:_

• settings– clock, etc.

Plan for VCR>timer recording>add program

findout(?x.program_position_to_store(x)),findout(?x.date_to_store(x)),findout(?x.start_time_to_store(x)),findout(?x.stop_time_to_store(x)),dev_do(vcr, 'AddProgram')

• (confirmations generated by update rule whenever some action is done, so not needed in plan)

Sample menu dialogue(with language change!)

S: Welcome to the VCR manager. Let’s see. What can I do for you?

U: Add a program today, channel five– request(add_program) -> plan is loaded– issue accommodation to integrate ”channel

five”

S: What time do you want to start recording?U: Five fifteen…

Other architectures

• Agent communication protocols– Open Agent Architecture– KQML

• Dialogue system toolkits– DARPA Communicator– SOAR, NL-SOAR

• Systems (implementing specific dialogue theory)– Verbmobil– VoiceXML

• have been compared to TrindiKit (see deliverables or ask)

relevant properties

• support for information state approach– which datastructures/datatypes are

supported?

• support for managing control flow– not limited to pipelining

• asynchronous processing• modularity• potential for scalability

OAA

• ”Facilitator” (hub) manages communication between ”agents” (spokes)

• facilitator can maintain global information state– what datastructures are available?

• asynchronous processing• modular• similar to scriptless version of DARPA

Communicator

KQML

• Knowledge Querying and Manipulation Language

• communication protocol between agents

DARPA Communicator

• Hub-and-spoke architecture• modularity, flexible control,

asynchronous processing• hub functions

– message routing between servers (spokes)– state maintenance– control flow; script decides which server to

call next

DARPA communicator cont’d

• limited support for information states– ”tokens”: frames with name, index, and list

of feature-value pairs– tokens processed by scripts

• set of rules determining when to call a server + which arguments (features) to pass

• Hub scripts too limited to do real dialogue management– most actual systems have separate

dialogue manager– hub mostly used for data routing

SOAR

• toolkit for modeling cognitive agents• similar to TrindiKit in some respects• keeps single information state

– however, only one datastructure whereas TrindiKit has extendible range of (possibly complex) datatypes

• update rules– some nice ways of triggering rules which can be

considered for TrindiKit

• supports ”chunking” of rules– however, may not be too useful in practice

VoiceXML

• Frame-based dialogue manager• not really an architecture, more like a

domain-independent dialogue system • Requires scripting dialogues in detail• Combine with GoDiS?

– use GoDiS as VoiceXML server, dynamically building VoiceXML scripts

– use VoiceXML specifications converted to GoDiS

Verbmobil

• not dialogue system per se– rather, dialogue translation system

• several local information states, not one global – ”partitioned blackboard”– which datastructures? extendible set?

• limited control support