a description of amitiés modules developed at suny hilda hardy amitiés consortium meeting...
TRANSCRIPT
A Description of Amitiés Modules Developed at SUNY
Hilda Hardy
Amitiés Consortium MeetingUniversity of Sheffield
May 2, 2002
2
Xinyang Zhang
3
Amitiés System Architecture
Hub
Speech Recognition
Dialogue Manager
Database Server
Text-to-speech Conversion
Nat’l Language UnderstandingFrench
German
Audio Server
Response Generation
FrenchGerman
CustomerDatabase
Engl.
LIMSI
Sheffield
SUNY
4
What we like about Galaxy,MITRE’s Galaxy Communicator SoftwareInfrastructure:• Distributed, message-based hub-and-spoke
infrastructure• Optimized for constructing spoken dialogue systems• Built-in Hub, implemented in C
– Maintains socket connections to servers– Handles message traffic
• Protocol for frames, functions, program files clearly defined
• Excellent on-line tutorial, documentation, many examples
5
What we don’t like about Galaxy (some minor points):
• Difficult to use an object-oriented approach for developing servers
• Not platform-independent• Frequent new versions, problems with memory
management
6
Hub program file: amities.pgmPGM_SYNTAX: extendedSERVICE_TYPE: UICLIENT_PORT: 14500SERVER: DialogueHOST: localhostPORT: 18500OPERATIONS: DoDialogue DoGreetingSERVER: Response_GeneratorHOST: localhostPORT: 10000OPERATIONS: GenerateSERVER: French_GeneratorHOST: localhostPORT: 15000OPERATIONS: GenerateSERVER: German_GeneratorHOST: localhostPORT: 15500OPERATIONS: Generate. . .
server info
port number determined by server
at server startup
7
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; MAIN OUTPUT BODY;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; This program handles the response from the dialogue;; manager which may take one of a number of ;; forms (database tuples to be described, or perhaps;; an already-formatted frame).PROGRAM: DialogueOutputRULE: :output_frame --> Response_Generator.GenerateIN: :output_frameOUT: :output_stringRULE: :output_string -->Display.DoDisplayIN: :output_stringOUT: none!PROGRAM: FrenchDialogOutputRULE: :output_frame --> French_Generator.GenerateIN: :output_frameOUT: :output_string. . .
amities.pgm: example Hub program
name of program, frame dispatch function
condition(s)Hub program
8
Dialogue Manager/Frame Router• Handles task identification
1. Change address
2. Replace card
3. Inquire about account balance
• Handles customer identification • Uses a mixed-initiative approach• Implements layers of abstraction for language and
database independence• Can recover from word recognition errors• Can perform mid-conversation language switching
9
Example frames: NLU to DM
{c NLU.Parse
:token_string (“Christopher” “Smith” “here”...)
:named_entities {c entities
:lname “Smith”
:fname “Christopher”}}
{c NLU.Parse :token_string (“Hello” “I” “wanted” “to” “notify” “you” “of” “a” “change” “of” “address”)}
frame type name
keys
list value
frame value
string values
10
Dialogue Server Dialogue Manager
Task ID Frame Router
User ID Frame Router
1 2
Task1
Keyword profilePrompt sequence
Task2 …
properties file
from NLU via Hub
11
Dialogue Manager: Some Details• Accomplishes goals in any order, even all at once• Loads properties file according to language ID• For task ID: entire utterance need not be recognized
as long as the salient words are identified• Queries database for named entities, uses “longest
common subsequence” dynamic programming technique. Example:
1 2 3 4 5 6
1 too 3 4 5 sick
1 X 3 4 5 X
1 2 3 4 5 6
1 2 tea or 5 6
1 2 X X 5 6
from database
from user
matching pattern
1 2 3 4 5 6
12
Dialogue Manager: Data Representation
A Working-space structure is maintained for each dialogue:– Primary keys, attributes, values, matched patterns– Matching-percentage scores
primary key
score
primary key
score
primary key
score
. . .
attribute
value
match_string
score
attribute
value
match_string
score
. . .
13
Database Server (pgdb_server)• Provides connectivity to a PostgreSQL database• Uses an extended data model: maps meta-attributes
to actual attributes• Sample table constructed from fictitious English,
French and German customer data• Example input from Dialogue Manager:
{c sql_query :query “select title, fname, lname, post_code, house_number from table where house_number = ‘1’ and account_number = ‘1234567887654321’” :maxrows 10 }
14
Database Server: Sample return frame
{c sql_query :query_result {c result
:values ( (“Mr.” “Stephen” “Jones” “EC4 5BE” “1” ) )
:column_names (“title” “fname” “lname”
“post_code”
“house_number”) :nfound 1 } }
one list for each hit
15
pgdb_serverDM/FR
Customer Database
Meta-attributes Meta-attributes
Actual attributes
fname lname post_code . . .
first name surname post code . . .
Database Server
properties file
via Hub
16
Multilingual Response Generator
• Implemented with separate servers in English, French and German
• Uses an extended data model: maps meta-names to actual words in the appropriate language
• Constructs responses according to our abstract representation of system utterances
17
Response Generator: Sample input, output frames
{c French_Generator.Generate OR FrenchDialogueOutput :output_frame {c output
:type “request” :repetition 0 :attributes ( “name”
“post_code” )
:modifiers ( “” “old” ) } }
{c French_Generator.Generate :output_string “Pouvez-vous me donner votre nom et votre ancien code postal, s’il vous plait?” }
from Dialogue Manager
to Display Server
18
Response Generator: Schema for representing system utterances
:type :repetition :attributes :modifiers
A: Amitiés, how can I help, [French], [German]?“prompt” 0A: Can I take your name and your old post code, please?“request” 0 (“name” “post_code”)(“” “old”)A: Thank you. I need to ask you some questions so that I can update
your address in our records.“respond” (“change_address”)A: How can I help?“prompt” 1A: I’m sorry, what can I do for you?“prompt”2A: Please hold while I transfer you to a Customer Service
Representative.“respond” (“hold”)
19
Dialogue Manager
Response Generator
German Generator
properties fileFrench Generator
meta-names
English namesFrench namesGerman names
Language Generator Servers
properties file
communication via the Hub
20
Response Generator: Meta-names and Engl/Fr/De equivalents
name name nom/m Namen/ffname first name prénom/m Vornamen/mlname last name nom/m Nachnamen/mhouse_number house number nombre de maison/f
Hausnummer/faddress address adresse/f Adresse/fpost_codepost code code postal/m Postleitzahl/faccount_number account number numéro de compte/m
Kontonummer/fbirthdatedate of birth date de naissance/f Geburtstag/mold old ancien/ancienne altenew new nouveau/nouvelle neue