dialogue system iso 24617 2
TRANSCRIPT
ISO 24617-2: A Semantically-based Standard for Dialogue Annotation
Bryan Hang Zhang
Universität des Saarlandes
2014 September
Outline:
• Motivation of building ISO Dialogue Annotation (DA)
• Intro to the Meta Model of DA
• Intro to the DiAML
( Dialogue Makeup language )
• Task: Automatic Annotation Task
2 Universität des Saarlandes
Motivation of ISO Dialogue Annotation Scheme
3 Universität des Saarlandes
Definition of dialogue act: 1. Dialogue act evolves from speech act. However, proper definitions and concept of
dialogue act varies.
4 Universität des Saarlandes
In reality there is a wide range of taxonomies and inventories about dialogue act.
5 Universität des Saarlandes
ISO 24617-2 dialogue act annotation is created
That’s why we need a standard for Dialogue act annotation which is inter-operable so annotated corpora can be reused.
6 Universität des Saarlandes
ISO 24617-2 Dialogue Act Annotation
• Based on DIT++ (Dynamic Interpretation Theory) taxonomy.
• Information State Approach:
Dialogue
acts
Information state of participants
UPDATE OPERATION
7 Universität des Saarlandes
Medamodel for Dialogue Annotation
8 Universität des Saarlandes
(DiAML) the Dialogue Act Markup Language
abstract syntax : specifies the possible annotation
structures in set-theoretical terms;
semantics : specifies the interpretation of the structures defined by the abstract syntax;
concrete syntax : defines an XML representation
9 Universität des Saarlandes
Representation Structure (XML)
Annotation structure
Defined Semantics
10 Universität des Saarlandes
Dialogue acts
Information state of participants
UPDATE OPERATION
11 Universität des Saarlandes
Dialogue acts
Information state of participants
UPDATE OPERATION
12 Universität des Saarlandes
Dialogue act
Semantic content Communicative Function
Description of objects, properties, actions.
Specification of how the information states should be
updated with semantic content
13 Universität des Saarlandes
Please consider these utterances:
Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.
Apology Answer
14 Universität des Saarlandes
Dialogue
acts
Semantic content
Communicative Function
Communicative Function
Communicative Function
…..
15 Universität des Saarlandes
Semantic content
Dialogue acts
Communicative Function
Communicative Function A Dimension 1
Dimension 2
Dimension 3
Dimension …
9 dimensions are formulated
Communicative Function A
Communicative Function A
Communicative Function A
16 Universität des Saarlandes
Semantic content
Dialogue acts
Communicative Function
Communicative Function A Dimension 1
Dimension 2
Dimension 3
Dimension …
10 dimensions are formulated
Communicative Function A
Communicative Function A
Communicative Function A
17 Universität des Saarlandes
Communicative Functions:
Dimension-specific functions
General purpose functions
Specific for a particular dimension
Turn Release function can be specific for Turn Management dimension
Can be applied for all dimension
18 Universität des Saarlandes
Medamodel for Dialogue Annotation
Minimal unit of dialogue annotation (the raw data)
19 Universität des Saarlandes
Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.
Functional Segment2
Functional Segment3
Apology Answer
D: TASK
Set question
Functional Segment 1
D: SOCAIL OBLIGATION
D: TASK
DA 1 Dialogue act 1
DA 3 DA 2
Semantic Content
Semantic Content Semantic Content
20 Universität des Saarlandes
Medamodel for Dialogue Annotation
Minimal unit of dialogue annotation (the raw data)
21 Universität des Saarlandes
Qualifiers
• Qualifiers are applicable to general purpose functions(GPFs).
Qualifiers Type Communicative functions
Sentiment qualifiers
Additive Q Accept happily
Conditional qualifiers
Specified Q
e.g. Promise, Offer…
Certainty qualifiers
e.g.Answer, Confirmation…
22 Universität des Saarlandes
(un)Certainty Qualifier
Uncertain
certain
23 Universität des Saarlandes
Medamodel for Dialogue Annotation
ENTITY STRUCTURE
DIALOGUE ACT STRUCTURE
24 Universität des Saarlandes
Relations
There are various relations in coherent dialogue: • Functional Dependence Relations • Feedback Dependence Relations • Rhetorical relations
25 Universität des Saarlandes
Dependence Relations 1.Functional Dependences
• Functional dependence relations occur with dialogue acts that are responsive in nature.
• Examples: Answer-Question,
Apology- Accept Apology,
26 Universität des Saarlandes
Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.
Functional Segment2
Functional Segment3
Apology Answer
D: TASK
Set question
Functional Segment 1
D: SOCAIL OBLIGATION
D: TASK
DA 1 Dialogue act 1
DA 3 DA 2
27 Universität des Saarlandes
fs1 (“Is there an earlier connection?”),
fs2 (“No there isn’t”) fs3 (“I’m sorry”)
• fs2 links to fs1 as the answer to the question, semantic content (No, there isn’t) is also needed for the link.
28 Universität des Saarlandes
• Functional dependences is now part of inherent attribute of the a functional segment.
• This justice the intuition that the functional dependence is an inherent part of a responsive dialogue act.
• An Answer cannot exist without a Question
Accept-apology cannot exist without an Apology.
29 Universität des Saarlandes
Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.
Functional Segment2
Functional Segment3
Functional Segment 1
Dialogue act 1
Dialogue Act 3 Dialogue Act 2
30 Universität des Saarlandes
In the past, a separate structure
31 Universität des Saarlandes
Now it is a built-in ATTRIBUTE
32 Universität des Saarlandes
Medamodel for Dialogue Annotation
33 Universität des Saarlandes
Dependence Relations 2.Feedback Dependences
• Feedback dependence relation is determined by semantic content of a previous dialogue act.
34 Universität des Saarlandes
• Speaker B here is checking the correctness of his perception of what A said, which sounded a bit strange
• (A may be hesitating between Tuesday and Thursday or maybe A was non-native speaker who had difficulty pronouncing ‘th’)
35 Universität des Saarlandes
A feedback can be a non-local and distant feedback,
36 Universität des Saarlandes
Functional Segment2
Functional Segment3
Functional Segment 1
Dialogue act 1
Dialogue Act 3 Dialogue Act 2
37 Universität des Saarlandes
38 Universität des Saarlandes
Medamodel for Dialogue Annotation
39 Universität des Saarlandes
Rhetorical Relations
Rhetorical Relations are the parts of a coherent text are connected explicitly or implicitly
In other terms such as
‘coherence relations’, or ‘discourse relations’
40 Universität des Saarlandes
• A: You keep losing them
• A: They easily slip behind the couch
• A: Where would you position your buttons?
• A: I think that has some impact on many things.
A1 Question Act A2 Inform Act Motivation Relation
Event A1 Event A2 Semantic Causal Relation
41 Universität des Saarlandes
Functional Segment2
Functional Segment3
Functional Segment 1
Dialogue act 1
Dialogue Act 3 Dialogue Act 2
42 Universität des Saarlandes
Element->
Attribute->
43 Universität des Saarlandes
Universität des Saarlandes 44
DiAML syntax and semantics
DiAML =Dialogue Act Markup Language
define:
• Abstract syntax:
set-theoretical annotation structure
• Concrete syntax:
XML representation of the structure.
45 Universität des Saarlandes
Abstract Syntax -Annotation Structure
Linguistic information added to the segments of language data, independent of format
46 Universität des Saarlandes
Medamodel for Dialogue Annotation
47 Universität des Saarlandes
Functional Segment 1
Holds the semantic content
Entity
Dialogue Act Structue Capture the function aspect
α
s
48 Universität des Saarlandes
• Entity structures: <s,α> or <s, α, E, δ> • s: functional segment
• α: Dialogue act structure
• E: a set of entity structures (α) that have a
function/ feedback dependence relations • • δ: specifies function or feedback dependence
relation. 49 Universität des Saarlandes
Medamodel for Dialogue Annotation
ENTITY STRUCTURE
DIALOGUE ACT STRUCTURE
50 Universität des Saarlandes
Customer: Is there wifi here?
waitress: No, I’m sorry, there isn’t.
Functional Segment2
Functional Segment3
Apology Answer
D: TASK
Set question
Functional Segment 1
D: SOCAIL OBLIGATION
D: TASK
DA 1 Dialogue act 1
DA 3 DA 2
51 Universität des Saarlandes
Dialogue Act Structure
Senders Addressee(S)
Dimension-specific communicative function
General-purpose function
specify dimension
Qualifier
52 Universität des Saarlandes
Medamodel for Dialogue Annotation
53 Universität des Saarlandes
ENTITY STRUCTURE e =
DIALOGUE ACT STRUCTURE
only captures the functional part of the Dialogue act.
Or
Semantics
Structure Interpretation
Complete Dialogue Act Interpretation
54 Universität des Saarlandes
• The Semantics of a Dialogue Act Structure
• :interpretation of the entity structure
• :semantic content of the segments
• So
55 Universität des Saarlandes
Semantics for a certain class of communicative functions
Functions Update semantics units
F(Answer)=U1∪ U2 ∪ U9 ∪ U7
56 Universität des Saarlandes
Semantic Updates units
Information-transfer functions
57 Universität des Saarlandes
Functional Part
Semantic Part
58 Universität des Saarlandes
An entity structure A non empty set of
entity structures
Rhetorical relations
Entities Link structures
Updates are sequential or unified
59 Universität des Saarlandes
Semantics of Dependence Relations and Qualifiers
60 Universität des Saarlandes
Concrete Syntax -representation in XML
The format in which annotation is rendered. Independent of content
61 Universität des Saarlandes
Conceptual Inventories
Abstract Syntax
Set-theoretical Annotation Structure
62 Universität des Saarlandes
Set-theoretical Annotation Structure
Representation Structure (XML)
63 Universität des Saarlandes
Representation Structure (XML)
Annotation structure
Defined Semantics
64 Universität des Saarlandes
Representation Structure (XML)
Annotation structure
Un-ambiguity Completeness
65 Universität des Saarlandes
Automatic Incremental Annotation
Petukhova and Bunt (2011) report on an incremental, token-based approach to the segmentation and annotation of spoken dialogue, with a focus on the recognition of their communicative functions.
66 Universität des Saarlandes
Local Classifiers
Global Classifiers
Communicative functions
AMI input data
features
Dimensions
Five previous and five following tokens
67 Universität des Saarlandes
ISO 24617-2 annotation scheme can be effectively used for automatic annotation.
F-scores for incremental token-based recognition of communicative functions.
68 Universität des Saarlandes
Conclusion
• The requirement that semantic annotations have a formal semantics was shown to have direct consequences for the design of annotation structures.
• For dialogue act annotation, functional and feedback dependence relations have to be expressed with attributes in the XML elements representing dialogue act structures. This simplifies the representations and make them semantically fully transparent.
• For the ISO standard for dialogue act annotation, this realization came just in time to prevent the proposal of incorrect annotation representations
69 Universität des Saarlandes
Reference
• Bunt, H. (2009a) The DIT++ taxonomy for functional dialogue markup. In D. Heylen, C. Pelachaud, R. Catizone and D. Traum (eds.)
• Bunt, H. (2011). The semantics of dialogue acts.
• Traum, D. & S. Larsson (2003) The Information State Approach
to Dialogue Act Management.
• ISO 24617-2: A semantically-based standard for dialogue annotation Harry Bunt*, Jan Alexandersson, Jae-Woong Choe, Alex Chengyu Fang, Koiti Hasida,Volha Petukhova, Andrei Popescu-Belis and David Traum
70 Universität des Saarlandes
Thank You! 謝謝大家! 감사합니다!
71 Universität des Saarlandes