arne jönsson 1 multimodal research in linköping arne jönsson nlplab/hcs/ida
Post on 20-Dec-2015
216 views
TRANSCRIPT
Arne Jönsson 2
NLPLAB
Upper row:Pernilla,Sonia,Lena,Annika,Micke,Magnus
Bottom row:Håkan,Lars A,Robert,Mustapha,Arne,Lise,Anders,Lars D
Missing:Nils, Bertil, Pontus, Gen
Arne Jönsson 3
Current research
• (Multimodal) information access- Natural language interaction- Text and document analysis
• Usability of multimodal interfaces
• Open-Source and development methods
Arne Jönsson 4
Dialogue system
Dialoguemanager
Generator
Interpreter
StructuredInformation
Dialogue system
Arne Jönsson 5
Ex TV programme guide
• Search for TV programs
• Mediaterminal
• Structured database
• Spoken input
• Developed from framework and method for iterative development
• With Nokia Home Communication
Arne Jönsson 11
Text and document analysis
Language analysis
StructuredInformation
Unstructured documents
Q&A
MT
Arne Jönsson 12
Ex. AskRSV
• Information extraction from text document
• Simple Q&A-system
• With Ida Infront
Arne Jönsson 14
Domain ontology in ASK-RSV
• Taxonomy of a limited set of objects, events, properties and relations in the tax domain
• Only IS-A and synonymy relations
OBJECT
THING
ACTOREVENT
PAPER DATE TAXATION TAX EVENT ORG.PEOPLE
TAX-RETURN ASSET DEBT INCOME TRANSACTIONCOMP. PUBL
Arne Jönsson 18
Combined system
StructuredInformation
Unstructured documents
Language analysis
Dialoguesystem
Lexicon Grammar Ontology
Arne Jönsson 19
Shared knowledge sources
• Ontology, lexicon and grammar- Different demands from IE and DS
• Design
• Development- Corpus analysis, user - Background system, database
• Automatic update
Arne Jönsson 20
BirdQuest
• Bird encyclopaedia- Tagged, templates
• Corpus with user questions- Analysed
• Based on dialogue systems framework
Arne Jönsson 26
Ötraf
• Time tables from local bus company
• Multimodal system- Dialogue- Handling of spatial and temporal
expressions
Arne Jönsson 28
Studies of multimodal interaction
• Wizard of Oz-simulation • Feedback study
- User’s experience- Interviews, questionnaires- Various dimensions: control, cooperation, ease of use,
efficiency, anxiety, hability etc
- Three conditions- No spoken feedback- Limited spoken feedback- Full spoken feedback
- 30 subjects
Arne Jönsson 29
Results
• All or nothing• More like a human in the full feedback
condition- Before the study subjects wanted to have a
timetable for the task, after they preferred a human
• No difference in experienced control and cooperation
• The speech synthesis better in full feedback mode- Same voice
Arne Jönsson 30
Open source
• Share research results between research groups- Stimulate exchange and co-operation
• Language technology readily available for industry- Increased use and insights- The companies do their own prototype
development• Robust code• nlpFarm
http://sourceforge.net/projects/nlpfarm/