research themes coordinated multi modal generation for virtual humans

Research themes Coordinated Multi Modal Generation for Virtual Humans Coordination of multi modal generation with perception input. Real-time coordinated interaction. Dynamic animation, synchronized with extrapolations of a stream of incoming events/observations. Conversational Agents: Theoretical Work The Virtual Conductor will serve as a context for theoretical investigation of dialog as a continuous perception/production feedback loop in which both interaction parties are at all times 'sending' as well as 'receiving'. Conducting Applications The Virtual Conductor (VC) may be used for several applications. The most relevant at this point are the use of the VC for rehearsals and for tutoring of conducting students . Music Perception Technology The VC presents a novel context to work on applications of music analysis technology. Activities Analyzing Conducting by Humans To implement the right behavior in a VC we need to know how different intentions are expressed by human conductors, and when. Music Perception Technology To conduct effectively, the VC needs to analyze the music in real time using algorithms implemented and/or adapted from literature. Multi Modal Generation Animation technology, adaptive (re)planning, synchronization, etc… Evaluation with Human Musicians Conduct tests with human musicians to evaluate the interaction models and the perception technology. Human Media Interaction P. Bos A. Nijholt D. Reidsma Zs. Ruttkay [email protected] Current State •Animated Virtual Conductor •Conducts a group of human musicians •Using 1-, 2-, 3- and 4-beat patterns. •Conducts volume and tempo (changes). •Tempo interaction: prepares, and leads the musicians through, tempo changes or corrections. •Can rehearse problematic passages with the ensemble The conductor has been evaluated several times with groups of human musicians. The musicians could follow the tempo and dynamic changes of the conductor reasonably well. The conductor could interact successfully with the musicians, correcting their tempo if they played too fast or to slow. •Interacting with a Virtual Conductor, Bos, P., Reidsma, D., Ruttkay, Z.M., and Nijholt, A. (2006) in: Proc. of 5th International Conference on Entertainment Computing, Cambridge, UK. pp 25-30, LNCS 4161 Springer Verlag, DOI 1007/11872320_3 •http://hmi.ewi.utwente.nl/showcases/Human Music Interaction/ The Virtual Conductor

Upload: murphy-stephenson

Post on 30-Dec-2015

27 views

Category:

Documents

0 download

Report

Download

Tags:

Embed Size (px):

DESCRIPTION

The Virtual Conductor. Research themes Coordinated Multi Modal Generation for Virtual Humans Coordination of multi modal generation with perception input. Real-time coordinated interaction. Dynamic animation, synchronized with extrapolations of a stream of incoming events/observations. - PowerPoint PPT Presentation

TRANSCRIPT

Research themes

Coordinated Multi Modal Generation for Virtual HumansCoordination of multi modal generation with perception input. Real-time coordinated interaction. Dynamic animation, synchronized with extrapolations of a stream of incoming events/observations.

Conversational Agents: Theoretical WorkThe Virtual Conductor will serve as a context for theoretical investigation of dialog as a continuous perception/production feedback loop in which both interaction parties are at all times 'sending' as well as 'receiving'.

Conducting ApplicationsThe Virtual Conductor (VC) may be used for several applications. The most relevant at this point are the use of the VC for rehearsals and for tutoring of conducting students.

Music Perception TechnologyThe VC presents a novel context to work on applications of music analysis technology.

Activities

Analyzing Conducting by HumansTo implement the right behavior in a VC we need to know how different intentions are expressed by human conductors, and when.

Music Perception TechnologyTo conduct effectively, the VC needs to analyze the music in real time using algorithms implemented and/or adapted from literature.

Multi Modal Generation Animation technology, adaptive (re)planning, synchronization, etc…

Evaluation with Human MusiciansConduct tests with human musicians to evaluate the interaction models and the perception technology.

Human Media InteractionP. BosA. NijholtD. ReidsmaZs. [email protected]

Current State•Animated Virtual Conductor•Conducts a group of human musicians •Using 1-, 2-, 3- and 4-beat patterns.•Conducts volume and tempo (changes). •Tempo interaction: prepares, and leads the musicians through, tempo changes or corrections.•Can rehearse problematic passages with the ensemble

The conductor has been evaluated several times with groups of human musicians. The musicians could follow the tempo and dynamic changes of the conductor reasonably well. The conductor could interact successfully with the musicians, correcting their tempo if they played too fast or to slow. •Interacting with a Virtual Conductor, Bos, P., Reidsma, D., Ruttkay, Z.M., and Nijholt, A. (2006) in: Proc. of 5th International Conference on Entertainment Computing, Cambridge, UK. pp 25-30, LNCS 4161 Springer Verlag, DOI 1007/11872320_3

•http://hmi.ewi.utwente.nl/showcases/Human Music Interaction/

The Virtual Conductor

Critical Period for Cross-Modal Plasticity in Blind Humans: A Functional MRI Study Norihiro Sadato, Tomohisa Okada, Manabu Honda, & Yoshiharu Yonekura

Coordinated System of Care · Coordinated System of Care • • • 1 . Coordinated System of Care. Quality Improvement Strategy . Overview . The Coordinated System of Care is designed

Live Free / Speak Free. Introduction What are the Modal Verbs Modal Verbs for Ability Modal Verbs for Advice Modal verbs to show obligation or necessity

Modal Analysis of a turbocharger shaft - crittm2a.com · Modal Analysis of a ... Agenda 1. Introduction 2. Shaft reverse engineering 3. Modal simulations 4. Modal testing 5 ... Results

MODAL VERBS & PERFECT MODAL VERBS

THe inTenSifying funcTiOn Of MOdAl PARTicleS And MOdAl

Homeostasis · 2020-05-05 · homeostasis in multi-organ organisms including humans. Based on our original discoveries, we will investigate the mechanisms underlying coordinated regulation

Neighborhood Semantics for Modal Logic An …epacuit/classes/esslli/...July 3, 2007 Neighborhood Semantics for Modal Logic 1.1 Relational Semantics for Modal Logic The basic modal

09 Modal Logic II - NUS Computingcs3234/slides/slides_09.color.pdf · Review of Modal Logic Some Modal Logics Natural Deduction in Modal Logic Knowledge in Multi-Agent Systems 09—Modal

Contextual Inter-modal Attention for Multi-modal Sentiment