New work-package WP5:Multimodal Processing and

Interaction

MUSCLE JPA3 Leaders:

Petros Maragos, ICCS-NTUA

Alexandros Potamianos, TSI-TUC

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

Outline of new WP5 Work in JPA3 T1. Book on Multimodal Processing and Interaction T2. Audio-Visual Speech Analysis and Recognition

T2.1 Audio-Visual Feature Extraction and Fusion T2.2: Dynamic Models for AV-ASR, Evaluation T2.3: Audio-Visual to Articulatory Speech Inversion

T3. Multimodal Integration for MM Analysis & Recognition T3.1: Video Analysis & Integration of Asynchronous Time-evolving

Modalities T3.2: Multimodal Saliency T3.3: Integrated Multimedia Content Analysis

T4. Interfaces to Multimedia T4.1: Multimodal Dialogue Interfaces T4.2: Eye-tracking Interfaces for Information Retrieval T4.3: Mobile Interfaces

T5. Coordination of research and Dissemination of results

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

Next steps & Proposed Research Orientations Merge research tasks of JPA1-JPA2/WP6+WP10 JPA3/WP5

Notify partners. The main axes of research in Multimodal Processing, Interaction,

Interfaces are fundamental and still timely (multimodal information fusion for multimedia understanding).

Focus on specific areas Need for even closer collaboration between the WP partners on specific

tasks of common interest (jointly planned research agendas, joint papers, fellowships, joint proposals for new added-value in-depth projects).

Contributions of each partner to deliverables should be more clearly specified to help people focus their research towards specific WP goals and collaborate with other groups.

Further develop Synergies with other WPs. E-teams Book

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

e-Teams: Goals & Objectives E-Team 10: Audio-Visual Speech Analysis & Recognition

AV Feature Extraction and Feature Fusion Dynamical Models for AV-ASR, Evaluation Audio-Visual to Articulatory Speech Inversion.

E-Team 11: Multimodal Processing & Multimedia Understanding Video Analysis and Integration of Asynchronous Time-evolving

Modalities Audio-Visual Attention Modeling and Salient Event Detection Integrated Multimedia Content Analysis

E-Team 12: Multimodal Interfaces Multimodal Recognition and Dialogue Systems Mobile Services Novel Interfaces (Eye-tracking)

MUSCLE Plenary, Dec. 2006, France MUSCLE MUSCLE WP5

BOOK Title: Multimodal Processing and Interaction: Audio, Video, Text Contents:

State-of-Art Reviews of WP6 + WP10 (updated) Contributed Research Chapters: New Work

Agenda: Scope and Thematic Areas discussed during Audio-Conf & Meetings Each interested participant emails preliminary title + abstract Table-of-Contents of selected chapters is discussed with all participants Publisher is contacted

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

Dissemination and Publicity

Necessary to attract better research and collaboration. Website Showcase Demonstrations of prototype systems sampling

MUSCLE research. Organize MUSCLE-related Special Sessions at related

Conferences: e.g.,

ICASSP, EUSIPCO, ICME, ICSL, ICIP, ECCV, ICCV Publish Updated State-of-the-Art report. Present & Publish joint papers at conferences and journals. Book.

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

Multimodal Research and Systems are Central in

MUSCLE Research Challenges & Goals

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

MMSP 2007

IEEE Workshop on Multimedia Signal Processing in 2007 co-organized by A. Potamianos (GC) and P. Maragos (TC)

Where: Chania, Crete

When: Oct 1-3, 2007

Synergy with MUSCLE Multimedia a central research area in MUSCLE

MUSCLE sponsors four student best paper MMSP 2007 awards

Possibility of organizing a MUSCLE special session

(spec. session proposal deadline March 9th)

Discussion of hosting a Plenary MUSCLE meeting in Chania as a satellite workshop to MMSP

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

CFP: Multimedia Interaction and Communication Multimedia Interaction

Modeling of multimodal perception Modeling of multimedia interaction Human-human and human-computer

dialog Multimodal fusion and fission

Multimedia Communication Multimedia peer-to-peer communication Multimedia wireless communication QoS in multimedia communication Cross-layer design for multimedia

communication Supporting Methods and Systems

Single- and multimedia analysis Single- and multimedia compression Single- and multimedia security Single- and multimedia indexing Single- and multimedia search

Emerging Applications Distributed multimedia Entertainment and gaming eHealth and telemedicine

Advanced Implementations Implementations on graphics cards Implementations on multimedia

processors Wearable multimedia

implementations

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

The VenueThe Venue

Panorama HotelPanorama Hotel Location: 10 km from Chania Location: 10 km from Chania

on the beachon the beach Conference rooms with Conference rooms with

capacity up to 800capacity up to 800 Easy access to restaurants Easy access to restaurants

nearby and town of Plataniasnearby and town of Platanias Numerous alternative hotels Numerous alternative hotels

nearbynearby

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

MUSCLE Plenary, Dec. 2006, France MUSCLEMUSCLEWP5

MMSP 2007 Important Dates Special Sessions (contact the respective chair by)

March 9, 2007

Papers (full paper, 4 pages, to be received by)

April 6, 2007

Notification of acceptance by June 8, 2007

Camera-ready paper submission by July 6, 2007

http://www.mmsp2007.org/

Posters and CFP available!Posters and CFP available!