Anna Esposito Antonietta M. EspositoAlessandro Vinciarelli Rüdiger HoffmannVincent C. Müller (Eds.)

CognitiveBehavioural SystemsCOST 2102 International Training SchoolDresden, Germany, February 21-26, 2011Revised Selected Papers

13

http://www.sophia.de

Volume Editors

Anna EspositoSeconda Università degli Studi di NapoliIIASS, Napoli, ItalyE-mail: iiass.annaesp@tin.it

Antonietta M. EspositoIstituto Nazionale di Geofisica e VulcanologiaSezione di Napoli Osservatorio Vesuviano, Napoli, ItalyE-mail: aesposito@ov.ingv.it

Alessandro VinciarelliUniversity of Glasgow, School of Computing Science, Glasgow, UKE-mail: alessandro.vinciarelli@glasgow.ac.uk

Rüdiger HoffmannTechnische Universität DresdenInstitut für Akustik und Sprachkommunikation, Dresden, GermanyE-mail: ruediger.hoffmann@ias.et.tu-dresden.de

Vincent C. MüllerAnatolia College/ACTDepartment of Humanities and Social Sciences, Pylaia, GreeceE-mail: vmueller@act.edu

ISSN 0302-9743 e-ISSN 1611-3349ISBN 978-3-642-34583-8 e-ISBN 978-3-642-34584-5DOI 10.1007/978-3-642-34584-5Springer Heidelberg Dordrecht London New York

Library of Congress Control Number: 2012950318

CR Subject Classification (1998): H.1.2, H.5.5, I.2.7, I.2.9-10, H.5.1-3, H.3.1, H.3.4,I.4.8, I.5.4, G.3, J.4, J.5

LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Weband HCI

© Springer-Verlag Berlin Heidelberg 2012This work is subject to copyright. All rights are reserved, whether the whole or part of the material isconcerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting,reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publicationor parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965,in its current version, and permission for use must always be obtained from Springer. Violations are liableto prosecution under the German Copyright Law.The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply,even in the absence of a specific statement, that such names are exempt from the relevant protective lawsand regulations and therefore free for general use.

Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

This book is dedicated to endings and to what is left behind

Preface

This volume brings together the advanced research results obtained by theEuropean COST Action 2102 “Cross Modal Analysis of Verbal and NonverbalCommunication”, primarily discussed at the EUCogII-SSPNet-COST2102 Inter-national Training School on “Cognitive Behavioural Systems”, held in Dresden,Germany, February 21–26 2011 (www.ias.et.tu-dresden.de/ias/cost-2102/).

The school was jointly sponsored by the European Cooperation in Science andTechnology (COST, www.cost.eu) in the domain of Information and Communi-cation Technologies (ICT) for disseminating the advances of the research activ-ities developed within the COST Action 2102: “Cross-Modal Analysis of Verbaland Nonverbal Communication” (cost2102.cs.stir.ac.uk); by the European Net-work of Excellence on Social Signal Processing (SSPNet, www.sspnet.eu) and bythe 2nd European Network for the Advancement of Artificial Cognitive Systems,Interaction and Robotics (EUCogII, www.eucognition.org/).

The main focus of the school was on Cognitive Behavioural Systems. In pre-vious meetings, EUCogII-SSPNet-COST2102 focused on the importance of dataprocessing for gaining enactive knowledge, as well as on the discovery of newprocessing possibilities that account for new data analysis approaches, coordi-nation of the data flow through synchronization and temporal organization andoptimization of the extracted features. The next step will be to discover morenatural and intuitive approaches for modelling and uncovering the wealth ofinformation conveyed by humans during interaction for developing realistic andsocially believable agents. This moves the research focus to cognitive systems andmodels of cognitive processes. It has been shown that cognitive processes – suchas inference, categorization and memory – are not independent of their physicalinstantiations. Individual choices, perception and actions emerge and are dy-namically affected/enhanced by the interaction between sensory-motor systemsand the inhabited environment (including the organizational, cultural and phys-ical context). This interplay carries up instantiations of cognitive behaviouralsystems.

How can these aspects be modelled in order to bring machine intelligenceclose to human expectations? Are existing paradigms sufficient or is more re-search needed on signals and data? How trustful, credible and satisfactory willemotionally-coloured multimodal systems appear to the end user? How will theirphysical instantiation and appearance affect the human-machine interplay?

The papers accepted in this volume were peer reviewed and include originalcontributions from early stage researchers. The volume presents new and orig-inal research results in the field of human-machine interaction inspired by cog-nitive behavioural human-human interaction features. The themes covered arecognitive and computational social information processing, emotional and so-cially believable Human-Computer Interaction (HCI) systems, behavioural and

VIII Preface

contextual analysis of interactions, embodiment, perception, linguistics, seman-tics and sentiment analysis in dialogues and interactions, and algorithmic andcomputational issues for the automatic recognition and synthesis of emotionalstates.

The contents have been divided into two scientific sections according to arough thematic classification. The first section, “Computational Issues in Cog-nitive Systems”, deals with models, algorithms, and heuristic strategies for therecognition and synthesis of behavioural data. The second section, “BehaviouralIssues in Cognitive Systems”, presents original studies that provide theoreticaland behavioural analyses on linguistic and paralinguistic expressions, actions,body movements and activities in human interaction.

The papers included in this book benefited from the lively interactions be-tween the many participants of the successful meeting in Dresden. Over 100senior and junior researchers gathered for the event.

The editors would like to thank the Management Board of the SSPNet andthe ESF COST- ICT Programme for their support in the realization of theschool and the publication of this volume. Acknowledgements go in particular tothe COST Science Officers, Giseppe Lugano, Matteo Razzanelli, and AranzazuSanchez, and the COST 2102 rapporteur, Guntar Balodis, for their constanthelp, guidance and encouragement. The event owes its success to more individu-als than can be named, but notably the members of the Dresden Local SteeringCommittee, who actively operated for the success of the event. Special appreci-ation goes to the President of the International Institute for Advanced ScientificStudies (IIASS), and to the Dean and the Director of the Faculty and the De-partment of Psychology at the Second University of Naples for making availablepeople and resources for editing this volume. The editors are deeply indebtedto the contributors for making this book a scientifically stimulating compilationof new and original ideas and to the members of the COST 2102 InternationalScientific Committee for their rigorous and invaluable scientific revisions, dedi-cation, and priceless selection process.

July 2012 Anna EspositoAntonietta M. Esposito

Alessandro VinciarelliRudiger HoffmannVincent C. Muller

Organization

International Steering Committee

Anna Esposito Second University of Naples and IIASS, ItalyMarcos Faundez-Zanuy University of Mataro, Barcelona, SpainRudiger Hoffmann Technische Universtat Dresden (TUD),

GermanyAmir Hussain University of Stirling, UKVincent Muller Anatolia College, Pylaia, GreeceAlessandro Vinciarelli University of Glasgow, UK

Local Steering Committee

Lutz-Michael Alisch TUD, Theory of Educational Sciences andResearch Methods

Rainer Groh TUD, Media DesignRudiger Hoffmann TUD, System Theory and Speech TechnologyKlaus Kabitzsch TUD, Technical Information Management

SystemsKlaus Meißner TUD, Multimedia TechnologyBoris Velichovsky TUD, Engineering Psychology and Cognitive

ErgonomicsGerhard Weber TUD, Human-Computer Interaction

COST 2102 International Scientific Committee

Alberto Abad INESC-ID Lisboa, PortugalSamer Al Moubayed Royal Institute of Technology, SwedenUwe Altmann Friedrich-Schiller-University Jena, GermanySigrun Marıa Ammendrup School of Computer Science, Reykjavik, IcelandHicham Atassi Brno University of Technology, Czech RepublicNikos Avouris University of Patras, GreeceMartin Bachwerk Trinity College, Dublin, IrelandIvana Baldasarre Second University of Naples, ItalySandra Baldassarri Zaragoza University, SpainRuth Bahr University of South Florida, USAGerard Bailly GIPSA-lab, Grenoble, FranceMarena Balinova University of Applied Sciences, Vienna, AustriaMarian Bartlett University of California, San Diego, USA

X Organization

Dominik Bauer RWTH Aachen University, GermanySieghard Beller Universitat Freiburg, GermanyStefan Benus Constantine the Philosopher University, Nitra,

Slovak RepublicNiels Ole Bernsen University of Southern Denmark, DenmarkJonas Beskow Royal Institute of Technology, SwedenPeter Birkholz RWTH Aachen University, GermanyHorst Bishof Technical University Graz, AustriaJean-Francois Bonastre Universite d’Avignon, FranceMarek Bohac Technical University of Liberec, Czech RepublicElif Bozkurt Koc University, Istanbul, TurkeyNikolaos Bourbakis ITRI, Wright State University, Dayton, USAMaja Bratanic University of Zagreb, CroatiaAntonio Calabrese Istituto di Cibernetica – CNR, Naples, ItalyErik Cambria University of Stirling, UKPaola Campadelli Universita di Milano, ItalyNick Campbell University of Dublin, IrelandValentın Cardenoso Payo Universidad de Valladolid, SpainNicoletta Caramelli Universita di Bologna, ItalyAntonio Castro-Fonseca Universidade de Coimbra, PortugalAleksandra Cerekovic Faculty of Electrical Engineering , CroatiaPeter Cerva Technical University of Liberec, Czech RepublicJosef Chaloupka Technical University of Liberec, Czech RepublicMohamed Chetouani Universite Pierre et Marie Curie, FranceGerard Chollet CNRS URA-820, ENST, FranceSimone Cifani Universita Politecnica delle Marche, ItalyMuzeyyen Ciyiltepe Gulhane Askeri Tip Academisi, Ankara, TurkeyAnton Cizmar Technical University of Kosice, SlovakiaDavid Cohen Universite Pierre et Marie Curie, Paris, FranceNicholas Costen Manchester Metropolitan University, UKFrancesca D’Olimpio Second University of Naples, ItalyVlado Delic University of Novi Sad, SerbiaCeline De Looze Trinity College, Dublin 2, IrelandFrancesca D’Errico Universita di Roma 3, ItalyAngiola Di Conza Second University of Naples, ItalyGiuseppe Di Maio Second University of Naples, ItalyMarion Dohen ICP, Grenoble, FranceThierry Dutoit Faculte Polytechnique de Mons, BelgiumLaila Dybkjær University of Southern Denmark, DenmarkJens Edlund Royal Institute of Technology, SwedenMatthias Eichner Technische Universitat Dresden, Germany

Organization XI

Aly El-Bahrawy Ain Shams University, Cairo, EgyptCigdem Eroglu Erdem Bahcesehir University, Istanbul, TurkeyEngin Erzin Koc University, Istanbul, TurkeyAnna Esposito Second University of Naples, ItalyAntonietta M. Esposito Osservatorio Vesuviano Napoli, ItalyJoan Fabregas Peinado Escola Universitaria de Mataro, SpainSascha Fagel Technische Universitat Berlin, GermanyNikos Fakotakis University of Patras, GreeceManuela Farinosi University of Udine, ItalyMarcos Faundez-Zanuy Universidad Politecnica de Cataluna, SpainTibor Fegyo Budapest University of Technology and

Economics, HungaryFabrizio Ferrara University of Naples “Federico II”, ItalyDilek Fidan Ankara University, TurkeyLeopoldina Fortunati Universita di Udine, ItalyTodor Ganchev University of Patras, GreeceCarmen Garcıa-Mateo University of Vigo, SpainVittorio Girotto Universita IUAV di Venezia, ItalyAugusto Gnisci Second University of Naples, ItalyMilan Gnjatovic University of Novi Sad, SerbiaBjorn Granstrom Royal Institute of Technology, SwedenMarco Grassi Universita Politecnica delle Marche, ItalyMaurice Grinberg New Bulgarian University, BulgariaJorge Gurlekian LIS CONICET, Buenos Aires, ArgentinaMohand-Said Hacid Universite Claude Bernard Lyon 1, FranceJaakko Hakulinen University of Tampere, FinlandIoannis Hatzilygeroudis University of Patras, GreeceImmaculada Hernaez University of the Basque Country, SpainJavier Hernando Technical University of Catalonia, SpainWolfgang Hess Universitat Bonn, GermanyDirk Heylen University of Twente, The NetherlandsDaniel Hladek Technical University of Kosice, Slovak RepublicRudiger Hoffmann Technische Universitat Dresden, GermanyHendri Hondorp University of Twente, The NetherlandsDavid House Royal Institute of Technology, SwedenEvgenia Hristova New Bulgarian University, Sofia, BulgariaStephan Hubler Dresden University of Technology, GremanyIsabelle Hupont Aragon Institute of Technology, Zaragoza,

SpainAmir Hussain University of Stirling, UKViktor Imre Budapest University of Technology and

Economics, Hungary

XII Organization

Ewa Jarmolowicz Adam Mickiewicz University, Poznan, PolandKristiina Jokinen University of Helsinki, FinlandJozef Juhar Technical University Kosice, Slovak RepublicZdravko Kacic University of Maribor, SloveniaBridget Kane Trinity College Dublin, IrelandJim Kannampuzha RWTH Aachen University, GermanyMaciej Karpinski Adam Mickiewicz University, Poznan, PolandEric Keller Universite de Lausanne, SwitzerlandAdam Kendon University of Pennsylvania, USAStefan Kopp University of Bielefeld, GermanyJacques Koreman University of Science and Technology, NorwayTheodoros Kostoulas University of Patras, GreeceMaria Koutsombogera Inst. for Language and Speech Processing,

GreeceRobert Krauss Columbia University, New York, USABernd Kroger RWTH Aachen University, GermanyGernot Kubin Graz University of Technology, AustriaOlga Kulyk University of Twente, The NetherlandsAlida Labella Second University of Naples, ItalyEmilian Lalev New Bulgarian University, BulgariaYiannis Laouris Cyprus Neuroscience and Technology Institute,

CyprusAnne-Maria Laukkanen University of Tampere, FinlandAmelie Lelong GIPSA-lab, Grenoble, FranceBorge Lindberg Aalborg University, DenmarkSaturnino Luz Trinity College Dublin, IrelandWojciech Majewski Wroclaw University of Technology, PolandPantelis Makris Neuroscience and Technology Institute, CyprusKenneth Manktelow University of Wolverhampton, UKRaffaele Martone Second University of Naples, ItalyRytis Maskeliunas Kaunas University of Technology, LithuaniaDominic Massaro University of California, Santa Cruz, USAOlimpia Matarazzo Second University of Naples, ItalyChristoph Mayer Technische Universitat Munchen, GermanyDavid McNeill University of Chicago, USAJirı Mekyska Brno University of Technology, Czech RepublicNicola Melone Second University of Naples, ItalyKatya Mihaylova University of National and World Economy,

Sofia, BulgariaPeter Mihajlik Budapest University of Technology and

Economics, HungaryMichal Mirilovic Technical University of Kosice, Slovak Republic

Organization XIII

Izidor Mlakar Roboti c.s. d.o.o, Maribor, SloveniaHelena Moniz INESC-ID, Lisboa, Portugal,Tamas Mozsolics Budapest University of Technology and

Economics, HungaryVincent C. Muller Anatolia College/ACT, Pylaia, GreecePeter Murphy University of Limerick, Limerick, IrelandAntonio Natale University of Salerno and IIASS, ItalyCostanza Navarretta University of Copenhagen, DenmarkEva Navas Escuela Superior de Ingenieros, Bilbao, SpainDelroy Nelson University College London, UKGeza Nemeth University of Technology and Economics,

Budapest, HungaryFriedrich Neubarth Austrian Research Inst. Artificial Intelligence,

AustriaChristiane Neuschaefer-Rube RWTH Aachen University, GermanyGiovanna Nigro Second University of Naples, ItalyAnton Nijholt Universiteit Twente, The NetherlandsJan Nouza Technical University of Liberec, Czech RepublicMichele Nucci Universita Politecnica delle Marche, ItalyCatharine Oertel Trinity College Dublin, IrelandStanislav Ondas Technical University of Kosice, Slovak RepublicRieks Op den Akker University of Twente , The NetherlandsKarel Palecek Technical University of Liberec, Czech RepublicIgor Pandzic Faculty of Electrical Engineering, CroatiaHarris Papageorgiou Institute for Language and Speech Processing,

GreeceKinga Papay University of Debrecen, HungaryPaolo Parmeggiani Universita degli Studi di Udine, ItalyAna Pavia Spoken Language Systems Laboratory, Lisbon,

PortugalPaolo Pedone Second University of Naples, ItalyTomislav Pejsa University of Zagreb, CroatiaCatherine Pelachaud CNRS, Telecom ParisTech, FranceBojan Petek University of Ljubljana, SloveniaHarmut R. Pfitzinger University of Munich, GermanyFrancesco Piazza Universita degli Studi di Ancona, ItalyNeda Pintaric University of Zagreb, CroatiaMatus Pleva Technical University of Kosice, Slovak RepublicIsabella Poggi Universita di Roma 3, ItalyGuy Politzer Universite de Paris VIII, FranceJan Prazak Technical University of Liberec, Czech RepublicKen Prepin Telecom ParisTech, FranceJirı Pribil Academy of Sciences, Czech RepublicAnna Pribilova Slovak University of Technology,

Slovak Republic

XIV Organization

Emanuele Principi Universita Politecnica delle Marche, ItalyMichael Pucher Telecommunications Research Center Vienna,

AustriaJurate Puniene Kaunas University of Technology, LithuaniaAna Cristina Quelhas Instituto Superior de Psicologia Aplicada,

Lisbon, PortugalKari-Jouko Raiha University of Tampere, FinlandRoxanne Raine University of Twente, The NetherlandsGiuliana Ramella Istituto di Cibernetica – CNR, Naples, ItalyFabian Ramseyer University Hospital of Psychiatry, Bern,

SwitzerlandJose Rebelo Universidade de Coimbra, PortugalPeter Reichl FTW Telecommunications Research Center,

AustriaLuigi Maria Ricciardi Universita di Napoli “Federico II”, ItalyMaria Teresa Riviello Second University of Naples and IIASS, ItalyMatej Rojc University of Maribor, SloveniaNicla Rossini Universita del Piemonte Orientale, ItalyRudi Rotili Universita Politecnica delle Marche, ItalyAlgimantas Rudzionis Kaunas University of Technology, LithuaniaVytautas Rudzionis Kaunas University of Technology, LithuaniaHugo L. Rufiner Universidad Nacional de Entre Rıos, ArgentinaMilan Rusko Slovak Academy of Sciences, Slovak RepublicZsofia Ruttkay Pazmany Peter Catholic University, HungaryYoshinori Sagisaka Waseda University, Tokyo, JapanBartolomeo Sapio Fondazione Ugo Bordoni, Rome, ItalyMauro Sarrica University of Padua, ItalyGellert Sarosi Budapest University of Technology and

Economics, HungaryGaetano Scarpetta University of Salerno and IIASS, ItalySilvia Scarpetta Salerno University, ItalyStefan Scherer Ulm University, GermanyRalph Schnitker Aachen University, GermanyJean Schoentgen Universite Libre de Bruxelles, BelgiumBjorn Schuller Technische Universitat Munchen, GermanyMilan Secujski University of Novi Sad, SerbiaStefanie Shattuck-Hufnagel MIT, Research Laboratory of Electronics, USAMarcin Skowron Austrian Research Inst. for Art. Intelligence,

AustriaJan Silovsky Technical University of Liberec, Czech RepublicZdenek Smekal Brno University of Technology, Czech RepublicStefano Squartini Universita Politecnica delle Marche, ItalyPiotr Staroniewicz Wroclaw University of Technology, PolandJan Stas Technical University of Kosice, SlovakiaVojtech Stejskal Brno University of Technology, Czech Republic

Organization XV

Marian Stewart-Bartlett University of California, San Diego, USAXiaofan Sun University of Twente, The NetherlandsJing Su Trinity College Dublin, IrelandDavid Sztaho Budapest University of Technology and

Economics, HungaryJianhua Tao Chinese Academy of Sciences, P. R. ChinaBalazs Tarjan Budapest University of Technology and

Economics, HungaryJure F. Tasic University of Ljubljana, SloveniaMurat Tekalp Koc University, Istanbul, TurkeyKristinn Thorisson Reykjavık University, IcelandIsabel Trancoso Spoken Language Systems Laboratory,

PortugalLuigi Trojano Second University of Naples, ItalyWolfgang Tschacher University of Bern, SwitzerlandMarkku Turunen University of Tampere, FinlandHenk Van den Heuvel Radboud University Nijmegen,

The NetherlandsBetsy van Dijk University of Twente, The NetherlandsGiovanni Vecchiato Universita “La Sapienza”, Roma, ItalyLeticia Vicente-Rasoamalala Alchi Prefectural Univesity, JapanRobert Vich Academy of Sciences, Czech RepublicKlara Vicsi Budapest University, HungaryHannes Hogni Vilhjalmsson Reykjavık University, IcelandJane Vincent University of Surrey, Guilford, UKAlessandro Vinciarelli University of Glasgow, UKLaura Vincze Universita di Roma 3, ItalyCarl Vogel Trinity College, Dublin, IrelandJan Volın Charles University, Czech RepublicRosa Volpe Universite de Perpignan, FranceMartin Vondra Academy of Sciences, Czech RepublicPascal Wagner-Egger Fribourg University, SwitzerlandYorick Wilks University of Sheffield, UKMatthias Wimmer Institute for Informatics, Munich, GermanyMatthias Wolf Technische Universitat Dresden, GermanyBencie Woll University College London, UKBayya Yegnanarayana International Institute of Information

Technology, IndiaVanda Lucia Zammuner University of Padua, ItalyJerneja Zganec Gros Alpineon, Development and Research, SloveniaGoranka Zoric Faculty of Electrical Engineering, Croatia

XVI Organization

Sponsors

The following organizations sponsored and supported the International Confer-ence

! European COST Action 2102 “Cross-Modal Analysis of Verbal andNonverbal Communication” (cost2102.cs.stir.ac.uk)

ESF Provide the COST Office through and EC contract

COST is supported by the EU RTD Framework programme

COST- the acronym for European Cooperation in Science and Technology-is the oldest and widest European intergovernmental network for cooper-ation in research. Established by the Ministerial Conference in November1971, COST is presently used by the scientific communities of 36 Europeancountries to cooperate in common research projects supported by nationalfunds.

The funds provided by COST - less than 1% of the total value of theprojects - support the COST cooperation networks (COST Actions) throughwhich, with EUR 30 million per year, more than 30 000 European scientistsare involved in research having a total value which exceeds EUR 2 billion peryear. This is the financial worth of the European added value which COSTachieves.

A “bottom up approach” (the initiative of launching a COST Actioncomes from the European scientists themselves), “a la carte participation”(only countries interested in the Action participate), “equality of access”(participation is open also to the scientific communities of countries notbelonging to the European Union) and “flexible structure” (easy implemen-tation and light management of the research initiatives) are the main char-acteristics of COST.

As precursor of advanced multidisciplinary research COST has a veryimportant role for the realisation of the European Research Area (ERA) an-ticipating and complementing the activities of the Framework Programmes,constituting a “bridge” towards the scientific communities of emerging coun-tries, increasing the mobility of researchers across Europe and fostering theestablishment of “Networks of Excellence” in many key scientific domainssuch as: Biomedicine and Molecular Biosciences; Food and Agriculture;Forests, their Products and Services; Materials, Physical and Nanosciences;Chemistry and Molecular Sciences and Technologies; Earth System Science

Organization XVII

and Environmental Management; Information and Communication Tech-nologies; Transport and Urban Development; Individuals, Societies Culturesand Health. It covers basic and more applied research and also addressesissues of pre-normative nature or of societal importance.Web: http://www.cost.eu

! SSPnet: European Network on Social Signal Processing, http://sspnet.eu/

The ability to understand and manage social signals of a person we arecommunicating with is the core of social intelligence. Social intelligence isa facet of human intelligence that has been argued to be indispensable andperhaps the most important for success in life. Although each one of usunderstands the importance of social signals in everyday life situations, andin spite of recent advances in machine analysis and synthesis of relevantbehavioral cues like blinks, smiles, crossed arms, head nods, laughter, etc.,the research efforts in machine analysis and synthesis of human social signalslike empathy, politeness, and (dis)agreement, are few and tentative. Themain reasons for this are the absence of a research agenda and the lack ofsuitable resources for experimentation.

The mission of the SSPNet is to create a sufficient momentum by integrat-ing an existing large amount of knowledge and available resources in SocialSignal Processing (SSP) research domains including cognitive modeling, ma-chine understanding, and synthesizing social behavior, and so:

• Enable the creation of the European and world research agenda in SSP;• Provide efficient and effective access to SSP-relevant tools and data

repositories to the research community within and beyond the SSPNet,and

• Further develop complementary and multidisciplinary expertise neces-sary for pushing forward the cutting edge of the research in SSP.

The collective SSPNet research effort is directed towards integration of ex-isting SSP theories and technologies, and towards identification and explo-ration of potentials and limitations in SSP. More specifically, the frameworkof the SSPNet will revolve around two research foci selected for their primacyand significance: Human-Human Interaction (HHI) and Human-ComputerInteraction (HCI). A particular scientific challenge that binds the SSPNetpartners is the synergetic combination of human-human interaction mod-els, and automated tools for human behavior sensing and synthesis, withinsocially-adept multimodal interfaces.

XVIII Organization

! EUCogII: 2nd European Network for the Advancement of Artifi-cial Cognitive Systems, Interaction and Robotics(http://www.eucognition.org/

! School of Computing Science, University of Glaskow, Scotland, UK! Department of Psychology, Second University of Naples, Caserta,

Italy! Technische Universitat Dresden, Institut fur Akustik und

Sprachkommunikation, Dresden, Germany! International Institute for Advanced Scientific Studies

“E.R. Caianiello” IIASS, www.iiassvietri.it/! Societa Italiana Reti Neuroniche, SIREN,

www.associazionesiren.org/! Regione Campania and Provincia di Salerno, Italy

Table of Contents

Computational Issues in Cognitive Systems

An Approach to Intelligent Signal Processing . . . . . . . . . . . . . . . . . . . . . . . . 1Matthias Wolff and Rudiger Hoffmann

The Analysis of Eye Movements in the Context of Cognitive TechnicalSystems: Three Critical Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Sebastian Pannasch, Jens R. Helmert, Romy Muller, andBoris M. Velichkovsky

Ten Recent Trends in Computational Paralinguistics . . . . . . . . . . . . . . . . . 35Bjorn Schuller and Felix Weninger

Conversational Speech Recognition in Non-stationary ReverberatedEnvironments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Rudy Rotili, Emanuele Principi, Martin Wollmer,Stefano Squartini, and Bjorn Schuller

From Nonverbal Cues to Perception: Personality and SocialAttractiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Alessandro Vinciarelli, Hugues Salamin, Anna Polychroniou,Gelareh Mohammadi, and Antonio Origlia

Measuring Synchrony in Dialog Transcripts . . . . . . . . . . . . . . . . . . . . . . . . . 73Carl Vogel and Lydia Behan

A Companion Technology for Cognitive Technical Systems . . . . . . . . . . . . 89Andreas Wendemuth and Susanne Biundo

Semantic Dialogue Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104Gunther Wirsching, Markus Huber, Christian Kolbl,Robert Lorenz, and Ronald Romer

Furhat: A Back-Projected Human-Like Robot Head for MultipartyHuman-Machine Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Samer Al Moubayed, Jonas Beskow, Gabriel Skantze, andBjorn Granstrom

VISION as a Support to Cognitive Behavioural Systems . . . . . . . . . . . . . . 131Luca Berardinelli, Dajana Cassioli, Antinisca Di Marco,Anna Esposito, Maria Teresa Riviello, and Catia Trubiani

XX Table of Contents

The Hourglass of Emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144Erik Cambria, Andrew Livingstone, and Amir Hussain

A Naturalistic Database of Thermal Emotional Facial Expressions andEffects of Induced Emotions on Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Anna Esposito, Vincenzo Capuano, Jiri Mekyska, andMarcos Faundez-Zanuy

Prosody Modelling for TTS Systems Using Statistical Methods . . . . . . . . 174Zdenek Chaloupka and Petr Horak

Modeling the Effect of Motion at Encoding and Retrieval for Same andOther Race Face Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Hui Fang, Nicholas Costen, Natalie Butcher, and Karen Lander

An Audiovisual Feedback System for Pronunciation Tutoring –Mandarin Chinese Learners of German . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Hongwei Ding, Oliver Jokisch, and Rudiger Hoffmann

Si.Co.D.: A Computer Manual for Coding Questions . . . . . . . . . . . . . . . . . . 198Augusto Gnisci, Enza Graziano, and Angiola Di Conza

Rule-Based Morphological Tagger for an Inflectional Language . . . . . . . . . 208Daniel Hladek, Jan Stas, and Jozef Juhar

Czech Emotional Prosody in the Mirror of Speech Synthesis . . . . . . . . . . . 216Jana Vlckova-Mejvaldova and Petr Horak

Pre-attention Cues for Person Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225Karel Palecek, David Geronimo, and Frederic Lerasle

Comparison of Complementary Spectral Features of Emotional Speechfor German, Czech, and Slovak . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

Jirı Pribil and Anna Pribilova

Form-Oriented Annotation for Building a Functionally IndependentDictionary of Synthetic Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Izidor Mlakar, Zdravko Kacic, and Matej Rojc

A Cortical Approach Based on Cascaded Bidirectional Hidden MarkovModels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

Ronald Romer

Modeling Users’ Mood State to Improve Human-Machine-Interaction . . . 273Ingo Siegert, R. Bock, and Andreas Wendemuth

Pitch Synchronous Transform Warping in Voice Conversion . . . . . . . . . . . 280Robert Vıch and Martin Vondra

Table of Contents XXI

ATMap: Annotated Tactile Maps for the Visually Impaired . . . . . . . . . . . 290Limin Zeng and Gerhard Weber

Behavioural Issues in Cognitive Systems

From Embodied and Extended Mind to No Mind . . . . . . . . . . . . . . . . . . . . 299Vincent C. Muller

Effects of Experience, Training and Expertise on MultisensoryPerception: Investigating the Link between Brain and Behavior . . . . . . . . 304

Scott A. Love, Frank E. Pollick, and Karin Petrini

Nonverbal Communication – Signals, Conventions and IncommensurableExplanations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

Lutz-Michael Alisch

A Conversation Analytical Study on Multimodal Turn-Giving Cues:End-of-Turn Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

Agnes Abuczki

Conversational Involvement and Synchronous Nonverbal Behaviour . . . . . 343Uwe Altmann, Catharine Oertel, and Nick Campbell

First Impression in Mark Evaluation: Predictive Ability of theSC-IAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

Angiola Di Conza and Augusto Gnisci

Motivated Learning in Computational Models of Consciousness . . . . . . . . 365James Graham and Daniel Jachyra

Are Pointing Gestures Induced by Communicative Intention? . . . . . . . . . . 377Ewa Jarmolowicz-Nowikow

TV Interview Participant Profiles from a Multimodal Perspective . . . . . . 390Maria Koutsombogera and Harris Papageorgiou

The Neurophonetic Model of Speech Processing ACT: Structure,Knowledge Acquisition, and Function Modes . . . . . . . . . . . . . . . . . . . . . . . . 398

Bernd J. Kroger, Jim Kannampuzha, Cornelia Eckers,Stefan Heim, Emily Kaufmann, and Christiane Neuschaefer-Rube

Coding Hand Gestures: A Reliable Taxonomy and a Multi-mediaSupport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

Fridanna Maricchiolo, Augusto Gnisci, and Marino Bonaiuto

Individuality in Communicative Bodily Behaviours . . . . . . . . . . . . . . . . . . . 417Costanza Navarretta

XXII Table of Contents

A Cross-Cultural Study on the Perception of Emotions: How HungarianSubjects Evaluate American and Italian Emotional Expressions . . . . . . . . 424

Maria Teresa Riviello, Anna Esposito, and Klara Vicsi

Affective Computing: A Reverence for a Century of Research . . . . . . . . . . 434Egon L. van den Broek

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449