laurence devillers & jean-claude martin limsi-cnrs fp6 ist humaine network of excellence /...
TRANSCRIPT
Laurence DEVILLERS & Jean-Claude MARTIN
LIMSI-CNRS
FP6 IST HUMAINE Network of Excellence / Association (http://emotion-research.net)W3C Emotion Incubator Group
.
Definition (Scherer 2000) An hypothetical construct
denoting a process of an organism’s reaction to significant events
Cognitive theories Focus on emotion as an
cognitive evaluation of events
Sequential evaluation of appraisal variables
Event
Coping potential?Which kind of
ressources he has to deal with the situation?
Compatibility with external and internal standards?Is the event
compatible with norms?
Conduciveness to goals?
Is this situation conductiveness goal ?
Global pleasantness?Is this situation
pleasant or not ?
Novelty?Is the event new ?
emotion
Example of a clip from French TV news (EMOTV): someone saying that « I thank you for saying that I was innocent » => a mixture of negative and positive emotions
Naturalistic clips often feature several events eliciting the emotional behavior:
Examples of events:
-event #1: the person that is videotaped has been accused although she is innocent - event #2: she was recognized innocent - event #3: she is interviewed
In this talk, « emotion » stands for « affective state » :emotions, moods, attitudes, …
Gap between emotions observed in artificial data (acted) and those observed with « real-life spontaneaous data in SITU » (Batliner, 2000)
Previous works (Ekman, Scherer, Campbell, Cowie, Devillers, Martin) have shown that there are many complex emotion mixtures in real-life audio or audio-visual data
Difference is mainly due to the context eliciting the emotion: the events that are at the origin of the emotions of a person
Existing work Scherer ‘appraisal variables’ are generally
studied in emotion recall experiments and have been used for predicting emotions from events.Most studies consider acted data collected in-lab
Our research questions How to annotate events in real life data? Can we apply the appraisal model to perception
of more general affective states from real life data?
This study explores how to code emotional events
Are we able to code the events that triggered spontaneous expressions of real-life affective states in audiovisual data?
What are the relevant dimensions of events? Are we able to infer them from short video
clips?
We already used appraisal dimensions in emotion perception investigations (Devillers et al., 2006): Method: subjects annotate their perception of the videotaped
person’s appraisal (28 clips (Belfast Naturalistic Data + French EmoTV), 5 coders, 16 appraisals dimensions)
Results: Several dimensions can be reliably annotated: conduciveness to goals, pleasantness , relation to expectations , controllability-conseqevt
Main difficulty: relations between simulatenous appraisals of multiple events and complex emotions mixtures
Complex Emotion mixtures
Event 1
Controllability consequence
events?Which kind of
ressources he has to deal with the
situation?
Compatibility with
external and
internal beliefs?
Is the event compatible
with beliefs?
Conductiveness goals?
Is this situation conductiveness
goal ?
Global pleasantn
ess?Is this
situation pkeasant or
not ?Novelty?Is the event
new ?
Event 2
Controllability consequence
events?Which kind of
ressources he has to deal with the
situation?
Compatibility with
external and
internal beliefs?
Is the event compatible
with beliefs?
Conductiveness goals?
Is this situation conductiveness
goal ?
Global pleasantne
ss?Is this
situation pkeasant or
not ?Novelty?Is the event
new ?
We use two corpora collected at LIMSI-CNRS: (www.emotion-research.net)
EmoTV : « provocative corpora » (Abrilian et al. , 2005) Naturalistic corpus extracted from FrenchTV
news 100 clips of few minutes (< 1 hour)
EmoTaboo (zara et al., 2007) Record of a two-players game designed in
order to induce emotions 10 clips of around 50 minutes each (8 hours)
Definition of an event An event is perceived in the video clips as
triggering the affective state Majority voting between 3 annotators
28 clips from EmoTV Three steps
1.Identify and annotate the emotional events
2.Annotate the temporal dimensions of event
3.Annotate appraisal dimensions of events
Three groups of emotional events are defined in the OCC model (Ortony, Clore, & Collins, 1988) depending on what is evaluated:
- the consequences of events for oneself or for others,- the actions of others - and the perception of objects.
In our corpus, we only observed events that are being evaluated with respect to their consequence for oneself or others and actions of others.
Our events scheme permits to annotate up to 3 emotional events for a given clip. Each event is annotated according to the temporal dimensions
Affective state labels Courage Disappointment
Emotional events1. Election campaign
▪ Temporality: past▪ Duration : months and
more
2. Election results ▪ Temporality: present(D-
day)▪ Duration : minutes
Emotional labels Serenity Pride
Emotional events1. Football cup
▪ Temporality: past▪ Duration : months and
more
2. Football match ▪ Temporality: close future▪ Duration : hours
Emotional labels Anger Despair Disappointment Disgust Helplessness Worry
Emotional events1. Lawsuit (current) :
▪ Temporality: past▪ Duration : months and
more2. Someone is accusing other
people▪ Temporality: present▪ Duration : hour
3. The trial has just finished▪ Temporality: present (D-
day)▪ Duration : minutes
In-lab induced protocol EmoTaboo : adaptation of the
game TABOO It involves interaction between two players One of them has to guess a word that the other
player is describing using his own speech and gestures without uttering five forbidden words
We use strategies for eliciting emotions connected to: the course of the game the selection of the cards The instructions given to the confederate
19
Dyadic interaction Word guessing
game
10 pairs naïve subjects:
4 women, 6 men confederates (close
relations with research staff): 3 women, 4 men
4 cameras 8 hours
Examples of events used for inducing emotions the card contains the word "palimpsest" the experimenter announces a penalty the confederate proposes words with no relation at all with what is
said by the naïve player the confederate finds the mimed word/does not find the mimed
word the confederate is ironic the confederate criticizes the naïve player
Ex: the card contains the word "palimpsest" is estimated by the player as: recent unexpected (the player did not expect a word that he did not
know) incompatible with his(her) purposes to win the game.
Timescale of events is not the same as in EmoTV
EmoTABOO events are rather short term
One long-term event: the game session
In this exploratory study, we tried to show the potential of a scheme of emotional events in order to better understand the emotional states
We proposed to code the temporal aspects of the events: date and duration (short-time, long-time)
Annotation of more dataAnnotate appraisal dimensions of
eventsStudy the relations between
temporal and appraisal annotations the events and the perceived complex
emotions
Goal: designing affective computing applications Possible approach:
Collect small set of real life data Annotate (events, emotion labels, …) Inspire from these annotations to define a protocol
close to the target application to elicit emotional behaviors in-lab
Collect large corpora of induced realistic emotional behavior relevant for the application
Opens a big question: how close to the intended context does training
material need to be?
Thanks for attention
The motivation of this special issue is to report innovative work on the modelling and generation of affect in real-life speech and spoken interaction (including human-human or human-machine interaction, multi-party interaction) or in “realistic” interactions (including realistic fictions).
Editors: Editors: Nick Campbell and Laurence Nick Campbell and Laurence DevillersDevillers
28
LIMSI – TV clips : Interviews from news - FrenchCause event: Unexpected bad Results in an ElectionCurrent event:Impact of election loss on political group
Multimodal Cues:+ speech content-Tense Smile (badControl of their Facial muscles)
-Emotion blends:- Disappointmentmasked by + courage
Cognitive appraisal theory (Scherer, 2000) argues that
- an organism may possess many distributed processes for interpreting this relationship (e.g., planning, explanation, perception, etc.)
-but that appraisal maps characteristics of these disparate processes into a common set of intermediate terms (intermediate between stimuli and response, between organism and environment) called ‘appraisal variables’
(Gratch and Marsella, 2004).