from affective interaction to emotion-oriented systems

From Affective Interactions to Emotion-Oriented Systems

Human-Computer Interaction

Master on Software Engineering Human-Computer Interaction

“I believe that logic + emotion are a winning combination. When useful and useable meet

delight great things happen. It’s about balance.”

David Armano

Stefan Negru – www.blankdots.com

http://www.flickr.com/photos/sskennel/


What are emotions, feelings and the other affect related concepts ?



Definition: “Affect is a feeling or emotion as distinguished from cognition, thought, or

action.”



“Emotion encompasses physiological, affective, behavioral, and cognitive components.”

Scott Brave & Clifford Nass, 2008



“Emotion is a reaction to events deemed relevant to the needs, goals, or concerns of an individual.”

Scott Brave & Clifford Nass, 2008



Four systems for emotion activation: Cognitive and noncognitive processes.

By Izard, Carroll E. Psychological Review, Vol 100(1), Jan 1993, 68-90.



neurological structure of emotion (LeDoux, 1996)



emotion versus mood

moods are not directed at any object in particular and are experienced as more diffuse, global, and general

state

a person can be sad about something - an emotionor generally depressed - a mood



emotion versus mood

An emotion can lead to or take part in establishing a certain mood

Sadness (& other factors) depression



emotion versus feeling

Feelings relate more to a property associated to and object or being, than to a person’s state of mind.

Feeling last more (days/weeks/years) than emotions (seconds) or moods (hours / days)



emotion versus feeling

Feelings are often pointed towards a class of objects, beings distinguishable through a certain property

“I love Adobe CS applications!”

“I hate text-editing applications”



emotion and memory

negative events, which tend to be highly arousing, are typically remembered better

than positive events



emotion and performance

keeping a user “happy” may, therefore,not only affect satisfaction, but may also lead

to efficiency and creativity



emotion and performance

keeping a user “happy” may also influence information processing and decision

making



emotion and communication

humans use emotions when communicating with other humans in order to:

show their internal state of mindalert others regarding some event/state

show empathy (understanding of situations)



How do we recognize and classify them?



A Circumplex model of affect (Russell, 1980).



Russell’s circumplex remodeled (Desmet, 2002; van Gorp, 2006)



emotion dimensions

Valuepleasant versus unpleasant

Arousal (psychological)

anxiety versus boredom



Anxiety, Boredom and Flow (Csikszentmihalyi, 1990; van Gorp 2006)



Flow

Achieving the desired state and keeping the user in that state is our goal.


Humane interfaces consider emotions



“Products were once designed for the functions they performed. But when all companies can make products that perform their functions equally well, the distinctive advantage goes to those who provide pleasure and enjoyment while maintaining the power. If functions are equated with cognition, pleasure is equated with emotion; today we want products that appeal to both cognition and emotion.”

Don Norman

“The Transmedia Design Challenge: Technology that is Pleasurable and Satisfying”,

ACM Interactions Volume 17 Issue 1, January + February 2010



Affective Interaction design

The need for products to convey positive emotions and avoid negative ones is critical to product

success.



Affective Interaction design

Designing expressive interfaces which make use of:Dynamic elements

AnimationsIcons, fonts, colors, graphical layouts

to influence how the effectiveness of an interface is perceived



Affective Computing

Affective Computing is computing that relates to, arises from, or deliberately influences emotion

and other affective phenomena.



Affective Computing

RecognizingInterpreting

Drawing out emotionsEmotional influence on behavior

Expressing emotions




http://www.interaction-design.org/encyclopedia/affective_computing.html

A general point of view

Affective Computing


It’s all nice, tidy and easy to understand right ?



It’s all nice, tidy and easy to understand right ?


Wishful thinking



Where’s the problem ?


Examples of affective interactions




http://getsatisfaction.com/spotify/topics/when_will_we_have_a_blackberry_client

http://satisfaction.mint.com/mint/topics/account_classifications_we_need_ability_to_change_them



http://satisfaction.mint.com/mint/topics/account_classifications_we_need_ability_to_change_them



http://www.nbcchicago.com/news/local-beat/Frank_lloyd-wright-fallingwater-lego.html


A few Problems

+ Context

+ Social Environment



A few more Problems – the engineering point of view

1. Detecting Emotions2. Labeling Emotions3. Processing Emotions4. Obtrusive interface



1. Detecting Emotions

Neurological responses

via electroencephalography (EEG) or magneto‐resonance imaging (MRI)

User input Facial expression

Voice



http://www.emotiv.com/based on electroencephalography (EEG) technology




“Lie to me”– Tv showhttp://www.imdb.com/media/rm817007104/tt1235099



METT – Micro Expression Training ToolSETT – Subtle Expression Training Tool



Asking our user how he feels.



Detecting emotions in an Emotion-Oriented system


2. Labeling Emotions

Which are basic emotions ?Group Emotions

Mixed versus pure emotions




http://wefeelfine.org/12 million human emotions recorded since 2005 (2009 statement)



Emotion-Oriented Systems The Humaine Handbook, Springer 2011


Basic emotions

anger, disgust, fear, happiness, sadness and surprise



Labeling emotions

Labeling is important but not for a machine



A label for the machine is a x,y coordinate on the circumplex.



Proposal of labeling emotions

FeelTagged an Arduino-based Social-Emotional Interactive System – Stefan Negru 2010



3. Processing emotions

Is it a mood/feeling/emotion ?What category/dimension ?

Multiple and/or complex emotions Emotion intensity, timing and context observed ?

etc.


Annotating emotions

<emotionml xmlns="http://www.w3.org/2009/10/emotionml">

<emotion>

<dimensions set="PleasantAnxiety">

<pleasant value="1.3"/> <!—- somewhat positive pleasant state -->

<anxiety value="1.9"/> 

<modality set="PersonalContext" mode="face voice" medium="visual acoustic"/>

</dimensions>

</emotion>

</emotionml>

EmotionML http://www.w3.org/TR/emotionml/


5. Have an unobtrusive interface

Providing a useful, easy to understand and adaptive interface – secondary goal

Unobtrusive interface- primary goal



emotion and attention

an interface capable of detecting, or predicting a user’s emotional or mood state could similarly assume an affect‐regulation

role, helping to guide attention away from negative and toward more positive stimuli



emotion and attention

Disregarding the flow state

“This system failed to understand your command.” – blaming itself

“The command was not understood.” – blaming no one

“You did not speak clearly enough for your command to be understood.” – blaming the user



Affective based interaction

Affective aspects of interaction (Rogers, 2007)

frustration

how we can detect and reduce it?



Affective based interaction

Affective aspects of interaction (Rogers, 2007)

persuasive technologies

how we can change the user attitude & behaviorby using the actual technologies?


http://www.happypills.es/


Affective Interaction

Example: a robot hardware platform called iCAT uses a set of logical rules to convey emotional states as it makes decisions, with the goal of improving human-robot interaction

Philips Research Laboratory 2005




http://www.research.philips.com/



http://pleoworld.com/

Pleo is an animatronic dinosaur toy designed to emulate the appearance and (imagined) behavior of a week-old baby Camarasaurus.



http://fostsvn.uopnet.plymouth.ac.uk/

ALIZ-E project



http://www.affectiva.com/q-sensor/

Q Sensor 2.0 - Measuring Emotion



http://blog.youeye.com/youeye-announces-webcam-emotion-tracking



http://www.affectiva.com/affdex/

“Affdex reads emotional states such as liking and attention from facial expressions using a webcam... to give marketers faster, more accurate insight into consumer response to brands and media. “


Ethical aspects

The goal is not to control people actions but to support them in order to improve their emotional state and to provide a better interaction based on

the user’s emotional state



Dark side of emotions

Humans could have an offensive behaviorregarding the digital entities

www.agentabuse.org





example (Tatsuya Nomura, 2005):

impolite behavior of the patients,when they discovered that the therapist is a virtual

agent





experiment (Christoph Bartneck et al., 2005):

humans tend to hurt more easily robots/devicesthan people



Unobtrusive versus Informative

Should users be informed that their affective states are being monitored and adapted to ?




Questions

from affective interaction to emotion-oriented systems

Education

applications stefan

moodstefan negru

creativitystefan negru

david armano stefan

emotion activation

moodan emotion

logic emotion

positive eventsstefan