![Page 1: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/1.jpg)
Affective Computing for Intelligent Agents
Introduction to Artificial Intelligence CIS 4930, Spring 2005
k i d h lGuest Speaker: Cindy Bethel
![Page 2: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/2.jpg)
Affective Computingp g• Affect:
h if i i lf d h f f– phenomena manifesting itself under the form of feelings or emotions
• Affective computing:Affective computing: – Where computers take into account that users have
emotions– … computers exhibit emotions
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 2
![Page 3: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/3.jpg)
Motivation(Pi d)(Picard)
• Picard (founder): “ if we want computers to be genuinely… if we want computers to be genuinely
intelligent, to adapt to us, and to interact naturally with us, then they will need the ability to recognize and express emotions, y g pto have emotions, and to have what has come to be called ‘emotional intelligence’.”
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 3
![Page 4: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/4.jpg)
Motivation (cont.)(Pi d)(Picard)
• Emotion is essential for R ti l d i i ki– Rational decision making
– Learning– PerceptionPerception– Managing sets of cognitive functions
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 4
![Page 5: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/5.jpg)
Related Definitions• Human-computer interaction (HCI)
Th t d f i t ti b t l d– The study of interaction between people and computers
• Human-robot interaction (HRI)– The study of interaction between people and robotsy p p
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 5
![Page 6: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/6.jpg)
Related Definitions• Social informatics
Th t d f i f ti d i ti t l– The study of information and communication tools in a social and cultural context
• Emotional intelligence– A kind of intelligence that involves the ability toA kind of intelligence that involves the ability to
perceive, assess, and influence one's own and other's emotions
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 6
![Page 7: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/7.jpg)
Applications (Breazeal Murphy)(Breazeal, Murphy)
• Facilitate collaboration between humans and agentsagents – Make interactions more natural (see Nass)– Make interactions more enjoyableMake interactions more enjoyable – Enable social learning (Remember Breazeal?)
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 7
![Page 8: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/8.jpg)
Applications (Breazeal Murphy)(Breazeal, Murphy)
• Simplify design and control of complex agentsE ti P f M it ( d f db k)– Emotions as Performance Monitor (and feedback)
– Sorting out among multiple processes• Knowing what mattersKnowing what matters• Knowing what action to try• Correcting errors and recognizing successes
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 8
![Page 9: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/9.jpg)
Example: Human-Robot Interactionp• Robots often have to team with people or work
in close proximityin close proximity
K ti• Key questions– How to divide up responsibilities or roles?
How to change them dynamically?– How to change them dynamically?– How do people like to interact with robots? – How do they interact most effectively?How do they interact most effectively?– Do robots and people need to “understand” each
other (e.g., have a shared cognitive model)?
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 9
![Page 10: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/10.jpg)
Emotions: Appraisal Mechanismpp• A mechanism for adapting to the world
U i i f ti i f ti l– Unconscious information processing of stimulus significance
– Leads to a conscious, subjective experienceLeads to a conscious, subjective experience
• Ex., fear– See a predator, start running– Later, say “I felt scared”
• Ex. Steadfast in cold– Reflex is to find shelter
E ti h l d t fl
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 10
– Emotions help adapt, overcome reflex
![Page 11: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/11.jpg)
A Simplified Neurological Model(Scherer Ortony)(Scherer, Ortony)
Percepts(from cognitive areas)
Amygdala
Raw Signals(from senses)(from senses)
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 11
![Page 12: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/12.jpg)
Can Rapidly Change Behaviorp y g
Percepts(from cognitive areas)
Amygdala
Raw Signals(from senses)(from senses)
Motivational-Behavioral(facial expressions,
Actions)
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 12
)
![Page 13: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/13.jpg)
Can Cause a Physiological ResponseResponse
Percepts(from cognitive areas)
Amygdala Somatic (endocrine system)
Raw Signals(from senses)
( y )
(from senses)Motivational-Behavioral
(facial expressions,Actions)
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 13
)
![Page 14: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/14.jpg)
Can Lead to a “Feeling”g
Conscious-InterpretativePercepts
(from cognitive areas)
(subjective emotion)
Amygdala Somatic (endocrine system)
Raw Signals(from senses)
( y )
(from senses)Motivational-Behavioral
(facial expressions,Actions)
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 14
)
![Page 15: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/15.jpg)
Appraising What? (Ortony)pp gEvent,AgentAgent
Goals Norms/ Tastes/Goals Standards Attitudes
desirability praiseworthiness appealingness
Goal-BasedEmotions
CompoundEmotions
Norm-BasedEmotions
Taste-BasedEmotions
Joy/DistressHope/Fear
Relief/
Anger/Gratitude
Gratification/
Pride/Shame
Admiration/Love/HateLike/Dislike
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 15
Disappoint. Remorse Reproach
![Page 16: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/16.jpg)
Subjective Nature of Emotionsj• “Fuzzy” subjective experience
• More of a spectrum than a single state– Joy/distress– Hope/fear
• This spectrum can be broken intoValence (where it is on the positive or negative side– Valence (where it is on the positive or negative side of the spectrum)
– Intensity (what the value is: a little? A lot?)
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 16
![Page 17: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/17.jpg)
Social Expression (Breazeal)(Breazeal)
• We show emotions to– Share control between agents
• Taking turnsi i• Expressing intent
• Does this require a shared cognitive model?
– Invoke a response• Communicate intent and confirm message was• Communicate intent and confirm message was
sent and received
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 17
![Page 18: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/18.jpg)
Bodemotes Research (Bethel)(Bethel)
Research Questions:• Can a robot without a face display affect
to human observers?• What kinds of inputs can be used to
determine an emotional output?p• How do we display emotion without the
use of a face?use of a face?
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 18
![Page 19: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/19.jpg)
Bodemotes Research (cont’d)(Bethel)(Bethel)
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 19
![Page 20: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/20.jpg)
Breazeal: KismetHigh Intensity
Surprise
PositiveValence
NegativeValence
AngryInterest
DisgustCalm
ValenceValenceContent
SadLow Intensity
From: http://www.ai.mit.edu/projects/sociable/facial-expression.html
![Page 21: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/21.jpg)
Multilevel Process Theory of Emotions (Leventhal and Scherer 1987)(Leventhal and Scherer, 1987)
Failure to make progress on tasks/goals changes emotional statewhich then produces multilevel response
Levels: Emotional Processes• Sensory-motor
– Sensed or internal events produce emotion which changes anSensed or internal events produce emotion which changes an agent’s behavior
– Example: pain causes an aggressive response
• Schematic– Emotions control which behaviors are active through prototypical
schemas
• Conceptual– Agent reasons about emotions and projects into the future
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 21
![Page 22: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/22.jpg)
Case Study of Multi-Process Theory: USF WaitersTheory: USF Waiters
• Hors D’euvres Anyone? Event– Cover the most area while
serving food at a reception– Fully autonomous– Interact with humans
• Approach– Two robots, one with more
sensors than the others– Sensor-endowed robot is waiterSensor endowed robot is waiter
because can interact with people better
– Less-endowed robot acts as a refiller, bringing trays upon g g y prequest to maximize coverage by waiter
– 1999: people trapped refiller (deadlock)
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 22
![Page 23: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/23.jpg)
With EmotionsESG uses a FSA to provide feedback: sensory-motor levelsensory motor level “tweaks” parameters, while schematic level triggers alternati etriggers alternative instantiations (escalating behavior))
The set of possible behaviors remains the same but thethe same, but the activation and dynamic adaptation make it more
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 23
reactive and opportunistic
![Page 24: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/24.jpg)
Results: Hurried Refill• Claims
– The Task Monitor has • Demonstrates
– Refiller has a sensory-motor indicated a condition that requires the Refiller to hurry up.W i d “H ”
ylevel change affected by a change in ESG (speeds up).
– Waiter sends a “Hurry” message to Refiller
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 24
![Page 25: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/25.jpg)
Results: Interceptp• Claims
– Waiter has a Schematic level • Demonstrates
– Emotional state change from change, e.g. SERVE to INTERCEPT.
– Task Monitor recognizes the “ di i ” h h R fill
gConcerned to Frustratedcauses Waiter to have a schematic level change.Thi difi i f“condition” that the Refiller
is not going to arrive in time to avoid the Waiter having to wait without the required
– This modification of behavior proactively avoids deadlock situation
qresource.
– This “condition” leads to the Emotional state change.
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 25
![Page 26: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/26.jpg)
Results: GoHome• Claims
– Waiter has a Schematic and• Demonstrates
– Waiter experiences AngerSensory-motor level changes in response to Anger.
p gwhen Refiller is unable to successfully complete its assignment. Waiter essentially ‘fires’ theessentially fires the Refiller. Changes from SERVING to GOHOME.
– Waiter avoids deadlock by ycompleting resupply itself.
– Waiter travels at the fastest speed possible.
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 26
![Page 27: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/27.jpg)
Emerging Applicationsg g pp• Games and interactive web-bots
– “Uncanny valley”Uncanny valley– Creating believable behavior
Lifelike human model by Andrey Kravchenko
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 27
Humanoid robot developed by Dr. Hiroshi Ishiguro
![Page 28: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/28.jpg)
Review Questions: Affective ComputingAffective Computing
• Define affective computingWhere computers take into account that users have emotions– Where computers take into account that users have emotions
• What is HRI?H h d b i i h h h– How humans and robots interact with each other
• Give two uses of affective computing in artificial intelligence.– Control or self-regulation– Naturalistic interfaces
• Can you have an emotional response without being conscious of it?– Yes
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 28
![Page 29: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/29.jpg)
Review (Cont.)
• What is valence and intensity of an emotion?– Valence is where the affect is on the emotion’s spectrumValence is where the affect is on the emotion s spectrum– Intensity is how strong the emotion is
• What are the four types of emotions in the neurological model (Scherer and Ortony)? – Goal-based, compound, norm-based, and taste-basedGoal based, compound, no m based, and taste based
• What are the three layers of the multi-level process theory of emotions? – Sensorimotor, schematic, conceptual
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 29
![Page 30: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/30.jpg)
Review (Cont.)( )• What did the following systems contribute in terms of
emotions?emotions?– Kismet
• Social interfaceUSF W it– USF Waiters
• Self-regulation, implementation of multi-process theory of emotions
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 30
![Page 31: Affective Computing for Intelligent Agentshexmoor/classes/CS404-S09/Affect.pdf · Affective Comppguting • Affect: – phh ifi ilf d hf fenomena manifesting itself under the form](https://reader035.vdocuments.us/reader035/viewer/2022063010/5fc3d67aa2be511c2d0955cb/html5/thumbnails/31.jpg)
Affective Computingp g• Materials responsible for:
L t t– Lecture notes
• Assignments associated with this topic:N– None
• Objectives:Define Affective Computing– Define Affective Computing
– Describe why social informatics is important in intelligenceg
CIS 4930 Introduction to AIAffective Computing for Intelligent Agents 31