social and emotional intelligence in ai and agents filecomputers in human behavior, 37, pp. 94-100....
TRANSCRIPT
Daniel McDuff
Social and Emotional Intelligence in AI and Agents
Applications
Personal Assistance
Education
Transportation
Gaming
Health and Care
Marketing
Governance
Applications
Personal Assistance
Education
Transportation
Gaming
Health and Care
Marketing
Governance
For an assistant to truly become valuable to the user, it must be natural to interact with and engaging.
• T Bickmore and J Cassell. 2001. Relational agents: a model and implementation of building user trust. Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 396-403. ACM.
• J Gratch, N Wang, J Gerten, E Fast, and R Duffy. 2007. Creating rapport with virtual agents. International Conference on Intelligent Virtual Agents, pp. 125-138. Springer.
• GM Lucas, J Gratch, A King, and LP Morency. 2014. It’s only a computer: virtual humans increase willingness to disclose. Computers in Human Behavior, 37, pp. 94-100.
For an assistant to truly become valuable to the user, it must be natural to interact with and engaging.
• T Bickmore and J Cassell. 2001. Relational agents: a model and implementation of building user trust. Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 396-403. ACM.
• J Gratch, N Wang, J Gerten, E Fast, and R Duffy. 2007. Creating rapport with virtual agents. International Conference on Intelligent Virtual Agents, pp. 125-138. Springer.
• GM Lucas, J Gratch, A King, and LP Morency. 2014. It’s only a computer: virtual humans increase willingness to disclose. Computers in Human Behavior, 37, pp. 94-100.
Sensing Camera
Physiological measurement (PPG)
Camera
Facial expressions
Scene Understanding
Cellphone
Personal Agent
Activities + Experience Sampling
Microphone
Speech-to-Text and Sentiment
Voice prosody
Example
PSI
Physiology (HR, HRV, Respiration)
Facial Actions and Expressions
Speech Sentiment
Voice Tone/Prosody
Scene Understanding + Object Detection
Webcam and Microphone
Multimodal
Combining modalities consistently improves recognition performance and can help generalize affect recognition.
Longitudinal
Emotions are short-lived and moods are longer, detecting an emotion without knowledge of a persons typical behavior is hard.
Personalized
Interpersonal variability is large, often larger than cultural or other demographic variability.
age gender culture
years men women migration
po
siti
ve e
mo
tio
n
po
siti
ve e
mo
tio
n
po
siti
ve e
mo
tio
n
Learning Across Datasets
Anonymized/Aggregated
Large-scale Meta Data
Dataset A Dataset B Dataset C