nao robot as a social mediator - imiimi.ntu.edu.sg/newsevents/events/pastseminars/documents/... ·...
TRANSCRIPT
Nao Robot as a Social Mediator
Supervisor: Asst Prof Justin DauwelsSchool of Electrical and Electronic Engineering, NTU
PhD Student: Yasir Tahir
Institute for Media Innovation (IMI), EEE, NTU
1
Co-Supervisor: Dr Daniel ThalmannInstitute for Media Innovation (IMI), NTU
Outline
SOCIOFEEDBACK SYSTEM OVERVIEW MOTIVATION
APPROACH
RESULTS
NAO as a SOCIAL MEDIATOR RELATED WORK
EXPERIMENTS
DEMO VIDEO
2
3
Sociofeedback System An Overview
Motivation When people communicate, their behavior and personality traits are
reflected in their speaking mannerism and body language.
Our objective is to analyze behavior in real-time and provide feedback based on user’s social state.
Approach We use audio-visual and depth data to extract behavioral cues. We
conduct data collection experiments and then get the recordings annotated. We then extract behavioral cues and train a machine learning classifier.
We determine levels of Dominance, Interest and Agreement and estimate social state using these values.
3
Functional Blocks of the System
4
Results
Numerical tests through the leave-one-out cross-validation indicate that the accuracy for inferring the sociometrics is in the range of 80-85%.
5
Rasheed, Umer, Yasir Tahir, Shoko Dauwels, Justin Dauwels, Daniel Thalmann, and Nadia Magnenat-Thalmann. "Real-time comprehensive sociometrics for two-person dialogs." In Human BehaviorUnderstanding, pp. 196-208. Springer International Publishing, 2013.
Possible Feedback Platforms
• Social Mediator via Nao
• Retrospective Feedback via Avatar Animation
• Socio-Feedback via Skype Application
• Socio-Feedback via Android Application
6
Feedback via Nao Robot
7
The following steps are discussed in this section:-
Related work in social robotics Concept of Nao as social mediator User studies conducted Demo video
Related Work: Human Robot Interaction
8
“Human beings seem to perceive affect from
robot motion, regardless of its shape and
outlook. Harris and Sharlin conducted a study to
understand different emotions that humans
perceive by movements of an abstract looking
robot”
Exploring the affect of abstract motion in social human-robotInteraction.J. Harris and E. Sharlin
“There is an experiment conducted to understand the affect humans elicit from the movement of a non-human like robot. The robots in this experiment were iCat robot and Roomba vacuum cleaning robot.
Perception of affect elicited by robot motion.M. Saerbeck and C. Bartneck
Can human beings identify emotions expressed by a humanoid robot using gestures? To address that question, a study was performed to explore whether humans can recognize emotions displayed by humanoid robot (Nao).”
Towards an affect space for robots to display emotional body language.A. Beck, L. CanameroInterpretation of emotional bodylanguage displayed by robots.A. Beck, A. Hiolle
Related Work: Social Robotics Applications
9
“In the recent past, many social robots
have been designed for real world
interactions e.g. Kismet, Mel, Pearl,
Robovie, and Robota , and Paro.”
Kismet: A context-dependent attention system for a social robot.Mel: A first experiment in engagement for human-robot interaction in hosting activities.Pearl: A mobile robotic assistantfor the elderly.Robovie: an interactivehumanoid robot
“Nowadays, social robots are successfully helping children in their social, emotional and communication deficits. They create interesting, appealing, and meaningful interplay situations that compel children to interact with them. One of the emerging applications of social robotics is the therapy of children with autism.”
An approach to the design of sociallyacceptable robots for children with autism spectrum disorders.K. C. Welch, U. LahiriMimicking and evaluating human motion to improve the imitation skill of children with autism through a robot.I. Fujimoto, T. Matsumoto
“Similarly, social robots are being actively deployed in nursing homes for assistance of the elderly. Those studies typically investigate what different social functions the device can play in the living environment of the elderly.”
A sociable robot to encourage social interaction among the elderlyC. D. Kidd, W. Taggart, and S. Turkle,
Nao as Social Mediator
SOCIAL MEDIATOR The role of social mediator means that Nao robot is interfaced with
sociofeedback system. The sociofeedback system analyses an ongoingconversation.
REAL-TIME FEEDBACK on social behavior Using the inference generated by sociofeedback system the robot can
provide relevant feedback using gestures and audio message.
10
Monitoring of Speakers + Feedback
Behavior Detection: Dominance, Interest, Discord, Consistency,
Mirroring etc
Feature Extraction
Visual Cues
Speech Cues
Audio and Video Data Acquisition
Feedback based on
SocialBehavior
Please carry on!Kindly wait for your
turn…
Please speak louder…
User Studies
12
So far we have conducted two user studies:-
First one to get user opinion about the feedback being delivered. Also to determine the most appropriate way of delivering feedback.
In the second study we interfaced Nao with sociofeedback system and asked participants to interact in certain manner. User speech was analysed and feedback was delivered.
User Study for appropriate Feedback
13
We called 20 people to receive feedback from Nao robot.
The participants did not have to participate in a conversation.
We provided feedback with only audio, only gestures and then by combining audio along with gestures.
The participants first identified the feedback and then rated the robot on Godspeed questionnaire.
• Yasir Tahir, Umer Rasheed, Keng Hui, Shoko Dauwels, Justin Dauwels. "`Nao as social mediator: A user study." In Robots in public spaces:towards multi-party, short-term, dynamic human-robot interaction, 2013.
Godspeed Questionnaire
14
AnthropomorphismMachinelike 1 2 3 4 5 HumanlikeMoving rigidly 1 2 3 4 5 Moving elegantly
AnimacyMechanical 1 2 3 4 5 OrganicInert 1 2 3 4 5 Interactive
LikabilityDislike 1 2 3 4 5 LikeUnfriendly 1 2 3 4 5 Friendly
Perceived IntelligenceIgnorant 1 2 3 4 5 KnowledgeableUnintelligent 1 2 3 4 5 Intelligent
Perceived SafetyCalm 1 2 3 4 5 AgitatedQuiescent 1 2 3 4 5 Surprised
• Bartneck, Christoph, et al. "Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots." International Journal of Social Robotics 1.1 (2009): 71-81.
Results
15
Speak Louder
Too noisy
Stop Interrupting
Calm down
Slowdown
Uninterested
Audio 84.2 68.4 89.4 89.4 89.4 78.9
Gestures 63.1 15.7 21 57.8 31.5 36.8
Combined 84.2 100 100 94.7 100 100
Results
16
Characteristics P-values Averagevalue (audio)
Average value(combined)
AnthropomorphismMachine/human like
0.017 2.94 3.52
Moving rigidly/elegantly 0.002 2.36 3.26
AnimacyMechanical/organic
0.129 2.89 3.26
Inert/interactive 0.046 3.15 3.63
LikabilityDislike/like
0.110 4.26 4.63
Unfriendly/friendly 0.008 3.73 4.26
Perceived IntelligenceIgnorant/knowledgeable
0.009 3.31 3.63
Unintelligent/intelligent 0.186 3.42 3.57
Perceived SafetyCalm/agitated
0.741 2.36 2.26
Quiescent/surprised 1.00 2.84 2.84
Human Perception of Nao as social mediator
17
We called 20 people to receive feedback from Nao robot.
The participants engaged in a conversation. The conversation was analyzed in real time and then feedback was provided at the end.
Based on the conclusions of previous studies we provided feedback using audio and gestures.
The participants rated the experiment on two questionnaires. One questionnaire was to rate the feedback and the second one to rate human perception using Godspeed questionnaire.
• Yasir Tahir, Umer Rasheed, Shoko Dauwels, Justin Dauwels. "`Perception of Humanoid Social Mediator in Two-Person Dialogs." Accepted as Late braking report at Human Robot Interaction (HRI 2014)
Questionnaire for Feedback assessment
18
Questions Average
valueQ1 Did you notice when the socio-feedback system was addressing you? 4.5
Q2 Did you notice when the socio-feedback system was addressing others? 4.5
Q3 Was the timing of socio-feedback appropriate? 3.3
Q4 Did the socio-feedback system interrupt the conversation? 2.3
Q5 Was the interaction natural? 4.2
Q6 Did you understand the message given by the socio-feedback? 4.8
Q7 Do you agree with the given feedback? 4.7
Q8 Did you enjoy using the socio-feedback system? 4.8
Demo Video
19
Q & A
Q&A
20