Automated Caricature of Robot Expressions in Socially Assistive Human-Robot InteractionRoss Mead and Maja J MatarićPresented by David Feil-Seifer

Interaction LabUniversity of Southern California

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

2 13

Outline

Motivation and Background

Approach and Methods

Robot Platform

Experimental Design

Q & A

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

3 13

Motivation and Background

Children with autism spectrum disorder (ASD) suffer from a deficit that prevents them from observing, interpreting, and learning social cues.

Clinical studies in social skills training have proposed methods, such as exaggeration, to enhance autism intervention strategies (Rao et al., 2008).

Socially assistive robotics has potential to improve social activity (Feil-Seifer & Matarić, 2005).

The Transporters is an animated series that has shown an improvement in face-based emotion recognition in children with ASD (Golan et al., 2009)…• However, expressive behaviors of the body still remain to be addressed.

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

4 13

Approach and Methods

Take inspiration from several principles of animation (Thomas & Johnston, 1981)…• Staging• Exaggeration• Anticipation• Secondary Action

Goal: Automated caricature of social interaction behaviors of robots.

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

5 13

Approach and Methods — Staging

Aims to provide some grounding for robot gestures in subsequent operations.Process of presenting a communicative act in as clear a way as possible…• Minimize or eliminate conflicting signals.• Isolate features that uniquely identify the content of the expression.

Caricaturing highlights such features, providing building blocks for expressions.Hypothesis: Well-staged expressions provide more clarity to a child with ASD than poorly-staged expressions.

Source: http://familypants.files.wordpress.com/2009/02/preston11.jpg

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

6 13

Approach and Methods — Exaggeration

Amplify distinct features that identify the expression to make content of the behavior more convincing and/or explicit.Use feature parameterizations isolated during the staging process to produce exaggerated expressions.Hypothesis: A child with autism will be more capable of interpreting an exaggerated expressive behavior than a non-exaggerated expressive behavior…• Supported by the peak shift principle (Ramachandran & Hirstein,

1999).

Source: http://www.animationbrain.com/2D/38principle.JPG

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

7 13

Approach and Methods — Anticipation

Anticipatory action often indicates or emphasizes the intent of the character.Staging and exaggeration provide insights pertaining to the dynamics of a communicative act…• We aim to utilize this to automatically generate motion paths for both

micro and macro anticipatory expressions that precede it.Hypothesis: Anticipatory actions will provide a child with ASD a better understanding of the intent of his or her social partner than non-anticipatory action.

Source: http://www.evl.uic.edu/ralph/508S99/gif/batter.gif

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

8 13

Approach and Methods — Secondary Action

The use of redundant signals in an expression to better communicate an idea.Redundancy plays a key role in social interaction (Birdwhistell, 1970).Signals must be isolated into distinct parts for proper staging, exaggeration, and anticipation to occur.We suspect that secondary action has potential with high-functioning children with autism (particularly, those who have participated in the Transporters studies)…• Might be overwhelming for children that are far in the spectrum.

Source: http://www.awn.com/tooninstitute/lessonplan/images/walk17.jpg

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

9 13

Robot Platform

Currently being implemented on the Sparky Minimatronic™ robot figure…• Courtesy of Walt Disney Imagineering

Research & Development.

Uses two servo controllers and 18 servos…• Lightweight and highly dexterous.• Movements are fluid and natural.• Articulated spine allows us to

manipulate posture-based expressions.

Will utilize an off-board sensor network that includes color cameras, lasers, Nintendo Wiimotes™, and desktop computer interface.

Being implemented as a tabletop agent, interacting verbally and nonverbally with user.

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

10 13

Experimental Design

Preliminary work discusses gesture repetition/persistence (Mead & Matarić, 2009).

Test techniques and hypotheses within the context of ASD social skills intervention.

Must validate with typically developed children…• Consider expressive behaviors that utilize the isolated features determined

during the staging process.• Consider their exaggerated, anticipatory, and secondary counterparts.

Conduct a study with children with ASD to determine the impact of each technique.

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

11 13

Acknowledgments

This work is supported in part by the National Science Foundation under:• Graduate Research Fellowship Program (GRFP)• Grant CNS-0709296 (“CRI: IAD - Computing Research Infrastructure for

Human-Robot Interaction and Socially Assistive Robotics”)• Grant IIS-0803565 (“Personalized Socially-Assistive Human-Robot

Interaction: Applications to Autism Spectrum Disorder”)• Grant IIS-0713697 (“HRI :Personalized Assistive Human-Robot Interaction:

Validation in Socially Assistive Robotics for Post-Stroke Rehabilitation”)

We would like to thank Akhil Madhani and Walt Disney Imagineering Research & Development for the use of the Sparky Minimatronic™ robot figure.

HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?

12 13

Selected References

1. S. Baron-Cohen, O. Golan, E. Chapman and Y. Granader, “Transported into a World of Emotion,” in The Psychologist, vol. 20, no. 2, pp. 76-77, 2007.

2. R. L. Birdwhistell, Kinesics and Context: Essays on Body Motion Communication. Philadelphia, PA: University of Pennsylvania Press, 1970.

3. S. E. Brennan, “Caricature Generator: The Dynamic Exaggeration of Faces by Computer,” in Leonardo, vol. 18, no. 3, pp. 170-178, 1985.

4. D. J. Feil-Seifer and M. J. Matarić, “Defining Socially Assistive Robotics,” in International Conference on Rehabilitation Robotics, pp. 465-468, 2005.

5. O. Golan, E. Ashwin, Y. Granader, S. McClintock, K. Day, V. Leggett, and S. Baron-Cohen, “Enhancing Emotion Recognition in Children with Autism Spectrum Conditions: An Intervention Using Animated Vehicles with Real Emotional Faces,” in the Journal of Autism and Developmental Disorders, DOI: 10.1007/s10803-009-0862-9, in press.

6. R. Mead and M. J. Matarić, “The Power of Suggestion: Teaching Sequences Through Assistive Robot Motions,” in the Proceedings of The 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI-09), pp. 317-318, 2009.

7. Z. Mo, J. P. Lewis, and U. Neumann, “Improved Automatic Caricature by Feature Normalization and Exaggeration,” SIGGRAPH 2004 Sketches and Applications, 2004.

8. V. S. Ramachandran and W. Hirstein, “The Science of Art: A Neurological Theory of Aesthetic Experience,” in the Journal of Consciousness Studies: Special Feature on Art and the Brain, vol. 6, no. 6-7, pp.15-51, 1999.

9. P. A. Rao, D. C. Beidel, and M. J. Murray, “Social Skills Interventions for Children with Asperger’s Syndrome or High-Functioning Autism: A Review and Recommendations,” in the Journal of Autism and Developmental Disorders, vol. 38, no. 2, pp. 353-361, 2008.

10. A. Tapus, M. J. Matarić, and B. Scassellati, “The Grand Challenges in Socially Assistive Robotics,” in IEEE Robotics and Automation Magazine, vol. 14, no. 1, pp. 35-42, 2007.

11. F. Thomas and O. Johnston, The Illusion of Life: Disney Animation. Hyperion, 1981.

Please address praise and easy questions torossmead@usc.edu.Please address criticism and difficult questions to Dave…

Q & A