usc / information sciences institute making pedagogical agents more socially intelligent lewis...

USC / Information Sciences Institute

Making Pedagogical AgentsMaking Pedagogical AgentsMore Socially IntelligentMore Socially Intelligent

Lewis JohnsonLewis JohnsonDirector, CARTEDirector, CARTE

USC / ISIUSC / ISIftp://ftp.isi.edu/isd/johnson/si/ftp://ftp.isi.edu/isd/johnson/si/


Background: Pedagogical Background: Pedagogical Agents (aka Guidebots)Agents (aka Guidebots)


Adele DemoAdele Demo


ClaimsClaims

Such guidebots requireSuch guidebots require Understanding of humans’ activitiesUnderstanding of humans’ activities Social interaction skills, i.e., Social interaction skills, i.e., social social

intelligenceintelligence

Most tutoring systems Most tutoring systems understand learner activities, understand learner activities, but lack social intelligencebut lack social intelligence

Challenge: to create guidebots Challenge: to create guidebots with SIwith SI

Without social intelligence:Without social intelligence:


Characteristics of Social Characteristics of Social AgentsAgents

Cognizance of other agentsCognizance of other agents Aware of their beliefs, attitudes, characteristicsAware of their beliefs, attitudes, characteristics

Sensitivity to social relationship, rolesSensitivity to social relationship, roles Sensitivity to social context, exchangeSensitivity to social context, exchange Able to manage interactions, taking Able to manage interactions, taking

above into accountabove into account


Social Intelligence ProjectSocial Intelligence Project

Develop models of social intelligence Develop models of social intelligence for educational softwarefor educational software Track learner cognitive and affective states, Track learner cognitive and affective states,

personality and learning characteristicspersonality and learning characteristics Manage interaction to maximize communication Manage interaction to maximize communication

effectiveness, persuasivenesseffectiveness, persuasiveness Adapt interaction to the learnerAdapt interaction to the learner Track learner-agent interaction as a social Track learner-agent interaction as a social

relationshiprelationship


Architecture of SI SystemArchitecture of SI System


Experimental Basis Experimental Basis

Videotaped sessions of computer-Videotaped sessions of computer-based learning with human tutorsbased learning with human tutors

Students read written tutorial on line, Students read written tutorial on line, completed simulation-based exercisescompleted simulation-based exercises

Tutors sat next to students, observed, Tutors sat next to students, observed, engaged in dialog as appropriateengaged in dialog as appropriate

Multiple sessions with each studentMultiple sessions with each student Intended to provide a model of Intended to provide a model of

appropriate guidebot interactionappropriate guidebot interaction


Conclusions from VideotapesConclusions from Videotapes

Dialog consisted of a series of exchangesDialog consisted of a series of exchanges On student side:On student side:

Differing degrees of understanding, as well as confidenceDiffering degrees of understanding, as well as confidence Differing preferences for social interactionDiffering preferences for social interaction Differing preferred divisions of rolesDiffering preferred divisions of roles

On tutor side:On tutor side: Monitoring learner activityMonitoring learner activity Sensitivity to understanding and confidenceSensitivity to understanding and confidence

On both sides:On both sides: Use of interaction tacticsUse of interaction tactics


Interaction TacticsInteraction Tactics

Intended to achieve a particular Intended to achieve a particular primary goal (communicative, primary goal (communicative, persuasive)persuasive)

Often address additional subsidiary Often address additional subsidiary goalsgoals

Listener response monitored to assess Listener response monitored to assess primary goal achievementprimary goal achievement

Tactics revised in response to Tactics revised in response to achievement failureachievement failure


ExampleExample

Tutor: So it’s asking for regression Student: Right, that wasn’t an option…

there’s no place… Tutor: You want to click on regression here…


Tutor Monitoring of Goal Tutor Monitoring of Goal AchievementAchievement

Look for student’s verbal Look for student’s verbal acknowledgement (or otherwise)acknowledgement (or otherwise)

Look for student actions indicating Look for student actions indicating understandingunderstanding

Rely on expectations of actions both Rely on expectations of actions both before and after before and after


Subsidiary Communicative Subsidiary Communicative GoalsGoals

Tutor phrased comments in order to Tutor phrased comments in order to reinforce learner control and joint reinforce learner control and joint activity. E.g.:activity. E.g.: ““Why don’t you go ahead and read your tutorial Why don’t you go ahead and read your tutorial

factory”factory” ““You want to save the factory”You want to save the factory” ““I’d skip this paragraph”I’d skip this paragraph” ““So why don’t we do that?”So why don’t we do that?”


Some Implications for Some Implications for GuidebotsGuidebots

Need to reduce disruptiveness of Need to reduce disruptiveness of human-guidebot communicationhuman-guidebot communication

Communication should be goal and Communication should be goal and tactic orientedtactic oriented

Communication should be situated in Communication should be situated in work contextwork context

A tactic-oriented approach could also A tactic-oriented approach could also help prevent and repair communication help prevent and repair communication breakdownsbreakdowns


A Tactic-Oriented Learner-A Tactic-Oriented Learner-Guidebot InterfaceGuidebot Interface

Both tutorial view and simulation interface Both tutorial view and simulation interface are instrumentedare instrumented

Learner communicates with guidebotLearner communicates with guidebot Directly using selected questions, typed commentsDirectly using selected questions, typed comments

Encoded as dialog moves using DISCOUNT schemeEncoded as dialog moves using DISCOUNT scheme Utilizes eDrama Learning’s NL parsing techniqueUtilizes eDrama Learning’s NL parsing technique

Indirectly via actions, focus of attentionIndirectly via actions, focus of attention

To be added soon:To be added soon: Vision tracking -> focus of attention monitoringVision tracking -> focus of attention monitoring Dialogs to assess learner confidence, update learner Dialogs to assess learner confidence, update learner

characteristics, assess progress in assessing social characteristics, assess progress in assessing social rolesroles


Next Step: Wizard-of-Oz Next Step: Wizard-of-Oz ExperimentExperiment

Student interacts with agent enhanced Student interacts with agent enhanced interfaceinterface Controlled by remote tutorControlled by remote tutor

Questions:Questions: Does tactic model permit appropriate tutorial Does tactic model permit appropriate tutorial

interaction? interaction? Will subjects interact with the agent the way they Will subjects interact with the agent the way they

interact face to face with tutors?interact face to face with tutors?


AcknowledgmentsAcknowledgments

Faculty:Faculty: Maged Dessouky, Chistoph v. d. Malsburg, Jeff Rickel Maged Dessouky, Chistoph v. d. Malsburg, Jeff Rickel

(USC)(USC) Richard Mayer (UCSB)Richard Mayer (UCSB) Helen Pain (U. of Edinburgh)Helen Pain (U. of Edinburgh)

Research staff:Research staff: Erin Shaw, Kate LaBore, Larry Kite, Kazunori Okada Erin Shaw, Kate LaBore, Larry Kite, Kazunori Okada

(USC)(USC)

Students:Students: Lei Qu, Ning Wang (USC)Lei Qu, Ning Wang (USC) Wauter Bosma, Sander Kole (U. of Twente)Wauter Bosma, Sander Kole (U. of Twente) Jason Finley (UCLA)Jason Finley (UCLA) Heather Collins (UCSB)Heather Collins (UCSB)

usc / information sciences institute making pedagogical agents more socially intelligent lewis...

Documents