integrating intelligent assistants into human teams katia sycara the robotics institute carnegie...
TRANSCRIPT
Integrating Intelligent Assistants into Human Teams
Katia Sycara
The Robotics Institute
Carnegie Mellon University
Pittsburgh, PA 15213
(412) 268-8225
www.cs.cmu.edu/~softagents
Michael Lewis
School of Information Sciences
University of Pittsburgh
Pittsburgh, PA 15260
(412) 624-9426
www.pitt.edu/~cmlewis
Team Members CMU
Liren Chen
Somesh Jha
Rande Shern
Dajun Zeng
Keith Decker
Anadeep Pannu
Vandana Verma
Prasad Chalasani
Kostya Domashnev
Onn Shehory
Team Members U. of Pittsburgh
Michael Lewis (PI)
Terry Lenox
Emily Roth
Talk Outline
• Goals
• Potential Impact for the Navy
• Approach
• Research Issues
• Progress
• Plan for Next Year
Overall Research Goal Increase the effectiveness of joint Command and Control Teams
through the incorporation of Agent Technology in environments that are:
• distributed
• time stressed
• uncertain
• open (information sources, communication links and agents dynamically appear and disappear)
Team members are distributed in terms of:
• time and space
• expertise
Impacts for Navy
• Reduce time for a C2 team to arrive at a decision
• Allow C2 teams to consider a broader range of alternatives
• Enable C2 teams to flexibly manage contingencies (replan, repair)
• Reduce time for a C2 team to form a shared model of the situation
• Reduce individual and team errors
• Support team cohesion and team work skills
• Increase overall team performance
Transition Opportunities
• Maritime Crisis planning
• Target identification training
• Air campaign planning
• Strike planning
• Aircraft maintenance
Overall Approach
• develop an adaptive, self-organizing collection of Intelligent Agents (the RETSINA infrastructure) that interact with the humans and each other.
– integrate multimedia information management and decision support
– anticipate and satisfy human information processing and problem solving needs
– perform real-time synchronization of human actions
– notify about significant changes in the environment
– adapt to user, task and situation
• develop model libraries of individual and team tasks
• develop verifiable useful human-agent interaction techniques
Overall Research Issues
• Agents and Agent Interactions
• Human Agent Interaction
• Information Filtering and Integration
Overall Research Issues: Agents and Agent Interactions
• interleaving planning, replanning, execution monitoring and information gathering in a multiagent setting
• single agent architecture and self-awareness
• agent coordination scheme
• finding appropriate agents
• agent interoperability
• agent-to-agent task delegation protocols
• learning through agent interactions
Overall Research Issues: Human Agent Interaction
• agent-based team aiding
• functional allocation between humans and agents (insert agents into military simulations and perform controlled experiments with human subjects to assess utility)
• human-agent trust
• development of task models (graphical task editor)
• user-guided instantiation of agents (agent editor)
Insert TeamAiding.ppt
Overall Research Issues: Information Filtering and Integration
• learning and tracking multiple interests of users
• increase relevance of retrieved information (refinement key words, relevance feedback, summary of most important information in documents)
• detecting ``interesting'' patterns from multiple data sources
• information integration and conflict resolution
Retsina Functional Organization
Characteristics of RETSINA Agents
• Agents act autonomously to accomplish objectives
– Goal-directed
– Taskable
– Running unassisted for long periods
– Proactive & Reactive
Characteristics of RETSINA Agents (Contd.)
• Agents engage in peer-to-peer interactions
– Agents are taskable, i.e. users or other agents can delegate tasks to them, user acceptability and trust an important issue
– Can interact as cooperative teams or self-interested individuals
– Interaction protocols
– Coordination Strategies
– Negotiation Protocols
• Agents adapt to their environment, user, task and each other
– Adapt both at the individual level and at the societal level
– Employ Alternate Methods
– Learn from (and about) users and each other
Progress
• RETSINA system infrastructure development
– Java implementation
• RETSINA agent architecture
– increased planning sophistication in individual agents
• Middle agents
• Agent interaction protocols
Middle Agent Types
PreferencesInitially Known By
Provider Only Provider +Middle Agent
Provider + Middle +Requester
Requester Only (Broadcaster) “Front-Agent” Matchmaker
Requester +Middle Agent
Anonymizer Broker Recommender
Requestor +Middle + Provider
Blackboard Introducer/Bodyguard
Arbitrator
Service Parameters Initially Known By
Service providers have capabilities and service parametersService requesters have service request and preferences
Retsina Agent Architecture
RETSINA Planning Mechanisms
• hierarchical task network-based formalism
• library of task reduction schemas
– alternative task reductions
– contingent plans, loops
• incremental task reduction, interleaved with execution
– information gathered during execution directs future planning
• resource and temporal constraints
A task Structure (Advertisement Task Structure)
Progress (Contd.)
• Agent interoperability
– language for capability advertisement (Aardvark)
– agent name server and distributed matchmakingª
• Human Agent Interaction
– Task Editor
– Agent Editor
– Human Agent Trust
– Team TANDEM experiments
________________________
ª www.cs.cmu.edu/~softagents/retsina/ans
Insert Aardvark.ppt:language for capability advertisement
Insert Interact.ppt: Agent Editor
Progress (Contd.)
• Applications
– Information filtering: Webmateª, DVINA
– Agents in team aiding: ModSAF, multiagent air patrol, agent-aided aircraft maintenance
___________________________
ª www.cs.cmu.edu/~softagents/webmate
This application is done in collaboration with the CMU wearable computer project.
ModSAF Vision
Insert AirMain.ppt:Aircraft Maintenance Task
Overview of the WebMate System
• Use the multiple TF-IDF vectors to keep track of user interests in different domains which are automatically learned
• Use the trigger pair model to automatically extract relevant words for refining search
• The user can provide multiple pages as relevance guidance for information search
Insert WebMate.ppt(more detailed description)
Insert WebMateDemo.ppt(detailed description of WebMate demo)
Overview of Informedia
• One of the six Digital Libraries Initiative projects funded by the NSF, DARPA, NASA and others in collaboration with WQED
• A multimedia library that will consist of over one thousand hours of digital video, audio, images, text and other related materials
• Uses combined speech, language and image understanding technology to transcribe, segment and index the linear video.
Plans for Next Year
• Continue enhancing the functionality of individual agents (e.g., more sophisticated planning mechanisms)
• Improve the robustness of the RETSINA infrastructure
• Finish the implementation of the agent advertisement language (Aardvark)
• Refine agent task delegation framework, particularly contingent task delegation
• Investigate situation-dependent agent coordination strategies
• Investigate information- and action-based conflict resolution
• Expand the ModSAF team-aiding scenarios by introducing agents of additional types and functionalities
Plans for Next Year (Contd.)
• Develop explicit agent tasking mechanisms
• Identify appropriate indexing mechanisms for task structure cases
• Expand the functionalities of agent editor
• Automatically learn individual and team coordination patterns from team activity traces
Plan for Integrating the Parts of CMU MURI
• Work with U. of Pittsburgh to identify additional agent requirements needed for agent-based team aiding
• U. of Pittsburgh will test the effectiveness of agent-based team aiding in ModSAF scenarios with human subjects
• Incorporate multimedia information from Informedia into agent-based team aiding
• Use the wearable computers as the platform for running the collaborative aircraft maintenance agents