scaling human robot teams
DESCRIPTION
Scaling Human Robot Teams. Prasanna Velagapudi Paul Scerri Katia Sycara Mike Lewis Robotics Institute Carnegie Mellon University Pittsburgh, PA. Large Multiagent Teams. 1000s of robots, agents, and people Must collaborate to complete complex tasks. Search and Rescue. Disaster - PowerPoint PPT PresentationTRANSCRIPT
Scaling Human Robot Teams
Prasanna VelagapudiPaul Scerri
Katia SycaraMike Lewis
Robotics InstituteCarnegie Mellon University
Pittsburgh, PA
Large Multiagent Teams
• 1000s of robots, agents, and people
• Must collaborate to complete complex tasks
Large Multiagent Teams
Large Multiagent Teams
• Network Constraints
Large Multiagent Teams
• Human Information Needs
Network Constraints
• Networks affect human interface design– Limited bandwidth– Significant latency– Lossy transmission– Partial/transient connectivity
Network Constraints
• How can we design robust tasks?– Feasible under network constraints– Tolerant of latency– Within bandwidth constraints– Robust to changes in information
Network Constraints
• Humans are a limited resource
Network Constraints
• Humans are a limited resource– Centralized, expensive– Limited attention and workload– Penalties for context switching– Necessary for certain tasks
• Complex visual perception• Meta-knowledge
Network Constraints
• How do we maximize the effectiveness of humans in these systems with respect to network constraints?
MrCSMulti-robot Control System
MrCSMulti-robot Control System
Waypoint Waypoint NavigationNavigation
TeleoperationTeleoperation
Video/ Video/ Image Image ViewerViewer
Status Status WindowWindow
Map Map OverviewOverview
Victims Found in USAR Task
Number of
Victims
[Velagapudi et al, IROS ’08]
Task decomposition[Velagapudi et al, IROS ’08]
Network Constraints
• How we divide tasks between agents may affect performance– What is the best way to factor tasks?– Where should we focus autonomy?
Large Multiagent Teams
• Human Information Needs
Human Information Needs
• Human operators need information to make good decisions
• In small teams, send everyone everything
• This doesn’t work in large systems
Human Information Needs
• Sensor raw datarates – Proprioception
• < 1kbps
– RADAR/LIDAR• 100kbps – 20Mbps
– Video• 300kbps – 80Mbps
Human Information Needs
• Can’t transmit every bit of information– Selectively forward data
• How do agents decide which pieces of information are important?
– Fuse the data• What information are we losing when we fuse
data?
Asynchronous Imagery
• Inspired by planetary robotic solutions– Limited bandwidth– High latency
• Multiple photographs from single location– Maximizes coverage– Can be mapped to virtual pan-tilt-zoom camera
Asynchronous Imagery
• Streaming Mode • Panorama Mode
Panoramas stored for later viewingPanoramas stored for later viewingStreaming live videoStreaming live video
[Velagapudi et al, ACHI ’08]
Victims Found[Velagapudi et al, ACHI ’08]
Average Average # of # of
victims victims foundfound
Accuracy ThresholdAccuracy Threshold
11
22
33
44
55
66
Within Within 0.75m0.75m
Within 1mWithin 1m Within 1.5mWithin 1.5m Within 2mWithin 2m00
PanoramaStreaming
Environmental Factors
• Colocated operators get extra information– Exocentric view of other agents– Ecological cues– Positional and scale cues
Conclusion
• Need to consider the practicalities of large network systems when designing for humans.
• Need to consider human needs when designing algorithms for large network systems.
Our Work
Cognitive modeling
• ACT-R models of user data
• Determine– What pieces of information users are using?– Where are the bottlenecks of the system?
Environmental Factors
• Colocated operators get extra information– Exocentric view of other agents– Ecological cues– Positional and scale cues
Utility-based information sharing
• It is hard to describe user information needs
• Agents often don’t know how useful information will be
• Many effective algorithms use information gain or probabilistic mass
• Can we compute utility for information used by people
MrCSMulti-robot Control System
MrCSMulti-robot Control System
Waypoint Waypoint NavigationNavigation
TeleoperationTeleoperation
Video/ Video/ Image Image ViewerViewer
Status Status WindowWindow
Map Map OverviewOverview
Victims Found
Number of
Victims
Task decomposition
NavigationNavigation
SearchSearch
Task decomposition
Asynchronous Data
• One way to address the latency of networks is to transition to asynchronous methods of perception and control.
• Asynchronous imagery– Decouples users from time constraints in
control
Asynchronous Imagery
• Inspired by planetary robotic solutions– Limited bandwidth– High latency
• Multiple photographs from single location– Maximizes coverage– Can be mapped to virtual pan-tilt-zoom camera
Asynchronous Imagery
• Streaming Mode • Panorama Mode
Panoramas stored for later viewingPanoramas stored for later viewingStreaming live videoStreaming live video
Victims Found
Average Average # of # of
victims victims foundfound
Accuracy ThresholdAccuracy Threshold
11
22
33
44
55
66
Within Within 0.75m0.75m
Within 1mWithin 1m Within 1.5mWithin 1.5m Within 2mWithin 2m00
PanoramaStreaming
Tools
• USARSim/MrCS
• VBS2
• Procerus UAVs
• LANdroids
• ACT-R
USARSim
[http://www.sourceforge.net/projects/usarsim]
• Based on UnrealEngine2
• High-fidelity physics• Realistic rendering
– Camera– Laser scanner
(LIDAR)
MrCSMulti-robot Control System
MrCSMulti-robot Control System
Waypoint Waypoint NavigationNavigation
TeleoperationTeleoperation
Video/ Video/ Image Image ViewerViewer
Status Status WindowWindow
Map Map OverviewOverview
VBS2
[http://www.vbs2.com]
• Based on Armed Assault and Operation Flashpoint
• Large scale agent simulation
• “Realistic” rendering– Cameras– Unit movements
Procerus UAVs
• Unicorn UAV• Developed at BYU• Foam EPP flying wing• Fixed and gimbaled
cameras• Integrated with
Machinetta agent middleware for full autonomy
LANdroids Prototype
• Based on iRobot Create platform
• Integrated 5GHz 802.11a based MANET
• Designed for warfighter networking
• Video capable
ACT-R
• Cognitive modeling framework
• Able to create generative models for testing