human-robot interface design for large swarms of...
TRANSCRIPT
![Page 1: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/1.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Speaking Swarmish
Physical
∧Human-Robot Interface Design for Large Swarms of Autonomous Mobile Robots
James McLurkin1, Jennifer Smith2, James Frankel3, David Sotkowitz4, David Blau5, Brian Schmidt6
1, 5, 6 2, 3, 4
![Page 2: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/2.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
My Position•My Axe: As a community, we don’t spend enough time
working on (or give enough respect to) user and development interfaces for robotic systems
•As the field matures, popular interfaces will inevitability arise. We can either design thoughtful interfaces (good) or let them arise spontaneously (bad)
•This work describes a set of simple user interfaces that use Physical I/O to interface an unaided user to large* groups of robots
•Oooo! A new Acronym:
HMRI = Human Multi-Robot InterfaceNow, we’re ready to get some work done!
*Large ≈ 100. Your definition may vary.
![Page 3: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/3.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
The HMRI Problem•Requirements๐Get information to all the robots (programs, commands)
๐Get information about what all the robots are doing (internal state)
๐Get information from all the robots (mission sensors)
•Some Terms:๐Virtual I/O: Interface with robots through computer
๐Physical I/O: Interface with robots directly
๐Development Interface: Interface for developers
๐User Interface: Interface for users
![Page 4: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/4.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Related Work
Daily, Cho, Martin, Payton, 2002 “World
Embedded Interfaces”
Collett and MacDonald, 2006, “Developer
Oriented Visualisation of a Robot Program”
StarCraft, Blizzard Entertainment, 1999
Kismet, Breazeal et. al., 1998-200X
![Page 5: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/5.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
HRI Design Space
![Page 6: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/6.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
This Work•Physical Infrastructure for hands-free operation
•Centralized Control Software for development, debugging and operation
•Physical I/O for in situ interaction๐ in situ = Allow the user to watch the robots while
reading telemetry about individual and group behaviors
๐Displays on robots are useless on multi-robot systems
๐The Human perceptual system can process a vast amount of multi-modal real-time data
๐Allow the trained, but unaided user to “read” the swarm
![Page 7: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/7.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Physical Infrastructure for HMRI•Problem: The robots outnumber the humans๐Anything you have to do to one robot, you have to do to
all of them. This does not scale well.
•Solution: Hands-Free Design๐The SwarmBot has dedicated hardware for the physical
user interface
๐Basic Swarm-level tasks must be hands-free:Remote Power On/Off
Autonomous Charging
Remote Programming
Efficient Transportation
Software for Centralized Command, Control, and Data Collection
![Page 8: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/8.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
SwarmBot Hardware Behavior LEDs
Radio
Charging Contacts
Camera
IR Inter-Robot Communications
1.1 Watt Audio System
Bump Skirt
orange = for physical UIgrey = for autonomous
operation
![Page 9: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/9.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
The Swarm “Extrastructure”
![Page 10: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/10.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Docking
![Page 11: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/11.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Long-Range Navigation
![Page 12: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/12.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Very Long-Range Navigation
![Page 13: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/13.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Remote Downloading System•[diagram of remote downloading procedure goes here]
(waive hands furiously)
•Status tones are separated by 25ms, and are still clear and easy to disambiguate
![Page 14: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/14.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
SwarmCraft GUI•Virtual I/O: Real-Time data collection and visualization•Necessary, but not sufficient
![Page 15: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/15.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
SwarmCraft GUI
![Page 16: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/16.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
SwarmCraft GUI
![Page 17: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/17.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Group Exercise: Conversational Swarmish
Three different color lights: red, green, blue
+ Two blinking patterns: sine and square
+ Many frequencies .
= 108 patterns than can each be read in ≤ 500ms
![Page 18: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/18.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Clumping
![Page 19: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/19.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Orientation
![Page 20: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/20.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Navigation
![Page 21: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/21.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Dispersing
![Page 22: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/22.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Dispersion
![Page 23: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/23.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
BubbleLine
![Page 24: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/24.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
BubbleLine: Flip
![Page 25: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/25.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
DTA: Extreme Comms-Line
![Page 26: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/26.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
DTA: Extreme Comms
![Page 27: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/27.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
DTA: Card Dealers (Clumping)
![Page 28: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/28.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
DTA: Tree-Recolor
![Page 29: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/29.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Application: “Directed Dispersion”
•FrontierBot•WallBot
•InteriorBot•ChargingBot•GuideBot
![Page 30: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/30.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Directed Dispersion: Big Room Explore
• Blue=Frontiers Red=Interior Green=Charging White=Guides
![Page 31: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/31.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Directed Dispersion: Big Room Return
![Page 32: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/32.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Directed Dispersion: Complex Rooms
![Page 33: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/33.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Limitations of Swarmish•Have not found a good way to represent continuous
quantities
•Current lexicon has about 108 usable patterns
•Sounds need to be carefully designed to work together ⇒ lots of trial and error
![Page 34: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/34.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
Future Work•Careful analysis of the human perceptual system and its
bandwidth limitations
•How to optimally leverage multi-modal input๐Rich non-speech and non-text interfaces
•Simple Augmented Reality Devices๐Hand-held tools to query robots: Tricoder-like device
![Page 35: Human-Robot Interface Design for Large Swarms of ...people.csail.mit.edu/jamesm/publications/McLurkinSmithEt...James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish Speaking](https://reader033.vdocuments.us/reader033/viewer/2022053008/5f0bdcd47e708231d4329527/html5/thumbnails/35.jpg)
James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish
ConclusionHMRI ⇒ Usability ⇒ Efficiency ⇒
((Debug Cycles)/Second)++ ⇒
((Ideas Tested)/Second)++ ⇒
(Software Quality)++ ⇒
(Publications/Second)++
Questions?