Download - Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen
![Page 1: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/1.jpg)
Mutual Empowerment in Human-Agent-Robot Teams
16 December 2010
HART Workshop
Jurriaan van Diggelen
![Page 2: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/2.jpg)
Problem statement
• Achieve more with less people
• Automation can help to:– Make better use of available semi-structured
information sources– Support decision makers in dealing with the
complexity of problems (war amongst the people)
![Page 3: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/3.jpg)
The big number cruncher
• Monolithic approach, BNC replaces existing infrastructure
• AI-complete
Sensor data
Twitter data
UAV images
Problem solution
![Page 4: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/4.jpg)
Towards a human-machine team solution
• Solution must be provided by a human machine team
• Mutual empowerment seeks to improve team performance by:– Compensating weaknesses of humans and
machines– Optimizing strengths of humans and
machines
![Page 5: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/5.jpg)
Types of Mutual Empowerment
human machineHMI
Intelligent Interfaces
human machine
CM
I
CC
I
HMI
User empowerment
DistributedArtificialIntelligence
CollectiveIntelligence
![Page 6: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/6.jpg)
ME handbook
Goal
![Page 7: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/7.jpg)
![Page 8: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/8.jpg)
![Page 9: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/9.jpg)
Methodology
Functional design
PrototypingValidation
•Use cases•Claims•Cognitive requirements•Ontologies•Performance measures•Tests/benchmarks
•System requirements•Functional modules•RDF interface specifications•Prototypes
•Mixed reality validation•Data collection
Tool support
Domain Exploration
•Domain•Human Factors•Technology
![Page 10: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/10.jpg)
Situated Cognitive Engineering
• Methodology supports– Incremental design– Reuse of earlier work (Prototypes, tests,
requirements, use cases) – Collaborative development
![Page 11: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/11.jpg)
Example
![Page 12: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/12.jpg)
Phase 1: domain exploration
• Domain– USAR– UGV, UAV– Operators in field
• Human Factors– Maintaining situation awareness– Cognitive overload– Adaptive teams
• Technology– Collaborative tagging, crowd sourcing– Mixed initiative systems– Adaptive/ adaptable automation
![Page 13: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/13.jpg)
Phase 2: Functional design (1)• Use cases
• Cognitive requirements
• Claims
UC 23• UAV classifies camera image as victim with certainty-level Unsure• Operator of Robot1 is notified of the potential victim and views the
camera images • Operator of Robot1 classifies the image as victim with certainty level Certain• Operator of Robot2 is notified about the victim• …
CR 5.1 Uncertainty managementOperators and agents can publish and change the certainty value of informationUse cases: UC 23
CR 5.1 • + improves situation awareness of operators and agents• - increases cognitive taskload
![Page 14: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/14.jpg)
Phase 2: Functional design (2)
• Ontologies
• Performance measures– E.g. situation awareness measure
• Tests/benchmarks– Test for evaluating performance
something
action event item
victimrobot
![Page 15: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/15.jpg)
Phase 3: Prototyping
• Develop system requirements that implement the cognitive requirements.
• Bundle system requirements in functional modules.
• Reuse existing base platform
![Page 16: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/16.jpg)
Trex
![Page 17: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/17.jpg)
Trex• Filter: which
data do you want to see? selection of semantic tags in Sparql
• Projection: How do you want to see the data?graphical object with attachment-points for semantic tags
![Page 18: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/18.jpg)
Functional modules supported by Trex
• User configurable information filters• User configurable information visualization• Realtime semi-structured data exploration• Collective relevance assessment• Uncertainty management• Human-in-the-loop AI
P Q R S THuman Machine Crowd Machine
Human-in-the-loop AI
![Page 19: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/19.jpg)
DEMO
![Page 20: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/20.jpg)
Future work
• Develop functional modules for:– Joint conflict resolution– Adaptive Interruptiveness– Network awareness– Policy awareness– Capability awareness– Activity awareness
![Page 21: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/21.jpg)
Conclusion
• Mutual Empowerment library provides a flexible way to– Increase application possibilities of AI– Employ potential of collective intelligence– Reuse and structure our knowledge of
human-machine collaboration tools
![Page 22: Mutual Empowerment in Human-Agent-Robot Teams 16 December 2010 HART Workshop Jurriaan van Diggelen](https://reader035.vdocuments.us/reader035/viewer/2022062217/5697bfec1a28abf838cb8722/html5/thumbnails/22.jpg)
Technology InvestigationDomain Analysis Human Factors
Metrics
Cognitive Requirements
ClaimsUse cases
OntologiesTests
Exploration
Functional design
Prototyping
Core functions
Functional modules
System Requirements
PrototypeRDF interfaces
TestingSimulation Test participants Empirical results