www.CrewAssistant.com 1
The MECA ProjectReasoning Agents on Mars
Leo Breebaart (S&T)
12 October 2006
www.CrewAssistant.com 2
The MECA Consortium
TNO Human Factors• Human behaviour and performance in technical high-demand environments;
methods to attune the environment to (momentary) human capacities.
S&T bv • Software development company with specific expertise on creating system
health management applications.
OK Systems• Software development company focusing on technology areas of AI, user
interfaces, databases and web-based systems (specially scheduling systems).
EADS-ST• Technologies, development, production and utilization of manned and
unmanned space missions, including experiments, space transportation systems, propulsion systems and support of these systems concerning operations, maintenance and mission handling.
www.CrewAssistant.com 3
Project Goal
The MECA Project aims to:
• Provide support to and increase autonomy of astronauts while:– Executing complex tasks– In a hostile and large unknown environment– Possibly disconnected from Mission Control
www.CrewAssistant.com 4
Objective & Vision
Objective: support mission goals (without injury or loss of life) by• empowering the cognitive capacities of human-machine teams
during planetary exploration missions. • in order to cope autonomously with unexpected, complex and
potentially hazardous situations.
Vision: crew support that • acts in a ubiquitous computing environment • as “electronic partner”, helping the crew
– to assess the situation, – to determine a suitable course of actions to solve a problem, – to safeguard the astronaut from failures.
www.CrewAssistant.com 5
MECA and Agents
Where does agent technology fit in?
• A MECA System can be considered as an instance of a group of software agents.
• The MECA Architecture will make use of the outcome of agent technology research.
www.CrewAssistant.com 6
This presentation
• Phase 1 (2005-2006)– “MECA 2017”– RB: Requirements Baseline
• Phase 2 (2006-2007)– “MECA 2007”– Demonstrator Prototype– Refined RB
www.CrewAssistant.com 7
Why the need for support?
• Current astronaut support rather antiquated.
• Long-distance manned explorations impose new challenges.
www.CrewAssistant.com 8
Background
www.CrewAssistant.com 9
State of the Art
LAPAP SCOPE
Our legacy:…
But MECA should extend this….
Operational Procedures Hardcopy +
www.CrewAssistant.com 10
MECA design process
Background • Operability-based design• Anticipate new Intelligent Interfaces• Anticipate new HFE standard
Approach• Human-Operation centered• Enabling Technology focus• Iterative process (specify-test-refine
cycles)• From abstract to detailed specifications• Sound theoretical and empirical foundation
10 -- 25 yrs
www.CrewAssistant.com 11
Example Scenario
Human and System Health Management
www.CrewAssistant.com 12
EVA 2 astronauts 2km from habitat, sample collection for scientific experiment.
Habitat
= MECA unit
www.CrewAssistant.com 13
MECA indicates a problem with the temperature regulation of one of the space suits.
Habitat
www.CrewAssistant.com 14
Astronaut, MECA and spacesuit collaborate on finding the cause of the failure.
Habitat
• MECA interrogates spacesuit to obtain more parameters for diagnosis.
www.CrewAssistant.com 15
The heater of the space suit is damaged and cannot be repaired locally.
Habitat
www.CrewAssistant.com 16
MECA simulates the consequences of the broken heater.
Habitat
• MECA predicts that astronaut has to be brought to habitat and has a risk of fainting.
www.CrewAssistant.com 17
Astronaut's MECA Unit and Habitat MECA Unit reschedule and plan safe return.
Habitat
• Habitat is preparing for treatment of hypothermic astronaut (preparing and activating resources)• Astronaut MECA asks for help of transporting astronaut (since he is likely to faint)
www.CrewAssistant.com 18
MECA Rovers respond to call for help, MECA habitat chooses rover
Habitat
• MECA habitat chooses rover that has enough power/resources to pick astronaut up and transport to habitat (fuel, location, speed, pressurized or unpressurized rover etc.).
www.CrewAssistant.com 19
MECA communicates adjusted schedule to astronaut(s) in the right manner (keeping in mind cognitive task load).
Habitat
• Hypothermic astronaut has a high task load due to high stress levels, MECA Unit shall adapt communication according to task load and affective state.• The astronaut accompanying him can be given information in a different manner.
www.CrewAssistant.com 20
Astronaut faints before rover arrives. MECA communicates fainting of astronaut.
Habitat
www.CrewAssistant.com 21
Astronaut has fainted earlier than predicted, MECA rover has to find a way to pick up astronaut (sensemaking).
Habitat
www.CrewAssistant.com 22
MECA of fainted astronaut and other astronaut collaborate to get fainted astronaut in rover
Habitat
www.CrewAssistant.com 23
Hypothermic astronaut is transported to habitat by rover.
Habitat
www.CrewAssistant.com 24
Hypothermic astronaut is in habitat being treated.
Habitat
www.CrewAssistant.com 25
Iterative Requirements Analysis
MECA Requirements
Refine HMC Performance
Simulation-Based Evaluation
Envisioned Technology
Human Factors Knowledge
OperationalDemands
PrototypeCurrent
Technology
System DesignReview
Comments
ScientificDiscourse
www.CrewAssistant.com 26
Operational Demands
MECA shall take account of:
• the high-level operation goals – e.g., safe return to earth
• the environment – e.g., radiation and social monotony
• task performance – e.g., people will get seriously ill
www.CrewAssistant.com 27
Human Factors Demands
• Cognitive Task Load
• Situation Awareness and Sense Making
• Diversity of Cognitive Capacities
• Trust and Emotion
• Collaboration
• Crew Resource Management
• Decision Making
www.CrewAssistant.com 28
Envisioned Technology (1)
habitatMEVhabitatfacility
MCC
Orbiter
crewrover
= meca unit
An infrastructure will be available for automatic distribution of data, software and reference documents.
www.CrewAssistant.com 29
Envisioned Technology (2)
• MECA shall make use of this infrastructure and can cope with possible failures
• Continuous analysis and extrapolation of emerging technologies, e.g. – multi-agent systems– automatic planning and scheduling– model-based health management
• Technical requirements, such as– maturity– graceful degradation– maintainability– fault tolerance
www.CrewAssistant.com 30
Requirements Baseline
Generic task level requirements
• Implement key Human Factor knowledge.• Enhance autonomy of individual actors and groups of
actors. • Support collaboration among the different actors. • Hardware and software systems must be highly reliable. • Manage environmental information.
www.CrewAssistant.com 31
Current Requirements
• Different types of requirements:– task level, – functional, – user interface, – technical interface, – operational and – technical requirements.
• All requirements are linked to use cases.
www.CrewAssistant.com 32
Use Case Template
ID 78
Title Hypothermic astronaut
Level Level 0
Goal Treat hypothermic astronaut that is on EVA…
Actor Personal MECA, MECA habitat, astronaut in habitat, rover..
Pre-condition Astronaut on EVA is hypothermic
Post-condition Hypothermic astronaut is in medical facility being treated
Frequency Not frequent
Main success scenario
Personal MECA detects hypothermia and communicates to hypothermic astronaut …
Alternative scenario
.. , Doctor is not in habitat, MECA will ask astronaut_1 to prepare…
Comment Derived from RefDoc3…
www.CrewAssistant.com 33
Outline of Functional Requirements
Process MECA function
Information Gathering detect needs for operations and training
Goal Setting select and prioritize goals for operations and training
Plan Generation or Selection generate plans, or select pre-generated plans and procedures, for operations and training
Plan Evaluation evaluate operational and training plans
Prepare for Execution prepare the resources for executing operational and training plans
Execution execute operational and training plans
Processing Evaluation of Results
evaluate execution results for operational and/or training purposes
www.CrewAssistant.com 34
Incremental and Iterative Prototyping
• Human-, task- and context-driven design and evaluation.
• Both MECA and the humans will show mutual adaptive behaviour, which effects should be well tested with realistic scenarios.
• Prototyping, simulation and testing is therefore essential to establish a sound and coherent set of requirements.
• A game-based simulation environment can provide an effective platform for testing the human-machine collaboration (e.g. the Unreal Tournament game-engine) in combination with other simulators.
www.CrewAssistant.com 35
Evaluation Criteria
• Long-term human in the loop effects• Standard usability measures
– effectiveness – efficiency – satisfaction – learnability
• Human experience measures, such as – situation awareness (perception, comprehension and
projection)– trust (persistence and behavioural competence,
servitude, and the understanding of the machine)– emotion (arousal and valence)
www.CrewAssistant.com 36
Break?
• Break!
www.CrewAssistant.com 37
Background
habitatMEVhabitatfacility
MCC
Orbiter
crewrover
= meca unit
Reasoning will involve complex system behaviorand scarce resources
www.CrewAssistant.com 38
Background
• Time is a scarce resource• Control of a wide array of tasks and experiments• Control of each task and experiment is complex
– Nominal -- Correct sequence of steps– Off-nominal -- Detect, isolate, and compensate failures
• Summary: – Optimize crew time by supporting control activities
• Check plan execution• Support control of complex equipment especially in the case of
malfunctions• Resource usage
www.CrewAssistant.com 39
Human decision making
Process UnderControl
Understand Act
•Situational awareness, e.g.:• Warning: At current rates, resources will be depleted within two days…
MECA Unit
• Must reduce load by 10% but must also generate more oxygen…• I lowered the consumption, but I still don’t see any change..
Complex, uncertain, and dangerous world
plan
www.CrewAssistant.com 40
Traditional support
Operational procedure:
verify
action
Next action
okay
Notokay
off-nominal proceduresguided bymission control
Based on operational procedure and Mission Control
www.CrewAssistant.com 41
Operational procedures
Nominal control
Off-nominal:Fault detection, isolation,repair
Operational procedures:•Huge pile of papers•No feedback of payload
www.CrewAssistant.com 42
The 2017 MECA system
• The 2017 MECA system helps the astronaut to make the right decisions in situations that :– Are novel, or near-novel, e.g., because equipment is
failing– Are complex, e.g., effects of the decision are hard to
predict• Intricate interactions between processes, systems,
components, …• The effects are noticeable only late in time
– Require human--human and human--machine cooperation
www.CrewAssistant.com 43
Automated support
Process UnderControl
Understand Act
MECAUnit
plan
•Improve situational awareness by interpretation measurement data, in particular: fault diagnosis
•Determine alternative plan (e.g., repair, reconfiguration)
•High level plan --> control action•Monitor plan execution
www.CrewAssistant.com 44
Automated support
Validate success of plan step
Automatic fault detection and isolationAdequate repair procedurespresentation of background info
Automatic (re)plan
www.CrewAssistant.com 45
Automated support
• Operational procedure is generalized by Actions of a Plan– Plan describes pre- and postconditions– Postconditions used for verification plan step and diagnosis
• Automated diagnosis
• Automated reconfiguration (repair, or redundancy management)
pre post pre post pre post
Supply pressure to fuel and oxidizer
Pb = high
www.CrewAssistant.com 46
Layered Reasoning
www.CrewAssistant.com 47
2017 MECA: Collaboration
Process UnderControl
Understand Act
PUC
Understand Act
I need your capabilities to repair the equipment
Resource level fromteam mate is enough tocomplete task…
www.CrewAssistant.com 48
Plan-based Architecture
PUCstate
PUC
Health mode &
Resource
Healthestimation
Resourcemonitor
Plan representation
goal
subgoal subgoal
task task
Plan/schedule
Reconfiguresystem
PossibleActions,Safety
constraints,utility
Derivecapabilities
Goals
www.CrewAssistant.com 49
MECA Unit Functional Decomposition
MMI
procedureexecutor
(PE)
what-ifsimulation &
rehearsal (PR)
executionmonitoring (SM)
sense-making
reconfiguration(CO)
physical equipment/facility interface (DA)
planning &scheduling
(PS)
derivecapabilities
(SA)
health &status
monitoring(FDIR)
inter-faceto
otherMECAunits
CTL
resourcemanager
healthmonitoring
plan
model
status+
history
collaboration
(simulated) PUC
other MECAunits
other MECAunits
usecases
www.CrewAssistant.com 50
Software Architecture
www.CrewAssistant.com 51
MECA 2017 is not MECA 2007!
For example MECA 2017 (‘Real’ MECA) MECA 2007 (Demonstrator)
Software Technologies Make generic choices: ontologies, agents, intelligent reasoning support.
Choose specific instantiations: OWL/RDF, Jade, Uptime.
Hardware Platforms Wearable / ubiquitous / brain-jack interface...
Tablet PC / Laptop.
Autonomy of components in Process under Control
Components can be independently autonomous (i.e. non-MECA intelligence).
No (interface to) 3rd-party intelligence, all autonomy implemented / controlled by MECA.
Reasoning Optimal combination of proactive and reactive.
Combination of proactive and reactive.
Unit complexity Completely hierarchical systems-of-systems nature of MECA Units.
Only simple aggregation and limited levels of containment for MECA Units.
Cooperation Adaptable to dynamic leadership, based on model on teamwork.
Limited models of teamwork.
Cognitive task load Aware of task load based on physiological sensors , imposed loads, and actual astronaut reactions.
Limit awareness of task load based on limited indicators.
www.CrewAssistant.com 52
Challenge Areas
• Data Architecture: how to ensure worldviews of different levels of MECA Units remain compatible? How is the worldview represented?
• (Unit) Collaboration: how to synchronize, authenticate, de-conflict, negotiate work share, re-plan when two MECA Units ‘meet up’?
• Good interfaces will be key!
www.CrewAssistant.com 53
Ontologies and reasoning
• Use ontologies to describe and reason about:– Design of MECA– Capabilities and Domains
• Equipment (PUC, Resources)• Environment• Tasks• Time• Communications• etc...
• Formal ontology representation allows:– automated validation of instance data– generation of documentation, templates, code– fit in with future “semantic web” approaches
www.CrewAssistant.com 54
Ontology example
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .@prefix meca: <http:///esa.int/meca#> .
# OWL Classes:
meca:Unit a owl:Class ;
# Owl Properties:
meca:isPersonalUnit a owl:DatatypeProperty; rdfs:domain meca:Unit; rdfs:range xsd:boolean .
meca:autonomyMode a owl:ObjectProperty; rdfs:domain meca:Unit .
meca:synchronisationPolicy a owl:ObjectProperty; rdfs:domain meca:Unit.
www.CrewAssistant.com 55
Planning & Scheduling - Design Concepts
• Mission objectives are predefined, and default general plans are arranged before mission begins.
• Short term goals and plans will be decided and controlled autonomously by crew during the mission, adapting to changing circumstances.
• Continuous planning, scheduling and rescheduling of the tasks assigned to a heterogeneous team is a complex process that consumes time and determines the chances of success.
• MECA should assist in the generation of plans, reduce the overload of crewmembers, check constraints and conditions, and optimize the usage of time and resources.
• Crewmembers should have quick and easy access to all information related with plans, be able to evaluate alternatives and make changes.
www.CrewAssistant.com 56
Planning & Scheduling - External Interfaces
www.CrewAssistant.com 57
Planning & Scheduling - Internal Components
• Rule system: editable by users, enabling explanation of decisions• Inference Engine: using Rules to generate and optimize Plans and Schedules• Representation: in a format that facilitates distribution & collaboration
www.CrewAssistant.com 58
Demonstrator – First Design Iteration
• Decompose scenario into Sequence Diagrams (bring time dimension into play, identify actors)
• Decompose scenario into Activity / State Diagrams
• Illustrate actor timelines with storyboards
• Identify order and depth to which storyboards are to be implemented
www.CrewAssistant.com 59
Demonstrator – Software Proposals
• Application Framework: Use EADS Java Framework for personal MECA Unit GUIs.
• Use Jade for implementing agent-aspects of Units themselves.
• Use Uptime for Model-Based Programming: autonomous planning, reconfiguration and fault diagnosis.
• Use RDF/XML+OWL for Knowledgebase.• Use Students’ models for PuC Components• ...
www.CrewAssistant.com 60
Questions?
Photo: E
SA
/Aurora