topic 7: collaborative distributed problem solving application domain type defining a mas...
Post on 11-Jan-2016
224 Views
Preview:
TRANSCRIPT
Topic 7: Collaborative Distributed Problem Solving
application domain type defining a MAS cooperation strategies
The Coordination Problem
Managing the interdependencies between the activities of agents. e.g. You and I both want to
leave the room.
We independently walk towards the door, which can only fit one of us.
I graciously permit you to leave first.
http://media.funmansion.com/funmansion/player/player.php?url=http://media.funmansion.com/content/flv/d4282006/crazy_foreign_traffic.flv
1. Understand application domain type (requirements) application characteristics, requirements, constraints “domain theory”
TOD Task-Oriented Domains WOD Worth-Oriented Domains SOD State-Oriented Domains
2. Define a MAS for the particular system objectives various aspects design issues / considerations interaction situations
3. Define cooperation strategies / examples AGV case examples
1. simple task allocation, task (re)allocation, jointly lifting, traffic mgt, …
1. Understand application domain type- Domain Theory
Application requirements obviously crucial for defining MAS-based software architecture
Domain theory: study of types / domains of MAS
Task Oriented Domains Agents have tasks to achieve Task (re)distribution
State Oriented Domains Goals specify acceptable final states Side effects Joint plan and schedules
Worth Oriented Domains Function rating states’ acceptability Joint plan, schedules, and goal relaxation
1.1 Task Oriented Domains
“Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994)
characteristics a set of agents a set of tasks to achieve a set of resources
agents can achieve tasks without help or interference from each other
however, agents may benefit by sharing some tasks
Task-oriented Domain: Example
Imagine you have 3 children, each of whom needs to be delivered to 3 different schools each morning. Your neighbour has 4 children who also need to be taken to school. Delivery of each child is a task. Assume that one of your children and one of your neighbour’s children both go to the same school. It obviously makes sense for both children to be taken together and only you or your neighbour needs to make the trip.
Task-oriented Domain: Example
Imagine you have 3 children, …
Post menPost OfficePost Office
a
c
d e
21
b
f
Task-oriented Domain: Example
Imagine you have 3 children, … Post men
Database queries Common DatabaseCommon Database
“All female employeeswith more than threechildren.”
2
1
“All female employeesmaking over $50,000 ayear.”
1.2 State-Oriented Domain
State-Oriented Domains actions lead to changes of system state
System objective==> attain a particular system state
either pre-defined/static, either evolving
typical additional requirements (e.g. efficiency, …)
11 22 33
11 22 33
2
1
e.g. slotted blocks worlde.g. furniture moving
State-Oriented Domain
1.3 Worth-Oriented Domains
Worth Oriented Domains actions lead to utility:
function rating states’ acceptability
System objective==> maximize utility
Joint plan, schedules, and goal relaxation
Worth-Oriented Domains
2 22
2
55
34
AB tilehole
obstacle
agents
example: tile world maximize rewards
example: traffic lights maximize traffic throughput
2. Define a MASfor particular system objectives
identify various ‘elements’ of MAS
design issues / considerations
interaction situations
2. Defining a MASfor particular System objectives
identify various aspects in defining the MAS
starting fromsystem objectives / requirements / constraints
define MAS for reaching the objective(s) identify agents identify individual goals identify individual skills (actions, comm, capabilities) identify their relation to resources (insufficient, ...) identify their cooperation
e.g. task allocation, coordination over resources, ...
identify suitable agent architecture
creative
task
2. Defining a MASfor particular System objectives
identify various aspects in defining the MAS
design issues / considerations within application boundaries obviously, i.e. if appl. allows
heterogeneity? all agents can perform tasks, vs. not
splittable tasks? specialization? grouping? distributed planning? task distribution predictiveness (anticipatory vs. in situ)
e.g. for avoiding conflicts over resources e.g. road load, deadlocks, ...
2. Defining a MASfor particular System objectives
identify various aspects in defining the MAS design issues / considerations
interaction situations what is an interaction ?
“dynamic relationship through a set of reciprocal actions” i.e. stem from actions whose consequences have an influence on
the future behaviour of the agents
e.g. agents helping each other data exchange between servers use of a printer by two programs simultaneously
Types of interaction situations
components / criteria for categorization of interaction situations types
compatible and incompatible goals relation to resources agent capacities / skills
Components of interaction situations
compatible and incompatible goals concurrent goals ? or contradictory or even opposed ?
incompatible antagonist situation achieving one goal other goal cannot be achieved
compatible cooperation situation
…
Components of interaction situations …
relation to resources resources: environmental elements which can be used in carrying out an
action objects, tools, space, time, …
limited resources conflicts agents needing the same tool at the same place in the same time programs sharing a CPU vehicles on busy roads
conflict resolution many methods
law of the strongest negotiation
e.g. insurances and court houses to resolve conflicts of interest due to accident
coordinating actions devices, regulations, supplementary actions e.g. traffic lights / highway code / … to avoid conflicts
resource spaceresources needed by A
resources needed by B
Components of interaction situations
…
capacities of agents in relation to tasks can an agent carry out a task alone ? or does it need others ?
particular tasks can be carried out either by a single agents
e.g. moving a block, doing a calculation, reading a file, … better by several agents / only by several agents
e.g. heavy blocks, different expertise, …
beneficial interactions
result can be more than sum of parts
Goals Resources(each has)
Skills Types of situation Involvement
Compatible Sufficient Sufficient Independence Indifference
Compatible Sufficient Insufficient Simple collaboration
Compatible Insufficient Sufficient Obstruction Cooperative
Compatible Insufficient Insufficient Coordinated collaboration
Incompatible Sufficient Sufficient Pure individual competition
Incompatible Sufficient Insufficient Pure collective competition
Antagonism
Incompatible Insufficient Sufficient Individual conflict over resources
Incompatible Insufficient Insufficient Collective conflict over resources
Types of interaction situations
Types of interaction situations:Examples
Compatible goals, insufficient resources, sufficient skills
obstruction e.g. motorway queue
Incompatible goals, insufficient resources, sufficient skills
individual conflict over resources e.g. only one liter of water, promotion, ...
...
3. Cooperation strategies - illustrated in AGV case
application domain type task-oriented domain
defining a MAS AGV agents, transport agents individual goals
AGV agents: perform tasks transport agents: ensure tasks are performed
skills AGV agents: jobs/actions/communication/… transport agents: communication/…
resources warehouse (lanes, stock) communication medium
cooperation …
3. Cooperation strategies - illustrated in AGV case (cont.)
…
interaction situations (in)compatible goals (in)sufficient skills (in)sufficient resources
3. Cooperation strategies - illustrated in AGV case (cont.)
1. (simple) task allocation CNET Gradient fields DynCNET
2. task (re)distribution negotiation
3. resource allocation - traffic management CNET
4. collision avoidance coordination protocol hull projection
5. joint action - jointly lift a (heavy) pallet joint intentions
6. batch order management1. team formation2. distributed planning
3.1 Simple task allocation
situationa task consists of
- a pick job (pick up pallet)- a move job (move towards destination)- a drop job (place pallet on target location)
a task can be executed by any AGVa task needs to be assigned / allocated to an AGV
characteristics“delayed commencement”dynamiccommunication overhead?
Simple task allocation (cont.)
situation
collaboration approaches
“Classic approach”: agents coordinate byexchanging messages
Contract NetDynCNet
Exploit the agent environment
Gradient Fields
Simple task allocation (cont.) situation
collaboration approaches1. Contract Net
transport agent issues request for bids(local broadcast)
if no response: stronger signalAGV agents can place bids
based oncurrent location (distance)other tasksbattery level…
transport agent assigns task to best bid
characteristics…
Simple task allocation (cont.) situation
collaboration approaches
1. Contract Net…
characteristics
communication overhead limited
does not take dynamics/opportunities into account
no rejection of assigned task
no re-allocation if better-suited AGV becomes available
issues
how should an AGV agent deal with multiple concurrent requests?
bid? value of bid? consequences?
ok approach for indiviual tasks, what for large numbers of tasks?
one task at a time
Simple task allocation (cont.) situation
collaboration approaches2. Dynamic Contract Net (DynCNET)
contract net, but taking into account dynamics
characteristicscommunication overheadcan take dynamics/opportunities into account
rejection of assigned taskre-allocation if better-suited AGV becomes available
issueshow should an AGV agent deal with multiple concurrent requests?
bid? value of bid? consequences?ok approach for indiviual tasks, what for large numbers of tasks?one task at a time
New Task Becomes Available
New AGV Becomes Available
DynCNET = Dynamic Contract Net
Simple task allocation (cont.) situation
collaboration approaches3. Gradient Fields
coordination through the environment
physical environment Restricts how agents can exploit the environment
agent environment virtual environment layer enables agents to share information and coordinate their
behavior
characteristicscfr. DynCNETreduced complexity of AGV agent!
Agent Environment for Agents
Exploit the Agent Environment Task Assignment (Field-Based Approach)
Exploit the Agent Environment
Task Assignment
Task Allocation experiments -Test Setting AGV System
AGV system “fish handling” (real scenario) 134 x 134 m; 56 pick/drop locations 14 AGVs; 0.7m/s; load manipulation 5s 140 transports/hour (standard test profile)
Tests Communication load Average waiting time Stress test: perform fixed number of transports as quick as possible
(e.g. handle the arrival of a truck with loads)
Communication Load Average Waiting Time
Test Results
Finished Transports in Stress Test
Two Qualities: Flexibility and Robustness
Flexibility (adapt to changes in the environment) FiTA: implicitly represented in the fields DynCNET: explicit points of choice
Robustness (to message loss) FiTA: graceful degradation DynCNET: requires additional functionality
3.2 Task (re)distribution
situationAGV agents have several tasks assigned to thema set of AGV agents could redistribute tasks amongst
each other for better results
e.g. after using several CNET protocols, the following assignmentexists:
AGV1: task 1 (NE corner), task 2 (SW corner) AGV2: task 3 (NE corner), task 4 (SW corner)
redistributing the tasks could seriously enhance task execution time
AGV1: task 1 (NE corner), task 3 (NE corner) AGV2: task 2 (SW corner), task 4 (SW corner)
Task (re)distribution (cont.)
situation
typical situation for negotiation in Task-Oriented Domains…
Task (re)distribution (cont.)
Task-Oriented Domains (TODs) Defined
a TOD is a triple<T, Ag, c>
where T is the (finite) set of all possible tasks Ag = {1,…,n} is the set of participating agents c = (T) + defines the cost of executing each subset of tasks
an encounter is a collection of tasks<T1,…,Tn>
where Ti T for each i Ag
Deals in TODs
given encounter <T1, T2>, a deal is an allocation of the tasks T1 T2 to the agents 1 and 2
the cost to i of deal = <D1, D2> is c(Di), and will be denoted costi()
the utility of deal to agent i is:utilityi() = c(Ti ) – costi()
the conflict deal, , is the deal <T1, T2> consisting of the tasks originally allocated.note that utilityi() = 0 for all i Ag
deal is individual rational if it weakly dominates the conflict deal
Negotiation
”The process of several agents searching for an agreement”
Reaching consensus
”Rules of Encouter” by Rosenchein and Zlotskin, 1994
Complexity of Negotiations
Some attributes that make the negotiation process complex are:
Multiple attributes: Single attribute (price) – symmetric scenario. Multiple attributes – several inter-related attributes, e.g. buying a
car.
The number of agents and the way they interact: One-to-one, e.g. single buyer and single seller. Many-to-one, e.g. multiple buyers and a single seller, auctions. Many-to-many, e.g. multiple buyers and multiple sellers.
Negotiation Components
Any negotiation setting will have 4 components: Negotiation set: represents the space of possible proposals
that agents can make
Protocol: defines the legal proposals that agents can make
Collection of strategies: (one for each agent) determines what proposals the agent will make
Rule: to determine when an agreement has been reached
The Negotiation Set
the set of deals over which agents negotiate are those that are: individual rational pareto efficient
The Negotiation Set Illustrated
Mechanisms, Protocols, Strategies
Negotiation is governed by a mechanism or a protocol: defines the ”rules of encounter” between the agents the public rules by which the agents will come to
agreements.
Given a particular protocol, how can a particular strategy be designed that individual agents can use?
Monotonic Concession Protocol
Negotiation proceeds in rounds
On round 1, agents simultaneously propose a deal from the negotiation set.
Agreement is reached if one agent finds that the deal proposed by the other agent is at least as good or better than its proposal.
Ai best deal Aj best deal
Monotonic Concession Protocol
If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals.
In round u+1, no agent is allowed to make a proposal that is less preferred by the other agent than the deal proposed at time u.
If neither agent concedes, then negotiation terminates with a conflict deal.
Monotonic Concession Protocol
Advantages: Symmetrically distributed (no agent plays a special role) Ensures convergence It will not go on indefinitely
Disadvantages: Agents can run into conflicts Inefficient – no quarantee that an agreement will be reached
quickly
Key Questions
3 key questions to be answered: What should an agent’s first proposal be?
It’s most preferred deal.
On any given round, who should concede?The agent least willing to risk conflict.
If an agent concedes, then how much should it concede?Just enough to change the balance of risk.
The Risk Factor
One way to think about which agent should concede is to consider how much each has to loose by running into conflict at that point.
Ai best deal Aj best deal
Conflict deal
How much am I willing to risk a conflict?
Maximum loss from conflict
Maximum loss from concession
The Zeuthen Strategy
Uses the risk evaluation strategy
Suppose you have conceded a lot. Then: Your proposal is now close to conflict deal. You are more willing to risk conflict.
An agent will be more willing to risk conflict if the difference in utility between its current proposal and the conflict deal is low.
Degree of willingness to risk a conflict can be defined as:
utility i loses by conceding and accepting j’s offerutility i loses by not conceding and causing conflictRiskt
i =
Nash Equilibrium…
the Zeuthen strategy is in Nash equilibrium under the assumption that one agent is using the strategy the
other can do no better than use it himself…
interesting to the designer of automated agents publicly known strategy
About MCP and Zeuthen Strategies
Advantages: Simple and reflects the way human negotiations work. Stability – in Nash equilibrium – if one agent is using the
strategy, then the other can do no better than using it him/herself.
Disadvantages: Computationally expensive – players need to compute the entire
negotiation set. Communication burden – negotiation process may involve
several steps.
3.3 Resource allocation - traffic management (cont.)
situation
approach
negotiation protocols
delegate MAS - see later
3.4 Collision avoidance
situation
AGV autonomously follow their own routes towards
source / target locations
warehouse layout may contain locations where
only 1 AGVs can pass at a time
e.g. cross roads, close lanes
Collision avoidance (cont.)
situation
approaches collision avoidance communication protocols
many possibilities local decision e.g. based on priority rules (priority to the right) CNET for bidding on ‘critical section’
quite complex in non-trivial situations n vehicles complex warehouse layouts dynamic set of AGVs to take into account needs to be integrated seamlessly in agent architecture… deadlock?
Collision avoidance (cont.)
situation
approaches collision avoidance communication protocols
hull projection environment as controlling entity environment mediates between AGV requests to use a lane
more easily extensible for more vehicles / complex layouts more easily integrated in agent architecture
Hull projection through the virtual environment
3.5 Joint action- jointly lift a (heavy) pallet
situation heavy pallet needs to be transported heavy = requires two AGVs
AGVs need to agree on this AGVs need to stick to the agreement
an AGV cannot suddenly drop the pallet because it sees another opportunity (e.g. another job)
Joint action - jointly lift a (heavy) pallet (cont.)
situation
approach joint intentions (Jennings, Tambe)
Recall intentions important for individual practical reasoning
Now: intentions also important for coordination distinguish
individual intentions (that may be coordinated) from intentions to cooperatively and coordinatedly achieve a goal as a team
commitment associated with an intention future directed, persitent, should not be dropped for no reason conventions exist that regulate when it‘s appropriate to drop an intention e.g. when lifting a heavy pallet together
Coordination through joint intentions (cont.)
Agents need individual commitments, and joint commitment to overall goal
State of joint commitment is distributed over agents
Convention regulates e.g. when joint commitment may be dropped and how other agents need to be informed about change of mind.
More formally: Joint persistant goals (JPGs)
JPG = (goal φ, motivation for goal ψ)
e.g.: φ = „having heavy object lifted onto truck“; ψ = „later transportation“
Conventions next to intentions
Initially each agent believes φ has not been satsified, and believes possibleToDo φ
Until termination condition is met, each agent has goal φ (more precisely intention φ)
Termination condition: It is mutually believed that either φ is satisfied φ is impossible ψ is no longer valid
…
…
Until termination condition is met:
if an agent believes that either the goal is met or the goal is impossible or the motivation no longer holds
it has the goal to make this mutually believed (the goal to convince all others about that)
Examples for JPG-like architectures: ARCHON (Jennings e.a.) Steam (Tambe)
Both encode conventions formally as rules, thus integrating them into the „normal“ reasoning process
3.6 Batch order handling
situation peek load in warehouse e.g. several loaded trucks need to be cleared
10s of pallets need to be transportedsome pallets should stay together (e.g. same colour
material)
complex tasks may need to be decomposed? stack of pallets --> smaller stacks or individual pallets
narrow unload location - coordination required
humans would make a team for efficiency? unloading chain?
Batch order handling (cont.)
situation approach
organisation-based approach coordinated planning
Batch order handling (cont.)
situation approach
organisation-based approach inspired by human organisations define roles, assign/negotiate about roles commitment
e.g. action chain roles:
head of chaintail of chainbody of chaintransport
head (unload)
body (pass on)
body (pass on)
tail (drop off)
transport
transport
Batch order handling (cont.)
situation approach
organisation-based approach
coordinated planning centralized (locally or globally)
plan building and distribution plan merging
distributed (G)PGP - (Generalized) Partial Global Planning (Durfee e.a.) SharedPlans (Grosz, Kraus)
Coordinated planning Centralized Planning
plan building and distribution plan merging
each agent makes local, partial plan (PP) the PPs are communicated to central coordinator
it resolves the conflicts and generates a conflict-free global plan (GP). GP is communicated to all agents. e.g. Georgeff in AAAI conference 1983 (National Conf on AI)
Distributed Planning no central controller each agent has some knowledge or model of other agents It has the capability to piece the PPs together into an agreeable GP e.g. Victor Lesser and group (University of Massachusettes, 1981)
Critique: agents share and process a huge amount of information requires lots of computing and communication resources centralised approaches introduce a single point of failure sometimes plans can also become obsolete very quickly
Conclusion
collaborative distributed problem solving at the heart of multi-agent systems
just a small sample of possible situations / cooperation strategies here …
distributed problem solving solves problems but creates new challenges many coordination situations many possible solutions
in the end: integration in one agent architecture coordination and coherence !!
coordinationManaging inter-dependencies between the activities of agents
coherence:“how well the MAS behaves as a unit along some dimension
of evaluation” (Bond et al.)
top related