distributed search by agents with personal preferences
DESCRIPTION
Distributed Search by Agents with Personal Preferences. Alon Grubshtein. Lessons learnt from applying distributed constraint reasoning to “realistic” agents with personal preferences. Before we begin…. In this talk:. Constraint Reasoning. Distributed Computing. - PowerPoint PPT PresentationTRANSCRIPT
Ben-Gurion University of the Negev
Department of Computer Science
Distributed Search by Agents with Personal Preferences
Alon Grubshtein
Lessons learnt from applying distributed constraint
reasoning to “realistic” agents with personal preferences
Ben-Gurion University of the NegevDepartment of Computer Science
Before we begin…
Ben-Gurion University of the NegevDepartment of Computer Science
Multi Agent Systems
Constraint Reasoning
Distributed Computing
Distributed
Constraint
Reasoning
In this talk:
Ben-Gurion University of the NegevDepartment of Computer Science
Sometime back in 2006
Check out this great phone I gotI can use it to work on my calendar!!! Who needs a
computer with such phones?
You can even write programs for it…
Lets write a distributed agent to automate meeting coordination
Ben-Gurion University of the NegevDepartment of Computer Science
Constraint reasoning (centralized)
A Constraint Reasoning problem:
• Variables• Domains• Constraints (relations)
A solution concept (target objective)
Ben-Gurion University of the NegevDepartment of Computer Science
Examples
Ben-Gurion University of the NegevDepartment of Computer Science
What’s in a constraint?
Two important classes of problems:
• Constraint Satisfaction (CSP)
• Constraint Optimization (COP)
A satisfying assignment
A minimal cost assignment
Ben-Gurion University of the NegevDepartment of Computer Science
Constraint algorithms
How do we find a solution?
• Enumerate feasible outcomes• Backtracking / Branch and Bound• Intelligent backtracking• Pre processing, forward checking and
heuristics• Local search algorithms
Ben-Gurion University of the NegevDepartment of Computer Science
From centralized to distributed
The problem itself is distributed across computational nodes – agents:• Privacy• “Difficulty”
Ben-Gurion University of the NegevDepartment of Computer Science
Constraint reasoning (distributed)
Distributed Constraint Reasoning (DCR) problem:
• Agents• Variables• Domains• Constraints (relations)
DCSP /
DCOP
Ben-Gurion University of the NegevDepartment of Computer Science
From centralized to distributed
• Computation on separate entities• Communication via messages• Each agent knows only a small
portion of the problem
• Allows for parallel computation
DISTRIBUTED =/= PARALLEL
Ben-Gurion University of the NegevDepartment of Computer Science
DCR algorithms
Ben-Gurion University of the NegevDepartment of Computer Science
Local Search for “real” problems
• Computationally hard• Simplistic myopic algorithms
(“local search”/“adaptive heuristics”)
• Example, DSA:1. Pick a random assignment2. While (stop condition):
a. Send assignment to all neighbors (receive)b. If can improve local state by changing assignment:
change with probability p
Ben-Gurion University of the NegevDepartment of Computer Science
A simple MAS example
Coordinating a meeting (e.g., seminar):
• Two alternatives: Morning or Evening• More participants – better• Prof. Lynn does not care when• If students disagree - morning• Alice prefers morning• Anna prefers evening
Prof. Lynn
Alice
Anna
5 1
0 2
M
M
E
E
AliceAnna
3 0
2 4
M
M
E
E
AliceAnna
5 , 3
1 , 0
0 , 2
2 , 4
M
M
E
E
AliceAnna
Ben-Gurion University of the NegevDepartment of Computer Science
Solving as a DCOP
Alice Anna
5 , 3 1 , 0
0 , 2 2 , 4
M
M
E
E
AliceBob
8 1
2 6
M
M
E
E
AliceBob
What if students can’t/won’t
communicate preferences?
Alice
Anna
M
M M
E
EE
Cost:
8 1 2 6
Ben-Gurion University of the NegevDepartment of Computer Science
Standard model solutions
• Easiest solution: Disclose preferences
• An alternate approach:Add unary constraints
• Problem: Can prove that this approach will fail on some instances
Ben-Gurion University of the NegevDepartment of Computer Science
How its done these days
The PEAV formulation:
A1 A2
x1 x2
x21 x1
2x y
a 3 6
b 7 5 x y
a 4 1
b 2 8mirror variables
hard constraints
x12
x2
x1
x21
• Modified search space Can’t be used with many local search algorithm!
• Requires more space
Ben-Gurion University of the NegevDepartment of Computer Science
Introducing ADCOPs
Different preferences on outcomes are not part of the standard model…
Asymmetric constraintsFormally:
Captures the idea that each agent has a personal “table” with costs/gains of each outcome
Ben-Gurion University of the NegevDepartment of Computer Science
ADCOPs
• ADCOPs:• At least as expressive as existing model• Succinct representation• Used with existing local search algorithms
Search can be improved by introducing cooperation/coordination
Ben-Gurion University of the NegevDepartment of Computer Science
ADCOP Local Search (quality)
0 20 40 60 80 100 120 140 160 180 2002000
3000
4000
5000
6000
7000
8000
9000
10000
11000
12000
MCS-MGMGCA-MGMACLSMGM2MGMDSA
Cycles
Solu
tion
Cost
0 20 40 60 80 100 120 140 160 180 200
Cycles
Solu
tion
Cost
0 20 40 60 80 100 120 140 160 180 200
Cycles
Solu
tion
Cost
DCOP ADCOP
Ben-Gurion University of the NegevDepartment of Computer Science
Multi Agent Systems
Constraint Reasoning
Distributed Computing
Distributed Constraint Reasoning
Ben-Gurion University of the NegevDepartment of Computer Science
Rethinking agents joint objective
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 3010
20
30
40
50
60
70
Normalized
Number of meetings
% Q
uality
Utilitarian
Difference in best and worst gains – Meeting Scheduling Problem
Ben-Gurion University of the NegevDepartment of Computer Science
Agreeing on an outcome(what is a fair solution?)
• Game Theory defines stable points:
• Assumptions:1. Self interested2. Rational (some knowledge)
Ben-Gurion University of the NegevDepartment of Computer Science
Graphical Games
• ADCOPs are Games played on a Graph• Closely related to Graphical Games• ADCOPs:
No knowledge assumed Agents are cooperative An even more succinct representation
• Can use DCR techniques to solve a game theoretic multi agent problem!
5 , 3
1 , 0
0 , 2
2 , 4
M
M
E
E
AliceAnna
Ben-Gurion University of the NegevDepartment of Computer Science
Asynchronous Nash BackTracking (ANT)
• Transform a MAS to a Distributed Constraint Problem
• A distributed, asynchronous, non-binary, asymmetric search
Two symmetric constraints
Three asymmetric constraints
Ben-Gurion University of the NegevDepartment of Computer Science
ANT
• A satisfaction problem• Inspired by ABT (ABT-1ph)• A solution always exists• Guaranteed to find an epsilon NE:
• More efficient than other distributed GG solvers
Ben-Gurion University of the NegevDepartment of Computer Science
Multi Agent Systems
Constraint Reasoning
Distributed Computing
Distributed Constraint Reasoning
Ben-Gurion University of the NegevDepartment of Computer Science
The quality of a stable solution
• A stable solution is not necessarily a good one…
• Why is that?• Competitive solution for cooperative
agents?
A2 \ A1 Cooperate Defect
Cooperate 4,4 6,0Defect 0,6 1,1
Ben-Gurion University of the NegevDepartment of Computer Science
Agreeing on an outcome(what is a fair solution?)
Cooperativ
e
Competitive
Utilitarian, Egalitarian, Leximin,…
Stable points: Nash (pure/mixed), Bayesian, Strong, Correlated, …
Ben-Gurion University of the NegevDepartment of Computer Science
A different approach
assume cooperation but try to incentivize agents by examining personal goals
• “Cost of Cooperation”• Baseline search
Ben-Gurion University of the NegevDepartment of Computer Science
The Cost of Cooperation (CoC) criteria:
The difference in an agent’s gain from the worst equilibrium (from its point of view) outcome and from cooperatively solving the problem
Possible solutions
U2(x)
U1(x)
Pareto front
Optimal solution (max sum)
Nash equilibrium solutions
Non positive CoC solutions
Ben-Gurion University of the NegevDepartment of Computer Science
u1=med
C2
A simple P2P example
a1 a8
a2
a3
a4
a5
a6
a7
C1
FS F
u1=low
u2=high
F S
S
• Agents only interact with neighbors (unknown topology)
• An agent’s gain is lowered when exerting resources on sharing (S)
• Gain is maximized if an agent can free ride the efforts of other agents (F)
• Gain is lowest if no one shares
Ben-Gurion University of the NegevDepartment of Computer Science
Competitive and Cooperative solutions
a1
a8
a2
a3
a4
a5
a6
a7
S S
F F
FF
F F
0.3 0.3
1 1
1
1 1
1
Cooperative Solution
a1 a8
a2
a3
a4
a5
a6
a7
S S
F F
FF
F F
0.3 0.31 1
0
0 0
0
A Bayesian stable solution (possible)
Ben-Gurion University of the NegevDepartment of Computer Science
Cost of Cooperation solution
• An improvement can be guaranteed (proved) for a set of interactions!
a1 a8
a2
a3
a4
a5
a6
a7
S S
F
F
F F
0.3 0.31 1
0
0S
S
0.3
0.3
35
Ben-Gurion University of the NegevDepartment of Computer Science
Applied to network games
ADCOP (CoC)
Maximizing utilities
Ben-Gurion University of the NegevDepartment of Computer Science
Multi Agent Systems
Constraint Reasoning
Distributed Computing
Distributed Constraint Reasoning
Ben-Gurion University of the NegevDepartment of Computer Science
Limits of the CoC approach
• So far we have seen several solutions:Fully cooperative (Utilitarian)Stable (Epsilon Nash Equilibrium)A combination:
Non positive Cost of Cooperation
• However…
A2 \ A1 Left Right
Up 2,5 4,1Down 6,1 0,3
Mixed NE: (1/2,1/3)Gain: (3, 7/3)
NO Cost of Cooperation solution!
Ben-Gurion University of the NegevDepartment of Computer Science
A framework for partial cooperation
• Agents gain is different • Do not “improve cooperatively”• Define cooperation with respect to
some baseline solution• Agents must agree on the baseline
(may need to apply a simple search algorithm).
Ben-Gurion University of the NegevDepartment of Computer Science
Modes of cooperation
• Define modes of cooperation within an Interaction Process: Non-Cooperative (NC) – agents are driven by
their own goals and act rationally. Can serve as a baseline solution
Guaranteed Personal Benefit (GPB) – agents seek an agreement and may take irrational steps. Guarantees a Pareto improvement
λ-cooperation – agents agree to a bounded loss from their NC gain (up to some predefined λ)
Ben-Gurion University of the NegevDepartment of Computer Science
Local Search and Partial Cooperation
Maintain threshold/guarantee:
1. Incorporate with distributed “anytime”
Can use any LS algorithm Focus on exploration
2. Tailor an algorithm maintain invariant (begins in a “legal”
state)
Ben-Gurion University of the NegevDepartment of Computer Science
Evaluation
Three key parameters:1. Compromise levels
(lambda)2. Agents’ degree3. Costs distribution
0 200 400 600 800 10001200140016001800200016000
17000
18000
19000
20000
21000
22000
23000
24000
Goods-MGMAGCMGMMGM2MCS-MGMGCA-MGM
Cycles
Solu
tion
Cost
Ben-Gurion University of the NegevDepartment of Computer Science
Multi Agent Systems
Constraint Reasoning
Distributed Computing
Distributed Constraint Reasoning
Ben-Gurion University of the NegevDepartment of Computer Science
SUMMARY & CONCLUSIONS
Ben-Gurion University of the NegevDepartment of Computer Science
Summary
Multi Agent Problem
DCSP/DCOP
Utilitarian(Minimal sum of costs)
Asymmetric Constraints
Stableε-Nash Equilibrium
Non positive Cost of Cooperation
Partial Cooperation
Represent
ation
Algorithm
Objecti
ve
Ben-Gurion University of the NegevDepartment of Computer Science
Conclusions
• Three points (‘up and down the ladder of abstraction’):
1. How to model the problem2. How does the model effect the means to
find a solution 3. What is a solution?
• Rethinking basic assumptions• Applying well established models to
simple realistic settings can reveal many of its shortcoming
Ben-Gurion University of the NegevDepartment of Computer Science
Journal publications:• Arnon Netzer, Alon Grubshtein and Amnon Meisels, “Concurrent Forward Bounding”, Artificial Intelligence, Vol. 193, pp. 186-216,
2012.• Roie Zivan, Alon Grubshtein and Amnon Meisels, “Hybrid Search for Dynamically changing CSPs”, Constraints, special issue on
constraint satisfaction for planning and Scheduling, Vol. 16, num. 3, pp. 228-249, 2011.• Alon Grubshtein and Amnon Meisels, “Cost of Cooperation for Scheduling Meetings”, Journal of Computer Science and Information
System (ComSIS), Vol. 7, num. 3, pp. 551-567, 2010.Conference and workshops publications :• Alon Grubshtein and Amnon Meisels, “Finding a Nash Equilibrium by Asynchronous Backtracking”, 18th Intl. Conf. on Principles and
Practice of Constraint Programming (CP’12), pp. 925-940, Quebec city, Canada, Oct. 2012.• Alon Grubshtein, Roie Zivan and Amnon Meisels, “Partial Cooperation in Multi Agent Local Search”, 20th European Conf. on Artificial
Intelligence,pp.378-383, Montpellier France, Aug. 2012• Roie Zivan, Alon Grubshtein, Michal Friedman and Amnon Meisels, “Partial Cooperation in Multi Agent Search”, (Extended Abstract)
Proc. 11th intern. Conf. Autonom. Agents Multi agent Sys. (AAMAS’12), Valencia, Spain.• Alon Grubshtein and Amnon Meisels, “A Distributed Cooperative Approach for Optimizing a Family of Network Games”, Proc. of the 5th
Intern. Symp. on Intelligent Distributed Computing (IDC’11), Delft, the Netherlands, pp. 49-62, October 2011.• Alon Grubshtein and Amnon Meisels, “A Distributed Cooperative Approach for Optimizing a Network Game”, Proc. 13th Intern.
Workshop on Dist. Constraints Reasoning (DCR’11), Barcelona, Spain, June 2011.• Alon Grubshtein, Nir Herschorn, Arnon Netzer, Guy Rapaport, Guy Yaffe and Amnon Meisels, “The Distributed Constraints (DisCo)
Simulation Tool”, Proc. 13th Intern. Workshop on Dist. Constraints Reasoning (DCR’11), Barcelona, Spain, June 2011.• Alon Grubshtein and Amnon Meisels, “Cooperation Mechanism for a Network Game”, Proc. 3rd Intern. Conf. Agents and AI
(ICAART’11), Rome, Italy, pp. 336-341, January 2011.• Alon Grubshtein, Tal Grinshpoun, Amnon Meisels and Roie Zivan, “Local Search for Distributed Asymmetric Optimization” , Proc. 9th
intern. Conf. Autonom. Agents Multi agent Sys. (AAMAS’10), Toronto, Canada, pp. 1015-1022, May 2010.• Arnon Netzer, Amnon Meisels and Alon Grubshtein, “Concurrent Forward Bounding for DCOPs”, Proc. 12th Intern. Workshop on Dist.
Constraints Reasoning (DCR’10) at AAMAS’10, Toronto, May 2010.• Alon Grubshtein, Nurit Gal-Oz, Tal Grinshpoun, Amnon Meisels and Roie Zivan, “Manipulating Recommendation Lists by Global
Considerations”, Proc. 2nd Intern. Conf. Agents and AI (ICAART’10),pp. 135-142, Valencia, Spain, January 2010.• Alon Grubshtein and Amnon Meisels, “Cost of Cooperation for Scheduling Meetings”, Proc. 3rdIntern.Symp. Intell. Dist. Comp. (IDC’09),
Vol. 237, pp. 227-236, Ayia Napa, Cyprus, October 2009.• Alon Grubshtein, Tal Grinshpoun, Amnon Meisels and Roie Zivan, “Asymmetric Distributed Constraint Optimization”, Proc. 11th Intern.
Workshop on Dist. Constraints Reasoning (DCR’09) at IJCAI-09, Pasadena CA, July 2009.• Ehud Gudes, Nurit Gal-Oz and Alon Grubshtein, “Methods for Computing Trust and Reputation While Preserving Privacy”, Proc. Data
and App. Security XXIII, 23rd Ann. IFIP WG 11.3 Working Conf. (DBSEC’09), Vol. 5645, pp. 291-298, Montreal, Canada, July 2009.• Amir Gershman, Alon Grubshtein, Amnon Meisels and Roie Zivan, “Scheduling Meetings by Agents”, Proc.7thintern. Conf. Practice and
Theory Auto. Timetabling (PATAT’08), Montreal, August 2008.
Thank you!