1 mechanism design for computationally bounded agents kate larson carnegie mellon university thesis...
TRANSCRIPT
1
Mechanism Design for Computationally Bounded Agents
Kate Larson
Carnegie Mellon University
Thesis Proposal
April 8, 2002
2
Introduction
• Fundamental problem in building open distributed systems:– There are users and agents acting in their own
self interest.– Designer wishes to guarantee optimal system
wide solutions while being unable to enforce particular behavior on users.
3
Introduction
• Techniques from game theory and mechanism design have been very useful:– Routing in networks– Resource allocation in operating systems– Auction design in electronic markets– Bidding agents for electronic auctions– ….
4
Introduction
• However, perfect rationality is often assumed!
• But in real systems…
Hard problemsCost Time constraints
5
Package Delivery and Vehicle Routing
Chicago to Home
Chicago to Toronto
Home to Chicago
Chicago
Toronto
Home
Depot
2
3 4
5
1
The delivery route can be computed.
Toronto to Home
6
Package Delivery and Vehicle Routing
Chicago to Home
Chicago to Toronto
Home to Chicago
Chicago
Toronto
Home
Depot
Toronto to Home
12
3
4
5
The new package can be easily fit into the delivery route. The cost can be kept low, and thus the bid also.
7
Computing of Valuations
• Agents may need to compute their valuations for
(bundles of) goods
– Vehicle routing problem in transportation exchanges
– Manufacturing scheduling problem in procurement
• Value of a set of items
=value of solution with items - value of solution without
• May involve solving multiple NP-complete problems
– Infeasible to solve optimally
8
Incentives• Game theory handles incentives
– System designer specifies the mechanism (auction rules)
– Each agent picks its own strategy
• Computational issues have to be handled also!– Computing as part of an agent’s strategy
• Interaction between incentives and computing– Equilibrium for rational agents is not always one
for computationally bounded agents!
9
My ApproachResource Bounded Reasoning from AI
Game Theory and Mechanism Design
Normative model of bounded rationality. Mechanism design for computationally bounded agents.
10
Thesis Statement
• It is possible to incorporate agents’ computational actions into a game theoretic model.
• Agents’ computational restrictions have a strong impact on the choice of strategies.
• It is possible to design mechanisms that take into account agents’ computational restrictions.
11
Structure
Model of bounded rationality
Game theoretic formulation
Impact on existing mechanisms
Mechanisms for bounded agents
Experiments
12
Outline• A model of bounded rationality
– A normative model of deliberation control for computationally bounded agents
• Game theory for computationally bounded agents
• The impact of agents’ computation • Mechanism design for computationally
bounded agents• Experiments• Schedule
13
Restricted Computational Resources
• Agents have limited computational resources– Deadlines
• All parcels must be delivered by Friday evening!
• Battery on my laptop only lasts 3 hours
– Cost associated with computation and deliberation
• Sending jobs to rendering farms
14
Anytime Algorithms• Anytime algorithms can be used to
approximate valuations– Anytime algorithms can return a solution at any time– Solution improves over time Allows a trade off between computing time and
solution quality
• Examples– Iterative refinement algorithms: Local search,
simulated annealing– Search algorithms: Depth first search, branch and
bound
15
Performance Profiles
• Performance profiles describe how computation changes the solution– Characterize the quality of an algorithm’s output as a
function of computing time
• Performance profile representation is important– Earlier methods were not normative: They did not
capture all possible ways an agent can control its deliberation
16
Performance Profiles
Computing time
Solution quality
Deterministic performance profile
Solution quality
Variance introduced by different problem instances
Computing time
[Horvitz 87, 89, Dean & Boddy 89]
Optimum
17
Conditioning on solution quality so far [Hansen & Zilberstein 96]
Ignores conditioning on the path
Table-based Representation of Uncertainty in Performance Profiles
Computing time
Solutionquality
[Zilberstein & Russell 91, 96]
.08 .19 .24
.15 .30 .17 .39
.16 .10 .16 .25 .30 .22
.08 .04 .17 .20 .22 .30 .24 .19 .15
.09 .10 .20 .22 .23 .37 .31 .13 .15
.11 .14 .33 .18 .21 .18 .08
.22 .17 .25 .24 .15 .13
.40 .31 .15 .19 .05
.15 .20 .03
.03
18
Performance Profile Tree
0
4
2
4
5
10
7
15
20
A
P(B|A) B
5
CP(C|A)Solution Quality
Value Node
Random Node
P(1)
P(2)
P(3)
[Larson & Sandholm 2001a]
Stores value of solution so far
Captures uncertainty from random algorithms, unknown algorithms
19
Properties of Performance Profile Trees
• Normative - allows conditioning on everything including– Path of solution quality– Path of other solution features– Problem instance features
• Can model uncertainty from– Randomized algorithms– Lack of knowledge about what algorithms other
agents use– Problem instances
20
Outline• A model of bounded rationality• Game theory for computationally bounded agents
– Roles of computation– Strategic computing– Deliberation equilibrium
• Effects of agents’ computation • Mechanism design for computationally bounded
agents• Experiments• Schedule
21
Roles of Computing• Improving:
– Agents compute to improve valuations
– Computation changes the valuation
• Example: Solving TSP’s to find cost of best route
• Refining:– Agents compute to
refine valuations– Computation
changes agents’ beliefs about valuations
• Example: Researching if an art work is a forgery
22
Roles of Computation• For computationally bounded agents,
computing has several strategic roles:1. Improves or refines the solution to the agent’s
own problem
2. Reduces uncertainty as to what future computing steps will bring
3. Improves the agent’s knowledge about others’ valuations
4. Improves the agent’s knowledge about what problems others may have computed on
23
Strategic Computing
• Good estimates of other agents’ valuations can allow an agent to tailor its strategy to achieve higher utility
• Strong strategic computing: Agent uses some of its deliberation resources to compute on others’ problems
• Weak strategic computing: Agent uses information from others’ performance profiles
24
Strategic Computing
• How an agent should allocate its computation can depend on– how others allocate their computation – what actions they all plan on taking
Incorporate computation actions into agents’ strategies. Analyze games for deliberation equilibria
25
Deliberation Equilibria
• A (Nash, dominant, etc.) deliberation equilibrium for computationally bounded agents is a (Nash, dominant, etc.) equilibrium where:1. The agents’ computing strategies form a
(Nash, dominant, etc) equilibrium, and
2. The agents’ noncomputing strategies form a (Nash, dominant, etc) equilibrium.
26
Outline• A model of bounded rationality• Game theory for computationally bounded agents• Effects of agents’ computation
– One-to-One Negotiation (Bargaining)– One-to-Many Negotiation (Auctions)– Many-to-Many Negotiation (Exchanges)
• Mechanism design for computationally bounded agents
• Experiments• Schedule
28
Bargaining[Larson and Sandholm, 2001a]
• Take-it-or-leave-it: One agent makes an offer to the other who can accept or reject.– Determined under what settings agents have
dominant strategies.– Determined under what settings agents will not split
their computation.– Designed algorithms for determining offline, agents
optimal online strategies
29
Bargaining[Larson and Sandholm, 2002]
• Alternating Offers Model: Agents take turns making proposals and counter proposals
• Despite richer strategy space, alternating offers model is similar to take-it-or-leave it model– In (Bayes Nash) deliberation equilibrium, agents behave as
though they were in take-it-or-leave-it setting
30
Future Bargaining Work
• So far have only worked in settings where computation is free but limited– What happens if computation is costly?
Conjecture: Costly computation will not change the results.
31
Auctions and Strategic Computing
yesyesno
Generalized Vickrey
On which agent, bundle pair to allocate next computation step ?
Multiple items
for sale
noEnglish (1st price ascending) yes
yes
no
nonoVickrey (2nd price sealed bid)
yesDutch (1st price descending) yesyes
yesyesyesFirst price sealed-bidSingle item for
sale
Costly computation
Limited computation
Strong strategic computingCounter-speculation by rational agents ?
Auction
mechanism
[Larson and Sandholm 2001b, Larson and Sandholm 2001c]
32
Exchanges• Exchanges are a generalization of
auctions where there are potentially multiple buyers and multiple sellers– Does strategic computing occur in
exchanges?– Does the type of restriction on agents’
computational capabilities determine the occurrence of strategic computation?
Conjecture: Strategic computation can occur with either type of restriction (costly or limited computation).
33
Outline• A model of bounded rationality• Game theory for computationally
bounded agents• Effects of agents’ computation • Mechanism design for computationally
bounded agents• Experiments• Schedule
34
Mechanism Design• So far have focused on standard
mechanisms and determining their strengths and limitations if agents have computational restriction.
• Change focus:1. Mechanisms for computationally
restricted agents.
2. Measures for mechanisms.
35
Issues for Mechanism Design• There are several issues that must be
addressed for computationally restricted agents– What should agents report to the mechanism?
• Valuations once computed?• Valuations agents expect to obtain?• Results of partial computation?
– What is the role of the mechanism?• To specify allocations only?• To specify allocations and computation policies?
36
Issues for Mechanism Design• Does there exist a mechanism that is:
– Strategy proof,– Efficient,– Individually rational, and – Limits wastage of computational resources.
37
Measuring Mechanisms: Miscomputing Ratio
• How does allowing agents free choice of computing policies effect the outcome, o?
• Miscomputing Ratio:
R=SW(o*)/SW(o(NE))
SW(o*)=Social welfare if a global controller enforces computing policies so as to maximize social welfare
SW(o(NE))=Social welfare of worst Nash Eq.
38
Results and Future Work• Results (Vickrey auctions):
– If agents have limited or costly computation then the miscomputing ratio can be unbounded.
– The choice of appropriate cost functions can motivate bidders to choose strategies that maximize social welfare.
• Future Work:– Constraining deadlines or cost functions– Using performance profile trees to predict R– Constraining performance profile trees – Comparing mechanisms using miscomputing ratio, R
39
Outline• A model of bounded rationality• Game theory for computationally
bounded agents• Effects of agents’ computation • Mechanism design for computationally
bounded agents• Experiments• Schedule
40
Experiments
• Goals:– To demonstrate that the performance
profile tree is a feasible model.– To demonstrate that the performance
profile tree model is general.– To determine if strategic computing is an
artifice of analysis or something that occurs in the “real world”.
• Tools for controlling computation and visualization of deliberation and negotiation
41
Status
Model of bounded rationality
Game theoretic formulation
Impact on existing mechanisms
Mechanisms for bounded agents
Experiments
completedpartiallyfuture work
42
Schedule
0 1 2 3 4 5 6 7
Writing
Experiments
Miscomputing Ratio
Mech. Design
Exchanges
Bargaining (Costly)
Sp02 Sm02 F02 Sp03 Sm03 F03
43
Related Publications• Larson and Sandholm, 2001a, Bargaining with Limited
Computation:Deliberation Equilibrium, Artificial Intelligence, 132(2), pp.183-217
• Larson and Sandholm 2001b, Computationally Limited Agents in Auctions, Workshop on Agent based approaches to B2B, Autonomous Agents 2001
• Larson and Sandholm, 2001c, Costly Valuation Computation in Auction, TARK VIII
• Larson and Sandholm, 2002, An Alternating Offers Bargaining Model for Computationally Limited Agents, AAMAS 2002
• Larson and Sandholm, 2002, Miscomputing Ratio: The Social Cost of Selfish Computing, draft.
44
Other Publications
• Larson and Sandholm, 2000, Anytime Coalition Generation: An Average Case Study, Journal of Experimental and Theoretical Artificial Intelligence, 12(2000),23-42.
• Sandholm, Larson, Andersson, Shehory, and Tohme, 1999, Anytime Coalition Structure Generation with Worst Case Guarantees, Artificial Intelligence, (111)1-2, pp209-238.
45
Future Auction Work: Reverse Auctions
• Are they different from standard auctions if agents have bounded computational resources?– Maybe or maybe not!– Reverse auctions differ from auctions in:
• Complexity of winner determination• Communication complexity
– At first look, it does not appear as though bidding strategies are different.
46
Related Work• Parkes 2001, Iterated combinatorial auctions so that
both the computational burden of agents and mechanism are reduced
• Persico 2000, Information acquisition in auctions
• Bergemann and Pesendorfer 2001, Model where the auctioneer controls what information agents have available to them
• Bergemann and Välimäki 2001, Costly information acquisition, but no clear model of how information is obtained
47
Related Work
• Nisan and Ronen 2000: Algorithmic mechanism design. What happens if the mechanism is computationally limited? Investigates ways of introducing approximation algorithms to compute outcomes without loosing incentive compatibility.
• Kfir-Dahav, Monderer, Tennenholtz 2000
48
Experiments• To demonstrate that the performance profile
tree is a feasible model:– Multi-item auction where agents submit bids where
the value is determined by the solution to a TSP problem
– Prebundling to simplify the winner determination problem and to restrict the number of optimization problems agents are faced with
– Focus on having agents use performance profile trees, determining how to make the performance profile tree model feasible for actual problems.
49
Experiments
• To demonstrate that the performance profile tree is general:– Single item auction where agents
valuations of the item is based on schedules they compute.
– Use off the shelf schedulers (Micro-Boss, or a stochastic scheduler from Steve Smith’s group) to show that the techniques I propose are general
50
Experiments• To determine if strategic computation is an
artifice of analysis or something that occurs in the “real world”:– Multi-item reverse auction using data from actual
delivery companies– Agents must solve multiagent vehicle routing
problems– How does one tailor performance profile trees to
actual data without losing their analytical power?– Does strategic computing occur?
51
Game Theory Concepts• Game theory: method of studying systems of
agents in settings where there are strategic interactions
• Game has – Set of agents, A– Set of outcomes, O– Set of actions for each agent
• Strategy: contingency plan that determines what action the agent will take at any point in the game
• Equilibrium is fundamental concept in game theory– Stable point in space of agents’ strategies
52
Game Theory Concepts• Different equilibrium concepts
– Dominant strategy equilibrium• Every agent has a strategy that they are best off playing,
independent of other agents actions
– Nash equilibrium• The strategy one agent plays is optimal given what
strategies other agents are playing
• Mechanism Design: – Study of how to implement good system wide solutions that
involve mulitple self-interested agents
53
Mechanism Design• Mechanism Design:
– Study of how to implement good system wide solutions that involve multiple self-interested agents
• Mechanism: M=(S1,…,Sn,g(.)) is a set of strategies Si available to each agent i, and an outcome function g(s).– Defines strategies available to agents– Describes methods used to determine outcome
based on agents’ strategies
54
Game Theory• Game theory studies phenomena that occur
when decision makers interact– It is assumed that decision makers are strategic.
If he bids $10 then I could bid $11 and win.
If he bids $11 then I could bid $12 and win.
55
Auctions with Individual Bidders
• In auctions where the bidders are individual consumers, valuations are often idiosyncratic
$5 for the bear
$100 for the bear
56
The Two Extremes
Hamlet: “What a piece of work is a man! How noble in reason! How infinite in faculties!”
Hamlet, II.2.319
Puck: “Lord, what fools these mortals be!”
Midsummer Night’s Dream, III.3.116
57
One Solution?
“It is evident that the rational thing to do is to be irrational, where deliberation and estimation cost more than they are worth.”
Frank Knight, Risk, Uncertainty and Profit, 1921