Preference Elicitation in Combinatorial Auctions
Tuomas Sandholm Carnegie Mellon University
Computer Science Department
(papers on this topic available via www.cs.cmu.edu/~sandholm)
Combinatorial auction
• Can bid on combinations of items [Rassenti,Smith & Bulfin 82]...
– Bidder’s perspective
• Allows bidder to express what she really wants– Avoids exposure problems– No need for lookahead / counterspeculationing of items
– Auctioneer’s perspective:
• Automated optimal bundling
• Binary winner determination problem: – Label bids as winning or losing so as to maximize sum of bid prices
» Each item can be allocated to at most one bid– NP-complete [Rothkopf et al 98 using Karp 72]
– Inapproximable [Sandholm IJCAI-99, AIJ-02 using Hastad 99]
Another complex problem in combinatorial auctions: “Revelation problem”
• In direct-revelation mechanisms (e.g. VCG), bidders bid on all 2#items combinations– Need to compute the valuation for exponentially many
combination• Each valuation computation can be NP-complete
• For example if a carrier company bids on trucking tasks: TRACONET [Sandholm AAAI-93]
– Need to communicate the bids
– Need to reveal the bids• Loss of privacy & strategic info
Revelation problem …
• Agents need to decide what to bid on– Waste effort on counter-speculation
– Waste effort making losing bids
– Fail to make bids that would have won • Reduces economic efficiency & revenue
Clearing algorithm
What info is needed from an agent depends on what others have revealed
Elicitor
Conen & S. IJCAI-01 workshop on Econ. Agents, Models & Mechanisms, ACMEC-01
Elicitor decides what to ask next based on answers it has received so far
$ 1,000 for
$ 1,500 for
? for
Elicitor [Conen & Sandholm 2001]
• Have auctioneer incrementally elicit information from bidders– based on the info received from bidders so far
Elicitation …
• Goal: minimize elicitation– Regardless of computational / storage cost– (Future work: explore tradeoffs across these)
• Approach:– At each phase:
• Elicitor decides what to ask (and from which bidder)• Elicitor asks that and propagates the answer in its data
structures• Elicitor checks whether the auction can already be cleared
optimally given the information in hand
Setting
Combinatorial auction: m items for sale• Private values auction, no allocative externalities
• Each bidder i has value function, vi: 2m R
• Unique valuations (to ease presentation)
Outline
• Query policy dependent (= rank lattice based) elicitor algorithms
• Policy independent elicitor algorithms
• Note: Private values model
Rank lattice
[1,1]
[1,2] [2,1]
[2,3]
[3,1]
[3,2]
[2,4]
[3,4] [4,3]
[3,3] [4,2]
[4,4]
[1,4] [4,1]
[2,2][1,3]
Infeasible
Feasible
Dominated
Bundle Ø A B ABRank for Agent 1 4 2 3 1Rank for Agent 2 4 3 2 1
A search algorithm for the rank latticeAlgorithm PAR “PAReto optimal“
OPEN [(1,...,1)]while OPEN [] do
Remove(c,OPEN); SUC suc(c);if Feasible(c) then
PAR PAR {c}; Remove(SUC,OPEN)else foreach node SUC do
if node OPEN and Undominated(node,PAR)then Append(node,OPEN)
• Thrm. Finds all feasible Pareto-undominated allocations (if bidders’ utility functions are injective)
• Welfare maximizing solution(s) can be selected as a post-processor by evaluating those allocations – Call this hybrid algorithm MPAR (for “maximizing” PAR)
Value-augmented rank lattice
Bundle Ø A B ABValue for Agent 1 0 4 3 8Value for Agent 2 0 1 6 9
17
14 13
9 10 12
98
[1,1]
[1,2] [2,1]
[2,3]
[3,1]
[3,2]
[2,4]
[3,4] [4,3]
[3,3] [4,2]
[4,4]
[1,4] [4,1]
[2,2][1,3]
Search algorithm family for the value-augmented rank lattice
Algorithm EBF “Efficient Best First“OPEN {(1,...,1)}loop if |OPEN| = 1 then c combination in OPEN else
M {k OPEN | v(k) = maxnode OPEN v(node) }if |M| 1 node M with Feasible(node) then return nodeelse choose c M such that c is not dominated by any node M OPEN OPEN \ {c} if Feasible(c) then return c else foreach node suc(c) do
if node OPEN then OPEN OPEN {node}
• From now on, assume quasilinear utility functions • Thrm. Any EBF algorithm finds welfare maximizing allocations• Thrm. VCG payments can be determined from the information already elicited
Best & worst case elicitation effort
• Best case: rank vector (1,...,1) is feasible – One bundle query to each agent, no value queries
– (VCG payments: 0)
• Thrm. Any EBF algorithm requires at worst (2#items #bidders – #bidders#items)/2 + 1 value queries– Proof idea. Upper part of the lattice is infeasible and
not less in value than the solution
• Not surprising because worst-case communication complexity of the problem is exponential [Nisan 01]
EBF minimizes feasibility checks
• Def: An algorithm is admissible if it always finds a welfare maximizing allocation
• Def: An algorithm is admissibly equipped if it only has– value queries, and– a feasibility function on rank vectors, and– a successor function on rank vectors
• Thrm: There is no admissible, admissibly equipped algorithm that requires fewer feasibility checks (for every problem instance) than any EBF algorithm
MPAR minimizes value queries
• Thrm. No admissible, admissibly equipped algorithm (that calls the valuation function for bundles in feasible rank vectors only) will require fewer value queries than MPAR
Differential-revelation• Extension of EBF• Information elicited: differences between valuations
– Hides sensitive value information
• Motivation: max ∑ vi(Xi) min ∑ [vi(r-1(1)) – vi(Xi)]
– Maximizing sum of value Minimizing difference between value of best ranked bundle and bundle in the allocation
• Thrm. Differences suffice for determining welfare maximizing allocations & VCG payments
• 2 low-revelation incremental ex post incentive compatible mechanisms ...
Differential elicitation ...
• Questions (start at rank 1)– “tell me the bundle at the current rank”– “tell me the difference in value of that bundle and
the best bundle“• increment rank
• Natural sequence: from “good” to “bad” bundles
Differential elicitation ...
• Variation: Bitwise decrement mechanism– Is the difference in value between the best
bundle and the bundle at the current rank greater than δ? • if „yes“ increment δ, requires min. Increment• allows establishing a „bit stream“ (yes/no
answers)
Policy independent elicitor algorithms
Some of our elicitor’s query types
• Order information: Which bundle do you prefer, A or B?
• Value information: What is your valuation for bundle A? (Answer: Exact or Bounds)
• Rank information: – What is the rank of bundle b?
– What bundle is at rank x?
– Given bundle b, what is the next lower (higher) ranked bundle?
General Algorithmic Framework for Elicitation
Algorithm Solve(Y,G) while not Done(Y,G) do
o = SelectOp(Y,G) Choose questionI = PerformOp(o,N) Ask bidderG = Propagate(I,G) Update data structures with answerY = Candidates(Y,G) Curtail set of candidate allocations
Output: Y – set of optimal allocations
Input: Y – set of candidate allocations (some may turn out infeasible, some suboptimal)
G – partially augmented order graph
(Partially) Augmented Order Graph
Ø B A AB∞ ∞ ∞ ∞
0 0 0 0
Ø A B AB0 0 1 6
4 0 3 6 2 6 1 9
Agent1
Agent2
A
B
Ø
B>
∞
1
∞
1
Allocations
AB6
1 9
Rank Upper Bound
Lower Bound
[1,1]
[1,2] [2,1]
[2,3]
[3,1]
[3,2]
[2,4]
[3,4] [4,3]
[3,3] [4,2]
[4,4]
[1,4] [1,4]
[2,2][1,3]
Some interesting procedures for combining different types of info
Constraint Network111
110 101 011
100 010 001
000
1 per agent
Constraint Network111
110 101 011
100 010 001
000
[0,]
[0,][0,][0,]
[0,] [0,] [0,]
[0]
Lower bound
Upper bound
Constraint Propagation111
110 101 011
100 010 001
000
[0,]
[0,][0,][5]
[0,] [0,] [0,]
[0]
vi(110)=5
Constraint Propagation111
110 101 011
100 010 001
000
[0,]
[0,][0,][5]
[0,] [0,] [0,]
[0]
vi(110)=5
Constraint Propagation111
110 101 011
100 010 001
000
[5,]
[0,][0,][5]
[0,5] [0,5] [0,]
[0]
vi(110)=5
Constraint Propagation111
110 101 011
100 010 001
000
[5,]
[0,][0,][5]
[0,5] [0,5] [0,]
[0]
vi(110)=5
Constraint Propagation111
110 101 011
100 010 001
000
[5,]
[0,][0,][5]
[0,5] [0,5] [0,]
[0 ,5]
vi(110)=5
000
Additional edges from order queries
What to query should the elicitor ask (next) ?
• Simplest answer: value query– Ask for the value of a bundle vi(b)
• How to pick b, i?– First try: Randomly (subject to not asking
queries whose answer can be inferred from info already elicited)
• Thrm. If the full-revelation (direct) mechanism makes Q value queries and the best value-elicitation policy makes q queries, we make
value queries
– Proof idea: We have q red balls, and the remaining balls are blue; how many balls do we draw before removing all q red balls?
– Universal revelation reducer
• Is it tight? Run experiments
Random elicitation with value queries only
Experimental setup for all graphs in this talk
• Simulations:– Draw agents’ valuation functions from a
random distribution where free disposal is honored
– Run the auction: auctioneer asks queries of agents, agents look up answer from a file
– Each point on plots is average of 10 runs
Random elicitation
• Not much better than theoretical bound
agentsitems3 4 5 6 7 8 92 10 2 3 4 5 6
queries
1
10
100
1000
20
40
60
80
queries
Full revelation
Queries
2 agents 4 items
Querying random allocatable bundle-agent pairs only…
• Bundle-agent pair (b,i) is allocatable if some yet potentially optimal allocation allocates bundle b to agent i
• How to pick (b,i)?– Pick a random allocatable one
• Asking only allocatable bundles means throwing out some queries
• Thrm. This restriction causes the policy to make at worst twice as many expected queries as the unrestricted random elicitor. (Tight)– Proof idea: These ignored queries are either
• Not useful to ask, or• Useful, but we would have had low probability of asking it, so no big difference in
expectation
Querying random allocatable bundle-agent pairs only…
• Much better– Almost (#items / 2) fewer queries than unrestricted random– Vanishingly small fraction of all queries asked !– Subexponential number of queries
agentsitems3 4 5 6 7 8 92 10 2 3 4 5 6
queries
1
10
100
1000
20
40
60
80
queries
Full revelation
Queries
Hudson & S. AMEC-02
Best value query elicitation policy so far
Number of items for sale
Fraction of values queried before optimal allocation found & proven
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5 6 7 8 9 10
2 agents
3 agents
4 agents
Focus on allocations that have highest upper bound.Ask a (b,i) that is part of such an allocation and among them, pick the one that affects (via free disposal) the largest number of bundles in such allocations.
Order queries
• Order query: “agent i, is bundle b worth more to you than bundle b’ ?”– Motivation: Often easier to answer than value
queries
• Order queries are insufficient for determining welfare maximizing allocations– How to interleave order, value queries?
• How to choose i, b, b’ ?
Value and order queries
• Interleave:– 1 value query (of random allocatable agent-bundle pair)– 1 order query (pick arbitrary allocatable i, b, b’ )
• To evaluate, in the graphs we have– value query costs 1– order query costs 0.1
Value and order queries …
• Elicitation cost reduced compared to value queries only– Cost reduction depends on relative costs of order & value queries
agentsitems3 4 5 6 7 8 92 10 2 3 4 5 6
queries
1
10
100
1000
20
40
60
80
queries
Order costValue costTotal costFull re
velation
Rank lattice based elicitation• Go down the rank lattice in best-first order (= EBF)• Performance not as good as value-based; why?
– #nodes in rank lattice is 2#bidders #items
– #feasible nodes is only #bidders#items
agentsitems4 6 82 10 2 3 4 5 6
queries
1
10
100
1000
20
40
60
80
queries
Full revelation
Queries
12
Bound-approximation queries• Often bidders can determine their valuations more
precisely by allocating more time to deliberation [S. AAAI-93, ICMAS-95, ICMAS-96, IJEC-00; Larson & S. TARK-01, AGENTS-01 workshop, SITE-02; Parkes IJCAI workshop-99]
• Get better bounds UBi(b) and LBi(b) with more time spent deliberating
• Idea: don’t ask for exact info if it is not necessary• Query: “agent i, hint: spend t time units tightening the
upper (lower) bound on b”• How to choose i, b, t, UB or LB ?
– For simplicity, in the experiment graph, fix t = 0.2 time units (1 unit gives exact)
Bound-approximation query policy
• Could choose the query randomly
• More sophisticated policy does slightly better:– Choose query that will change the bounds on
allocatable bundles the most– Don’t know exactly how much bounds will change– Assume all legal answers equiprobable, sample to get
expectation
Bound-approximation queries• This policy does quite well
• Future work: try other related policies
agentsitems3 4 5 6 7 8 92 10 2 3 4 5 6
queries
1
10
100
1000
40
80
120
160
queries
Full revelation
Query cost
Supplementing bound-approximation queries with order queries
• Integrated as before– Computationally more expensive
agentsitems3 4 5 6 7 8 92 10 2 3 4 5 6
queries
1
10
100
1000
40
80
120
160
queries
Full revelation
Total cost
Value cost Order cost
Incentive compatibility
• Elicitor’s questions leak information about others’ preferences
• Can be made ex post incentive compatible– Ask enough questions to determine VCG prices
• Worst approach: #bidders+1 “elicitors”
– Could interleave these “extra” questions with “real” questions• To avoid lazyness; Not necessary from an incentive perspective
– Agents don’t have to answer the questions & may answer questions that were not asked
• Unlike in price feedback (“tatonnement”) mechanisms [Bikhchandani-Ostroy, Parkes-Ungar, Wurman-Wellman, Ausubel-Milgrom, Bikhchandani-deVries-Schummer-Vohra, …]
• Push-pull mechanism
Universal revelation reducers
Universal revelation reducer
• Def. A universal revelation reducer is an elicitor that will ask less than everything whenever the shortest certificate includes less than all queries
• Thrm [Hudson & Sandholm 03] No determionistic universal revelation reducer exists
• A randomized one exists– (E.g., the one that asks random unknown value queries)
Elicitation where worst-case number of queries is polynomial in items
Read-once valuations
• Thrm. If an agent has a read-once valuation function, the number of queries needed to elicit the function is polynomial in items
• Thrm. If an agent’s valuation function is approximable by a read-once function (with only MAX and PLUS nodes), elicitor finds an approximation in a polynomial number of queries
PLUS
ALL
MAX
ALL
500 400 200 100
1000
150
GATEk,m
[Zinkevich, Blum, Sandholm ACMEC-03]
Zinkevich, Blum & S. ACMEC-03
Toolbox valuations
• Items are viewed as tools• Agent can accomplish multiple goals
– Each goal has a value & requires some subset of tools – Agent’s valuation for a package of items is the sum of
the values of the goals that those tools allow the agent to accomplish
• E.g. items = medical patents, goals = medicines• Thrm. If an agent has a toolbox valuation function,
it can be elicited in O(#items #goals) queries
Conitzer, S. & Santi Draft-03
0+1+2 = 3
2-wise dependent valuations
• Thrm. If an agent has a 2-wise dependent valuation function, elicitor finds it in n(n+1)/2 queries
• Thrm. If an agent’s valuation function is approximately 2-wise dependent, elicitor finds an approximation in n(n+1)/2 queries
– Thrm. Every super-additive valuation function is approximately 2-wise dependent
• Thrm. These results generalize to k-wise dependent valuations
1
3
3-2
0
2
1Node = item
n items
Combining polynomially elicitable classes
• Thrm. [Conitzer, Sandholm, Santi 03] If Class C1 is elicitable in polytime and class C2 is elicitable in polytime, then C1 U C2 is elicitable in polytime
Blum, Jackson, S. & Zinkevich COLT-03
Power of multi-agent elicitation
• Thrm. For some classes of valuation functions,
– eliciting the function requires an exponential number of queries,
– but a polynomial number of queries suffices for allocating the items optimally among the agents
Ascending combinatorial auctions
Demand queries
• “If these were the prices, which bundle would you buy?”
• A value query can be simulated by a polynomial number of demand queries
• A demand query cannot be simulated in a polynomial number of value queries [Nisan]
Ascending combinatorial auctions• Increase prices until each item is demanded only once• Item prices vs. bundle prices
– E.g. where there exist no appropriate item prices
• Discriminatory vs. nondiscriminatory prices
BundleBidder 1’s valuation
Bidder 2’s valuation
{1} $0 $2
{2} $0 $2
{1,2} $3 $2
Competitive equilibrium
• Def. Competitive equilibrium (CE)– For each bidder, payoff = max [vi(S) – pi(S), 0]– Seller’s payoff = maxS Feasibles i pi(S)– Prices can be on bundles and discriminatory
• Thrm. Allocation S* is supported in CE iff it is an efficient allocation
• Thrm [Parkes 02; Nisan&Segal 03]. In a combinatorial auction, the information implied by best-responses to some set of CE prices is necessary and sufficient as a certificate for the optimal allocation
Communication complexity of ascending auctions
• Exponential in items in the general case – (like any other preference elicitation scheme)
• If items are substitutes (for each agent), then a Walrasian equilibrium exists, – i.e., nondiscriminatory per-item prices suffice
for agents to self-select the right items– Number of queries needed to find such prices is
polynomial in items [Nisan & Segal 03]
Conclusions on preference elicitation in combinatorial auctions
• Combinatorial auctions are desirable & winner determination algorithms now scale to the large
• Another problem: “The Revelation Problem”– Valuation computation / revelation / communication
• Introduced an elicitor that focuses revelation – Provably finds the welfare maximizing (or Pareto efficient) allocations– Policy dependent search algorithms for elicitation
• Based on topological observations• Optimally effective among admissibly equipped elicitors • Eliciting value differences suffices
– Policy independent general elicitation framework• Uses value, order & rank queries (etc)• Bound-approximation queries takes incremental revelation further• Several algorithms, data structures & query policies in the paper• Only elicits a vanishingly small fraction of the valuations
• Presented a way to make the elicitor incentive compatible• Yields a push-pull partial-revelation mechanism
Conclusions on ascending combinatorial auctions
• Demand queries (exponentially more powerful than value queries)– Per-item prices vs. bundle prices– Discriminatory vs. nondiscriminatory prices
• Exponential communication complexity, but polynomial in special classes (e.g., when items are substitutes)– To allocate optimally, enough info has to be elicited to
determine the minimal competitive equilibrium prices
• Could also use descending prices
Future research on multiagent preference elicitation
• Scalable general elicitors (in queries, CPU, RAM)– Current run-time: exp in #items, poly in #agents– Current space: exp in #items, linear in #agents– More powerful queries, e.g. side constraints– New query policies
• New polynomially elicitable valuation classes• Using models of how costly it is to answer different queries [Hudson
& S. AMEC-02]
• Decision-theoretic elicitation using priors• Elicitors for markets beyond combinatorial auctions
– (Combinatorial) reverse auctions & exchanges– (Combinatorial) markets with side constraints– (Combinatorial) markets with multiattribute features– …
• Other applications (e.g. voting [Conitzer & S. AAAI-02])