Combinatorial Laplacian and Rank Aggregation
Combinatorial Laplacian and Rank Aggregation
Yuan Yao
Stanford University
ICIAM, Zurich, July 16–20, 2007
Joint work with Lek-Heng Lim
Combinatorial Laplacian and Rank Aggregation
Outline
1 Two Motivating Examples
2 Reflections on RankingOrdinal vs. CardinalGlobal, Local, vs. Pairwise
3 Discrete Exterior Calculus and Combinatorial LaplacianDiscrete Exterior CalculusCombinatorial Laplacian Operator
4 Hodge TheoryCyclicity of Pairwise RankingsConsistency of Pairwise Rankings
5 Conclusions and Future Work
Combinatorial Laplacian and Rank Aggregation
Two Motivating Examples
Example I: Customer-Product Rating
Example (Customer-Product Rating)
m-by-n customer-product rating matrix X ∈ Rm×n
X typically contains lots of missing values (say ≥ 90%).
The first-order statistics, mean score for each product, might sufferfrom
most customers just rate a very small portion of the products
different products might have different raters, whence meanscores involve noise due to arbitrary individual rating scales
Combinatorial Laplacian and Rank Aggregation
Two Motivating Examples
From 1st Order to 2nd Order: Pairwise Rankings
The arithmetic mean of score difference between product iand j over all customers who have rated both of them,
gij =
∑k(Xkj − Xki )
#{k : Xki ,Xkj exist},
is translation invariant.
If all the scores are positive, the geometric mean of score ratioover all customers who have rated both i and j ,
gij =
(∏k
(Xkj
Xki
))1/#{k:Xki ,Xkj exist}
,
is scale invariant.
Combinatorial Laplacian and Rank Aggregation
Two Motivating Examples
More invariant
Define the pairwise ranking gij as the probability that productj is preferred to i in excess of a purely random choice,
gij = Pr{k : Xkj > Xki} −1
2.
This is invariant up to a monotone transformation.
Combinatorial Laplacian and Rank Aggregation
Two Motivating Examples
Example II: Purely Exchange Economics
Example (Pairwise ranking in exchange market)
n goods V = {1, . . . , n} in an exchange market, with anexchange rate matrix A, such that
1 unit i = aij unit j , aij > 0.
which is a reciprocal matrix, i.e. aij = 1/aji
Ideally, a product triple (i , j , k) is called triangulararbitrage-free, if aijajk = aik
Money (universal equivalent): does there exist a universalequivalent with pricing function p : V → R+, such that
aij = pj/pi?
Combinatorial Laplacian and Rank Aggregation
Two Motivating Examples
From Pairwise to Global
Under the logarithmic map, gij = log aij , we have anequivalent theory:• the triangular arbitrage-free is equivalent to
gij + gjk + gki = 0
• universal equivalent is a global ranking f : V → R(fi = log pi ) such that
gij = fj − fi =: (δ0f )(i , j)
Here• Global ranking ⇔ universal equivalent (price)• Pairwise ranking ⇔ exchange rates
Combinatorial Laplacian and Rank Aggregation
Two Motivating Examples
Observations
In both examples,
contain cardinal information
involve pairwise comparisons
How important are they?
Combinatorial Laplacian and Rank Aggregation
Reflections on Ranking
Ordinal vs. Cardinal
Ordinal Rank Aggregation
Problem: given a set of partial/total order {�i : i = 1, . . . , n}on a common set V , find
(�1, . . . ,�n) 7→�∗,
as a partial order on V , satisfying certain optimal condition.
Examples:• voting• Social Choice Theory
Notes:• Impossibility Theorems (Arrow et al.)• Hardness in solving (NP-hard for Kemeny optimality etc.)
Combinatorial Laplacian and Rank Aggregation
Reflections on Ranking
Ordinal vs. Cardinal
Cardinal Rank Aggregation
Problem: given a set of functions fi : V → R (i = 1, . . . , n),find
(f1, . . . , fn) 7→ f ∗
as a function on V , satisfying certain optimal condition.
Examples:• customer-product rating, e.g. Amazon, Netflix• stochastic choice with f as probability distributions on V ,e.g. Google search, cardinal utility in Economics
Notes• relaxations leave rooms for ‘possibility’• ordinal rankings induced from cardinal rankings, but withinformation loss
Combinatorial Laplacian and Rank Aggregation
Reflections on Ranking
Global, Local, vs. Pairwise
Global, Local, and Pairwise Rankings
Global ranking is a function on V , f : V → RLocal (partial) ranking: restriction of global ranking on asubset U, f ′ : U → RPairwise ranking: g : V × V → R (with gij = −gji )• Note: pairwise rankings are simply skew-symmetric matricessl(n) or certain equivalence classes in sl(n). Also we may viewpairwise rankings as weighted digraphs.
Combinatorial Laplacian and Rank Aggregation
Reflections on Ranking
Global, Local, vs. Pairwise
Why Pairwise Ranking?
Human mind can’t make preference judgements on moderatelylarge sets (e.g. no more than 7± 2 in psychology study)
But human can do pairwise comparison more easily andaccurately
Pairwise ranking naturally arises in tournaments, exchangeEconomics, etc.
Pairwise ranking may reduce the bias caused by thearbitrariness of rating scale
Pairwise ranking may contain more information than globalranking (to be seen soon)!
Combinatorial Laplacian and Rank Aggregation
Discrete Exterior Calculus and Combinatorial Laplacian
Our Main Theme
Below we’ll outline an approach to analyze
cardinal, and
pairwise
rankings, in a perspective from discrete exterior calculus.
Briefly, we’ll reach an orthogonal decomposition of pairwiserankings, by Hodge Theory,
Pairwise = Global + Consistent Cyclic + Inconsistent Cyclic
Combinatorial Laplacian and Rank Aggregation
Discrete Exterior Calculus and Combinatorial Laplacian
Discrete Exterior Calculus
Simplicial Complex of Products
Let V = {1, . . . , n} be the set of products or alternatives to beranked. Construct a simplicial complex K :
0-simplices K0: V
1-simplices K1: edges {i , j} such that comparison (i.e.pairwise ranking) between i and j exists
2-simplices K2: triangles {i , j , k} such that• every edge exists in K1
• more considerations on consistency, like triangulararbitrage-free
Note: it suffices here to construct K up to dimension 2!
Combinatorial Laplacian and Rank Aggregation
Discrete Exterior Calculus and Combinatorial Laplacian
Discrete Exterior Calculus
Cochains
k-cochains C k(K , R): vector space of k + 1-alternatingtensors associated with Kk+1
{u : V k+1 → R, uiσ(0),...,iσ(k)= sign(σ)ui0,...,ik}
for (i0, . . . , ik) ∈ Kk+2, where σ ∈ Sk+1 is a permutation on(0, . . . , k).
Inner product in C k(K , R): standard Euclidean
In particular,• global ranking: 0-cochains f ∈ C 0(K , R) ∼= Rn
• pairwise ranking: 1-cochains g ∈ C 1(K , R), gij = −gji
Combinatorial Laplacian and Rank Aggregation
Discrete Exterior Calculus and Combinatorial Laplacian
Discrete Exterior Calculus
Coboundary Maps
k-dimensional coboundary maps δk : C k(V , R)→ C k+1(V , R)are defined as the alternating difference operator
(δku)(i0, . . . , ik+1) =k+1∑j=0
(−1)j+1u(i0, . . . , ij−1, ij+1, . . . , ik+1)
δk plays the role of differentiation
δk+1 ◦ δk = 0
In particular,• (δ0f )(i , j) = fj − fi is gradient of global ranking f• (δ1g)(i , j , k) = gij + gjk + gki is curl of pairwise ranking g
Combinatorial Laplacian and Rank Aggregation
Discrete Exterior Calculus and Combinatorial Laplacian
Discrete Exterior Calculus
A View from Discrete Exterior Calculus
We have the following cochain complex
C 0(K , R)δ0−→ C 1(K , R)
δ1−→ C 2(K , R),
in other words,
Globalgrad−−→ Pairwise
curl−−→ Triplewise
andcurl ◦ grad(Global Rankings) = 0
Pairwise rankings = alternating 2-tensors = skew-symmetricmatrices = log of Saaty’s reciprocal matrices
Triplewise rankings = alternating 3-tensors
See also: Douglas Arnold’s talk on Tuesday
Combinatorial Laplacian and Rank Aggregation
Discrete Exterior Calculus and Combinatorial Laplacian
Discrete Exterior Calculus
What does it tell us?
Globalgrad−−→ Pairwise
curl−−→ Triplewise
grad(Global) (i.e. im(δ0)): a proper subset of pairwiserankings induced from global
curl(Pairwise) (i.e. im(δ1)): measures theconsistency/triangular arbitrage on triangle {i , j , k}
(δ1g)(i , j , k) = gij + gjk + gki
• ker(curl) (i.e. ker(δ1)): consistent, curl-free, triangulararbitrage-free, in particular —• curl ◦ grad(Global) = 0 (i.e. δ1 ◦ δ0 = 0) says global rankingsare consistent/curl-free
Combinatorial Laplacian and Rank Aggregation
Discrete Exterior Calculus and Combinatorial Laplacian
Discrete Exterior Calculus
Reverse direction: conjugate operators
Gradientgrad∗(=− div)←−−−−−−−− Pairwise
curl∗←−−− Triplewise
grad∗: δT0 under Euclidean inner product, gives the total
inflow-outflow difference at each vertex (negative divergence)
(δT0 g)(i) =
∑g∗i −
∑gi∗
• ker(δT0 ), as divergence-free, is cyclic (interior/boundary)
curl∗: δT1 , gives interior cyclic pairwise rankings along
triangles in K2, which are inconsistent
Combinatorial Laplacian and Rank Aggregation
Discrete Exterior Calculus and Combinatorial Laplacian
Combinatorial Laplacian Operator
Combinatorial Laplacian
Define the k-dimensional combinatorial Laplacian,∆k : C k → C k by
∆k = δk−1δTk−1 + δT
k δk , k > 0
k = 0, ∆0 = δT0 δ0 is the well-known graph Laplacian
k = 1,∆1 = curl ◦ curl∗− div ◦ grad
Important Properties:• ∆k positive semi-definite• ker(∆k) = ker(δT
k−1) ∩ ker(δk): k-harmonics, dimensionequals to k-th Betti number• Hodge Decomposition Theorem
Combinatorial Laplacian and Rank Aggregation
Hodge Theory
Hodge Decomposition Theorem
Theorem
The space of pairwise rankings, C 1(V , R), admits an orthogonaldecomposition into three
C 1(V , R) = im(δ0)⊕ H1 ⊕ im(δT1 )
whereH1 = ker(δ1) ∩ ker(δT
0 ) = ker(∆1).
Combinatorial Laplacian and Rank Aggregation
Hodge Theory
Hodge Decomposition Illustration
Figure: Hodge Decomposition for Pairwise Rankings
Combinatorial Laplacian and Rank Aggregation
Hodge Theory
An Example from Jester Dataset
Figure: Hodge Decomposition for a pairwise ranking on four Jester jokes(No.1 - 4): g1 gives a global ranking (order: 1 > 2 > 3 > 4) whichaccounts for 90% of the total norm; g2 is the consistent cyclic part ontriangles {{123}, {124}} with 7% norm; and g3 is the inconsistent cyclicpart.
Combinatorial Laplacian and Rank Aggregation
Hodge Theory
Cyclicity of Pairwise Rankings
Acyclic-Cyclic Decomposition
Corollary
Every pairwise ranking admits a unique orthogonal decomposition,
g = projim(δ0) g + projker(δT0 ) g
i.e.Pairwise = grad(Global) + Cyclic
Note: Pairwise rankings induced from global are exactly acycliccomponent, as the orthogonal complement of cyclic pairwiserankings.
Combinatorial Laplacian and Rank Aggregation
Hodge Theory
Consistency of Pairwise Rankings
Consistency
Definition
A pairwise ranking g is consistent on a triangle (2-simplex) (i , j , k)if gij + gjk + gki = 0, in other words, (δ1g)(i , j , k) = 0.
Note:
Consistency depends on the triangles(2-simplices), so for a pairwise rankingg , | curl(g)(i , j , k)| measures the curldistribution over triangles (2-simplices)in K2
−0.2 −0.15 −0.1 −0.05 0 0.05 0.1 0.150
1
2
3
4
5
6
7
8
9x 10
4 Curl distribution of Jester dataset
Combinatorial Laplacian and Rank Aggregation
Hodge Theory
Consistency of Pairwise Rankings
Consistent Decomposition
Corollary
1 A consistent pairwise ranking g associated with K, has aunique orthogonal decomposition
g = projim(δ0) g + projH1g = grad(Global) + Harmonic
i.e. where harmonic is cyclic on the “holes” of the complex K.
2 Every consistent pairwise ranking on a contractible K, isinduced from a global ranking.
Note: (2) rephrases the famous theorem in exchange Economics:triangular arbitrage-free implies arbitrage-free and the existence ofuniversal equivalent.
Combinatorial Laplacian and Rank Aggregation
Conclusions and Future Work
Conclusions and Future Work
Conclusions
Hodge Theory provides an orthogonal decomposition forpairwise rankings
Such decomposition is helpful to characterize the cyclicity and(triangular) consistency of pairwise rankings
Future
Comparisons with other spectral methods• Fourier Analysis on symmetry groups (Diaconis)• Markov Chain based methods (PageRank, etc.) as graphLaplacians
Design new algorithms
Applications on large scale data sets, e.g. Netflix dataset.
Combinatorial Laplacian and Rank Aggregation
Acknowledgements
Gunnar Carlsson (Stanford)
Persi Diaconis (Stanford)
Nick Eriksson (Stanford)
Fei Han (UCB)
Susan Holmes (Stanford)
Xiaoye Jiang (Stanford)
Ming Ma (UCB and Beijing Institute of Technology)
Michael Mahoney (Yahoo! Research)
Steve Smale (TTI-U Chicago and UCB)
Shmuel Weinberger (U Chicago)