combinatorial optimization and the mean field model
DESCRIPTION
Combinatorial optimization and the mean field model. Johan Wästlund Chalmers University of Technology Sweden. Random instances of optimization problems. Random instances of optimization problems. Random instances of optimization problems. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/1.jpg)
Combinatorial optimization and the mean field modelJohan WästlundChalmers University of TechnologySweden
![Page 2: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/2.jpg)
Random instances of optimization problems
![Page 3: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/3.jpg)
Random instances of optimization problems
![Page 4: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/4.jpg)
Random instances of optimization problems Typical distance between nearby points is of
order n-1/2
![Page 5: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/5.jpg)
Random instances of optimization problems A tour consists of n links, therefore we
expect the total length of the minimum tour to scale like n1/2
Beardwood-Halton-Hammersley (1959):
73.02/1 pN
nL
92.062.0
![Page 6: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/6.jpg)
Mean field model of distance
Distances Xij chosen as i.i.d. variables Given n and the distribution of distances,
study the random variable Ln If the distribution models distances in d
dimensions, we expect Ln to scale like n1-1/d
In particular, pseudo-dimension 1 means Ln is asymptotically independent of n
![Page 7: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/7.jpg)
Mean field model of distance
The edges of a complete graph on n vertices are given i. i. d. nonnegative costs
Exponential(1) distribution.
![Page 8: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/8.jpg)
Mean field model of distance
We are interested in the cost of the minimum matching, minimum traveling salesman tour etc, for large n.
![Page 9: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/9.jpg)
Mean field model of distance
Convergence in probability to a constant?
![Page 10: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/10.jpg)
Matching Set of edges that gives a pairing of all points
![Page 11: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/11.jpg)
Statistical Physics / C-S Spin configuration Hamiltonian Ground state energy
Temperature Gibbs measure Thermodynamic limit
Feasible solution Cost of solution Cost of minimal
solution Artificial parameter T Gibbs measure n→∞
![Page 12: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/12.jpg)
Statistical physics
Replica-cavity method of statistical mechanics has given spectacular predictions for random optimization problems
M. Mézard, G. Parisi 1980’s Limit of /12 for minimum matching on the
complete graph (Aldous 2000) Limit 2.0415… for the TSP (Wästlund 2006)
![Page 13: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/13.jpg)
A. Frieze (2004): “Up to now there has been almost no progress analysing this random model of the travelling salesman problem.”
N. J. Cerf et al (1997): “Researchers outside physics remain largely unaware of the analytical progress made on the random link TSP.”
![Page 14: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/14.jpg)
Non-rigorous derivation of the /12 limit Matching problem on Kn for large n. In principle, this requires even n, but we shall
consider a relaxation Let the edges be exponential of mean n, so
that the sequence of ordered edge costs from a given vertex is approximately a Poisson process of rate 1.
![Page 15: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/15.jpg)
Non-rigorous derivation of the /12 limit The total cost of the minimum matching is of
order n. Introduce a punishment c>0 for not using a
particular vertex. This makes the problem well-defined also for
odd n. For fixed c, let n tend to infinity. As c tends to infinity, we expect to recover
the behavior of the original problem.
![Page 16: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/16.jpg)
Non-rigorous derivation of the /12 limit For large n, suppose that the problem
behaves in the same way for n-1 vertices. Choose an arbitrary vertex to be the root What does the graph look like locally around
the root? When only edges of cost <2c are considered,
the graph becomes locally tree-like
![Page 17: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/17.jpg)
Non-rigorous derivation of the /12 limit Non-rigorous replica-cavity method Aldous derived equivalent equations with the
Poisson-Weighted Infinite Tree (PWIT)
![Page 18: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/18.jpg)
Non-rigorous derivation of the /12 limit Let X be the difference in cost between the
original problem and that with the root removed.
If the root is not matched, then X = c. Otherwise X = i – Xi, where Xi is distributed like X, and i is the cost of the i:th edge from the root.
The Xi’s are assumed to be independent.
![Page 19: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/19.jpg)
Non-rigorous derivation of the /12 limitIt remains to do some calculations.
We have
where Xi is distributed like X
),,min( ii XcX
![Page 20: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/20.jpg)
Non-rigorous derivation of the /12 limit Let )(exp)(exp)()( ufdttFuXPuF
u
X
-u
![Page 21: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/21.jpg)
Non-rigorous derivation of the /12 limit Then if u>-c,
)()()(' ufeuFuf
)()(' ufeuf
![Page 22: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/22.jpg)
Non-rigorous derivation of the /12 limit
)()()()( )(')(')(')(' ufufufuf edudeufe
dudeufufuf
Hence )()( ufuf ee is constant
![Page 23: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/23.jpg)
Non-rigorous derivation of the /12 limit
The constant depends on c and holds when
–c<u<c
f(-u)
f(u)
![Page 24: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/24.jpg)
Non-rigorous derivation of the /12 limit From definition, exp(-f(c)) = P(X=c) =
proportion of vertices that are not matched, and exp(-f(-c)) = exp(0) = 1
e-f(u) + e-f(-u) = 2 – proportion of vertices that are matched = 1 when c = infinity.
![Page 25: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/25.jpg)
Non-rigorous derivation of the /12 limit
1 ee yx
6
2Area
![Page 26: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/26.jpg)
Non-rigorous derivation of the /12 limit What about the cost of the minimum
matching?
![Page 27: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/27.jpg)
Non-rigorous derivation of the /12 limit
![Page 28: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/28.jpg)
Non-rigorous derivation of the /12 limit
![Page 29: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/29.jpg)
Non-rigorous derivation of the /12 limit Hence J = area under the curve when f(u) is
plotted against f(-u)! Expected cost = n/2 times this area In the original setting = ½ times the area = /12.
![Page 30: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/30.jpg)
The equation has the explicit solution
This gives the cost
)1log()( ueuf
121)1log(
21 2
dueeu
u
![Page 31: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/31.jpg)
![Page 32: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/32.jpg)
The exponential bipartite assignment problem
n
![Page 33: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/33.jpg)
The exponential bipartite assignment problem Exact formula conjectured by Parisi (1998)
Suggests proof by induction Researchers in discrete math, combinatorics and
graph theory became interested Generalizations…
2
191
411)(
nCE n
![Page 34: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/34.jpg)
Generalizations
by Coppersmith & Sorkin to incomplete matchings
Remarkable paper by M. Buck, C. Chan & D. Robbins (2000)
Introduces weighted vertices Extremely close to proving Parisi’s
conjecture!
![Page 35: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/35.jpg)
Incomplete matchings
nm
![Page 36: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/36.jpg)
Weighted assignment problems Weights 1,…,m, 1,…, n on vertices Edge cost exponential of rate ij
Conjectured formula for the expected cost of minimum assignment
Formula for the probability that a vertex participates in solution (trivial for less general setting!)
![Page 37: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/37.jpg)
The Buck-Chan-Robbins urn process Balls are drawn with probabilities proportional
to weight
1 2
3
![Page 38: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/38.jpg)
Proofs of the conjectures
Two independent proofs of the Parisi and Coppersmith-Sorkin conjectures in 2003 (Nair, Prabhakar, Sharma and Linusson, Wästlund)
![Page 39: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/39.jpg)
Rigorous method
Relax by introducing an extra vertex Let the weight of the extra vertex go to zero Example: Assignment problem with 1=…=m=1, 1=…=n=1, and m+1 =
p = P(extra vertex participates) p/n = P(edge (m+1,n) participates)
![Page 40: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/40.jpg)
Rigorous method
p/n = P(edge (m+1,n) participates) When →0, this is
Hence
By Buck-Chan-Robbins urn theorem,
1,,1,,1,,1,,,1 nmknmknmknmknm CCCClP
n
pCCE nmknmk 01,,1,, lim
11
kmmmp
![Page 41: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/41.jpg)
Rigorous method
Hence
Inductively this establishes the Coppersmith-Sorkin formula
nkmnmmn
CECE nmknmk )1(1
)1(11
1,,1,,
)1(
1)1(
1)1(
1)1(
11,,
knmnkmnmnmmn
CE nmk
![Page 42: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/42.jpg)
Rigorous results
Much simpler proofs of Parisi, Coppersmith-Sorkin, Buck-Chan-Robbins formulas
Exact results for higher moments Exact results and limits for optimization
problems on the complete graph
![Page 43: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/43.jpg)
The 2-dimensional urn process
2-dimensional time until k balls have been drawn
![Page 44: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/44.jpg)
Limit shape as n→∞
Matching:
TSP/2-factor:
1 ee yx
12
12
1
ee yx yx
041548.2)(21
0
dxxy
6)(
2
0
dxxy
![Page 45: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/45.jpg)
Mean field TSP
If the edge costs are i.i.d and satisfy P(l<t)/t→1 as t→0 (pseudodimension 1), then as n →∞,
...041548.2* LLp
n
![Page 46: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/46.jpg)
For the TSP, the replica-cavity approach gives
)())(1()(' ufeufuf
)())(1()(' ufeufuf
![Page 47: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/47.jpg)
It follows that
is constant, and = 1 by boundary conditions Replica-cavity prediction agrees with the
rigorous result (Parisi 2006)
)()(
2)(1
2)(1 ufuf eufeuf
![Page 48: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/48.jpg)
Further exact formulas
nnn
n kn
kkC1
32
1
2
1
4 /114/12/15var
2
1)3(4)2(4n
On
![Page 49: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/49.jpg)
LP-relaxation of matching in the complete graph Kn
121
91
411)(
2
2
nCE n
![Page 50: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/50.jpg)
Future work
Explain why the cavity method gives the same equation as the limit shape in the urn process
Reprove results of one method with the other Find the variance with the replica method Find rigorously the distribution of edge costs
participating in the solution (there is an exact conjecture)
![Page 51: Combinatorial optimization and the mean field model](https://reader036.vdocuments.us/reader036/viewer/2022062501/568167b8550346895ddd0652/html5/thumbnails/51.jpg)