dimension reduction for finite trees in l 1
DESCRIPTION
James R. Lee. Mohammad Moharrami. Dimension reduction for finite trees in L 1. University of Washington. Arnaud De Mesmay. École Normale Supérieure. TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A A A A A. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/1.jpg)
Dimension reduction for finite trees in L1
University of Washington
James R. Lee
École Normale Supérieure
Arnaud De Mesmay
Mohammad Moharrami
0 1
0 0
0
0
01
1 1
1
1 10
0 1
0 0
0
0
01
1 1
1
1 10
![Page 2: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/2.jpg)
dimension reduction in Lp
Given an n-point subset X µ Rd, find a mapping
kf (x) ¡ f (y)k1 =
F : X ! Rk
such that for all x, y 2 X,kx ¡ ykp · kF (x) ¡ F (y)kp · D ¢kx ¡ ykp
k = target dimensionD = distortion
n = size of X
Dimension reduction as “geometric information theory”
![Page 3: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/3.jpg)
the case p=2
When p=2, the Johnson-Lindenstrauss transform gives, for everyn-point subset X µ Rd and " > 0,
kf (x) ¡ f (y)k1 =
k = O³
logn"2
´D · 1+ "
Applications to…- Statistics over data streams- Nearest-neighbor search- Compressed sensing- Quantum information theory- Machine learning
![Page 4: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/4.jpg)
dimension reduction in L1
Natural to consider is p=1.
History:
k = target dimensionD = distortion
n = size of X
- Caratheodory’s theorem yields D=1 and k = ¡n2¢
- [Schechtman’87, Bourgain-Lindenstrauss-Milman’89, Talagrand’90] Linear mappings (sampling + reweighting) yield
k = O³
n log n"2
´D · 1+" and
- [Batson-Spielman-Srivastava’09, Newman-Rabinovich’10] Sparsification techniques yield
k = O ¡ n"2
¢D · 1+" and
![Page 5: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/5.jpg)
the Brinkman-Charikar lower bound
[Brinkman-Charikar’03]: There are n-point subsets such that distortion D requires
k ¸ n (1=D 2)
Very technical argument based on LP-duality.[L-Naor’04]: One-page argument based on uniform convexity.
[Brinkman-Karagiozova-L 07] Lower bound tight for these spaces
![Page 6: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/6.jpg)
more lower bounds
[Andoni-Charikar-Neiman-Nguyen’11]: There are n-point subsets such that distortion 1+" requires
k ¸ n1¡ O(1=log(1="))
[Regev’11]: Simple, elegant, information-theoretic proof of both the Brinkman-Charikar and ACNN lower bounds.
Low-dimensional embedding ) encoding scheme
![Page 7: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/7.jpg)
4
the simplest of L1 objects
A tree metric is a graph theoretic tree T=(V, E) together withnon-negative lengths on the edgeslen : E ! [0;1 )
Easy to embed isometrically into RE equipped with the L1 norm.
![Page 8: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/8.jpg)
dimension reduction for trees in L1
Charikar and Sahai (2002) showed that for trees one can achievek = O
³log3 n
"2
´D · 1+ "
A. Gupta improved this to k = O³
log2 n"2
´
In 2003 in Princeton with Gupta and Talwar, we asked:
Is possible?k = O(logn)D = O(1)
even for complete binary trees?
![Page 9: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/9.jpg)
dimension reduction for trees in L1
Theorem: For every n-point tree metric, one can achievek = O ¡ 1
"4 log¡ 1"¢¢¢logn
D · 1+ "and
(Can get for “symmetric” trees.)k = O³
logn"2
´
Complete binary tree using local lemmaSchulman’s tree codesComplete binary tree using re-randomizationExtension to general trees
![Page 10: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/10.jpg)
dimension reduction for the complete binary tree
0 1
0 0
0
0
01
1 1
1
1 1 1
001 110
Every edge gets B bits ) target dimension = B log2nChoose edge labels uniformly at random.Nodes at tree distance have probability to get labels with hamming distance
(logn) n¡ O(B )
(logn)
![Page 11: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/11.jpg)
dimension reduction for the complete binary tree
0 1
0 0
0
0
01
1 1
1
1 1 1
001 110
Every edge gets B bits ) target dimension = B log2nChoose edge labels uniformly at random.
Siblings have probability 2-B to have the same label, yetthere are n/2 of them.
![Page 12: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/12.jpg)
Lovász Local Lemma
Pairs at distance L have probability to be “good”1¡ 2¡ (L )
Number of dependent “distance L” events is2O(L )
LLL + sum over levels ) good embedding
![Page 13: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/13.jpg)
Schulman’s tree codes
- LLL argument difficult to extend to arbitrary trees. - Same as construction of Schulman’96: Tree codes for interactive communication
0 1
0 0
0
0
01
1 1
1
1 1 1
001 110
![Page 14: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/14.jpg)
re-randomization
0 1
0 0
0
0
01
1 1
1
1 10
0 1
0 0
0
0
01
1 1
1
1 10
0 1
For every level on the right, exchange 0’s and 1’s with probability half(independently for each level)
Random isometry:
![Page 15: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/15.jpg)
re-randomization
0 1
0 0
0
0
01
1 1
1
1 10
0 1
0 0
0
0
01
1 1
1
1 10
0 1
Pairs at distance L have probability to be “good”1¡ 2¡ (L )
Number of pairs at distance L is 2O(L )
![Page 16: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/16.jpg)
extension to general trees
How many coordinates to change per edge, and bywhat magnitude?
Unfortunately, the general case is technical (paper is 50 pages)Obstacles:
General trees do not have O(log n) depthUse “topological depth” of Matousek.
Multi-scale entropy functional
![Page 17: Dimension reduction for finite trees in L 1](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815ecf550346895dcd5c46/html5/thumbnails/17.jpg)
open problems
Coding/dimension reduction:Extend/make explicit the connection between L1 dimensionreduction and information theory.
Close the gap: For distortion 10, is the right target dimension
n or n0:01 ?Other Lp norms: Nothing non-trivial is known forp =2 f1;2;1 g