communication complexity of randomness...
TRANSCRIPT
of 17May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 1
Communication Complexity of Randomness Manipulation
Madhu SudanHarvard University
Joint work with Mitali Bafna (Harvard), Badih Ghazi (Google), and Noah Golowich (Harvard)
of 17
Randomness Processing Industry
Dispersers, Extractors, Merges, Condensers, PRGs …
Interest not in the exact function. (E=RSA most boring.)
Rather in its manipulation of distributions … Distribution of 𝑋𝑋 vs. that of 𝐸𝐸(𝑋𝑋)
Long history … omitted. Key ingredients single processor “𝐸𝐸” unknown source 𝑋𝑋 ∈ 𝒳𝒳
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 2
𝐸𝐸𝑋𝑋 𝐸𝐸(𝑋𝑋)
of 17
2-player setting:
Is 𝛿𝛿 = 0 possible? If not minimize 𝛿𝛿! Etc. Alice + Bob can use private randomness Zero communication version: “Non-Interactive
Simulation” (NIS).
Distributed Randomness Processing
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 3
Alice 𝑈𝑈
Bob𝑌𝑌 𝑉𝑉
≤ 𝐶𝐶 bits, ≤ 𝑟𝑟 Rounds
𝑋𝑋𝑋𝑋1,𝑋𝑋2, …
𝑌𝑌1,𝑌𝑌2, …
𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃 Goal: 𝑈𝑈,𝑉𝑉 ≈𝛿𝛿 𝑄𝑄
Infinite sequence
of 17
A classical example
Zero communication 𝑋𝑋,𝑌𝑌 ∼ Unif 0,0 , 0,1 , 1,0 𝑈𝑈,𝑉𝑉 ∼ 𝜌𝜌-correlated bits.
𝑈𝑈,𝑉𝑉 ∼ Unif 0,1 , Pr 𝑈𝑈 = 𝑉𝑉 = 1+𝜌𝜌2
[Witsenhausen ‘70s]: Can’t achieve 𝜌𝜌 = 1. Not even 𝜌𝜌 = .51
(Naïve strategy achieves 𝜌𝜌 = 14)
Can achieve 𝜌𝜌 = 13
by a general method.
Best 𝜌𝜌? Open!!
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 4
of 17
Witsenhausen …
Intermediate Problem: Output (𝐴𝐴,𝐵𝐵) w. 𝐴𝐴,𝐵𝐵 ∼ 𝑁𝑁 0,1with maximum correlation.
Definition: Max-Corr 𝜌𝜌 𝑋𝑋,𝑌𝑌 ≜ max𝑓𝑓,𝑔𝑔
𝔼𝔼𝑋𝑋,𝑌𝑌 𝑓𝑓 𝑋𝑋 𝑔𝑔(𝑌𝑌)
Where 𝑓𝑓,𝑔𝑔:Ω → ℝ,𝔼𝔼𝑋𝑋 𝑓𝑓 𝑋𝑋 = 𝔼𝔼𝑌𝑌 𝑔𝑔 𝑌𝑌 = 0𝔼𝔼𝑋𝑋 𝑓𝑓 𝑋𝑋 2 = 𝔼𝔼𝑌𝑌 𝑔𝑔 𝑌𝑌 2 = 1
Thm 1: For any 𝑈𝑈,𝑉𝑉 output, 𝜌𝜌 𝑈𝑈,𝑉𝑉 ≤ 𝜌𝜌(𝑋𝑋,𝑌𝑌) Thm 2: For Gaussian output (𝐴𝐴,𝐵𝐵), can achieve
𝜌𝜌 𝐴𝐴,𝐵𝐵 = 𝜌𝜌(𝑋𝑋,𝑌𝑌) Thm 3: For binary 𝑈𝑈,𝑉𝑉 , can achieve
1 −2 ⋅ cos−1 𝜌𝜌 𝑋𝑋,𝑌𝑌
𝜋𝜋≤ 𝜌𝜌 𝑈𝑈,𝑉𝑉 ≤ 𝜌𝜌 𝑋𝑋,𝑌𝑌
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 5
For Bernoulli/GaussiansMax-Correlation = Correlation
of 17
“Tensorization”
“Single-letter” Problems: Given (1) structure 𝑆𝑆 (graph, distribution, game) (2) product operation 𝑆𝑆⊗𝑡𝑡, (3) measure 𝑀𝑀, compute lim
𝑡𝑡→∞𝑀𝑀 𝑆𝑆⊗𝑡𝑡
Shannon capacity, Compression length, Channel capacity, parallel repetition value of 2-prover game, Direct sum complexity, NIS
“Tensorizing bound”: Typically: 𝑀𝑀 𝑆𝑆⊗𝑡𝑡 ≥ 𝑀𝑀(𝑆𝑆). Find: ℳ(⋅) s.t. 𝑀𝑀 𝑆𝑆 ≤ ℳ(𝑆𝑆) and ℳ 𝑆𝑆⊗𝑡𝑡 = ℳ(𝑆𝑆)
Max-Correlation “Tensorizes” If 𝑋𝑋,𝑌𝑌 ∼ 𝜇𝜇 & 𝑋𝑋𝑡𝑡,𝑌𝑌𝑡𝑡 ∼ 𝜇𝜇⊗𝑡𝑡 then 𝜌𝜌 𝑋𝑋𝑡𝑡 ,𝑌𝑌𝑡𝑡 = 𝜌𝜌(𝑋𝑋,𝑌𝑌) Proof idea: 𝜇𝜇 described by matrix 𝑃𝑃; 𝜌𝜌 related to its
singular values, 𝜇𝜇⊗𝑡𝑡 described by 𝑃𝑃⊗𝑡𝑡
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 6
of 17
Single-letter EmbarassmentProblem Base Single-letter
versionCompressibility P P
Channel capacity NP P
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 7
Glossary: 0 ≤ P ≤ 𝑁𝑁𝑃𝑃 ≤ EXP ≤ Computable≤ CA Computably Approximable ≤ ∞
Problem Base Single-letterversion
Compressibility P P
Channel capacity NP P
Independent set/Shannon capacity
NP [0,∞]
Value of 2-prover game NP [NP,∞]
Problem Base Single-letterversion
Compressibility P P
Channel capacity NP P
Independent set/Shannon capacity
NP [0,∞]
Value of 2-prover game NP [NP,∞]
Non-interactive-simulation (“correlation”)
NP [0,𝐶𝐶𝐴𝐴]
Communication Complexity [NP,Exp?] 0,𝐶𝐶𝐴𝐴
Problem Base Single-letterversion
Compressibility P P
Channel capacity NP P
Independent set/Shannon capacity
NP [0,∞]
Value of 2-prover game NP [NP,∞]
Non-interactive-simulation (“correlation”)
NP [0,𝐶𝐶𝐴𝐴]
Communication Complexity [NP,Exp?] 0,𝐶𝐶𝐴𝐴
? P [NP,∞]
of 17
Complexity of NIS
“Computably approximable” [Ghazi,Kamath,S.], [De,Mossel,Neeman]2,
[Ghazi,Kamath,Raghavendra] There exists a finite time algorithm determining if we can
get 𝜖𝜖-close to (𝑈𝑈,𝑉𝑉) from i.i.d. samples of (𝑋𝑋,𝑌𝑌)
Idea: “Invariance Priniciple” [Mossel] Either there is a method using few samples of (𝑋𝑋,𝑌𝑌) or
many samples 𝑋𝑋1,𝑌𝑌1 , … , 𝑋𝑋𝑛𝑛,𝑌𝑌𝑛𝑛 each with low influence on 𝑈𝑈,𝑉𝑉 .
If latter, can replace 𝑋𝑋,𝑌𝑌 by finite # Gaussians. Gives computable upper bound on #samples needed to
get close to optimal strategy.
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 8
of 17
Communication?
Restrict to “Common Randomness Generation” (𝑘𝑘-CRG): Output (𝑈𝑈 = 𝑉𝑉), with 𝑈𝑈 ∼ Unif( 0,1 𝑘𝑘)
First considered ~70 [Ahlswede-Csizar]: “Characterization”: Can communicate 𝑅𝑅 ⋅ 𝑘𝑘 +𝑜𝑜 𝑘𝑘 bits, where 𝑅𝑅 = min
ΠICint(Π)ICext Π
Computable? Computably approximable? For one-way communication: 𝑅𝑅 = 1 − 𝜌𝜌 𝑋𝑋,𝑌𝑌 2
[Zhao-Chia] (see also [Guruswami-Radhakrishnan, Ghazi-Jayram, S.-Tyagi-Watanabe])
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 9
Alice 𝑈𝑈
Bob 𝑉𝑉
≤ 𝐶𝐶 bits, ≤ 𝑟𝑟 Rounds
𝑋𝑋1,𝑋𝑋2, …
𝑌𝑌1,𝑌𝑌2, …
𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃 Goal: 𝑈𝑈,𝑉𝑉 ≈𝛿𝛿 𝑄𝑄
of 17
This talk: Rounds in CRG
Does increasing #rounds ease CRG? (For some (𝑋𝑋,𝑌𝑌)?) … Is the following true?
Question: ∀𝜖𝜖 > 0 , 𝑟𝑟, ∃(𝑋𝑋,𝑌𝑌) s.t. ∀𝑘𝑘 ∃ 𝑟𝑟, 𝜖𝜖𝑘𝑘 -protocol for 𝑘𝑘-CRG 𝑋𝑋,𝑌𝑌 No 𝑟𝑟 − 1, 1 − 𝜖𝜖 .𝑘𝑘 -protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌)
Still don’t know. Partial progress. Thm. [BGGS’19] : ∀𝑛𝑛,𝑘𝑘, 𝑟𝑟 there exists 𝑋𝑋,𝑌𝑌 s.t.
∃ 𝑟𝑟,𝑂𝑂(𝑟𝑟 log 𝑛𝑛 -protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌).
No 𝑟𝑟2− 3, min 𝑘𝑘, 𝑛𝑛
polylog 𝑛𝑛-protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌)
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 10
of 17
Our Source:
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 11
𝑖𝑖0
𝜋𝜋1
𝜋𝜋2
𝜋𝜋3
𝜋𝜋𝑟𝑟
⋮
𝐴𝐴1 𝐴𝐴2 𝐴𝐴3 𝐴𝐴𝑛𝑛 𝐵𝐵1 𝐵𝐵2 𝐵𝐵3 𝐵𝐵𝑛𝑛… …
Alice’s Inputs𝑋𝑋 = (𝜋𝜋1,𝜋𝜋3, … ;𝐴𝐴1, … ,𝐴𝐴𝑛𝑛)
Bob’s Inputs𝑌𝑌 = (𝑖𝑖0;𝜋𝜋2,𝜋𝜋4, … ;𝐵𝐵1, … ,𝐵𝐵𝑛𝑛)
𝑖𝑖0 ∼ [𝑛𝑛]
𝜋𝜋1, … ,𝜋𝜋𝑛𝑛 ∼ 𝑆𝑆𝑛𝑛𝐴𝐴1, … ,𝐴𝐴𝑛𝑛 ∼ 0,1 𝑘𝑘
𝐵𝐵1, … ,𝐵𝐵𝑛𝑛 ∼ 0,1 𝑘𝑘
𝐴𝐴𝑖𝑖𝑟𝑟 = 𝐵𝐵𝑖𝑖𝑟𝑟where 𝑖𝑖𝑗𝑗 = 𝜋𝜋𝑗𝑗 𝑖𝑖𝑗𝑗−1
“Pointer Chasing Source” (PCS)
of 17
CRG from Pointer Chasing Source
Easy direction easy: (𝑟𝑟, 𝑟𝑟 log𝑛𝑛)-protocol for 𝑘𝑘-CRG Hardness?
Pointer chasing is hard: [Duris-Galil-Schnitger, Nisan-Wigderson,…].
CRG from PCS requires pointer chasing? No! Can solve problem without pointers! Small non-deterministic complexity! Hardness needs to use hardness of
disjointness? And of pointer-chasing? How to combine modularly?
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 12
of 17
Our solution: Pointer Verification Problem
PVP: Distinguish 𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃𝑉𝑉YES from 𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃𝑉𝑉No 𝑃𝑃𝑉𝑉YES:𝑋𝑋 = 𝜋𝜋1,𝜋𝜋3, …𝜋𝜋𝑟𝑟−1 ; 𝑌𝑌 = 𝑖𝑖0, 𝑖𝑖𝑟𝑟; 𝜋𝜋2,𝜋𝜋4, …𝜋𝜋𝑟𝑟 𝑃𝑃𝑉𝑉NO:𝑋𝑋 = 𝜋𝜋1,𝜋𝜋3, …𝜋𝜋𝑟𝑟−1 ; 𝑌𝑌 = 𝑖𝑖0, 𝑖𝑖𝑟𝑟′ ; 𝜋𝜋2,𝜋𝜋4, …𝜋𝜋𝑟𝑟
YES instance satisfy: 𝑖𝑖𝑗𝑗 = 𝜋𝜋𝑗𝑗 𝑖𝑖𝑗𝑗−1 NO instance: 𝑖𝑖𝑟𝑟′ random
Claims: Hardness of (Unique) Set Disjointness ⇒
Protocol for 𝑘𝑘-CRG(PCS) solves PVP.
No 𝑟𝑟2− 𝑂𝑂 1 , 𝑛𝑛
polylog 𝑛𝑛-protocol for PVP.
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 13
of 17
PVP ≤ CRG
Let 𝜇𝜇𝑋𝑋𝑌𝑌 denote dist. of PCS source. 𝜇𝜇𝑋𝑋, 𝜇𝜇𝑌𝑌 marginals
CRG Hard ⇐ 𝜇𝜇𝑋𝑋𝑌𝑌 indist. from 𝜇𝜇𝑋𝑋 × 𝜇𝜇𝑌𝑌 (to 𝑟𝑟2
,𝐶𝐶 -protocols)
Intermediate distribution: 𝜇𝜇mid 𝑋𝑋 = (𝜋𝜋odd,𝐴𝐴1, … ,𝐴𝐴𝑛𝑛); 𝑌𝑌 = 𝑖𝑖0,𝜋𝜋even,𝐵𝐵1, … ,𝐵𝐵𝑛𝑛 ,
with 𝐴𝐴𝑗𝑗 = 𝐵𝐵𝑗𝑗 for random 𝑗𝑗. (Correlation exists but pointers don’t point to it.)
Claims: 𝜇𝜇𝑋𝑋𝑌𝑌 indist. from 𝜇𝜇mid ⇐ PVP is hard 𝜇𝜇mid indist. from 𝜇𝜇𝑋𝑋 × 𝜇𝜇𝑌𝑌 ⇐ (Unique) disjointness
hard.
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 14
𝑖𝑖
⋮
of 17
Bulk of paper: PVP is hard
Main idea: “Round elimination” a la NW ’93 No black-box use No simple variant
Induction on #rounds Many invariants … roughly
𝐻𝐻 𝜋𝜋odd,𝜋𝜋even transcript) ≈ 𝑀𝑀𝑀𝑀𝑀𝑀 − 𝑂𝑂 𝐶𝐶 𝐻𝐻 𝑖𝑖𝑡𝑡 transcript) ≈ log𝑛𝑛 − 𝑜𝑜 1 𝑋𝑋 ⫫ 𝑌𝑌 | path, transcript
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 15
of 17
Conclusions
Many interesting problems in distributed randomness manipulation.
Complexity of the “Single-letter characterizations”.
Tight characterization of round complexity of CRG.
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 16
of 17
Thank You!
May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 17