communicationcomplexity1_jieren

77
Communication Complexity Jie Ren Adaptive Signal Processing and Information Theory Group Nov 3 rd , 2014 Jie Ren (Drexel ASPITRG) CC Nov 3 rd , 2014 1 / 77

Upload: jie-ren

Post on 07-Aug-2015

16 views

Category:

Documents


0 download

TRANSCRIPT

Communication Complexity

Jie Ren

Adaptive Signal Processing and Information Theory Group

Nov 3rd, 2014

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 1 / 77

1 E. Kushilevitz and N. Nisan, “Communication Complexity,”Cambridge University Press, 1997.

2 L. Lovasz, “Communication Complexity: A Survey,” in Paths, Flows,and VLSI Layout, B. H. Korte, Ed., Springer Verlag: Berlin 1990.

3 T. Lee and A. Shraibman, “Lower Bounds in CommunicationComplexity: A Survey,” Now Publishers Inc., 2009.

4 A. C. Yao, “Some Complexity Questions Related to DistributedComputing,” Proc. of 11th ACM Symposium on Theory ofComputing, 1981, 308-311.

5 P. Beame and J. Lawry, “Randomized versus NondeterministicCommunication Complexity,” Proc. of 24th ACM Symposium onTheory of Computing, 1992, 188-199.

6 A. K. Chandra, M. L. Furst and R. J. Lipton, “Multi-partyProtocols,” Proc. of 15th ACM Symposium on Theory of Computing,1983, 94-99.

7 P. B. Miltersen, N. Nisan, S. Safra and A. Wigderson, “On DataStructures and Asymmetric Communication Complexity,” Proc. of27th ACM Symposium on Theory of Computing, 1995, 103-111.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 2 / 77

Deterministic Communication Complexity

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 3 / 77

Deterministic Communication Complexity Problem Setup

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 4 / 77

Deterministic Communication Complexity Problem Setup

Problem Setup

Alice Bob

x ∈ X y ∈ Y

f(x, y) ∈ {0, 1}· · ·f(x, y)

0/1

0/1

• Two party communication

• Each knows an input x ∈ X/y ∈ Y• Let one/both sides compute a function f with no error

• Only care about communication cost

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 5 / 77

Deterministic Communication Complexity Problem Setup

Problem Setup

Alice Bob

x ∈ X y ∈ Y

f(x, y) ∈ {0, 1}· · ·f(x, y)

0/1

0/1

• Sending binary messages

• f usually binary

• Deterministic protocol P: who to talk/what to send

• Communication cost: sum of total bits/rounds CC (P)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 6 / 77

Deterministic Communication Complexity Problem Setup

Deterministic Communication Complexity

Alice Bob

x ∈ X y ∈ Y

f(x, y) ∈ {0, 1}· · ·f(x, y)

0/1

0/1

D(f ) = minP

max(x ,y)∈X×Y

CC (P) (1)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 7 / 77

Deterministic Communication Complexity Problem Setup

A Naive Upper Bound

Proposition (naive upper bound): For every function f : X × Y → Z

D(f ) ≤ log2 |X |+ log2 |Z | (2)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 8 / 77

Deterministic Communication Complexity Problem Setup

A Naive Upper Bound

Example: MAX of the unionAlice and Bob hold subsets x , y ⊆ {1, . . . , n} respectively, and they with tocompute MAX (x , y).

D(MAX ) ≤ 2 log2 n (3)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 9 / 77

Deterministic Communication Complexity Protocol Tree

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 10 / 77

Deterministic Communication Complexity Protocol Tree

Definition: Protocol Tree

Definition: A protocol P over domain X × Y with range Z is a binary tree whereeach internal node v is labeled either by a function av : X → {0, 1} or by afunction bv : Y → {0, 1}, and each leaf is labeled with an element z ∈ Z .The communication cost CC (P) will be the depth of the protocol tree.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 11 / 77

Deterministic Communication Complexity Protocol Tree

Example: Protocol Tree

a1(x = 1, 2) = 0

a1(x = 3) = 1

b2(y = 1, 2) = 0

b2(y = 3) = 1

0b3(y = 1) = 1

b3(y = 2, 3) = 0

1

1

a4(x = 1) = 0

a4(x = 2, 3) = 1

0 1

f(x, y) =

{1 x ≥ y0 OTH

x ∈ {1, 2, 3}y ∈ {1, 2, 3}

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 12 / 77

Deterministic Communication Complexity Protocol Tree

Why Binary Message?

• Entropy not involved - simple?

• No block coding (compute a single function)

• Worst case - always exists p = 1/2 s.t. h2(p) = 1

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 13 / 77

Deterministic Communication Complexity Protocol Tree

One Side Compute f Vs. Both Sides Compute f

• Equivalent setup

• ⇒ Need one more bit if f is binary

• ⇐ Second last round: one side must know f (x , y)

Some Lower Bounds of Communication Complexity

• D(f ) : unknown for general f

• Interested in lower bounds

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 14 / 77

Deterministic Communication Complexity Combinatorial Rectangles

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 15 / 77

Deterministic Communication Complexity Combinatorial Rectangles

Combinatorial Rectangles

Definition: Let P be a protocol and v be a node of the protocol tree. Rv

is the set of inputs (x , y) that reach node v .Proposition: If L is the set of leaves of a protocol P, then {R`, ` ∈ L} is apartition of X × Y .Definition: A combinatorial rectangle is a subset R ⊆ X × Y such thatR = A× B for some A ⊆ X and B ⊆ Y .

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 16 / 77

Deterministic Communication Complexity Combinatorial Rectangles

Proposition: R ⊆ X × Y is a rectangle iff

(x1, y1) ∈ R & (x2, y2) ∈ R ⇒ (x1, y2) ∈ R. (4)

Proposition: For every protocol P and leaf `, R` is a rectangle

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 17 / 77

Deterministic Communication Complexity Combinatorial Rectangles

a1(x = 1, 2) = 0

a1(x = 3) = 1

b2(y = 1, 2) = 0

b2(y = 3) = 1

0b3(y = 1) = 1

b3(y = 2, 3) = 0

1

1

a4(x = 1) = 0

a4(x = 2, 3) = 1

0 1

x=1

x=3

x=2

y=1 y=2 y=3

1 0 0

1

1

1 0

1 1

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 18 / 77

Deterministic Communication Complexity Combinatorial Rectangles

Rectangle lower bound

Definition: A subset R ⊆ X × Y is called f -monochromatic if f is fixedon R.Theorem 1.17 (Kushilevitz & Nisan): If any partition of X × Y intof -monochromatic rectangles requires at least t rectangles, then

log2 t ≤ D(f ) (5)

• P partitions X × Y into monochromatic rectangles

• Depth of its protocol tree: ≥ log2 t

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 19 / 77

Deterministic Communication Complexity Combinatorial Rectangles

x=1

x=3

x=2

y=1 y=2 y=3

1 0 0

1

1

1 0

1 1

D(f) ≥ log2 5

D(f) = 3

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 20 / 77

Deterministic Communication Complexity Fooling Sets

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 21 / 77

Deterministic Communication Complexity Fooling Sets

Motivation: If we exhibit a large set of input pairs such that no two ofthem can be in a single monochromatic rectangle, then the number ofpartitions of P must be large

z ?

? z

x1

y2y1

x2

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 22 / 77

Deterministic Communication Complexity Fooling Sets

Definition : Let f : X × Y → {0, 1}. A set S ⊆ X × Y is called a foolingset if there exits a value z ∈ {0, 1} such that

• For every (x , y) ∈ S , f (x , y) = z

• For every two distinct pairs (x1, y1) and (x2, y2) in S , eitherf (x1, y2) 6= z or f (x2, y1) 6= z

z ?

? z

x1

y2y1

x2

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 23 / 77

Deterministic Communication Complexity Fooling Sets

Fooling set lower bound

Theorem 1.20 (Kushilevitz & Nisan) : If f has a fooling set S of size t,then

log2 t ≤ D(f ) (6)

Proof : No monochromatic rectangle contains more than one element of S

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 24 / 77

Deterministic Communication Complexity Fooling Sets

Example: Alice and Bob each hold an n-bit integer 0 ≤ x , y < 2n. The“greater than or equal to” function, GTE (x , y), is defined to be 1 iffx ≥ y .

D(GT ) = n + 1 (7)

x=0

x=1

x=2

x=3

y=0 y=1 y=2 y=3

1

1

1

1

0 0 0

1 0 0

1 1 0

1 1 1

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 25 / 77

Deterministic Communication Complexity Fooling Sets

Example: Alice and Bob each hold an n-bit integer 0 ≤ x , y < 2n. The“greater than or equal to” function, GTE (x , y), is defined to be 1 iffx ≥ y .

D(GT ) = n + 1 (8)

x=0

x=1

x=2

x=3

y=0 y=1 y=2 y=3

1

1

1

1

0 0 0

1 0 0

1 1 0

1 1 1

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 26 / 77

Deterministic Communication Complexity Rectangle Rank

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 27 / 77

Deterministic Communication Complexity Rectangle Rank

Motivation: Give communication complexity lower bound in an algebraicwayDefinition: Associate with every function f : X × Y → {0, 1} a matrixMf of dimensions |X | × |Y |. The rows/columns of Mf are indexed by theelements of X/Y . Then rank(f ) is the linear rank of Mf over the field ofreals.

x=0

x=1

x=2

x=3

y=0 y=1 y=2 y=3

1

1

1

1

0 0 0

1 0 0

1 1 0

1 1 1

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 28 / 77

Deterministic Communication Complexity Rectangle Rank

Rank lower bound

Theorem 1.28 (Kushilevitz & Nisan): For any functionf : X × Y → {0, 1}

log2 rank(f ) ≤ D(f ) (9)

x=0

x=1

x=2

x=3

y=0 y=1 y=2 y=3

1

1

1

1

0 0 0

1 0 0

1 1 0

1 1 1

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 29 / 77

Deterministic Communication Complexity Rectangle Rank

Proof: Let L1 be the set of leaves in which the output is 1. For each` ∈ L1,

M`(x , y) =

{1 if (x , y) ∈ R`

0 otherwise(10)

Mf =∑

`∈L1

M` (11)

andrank(Mf ) ≤

`∈L1

rank(M`) ≤ |L1| ≤ |L| (12)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 30 / 77

Deterministic Communication Complexity Rectangle Rank

Rank lower bound

Theorem 1.28 (Kushilevitz & Nisan): For any functionf : X × Y → {0, 1}

log2 rank(f ) ≤ D(f ) (13)

x=0

x=1

x=2

x=3

y=0 y=1 y=2 y=3

1

1

1

1

0 0 0

1 0 0

1 1 0

1 1 1

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 31 / 77

Deterministic Communication Complexity Rectangle Rank

Rank upper bound

Proposition 2.3 (Lovasz 1990): For any function f : X × Y → {0, 1}

D(f ) ≤ rank(f ) (14)

Proof: We know that row rank = column rank = rank(f ), and we canform the row vector space with dimension rank(f ). We then claim thatthere are at most 2rank(f ) distinct row vectors, the reason is because,although the coefficients for the polynomials that represent the rowvectors can be real, the entries of the matrix M(f ) can only be 0 or 1.Hence we can build a protocol as follows: Alice merge the repeated rowsof M(f ) on the table to have M ′(f ), and then sends the index of row inM ′(f ) that contains x . Bob compute f (x , y) based on what he received.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 32 / 77

Deterministic Communication Complexity Rectangle Rank

Summary

• Concept: protocol tree, combinatorial rectangles, fooling sets, rank

• Naive upper bound: log2 |X |+ 1

• Rectangle lower bound: log2 tr

• Fooling set lower bound: log2 tf

• Rank lower bound: log2 rank(f )

• Rank upper bound: rank(f )

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 33 / 77

Nondeterministic CC & Randomized CC

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 34 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: Motivation

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 35 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: Motivation

Motivation

• How good are the rectangle lower bounds?

• Relaxing the need to partition by allowing covering of the same space

x=0

x=1

x=2

x=3

y=0 y=1 y=2 y=3

1

1

0

0

1 0 0

1 1 0

1 1 0

0 0 0

x=0

x=1

x=2

x=3

y=0 y=1 y=2 y=3

1

1

0

0

1 0 0

1 1 0

1 1 0

0 0 0

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 36 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: Motivation

Motivation: Alice has a n-bit string x ∈ {0, 1}n, Bob has a n-bit stringy ∈ {0, 1}n, either side wants NEQ(x , y).

D(NEQ) = n (15)

Now assume a third person knows everything: x ,y and NEQ(x , y) andwant to convince Alice and Bob, Alice and Bob need to check thecorrectness

• Sends the index of the first bit differs

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 37 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: Motivation

Setup: A prover, who sees both x and y , is trying to convince Alice andBob that “f (x , y) = 1”. If f (x , y) 6= 1, then Alice and Bob must be ableto detect the proof is wrong.

f (x , y) = 1⇒ ∃ z P(x , y , z) = 1 (16)

f (x , y) = 0⇒ ∀ z P(x , y , z) = 0 (17)

• Two-stage nondeterministic protocol PN

1 Alice and Bob receive a message z from the third person.2 Alice and Bob run a deterministic protocol PD,z based on z .

• The interesting cost in this protocol is the maximum length of z plusthe number of bits exchanged over all x , y .

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 38 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: Motivation

Alternative Setup: Let f : X × Y → {0, 1} be a function. LetL = {(x , y) : f (x , y) = 1}. A successful nondeterministic protocol for fconsists of functions fA : X × {0, 1}k → {0, 1} andfB : Y × {0, 1}k → {0, 1} such that

1 ∀(x , y) ∈ L, ∃z ∈ {0, 1}k s.t. fA(x , z) ∧ fB(y , z) = 1

2 ∀(x , y) 6∈ L, ∀z ∈ {0, 1}k , fA(x , z) ∧ fB(y , z) = 0

• One stage nondeterministic protocol

1 Alice and Bob receive a message z and compute f (x , y) successfully.

• The interesting cost in this protocol is the length of z only.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 39 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: Motivation

Two-stage ⇒ One-stage: Given a two-stage nondeterministic protocolwith k bits first stage cost and d bits second stage cost, we can alwaysbuild a one-stage nondeterministic protocol by adding the d bitsdeterministic communication to the witness z with each party accepting ifthe message agrees with what Alice and Bob would have said in theprotocol.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 40 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: Motivation

In the language of “Rectangles”: A prover, who sees both x and y , istrying to convince Alice and Bob that “f (x , y) = 1” by broadcasting a1-monochromatic rectangle that cover (x , y).

• By “Nondeterministic” we mean: this 1-monochromatic rectanglemay not be unique

x=0

x=1

x=2

x=3

y=0 y=1 y=2 y=3

1

1

0

0

1 0 0

1 1 0

1 1 0

0 0 0

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 41 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: definitions

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 42 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: definitions

Definitions: Let f : X × Y → {0, 1} be a binary function.

• CP(f ): the smallest number of leaves in a protocol P• CD(f ): the smallest number of monochromatic rectangles that

partition X × Y

• C (f ): the smallest number of monochromatic rectangles needed tocover X × Y

• C z(f ): the smallest number of monochromatic rectangles needed tocover the z-inputs

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 43 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: definitions

Proposition 2.2 (Kushilevitz & Nisan): For all f : X × Y → {0, 1},

log2(C 0(f ) + C 1(f )

)≤ log2 C

D(f ) ≤ log2 CP(f ) ≤ D(f ) (18)

Theorem 29 (Lee & Shraibman): Let f : X ×Y → {0, 1} be a function,

N1(f ) = dlog2 C1(f )e (19)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 44 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: definitions

Proof:

• N1(f ) ≤ dlog2 C1(f )e: Let {R`} be a cover. If f (x , y) = 1, the

players receive the index ` that (x , y) ∈ R`• N1(f ) ≥ dlog2 C

1(f )e: Let k = N1(f ), and letfA : X × {0, 1}k → {0, 1}, fB : Y × {0, 1}k → {0, 1} be functions inthe one-stage nondeterministic protocol. DefineRz = {(x , y) : fA(x , z) ∧ fB(y , z) = 1}, Rz is a rectangle. We claim{Rz , z ∈ {0, 1}k} is a cover of the 1s. This is because by thedefinition of nondeterministic protocol:

• ∀(x , y) pairs that f (x , y) = 1, there must exists some z s.t.(x , y) ∈ Rz .

• ∀(x , y) pairs that f (x , y) = 0, (x , y) 6∈ Rz for all z .

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 45 / 77

Nondeterministic CC & Randomized CC Nondeterministic Communication Complexity: definitions

Definition (Lee & Shraibman):

N1(f ) = dlog2 C1(f )e (20)

N0(f ) = dlog2 C0(f )e (21)

N(f ) = max(N1(f ),N0(f )) (22)

Definition (Kushilevitz & Nisan):

N1(f ) = log2 C1(f ) (23)

N0(f ) = log2 C0(f ) (24)

N(f ) = log2(C 0(f ) + C 1(f )

)(25)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 46 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 47 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Motivation:

• Introduce randomness in the protocol rA and rB : flip coins

• Allow protocols that may have error

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 48 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Randomized Protocol Tree

Definition: A randomized protocol P over domain X × Y with range Z is a binarytree where each internal node v is labeled either by a functionav : X × RA → {0, 1} or by a function bv : Y × RB → {0, 1}, and eachleaf is labeled with an element z ∈ Z .

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 49 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Definition: Let P be a randomized protocol. All the probabilities beloware over random choices of rA and rB .

• P computes a function f with zero error

• P computes a function f with ε−error if for all (x , y)

P[P(x , y) = f (x , y)] ≥ 1− ε (26)

• P computes a function f with one-sided ε−error if for all (x , y) suchthat f (x , y) = 0

P[P(x , y) = 0] = 1, (27)

and for all (x , y) such that f (x , y) = 1,

P[P(x , y) = 1] ≥ 1− ε (28)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 50 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Definition: Let f : X × Y → {0, 1} be a binary function. We consider thefollowing complexity measure for f

• R0(f ) is the minimum average case cost of a randomized protocolthat computes f with zero error

• Rε(f ) is the minimum worst case cost of a randomized protocol thatcomputes f with error ε. We typically use ε = 1/3

• R1ε (f ) is the minimum worst case cost of a randomized protocol that

computes f with one-sided error ε.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 51 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Why we care these measures:

• worst case zero error = deterministic

• for all average case ε error, there exists a worst case problem that canconvert to

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 52 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Proposition: Given a protocol P that makes an error ε/2 and the averagenumber of bits exchanged is t, it can always be modified as follows:execute P as long as at most 2t/ε bits are exchanged, if the protocolfinishes, use its output, otherwise output 0. This gives a worst case cost2t/ε with error upper bounded by ε.Proof:

t =∑

ra,rb,x ,y

cc · p(ra, rb, x , y)

=∑

cc≤2t/ε

cc · p(ra, rb, x , y) +∑

cc>2t/ε

cc · p(ra, rb, x , y)

≥∑

cc>2t/ε

cc · p(ra, rb, x , y) ≥ 2t

εPr [cc > 2t/ε]

(29)

Hence,

Pr [err ] ≤ ε

2Pr [P ends] + 1 · Pr [P not ends]

≤ ε

2+

t

2t/ε= ε

(30)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 53 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

1 For all 0 < ε ≤ ε′ < 1/2,

Rε(f ) ≤ O(logε′ ε · Rε′ (f )) (31)

2 For all 0 < ε ≤ 1/2,

Rε(f ) ≤ R1ε (f ) ≤ O(log ε−1R0(f )) (32)

3 For all 0 < ε ≤ 1/2,

R0(f ) = Θ(max[R1ε (f ),R1

ε (not(f ))]) (33)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 54 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Proof of Property 1: We first prove a similar result for the 1-sided errorproblem: for all 0 < ε ≤ ε′ < 1/2,

R1ε (f ) ≤ O(logε′ ε · R1

ε′ (f )) (34)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 55 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Proof of Property 1: Given a randomized protocol P with worst casecost T bits and one-sided error no greater than ε

′< 1/2, we can build a

new protocol P ′with worst case cost nT bits by simply running P n

times. In the new protocol, Alice and Bob will claim f (x , y) = 1 if andonly if there exists at least one time among the n repeating protocols thatthey will output 1. Now we bound the error for the new protocol P ′

:

P[err |f (x , y) = 0] = 0 (35)

P[err |f (x , y) = 1] = P[all n trails output 0|f (x , y) = 1]

= (ε′)n

(36)

Therefore, if we repeat P logε′ ε times, we can guarantee to reduce theone-sided error to ε.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 56 / 77

Nondeterministic CC & Randomized CC Randomized Communication Complexity

Proof of Property 1: Now we prove property 1. We still run P n times,each gives an output Xi , i ∈ {1, . . . , n}. In the new protocol, Alice andBob will claim f (x , y) = 1 if and only if

1

n

i

Xi >1

2(37)

Now we bound the error for the new protocol P ′:

P[err |f (x , y) = 0] ≤n∑

i=dn/2e

(n

i

)(ε

′)i (1− ε′)n−i

≤ (ε′)n

(38)

P[err |f (x , y) = 1] ≤n∑

i=dn/2e

(n

i

)(ε

′)i (1− ε′)n−i

≤ (ε′)n

(39)

Therefore, if we repeat P logε′ ε times, we can also guarantee to reducethe two-sided error to ε.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 57 / 77

Nondeterministic CC & Randomized CC Distributional Complexity and Discrepancy

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 58 / 77

Nondeterministic CC & Randomized CC Distributional Complexity and Discrepancy

Distributional Complexity

Motivation: Consider probability distributions over the inputsDefinition: Let µ be a probability distribution on X × Y . The(µ, ε)-distributional communication complexity of f , Dµ

ε (f ), is the cost ofthe best deterministic protocol that gives the correct answer for f with aprobability at least 1− ε.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 59 / 77

Nondeterministic CC & Randomized CC Distributional Complexity and Discrepancy

Discrepancy

Motivation: Allow those rectangles that partition the support to be“almost” f -monochromatic.Definition: Let f : X × Y → {0, 1} be a function, R be any rectangle,and µ be a probability distribution on X × Y , Denote

Discµ(R, f ) = |Pµ

[f (x , y) = 0 & (x , y) ∈ R]− Pµ

[f (x , y) = 1 & (x , y) ∈ R]|(40)

The discrepancy of f according to µ is,

Discµ(f ) = maxR

Discµ(R, f ) (41)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 60 / 77

Nondeterministic CC & Randomized CC Distributional Complexity and Discrepancy

Discrepancy

Proposition 3.28 (Kushilevitz & Nisan): For every functionf : X × Y → {0, 1}, every probability distribution µ on X × Y , and everyε ≥ 0,

Dµ1/2−ε(f ) ≥ log2(2ε/Discµ(f )) (42)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 61 / 77

Nondeterministic CC & Randomized CC Distributional Complexity and Discrepancy

Discrepancy

Proof: Given any P with c bits to compute f , we have

2ε ≤ P[P(x , y) = f (x , y)]− P[P(x , y) 6= f (x , y)]

=∑

`

(P[P(x , y) = f (x , y)&(x , y) ∈ R`]

−P[P(x , y) 6= f (x , y)&(x , y) ∈ R`])

≤∑

`

|Pµ

[f (x , y) = 0 & (x , y) ∈ R`]− Pµ

[f (x , y) = 1 & (x , y) ∈ R`]|

≤ 2c · Discµ(f )(43)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 62 / 77

Some Analysis

Outline

1 Deterministic Communication ComplexityProblem SetupProtocol TreeCombinatorial RectanglesFooling SetsRectangle Rank

2 Nondeterministic CC & Randomized CCNondeterministic Communication Complexity: MotivationNondeterministic Communication Complexity: definitionsRandomized Communication ComplexityDistributional Complexity and Discrepancy

3 Some Analysis

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 63 / 77

Some Analysis

Recall

Definitions: Let f : X × Y → {0, 1} be a binary function.

• CP(f ): the smallest number of leaves in a protocol P• CD(f ): the smallest number of monochromatic rectangles that

partition X × Y

• C (f ): the smallest number of monochromatic rectangles needed tocover X × Y

• C z(f ): the smallest number of monochromatic rectangles needed tocover the z-inputs

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 64 / 77

Some Analysis

Recall

Proposition: For all f : X × Y → {0, 1},

log2 C (f ) ≤ log2 CD(f ) ≤ log2 C

P(f ) ≤ D(f ) (44)

C (f ) = C 0(f ) + C 1(f ) (45)

Definition: The nondeterministic communication complexity,

N1(f ) = log2 C1(f ) (46)

N0(f ) = log2 C0(f ) (47)

N(f ) = log2 C (f ) (48)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 65 / 77

Some Analysis

Protocol partition number

Theorem 2.8 (Kushilevitz and Nisan): The protocol partition numberof a function determines the deterministic communication complexity.

log2 CP(f ) ≤ D(f ) ≤ 2 log3/2 C

P(f ) (49)

Proof: Given any protocol P with t number of leaves, it can be convertedinto a “balanced” protocol.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 66 / 77

Some Analysis

Protocol partition number

Proof: Given any protocol P with t number of leaves, there must exist aninternal node v such that

t/3 < tv ≤ 2t/3 (50)

We build a new protocol based on this internal node:

1 Alice and Bob determine whether or not (x , y) ∈ Rv

2 If (x , y) ∈ Rv , Alice and Bob recursively solve f in the rectangle Rv .

3 If (x , y) 6∈ Rv , Alice and Bob recursively solve f ′ on X × Y where

f ′(x1, y1) =

{f (x1, y1) if (x1, y1) 6∈ Rv

0 otherwise(51)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 67 / 77

Some Analysis

Protocol partition number

Analysis:

• Step 1 Requires 2 bits

• In Step 3, we take P and replace Tree(v) by a 0-leaf, we get aprotocol for f ′ with t − tv + 1 leaves, hence

D(t) ≤ 2 + D(2t/3) (52)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 68 / 77

Some Analysis

Recall

Proposition: For all f : X × Y → {0, 1},

log2 C (f ) ≤ log2 CD(f ) ≤ log2 C

P(f ) ≤ D(f ) ≤ 2 log3/2 CP(f ) (53)

Definition: The nondeterministic communication complexity,

N1(f ) = log2 C1(f ) (54)

N0(f ) = log2 C0(f ) (55)

N(f ) = log2 C (f ) (56)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 69 / 77

Some Analysis

Deterministic CC Vs. Nondeterministic CC

How good is the rectangle lower bound?

D(f )?= O(logCD(f )) (57)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 70 / 77

Some Analysis

Deterministic CC Vs. Nondeterministic CC

Theorem 2.11 (Kushilevitz & Nisan): For every functionf : X × Y → {0, 1},

D(f ) = O(N0(f )N1(f )) (58)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 71 / 77

Some Analysis

Deterministic CC Vs. Nondeterministic CC

Property: Let R = S × T be a 0-monochromatic rectangle, and letR ′ = S ′ × T ′ be a 1-monochromatic rectangle, then either S ∩ S ′ = ∅ orT ∩ T ′ = ∅.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 72 / 77

Some Analysis

Deterministic CC Vs. Nondeterministic CC

Proof of Theorem 2.11 (Kushilevitz & Nisan): We give a protocol Pas follows, Alice and Bob search for a 0-rectangle that contains the input(x , y), and they conclude f (x , y) = 1 if they fail. In each round, Alice andBob exchange log2 C

1(f ) + 1 bits and reduce the number of “alive”0-rectangles by a factor of 2. There will be no more than log2 C

0(f )rounds, hence

D(f ) ≤ CC (P) = O(log2 C0(f )(log2 C

1(f ) + 1)) = O(N0(f )N1(f )) (59)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 73 / 77

Some Analysis

Deterministic CC Vs. Nondeterministic CC

Proof of Theorem 2.11 (Kushilevitz & Nisan): In each round, theplayers do the following:

1 Alice outputs f (x , y) = 0 if no 0-rectangles are alive. Otherwise, Alicelooks for a 1-rectangle that contains row x and intersects in rows withat most half of the alive 0-rectangles and send the name of this1-rectangle.

2 Bob looks for a 1-rectangle that contains column y and intersects incolumns with at most half of the alive 0-rectangles and send thename of this 1-rectangle. Otherwise, Bob outputs f (x , y) = 0

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 74 / 77

Some Analysis

Deterministic CC Vs. Nondeterministic CC

Protocol Analysis:

• If f (x , y) = 0, it must belong to some 0-rectangle R, then R remainsalive during the protocol. Therefore, if no 0-rectangle is alive, f (x , y)must be 1

• If neither Alice nor Bob can find a 1-rectangle to announce (whichmeans both of them output f (x , y) = 1), we claim this output mustbe correct by the property.

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 75 / 77

Some Analysis

Public coin

Theorem (Theo. 3 in Newman 1991, Theo. 3.14 in K & N): Letf : {0, 1}n × {0, 1}n → {0, 1} be a function. For every δ > 0 and everyε > 0, we have

Rδ+ε(f ) ≤ Rpubε (f ) + O(logn + logδ−1) (60)

• ∃ a set of t(δ, n) = O(n/δ2) public coin protocols with error ε+ δ

• Alice tells Bob which protocol to use log2 t = O(log n + log δ−1)

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 76 / 77

Some Analysis

Randomized CC Vs. Distributional CC

Theorem (Theo. 3 in Yao 1979, Theo. 3.20 in K & N):

Rpubε (f ) = max

µDµε (f ) (61)

• ⇒ The randomized protocol is correct for every distribution µ withprobability ≥ 1− ε

• ⇐ Use min-max theorem of zero-sum game

Jie Ren (Drexel ASPITRG) CC Nov 3rd, 2014 77 / 77