UnconditionalWeak derandomization of weak
algorithms
Explicit versions of Yao’s lemma
Ronen Shaltiel, University of Haifa
:
Derandomization: The goalMain open problem: Show that BPP=P.
(There is evidence that this is hard [IKW,KI])
More generally:Convert: randomized algorithm A(x,r)into: deterministic algorithm B(x)
We’d like to:1. Preserve complexity:
complexity(B) ≈ complexity(A) (known BPPEXP).2. Preserve uniformity:
transformation A → B is “explicit” (known BPPP/poly).
n bit long input m bit long “coin tosses”
Strong derandomization is sometimes impossible
Setting: Communication complexity.
x=(x1,x2) where x1,x2 are shared between two players.
Exist randomized algorithms A(x,r) (e.g. for Equality) with logarithmic communication complexity s.t. any deterministic algorithm B(x) requires linear communication.
Impossible to derandomize while preserving complexity.
(The easy direction of) Yao’s Lemma: A straight forward averaging argument
Given randomized algorithm that computes a function f with success 1-ρ on the worst case, namely:
Given A:{0,1}n£{0,1}m →{0,1} s.t.
∀x: PrR←Um[A(x,R)=f(x)]¸1-ρ
r’2{0,1}m s.t. the deterministic algorithm B(x)=A(x,r’)
computes f well on average, namely:
PrX←Un[B(X)=f(X)]¸1-ρ
Useful tool in bounding randomized algs. Can also be viewed as “weak derandomization”.
Yao’s lemma as “weak derandomization”
Advantages
Applies to any family of algorithms and any complexity measure.
Communication complexity. Decision tree complexity. Circuit complexity classes.
Construction: B(x)=A(x,r’) preserves complexity.
e.g. if A has low communication complexity then B has low communication complexity.
Drawbacks
Weak derandomization is weak: deterministic alg B succeeds on most but not all inputs.
Let’s not be too picky. In some scenarios (e.g.
communication complexity) strong derandomization is impossible.
The argument doesn’t give an explicit way to find r’ and produce B(x)=A(x,r’).
Uniformity is not preserved: Even if A is uniform we only get that B(x)=A(x,r’) is nonuniform (B is a circuit).
The goal:Explicit versions of Yao’s Lemma
Given randomized algorithm that computes a function f with success 1-ρ on the worst case, namely:
Given A:{0,1}n£{0,1}m →{0,1} s.t.
∀x: PrR←Um[A(x,R)=f(x)]¸1-ρ
Give explicit construction of a deterministic algorithm B(x) s.t.:
B computes f well on average, namely:
PrX←Un[B(X)=f(X)]¸1-ρ
Complexity is preserved: complexity(B) ≈ complexity(A).
We refer to this as “Explicit weak derandomization”.
Adelman’s theorem (BPPP/poly) follows from Yao’s lemma
Given randomized algorithm A that computes a function f with success 1-ρ on the worst case.
1. (amplification) amplify success prob. to 1-ρ for ρ=2-(n+1)
2. (Yao’s lemma) deterministic circuit B(x) such that:
b := PrX←Un[B(X)≠f(X)]<ρ<2-(n+1) ⇒ b=0 ⇒ B succeeds on all inputs.
Corollary: Explicit version of Yao’s lemma for general poly-time algorithms ⇒ BPP=P.
Reminder of talk: Explicit versions of Yao’s lemma for “weak algorithms”: Communication games, Decision trees, Streaming algorithms, AC0 algorithms.
Related work: Extracting randomness from the input
Idea [Goldreich and Wigderson]: Given a randomized alg A(x,r) s.t. |r|·|x| consider the deterministic alg:
B(x)=A(x,x).
Intuition: If input x is chosen at random then random coins r:=x is chosen at random.
Problem: Input and coins are correlated. (e.g. consider A s.t. 8input x, coin x is bad for x).
GW: Does work if A has the additional property that whether or not a coin toss is good does not depend on the input.
GW: It turns out that there are A’s with this property.
The role of extractors in [GW]
In their paper Goldreich and Wigderson actually use:
B(x)=maj seeds y A(x,E(x,y))
Where E(x,y) is a “seeded extractor”.
Extractors are only used for “deterministic amplification” (that is to amplify success probability).
Alternative view of the argument:1. Set A’(x,r)=maj seeds y A(x,E(r,y))
2. Apply construction B(x)=A’(x,x).
Randomness extractors
Daddy, how do
computers get random
bits?
Do we have to tell that same old story again?
Seeded
Randomness Extractors: Definition and two flavors
C is a class of distributions over n bit strings “containing” k bits of (min)-entropy.
A deterministic (seedless) C-extractor is a function E such that for every XєC, E(X) is ε-close to uniform on m bits.
A seeded extractor has an additional (short i.e. log n) independent random seed as input.
For Seeded extractors C={all X with min-entropy ¸ k}
source distribution from C
Extractorseed
random output
Deterministic
• A distribution X has min-entropy ≥ k if ∀x: Pr[X=x] ≤ 2-k
• Two distributions are ε-close if the probability they assign to any event differs by at most ε.
Zimand: explicit version of Yao’s lemma for decision trees
Zimand defines and constructs a stronger variant of seeded extractors E(x,y) called “exposure resilient extractors” . He considers:
B(x)=maj seeds y A(x,E(x,y))
Thm: [Zimand07] If A is a randomized decision tree with q queries that tosses q random coins then:
B succeeds on most inputs. (a (1-ρ)-fraction). B can be implemented by a deterministic
decision tree with qO(1) queries.
Zimand states his result a bit differently.
We improve to
O(q)
Our results Develop a general technique to prove explicit
versions of Yao’s Lemma (that is weak derandomization results).
Use deterministic (seedless) extractors that is B(x)=A(x,E(x)) where E is a seedless extractor.
The technique applies to any class of algorithms with |r|·|x|. Can sometimes handle |r|>|x| using PRGs.
More precisely: Every class of randomized algorithm defines a class C of distributions. An explicit construction of an extractor for C immediately gives an explicit version of Yao’s Lemma (as long as |r|·|x|).
Explicit version of Yao’s lemma for communication games
Thm: If A is a randomized (public coin) communication game with communication complexity q that tosses m<n random coins then set B(x)=A(x,E(x)) where E is a “2-source extractor”.
B succeeds on most inputs. A (1-ρ)-fraction (or even a (1-2-(m+q))-fraction).
B can be implemented by a deterministic communication game with communication complexity O(m+q).
Dfn: A communication game is explicit if each party can compute its next message in poly-time (given his input, history and random coins).
Cor: Given an explicit randomized communication game with complexity q and m coins there is an explicit deterministic communicaion game with complexity O(m+q) that succeeds on a (1-2-(m+q)) fraction of the inputs.
Both complexity and uniformity are
preserved
Explicit weak derandomization results
AlgorithmsExtractors
Decision trees(Improved alternative proof of Zimand’s result).
Extractors for bit-fixing sources [KZ03,GRS04,R07]
Communication games2-source extractors [CG88,Bou06]
Streaming algorithms (can handle |r| ¸ |x|).
Construct from 2-source extractors. Inspired by [KM06,KRVZ06].
AC0 (constant depth) (can handle |r| ¸ |x|).
We construct inspired by PRGs for AC0. [N,NW]
Poly-time algorithms (can handle |r| ¸ |x|).
We construct using “low-end” hardness assumptions.
Constant depth algorithmsConsider randomized algorithms A(x,r) that are computable
by uniform families of poly-size constant depth circuits.
[NW,K] : Strong derandomization in quasi-poly time. Namely, there is a uniform family of quasi-poly-size circuits that succeed on all inputs.
Our result: Weak derandomization in poly-time. Namely, there is a uniform family of poly-size circuits that succeed on most inputs. (can also preserve constant depth).
High level idea: Reduce # of random coins of A from nc to (log n)O(1) using
a PRG. (Based on the hardness of the parity function [H,RS])
Extract random coins from input x using an extractor for “sources recognizable by AC0 circuits”.
Construct extractors using the hardness of the parity function and ideas from [NW,TV].
High level overview of the proof
To be concrete we consider communication games
PreparationsThm: If A is a randomized communication game with
communication complexity q that tosses m random coins then set B(x)=A(x,E(x)) where E is a 2-source extractor.
B succeeds on most inputs. A (1-ρ)-fraction.
B can be implemented by a deterministic communication game with communication complexity O(m+q).
Define independent random variables X,R by: X←Un, R←Um.
We have that: ∀x: Pr[A(x,R)=f(x)]¸1-ρ
It follows that: a := Pr[A(X,R)=f(X)]¸1-ρ
We need to show: b := Pr[A(X,E(X))=f(X)]¸1-ρ – (2ρ+2-2m).
The plan is to show that b ¸ a – (2ρ + 2-2m).
High level intuition
At the end of protocol all inputs in a rectangle answer the same way.
Consider the entropy in the variable X|rectangle = (X|Qr(X)=v).
Independent of answer. Idea: extract the randomness from this
entropy.Doesn’t make sense: rectangle is defined
only after random coins r are fixed.
• For every choice of random coins r the game A(∙,r) is deterministic w/complexity q.
• It divides the set of strings x of length n into 2q rectangles.
• Let Qr(x) denote the rectangle of x.
x1
x2
Qr(x1,x2
)
Rectangle
=
2-source
Averaging over random coins and rectangles
a = Pr[A(X,R)=f(X)]
= Σr Pr[A(X,R)=f(X) ⋀ R=r]
= Σr Σv Pr[A(X,R)=f(X) ⋀ R=r ⋀ Qr(X)=v]
= Σr Σv Pr[A(X,r)= f(X) ⋀ R=r ⋀ Qr(X)=v]
= Σr Σv Pr[Qr(X)=v]∙Pr[R=r|Qr(X)=v]∙Pr[A(X,r)=f(X)|R=r⋀Qr(X)=v]
• For every choice of random coins r the game A(∙,r) is deterministic w/complexity q.
• It divides the set of strings x of length n into 2q rectangles.
• Let Qr(x) denote the rectangle of x.
x1
x2
Qr(x1,x2
)
Averaging over random coins and rectangles
b = Pr[A(X,E(X))=f(X)]
= Σr Pr[A(X, E(X))=f(X) ⋀ E(X)=r]
= Σr Σv Pr[A(X,E(X))=f(X) ⋀ E(X)=r ⋀ Qr(X)=v]
= Σr Σv Pr[A(X,r)= f(X) ⋀ E(X)=r ⋀ Qr(X)=v]
= ΣrΣvPr[Qr(X)=v]∙Pr[E(X)=r|Qr(X)=v] ∙ Pr[A(X,r)=f(X)|E(X)=r⋀Qr(X)=v]
• For every choice of random coins r the game A(∙,r) is deterministic w/complexity q.
• It divides the set of strings x of length n into 2q rectangles.
• Let Qr(x) denote the rectangle of x.
x1
x2
Qr(x1,x2
)
b = Pr[A(X,E(X))=f(X)]
Proof (continued)
ΣrΣvPr[Qr(X)=v]∙Pr[E(X)=r|Qr(X)=v]∙Pr[A(X,r)=f(X)|E(X)=r⋀Qr(X)=v]
ΣrΣvPr[Qr(X)=v]∙Pr[R=r |Qr(X)=v]∙Pr[A(X,r)=f(X)|R=r ⋀Qr(X)=v]
a = Pr[A(X,R)=f(X)]
Problem: It could be that A(•,r) does well on rectangle but poorly on {E(X)=r}
Note: A(•,r) is constant over rectangle.
b = Pr[A(X,E(X))=f(X)]
Proof (continued)a = Pr[A(X,R)=f(X)]
v
v
ΣrΣvPr[Qr(X)=v]∙Pr[R=r |Qr(X)=v]∙Pr[A(X,r)=f(X)|R=r ⋀Qr(X)=v]
ΣrΣvPr[Qr(X)=v]∙Pr[E(X)=r|Qr(X)=v]∙Pr[A(X,r)=f(X)|E(X)=r⋀Qr(X)=v]
2-m
R is uniform and independent of X
≈2-m
E is a 2-source extractor and {Qr(X)=v} is a
rectangle
Qr(x1,x2)=v
x1
x2
{E(X)=r}
Would be fine if f was
also constant
over rectangle
Modifying the argumentWe have that: Pr[A(X,R)=f(x)]¸1-ρ
By Yao’s lemma deterministic game F w/complexity q
Pr[F(X)=f(X)]¸1-ρ
Consider randomized algorithm A’(x,r) which Simulates A(x,r) Simulates F(x)
Let Qr(x) denote the rectangle of A’ and note that:• A(∙,r) is constant on rectangle {Qr(X)=v}.• F(x) is constant on rectangle {Qr(X)=v}.
Problem: It could be that A(•,r) does well on rectangle but poorly on {E(X)=r}
Note: A(•,r) is constant over rectangle.
b = Pr[A(X,E(X))=f(X)]
Proof (continued)a = Pr[A(X,R)=f(X)]
v
v
ΣrΣvPr[Qr(X)=v]∙Pr[R=r |Qr(X)=v]∙Pr[A(X,r)=f(X)|R=r ⋀Qr(X)=v]
ΣrΣvPr[Qr(X)=v]∙Pr[E(X)=r|Qr(X)=v]∙Pr[A(X,r)=f(X)|E(X)=r⋀Qr(X)=v]
2-m
R is uniform and independent of X
≈2-m
E is a 2-source extractor and {Qr(X)=v} is a
rectangle
Qr(x1,x2)=v
x1
x2
{E(X)=r}
Would be fine if f was
also constant
over rectangle
Problem: It could be that A(•,r) does well on rectangle but poorly on {E(X)=r}
Note: A(•,r) is constant over rectangle.
b’ = Pr[A(X,E(X))=F(X)]
Proof (replace f→F)a’ = Pr[A(X,R)=F(X)]
v
v
ΣrΣvPr[Qr(X)=v]∙Pr[R=r |Qr(X)=v]∙Pr[A(X,r)=F(X)|R=r ⋀Qr(X)=v]
ΣrΣvPr[Qr(X)=v]∙Pr[E(X)=r|Qr(X)=v]∙Pr[A(X,r)=F(X)|E(X)=r⋀Qr(X)=v]
2-m
R is uniform and independent of X
≈2-m
E is a 2-source extractor and {Qr(X)=v} is a
rectangle
Qr(x1,x2)=v
x1
x2
{E(X)=r}
We have that F is constant
over rectangle!|a’-a|· ρ
|b’-b|· ρ
Finishing upThm: If A is a randomized communication game with
communication complexity q that tosses m random coins then set B(x)=A(x,E(x)) where E is a 2-source extractor.
B succeeds on most inputs. A (1-ρ)-fraction.
B can be implemented by a deterministic communication game with communication complexity O(m+q).
2-source extractors cannot be computed by communication games.
However, we need extractors for “relatively large rectangles”. Namely 2-source extractors for min-entropy n-(m+q).
Each of the two parties can send the first 3(m+q) bits of his input. The sent strings have entropy rate at least ½.
Run explicit 2-source extractor on substrings.
q.e.d.
???
Generalizing the argument Consider e.g. randomized decision trees A(x,r). Define Qr(x) to be the leaf the decision tree A(∙,r)
reaches when reading x. Simply repeat argument noting that {Qr(X)=v} is a
bit-fixing source. More generally, for any class of randomized
algorithms we can set Qr(x)=A(x,r) Can do the argument if we can explicitly construct
extractors for distributions that are uniform over {Qr(X)=v} = {A(X,r)=v}.
Loosely speaking, need extractors for sources recognizable by functions of the form A(∙,r).
There is a generic way to construct them from a function that cannot be approximated by functions of the form A(∙,r).
Conclusion and open problem
Loosely speaking: Whenever we have a function that is hard on average against a nonuniform version of a computational model we get an explicit version of Yao’s lemma (that is explicit weak derandomization) for the model.
Can handle AC0 using the hardness of parity. Gives a conditional weak derandomization for
general poly-time algorithms. Assumption is incomparable to [NW,GW].
Open problems: Other ways to handle |r| > |x|. Distributions that aren’t uniform.
That’s it…