![Page 1: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/1.jpg)
Pseudorandom Generators from Invariance Principles
1
Raghu MekaUT Austin
![Page 2: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/2.jpg)
What are Invariance Principles?
2
![Page 3: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/3.jpg)
Example 1: Central Limit Theorem
3
Let iid with finite mean and variance.(after appropriate normalization)
Trivia: CLT is how Gaussian density came about ...
![Page 4: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/4.jpg)
Example 2: Mossel, O’Donnell, Oleszkiewicz ‘05
4
![Page 5: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/5.jpg)
Ex 3: Discrete Central Limit Theorem
5
Let independent indicator random variables.(total variance is large)
![Page 6: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/6.jpg)
Hardness of Approximatio
n
Computational Learning
Voting Theory Communication
Complexity
Invariance Principles in CS
Property Testing
Invariance
Principles
![Page 7: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/7.jpg)
This Talk …
7
Applications to construction of pseudorandom generators.
PRGs from invariance principlesIPs give us nice target distributions to aim.Error depends on first few moments –
manage with limited independence + hashing.
![Page 8: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/8.jpg)
Outline of Talk
8
1. PRGs for polynomial threshold functionsM, Zuckerman 10.Featured IP’s: Berry-Esseen theorem, MOO
05.
2. PRGs fooling linear forms in statistical distanceGopalan, M, Reingold, Zuckerman 10. “Discrete central limit theorems”
![Page 9: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/9.jpg)
Polynomial Threshold Functions
9
Applications: Complexity theory, learning theory, voting theory, quantum computing
![Page 10: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/10.jpg)
Halfspaces
10
Applications: Perceptrons, Boosting, Support Vector Machines
![Page 11: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/11.jpg)
11
Good PRGs for PTFs? This Work
First nontrivial answer for degrees > 1.Significant improvements for degree 1.
Generic technique: PRGs from CLTs
Important in Complexity theory.
Algorithmic applications: explicit Johnson-
Lindenstrauss families, derandomizing Goemans-
Williamson.
![Page 12: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/12.jpg)
Fraction of Positive Universe points~ Fraction of Positive PRG points
PRGs for PTFs … Visually
12
Small set preserving fraction of +’ve points for all PTFs
Universe of PointsSmall set of PRG Points
![Page 13: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/13.jpg)
PRGs for PTFs Stretch r bits to n bits and fool
degree d PTFs.
13
![Page 14: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/14.jpg)
Previous Results
14
This work Degree d PTFsThis work Halfspaces
Reference Function Class Seed LengthNo nontrivial PRGs for degree > 1
Nis90, INW94
Halfspaces with poly. weights
DGJSV09 HalfspacesRabani, Shpilka 09
Halfspaces, Hitting sets
KRS 09 Spherical caps, Digons
Our Results
Similar results for spherical caps
![Page 15: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/15.jpg)
Independent Work
15
Diakonikolas, Kane and Nelson 09: -wise independence fools degree 2 PTFs.
Ben-Eliezer, Lovett and Yadin 09: Bounded independence fools a special class of degree d PTFs.
![Page 16: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/16.jpg)
Outline of Constructions
16
1. PRGs for regular PTFsLimited dependence and hashingBerry-Esseen theorem and invariance
principle
2. Reduce arbitrary PTFs to regular PTFsRegularity lemma (Servedio 06, DGJSV 09)
and bounded independence
3. PRGs for logspace machines fool halfspaces
halfspaces.
Essentially a simplification of the hitting set of Rabani and Shpilka.
![Page 17: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/17.jpg)
Regular Halfspaces
17
All variables have low “influence”.
Why regular? By CLT: Nice target distributions:• Enough to find G such that
![Page 18: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/18.jpg)
Berry-Esseen Theorem Quantitative central limit theorem
18
Error depends only on first four moments! Crucial for our analysis.
![Page 19: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/19.jpg)
Toy Example: Majority
19
For simpliciy, let . BET: For
Idea: Error in BET depends only on first four moments. Let’s exploit that!
![Page 20: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/20.jpg)
Fooling Majority
20
Let Partition [n] into t blocks.
Observe: Y’s are independent Sum of fourth moments small
Block 1 Block t
Conditions of BET:
![Page 21: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/21.jpg)
Fooling Majority
21
Y’s are independent Sum of fourth moments small
Conditions of BET:
Y’s independent
First Four Moments
Blocks independentEach block 4-wise
independentProof still works: Randomness used:
![Page 22: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/22.jpg)
Fooling Regular Halfspaces
22
Problem for general regular: weights skewed in a blockExample:
Solution - RS 09: partition into blocks at randomAnalysis reduces to the case of majorities.Enough to use pairwise-independent hash
functions.Some notation:
Hash family 4-wise independent generator
![Page 23: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/23.jpg)
Main Generator Construction
x1
x2
x3 … x
nx5
x4
xk … x
1x3
xk
x5
x4
x2
1 2 t
… xn… x
5x4
x2
2 t
xnxn
23
Randomness:
![Page 24: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/24.jpg)
Analysis for Regular Halfspaces
x1
x3
xk
1
… … x5
x4
x2
2 t
xn
For fixed h, are independent.For random h, sum of fourth moments small.Analysis same as for majorities.24
![Page 25: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/25.jpg)
Summary for Halfspaces
25
1. PRGs for Regular halfspacesLimited independence, hashingBerry-Esseen theorem
2. Reduce arbitrary case to regular caseRegularity lemma, bounded
independence
3. PRGs for ROBPs fool Halfspaces
PRG for Halfspaces
![Page 26: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/26.jpg)
Subsequent Work
26
Reference ResultGopalan et al.[GOWZ10]
PRGs for functions of halfspaces under product distributions
Harsha et al. [HKM10](new IP + generator)
Quasi-polynomial time approx. counting for “regular” integer programs
![Page 27: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/27.jpg)
PRGs for PTFs
27
1. PRGs for regular PTFsLimited independence and hashingInvariance principle of Mossel et al. [MOO05]
2. Reduce arbitrary PTFs to regular PTFsRegularity lemmas of BELY09, DSTW09,
HKM09.
Same generator with stronger .Analysis more complicated:
Cannot use invariance principle as black box
New ‘blockwise’ hybrid argument
![Page 28: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/28.jpg)
Outline of Talk
28
1. PRGs for polynomial threshold functionsM, Zuckerman 10.
2. PRGs fooling linear forms in statistical distanceGopalan, M, Reingold, Zuckerman 10.
2. PRGs fooling linear forms in statistical distance
Uses result for halfspaces.Similar outline: regular/non-regular,
etc. We give something back …
![Page 29: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/29.jpg)
Fooling Linear Forms in Stat. Dist.
29
Fact: For
Question: Can we have this “pseudorandomly”?
Generate ,
![Page 30: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/30.jpg)
Why Fool Linear Forms?
30
Special case: epsilon-bias spaces
Symmetric functions on subsets.Previous best: Nisan, INW.
Been difficult to beat Nisan-INW barrier for natural cases.
Question: Generate ,
![Page 31: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/31.jpg)
PRGs for Statistical Distance
31
Thm: PRG fooling 0-1 linear forms in TV with seed
.
Fits the ‘PRGs from invariance principles’ theme.
Leads to an elementary approach to discrete CLTs.
We do more … “combinatorial shapes”
![Page 32: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/32.jpg)
Discrete Central Limit Theorem
Closeness in statistical distance to binomial distributions
32
Optimal error: .• Barbour-Xia, 98. Proof analytical –
Stein’s method.
![Page 33: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/33.jpg)
Outline of Construction
33
1. Fool 0-1 linear forms in cdf distance.
2. PRG on n/2 vars + PRG fooling in cdf PRG for linear forms for large test sets.
3. Fool 0-1 linear forms for small test sets in TV.
2. Convolution Lemma: close cdfs close in TV.
Analysis of recursionElementary proof of discrete CLT.
![Page 34: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/34.jpg)
Recursion Step for 0-1 Linear Forms
34
For intuition consider
X1 Xn/2+1 Xn… Xn/2 …
PRG -fool in TV PRG -fool in CDF
PRG -fool in TV
True randomness
PRG -fool in TV
![Page 35: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/35.jpg)
Recursion Step: Convolution Lemma
35
Lem:
![Page 36: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/36.jpg)
Convolution Lemma
36
Problem: Y could be even, Z odd.Define Y’:Approach:
Lem:
![Page 37: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/37.jpg)
37
![Page 38: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/38.jpg)
38
Convexity of : Enough to study
![Page 39: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/39.jpg)
Recursion Step
39
For general case similar: Hash …
Recycle randomness across recursions using INW.
![Page 40: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/40.jpg)
Take Home …
40
PRGs from invariance principlesIPs give us nice target distributions to
aim.Error depends on first few moments –
manage with limited independence + hashing.
![Page 41: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/41.jpg)
Open Problems
Optimal non-explicit:Possible approach: recycle randomness as
was done for halfspaces.
41
Better PRGs for PTFs?
![Page 42: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/42.jpg)
Open Problems
42
More applications of ‘PRGs from invariance principles’?
![Page 43: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/43.jpg)
43
Thank You
![Page 44: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/44.jpg)
Combinatorial Shapes
44
Generalize combinatorial rectangles.
What about
Results: Hitting sets – LLSZ 93,PRGs – EGLNV92, Lu02.
Applications: Volume estimation, integration.
![Page 45: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/45.jpg)
Combinatorial Shapes
45
![Page 46: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/46.jpg)
PRGs for Combinatorial Shapes
46
Unifies and generalizesCombinatorial rectangles – symmetric
function h is ANDSmall-bias spaces – m = 2, h is parity0-1 halfspaces – m = 2, h is shifted majority
![Page 47: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/47.jpg)
PRGs for Combinatorial Shapes
47
Thm: PRG for (m,n)-Combinatorial shapes with seed
.
Independent work – Watson 10: Combinatorial Checkerboards.• Symmetric function h is parity.• Seed:
![Page 48: Pseudorandom Generators from Invariance Principles](https://reader036.vdocuments.us/reader036/viewer/2022062310/568163f2550346895dd573c9/html5/thumbnails/48.jpg)
This Talk: Linear Forms in Stat. Dist.
48
Fact: For
Question: Can we have this “pseudorandomly”?
Generate ,