expander graphs, randomness extractors and list-decodable codes

Expander Graphs,Randomness Extractors

and List-Decodable Codes

Salil VadhanHarvard University

Joint work with Venkat Guruswami (UW) & Chris Umans (Caltech)

[GW94,WZ95,TUZ01,RVW00,CRVW02]

Connections in Pseudorandomness

RandomnessExtractors

Expander Graphs

List-DecodableError-Correcting

Codes

PseudorandomGenerators

Samplers

[Tre99,RRV99,ISW99,SU01,U02]

[Tre99,TZ01,TZS01,SU01]

[CW89,Z96]

This Work

[PV05,GR06]

This Work

Outline

• Expander Construction

• Application to Extractors

• Connections

• Conclusions

(Bipartite) Expander Graphs

Goals:• Minimize D• Maximize A• Minimize M

|(S)| A¢|S|D

NM

S, |S| K

Nonconstructive:• D = O(log(N/M)/)

• A = (1-)¢D

• M = (KD/

“(K,A) expander”

O(1) if M=N

log N)

if M·pN

=

Applications of Expanders• Fault-tolerant networks (e.g., [Pin73,Chu78,GG81])• Sorting in parallel [AKS83]• Complexity theory [Val77,Urq87]• Derandomization [AKS87,INW94,Rei05,…]• Randomness extractors [CW89,GW94,TUZ01,RVW00]• Ramsey theory [Alo86]• Error-correcting codes [Gal63,Tan81,SS94,Spi95,LMSS01]

• Distributed routing in networks [PU89,ALM96,BFU99].

• Data structures [BMRS00].

• Distributed storage schemes [UW87].

• Hard tautologies in proof complexity [BW99,ABRW00,AR01].• Other areas of Math [KR83,Lub94,Gro00,LP01]

Need explicit constructions (deterministic, time poly(log N)).

Advantage of Expansion (1-)¢D

• At least (1-2) D |S| elements of (S) areunique neighbors: touch exactly one edge from S

|(S)| (1-) D |S|D

NM

S, |S| K

x

• Fault tolerance: Even if an adversary removes most (say ¾) edges from each vertex, lossless expansion maintained (with =4)

Application to Data Structures [BMRS00]

• Goal: store small S½[N] s.t. can test membership by (probabilistically) reading 1 bit.

• Expansion (1-)¢D ) 9 0,1 assignment to [M] s.t. for every x2[N], a 1-O() fraction of neighbors have correct answer!

D

NM

S, |S| K |(S)| (1-)¢ D¢|S|

/2

000110110

Application to Data Structures [BMRS00]

Size: M=O(K¢ log N) with optimal expander• (K¢log N) necessary to represent set.• Perfect hashing: same size, but read O(log N)-bit word

D

NM

S, |S| K /2

000110110

Explicit Constructions

Nonconstructive O(log(N/M)) (1-)¢D O(KD

Ramanujan graphs[…LPS86,M88]

O(1) ¼ D/2[Kah94]

N

Zig-zag CRVW02] O(1) (1-)¢D N

Ta-Shma, Umans, Zuckerman [TUZ01]

polylog(N)quasipoly(log N)

(1-)¢D(1-)¢D

quasipoly(KD) poly(KD)

Our Result polylog(N) (1-)¢D poly(KD)

degree D expansion A |right-side| M

arbitrary constant. quasipoly(t)=exp(polylog t)

Our Result

Thm: For every N, K, >0, 9 explicit (K,A) expander with• degree D = poly(log N, 1/)

• expansion A = (1-)¢D

• #right vertices M = D2¢ K1.01.

|(S)| A¢|S|D

NM

S, |S| K

Our Construction

Left vertices = Fqn = polys of degree · n-1 over Fq

Degree = q

Right vertices = Fqm+1

(f,y) = y’th neighbor of f = (y, f(y), (fh mod E)(y), (fh2 mod E)(y), …, (fhm-1 mod E)(y))

where E(Y) = irreducible poly of degree n h = a parameter

Thm: This is a (K,A) expander with K=hm, A = q-hnm.

Setting Parameters

(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))

f = poly of degree · n-1, E = irreducible of degree n

N = Fqn , D = q, M = Fq

m+1


Set h = poly(nm/)q = h1.01Then:

• D = q = poly(log N, 1/) • A = q-hnm ¸ (1-)¢ D

• M = qm+1 = q¢ (h1.01)m = D¢ K1.01

Rel’n to Parvaresh-Vardy Codes [PV05]




• (f,y) = (y, y’th symbol of PV encoding f)

• Proof of expansion inspired by list-decoding algorithm for PV codes.

List-Decoding View of Expanders

• For Tµ [M], define LIST(T) = {x2 [N] : (x)µT}

• Lemma: G is a (=K,A) expander iff for all Tµ [M] of size AK-1, we have |LIST(T)| · K-1

|(S)| A¢ KD

N

S, |S|=K

M

“(=K,A) expander”

Comparing List-Decoding Views

: [N] £ [D] ! [D] £ [M]

T µ [D] £ [M]

Object Interpretation x2 LIST(T) iff Decoding Problem

expanders x,y) = y’th nbr of x

8 y (x,y)2 T |T| < AK ) |LIST(T)| < K

list-decodable codes

x,y) = (y,ECC(x)y)

Pry [(x,y) 2 T]

¸ 1/M + T = {(y,ry)}) |LIST(T)| < K

Proof of Expansion



Thm: For A=q-nmh and any K· hm, we have

Tµ Fqm+1 of size AK-1) |LIST(T)|· K-1

Proof Outline (following [S97,GS99,PV05]):• Find a low-degree poly Q vanishing on T.• Show that every f 2LIST(T) is a “root” of a related

polynomial Q’.

• Show that deg(Q’) · K-1

=

Proof of Expansion: Step 1



Thm: For A=q-nmh, K= hm, |T|·AK-1) |LIST(T)|· K-1.

Step 1: Find a low-degree poly Q vanishing on T.

• Take Q(Y,Z1,…,Zm) to be of degree · A-1 in Y,degree · h-1 in each Zi.

• # coefficients = A K > |T| = # constraints ) nonzero solution

• WLOG E(Y) doesn’t divide Q(Y,Z1,…,Zm).





Step 1: 9 Q vanishing on T, deg · A-1 in Y, h-1 in Zi, E-Q

Step 2: Every f 2LIST(T) is a “root” of a related Q’.

f(Y) 2LIST(T)

) 8 y2 Fq Q(y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y)) = 0

) Q(Y, f(Y), (fh mod E)(Y), …, (fhm-1 mod E)(Y)) 0

) Q(Y, f(Y), f(Y)h, …, f(Y)hm-1) 0 (mod E(Y))

) Q’(f) = 0 in Fq[Y]/E(Y), where

Q’(Z) = Q(Y,Z,Zh,…,Zhm-1) mod E(Y)






Step 2: 8 f2LIST(T) Q’(f) = 0 where Q’(Z) = Q(Y,Z,Zh,…,Zhm-1) mod E(Y)

Step 3: Show that deg(Q’) · K-1

• Q’(Z) nonzero because Q(Y,Z1,….,Zm) not divisible by E(Y) & is of deg · h-1 in Zi

• deg(Q’(Z)) · h-1+(h-1)¢ h++(h-1)¢ hm-1 = hm-1 = K-1

Proof of Expansion: Wrap-Up





Step 2: 8 f2LIST(T) Q’(f) = 0 where Q’(Z) = Q(Y,Z,Zh,…,Zhm-1) mod E(Y)

Step 3: Show that deg(Q’) · K-1

Proof of Thm:

|LIST(T)| · deg(Q’) · K-1. ¥

Our Result



• #right vertices M = D2¢ K1.01.

|(S)| A¢|S|D

NM

S, |S| K

Outline

Expander Construction

• Application to Extractors

• Connections

• Conclusions

Extractors: Original Motivation[SV84,Vaz85,VV85,CG85,Vaz87,CW89,Zuc90,Zuc91]

• Randomization is pervasive in CS– Algorithm design, cryptography, distributed computing, …

• Typically assume perfect random source.– Unbiased, independent random bits– Unrealistic?

• Can we use a “weak” random source?– Source of biased & correlated bits.– More realistic model of physical sources.

• (Randomness) Extractors: convert a weak random source into an almost-perfect random source.

Applications of Extractors

• Derandomization of (poly-time/log-space) algorithms [Sip88,NZ93,INW94, GZ97,RR99, MV99,STV99,GW02]

• Distributed & Network Algorithms[WZ95,Zuc97,RZ98,Ind02].

• Hardness of Approximation [Zuc93,Uma99,MU01,Zuc06]

• Data Structures [Ta02]

• Cryptography [BBR85,HILL89,CDHKS00,Lu02,DRS04,NV04]

• Metric Embeddings [Ind06]

• Def: A (k,)-extractor is Ext : {0,1}n £{0,1}d ! {0,1}m s.t.

8 k-source X, Ext(X,Ud) is -close to Um.

8x Pr[X=x] · 2-k

Extractors [NZ93]

d random bits

“seed”

• Optimal (nonconstructive): d = log(n-k)+2log(1/)+O(1)m = k+d-2log(1/)-O(1)

EXT

k-source of length n

m almost-uniform bits

in variationdistance

Our Result

d random bits

“seed”

EXT


m almost-uniform bits

Thm: For every n, k, >0, 9 explicit (k,) extractor with seed length d=O(log(n/)) and output length m=.99k.

• Previously achieved by [LRVW03]

– Only worked for ¸ 1/no(1)

– Complicated recursive construction

Approach: Condensers [RR99,RSW00]

d random bits

“seed”

CON


¼ k’-source of length m

Def: A k!k’ condenser is Con : {0,1}n £{0,1}d ! {0,1}m s.t.

8 k-source X, Con(X,Ud) -close to some k’-source.

• Can extract from output: easier if k’/m > k/n.

• Called lossless if k’=k+d.

2k

Lossless Condensers Expanders

Lemma [TUZ01]: Con : {0,1}n £{0,1}d ! {0,1}m is a k!k+d condenser iff it defines a (2k,(1-)¢2d) expander.

Proof ((): • Suffices to condense sources uniform on 2k strings.• Expansion ) can make 1-1 by moving fraction of edges

{0,1}n

{0,1}m

2d

¸ (1-) 2d¢ 2k

n-bit k-source

¼ m-bit (k+d)-source

d-bit seed CON

x

Con(x,y)

y

Our Condenser



• #right vertices M = D2¢ K1.01. (f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))

Thm: For every n, k, >0, 9 explicit k!k+d condenser w/

• seed length d = O(log n+log(1/))

• output length m=2d+1.01¢k

• Con(f,y) = (y, f(y), (fh mod E)(y), …, (fhm-1 mod E)(y))

Our Extractor

Condense: 9 explicit k!k+d condenser w/


• output length m ¼ 1.01k


Then Extract: apply extractor for min-entropy rate .99:

• Constant – Ext(x,y) = y’th vertex on expander walk specified by x.

– Extraction follows from Chernoff bound for expander walks [G98], via equivalence of extractors and samplers [Z96].

Our Extractor

Condense: 9 explicit k!k+d condenser w/




Then Extract: apply extractor for min-entropy rate .99:

• Arbitrary – Zuckerman’s extractor for constant min-entropy rate [Z96].

Variations on the Condenser

Thm: 9 explicit k!k+d condenser w/




Variations (lose constant fraction of min-entropy):

• “Repeated roots” [GS99] in analysis

– seed length d = log n+log(1/)+O(1)

– output length m = O(k ¢ log(n/))

Variations on the Condenser

Thm: 9 explicit k!k+d condenser w/




Variations (lose constant fraction of min-entropy):

• E(Y) = Yq-1 - , for primitive root [GR06]

) (fhi mod E)(y) = f (i y)

) univariate analogue of Shaltiel-Umans extractor [SU01].

Outline


Application to Extractors

• Connections

• Conclusions

Comparing List-Decoding Views

: {0,1}n £{0,1}d ! {0,1}d £ {0,1}m

T µ {0,1}d£ {0,1}m N=2n,D=2d,…

Object Interpretation x2 LIST(T) iff Decoding Problem

expanders x,y) = y’th nbr of x

8 y (x,y)2 T |T| < AK ) |LIST(T)| < K

list-decodable codes

x,y) = (y,ECC(x)y)

Pry [(x,y) 2 T]

¸ 1/2m + T = {(y,ry)}) |LIST(T)| < K

extractors Pry [(x,y) 2 T]

¸ |T|/2m+d + 8 T |LIST(T)| < K

lossy condensers

Pry [(x,y) 2 T]

¸ |T|/2m+d + |T|· K’-1 ) |LIST(T)| · K

Outline


Application to Extractors

Connections

• Conclusions

Conclusions

• List-decoding view ) best known constructions of– Highly unbalanced expanders– Lossless condensers– Randomness extractors

• Push it further?– Nonbipartite expanders– Direct construction of extractor– Extractors optimal up to additive constants– Better list-decodable codes

expander graphs, randomness extractors and list-decodable codes

Documents