The Goldreich-Levin Theorem:List-decoding the Hadamard code

Amnon Aaronsohn

ECC Course, TAU

2

Outline

Motivation Probability review Theorem and proof

3

Decoding

Fix an (n, k, d) code C, and suppose there is an unknown message sk

We are given a vector yn which is equal to the codeword C(s) with at most m of the places corrupted

Suppose we want to find possible values sk for the original message so that d

H(C(s),y)m

If m<d/2 then there's a unique solution If d/2<m<d there could be multiple solutions

4

Hadamard Codes

[2n, n, 2n-1]2 linear code

The encoding for a message xFn is given by

all 2n scalar products <x,y> for yFn

(Note: all string related math here is mod 2.) Why is the relative distance 1/2? We will see a probabilistic algorithm that

provides list decoding for Hadamard codes when up to 1/2- of the bits are corrupted

5

Basic probability theory review

Random variables (discrete) Expected value ()

E(X) xp(x) Variance (2)

Var(X) = E[(X-E(X))2]

= E[X2]-E[X]2

6

Binary random variables

Pr(X=1)=p, Pr(X=0)=1-p

Often used as indicator variables

E(X)=…

Var(X) = p(1-p) ≤ 1/4

7

Majority votes

Consider a probabilistic algorithm that returns a binary value (0 or 1), with probability > 1/2 of returning the correct result

We can amplify the probability of getting the correct answer by calling the algorithm multiple times and deciding by the majority vote

In order for this to work well there should be some independence between the algorithm’s results in each invocation

8

Independence

Events A1,...,A

n are independent if

Pr[A1,...,A

n] = Pr[A

1]...Pr[A

n]

Likewise, random variables X1,...,X

n are

independent if for each possible assignment x

1,...,x

n:

Pr[X1=x

1,...,X

n=x

n] = Pr[X

1=x1]...Pr[X

n=x

n]

9

Pairwise independence

A set of r.v.'s (or events) is pairwise independent if each pair of the set is independent

Does one type of independence imply the other?

10

Example: xors of random bits

Let X1,…,Xk be independent binary r.v.’s with p=1/2

For each non-empty subset of indexes J define XJ = iJ xi (= iJ xi)

The XJs are

(1) uniformly distributed

(2) not mutually independent

(3) pairwise independent Can be trivially extended to random vectors

11

Chernoff bound

Reminder: we want to improve the accuracy of an algorithm by calling it multiple times and deciding by majority vote

The probability of not getting a simultaneous occurance of the majority of n independent events, each having probability p≥1/2+ has the upper bound

Pr(error) ≤ exp{-2n

12

Chebyshev inequality

For any r.v. X with expected value μ and variance 2:

Pr(|X-|≥) ≤ 2/2

Can be used to get an upper bound for the probability of not getting a majority of n pairwise independent events with p≥1/2+:

Pr(error) ≤ 1/(4n2)

13

Back to the decoding problem

Message space {0,1}n

Think of codewords as binary functions:c=Had(s) x c(x)=<s,x>

Input: function f:{0,1}n{0,1}, representing a codeword with noise

Output: a list L of possible messages s.t. for each sL, f agrees with Had(s) at pfraction of the function inputs:

Prx[f(x)=<s,x>] = p Time complexity in terms of calls to f

14

No error case: p = 1

Unique decoding In this case we can recover the ith bit of the

message by computing f(ei) where ei is the string with 1 at the ith position and 0 everywhere else.

15

Low error case: p = 3/4+

Unique decoding Why not simply use f(ei) as before? Probabilistic algorithm:

Estimate-Had(x):For j = 1…k (k to be fixed)

Choose rj{0,1}n randomly

aj f(rj+x) f(rj)

Return majority(a1,…,ak) Now set the ith bit of the solution to

Estimate-Had(ei)

16

Analysis

Consider this part:Choose rj{0,1}n randomly

aj f(rj+x) f(rj)

If both f(rj+x) andf(rj) are correct then

aj = f(rj+x) f(rj) = <s, rj+x> <s, rj> = <s,x>

Using a union bound we get

Pr[aj <s,x>] ≤ 2(1p) = 1/22

17

Analysis (contd.)

Since we take a majority vote of a1,…,ak we can use the fact that they’re independent to get a Chernoff bound of at most e(k) on the probability of error

The probability of getting some bit wrong is Pr[Estimate-Had(ei) is wrong for some i] ≤ ne(k)

Taking k = O(logn/) gives an O(nlogn/) algorithm with arbitrarily small error

Note that the error probability is doubled, so doesn’t work with p<3/4

18

General case: p = 1/2+

List decoding The Goldreich-Levin theorem gives a probabilistic

algorithm for this problem. Specifically: Input: Function f() as before Output: List L of strings such that each possible

solution s appears with high probability:

Prx[f(x)=<s,x>] ≥ 1/2+Pr[sL] ≥1/2

Run time: Poly(n/)

19

The algorithm (almost)

Suppose that we somehow know the values of Had(s) in m places. Specifically, we are given the strings r1,…,rm and the values b1,…,bm where bj = <s,rj>, for an unknown s

We can then try to compute the value of Had(s) in any x:

Estimate-With-Guess(x , r1,…,rm , b1,…,bm):

For J {1,...,m} (J)

aJ f(x+jJ rj) jJ bj

Return majority of all aJ

Now get the bits of s by calling Estimate-With-Guess with ei

as before

20

Analysis

The idea here is that due to linearity we can get the correct values in more places than we are given

For any J {1,...,m} define rJ=jJ rj.Then <s, rJ>=<s, jJrj>=jJ<s, rj >=jJ bj

If the rjs are uniformly random so are the rJs

The probability of getting aJ wrong is therefore the probability of getting f(x+rJ) wrong, which is bounded by 1/2-

21

But!

The rJs are not independent, so Chernoff bound can’t be used

However, they are pairwise independent so we can use Chebyshev

Pr[EWG(x , r1,…,rm , b1,…,bm) <s,x>] ≤ 1/(2m2) when the ris are independent and chosen uniformly and for each i, bi=<s,ri>

We can recover all bits with an error of at most n/(2m2). Taking 2m = O(n/) gives an O(n2/) algorithm with arbitrarily small error

22

Completing the algorithm

We don’t actually have the correct values for the bis But if m is small we can try all 2m combinations – for

each solution one of them must be correct! The final algorithm:

1. Choose r1,…,rm randomly

2. For each (b1,…,bm){0,1}m:2.1 For i=1,..,n

aiEWG(ei , r1,…,rm , b1,…,bm)

2.2 Output (a1,…,an) Complexity: O(n3/)

23

Finally

Now that we can generate a list where every possible solution appears with probability 1/2, we can re-run the algorithm a constant number of times to get an arbitrary small probability to miss a given solution

24

Summary

We saw a list decoding algorithm for Hadamard code, enumerating with high probability all strings with distance arbitrarily close to 1/2 to a given string

Sample f() at uniformly distributed points so that the adversary won’t be able to affect result

Generate points in a linear subspace spanned by a small number of points, for which we can try all combinations

Results in pairwise independent trials, so we can apply Chebyshev inequality