04 hamming codes and performance

7/30/2019 04 Hamming Codes and Performance

1/16

EE4-07 Introduction to Coding Theory

Dr. Wei DaiImperial College London (IC)

Autumn 2011

Dr. Wei Dai (Imperial College) Introduction to Coding Theory Autumn 2011 1
http://find/


2/16

Section 4 Hamming Codes and Performance

The Hamming code: definition and properties

The dual of the Hamming codeSeveral bounds in coding theory

http://find/


3/16

Binary Hamming Codes

The H [7, 4, 3] Binary Hamming code is a linear code with

H= 0 0 0 1 1 1 10 1 1 0 0 1 1

1 0 1 0 1 0 1

1 2 3 4 5 6 7

(contains all the nonzero columns of length 3)

Let r 2. The parity-check matrix of the binary Hamming codeH [2r 1, 2r 1 r, 3] consists of all the nonzero vectors (columns) oflength r. (Also denoted by Hr.)

d = 3 because every 2 columns are linearly independent andthere are 3 columns that are linearly dependent.

Correct exactly 1 error.

http://find/


4/16

Decoding a Binary Hamming Code

Syndrome decoding:

Arrange columns of H in the alphabetically increasing order1 Compute the syndrome s = yHT.

2 The syndrome s gives the location of error.

3 Flip the bit located at s. x1 = y1, ,xs = ys + 1, ,xn = yn.

Easy to decode.

http://find/


5/16

Coding Bounds

Given n and d, what is the maximum r or k or M?

Given n and k, what is the maximum d?

Sphere packing (Hamming) boundSphere covering (Gilbert-Varshamov) bound

Improved Gilbert-Varshamov bound for linear codes

Plotkin bound

Singleton bound

http://find/http://goback/


6/16

Hamming Bound and Perfect Codes

Spheres in Fnq : B (x, t) = y Fnq : d (x,y) t

.

Volume: Vol (B (x, t)) =t

i=0ni

(q 1)i

.Remark: Volume does not depend on the center. Denote it by V (t)

Hamming bound (sphere-packing bound): For a code of length n anddistance d, the number of codewords satisfies

M qn/V

d12

= qn/

d12 i=0

ni

(q 1)i

.

Proof: Let t =d12

. Let C = {c1, , cM}. Clearly, B (ci, t)s are

disjoint, 1 i M. Hence, M V (B (e)) qn.

Perfect codes: the codes that attain the Hamming bound.

Hamming code is a perfect code.

Perfect codes are rare (Hamming codes & Golay codes).

http://find/http://goback/


7/16

Gilbert-Varshamov Bound

Gilbert-Varshamov (sphere covering) bound: For given code length nand distance d, there exists a code with number of codewords

qn/V (d 1) M.Proof: Its proved by construction. Let M0 = q

n/V (d 1) > 1.Take an arbitrary c1 F

nq . Since F

nq B (c1, d 1) = , take arbitrary

c2 Fn

q B (c1, d 1). Clearly, d (c1, c2) d.Inductively, suppose that we have obtained codewords c1, , cM01in this way.

Since VolM01

i=1 B (ci, d 1)

(M0 1) V (d 1) < qn,

Fn

q

M01i=1

B (ci, d 1) = . Take arbitrary

cM0 Fnq

M01i=1 B (ci, d 1) = . Clearly, d (cM0 , ci) d as

cM0 / B (ci, d 1) for all 1 i M0 1.

http://find/


8/16

Illustration for Sphere Packing and Covering

Sphere Packing Sphere Covering

http://find/


9/16

Improved Gilbert-Varshamov Bound for Linear Codes

Improved G-V bound for linear codes: For given n,d,k such that

d2i=0

n1i

(q 1)i

< qnk

, there exists an [n, k] linear code over Fqwith minimum distance at least d.Proof: We shall show that an (n k) n matrix H such that everyd 1 columns of H are linearly independent. We construct H asfollows.

Let h1be any nonzero vector in Fnkq . Let h2 be any vector not inspan (c1). Let h3 be any vector not in span ([c1, c2]). .Inductively, suppose that we have constructed H of size

(n k) (n 1) such that every d 1 columns of H are linearlyindependent. Note that the number of vectors in the linear span of

d 2 or fewer columns of H is given by

d2i=0

n1i

(q 1)i. Sinced2

i=0

n1i

(q 1)i < qnk, h Fnkq such that every d 1 columns

of the resulting H= [H,h] are linearly independent.

http://find/


10/16

Plotkin Bound

Plotkin bound: For a code of length n. Suppose that rn < d where

r = 1 q1. Then the number of codewords M

ddrn

.

(M

2d2dn

when q = 2.)

Proof: 1) Let T =

cC

cC

d (c, c). Since d d (c, c) for c = c, wehave M(M 1) d T.

2) Let A be the M n matrix whose rows are the codewords. Let ni,adenote the number of entries in the ith column of A that are equal to a.Then

aF ni,a = M for all i. We have T =

ni=1 (

c

c d (ci, c

i)) =

ni=1

aF ni,a (M ni,a) = M

2n

ni=1

aF n

2i,a.

3) By Cauchy-Schwarz inequality,

aF ni,a2

q

aF n2i,a. Hence,

T M2n n

i=1 q1

aF ni,a2

= rM2n. Therefore,M2 (d rn) M d.

http://find/


11/16

Cauchy-Schwarz Inequality and Proof

Cauchy-Schwarz Inequality: For real numbers x1, , xn andy1, , yn,

(

mi=1 xiyi)

2 (

mi=1 xi)

2 (

mi=1 yi)

2.

Proof: It hold if either x or y is 0.

Assume that both x and y are nonzero. Define z = rx + y.Then 0 z22 = r

2 x22 + 2r x,y + y22 for all r R.

This implies b2 4ac = 4 x,y2 4 x22 y22 0, i.e.,

x,y2 x22 y22.

http://find/


12/16

Dual of Binary Hamming Codes

(Hr) = Sr

2r 1, r, (n + 1) /2 = 2r1

is called simplex code.

A very low rate code with very large distance.

Proof of d (Sr) = 2r1: by induction.

Simplex codes attain Plotkin bound: M = 2r; 2d2dn =2r

2r(2r1) = 2r.

http://goforward/http://find/http://goback/


13/16

Singleton Bound and MDS

Theorem (Singleton Bound):

This distance of any code C Fnq with |C| = M satisfies

M qnd+1.In particular, if the code is linear and M = qk, then

d n k + 1.

Proof: For a code with length n and distance d, take arbitrary ci = cj .Let ci,1:nd+1 be the first n d + 1 entries of ci and ci,nd+2:n be thelast d 1 entries of ci. Thend dH (ci, cj) = dH (ci,1:nd+1, cj,1:nd+1) + dH (ci,nd+2:n, cj,nd+2:n) .But dH (ci,nd+2:n, cj,nd+2:n) d 1. Hence,

dH (ci,1:nd+1, cj,1:nd+1) 1, i.e., the first n d + 1 coordinates of allcodewords are distinct. Hence, M qnd+1.

Codes meet singleton bound are maximum distance separable (MDS).

http://find/


14/16

Properties of MDS codes

Properties of MDS codes:Let C[n,k,d] be a linear code over Fq. Let H

and G be the corresponding parity-check and generator matrices.Then the following statements are equivalent

1 C is an MDS code.

2 Every set of n k columns of H is linear independent.

3

Every set of k of G is linear independent.4 C is an MDS code.

Proof: It is clear that 12 and 34 as G is the parity-check matrix ofC. We prove that 14. Once we proved that, it is clear that 41.

To show C is MDS, it suffices to show that the minimum distance d isk + 1.

http://find/


15/16

Proof of Properties of MDS codes

Suppose that d k. Then a nonzero codeword c C with at most

k nonzero entries and at least n k zeros. Since permuting thecoordinates reserves the codeword weights (i.e., the distance), w.l.o.g.,

assume that the last n k coordinates of c are 0.

Write H= [A, H] where A F(nk)kq and H F

(nk)(nk)q . From

assumption 1 (and hence 2), the columns of H are linear

independent. Hence, H is invertible. The only way to obtain 0 in all

the last n k coordinates of c = sH is s = 0. It contradicts with theassumption c = 0. Hence, d k + 1. Together with the Singleton

bound, it follows that d

= k + 1. Hence, 14.

Hamming codes are not MDS in general.


S
http://find/


16/16

Summary

Examples of linear codes

Hamming codes Simplex codes

Coding Bounds

Sphere packing and covering bounds Plotkin bound Singleton bound and MDS

http://goforward/http://find/http://goback/

04 hamming codes and performance

Documents