ec 515 introduction

7/29/2019 EC 515 Introduction

1/37

Introduction to Channel Coding

Introduction to Algebra

EC 515: Coding Theory


2/37

Channel coding theorem

Let a DMS with an alphabetXhave entropy H(X) and

produce symbols once every Ts sec. Let a DMC have

capacity Cand be used once every Tcsec. Then, if

there exists a coding scheme for which the source

output can be transmitted over the channel and bereconstructed with an arbitrarily small prob. of error.

csTC

TXH )(


3/37

Code rate

The parameterC/ Tc is called the critical rate. When

the above expression is satisfied with the equality

sign, the system is said to be signaling at critical rate.

Code rate: a block ofkbits mapped to n bits byadding (n k) redundant bits. Then code rate is given

as r= k/ n

Example: The source generates bits 0 or 1 with

equal probability, one bit in Ts sec. H(X) = 1

The channel is binary symmetric channel with bit-

error probabilityp, channel carrying one bit in Tcsec


4/37

Code rate

Channel capacity

C= 1 +plog2p+ (1 p) log2(1 p) = 1 H(p)

For every 1/Ts bits generated per sec, 1/Tcbits are

transmitted over the channel per sec

Code rate r= Tc / Ts .

Then for reliable transmission we require r C.


5/37

Channel coding basics

While source coding reduces redundancy, channelcoding introduces redundancy.

Idea is to increase Hamming distance betweenevery pair of codes; If min. dist. among all pairs of

codes is dmin and the max. possible bit-error is t, then

Error can be detected if

Error can also be corrected if

One very trivial form of channel coding is repetitivecoding.

1min dt

2

1mindt


6/37

Groups

A set of objects G, along with a binary operation * onthe elements of the set, is called a group if thefollowing properties are satisfied:

Closure: G is closed under *

Associativity

Guaranteed existence of an identity element.

Existence of an inverse element for every

element.

The group is denoted by {G, * }.


7/37

Groups

Theorem: The identity element in a group G is

unique.

Theorem: The inverse of a group element is unique.

Orderof a group = number of elements

Finite group = finite order group

Abelian group = if the operation on the set elements

is commutative.


8/37

Groupsexamples

Set of all integers under real addition (abelian)

Set of all rational numbers excluding 0 under real

multiplication (abelian)

Set of binary numbers 0 and 1 under binary operationXOR (finite, abelian)

Additive group: finite and abelian

(a + b) mod m, where G = {0, 1, 2, ., m1)

Multiplicative group: finite and abelian

(a b) mod p, where G = {1, 2, ., p1), p is prime


9/37

Subgroups and cosets

Subgroup: subset H of G if closed under the groupoperation * of G and satisfies all conditions of agroup.

Left coset: a * H = {a * h}, where a is an element of

G and h is an element of H

Right coset: H * a

For commutative group, left coset = right coset

Theorem: For any element h in H, its inverse hisalso in H.


10/37

Subgroups and cosets

Theorem: No two elements in a coset of H are

identical.

Theorem: No two elements in two different (distinct)

cosets of H are identical. Every element in G appears in one and only one

coset of H.

All the distinct cosets of H are disjoint.

The union of all the distinct cosets of H forms the

group G.


11/37

Fields

A set F together with two binary operations addition(denoted by +) and multiplication (denoted by ) isa field if

The set F is a commutative group under addition.

Identity element w.r.t. addition is called additiveidentity or zero element (denoted by 0)

The set F minus the zero element is acommutative group under multiplication.

Identity element w.r.t. multiplication is calledmultiplicative identity or unit element (denoted by1)

Multiplication is distributive over addition.


12/37

Fields

Subtracting one field element b from another field

element a = adding additive inverse of b to a

Similarly, dividing a by b = multiplying a with

multiplicative inverse of b Note, addition, multiplication, subtraction and division

here are field operations and need not be same as

ordinary addition, multiplication, subtraction and

division. Order of field = number of elements in F.

Finite order field is known as Galois Field.


13/37

Field properties

a 0 = 0 a = 0

a b 0, for any two non-zero elements a, b

a b = 0 and a 0 implies b = 0

(a b) = (a) b = a (b)

a b = a c implies b = c, if a 0


14/37

Examples of Galois field

Set {0, 1} under mod-2 addition (XOR) and

multiplication (AND)

Called binary field denoted by GF(2)

{0, 1, .., p1} under mod-p addition and mod-pmultiplication, where p is prime.

Called prime field denoted by GF(p)

Extension field: prime field GF(p) extended to a fieldof pm elements, denoted as GF(pm), m is pos. integer

The order of any finite field is a power of a prime.


15/37

Binary field arithmetic

We are concerned with binary field GF(2) or its

extension in digital system using binary coding.

Binary mod-2 addition mod-2 subtraction.

Solving equations in binary arithmetic follows thesame rule as in ordinary arithmetic; Cramers rule is

applicable.

Polynomialover GF(2) with one variable and

coefficients 0 or 1:

f(X) = a0 + a1X + ..+ anXn


16/37


Degree of polynomial = largest power of X with non-

zero coefficeint, i.e. degree of f(X) is n if an 0.

We can have 2n polynomials with degree n.

Polynomials over GF(2) can be added, subtracted,multiplied and divided in the usual way where

multiplication and addition of coefficients are mod-2.

Addition and multiplication of polynomials are

commutative and associative.

Multiplication is distributive over addition.

Euclids division algorithm: f(X) = q(X).g(X) + r(X)


17/37


f(X) divisible by (Xa) (X+a) if a is a root of f(X), i.e.f(a) = 0.

It follows, f(X) divisible by (X+1) if it has an evennumber of terms.

Irreducible polynomial = polynomial p(X) of degreem not divisible by any polynomial of degree less thanm but greater than 0.

For any m 1, there exists at least one irreduciblepolynomial of degree m.

Theorem: Any irreducible polynomial p(X) ofdegree m always divides Xn + 1, where n = 2m 1.


18/37


Primitive polynomial: when p(X) does not divide

any other polynomial of the form XK + 1, where K < n

= 2m 1.

All irreducible polynomials are not primitivepolynomials.

Example: X4 + X + 1 and X4 + X3 + X2 + X + 1 are

both irreducible polynomial of degree 4, but only

the first polynomial is also primitive polynomial(the second polynomial can divide X5 + 1)

f2(X) = f(X2) [f(X)]2k = f(X2k), k 0.


19/37

Construction of GF(2m)

Choose a primitive polynomial p(X) over GF(2) oforder m > 1. (Why only a primitive polynomial?)

Determine the root of p(X).

Let, be the root of p(X) p() = 0 and 0,1since p(X) not divisible by X or (X+1).

Then, F = {0, 1, , 2, , n1} forms a Galois fieldof order 2m= n + 1 under binary field addition + andmultiplication

Multiplication of two non-zero elements of F givesi j = i+j (for i, j 0, where 0 = 1), which iscontained in F (in case i + j n, check that n = 1)


20/37


Divide every element of F by p() and obtain theremainder.

Dividing 0 gives remainder 0.

Dividing i

(0 i < m) gives non-zero remainderri() =

i since 0.

Dividing i (m i < n) gives remainder ri() whichis a non-zero polynomial of degree less than m.

Check that

All ri(), for 0 i < n, are distinct.

i = ri() for 0 i < n


21/37


Thus, F = {0, 1, , 2, , m1, rm(),, rn1()} is

the Galois field of order 2m.

Check that

Elements of F are distinct polynomials (including0) of degree less than m.

Number of non-zero polynomials of degree less

than m is n = 2m 1.

Therefore, adding any two elements of F gives a

polynomial (including 0) of degree less than m

which is also contained in F.


22/37


Thus, GF(2m) may be represented in following ways:

Power representation

F = {0, 1, , 2, , n1}

Polynomial representation

F = {0, 1, , 2, , m1, rm(),, rn1()}

m-tuple representation: (ai,0 ai,1.. ai,m1) where

ai,k are the coefficients of the polynomial ri()

Example: Construct GF(24) from the generator

polynomial p(X) = 1 + X + X4


23/37

Basic properties of GF(2m)

If an element in GF(2m) is root of f(X), then P, P =2l, l 0 is also root of f(X).

All n = 2m 1 non-zero elements of GF(2m) form allthe roots of Xn + 1.

All elements of GF(2m), including 0, form all theroots of Xn+1 + X.

Minimal polynomial(X) of an element in GF(2m)is the polynomial of smallest degree whose root is ,i.e. () = 0.

Min. polynomial for 0 and 1 are X and X + 1,respectively.


24/37


Minimal polynomial (X) of an element in GF(2m) isirreducible.

If element in GF(2m) is root of f(X), then (X)divides f(X).

It follows, (X) divides Xn+1 + X, n = 2m 1.

If element in GF(2m) is root of an irreduciblepolynomial f(X), then the minimal polynomial of is(X) = f(X).

Let, e be the smallest non-negative integer for anelement in GF(2m), so that P = , P = 2e then thepolynomial f(X) given below is irreducible.


25/37


The min. polynomial (X) for the element is givenby the polynomial f(X) above.

If the degree of the minimal polynomial (X) for anelement in GF(2m) is e then e m, and P = ,where P = 2e

The element P, P = 2l is called a conjugate of.

2, , P, P = 2e1 form distinct conjugates of.

All conjugates of are all the roots of(X).

Check that n = 1n+1 = e m

1

0

2)(

e

i

i

XXf


26/37


Cyclic group: If there exists an element whose

powers constitute the whole group.

Order of a field element a is the smallest positive

integer N such that aN

= 1. Primitive element: If the order of an element in

GF(q) is q 1.

It may be proved that aq1 = 1 for any element a in

GF(q).

Powers of a primitive element gives all the non-

zero elements of GF(q).


27/37


Example: 3 is a primitive element in GF(7), while

2 is not.

If is a primitive element in GF(2m) then its

conjugates are also primitives. Example: For GF(24) = {0, 1, , 2, , 14},

check that = 7 is a primitive. Accordingly, we

may check that its conjugates 14, 13, 11 are also

primitives. If an element in GF(2m) has order N, then all its

conjugates also have order N.


28/37

Vector spaces

F is a field whose elements are called scalars.

V is a set of elements on which a binary operation +is defined.

A multiplication operation between elements of Fand V is defined.

V is called a vector space over the field F if

V is a commutative group under +

a v is in V where a is in F and v in V

Distributive laws for any two elements a, b in Fand any two elements u, v in V.


29/37

Vector spaces

Associative laws for any two elements a, b in F

and any element v in V.

1 v = v, where 1 is the unit element in F.

Elements of V are called vectors.

Addition + on V is called vector addition.

Multiplication combining a scalar and a vector is

called scalar multiplication or scalar product. Additive identity in V is denoted by 0.


30/37

Properties of vector spaces

0 v = 0, where 0 is the zero element in F.

a 0 = 0

(a) v= a (v) = (a v)

Therefore, (a) vor a (v) is the additive inverseof a v

Consider vector space Vn over GF(2) that haselements given by n-tuple over GF(2) total 2ndistinct n-tuples.

Vector0 is all 0 n-tuple (additive identity)

Vector addition is mod-2 addition.

Additive inverse of an n-tuple is itself.


31/37


Scalar multiplication is mod-2 multiplication.

Subspace: Subset S of V that is also a vector space

over F is called subspace of V.

Let S be a nonempty subset of a vector space V overa field F. Then, S is a subspace of V if

For any two vectors u, v in S, u + v is also a

vector in S.

For any element a in F and any vectorvin S, a v

is also a vector in S.


32/37


Linear combination of vectors v1, v2, .., vN in V:

a1v1 + a2v2+ + aNvN

Sum of two linear combinations ofv1, v2, .., vN is

also a linear combination ofv1, v2, .., vN Product of a scalar in F and a linear combination

ofv1, v2, .., vN is also a linear combination ofv1,

v2, .., vN

Linearly dependent vectors: a1v1+ + aNvN = 0

Linearly independent vectors: a1v1+ + aNvN0

except when a1 = a2= = aN = 0


33/37

Basis of vector space

B = A set of linearly independent vectors so that anyvector in V can be expressed as a linear combinationof the vectors in B:

B spans the vector space V and is called the

basis or base of the vector space V. At least one basis can be found.

Number of basis vectors is the dimension of thevector space.

Dimension of Vn = n. A set of k (where k < n) linearly independent

vectors from Vn form a k-dimensional subspace ofVn consisting of 2

k vectors.


34/37

Dot product

Inner product ordot product: gives scalar result, u

v = u0 v0 + u1 v1+ .. + un1 vn 1 (mod-2

addition)

Orthogonal vectors: u v = 0 Properties of dot product:

u v = v u

u (v + w)= u v + u w (au) v =a(u v)


35/37

Null space

S is a subspace of Vn.

Define Sd as another subspace of Vn so that u v = 0,

where u and v are any vectors from S and Sd,

respectively.

Check that Sd must contain vector0, and so is non-

empty subset of Vn.

For any element a in GF(2), a v = 0 orv which is

also contained in Sd.

Also, u (v + w) = 0 where v and w are any two

vectors from Sd


36/37

Null space

This means, v and w, and also v + w are

orthogonal to u

Hence, v + w is also in Sd

Sd is the null or dual space of S

Sd is the subset of all vectors that are orthogonal to

every vector in S.

Check that, vice-versa is also true.

Hence, conversely, S is the dual space of Sd

Dimension of Sd is n k.


37/37

Example

Let n = 3, k = 2; Vn is the vector space consisting of

all the 8 number of 3-tuples: 000, 001, 010, 011, 100,

101, 110, 111.

S = {000, 011, 101, 110} Check that S forms a subspace.

We can find a set of 2 linearly independent vectors

that spans S: {011, 101} or {101, 110} or {011, 110}

2-diemnsional vector space.

Null-space for S is given by Sd = {000, 111}; vector

111 only spans the vector space Sd 1-dimensional

vector space

ec 515 introduction

Documents