channel capacity. techniques to reduce errors in digital communication systems automatic repeat...

Channel Capacity

Information Measure

Information in message j

bitsP

Ij

j

1log2

jP Probability of transmitting the jth message Average information:

m

j

m

j

jjj PPIPH

1 2

1log

)/(/ sbitsTHR Source Rate (T is the time required to send a message)

Channel Capacity

NSBC 1log

2

If CR Shannon demonstrated that for signal plus white noise, it is possible to build a system with 0

eP If R < C

Coding, Channel Coding

Convolutional Codes

Techniques to reduce errors in digital communication systems

Techniques to reduce errors in digital communication systems

Automatic repeat request (ARC)Forward error correction (FEC) Channel coding as

opposed to source codingARC is typical of computer systems.FEC is typical of large transmission delay systems.

We will concentrate in FEC.

Codes Block Codes are maps of k input binary symbols into n

binary symbols n>k (memoryless) nkR / typically from ¼ to 7/8 with k>3

Convolutional codes are employed in memory systems where the current k binary symbols are combined with information from previous symbols to produce vkn binary symbols. nkR / Typically from ¼ to 7/8 1<k<8 and 2<v<60.

R is the code rate different from bit rate or information rate

Block Codes

Definitions Hamming weight number of binary 1 bits in a word Hamming distance “d” number of positions in which two words differs.

If 1 tsd s errors can be detected and t errors corrected. If s= t; d>2t+1

rkppppiiii ........

321321 typical, but not unique, transmission format

Hamming codes correct 1 error, s=t=1 d>3 mkn mm 12,12, . Posibble values (7,4), (15,11), (31,26)

There are many other possibilities for block codes like cyclic codes

Convolutional Codes

xhy i

jj

i

0

http://en.wikipedia.org/wiki/Convolutional_code

Rate 1/3 non-recursive, non-systematic convolutional encoder with constraint length 3

http://en.wikipedia.org/wiki/Convolutional_code

Sixteen-state recursive systematic convolutional (RSC) code.

http://en.wikipedia.org/wiki/Convolutional_code

Rate 1/2 8-state recursive systematic convolutional encoder.

Convolutional Codes

Convolutional Codes

Encoded bits in reverse order

http://home.netcom.com/~chip.f/viterbi/algrthms2.html

t = 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17State 002 0 0 2 3 3 3 3 4 1 3 4 3 3 2 2 4 5 2State 012 3 1 2 2 3 1 4 4 1 4 2 3 4 4 2State 102 2 0 2 1 3 3 4 3 1 4 1 4 3 3 2State 112 3 4 2 1 1 3 4 4 3 4 2 3 4 4

00 11 11 00 01 10 01 11 11 10 00 00 11 00 11 10 11

t = 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17State 002 0State 012

State 102

State 112

00 11 11 00 01 10 01 11 11 10 00 00 11 00 11 10 11

t = 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17State 002 0 2 3 3 3 3 4 1 3 4 3 3 2 2 4 5 2State 012 3 1 2 2 3 1 4 4 1 4 2 3 4 4 2State 102 2 0 2 1 3 3 4 3 1 4 1 4 3 3 2State 112 3 1 2 1 1 3 4 4 3 4 2 3 4 4

00 11 11 00 01 10 01 11 11 10 00 00 11 00 11 10 11

t = 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17State 002 0 2 3 3 3 3 4 1 3 4 3 3 2 2 4 5 2State 012 3 1 2 2 3 1 4 4 1 4 2 3 4 4 2State 102 2 0 2 1 3 3 4 3 1 4 1 4 3 3 2State 112 3 1 2 1 1 3 4 4 3 4 2 3 4 4

00 11 11 00 01 10 01 11 11 10 00 00 11 00 11 10 11

t = 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17State 002 0 2 3 3 3 3 4 1 3 4 3 3 2 2 4 5 2State 012 3 1 2 2 3 1 4 4 1 4 2 3 4 4 2State 102 2 0 2 1 3 3 4 3 1 4 1 4 3 3 2State 112 3 1 2 1 1 3 4 4 3 4 2 3 4 4

00 11 11 00 01 10 01 11 11 10 00 00 11 00 11 10 11

0 1 0 1 1 1 0 0 1 0 1 0 0 0 1 0 0