channel coding notes
TRANSCRIPT
-
7/30/2019 Channel Coding Notes
1/15
Channel Coding
Channel Coding is an encoding process that is done to add redundancy to thedata stream (block) before modulation or transmission. This redundancy is
added just for one specific reason: To make the reception of the transmitted
data at the receiver reliable (error free). So to sum it all up we can say that
Channel Coding is done to reduce or remove the error from the received data
at the receiver which gets added to the transmitted message during
transmission in the channel.
Difference between Channel Coding and Source Coding:
As mentioned earlier that Channel Coding is used to make the transmitted
data more reliable by adding redundancy to the actual data to be transmitted.
Whereas Source Coding is done to accomplish another important aspect as far
as the data transmission point of view is concerned that is Data Compression,
its basic function is to reduce that size of the data to be transmitted by the useof various Encoding Algorithms.
So as seen from the above given explanation that Channel Coding and Source
Coding are two completely different processes in the Digital Communication
System Block Diagram.
Note: Always Source Coding is done prior to Channel Coding is almost every
Digital Communication system.
Need for Channel Coding:
A general objective, however, is often to achieve maximum data transfer, in a
minimum bandwidth while maintaining an acceptable quality of transmission.
The quality of transmission, in the context of digital communications, is
essentially concerned with the probability of bit error (Pe) at the receiver.
-
7/30/2019 Channel Coding Notes
2/15
The Shannon Hartley Law (equation 1) demonstrates two basic
things:
Firstly it shows (quantitatively) how bandwidth (B) and signal power(S/N) may be traded in an ideal system.
Secondly it gives a theoretical limit for the transmission rate of (reliable,i.e. error free) data (Rmax) from a transmitter of given power, over a
channel with a given bandwidth, operating in a given noise environment.
Equation 1:
Rmax = B . log2. (1 + S: N) bits/sec
Where:
B = Bandwidth S:N = Signal to Noise Ratio Rmax = Maximum Information that can be transmitted
So for the communication system to operate at the Optimum Level, means to
send maximum amount of information in minimum possible bandwidth and
least transmitter power, the Signal to Noise Ratio should be reduced
considerably and made almost negligible, this can be done by the elimination
of errors in transmitted data for which Channel Coding is a must.
Classification of Channel Coding:
Channel Coding can be broadly classified into 2 main categories:
1.
Block Coding: In this type of channel coding the data to be transmittedis grouped into blocks and the redundancy is added separately to each
data block to form a codeword which is then transmitted.
2. Convolution Coding: In this type of channel coding the data to betransmitted is split in to a bit stream and redundancy is added to each
bit and is then transmitted.
-
7/30/2019 Channel Coding Notes
3/15
These 2 categories are further classified into sub groups and finally into
different techniques:
Block Coding:This main category can be divided into two sub groups:
RECC (Reverse Error Correcting Codes) FECC (Forward Error Correcting Codes)
The proper classification of Channel Coding is given in the following Tree
Diagram:
Tree Diagram:
So the above tree shows the classifications of the different coding techniques
used under the Channel Coding Block.
-
7/30/2019 Channel Coding Notes
4/15
RECC (Reverse Error Correcting Codes):
In this type of Error Correcting Code, the redundancy added to the
actual data can only help in detection of transmission errors or errors
due to the propagation through the channel of transmission. These
detected errors are corrected by sending a signal on a reverse channel to
the Transmitter to resend the corresponding block of data. Hence they
are called Reverse Error Correction Codes.
Popular Examples:Parity Check Code
Checksum
CRC (Cyclic Redundancy Check)
FECC (Forward Error Correcting Codes):
In this type of Error Correcting Code, the redundancy added not only
helps to detect errors due to transmission or propagation through the
channel but also helps to correct them at the Receiver without the needfor any resending of data and eliminating the need of a Reverse channel
to the Transmitter, hence making them more reliable and efficient. Since
the data transmission is only one way, (Transmitter to Receiver) they are
called Forward Error Correcting Codes.
Popular Examples: 3 Repetition Code
Hamming Code
LDPC (Low Density Parity Check Codes) / Gallager Code
Reed Solomon Code
Important Abbreviations:
http://www.telecomvt.org/documents/acronyms-glossary-SCG.pdf
http://www.telecomvt.org/documents/acronyms-glossary-SCG.pdfhttp://www.telecomvt.org/documents/acronyms-glossary-SCG.pdfhttp://www.telecomvt.org/documents/acronyms-glossary-SCG.pdf -
7/30/2019 Channel Coding Notes
5/15
Channel Coding Techniques:
RECC (Reverse Error Correcting Code):
Parity Check:
In this type of Error Correcting Code, the no. of 1s is counted and an
additional parity bit is added to define the count. There are two kinds of Parity
Checks:
Even Parity:The additional parity bit is 0 for even no. of 1s and is1 for odd no. of 1s.
Odd Parity:The additional parity bit is 1 for even no. of 1s and is0 for odd no. of 1s.
During data transmission, the additional parity bit is sent along with the
corresponding Data block and at the Receiver the received data block is
matched with its parity bit, mismatch will occur due to any transmission errors
then the Receiver sends a signal to the Transmitter on a Reverse Channel
asking for retransmission of the corresponding data block.
Checksum:
In this type of Error Correction Code, the data block is split into individual bits
and each of the bits is EXORed (Modulo - 2 Addition) to get a final resultant bit
which is transmitted along with the corresponding block of data.
Example: Data Block = 101011, then 1 0 1 0 1 1 = 0, therefore theChecksum of the above given data block = 0 so the codeword transmitted will
be 1010110 instead of 101011.
At the Receiver the received data block is matched with its Checksum, if a
mismatch occurs due to any transmission errors then the Receiver sends a
signal to the Transmitter on a Reverse Channel asking for retransmission of the
corresponding data block.
-
7/30/2019 Channel Coding Notes
6/15
CRC (Cyclic Redundancy Check):
In this type of Error Correcting Code, for each data block a short finite length
binary sequence called as the CRC is generated and sent along with the
corresponding Data block, this CRC is generated based on a Polynomial Long
Division method in which the quotient is discarded and the remainder is
considered to be the CRC.
Example:11010011101100
-
7/30/2019 Channel Coding Notes
7/15
FECC (Forward Error Correcting Code):
Repetition Code:
In this type of Error Correcting Code, the data bits to be transmitted over the
channel are repeated and then sent. This code is specified using the following
specification.
(n,m) Repetition Code, where n = total no. of bits (size of the codeword)
and m = no. of message bits (data bits).
k (redundancy) = n (codeword)m (data)
Consider n = 3 & m = 1, therefore the code can be given by (3,1) Repetition
Code or simply by 3 Repetition Code.
So this can be described by saying that each bit to be transmitted is repeated 3
times and transmitted.
Example: If 100110 is to be transmitted then 111 000 000 111 111 000 the
codeword will be sent instead of the actual data, so as can be seen from the
codeword that each data bit is repeated 3 times.
This Code can detect errors equivalent up to k Hamming Distance (k = 2) and
can correct errors up to m Hamming Distance only (m = 1).
Hamming Distance:
** Red path defines a Hamming Distance of 3 whereas the Blue path defines that of 2 **
Definition of 1 Hamming Distance: Its the distance from one point in the
lattice to another point in the lattice allowing only one of three bits to change.
-
7/30/2019 Channel Coding Notes
8/15
Error Correcting Property:
Consider 10110 is transferred over a channel in the form of 111 000 111 111
000, suppose at the Receiver the codeword is received along with some errors
in it. Suppose for the above given codeword, a code word corresponding to110 000 111 101 010 is received then the receiver corrects the data all by itself
by the use of a Majority Vote algorithm, in which 110 has majority of 1s so it
must have been 111 at the transmitter which corresponds to 1. Similarly for
010 = 000 = 0. Note that the errors detected and corrected here are at 1
Hamming Distance.
Now for Errors more than 1 Hamming Distance, like for example 1 = 111 was
transmitted and received as 001, then the Receiver will detect an Error and the
following assumptions will be made at the Receiver, 001 = 000 = 0, so the
error will be detected but Not Corrected at the Receiver.
To increase the Error correction capability, no. of repetitions should be more
which will compromise on the Code Rate.
Code Rate:
Code Rate can be defined as the Ratio of the Actual Data Bits to the Codeword
Transmitted. Lesser the Code Rate, lesser is the Efficiency of the Code.
Code Rate = no. of data bits = m = 1 = 0.33334
Codeword bits n 3
So the Code Rate for 3 Repetition Code = 0.333334
Advantages of Repetition Codes Disadvantages of Repetition Codes
Can Detect & Correct errors up to 1
Hamming Distance
Can Detect but cannot Correct Errors
of >1 Hamming Distance
Easy to Implement Inefficient due to Low Code Rate
-
7/30/2019 Channel Coding Notes
9/15
Hamming Code:
In this type of Error Correction Code, the data message bits (m) to be sent over
the channel is added with an additional check bits (k) to form a codeword of(n) bits, which is then transmitted over the channel using various modulation
techniques. This added redundancy at the transmitter helps in detection and
correction of the errors at the receiver and to obtain the original data bits (m)
from the Codeword (n), the redundancy bits (k) are removed.
Hamming Code is specified in a similar manner: (n,m) Hamming Code.
Generally (7,4) Hamming Code is used for small scale operations.
Generation of Hamming Code:
The Hamming Code (X) is generated using multiplication of two Matrices, the
first matrix is that of the Message bit stream (m bits) and the second is the
Generator Matrix which is nothing but the concatenation of the Identity Matrix
and the Parity Check Matrix.
Order of Matrices: Message Matrix(M) (1 x m) | m = 4
Generator Matrix(G) (m x n) | n = 7 & k = 3
Hamming Code Matrix(X) (1 x n)
Therefore it can be written as: X = M x G
-
7/30/2019 Channel Coding Notes
10/15
Message Matrix(M): Its nothing but the actual data bit stream to be
transmitted.
Parity Check Matrix: It is a k bit number having a Hamming Distance
minimum of 2 arranged in any order but not repeated.
Hamming Code Matrix(X):Its the multiplication of the Message Matrix(M)
and Generator Matrix(G).
Assuming the above given data, the Hamming Matrix is calculated as follows
using Matrix Multiplication and Mod-2 Addition:
(X) = [0110011]1x7, this is the Hamming Code for Data = 0110 in a (7,4)
Hamming Code.
Error Detection & Correction using Hamming Codes:
So at the Receiver, Codeword is received along with the Error signal, this is
decoded using a Syndrome Detection & Correction Technique.
Let Y (Received Codeword) = X(Codeword) + E(Error)
Syndrome(S) = HTx Y = H
T.X + H
T.E
If Syndrome(S) = 0 then there is no error but if (S) is not equal to 0 then Error
exists.
Where HT= Transpose of the concatenation of Parity Check Matrix / Identity
Matrix.
-
7/30/2019 Channel Coding Notes
11/15
Process at the Receiver:
At the Receiver, the Syndrome is calculated first using the relations given
above and then depending on the Syndrome, most likely possible Error patternis selected from the Look-Up Table, hence the error is detected and the Most
likely error pattern is EXORed with the Received codeword to get the
Corrected Data Block.
Analysis: Since n = 7 and k = 3;
Possible Error Patterns = 27
= 128
Possible Syndromes = 23 = 8
So: Possible Error Patterns = 128 = 16
Possible Syndromes 8
Therefore 16 different Error patterns can Generate 1 Syndrome so many errors
are undetected and hence uncorrected.
This can be overcome by increasing the no. of check bits in the Coding, whichnot only helps to improve the error detection and correction capacity but also
improves the Code Rate.
How increasing k increases the Code Rate?
Code Rate = m = nk = 2k 1k
n n 2k 1
So as k increase, Code Rate tends to be equal to 1, which makes this Code
highly efficient.
-
7/30/2019 Channel Coding Notes
12/15
Advantages of Hamming Code Disadvantages of Hamming Code
Very Reliable Code Complexity in Algorithms
High Code Rate Not all Errors can be detected
Cheap to Implement Look-Up Table required at Receiver
LDPC (Low Density Parity Check Code):
In this kind of FECC, these codes are highly efficient since they work at almost
the channel capacities of various different channels and also at the same time
have a huge error detection and correction capacity.
Representation of LDPC:
They can be represented in two ways:
1. Matrix Method2. Graphical Method
Matrix Method Representation:
To know this method lets look at the below given example of a (8,4) Parity
Check Matrix. Let the dimensions of the matrix be n = 8, m = 4 respectively.
[The no. of 1s in each row = Wr& the no. of 1s in each column = Wc ]
For this to be a LDPC matrix, Wr
-
7/30/2019 Channel Coding Notes
13/15
The Graphical method consists of a Bi-partite graph where v_nodes are
Variable Nodes and c_nodes are Fixed Nodes also the Points Co to C7 are
variable as shown above and also f0 to f3 are fixed.
Note: Wr = 2 & Wc = 4
Relation between the Matrix form and Graphical form is simple that is theRows in the Matrix represent the points f0 to f3 and the Columns of the matrix
are nothing but the points Co to C7 . All those co- ordinates in the matrix where
there is a 1 are the lines joining those two corresponding nodes in the Graph.
Performance & Complexity of LDPC:
To improve the error detection and correction capacity the length of the data
block (bit stream) has to be large which also serves the purpose of increasingthe Code Rate and hence the efficiency but requires a huge bandwidth and to
design such an Algorithm is very complex as well as the involving complexities
at the Decoder, so as a compromise the length of the data block is always kept
fixed and an appropriate length is taken for optimum performance.
Example:Within 0.004dB of the Shannons Limit, with an error probability of
10-6
can be achieved by taking block length of 107
bits.
-
7/30/2019 Channel Coding Notes
14/15
Decoding LDPC:
The decoding of this code can be done using various algorithms, some of the
popular algorithms are:
Belief Propagation Algorithm Message Passing Algorithm Sum-Product Algorithm
These are really complex, so to demonstrate the actual decoding of the LDPC,
we use the Hard Decoding Technique.
But first assume that the data transmitted is c = [1 0 0 1 0 1 0 1] and the
received codeword c = [1 1 0 1 0 1 0 1] where we can see that due to the error
in the channel, the bit at C1got flipped from 0 to 1.
Now to detect and correct the received codeword involves a series of steps
based on the Algorithm used.
Step1: In this step the Variable nodes send the message bits to their
corresponding Fixed nodes as shown below.
-
7/30/2019 Channel Coding Notes
15/15
Step2: Here individual responses are calculated by the Fixed node on the basis
of the information sent to them by 3 Variable node and it decides the value of
the fourth Variable node to satisfy the parity check matrix and sends it to that
particular Variable node this is again shown in the table above.
Step3: In this step the Variable node decides whether the bit it received
initially was correct or not based on the majority vote of 3 inputs to it, 2 by
Fixed Nodes and 1 by Input itself, as shown below if found to be incorrect it is
corrected and the process for that particular Variable node is terminated.
Step4: This jumps to Step 2 and repeats steps 2 & 3 for all the given Variable
nodes. Once done the entire process terminates.
Advantages of LDPC Disadvantages of LDPC
High Code Rate Very Complex to Implement
High Efficiency Needs large Bandwidth
Highly Reliable Complex Algorithms
Highly Compatible