![Page 1: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/1.jpg)
Constructing ErrorConstructing Error--Correction Codes Correction Codes from Scalefrom Scale--Free NetworksFree Networks
Francis C.M. LauFrancis C.M. Lau
Department of Electronic and Information EngineeringHong Kong Polytechnic University
International Workshop on Complex Systems and Networks 2007 Guilin, China
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
![Page 2: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/2.jpg)
IWCSN' 2007, Guilin, China 2
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 1: Communications and CodingPart 1: Communications and Coding
![Page 3: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/3.jpg)
IWCSN' 2007, Guilin, China 3
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Communications without CodingHow are you today ?
Hxw au& u%$ wqo .
Welf affi zv iol bxg.
How aruyox tuday ?
Information can be easily corrupted when sent through a channel !
![Page 4: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/4.jpg)
IWCSN' 2007, Guilin, China 4
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
More Reliable Communications
How are you today ?How are you today ?How are you today ?
How aru yox tuday ?How aee yeu todey ?Hoe are you toxak ?
How are you today ?
Error-correction capability
channel
![Page 5: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/5.jpg)
IWCSN' 2007, Guilin, China 5
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Bit-level Communications
111100
011010
101001
110000
10
Without coding schemes:
0,1 0,0
InfoSource
channel InfoSink
noise
0,1 0,1000, 111 000, 110
1 information bit2 check bits
Code Rate=number of information bits/ block length=1/3
Block Length =1+2=3
With coding schemes:Info
SourceEncoder channel Decoder Info
Sink
noise
![Page 6: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/6.jpg)
IWCSN' 2007, Guilin, China 6
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Reliable Communications
Add redundant information at transmitterDecode information intelligently at receiver
Error-Correction Capability
Any better ways than to repeat the information several times?Any performance bounds?
![Page 7: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/7.jpg)
IWCSN' 2007, Guilin, China 7
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Shannon’s Capacity Theorem
Channel
Additive White Gaussian NoiseBandwidth W
Average Received Signal Power SAverage Noise Power N
⎟⎠⎞
⎜⎝⎛ +=
NSWC 1log2
System Capacity of the channel
C. E. Shannon “A mathematical theory of communications,” Bell Syst. Tech. J., vol. 27, pp. 379–423, 623–656, 1948.
![Page 8: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/8.jpg)
IWCSN' 2007, Guilin, China 8
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Shannon’s Capacity Theorem
possible theoretically to transmit information at any rate R ≤ C with an arbitrarily small error probability (with coding)if R > C, not possible to transmit information with an arbitrarily small error probability (even with coding)
⎟⎠⎞
⎜⎝⎛ +=
NSWC 1log2
ChannelInformation with rate R
![Page 9: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/9.jpg)
IWCSN' 2007, Guilin, China 9
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Normalized channel capacity versus SNR
⎟⎠⎞
⎜⎝⎛ +=
NS
WC 1log2
![Page 10: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/10.jpg)
IWCSN' 2007, Guilin, China 10
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Normalized channel bandwidth versus SNR
⎟⎠⎞
⎜⎝⎛ +
=
NSC
W
1log
1
2
The graph is not telling the whole story !
![Page 11: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/11.jpg)
IWCSN' 2007, Guilin, China 11
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Because …
noise power is proportional to bandwidth
WNN 0=
⎟⎠⎞
⎜⎝⎛ +
=
NSC
W
1log
1
2
![Page 12: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/12.jpg)
IWCSN' 2007, Guilin, China 12
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Further …
when bit rate R equals channel capacity C
CS
RSEb ==energy per bit
![Page 13: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/13.jpg)
IWCSN' 2007, Guilin, China 13
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Shannon Limit
⎟⎠⎞
⎜⎝⎛ +=
NS
WC 1log2
( )12 /
0
−= WCb
CW
NE
WNN 0=
CS
RSEb ==
rearrangement
![Page 14: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/14.jpg)
IWCSN' 2007, Guilin, China 14
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
( )12 /
0
−= WCb
CW
NE
Shannon Limit
0//
dB 59.1/ 0
→⇔∞→⇒−→
WCCW
NEb
Channel capacity approaches zero, regardless of the channel bandwidth
No error-free communications below dB 59.1/ 0 −=NEb
![Page 15: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/15.jpg)
IWCSN' 2007, Guilin, China 15
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Typical error performance of coded and uncodedmodulations
![Page 16: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/16.jpg)
IWCSN' 2007, Guilin, China 16
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Improved error performance;more bandwidth required to
add redundancy bits
![Page 17: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/17.jpg)
IWCSN' 2007, Guilin, China 17
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Reduction in requirement;more bandwidth required to add
redundancy bits
0/ NEb
coding gain
![Page 18: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/18.jpg)
IWCSN' 2007, Guilin, China 18
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Significance/Conclusions of Shannon’s work
proved theoretically that there exists codes that could improve the error probability performance from uncoded modulation schemesthere is a minimum requirement0/ NEb
![Page 19: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/19.jpg)
IWCSN' 2007, Guilin, China 19
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BUT ….
![Page 20: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/20.jpg)
IWCSN' 2007, Guilin, China 20
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
How should we design coding schemes, with reasonable
complexity, that work as close to the Shannon limit as possible?
![Page 21: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/21.jpg)
IWCSN' 2007, Guilin, China 21
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Solutions
Not provided by Shannon !
So do research on Coding !
![Page 22: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/22.jpg)
IWCSN' 2007, Guilin, China 22
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 2: ParityPart 2: Parity--Check CodesCheck Codes
![Page 23: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/23.jpg)
IWCSN' 2007, Guilin, China 23
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Codes
single-parity-check code
110011010100
parity bit
message bits
![Page 24: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/24.jpg)
IWCSN' 2007, Guilin, China 24
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Codes
single-parity-check code
even-parity code
can detect all single-and triple-error patterns (e.g. 0100 or 0010) but cannot correct errors
![Page 25: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/25.jpg)
IWCSN' 2007, Guilin, China 25
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
001111110011110000000110100001101011
Parity-Check Codes
rectangular code (or product code)
horizontal parity check
vertical parity check
message
![Page 26: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/26.jpg)
IWCSN' 2007, Guilin, China 26
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
001111110011110000000110100001101011
Parity-Check Codes
rectangular code (or product code)
can correct a single error pattern
001111110011110000001110100001101011
horizontal parity check fails
vertical parity check fails
bit in error
channel
![Page 27: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/27.jpg)
IWCSN' 2007, Guilin, China 27
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Linear Block Codes
a class of parity-check codesdenoted by (n, k)
codeword length message length
maps k-bit messages (k-tuples) linearlyand uniquely to n-bit codewords (n-tuples)
![Page 28: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/28.jpg)
IWCSN' 2007, Guilin, China 28
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Subset S of a vector space is a subspace if
it contains the all-zeros vectorsum of any two vectors in S is also in S
![Page 29: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/29.jpg)
IWCSN' 2007, Guilin, China 29
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Packing as many Packing as many codewordscodewords in the entire in the entire space as possible improves space as possible improves coding efficiency coding efficiency
Putting the Putting the codewordscodewords as as far apart from one far apart from one another as possible another as possible increases the chance of increases the chance of decoding the decoding the codewordscodewordscorrectlycorrectly
![Page 30: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/30.jpg)
IWCSN' 2007, Guilin, China 30
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
(6, 3) Code Example
form a subspace
011101+
Modulo-2 Addition
all-zeros vector
![Page 31: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/31.jpg)
IWCSN' 2007, Guilin, China 31
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Modulo-2 Multiplication• 0 10 0 01 0 1
Modulo-2 Addition and Multiplication
addition can be accomplished electronically using an Exclusive-OR gatemultiplication can be accomplished using an AND gate
Modulo-2 Addition+ 0 10 0 11 1 0
![Page 32: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/32.jpg)
IWCSN' 2007, Guilin, China 32
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Encoding the Messages
Table look-up is possible for small kFor large k, table look-up may become extremely difficult
e.g., 301026.12100 ×≈⇒= kk
Use of Generator Matrix
![Page 33: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/33.jpg)
IWCSN' 2007, Guilin, China 33
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Generator Matrix G (size k x n)
a basis set of k linearly independent n-tuples that spans the subspace
n-tuple
n-tuple
n-tuple
][ 21 kmmm L=mmessage
mGU =codeword (size 1 x n)
![Page 34: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/34.jpg)
IWCSN' 2007, Guilin, China 34
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
(6, 3) Code Example
][ 654321 uuuuuu=
![Page 35: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/35.jpg)
IWCSN' 2007, Guilin, China 35
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Matrix H (size (n-k) x n)
For each generator matrix G, there exists an (n-k) x nmatrix H such that rows of G are orthogonal to rows of H.
0GH =Tk x (n-k) all-zeros matrix
0mGHUH == TT
H can be used to test whether a received vector is a valid codeword.
HG ↔
![Page 36: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/36.jpg)
IWCSN' 2007, Guilin, China 36
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Matrix H (size (n-k) x n)
0GH =T
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
110100011010101001
H
rows areorthogonal
![Page 37: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/37.jpg)
IWCSN' 2007, Guilin, China 37
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Matrix H
000
101110011100010001
][
653
542
641
654321
=++=++=++
⇒
=
⎥⎥⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢⎢⎢
⎣
⎡
⇒=
uuuuuuuuu
uuuuuuT 00UH
![Page 38: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/38.jpg)
IWCSN' 2007, Guilin, China 38
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Bipartite Graph
)0(5422 =++= uuuc
)0(6533 =++= uuuc
1u
2u
3u
4u
5u
6u
variable nodes
check nodes
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
110100011010101001
H
)0(6411 =++= uuuc
![Page 39: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/39.jpg)
IWCSN' 2007, Guilin, China 39
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Decoding
received vector r’
Codeword UAdditive White Gaussian Noise Channel0 +1 volt
1 −1 volt
![Page 40: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/40.jpg)
IWCSN' 2007, Guilin, China 40
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Eight codewords in a 6-tuple space
Hard Decoding
received vector r’(AWGN channel)
decoded codeword after error correction
after making hard decision on each bit
ri > 0 volt 0ri < 0 volt 1
![Page 41: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/41.jpg)
IWCSN' 2007, Guilin, China 41
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
r’
)'|011101( rU =P
)'|101110( rU =P
)'|110100( rU =P
maximuma posteriori (MAP) decision rule: Select codeword U that has the largest )'|( rUP
)'|101001( rU =Pa posteriori probability (APP)
Soft Decoding
![Page 42: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/42.jpg)
IWCSN' 2007, Guilin, China 42
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Performance of Some Well-known Block Codes (Coherent BPSK over an AWGN channel)
t = maximum number of guaranteed correctable errors per codeword
![Page 43: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/43.jpg)
IWCSN' 2007, Guilin, China 43
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Performance of BCH Codes (Coherent BPSK over an AWGN channel)
![Page 44: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/44.jpg)
IWCSN' 2007, Guilin, China 44
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 3: LowPart 3: Low--DensityDensity--ParityParity--Check (LDPC) CodesCheck (LDPC) Codes
![Page 45: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/45.jpg)
IWCSN' 2007, Guilin, China 45
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Matrix H (size (n-k) x n)
0GH =T
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
110100011010101001
H
![Page 46: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/46.jpg)
IWCSN' 2007, Guilin, China 46
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Low-Density-Parity-Check Codes
proposed by Gallager(1960)parity-check matrix H
sparse (most elements are zeros)fraction of 1’s ~ O(n)
elements of H determine the connections between variable nodes and check nodes degree of variable
node u6 = 2
degree of checknode c3 = 3
![Page 47: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/47.jpg)
IWCSN' 2007, Guilin, China 47
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Low-Density-Parity-Check Codes
sparse (low-density) parity-check matrix Himplies that all variable nodes and check nodes have very few connections
HG ↔
![Page 48: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/48.jpg)
IWCSN' 2007, Guilin, China 48
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Error rates achieved by different coding schemes under the binary AWGN channel.Codeword length = 106. Rate =0.5.
![Page 49: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/49.jpg)
IWCSN' 2007, Guilin, China 49
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Types of LDPC Codes
Regular LDPC all nodes of the same type (variable node or check node) have the same degree
A (3, 6)-regular LDPC code of length 10 and rate one-half.
check node degree
variable node degree
![Page 50: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/50.jpg)
IWCSN' 2007, Guilin, China 50
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Types of LDPC Codes
Irregular LDPC: the degrees of each set of nodes are chosen according to some distribution
![Page 51: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/51.jpg)
IWCSN' 2007, Guilin, China 51
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Degree Distribution of Nodes
Degree distribution of variable nodes
Degree distribution of check nodes
fraction of edges connected to the variable nodes with degree k
∑=
−=vd
k
kk xx
2
1)( λλ
∑=
−=cd
k
kk xx
2
1)( ρρfraction of edges connected to the
check nodes with degree k
maximum variable node degree
maximum check node degree
![Page 52: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/52.jpg)
IWCSN' 2007, Guilin, China 52
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Code Rate
∫∫−= 1
0
1
0
d)(
d)(1
xx
xxR
λ
ρ
![Page 53: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/53.jpg)
IWCSN' 2007, Guilin, China 53
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Question
Given specific
∑=
−=vd
k
kk xx
2
1)( λλ ∑=
−=cd
k
kk xx
2
1)( ρρand .
How would the LDPC code perform?
![Page 54: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/54.jpg)
IWCSN' 2007, Guilin, China 54
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
AnswerDependent on the actual design
(connections),
the channel type, e.g. AWGN, binary symmetric channel (BSC),
binary erasure channel (BEC)
and the decoding algorithm.
![Page 55: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/55.jpg)
IWCSN' 2007, Guilin, China 55
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
More Question
Any idea on the optimal performance of
practical LDPC decoders?
![Page 56: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/56.jpg)
IWCSN' 2007, Guilin, China 56
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 4: Part 4: Belief Propagation (BP) Decoding Belief Propagation (BP) Decoding AlgorithmAlgorithm
![Page 57: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/57.jpg)
IWCSN' 2007, Guilin, China 57
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Belief Propagation (BP) Decoding Algorithm
A kind of message-passing decoding algorithmApplicable to both regular and irregular LDPC codesProduces very good error performance
![Page 58: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/58.jpg)
IWCSN' 2007, Guilin, China 58
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Belief Propagation (BP) Decoding Algorithm
Define Log Likelihood Ratio (LLR):
⎥⎦
⎤⎢⎣
⎡==
info)|1bit(info)|0bit(log
PP
variable nodes
check nodes
![Page 59: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/59.jpg)
IWCSN' 2007, Guilin, China 59
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Compute initial Log Likelihood Ratio (LLR) for each variable node based on the received signal vector r (real number elements)
⎥⎦
⎤⎢⎣
⎡==
)|1bit()|0bit(log
i
i
rPrP
1r)(LLR 10 r
iterationnumber
ir
![Page 60: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/60.jpg)
IWCSN' 2007, Guilin, China 60
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Set iteration number k = 1Pass the LLR messages from variable nodes to the connected check nodes
)(LLR 10 r
![Page 61: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/61.jpg)
IWCSN' 2007, Guilin, China 61
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Check nodes received the LLR messages from the connected variable nodes
![Page 62: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/62.jpg)
IWCSN' 2007, Guilin, China 62
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
![Page 63: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/63.jpg)
IWCSN' 2007, Guilin, China 63
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
![Page 64: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/64.jpg)
IWCSN' 2007, Guilin, China 64
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
![Page 65: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/65.jpg)
IWCSN' 2007, Guilin, China 65
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
![Page 66: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/66.jpg)
IWCSN' 2007, Guilin, China 66
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
![Page 67: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/67.jpg)
IWCSN' 2007, Guilin, China 67
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
![Page 68: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/68.jpg)
IWCSN' 2007, Guilin, China 68
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each variable node update its LLR based on the messages passed from the check nodes and the initial LLRBased on the updated LLR, estimate the codeword
)(LLR)(LLR 1110 rr →
![Page 69: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/69.jpg)
IWCSN' 2007, Guilin, China 69
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Estimate the codeword as
)(LLR)(LLR 1110 rr →
[ ]nccc ˆˆˆˆ 21 L=c
where ⎭⎬⎫
⎩⎨⎧
<>
=0)(LLR if1 0)(LLR if0
ˆ1
1
rr
ck
ki
If , is the decoded codeword.
0Hc =Tˆ c
![Page 70: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/70.jpg)
IWCSN' 2007, Guilin, China 70
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
)(LLR 10 r
If , increment the iteration number k.Each variable node computes a message for each of its connected check nodes, based on its initial LLR and the messages from all other connected check nodes
0Hc ≠Tˆ
![Page 71: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/71.jpg)
IWCSN' 2007, Guilin, China 71
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
)(LLR 10 r
Each variable node computes a message for each of its connected check nodes, based on its initial LLR and the messages from all other connected check nodes
![Page 72: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/72.jpg)
IWCSN' 2007, Guilin, China 72
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
)(LLR 10 r
Each variable node computes a message for each of its connected check nodes, based on its initial LLR and the messages from all other connected check nodes
![Page 73: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/73.jpg)
IWCSN' 2007, Guilin, China 73
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Check nodes received the LLR messages from the connected variable nodes
![Page 74: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/74.jpg)
IWCSN' 2007, Guilin, China 74
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodesSame iterative process repeated …. until convergence to a valid codeword or maximum number of iterations exceeded
![Page 75: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/75.jpg)
IWCSN' 2007, Guilin, China 75
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Capacity (Threshold) of LDPC codesGiven the degree distributions
and the channel type (AWGN, BSC or BEC)and the use of BP decoding algorithm.
∑=
−=vd
k
kk xx
2
1)( λλ ∑=
−=cd
k
kk xx
2
1)( ρρ
Richard and Urbanke (2001) proposed an effective algorithm – density evolution – to determine the capacity (threshold) of LDPC codes.
T. J. Richardson and R. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,” IEEE Trans. Inform. Theory, vol. 47, pp. 599–618, Feb. 2001.
![Page 76: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/76.jpg)
IWCSN' 2007, Guilin, China 76
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Density Evolution Algorithm
channel type
∑=
−=vd
k
kk xx
2
1)( λλ
∑=
−=cd
k
kk xx
2
1)( ρρDensity
Evolution Algorithm
iterations
threshold value *σ
a higher threshold value indicates a higher achievableachievable performance of the code
![Page 77: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/77.jpg)
IWCSN' 2007, Guilin, China 77
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Good Degree Distribution Pairs (Rate = 0.5)
T. J. Richardson et al., “Design of capacity-approaching irregular low-density parity-check codes,”IEEE Trans. Inform. Theory, vol. 47, pp. 619–637, Feb.2001.
![Page 78: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/78.jpg)
IWCSN' 2007, Guilin, China 78
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Good Degree Distribution Pairs (Rate = 0.5)
![Page 79: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/79.jpg)
IWCSN' 2007, Guilin, China 79
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
the threshold value , for example indicates the maximum noise power that can be tolerated for error-free communication in AWGN channelsthe threshold value can be achievedachieved if
the message-passing process does not contain any cyclesnumber of iterations tends to infinitycodeword length is infinitecodeword length is infinite
Density Evolution Algorithm*σ
*σ
![Page 80: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/80.jpg)
IWCSN' 2007, Guilin, China 80
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Density Evolution AlgorithmProblems:
optimizing the codes based on DE algorithm is not a simple task
codeword length cannot be infiniteoptimizing the threshold value may give a more complex code
number of connections
∑=
−=vd
k
kk xx
2
1)( λλ ∑=
−=cd
k
kk xx
2
1)( ρρ
vary to maximize the threshold value
![Page 81: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/81.jpg)
IWCSN' 2007, Guilin, China 81
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 5: Review of Complex NetworksPart 5: Review of Complex Networks
![Page 82: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/82.jpg)
IWCSN' 2007, Guilin, China 82
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Basic properties
Path length: the distance between two nodes, which is defined as the number of edges along the shortest path connecting them
![Page 83: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/83.jpg)
IWCSN' 2007, Guilin, China 83
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Basic properties
Betweenness centrality: the fraction of shortest paths going through a given nodeAssortative mixing: preference of high-degree nodes attach to other high-degree nodesDisassortative mixing: preference of high-degree nodes attach to low-degree nodes
![Page 84: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/84.jpg)
IWCSN' 2007, Guilin, China 84
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex Networks
Random NetworksGiven a network with N nodes. Each pair of nodes are connected with a probability of p. Poisson distribution
( )!
kep kk
μμ −
=
μ =
0.1ERp =
![Page 85: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/85.jpg)
IWCSN' 2007, Guilin, China 85
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex Networks
Regular Coupled Networks
high clusteringlarge average path length
Fully-connected Networks
2 2.5 3 3.5 4 4.50
0.2
0.4
0.6
0.8
1
<k>
P(k
)
![Page 86: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/86.jpg)
IWCSN' 2007, Guilin, China 86
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex Networks
Small-World NetworksEach edge of a regular coupled network is re-wired with a probability of phigh clusteringsmall average path length
0WSp =
0.2WSp =
![Page 87: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/87.jpg)
IWCSN' 2007, Guilin, China 87
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex Networks
Scale-Free Networks
![Page 88: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/88.jpg)
IWCSN' 2007, Guilin, China 88
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex NetworksScale-Free (SF) Networks
γii nn −~)Pr( γxxf −~)(
![Page 89: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/89.jpg)
IWCSN' 2007, Guilin, China 89
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Characteristics of Typical Complex Networks
HighUniformLongRegular Coupled Network
-
High
Low
Clustering Coefficient
Power-LawVery ShortScale Free Networks
-Short Small World Networks
PoissonShortRandom Networks
Degree Distribution
Average Distance
(log( ))NΟ
(log(log( )))*NΟ
*The exponent parameter should be valued between 2 and 3.See reference “Scale-Free Networks Are Ultrasmall”, PRL, vol. 90, no. 5
(log( ))NΟ
( )NΟ
Fast to disseminate information
![Page 90: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/90.jpg)
IWCSN' 2007, Guilin, China 90
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 6: ScalePart 6: Scale--free Networks to free Networks to LDPC CodesLDPC Codes
![Page 91: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/91.jpg)
IWCSN' 2007, Guilin, China 91
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Scale-free Networks meet LDPC Codes
Can the “very short distance” property of scale-free network helps passing/spreading messages quickly in the decoding of LDPC codes?
If so, how?
![Page 92: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/92.jpg)
IWCSN' 2007, Guilin, China 92
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Scale-free Networks meet LDPC Codes
??
![Page 93: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/93.jpg)
IWCSN' 2007, Guilin, China 93
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
From Bipartite Graph to Unipartite Graph
![Page 94: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/94.jpg)
IWCSN' 2007, Guilin, China 94
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
From Bipartite Graph to Unipartite Graphpower-law degree distribution
Power-law degree distribution !
![Page 95: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/95.jpg)
IWCSN' 2007, Guilin, China 95
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Typical node degree distribution of the corresponding unipartite graph. Codeword length = 10000 and maximum variable node degree = 20.X. Zheng, F.C.M. Lau and C.K. Tse, " Study of LDPC Codes Built on Scale-Free Networks," Proceedings, NOLTA'06, Bologna, Italy, September 2006, pp. 563-566.
![Page 96: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/96.jpg)
IWCSN' 2007, Guilin, China 96
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Building LDPC Codes From SF Networks
Assume that the variable nodes have the power-law degree distribution and the check nodes obey the Poisson-law degree distribution
Use the (Density Evolution) DE to select the optimized parameters and .
( ) ~P k k γλ
−
γ μ
!)(
lelP
l μ
ρμ −
=
![Page 97: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/97.jpg)
IWCSN' 2007, Guilin, China 97
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Threshold value and average variable node degrees <k> of LDPC codes built from scale-free networks and the optimized ones reported in [1] for an AWGNchannel. Rate equals 0.5.
*σ
[1]
[1] T. J. Richardson et al., “Design of capacity-approaching irregular low-density parity-check codes,”IEEE Trans. Inform. Theory, vol. 47, pp. 619–637, Feb.2001.
![Page 98: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/98.jpg)
IWCSN' 2007, Guilin, China 98
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Typical node degree distribution of the corresponding unipartite graph. Codeword length = 1000 and maximum variable node degree = 15.
X. Zheng, F.C.M. Lau and C.K. Tse, “Error Performance of Short-Block-Length LDPC Code Built on Scale-Free Networks,” Proceedings, The Third Shanghai International Symposium on Nonlinear Sciences and Applications, Shanghai, China, June 2007, pp. 55-57.
![Page 99: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/99.jpg)
IWCSN' 2007, Guilin, China 99
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Building LDPC Codes From SF Networks
Threshold values lower compared with those reported in the literatureAverage variable node degrees <k> lower compared with those reported in the literature
*σ
Which one is better in practice ?
![Page 100: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/100.jpg)
IWCSN' 2007, Guilin, China 100
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
The threshold value can be achievedachieved ifthe message-passing process does not contain any cycles andnumber of iterations tends to infinity andcodeword length is infinitecodeword length is infinite
Threshold Value
*σ
![Page 101: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/101.jpg)
IWCSN' 2007, Guilin, China 101
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
PEG and Enhanced PEG
Progressive Edge-Growth algorithm (PEG)an effective method to construct codes with girth average as large as possible based on the given degree distributions
Enhanced PEG (E-PEG) proposed by usstopping set and the near codeword are also checked after each variable node is added.
X. Y. Hu, E. Eleftheriou and D. M. Arnold, “Regular and irregular progressive edge-growth tanner graphs,” IEEE Trans. Inform. Theory, vol. 51, no. 1, pp. 386–398, 2005.
![Page 102: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/102.jpg)
IWCSN' 2007, Guilin, China 102
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 7: Simulation ResultsPart 7: Simulation Results
![Page 103: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/103.jpg)
IWCSN' 2007, Guilin, China 103
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Block Error RatesBlock length=1008Code rate=0.5Max. no. of iterations = 50
![Page 104: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/104.jpg)
IWCSN' 2007, Guilin, China 104
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Bit Error Rates
Block length=1008Code rate=0.5Max. no. of iterations = 50
![Page 105: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/105.jpg)
IWCSN' 2007, Guilin, China 105
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
0 0.5 1 1.5 2 2.5 310-6
10-5
10-4
10-3
10-2
10-1
100
SNR(dB)
Blo
ck E
rror R
ate
DE10 (PEG), <k>=3.66 DE10 (E-PEG), <k>=3.66
PEG and E-PEG Algorithms
Block length=1008Code rate=0.5Max. no. of iterations = 50
![Page 106: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/106.jpg)
IWCSN' 2007, Guilin, China 106
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
0 0.5 1 1.5 2 2.5 310-8
10-6
10-4
10-2
100
SNR(dB)
Bit
Erro
r Rat
e
DE10 (PEG), <k>=3.66 DE10 (E-PEG), <k>=3.66
PEG and E-PEG Algorithms
Block length=1008Code rate=0.5Max. no. of iterations = 50
![Page 107: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/107.jpg)
IWCSN' 2007, Guilin, China 107
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
0 0.5 1 1.5 2 2.5 310-7
10-6
10-5
10-4
10-3
10-2
10-1
100
SNR(dB)
Blo
ck E
rror R
ate
SF20 (PEG), <k>=3.72 SF20 (E-PEG), <k>=3.72
PEG and E-PEG Algorithms
Block length=1008Code rate=0.5Max. no. of iterations = 50
![Page 108: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/108.jpg)
IWCSN' 2007, Guilin, China 108
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
0 0.5 1 1.5 2 2.5 310-8
10-6
10-4
10-2
100
SNR(dB)
Bit
Erro
r Rat
e
SF20 (PEG), <k>=3.72 SF20 (E-PEG), <k>=3.72
PEG and E-PEG Algorithms
Block length=1008Code rate=0.5Max. no. of iterations = 50
![Page 109: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/109.jpg)
IWCSN' 2007, Guilin, China 109
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 8: SummaryPart 8: Summary
![Page 110: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/110.jpg)
IWCSN' 2007, Guilin, China 110
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Summary
Coding for a reliable communicationOperation principles of parity-check codesLow-density-parity-check (LPDC) codes Belief propagation decoding algorithm Density evolution
![Page 111: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/111.jpg)
IWCSN' 2007, Guilin, China 111
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
SummaryLPDC codes with scale-free variable-node-degree distribution achieve very good theoretical threshold (error correction performance)Short LPDC codes built with scale-free variable-node-degree distribution outperform other well-known LPDC codes with similar complexity
![Page 112: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/112.jpg)
IWCSN' 2007, Guilin, China 112
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Some of Our Related Work1. X. Zheng, F.C.M. Lau and C.K. Tse, “Error Performance of Short-Block-Length
LDPC Code Built on Scale-Free Networks,” Proceedings, The Third Shanghai International Symposium on Nonlinear Sciences and Applications, Shanghai, China, June 2007, pp. 55-57.
2. X. Zheng, F.C.M. Lau, Chi K. Tse and S.C. Wong, “Study of Bifurcation Behavior of LDPC Decoders", International Journal of Bifurcation and Chaos, vol. 16, no. 11, pp. 3435-3449, Nov. 2006.
3. X. Zheng, F.C.M. Lau and Chi K. Tse, “Study of LDPC Codes Built on Scale-Free Networks,” Proceedings, International Symposium on Nonlinear Theory and Its Applications (NOLTA'06), Bologna, Italy, September 2006, pp. 563-566.
4. X. Zheng, F.C.M. Lau, C.K. Tse and S.C. Wong, “Techniques for Improving Block Error Rate of LDPC Decoders,” Proceedings, IEEE International Symposium on Circuits and Systems (ISCAS'06), Kos, Greece, May 2006, pp. 2261-2264.
5. X. Zheng, F.C.M. Lau, Chi K. Tse and S.C. Wong, “Study of Nonlinear Dynamics of LDPC Decoders", Proceedings, European Conference on Circuit Theory and Design (ECCTD ‘2005), Dublin, Ireland, August 2005, paper 207. (CD version)
![Page 113: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/113.jpg)
IWCSN' 2007, Guilin, China 113
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Collaborators
Dr Wai-man TAMProf. Chi K. TSEDr Siu C. WONGMiss Xia ZHENG
![Page 114: Constructing Error-Correction Codes from Scale-Free Networks](https://reader033.vdocuments.us/reader033/viewer/2022060400/6294b404cef16370092bf28a/html5/thumbnails/114.jpg)
Constructing ErrorConstructing Error--Correction Codes Correction Codes from Scalefrom Scale--Free NetworksFree Networks
Francis C.M. LauFrancis C.M. Lau
Department of Electronic and Information EngineeringHong Kong Polytechnic University
International Workshop on Complex Systems and Networks 2007 Guilin, China
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Thank You !