exercise in the previous class (1) 0 1 1 0 1 0 0 1 0 1 0 1...
TRANSCRIPT
exercise in the previous class (1)
Define (one of) (15, 11) Hamming code:
construct a parity check matrix, and
determine the corresponding generator matrix
1
100011010011101
010010101011011
001001100110111
000100011101111
1100
1010
0110
1001
0101
0011
1110
1101
1011
0111
1111
I
generator matrix
parity check matrix
m = 4
24 – 1 nonzero vectors
exercise in the previous class (2)
Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4.
2
.
10001110
01001101
00101011
00010111
H
weight 1: HuT = hi = 0...contradiction
weight 2: HuT = hi + hj= 0 ...contradiction
weight 3: HuT = hi + hi + hl = 0, but...
column vectors have odd weight, and hi + hi has even weight
hi + hi = hl never happens...contradiction
weight 4: we have 10001110C
previous class
Hamming code
one-bit error correcting, one of perfect codes
minimum Hamming distance
smallest distance between two codewords
gives the maximum number of correctable errors
easy but “rough” estimation of performance
3
today’s class
two independent topics:
performance evaluation using weight distribution
slightly complicated (ちょっと複雑)
precise and detailed estimation is possible
cyclic code
a subclass of linear codes
advantages in realizations
4
is the minimum distance everything?
If the minimum Hamming distance of C is dmin,
then C can correct (dmin – 1) / 2 or less bit errors.
5
C1 C2 C3
< = ?
Consider three codes with the same code length...
many neighbors small # of neighbors
dmin dmin dmin
C2 seems better than C3...
1 codeword at distance 0 4 codewords at distance 3 3 codewords at distance 4
1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4
weight distribution
6
C={000000,100111,010110,110001, 001011,101100,011101,111010}.
weight distribution
in linear codes, all codewords have the same distance distribution
3
4 000000
distance distribution from 000000
from the view of 000000,
performance evaluation with weight distribution
assume...
BSC with bit error probability p
one-bit error correcting
send all-zero codeword and evaluate the probabilities of...
correctly decoded
wrongly decoded
error detected
7
C={000000,100111,010110,110001, 001011,101100,011101,111010}.
r 000000 decoder ?
received vector
1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4
the domain of the received vector
the received vector rV6
V6 contains eight decoding territories
8
C={000000,100111,010110,110001, 001011,101100,011101,111010}.
correctly decoded;
if r falls in the territory of 000000
wrongly decoded;
if r falls in the territory of the other codewords
error detected; if r does not fall in the territories
(not all vectors shown)
probability of correct decoding
correctly decoded if r belongs to
T = {000000, 000001, 000010, 000100, 001000, 010000, 100000}
9
r is decoded correctly with probability Pc = (1 – p)6 + 6p(1-p)5
prob. that rT and wH(r) = 0
prob. that rT and wH(r) = 1
(1 – p)6
6p( 1 – p)5
(1 – p)6
p( 1 – p)5
probability of wrong decoding (1)
case1: r belongs to the territory of a weight-3 codeword 001011
T = {001011, 001010, 001001, 001111, 000011, 011011, 101011}
10
prob. that rT and wH(r) = 3
prob. that rT and wH(r) = 2
p3(1 – p)3
3p2( 1 – p)4
prob. that rT and wH(r) = 4 3p4( 1 – p)2
r is decoded to the wrong codeword 001011 with probability 3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2
... the same discussion holds for other codewords with weight 3
probability of wrong decoding (2)
case2: r belongs to the territory of a weight-4 codeword 100111
T = {100111, 100110, 100101, 100011, 101111, 110111, 000111}
11
prob. that rT and wH(r) = 4
prob. that rT and wH(r) = 3
p4(1 – p)2
4p3( 1 – p)3
prob. that rT and wH(r) = 5 2p5( 1 – p)
r is decoded to the wrong codeword 100111 with probability 4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p)
... the same discussion holds for other codewords with weight 4
probability of wrong decoding (3)
wrongly decoded;
Pe = 4×(falls in the territory of one of weight-3 codewords)
+ 3×(falls in the territory of one of weight-4 codewors)
= 4×(3p2(1 – p)4 + p3(1 – p)3 + 3p4(1 – p)2 )
+ 3×(4p3(1 – p)3 + p4(1 – p)2 + 2p5(1 – p))
= 12p2(1 – p)4 + 16p3(1 – p)3 + 15p4(1 – p)2 + 6p5(1 – p)
error detected with probability Pd = 1 – Pc – Pe
12
C={000000,100111,010110,110001, 001011,101100,011101,111010}.
1 codeword with weight 0 4 codewords with weight 3 3 codewords with weight 4
the probabilities in a graph
13
00.10.20.30.40.50.60.70.80.9
1
0 0.1 0.2 0.3 0.4 0.5
pro
bab
iliti
es
bit error probability p
Pc
Pe
Pd
comparison example
(7,4) Hamming code: 0000000, 1000101, 0100111, 0010110,
0010110, 1010011, 0110001, 1110100,
0001011, 1001110, 0101100, 1101001,
0011101, 1011000, 0111010, 1111111
(9, 4) 2D code: 000000000, 000101011, 001001101, 001100110,
010010011, 010111000, 011011110, 011110101,
100010101, 100111110, 101011000, 101110011,
110000110, 110101101, 111001011, 111100000
14
weight 0 3 4 7
#codewords 1 7 7 1
weight 0 4 6
#codewords 1 9 6
comparison of two codes
15
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5
Hamm, Pc
Hamm, Pe
Hamm, Pd
2D, Pc
2D, Pe
2D, Pd
Hamming code seems better, but...
“retransmission”
re-transmission of broken data is allowed in some systems
16
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5
Hamm, Pc
Hamm, Pe
Hamm, Pd
2D, Pc
2D, Pe
2D, Pd
2D, Pc (retrans)
2D code + one-time re-transmission is better for p ≤ 0.28.
half the way...
two independent topics:
performance evaluation using weight distribution
slightly complicated (ちょっと複雑)
precise and detailed estimation is possible
cyclic code
a subclass of linear codes
advantages in realizations
17
cyclic codes
Linear codes are characterized by matrix operations.
Encoders/decoders are realized by combinatorial circuits.
simple but not scalable ... O(n2) for the code length n
cyclic codes (巡回符号)
a subclass of linear codes
Encoders/decoders are realized by shift registers.
complex but scalable ... O(n) complexity
18
codes linear codes
cyclic codes
preliminary (1)
We represent binary vectors by binary polynomials (多項式).
19
11101 x4 + x3 + x2 + x + 1
addition of polynomials: (subtraction)
x4 + x3 + x2 + 1 x3 + x + 1 +)
x4 + x2 + x
11101 01011
10110
+)
multiplication of polynomials:
x4 + x3 + x2 + 1 x3 + x + 1 ×)
x4 + x3 + x2 + 1 x5 + x4 + x3 + x
x7 + x6 + x5 + x3
x7 + x6 + x3 + x2+ x + 1 11001111
11101 01011 ×)
11101 11101
11101 multiply by xm
= left-shift of m bits
preliminary (2)
20
division of polynomials:
x6 + x4 x4 + x3 + x2 + 1 )
x2 + x + 1
x6 + x5 + x4 + x2
x5 + x2 x5 + x4 + x3 + x
x4 + x3 + x2 + x x4 + x3 + x2 + 1
x + 1
11101 ) 1010000 11101
10010 11101
11110 11101
11
111
addition = subtraction = XOR of coefficients
the division circuit is easily implemented by a shift register
division circuit (1)
divide p(x) = x6 + x4 by q(x) = x4 + x3 + x2 + 1...
21
1 1 1 0 1
1 0 1 0 0 0 0
q(x)
p(x)
quotient (商) remainder (剰余)
1. store p(x) to the registers 2. if MSB = 1, then AND gates are activated,
and registers are XOR’ed with q(x) 3. left-shift, and go to step 1
division circuit (2)
22
0 1 0 0 1
1 1 1 0 1
0 0
11101 ) 1010000 11101
010010 11101
11110 11101
11
111
1 0 1 0 0
1 1 1 0 1
0 0
one-cycle of operations
= one-step of the calculus
contents of the register
= remainder of the division
definition of cyclic codes
How to construct an (n, k) cyclic code:
length n, k information symbols, m = n – k parity symbols
Step 1: choose an order-m polynomial G(x) which divides xn + 1.
Step 2: C = {multiples of G(x) with order < n}
Example: n = 7, m = 4:
G(x) = x4 + x3 + x2 + 1 divides x7+1,
because x7+1 = (x3 + x2 + 1) (x4 + x3 + x2 + 1).
G(x)...generator polynomial
23
10000001 11101 )
11101 1101001
11101
11101 11101
0
1101
G(x) x7+1
construction of a cyclic code
Step 1: choose an order-m polynomial G(x) which divides xn + 1.
Step 2: C = {multiples of G(x) with order < n}
n = 7, m = 4: G(x) = x4 + x3 + x2 + 1
24
G(x) (x2+x+0)×
C=
0000000 0011101 0111010 0100111
1110100 1101001 1001110 1010011
G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)× G(x) (x2+x+1)×
=
properties of cyclic codes (1)
Lemma: a cyclic code is a linear code
proof: show that, for c1, c2C, we have c1+c2C
c1 C c1 = f1(x)G(x)
c2 C c2 = f2(x)G(x)
c1+c2 = (f1(x)+f2(x)) G(x) C
25
properties of cyclic codes (2)
Lemma: if (an-1, an-2, ..., a0)C, then (an-2, ..., a0, an-1)C
proof:
Let W(x) = an-1xn-1 + ... + a0 and W’(x) = an-2xn-1 + ... + a0x + an-1.
W(x) is a multiple of G(x)
W’(x) = an-2xn-1 + ... + a0x + an-1
= an-1xn + an-2xn-1 + ... + a0x + an-1+ an-1xn
= xW(x) + an-1(xn + 1)
26
multiple of G(x) multiple of G(x)
W’(x) is a multiple of G(x), and (an-2, ..., a0, an-1)C.
A cyclic code C is closed for a cyclic shift.
three approaches for encoding
three approaches for an encoding procedure:
matrix approach
use a generator matrix ... no advantage of cyclic codes
multiplication approach
codeword = (info. symbols) × G(x)
the code not systematic ( p.24)
division approach
slightly complicated (for human)
make the code systematic
easily implemented by shift registers
27
encoding by division
1. represent information symbols by a polynomial A(x)
2. divide A(x)xm by G(x), and let B(x) be the remainder
3. let A(x)xm + B(x) be a codeword
encoding 011, with n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1
1. A(x) = x + 1
2. A(x)x4 = x5 + x4 = x(x4 + x3 + x2 + 1) + (x3 + x), and B(x) = x3 + x
3. A(x)x4 + B(x)= x5 + x4 + x3 + x, the codeword = 0111010
28
A(x)
D(x) G(x)
B(x)
x4
÷ divisor
dividend
quotient
remainder
W(x) = A(x) x4 + B(x)
did we really make “encoding”?
simple question: Is A(x)xm + B(x) really a codeword?
is A(x)xm + B(x) divided by G(x)?
Yes, note that...
B(x) is a remainder of A(x)xm
in a binary world, A(x)xp + B(x) = A(x)xp – B(x)
“A(x)xp – B(x)” = “remove the remainder”
if A(x)xp – B(x) is divided by G(x), then there is no remainder...
29
example
n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1
30
data 000 001 010 011 100 101 110 111
A(x) x2+x+0 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1 x2+x+1
A(x)x4
0000000 0010000 0100000 0110000 1000000 1010000 1100000 1110000
B(x) x3+x2+x+0 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1 x3+x2+x+1
A(x)xm + B(x) 0000000 0011101 0100111 0111010 1001110 1010011 1101001 1110100
systematic code
encoder ≈ division circuit O(n) < O(n2) of matrix operation
error “detection” of cyclic codes
error “detection” is easy for cyclic codes
u C u (in a polynomial representation) is divided by G(x)
31
G(x)
= 0 ... no error 0 ... error ÷ divisor
dividend
quotient
remainder received u
one division circuit is used for encoding and error detection
reduces the cost of realization
Cyclic Redundancy Check (CRC)
... used in many communication systems
error “correction” of cyclic codes
general algorithm for all cyclic codes
error-trapping decoder [Kasami 61]
32
Tadao Kasami
1930-2007
E. Berlekamp, 1940- J. L. Massey, 1934-
I. Reed, 1923- (left)
and
G. Solomon,
1930-1996
special algorithms for special cyclic codes
Berlekamp-Massey algorithm for...
BCH (Bose-Chaudhuri-Hocquenghem) codes
Reed-Solomon Codes
summary
performance evaluation using weight distribution
slightly complicated (ちょっと複雑)
precise and detailed estimation is possible
cyclic code
a subclass of linear codes
advantages in realizations
33
exercise
Consider the following code C.
determine the weight distribution of C
compute the “three” probabilities (p. 7), and draw a graph
34
.
10001110
01001101
00101011
00010111
H
We want to construct a cyclic code with n = 7, k = 4, and m =3.
confirm that G(x) = x3 + x2 + 1 can be a generator polynomial
encode 0110
decide if 0001100 is a correct codeword or not