digital communication systems 452 - 3 - discrete...1 digital communication systems ecs 452 asst....

1

Digital Communication SystemsECS 452

Asst. Prof. Dr. Prapun [email protected]

3 Discrete Memoryless Channel (DMC)

Office Hours: Check Google Calendar on the course website.Dr.Prapun’s Office:6th floor of Sirindhralai building, BKD

2



3.1 DMC Models

3

Digital communication system

System considered back in CH2

System Under Consideration

4

Digital communication system

System considered in Section 3.1

System Under Consideration

Noisy Channel

5

EquivalentChannel

X: channel input

Y: channel output

10

1111111111111111111111111111111111111111111111111111111111111111111111111111111111000000000000111111…

10

1101101111011110111110111111101010011110111101111111100111111111111001111111011111001000000000101101…

[BSC_demo_image.m]

MATLAB

6

x_raw = imresize(imread('SIIT_25LOGO.png'),[150 150]); x_raw = [rgb2gray(x_raw) > 128]; % convert to 0 and 1n = prod(size(x_raw)); x = reshape(x_raw,1,n); % convert to row vector

figurex_img = reshape(x,size(x_raw));imshow(x_img*255);

[BSC_demo_image.m]

Another Noisy Channel

7

EquivalentChannel

X: channel input

Y: channel output

[DMC_demo_2.m]

TTTTTTTTTTTTTTTHTHTTTTTHTTTHTTTTTTTHTTTTHTTHTTTTTTTTTTTTTTTTHTTTHTTTHTHTTTTTTTTHHTTTTTTTTTHTTHTHTTTT

HEHHEHTTTHEETETHETHHHETTHEEHEEHEHTHHEHEETTHTHEETETETEEEEHHTHHETEHEETHTEEHEEHEHEEHHEHTTTEEHTTTHEHEEHH

MATLAB

8

%% Generating the channel input xx = randsrc(1,n,[S_X;p_X]); % channel input

%% Applying the effect of the channel to create the channel output yy = DMC_Channel_sim(x,S_X,S_Y,Q); % channel output

function y = DMC_Channel_sim(x,S_X,S_Y,Q)%% Applying the effect of the channel to create the channel output yy = zeros(size(x)); % preallocationfor k = 1:length(x)

% Look at the channel input one by one. Choose the corresponding row% from the Q matrix to generate the channel output.y(k) = randsrc(1,1,[S_Y;Q(find(S_X == x(k)),:)]);

end

[DMC_Analysis_demo.m]

[DMC_Channel_sim.m]

Ex 3.3: BSC

9

>> BSC_demoans =1 0 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1ans =1 1 1 1 1 1 1 1 1 0 1 1 0 1 0 1 1 1 1 1

Elapsed time is 0.134992 seconds.

%% Simulation parameters% The number of symbols to be transmittedn = 20; % Channel Input S_X = [0 1]; S_Y = [0 1];p_X = [0.3 0.7];% Channel Characteristicsp = 0.1; Q = [1-p p; p 1-p];

[BSC_demo.m]

p_X =0.3000 0.7000

p_X_sim =0.1500 0.8500

q =0.3400 0.6600

q_sim =0.1500 0.8500

Q =0.9000 0.10000.1000 0.9000

Q_sim =0.6667 0.33330.0588 0.9412

PE_sim =0.1000

PE_theretical =0.1000

Rel. freq. from the simulation

10

%% Statistical Analysis% The probability values for the channel inputsp_X % Theoretical probabilityp_X_sim = hist(x,S_X)/n % Relative frequencies from the simulation% The probability values for the channel outputsq = p_X*Q % Theoretical probabilityq_sim = hist(y,S_Y)/n % Relative frequencies from the simulation% The channel transition probabilities from the simulationQ_sim = [];for k = 1:length(S_X)

I = find(x==S_X(k)); LI = length(I);rel_freq_Xk = LI/n; yc = y(I);cond_rel_freq = hist(yc,S_Y)/LI; Q_sim = [Q_sim; cond_rel_freq];

endQ % Theoretical probabilityQ_sim % Relative frequencies from the simulation

[DMC_Analysis_demo.m]

Ex 3.3: BSC

11

>> BSC_demoans =1 0 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1ans =1 1 1 1 1 1 1 1 1 0 1 1 0 1 0 1 1 1 1 1


%% Simulation parameters% The number of symbols to be transmittedn = 20; % Channel Input S_X = [0 1]; S_Y = [0 1];p_X = [0.3 0.7];% Channel Characteristicsp = 0.1; Q = [1-p p; p 1-p];

[BSC_demo.m]

p_X =0.3000 0.7000

p_X_sim =0.1500 0.8500

q =0.3400 0.6600

q_sim =0.1500 0.8500

Q =0.9000 0.10000.1000 0.9000

Q_sim =0.6667 0.33330.0588 0.9412

PE_sim =0.1000


Because there are only 20 samples, we can’t expect the relative freq. from the simulation to match the specified or calculated probabilities.

Ex 3.3: BSC

12

%% Simulation parameters% The number of symbols to be transmittedn = 1e4; % Channel Input S_X = [0 1]; S_Y = [0 1];p_X = [0.3 0.7];% Channel Characteristicsp = 0.1; Q = [1-p p; p 1-p];

[BSC_demo.m]

>> BSC_demo

p_X =0.3000 0.7000

p_X_sim =0.3037 0.6963

q =0.3400 0.6600

q_sim =0.3407 0.6593


Q =0.9000 0.10000.1000 0.9000

Q_sim =0.9078 0.09220.0934 0.9066

PE_sim =0.0930


Ex 3.9: DMC

13

p_X =0.2000 0.8000

p_X_sim =0.2000 0.8000

q =0.3400 0.3600 0.3000

q_sim =0.4000 0.3500 0.2500

Q =0.5000 0.2000 0.30000.3000 0.4000 0.3000

Q_sim =0.7500 0 0.25000.3125 0.4375 0.2500

>> sym(Q_sim)ans =[ 3/4, 0, 1/4][ 5/16, 7/16, 1/4]PE_sim =

0.7500

>> DMC_demoans =1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 0 1 1 0 1ans =1 3 2 2 1 2 1 2 2 3 1 1 1 3 1 3 2 3 1 2

x:

y:

0

1

1

3

0.5

0.3

X Y

%% Simulation parameters% The number of symbols to be transmittedn = 20; % General DMC% Ex. 3.6 amd 3.12 in lecture note% Channel Input S_X = [0 1]; S_Y = [1 2 3];p_X = [0.2 0.8];% Channel CharacteristicsQ = [0.5 0.2 0.3; 0.3 0.4 0.3];

[DMC_demo.m]

2𝒑 𝟎 𝟎. 𝟐

𝒑 𝟏 𝟎. 𝟖

Ex 3.9 & 3.13: DMC

14

>> p_X_sim * Q_simans =

0.4000 0.3500 0.2500

p_X =0.2000 0.8000

p_X_sim =0.2000 0.8000

q =0.3400 0.3600 0.3000

q_sim =0.4000 0.3500 0.2500

Q =0.5000 0.2000 0.30000.3000 0.4000 0.3000

Q_sim =0.7500 0 0.25000.3125 0.4375 0.2500

>> sym(Q_sim)ans =[ 3/4, 0, 1/4][ 5/16, 7/16, 1/4]PE_sim =

0.7500

q pQ

Review: Evaluation of Probability from the Joint PMF Matrix

15

Consider two random variables X and Y.

Suppose their joint pmf matrix is

Find

0.1 0.1 0 0 0 0.1 0 0 0.1 00 0.1 0.2 0 00 0 0 0 0.3

2 3 4 5 6xy

1346

3 4 5 6 75 6 7 8 96 7 8 9 108 9 10 11 12

2 3 4 5 6xy

1346

Step 1: Find the pairs (x,y) that satisfy the condition“x+y < 7”

One way to do this is to first construct the matrix of x+y.

,X YP

x y


16



Find

0.1 0.1 0 0 0 0.1 0 0 0.1 00 0.1 0.2 0 00 0 0 0 0.3

2 3 4 5 6xy

1346

3 4 5 6 75 6 7 8 96 7 8 9 108 9 10 11 12

2 3 4 5 6xy

1346

Step 2: Add the corresponding probabilities from the joint pmf (matrix)

,X YP

x y𝑃 𝑋 𝑌 7 0.1 0.1 0.1

0.3


17



Find

0.1 0.1 0 0 0 0.1 0 0 0.1 00 0.1 0.2 0 00 0 0 0 0.3

2 3 4 5 6xy

1346

,X YP

Review: Sum of two discrete RVs

18



Find

0.1 0.1 0 0 00.1 0 0 0.1 00 0.1 0.2 0 00 0 0 0 0.3

2 3 4 5 6xy

1346

3 4 5 6 75 6 7 8 96 7 8 9 108 9 10 11 12

2 3 4 5 6xy

1346

x y𝑃 𝑋 𝑌 7 0.1

,X YP

Summary: DMC

19

|

,

Channel Input Alphabet

𝑃 𝑋 𝑥

𝑃 𝑌 𝑦

𝑃 𝑌 𝑦|𝑋 𝑥

𝑃 𝑋 𝑥, 𝑌 𝑦

matrix

matrixrow vector

row vector “AND”

Put together All values at corresponding positions in the matrix.

y

Q

1

i

m

p x

p x

p x

,

1

,X Y i ji

m

x

x

x

p x y

y 1 j ny y y

jq y q pQ

P

1

i

m

x

x

x

1

j

n

y

y

y

j iQ y x

1

j ii

m

x

yx

x

Q x

1 j ny y y

Matrix Multiplication

20

diag

diag

0 0 00 0 0

1 1 1

0 0 0 5 4 6 6 5 4 6

5 4 6 66

60 0 0 6 1 6 3 6 6 30

1 6 31 1 1 1 18 11 19 15

1 2 1

10 0 00 0

0 0 1 2 1 5 2 50 0 0 6 4 6 1 6 4

56 4 6 1

6

0a b c d

a b c d

a a a a ab b b b b

c c c c cd d d d d

ab

cd

a b c d

Ex: DMC

21

>> p = [0.2 0.8];>> Q = [0.5 0.2 0.3; 0.3 0.4 0.3];>> p*Qans =

0.3400 0.3600 0.3000>> P = (diag(p))*QP =

0.1000 0.0400 0.06000.2400 0.3200 0.2400

>> sum(P)ans =

0.3400 0.3600 0.3000

22



3.2 Decoder and

for Naïve Decoder

23 [DMC_Analysis_demo.m]

%% Naive Decoderx_hat = y;

%% Error ProbabilityPE_sim = 1-sum(x==x_hat)/n % Error probability from the simulation

% Calculation of the theoretical error probabilityPC = 0;for k = 1:length(S_X)

t = S_X(k);i = find(S_Y == t);if length(i) == 1

PC = PC+ p_X(k)*Q(k,i);end

endPE_theretical = 1-PC

Formula derived in 3.24 of lecture notes

[Ex. 3.25]: Naïve Decoder and BAC

24

p_X =0.5000 0.5000

p_X_sim =0.7000 0.3000

q =0.5500 0.4500

q_sim =0.6500 0.3500

Q =0.7000 0.30000.4000 0.6000

Q_sim =0.7143 0.28570.5000 0.5000

PE_sim =0.3500


>> BAC_demoans =0 0 0 1 1 0 0 1 0 0 0 0 1 0 0 1 0 1 0 0ans =0 0 1 1 0 0 0 1 1 1 0 0 1 0 0 0 0 0 1 0

x:

y:

0

1

0

1

0.3

0.7

0.4

0.6

X Y

%% Simulation parameters% The number of symbols to be transmittedn = 20; % Binary Assymmetric Channel (BAC)% Ex 3.8 in lecture note (11.3 in [Z&T, 2010])% Channel Input S_X = [0 1]; S_Y = [0 1];p_X = [0.5 0.5];% Channel CharacteristicsQ = [0.7 0.3; 0.4 0.6];

[BAC_demo.m]

𝒑 𝟎 𝟎. 𝟓

𝒑 𝟏 𝟎. 𝟓

720

25

p_X =0.5000 0.5000

p_X_sim =0.5043 0.4957

q =0.5500 0.4500

q_sim =0.5532 0.4468

Q =0.7000 0.30000.4000 0.6000

Q_sim =0.7109 0.28910.3928 0.6072

PE_sim =0.3405


0

1

0

1

0.3

0.7

0.4

0.6

X Y

%% Simulation parameters% The number of symbols to be transmittedn = 1e4; % Binary Assymmetric Channel (BAC)% Ex 3.8 in lecture note (11.3 in [Z&T, 2010])% Channel Input S_X = [0 1]; S_Y = [0 1];p_X = [0.5 0.5];% Channel CharacteristicsQ = [0.7 0.3; 0.4 0.6];

𝒑 𝟎 𝟎. 𝟓

𝒑 𝟏 𝟎. 𝟓

10 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

10010 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

100

[BAC_demo.m]

[Ex. 3.25]: Naïve Decoder and BAC

[Ex. 3.16]:

26

p_X_sim =0.2326 0.7674

q =0.4698 0.5302

q_sim =0.4672 0.5328

Q =0.7000 0.30000.4000 0.6000

Q_sim =0.6928 0.30720.3989 0.6011

PE_sim =0.3776


0

1

0

1

0.3

0.7

0.4

0.6

X Y

S_X = [0 1]; S_Y = [0 1];p_X = [0.5 0.5];% Channel CharacteristicsQ = [0.7 0.3; 0.4 0.6];x_raw = imresize(imread('SIIT_25LOGO.png'),[150 150]); x_raw = [rgb2gray(x_raw) > 128]; % convert to 0 and 1x = reshape(x_raw,1,n);

[BAC_demo_image.m]

Naïve Decoder and BAC

[Ex. 3.26]: Naïve Decoder and DMC

27

p_X =0.2000 0.8000

p_X_sim =0.2000 0.8000

q =0.3400 0.3600 0.3000

q_sim =0.4000 0.3500 0.2500

Q =0.5000 0.2000 0.30000.3000 0.4000 0.3000

Q_sim =0.7500 0 0.25000.3125 0.4375 0.2500

PE_sim =0.7500


>> DMC_demoans =1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 0 1 1 0 1ans =1 3 2 2 1 2 1 2 2 3 1 1 1 3 1 3 2 3 1 2

x:

y:

0

1

1

3

0.5

0.3

X Y

%% Simulation parameters% The number of symbols to be transmittedn = 20; % General DMC% Ex. 3.16 in lecture note% Channel Input S_X = [0 1]; S_Y = [1 2 3];p_X = [0.2 0.8];% Channel CharacteristicsQ = [0.5 0.2 0.3; 0.3 0.4 0.3];

2𝒑 𝟎 𝟎. 𝟐

𝒑 𝟏 𝟎. 𝟖

20 420

[Same samples as in Ex. 3.6]

[DMC_demo.m]

28

p_X =0.2000 0.8000

p_X_sim =0.2011 0.7989

q =0.3400 0.3600 0.3000

q_sim =0.3387 0.3607 0.3006

Q =0.5000 0.2000 0.30000.3000 0.4000 0.3000

Q_sim =0.4943 0.1914 0.31430.2995 0.4033 0.2972

PE_sim =0.7607


0

1

1

3

0.5

0.3

X Y

%% Simulation parameters% The number of symbols to be transmittedn = 1e4; % General DMC% Ex. 3.16 in lecture note% Channel Input S_X = [0 1]; S_Y = [1 2 3];p_X = [0.2 0.8];% Channel CharacteristicsQ = [0.5 0.2 0.3; 0.3 0.4 0.3];

2𝒑 𝟎 𝟎. 𝟐

𝒑 𝟏 𝟎. 𝟖

10 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

10010 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

100

[DMC_demo.m]

[Ex. 3.26]: Naïve Decoder and DMC

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

DigitalDemodulator

Transmitter

Remove redundancy

Add systematic redundancy

EquivalentChannel

X: channel input

Y: channel outputSource Decoder

Channel Decoder(Detector)

Receiver

Decodedvalue

ˆ ˆX x Y

System Model for Section 3.2-3.3

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

DigitalDemodulator

Transmitter

Remove redundancy


X: channel input



Decodedvalue

[Ex. 3.12, 3.18, 3.26]

0

1

1

3

0.5

0.3

2

10111110110111100101…

21133122121231131311…

Receiver ˆ ˆX x Y

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

DigitalDemodulator

Transmitter

Remove redundancy


X: channel input


Decodedvalue

[Ex. 3.26]: Naïve Decoder

0

1

1

3

0.5

0.3

2

10111110110111100101…

21133122121231131311…

ˆ12

1

3 32

x yy

ˆ01

1

3 02

x yy

21133122121231131311…

Naïve Decoder

ˆ 0.76P P X X

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

DigitalDemodulator

Transmitter

Remove redundancy


X: channel input


Decodedvalue

[Ex. 3.27]: DIY Decoder

0

1

1

3

0.5

0.3

2

10111110110111100101…

21133122121231131311…

ˆ01

1

3 02

x yy

ˆ01

1

3 02

x yy

10000011010100000000…

ˆ 0.52P P X X


33

>> DMC_decoder_DIY_demoans =1 0 1 1 1 1 1 0 1 1 0 1 1 1 1 0 0 1 0 1ans =2 1 1 3 3 1 2 2 1 2 1 2 3 1 1 3 1 3 1 1ans =1 0 0 0 0 0 1 1 0 1 0 1 0 0 0 0 0 0 0 0PE_sim =

0.5500PE_theretical =

0.5200Elapsed time is 0.081161 seconds.

%% Simulation parameters% The number of symbols to be transmittedn = 20; % General DMC% Ex. 3.16 in lecture note% Channel Input S_X = [0 1]; S_Y = [1 2 3];p_X = [0.2 0.8];% Channel CharacteristicsQ = [0.5 0.2 0.3; 0.3 0.4 0.3];

%% DIY DecoderDecoder_Table = [0 1 0]; % The decoded values corresponding to the received Y

[DMC_decoder_DIY_demo.m]

X

Y

𝑋

ˆ01

1

3 02

x yy


34


% Decode according to the decoder tablex_hat = y; % preallocationfor k = 1:length(S_Y)

I = (y==S_Y(k));x_hat(I) = Decoder_Table(k);

end

PE_sim = 1-sum(x==x_hat)/n % Error probability from the simulation

% Calculation of the theoretical error probabilityPC = 0;for k = 1:length(S_X)

I = (Decoder_Table == S_X(k));q = Q(k,:); PC = PC+ p_X(k)*sum(q(I));



Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

DigitalDemodulator

Transmitter

Remove redundancy


X: channel input



0

1

1

3

0.5

0.3

2

ˆ01

1

3 02

x yy

ˆ 0.52P P X X

10 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

100


36

>> DMC_decoder_DIY_demoPE_sim =

0.5213PE_theretical =

0.5200Elapsed time is 2.154024 seconds.

%% Simulation parameters% The number of symbols to be transmittedn = 1e4; % General DMC% Ex. 3.16 in lecture note% Channel Input S_X = [0 1]; S_Y = [1 2 3];p_X = [0.2 0.8];% Channel CharacteristicsQ = [0.5 0.2 0.3; 0.3 0.4 0.3];


10 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

10010 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

10010 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

100


ˆ 0.52P P X X

𝑋 𝑋𝑌

Recipe for finding of any decoder

37

Use the 𝐏 matrix. If unavailable, can be found by scaling each row of the 𝐐 matrix by its

corresponding 𝑝 𝑥 . Write 𝑥 𝑦 values on top of the 𝑦 values for the 𝐏 matrix. For column 𝑦 in the 𝐏 matrix, circle the element whose corresponding

𝑥 value is the same as 𝑥 𝑦 . 𝑃 the sum of the circled probabilities. 𝑃 1 𝑃 .

0.2

0.8

00 01

0.10 0.04 0.060.24 0.

.5 0.21

0.30.3 0.4 0.3 32 0.24

Q P

1 2 3 1 2 3x y xy1 1 0 x̂ y

0

1

1

3

0.5

0.3

X Y2

ˆ11

1

3 02

x yy

[Ex. 3.28]

[3.29]

38



3.3 Optimal Decoder

Searching for the Optimal Detector

39

>> DMC_decoder_ALL_demoans =

0 0 0 0.80000 0 1 0.62000 1 0 0.52000 1 1 0.34001 0 0 0.66001 0 1 0.48001 1 0 0.38001 1 1 0.2000

Min_PE =0.2000

Optimal_Detector =1 1 1


𝑥 1 𝑥 2 𝑥 3 𝑃 ℰ

Ex. 3.27

Ex. 3.28

Review: ECS315 (2018)

40

Guessing Game 1

41

There are 15 cards. Each have a number on it. Here are the 15 cards:

1 2 2 3 3 3 4 4 4 4 5 5 5 5 5 One card is randomly selected from the 15 cards.

You need to guess the number on the card.

Have to pay 1 Baht for incorrect guess.

The game is to be repeated n = 10,000 times.

What should be your guess value?

42

close all; clear all;

n = 5; % number of time to play this game

D = [1 2 2 3 3 3 4 4 4 4 5 5 5 5 5];X = D(randi(length(D),1,n));

if n <= 10X

end

g = 1cost = sum(X ~= g)

if n > 1averageCostPerGame = cost/nend

>> GuessingGame_4_1_1X =

3 5 1 2 5g =

1cost =

4averageCostPerGame =

0.8000

43


n = 5; % number of time to play this game


if n <= 10X

end

g = 3.3cost = sum(X ~= g)


>> GuessingGame_4_1_1X =

5 3 2 4 1g =

3.3000cost =

5averageCostPerGame =

1

44


n = 1e4; % number of time to play this game


if n <= 10X

end

g = ?cost = sum(X ~= g)


Guessing Game 1

451 1.5 2 2.5 3 3.5 4 4.5 5

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Guess value

AV

erag

e C

ost P

er G

ame

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 60.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Guess value

AV

erag

e C

ost P

er G

ame

Guessing Game 1

46

Optimal Guess: The most-likely value

Summary: MAP vs. ML Decoders

47

Decoder is derived from the 𝐏 matrix Select (by circling) the maximum value

in each column (for each value of 𝑦) in the 𝐏 matrix.

The corresponding 𝑥 value is the value of 𝑥 𝑦 .

MAP ,

optimal

ˆ argmax ,

ˆ

argmax

argmax

X Yx

x

x

x y p x y

x y

P X x Y y

Q y p xx

MLˆ argmaxx

x y Q y x

Decoder is derived from the 𝐐 matrix Select (by circling) the maximum value

in each column (for each value of 𝑦) in the 𝐐 matrix.

The corresponding 𝑥 value is the value of 𝑥 𝑦 .

Once the decoder (the decoding table) is derived 𝑃 and 𝑃 are calculated by adding the corresponding probabilities in the 𝐏 matrix.

Optimal at least when 𝑝 𝑥 is uniform (the channel inputs are equally likely)

a posteriori probability

likelihood funtion

Can be derived without knowing the channel input probabilities.

prior probability

Example: MAP Decoder [Ex. 3.36]

48

0.2

0.8

00 01

0.10 0.04 0.060.24 0.

.5 0.21

0.30.3 0.4 0.3 32 0.24

Q P

1 2 3 1 2 3x y xy1 1 1 MAPx̂ y

0

1

1

3

0.5

0.3

X Y2

( ) 0.24 0.32 0.( ) 1 21 24 0.P P

MAPˆ

13 1

1 12

x yy

Example: ML Decoder [Ex. 3.47]

49

( ) 0.10 0.( 32 0.06) 1 1 0.52PP

( ) 0.10 0.( 32 0.24) 1 1 0.34PP

0.2

0.8

00 01

0.10 0.04 0.060.24 0.

.5 0.21

0.30.3 0.4 0.3 32 0.24

Q P

1 2 3 1 2 3x y xy0 1 0 MLx̂ y

0

1

1

3

0.5

0.3

X Y2

0.2

0.8

00 01

0.10 0.04 0.060.24 0.

.5 0.21

0.30.3 0.4 0.3 32 0.24

Q P

1 2 3 1 2 3x y xy

0

1

1

3

0.5

0.3

X Y2

MLx̂ y0 1 1

Sol 1:

Sol 2:

ML

1ˆ01

3 12

yy x

ML

1ˆ01

3 02

yy x[Same as Ex. 3.27]

[Agree with Ex. 3.33]

MAP Decoder

50

%% MAP DecoderP = diag(p_X)*Q; % Weight the channel transition probability by the

% corresponding prior probability.[V I] = max(P); % For I, the default MATLAB behavior is that when there are

% multiple max, the index of the first one is returned.Decoder_Table = S_X(I) % The decoded values corresponding to the received Y

%% Decode according to the decoder tablex_hat = y; % preallocationfor k = 1:length(S_Y)


end


%% Calculation of the theoretical error probabilityPC = 0;for k = 1:length(S_X)

I = (Decoder_Table == S_X(k));Q_row = Q(k,:); PC = PC+ p_X(k)*sum(Q_row(I));


[DMC_decoder_MAP_demo.m]

ML Decoder

51

%% ML Decoder[V I] = max(Q); % For I, the default MATLAB behavior is that when there are

% multiple max, the index of the first one is returned.Decoder_Table = S_X(I) % The decoded values corresponding to the received Y

%% Decode according to the decoder tablex_hat = y; % preallocationfor k = 1:length(S_Y)


end


%% Calculation of the theoretical error probabilityPC = 0;for k = 1:length(S_X)

I = (Decoder_Table == S_X(k));Q_row = Q(k,:); PC = PC+ p_X(k)*sum(Q_row(I));


[DMC_decoder_ML_demo.m]

Optimal Decoder for BSC

57

close all; clear all;tic

%% Simulation parameters% Channel Input S_X = [0 1]; S_Y = [0 1];p0 = 0.8; p1 = 1-p0; p_X = [p0 p1];% Channel Characteristicsp = 0.1; Q = [1-p p; p 1-p];

%% All possible "reasonable" decoders% X_hat = Y; X_hat = 1-Y; X_hat = 1; X_hat = 0Decoder_Table_ALL = [0 1; 1 0; 1 1; 0 0]

%% Calculate the error probability for each of the decoder PE_ALL = [];for k = 1:size(Decoder_Table_ALL,1)

Decoder_Table = Decoder_Table_ALL(k,:);PC = 0;for k = 1:length(S_X)

I = (Decoder_Table == S_X(k));Q_row = Q(k,:);PC = PC + p_X(k)*sum(Q_row(I));

endPE_theretical = 1-PC;PE_ALL = [PE_ALL; PE_theretical];

end

%% Display the results[Decoder_Table_ALL PE_ALL]

%% Find the optimal detectors[V I] = min(PE_ALL);Optimal_Detector = Decoder_Table_ALL(I,:)Min_PE = V

toc

>> BSC_decoder_ALL_demoDecoder_Table_ALL =

0 11 01 10 0

ans =0 1 0.10001 0 0.90001 1 0.80000 0 0.2000

Optimal_Detector =0 1

Min_PE =0.1000


[BSC_decoder_ALL_demo.m]

𝑥 0 𝑥 1 𝑃 ℰ

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

DigitalDemodulator

Transmitter

Remove redundancy


EquivalentChannel

X: channel input



Receiver

Decodedvalue

System Model for Section 3.2-3.3

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

DigitalDemodulator

Transmitter

Remove redundancy


EquivalentChannel

Source Decoder


Receiver

Decodedvalue

System Model for Section 3.4

X: channel input

Y: channel output

Some results from Section 3.3-3.4

60

Information Source

Destination

Message

Recovered Message

Source Encoder

Channel Encoder

Transmitter

Remove redundancy


EquivalentChannel

X: channel input



Receiver

Decodedvalue

System Model for Section 3.5

Channel Encoder

[3.62] Block Encoding

k bits k bits k bits n bits n bits n bits

X

𝒏, 𝒌 code

Code rate = 𝒏𝒌

Channel Encoder

X

[3.62] Block Encoding

k bits n bits

Codebook

𝒔 𝐬𝐬

𝐬

𝐱

𝐱

𝐱

𝐱

𝑀 2 possibilities

Choose 𝑀 2 from 2 possibilities to be used as codewords.

[Figure 13]

64



3.5 Repetition Code in Communications Over BSC

Hypercube

65 https://www.youtube.com/watch?v=Q_B5GpsbSQw

Hypercube

66 https://www.youtube.com/watch?v=Q_B5GpsbSQw

n-bit space

67

n = 1

n = 2

n = 3

n = 4

Channel Encoder and Decoder

68

Noise & In

terferen

ce

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

Source Decoder

Channel Decoder

DigitalDemodulator

Transmitter

Receiver


X: channel input

Y: channel output

0

1

0

1

p

1-p

p

1-p

S

𝐒

Better Equivalent Channel

69

Noise & In

terferen

ce

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

Source Decoder

Channel Decoder

DigitalDemodulator

Transmitter

Receiver

Remove redundancy

Add systematic redundancy Better

EquivalentChannel

S

𝐒

Example: Repetition Code

70

Original Equivalent Channel:

BSC with crossover probability p = 0.01

New (and Better) Equivalent Channel:

Use repetition code with n = 5 at the transmitter Use majority vote at the receiver New BSC with

0

1

0

1

p

1-p

p

1-p

0

1

0

1

p

1-p

p

1-p

Repetition Code with n = 5

Majority Vote

0

1

0

1

𝑝 1053 𝑝 1 𝑝 5

4 𝑝 1 𝑝 55 𝑝 1 𝑝

Example: ASCII Encoder and BSC

71

Noise & In

terferen

ce

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

Source Decoder

Channel Decoder

DigitalDemodulator

Transmitter

Receiver


X: channel input

Y: channel output

“LOVE”

“1001100100111110101101000101”

0

1

0

1

p

1-p

p

1-p

The ASCII Coded Character Set

72[The ARRL Handbook for Radio Communications 2013]

0 16 32 48 64 80 96 112

Example: ASCII Encoder

73

Character Codeword

⋮E 1000101

⋮L 1001100

⋮O 1001111

⋮V 1010110

⋮

SourceEncoder

Information Source

“LOVE”“1001100100111110101101000101”

>> M = 'LOVE';>> X = dec2bin(M,7);>> X = reshape(X',1,numel(X))X =1001100100111110101101000101

MATLAB:

c(“L”) c(“O”) c(“V”) c(“E”)

System considered

74

Noise & In

terferen

ce

Information Source

Destination

Channel

ReceivedSignal

TransmittedSignal

Message

Recovered Message

Source Encoder

Channel Encoder

DigitalModulator

Source Decoder

Channel Decoder

DigitalDemodulator

Transmitter

Receiver


X: channel input

Y: channel output

0

1

0

1

p

1-p

p

1-p

THE WIZARD OF OZ (1900)written by L. Frank Baum

Introduction

Folklore, legends, myths and fairy tales have followed childhoodthrough the ages, for every healthy youngster has a wholesome andinstinctive love for stories fantastic, marvelous and manifestlyunreal. The winged fairies of Grimm and Andersen have brought morehappiness to childish hearts than all other human creations.Yet the old time fairy tale, having served for generations, may

101010010010001000101010000010101111001001101101010000011010010100010001000001001111100011001000001001111101101001000000100000010000001010000110001011100101100000110000010100100010100001001111011111100101101001111010011101001100101110111001000001100010111100101000001001100010111001000001000110111001011000011101110110101101000001000010110000111101011101101000101000010100001010100100111011101110100111001011011111100100111010111000111110100110100111011111101110000101000010100100000010000010001101101111110110011010111101100110111111100101100101010110001000001101100110010111001111100101110111011001001110011010110001000001101101111100111101001101000111001101000001100001110111011001000100000110011011000011101001111001011110010100000111010011000011101100110010111100110100000110100011000011110110110010101000001100110110111111011001101100110111111101111100101110010001000001100011110100011010011101100110010011010001101111110111111001000001010111010011010001110010110111111101011100111110100001000001110100110100011001010100000110000111001111100101111001101011000100000110011011011111110010010000011001011110110110010111100101111001010000011010001100101110000111011001110100110100011110010100000111100111011111110101110111011001111110011111010011001011110010010000011010001100001111001101000001100001010000011101111101000110111111011001100101111001111011111101101110010101000001100001110111011001000001010110100111011101110011111010011010011101110110001111101001101001111011011001010100000110110011011111110110110010101000001100110110111111100100100000111001111101001101111111001011010011100101111001101000001100110110000111011101110100110000111100111110100110100111000110101100010000011011011100001111001011101101100101110110011011111110101111001101000001100001110111011001000100000110110111000011101110110100111001101100101111001111101001101100111100100010101110101110111011100101100101110000111011000101110010000010101001101000110010101000001110111110100111011101100111110010111001000100000110011011000011101001111001011010011100101111001101000001101111110011001000001000111111001011010011101101110110101000001100001110111011001000100000100000111011101100100110010111100101110011110010111011100100000110100011000011110110110010101000001100010111001011011111110101110011111010001110100010000011011011101111111001011001010001010110100011000011110000111000011010011101110110010111100111110011010000011101001101111010000011000111101000110100111011001100100110100111100111101000010000011010001100101110000111100101110100111001101000001110100110100011000011101110010000011000011101100110110001000001101111111010011010001100101111001001000001101000111010111011011100001110111001000001100011111001011001011100001111010011010011101111110111011100110101110000101001000000100000101100111001011110100010000011101001101000110010101000001101111110110011001000100000111010011010011101101110010101000001100110110000111010011110010111100101000001110100110000111011001100101010110001000001101000110000111101101101001110111011001110100000111001111001011110010111011011001011100100010000011001101101111111001001000001100111110010111011101100101111001011000011110100110100111011111101110111001101011000100000110110111000011111001

𝑝 0.01

[ErrorProbabilityoverBSC.m]

Results

75

101010010010001000101010000010101111001001101101010000011010010100010001000001011111100011001000001001111101101001000000100000010000001010000110001011100101100000110000011100100110100001001111011111100101101001111010011101001100101110111001000001100010111100101000001001100010111001000001000110111001011000011101110110101101100001000010110000111101011101101000101000010100001010100100111011101110100111001011011111100100111010111000111110100110100111011111101110000101000010100100000011000010001101101111110110011010111101100110111111100101100101010111000000001101100110010111001111100101110111011001001010011000110001000001101101111100111101001101000111001101000001100001110111011001000100000110011011000011101001110001011110010100000111010011000011101100110011111100110100000110100011000011110110110010101000001100110110110111011001101100110111111101111100101110010001000001100011110100011010011101100110010011010001101111110111111001000001010111010011010001110010110111111101011100111110100001000001110100110100011001010100000110000111001111100101111001101011000000000110011011011111110010010010011001011110110110010111100101111001010000011011101100101110000111011001110100110100011110010100000111100111011111110101110111011001111110011111010011001011110010010000011010001100001111001101000001100001010000011101111101000110111111011001100101111001111011111101101110010101000001100001110111011001000001010110100111011001110011111000011110011101110110001111101001101001111001011001010100000110110011011111110110110010101000001100110110111111100100100000111001111101001101111111001011010011100101111000101000001100110110000101011101110100110000111100111110100110100111000110101100010000011011011100001111001011101101100101110110011011111110101011001101000001100101110111011001000100000110110111000011101110110100111001101100101111001111101001101100111100100010101110101110111011100101100101110000111011000101110010000010101001101000110010101000001110111110100111011101100111110010111001000100000110011011000011101001111001011010011100101111001101000001101111110011001000001000111111001011010011101101110110101000001100001110111011001000101010100000111011101100100110010111100101110011110010111011100100000110100011000011110110100010101000001100010111001011011111110101110011111010001110100010000011011011101111111001011001010001010110100011000011110000111000011010011101110110010111100111110011000000011101001101111010000011000111101000110100111010001100100110100111100111101000010000011010001100101110000111100101110100111001101000001110100110100011000011101110010000011000011101100110110001000001101111110010011010001110101111001001000001101000111010111011011100001110111001000001100011111001011001011100001111010011010011101111110111011100110101111000101001000000100000101100111001011110100010000011101001101000110010101100001101111110110011001000100010111010011010011101101110110101000001100110110000111010010110010111100101000001110100110000111011001100101010110001000001101000110000111101101101001110111011001110100000111000111001011110010111011011001011100100010000011001101101111111001001000001100111110010111011101100101111001011000011110100110100111011111101110111000101011000100000110110111000011111001

101010010010001000101010000010101111001001101101010000011010010100010001000001001111100011001000001001111101101001000000100000010000001010000110001011100101100000110000010100100010100001001111011111100101101001111010011101001100101110111001000001100010111100101000001001100010111001000001000110111001011000011101110110101101000001000010110000111101011101101000101000010100001010100100111011101110100111001011011111100100111010111000111110100110100111011111101110000101000010100100000010000010001101101111110110011010111101100110111111100101100101010110001000001101100110010111001111100101110111011001001110011010110001000001101101111100111101001101000111001101000001100001110111011001000100000110011011000011101001111001011110010100000111010011000011101100110010111100110100000110100011000011110110110010101000001100110110111111011001101100110111111101111100101110010001000001100011110100011010011101100110010011010001101111110111111001000001010111010011010001110010110111111101011100111110100001000001110100110100011001010100000110000111001111100101111001101011000100000110011011011111110010010000011001011110110110010111100101111001010000011010001100101110000111011001110100110100011110010100000111100111011111110101110111011001111110011111010011001011110010010000011010001100001111001101000001100001010000011101111101000110111111011001100101111001111011111101101110010101000001100001110111011001000001010110100111011101110011111010011010011101110110001111101001101001111011011001010100000110110011011111110110110010101000001100110110111111100100100000111001111101001101111111001011010011100101111001101000001100110110000111011101110100110000111100111110100110100111000110101100010000011011011100001111001011101101100101110110011011111110101111001101000001100001110111011001000100000110110111000011101110110100111001101100101111001111101001101100111100100010101110101110111011100101100101110000111011000101110010000010101001101000110010101000001110111110100111011101100111110010111001000100000110011011000011101001111001011010011100101111001101000001101111110011001000001000111111001011010011101101110110101000001100001110111011001000100000100000111011101100100110010111100101110011110010111011100100000110100011000011110110110010101000001100010111001011011111110101110011111010001110100010000011011011101111111001011001010001010110100011000011110000111000011010011101110110010111100111110011010000011101001101111010000011000111101000110100111011001100100110100111100111101000010000011010001100101110000111100101110100111001101000001110100110100011000011101110010000011000011101100110110001000001101111111010011010001100101111001001000001101000111010111011011100001110111001000001100011111001011001011100001111010011010011101111110111011100110101110000101001000000100000101100111001011110100010000011101001101000110010101000001101111110110011001000100000111010011010011101101110010101000001100110110000111010011110010111100101000001110100110000111011001100101010110001000001101000110000111101101101001110111011001110100000111001111001011110010111011011001011100100010000011001101101111111001001000001100111110010111011101100101111001011000011110100110100111011111101110111001101011000100000110110111000011111001


Introduction


THE WIZARD _F OZ (19009 written by L. Frank0Baum

Introduction

0Folklore. legendS myths and faiby talgs have fmllowed childhoodthrough the ages, for$every nealthy youngster has a wholesome andilspynctire love for storieq fa.tastic, marvelou3 end manifestlyunreal. The winged fairies of Grimm and*Andersen havE brought morehappiness to chihdish hearts than all odhur human creations/Yet the0old"timm fai2y tale, having qerved for generationq, may

Results

76


Introduction


THE WIZARD _F OZ (19009 written by L. Frank0Baum

Introduction

0Folklore. legendS myths and faiby talgs have fmllowed childhoodthrough the ages, for$every nealthy youngster has a wholesome andilspynctire love for storieq fa.tastic, marvelou3 end manifestlyunreal. The winged fairies of Grimm and*Andersen havE brought morehappiness to chihdish hearts than all odhur human creations/Yet the0old"timm fai2y tale, having qerved for generationq, may

The whole book which is saved in the file “OZ.txt” has 207760 characters (symbols).

The ASCII encoded string has 207760×7 = 1454320 bits.

The channel corrupts 14545 bits.

This corresponds to 14108 erroneous characters.

Results

77

The file “OZ.txt” has 207760 characters (symbols).



This corresponds to 14108 erroneous characters.

>> ErrorProbabilityoverBSCbiterror =

14545BER =

0.010001237691842theoretical_BER =

0.010000000000000characterErrror =

14108CER =

0.067905275317674theoretical_CER =

0.067934652093010

Results

78

The file “OZ.txt” has 207760 characters (symbols).



This corresponds to 14108 erroneous characters (symbols).

CER A character (symbol) is successfully recovered if and only if none of its bits are corrupted.

BSC’s crossover probability

Crossover probability and readability

79

When the first novel of the series, Harry Potter and the Philosopher's Stone (published in some countries as Harry Potter and the Sorcerer's Stone), opens, it is apparent that some significant event has taken place in the wizarding world--an event so very remarkable, even the Muggles notice signs of it. The full background to this event and to the person of Harry Potter is only revealed gradually through the series. After the introductory chapter, the book leaps forward to a time shortly before Harry Potter's eleventh birthday, and it is at this point that his magical background begins to be revealed.

When the first novel of the series, Harry Pottez and the Philosopher's Stone (p5blished in some countries as Harry Potter cnd the Sorcerep's Stone), opens, it i3 apparent that soMecignifacant event!haS taken0place in the wi~arding 7orld--ao event so`very!bemark!blu, even the Mufgles nodice signs"of it. The fuld background to this event and to the person of Harry P/tTer is only revealed gradually through th series. After the introfuctory chapter, the boo+ leaps forward to a time shortly before Harpy Potteb7s eleventh`birthday, and )t is at this poi~t that his -agikal bac{ground begins to be revealed.

𝑝 0.01 CER 0.07

Original


80

When the first novel of the series, Harry Pottez and the Philosopher's Stone (p5blished in some countries as Harry Potter cnd the Sorcerep's Stone), opens, it i3 apparent that soMecignifacant event!haS taken0place in the wi~arding 7orld--ao event so`very!bemark!blu, even the Mufgles nodice signs"of it. The fuld background to this event and to the person of Harry P/tTer is only revealed gradually through th series. After the introfuctory chapter, the boo+ leaps forward to a time shortly before Harpy Potteb7s eleventh`birthday, and )t is at this poi~t that his -agikal bac{ground begins to be revealed.

𝑝 0.01 CER 0.07

Human may be able to correct some (or even all) of these errors.


81

w(en th% birst .ovo,òf the serieq, Hcrry Potter(and the Phidosopèr's Suone (xub|ishe$(in some!Countries as @arry Potter and tè Snr#erer's S|ong)- opens, it is apparent thatsoeesmgfificant erent ha3 taieN place in the wizardino world-,an event!so very remarkable, even thEEugglds notiae qigns of it. Tledfull back'tound to this event ane to the perron of Harry Popter is onl{ reveqned gsadeally thro}gH th%$serias. After the int2oducpory chcptur, the0jook deapsforward to"a!tmme shmrtly befosE Harry"Potter's eleventh jirthdiy cnd ht is a| thi3 po{nt tHat@is mAgiial background begijs to rm rerealed.

Whethe first nOwgl nf the susi-Q-@Harr} PoutEr(and |he PhilosoxhEr's Ctonepuclyshed in som% coultries as Harrx @ tter and the S_rcermr7s Spone), opdns, id is apparent that {omg`signifikant evmnt ias taKen!Placa in tHe 7ijardIng world--an event so Very remaroqble, eve.!thE MugglaC fotice signc of"it. Uhe full backf2ound`to thas even| ant`0o the pEssoj of @arry Qotteb iw only revealed gradu!lly vhvoug` the rerier. Afte2 the IndRoductori chaptar,tèbook leats ForwaRf tc a 4imE shostl= before!Hcssy potter's u|Eveoth$firthdA{, and iT is ad this pomNt uhav ìr magica, back'bound cegins to bE 2evealed.

𝑝 0.02 CER 0.13

𝑝 0.03 CER 0.19


82

When phe fir v okval ov"th% serie3, @`rry0Pntter efdxtxeThil soph%rs Stone0(p}blisjed!in{ooe c un|pye{ agav0y Potter aj`(the sorcerer"s S4o|e)< opdns- mt"Is!apParEnt 4hat somusiwnidiga.v evant iAs take."plhge in(uhe w)zard)ng wo{|d--An event so very Rumar{ableleteN0Dhe %ugcles$n t)ce signs of$At. Tje!&ul|!backep/und Dk thkwèvent ajt(to vhd per{On of8Ikxry P_Pter is oN,y rereAeud gredualli 4hroufh5ie qeriesn Af|ir the )~trofUctkry!ciapter,$tler%ok lE`ps for erd8to!a d)hg 3Hostly redobd HArry(Potter/r elaventI(birpl%ay,))nd(iD i3 1t tlishohlt vhat iis$iagical bac+gropnd bedans to bg rEve!ied/

Whef th% &i2sv nkvdl"On(txE"serm-s< HaRtY Qo|p%R$anlthe Phi$)qop8gb'r YtoNe (puclirhedin23/ee c uNpr9es aZ Harby!PovDdZ qnd0THA!Uorojev's Qpof'), pegsL iT is0aqazenP Tiet`{nlesau*!fICQ~t eve.t`xA# raken pOqb%%)D }Hm`wizprdYjv"wOrnd--a~%W%Jv s' tury2maskABdd$èden(tl| LuxGxec`nOtike c)gzq of ktTiu!f5mm"cackG@ Ud(to"vhhQ a~aNd alt tn0vid veRckn of HaRvq$Xntter#isxohk{ regea,ed@&saduadLy u(2otGh"tau griEs."AfTex0T`g mntr DUCt ry kh `ter,$thd(fomN0jàpv ngrwarTt-0c t,me"1xortly bEemsL |ar2q Pnfter'3 aMen-n5i@Fipth$`q, aoh It i3d1t piac0pmhnP d*if Zas mafibin"je#k7poUndpb%dins tk`be qe6e!lgd.

𝑝 0.05 CER 0.30

𝑝 0.10 CER 0.52

BER vs. CER

830 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Bit Error Rate (BER)

Cha

ract

er E

rror r

ate

(CE

R)

[BERCER.m]

BER vs. CER

84 10-2 10-1 10010-2

10-1

100

Bit Error Rate (BER)

Cha

ract

er E

rror r

ate

(CE

R)

[BERCER.m]

digital communication systems 452 - 3 - discrete...1 digital communication systems ecs 452 asst....

Documents