decision rules for real channels

8/2/2019 Decision Rules for Real Channels

1/22

Chapter 2

Decision Rules for Real Channels

SUMMARY: Here we will first consider transmission of information over a channel witha single real-valued input and output. Again the optimum receiver is determined. As an

example we investigate the additive Gaussian noise channel, i.e. the channel that adds

Gaussian noise to the input signal.

Then we discuss the channel whose input and output are vectors of real-valued compo-

nents. An important example of such a vector channel is the additive Gaussian noise (AGN)

vector channel. This channel adds zero-mean Gaussian noise to each input signal compo-

nent. All noise samples are assumed to be independent of each other and have the same

variance. For the AGN vector channel we determine the optimum receiver. If all messages

are equally likely this receiver chooses the input vector that is closest to the received vector

in Euclidean sense. For the error probability in this case we derive an upper bound basedon the union bound.

We also investigate the multi-vector channel here, i.e. a channel with more than one out-

put vector. More precisely we discuss a sufficient condition under which one of the output

vectors is irrelevant and can be discarded without making the average error probability

larger.

2.1 The Q-function

To compute the error probabilities the so-called Q-function (see figure 2.1) is very useful.

Definition 2.1 (Q-function) This function of x (, ) is defined as

Q(x)=

x

12

exp(2

2)d. (2.1)

It is the probability that a Gaussian random variable with mean 0 and variance1 assumes a value

larger than x .

A useful property of the Q-function is

Q(x ) + Q(x) = 1. (2.2)

15


2/22

16 CHAPTER 2. DECISION RULES FOR REAL CHANNELS

4 3 2 1 0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Figure 2.1: Gaussian probability density function for mean 0 and variance 1. The shaded area

corresponds to Q(1).

In table 2.1 we tabulated Q(x) for several values of x .

x Q(x) x Q(x) x Q(x)

0.00 0.50000 1.5 6.6681 102 5.0 2.8665 1070.25 0.40129 2.0 2.2750 102 6.0 9.8659 10100.50 0.30854 2.5 6.2097 103 7.0 1.2798 10120.75 0.22663 3.0 1.3499

103 8.0 6.2210

1016

1.00 0.15866 4.0 3.1671 105 10.0 7.6200 1024

Table 2.1: Table of Q(x) for some values of x .

2.2 Problem Description

In this chapter we are interested in finding the optimum receiver when the channel adds Gaussian

noise to the input signal, which can be a scalar signal or a vector. We also want to find an

expression (or an upper bound) for the error probability for such situations.

2.3 A System Based on a Real Scalar Channel

A communication system based on a channel with real-valued input and output alphabet, i.e. a

real scalar channel does not differ very much from a system based on a discrete channel (see

figure 2.2).

SOURCE An information source produces the message m M = {1, 2, , |M|} with a-priori probability Pr{M = m} for m M. Again M is the name of the random variable


3/22

2.4. MAP AND ML DECISION RULES 17

EEEE realsm

source transmitter receiver destin.

m mr

channel

Figure 2.2: Elements in a communication system based on a real scalar channel.

associated with this mechanism.

TRANSMITTER The transmitter now sends a real-valued scalar signal sm if message m is

to be transmitted. The scalar signal is input to the channel. It assumes a value in the

range (, ). The random variable corresponding to the signal is denoted by S. Thecollection of used signals is s1, s2,

, s

|M

|.

REAL SCALAR CHANNEL The channel now produces an output r in the range (, ).When the input signal is the real-valued scalar s the channel output is generated according

to the conditional probability density function pR(r|S = s) and thus R is a real-valuedscalar random variable. The probability of receiving a channel output r R < r + dr isequal to pR (r|S = s)dr for an infinitely small interval dr.When message m M occurs, signal sm is chosen by the transmitter as channel input.Then the conditional probability density function

pR(r|M = m) = pR (r|S = sm ) for all r (, ), (2.3)

describes the behavior of the transmitter followed by the real scalar channel.

RECEIVER The receiver forms an estimate m of the transmitted message (or signal) based on

the received real-valued scalar channel output r, hence m = f(r). The mapping f() isagain called the decision rule.

DESTINATION The destination accepts the estimate m.

2.4 MAP and ML Decision Rules

For the real scalar channel we can write the probability of correct decision as

PC =

Pr{M = f(r)}pR (r|M = f(r))dr. (2.4)

An optimum decision rule is obtained if, after receiving the scalar R = r, the decision f(r) istaken in such a way that

Pr{M = f(r)}pR (r|M = f(r)) Pr{M = m}pR (r|M = m) for all m M. (2.5)This leads to the definition of the decision variables given below.


4/22


Alternatively we can assume that a value r R < r + dr was received. Then the decisionvariables would be

Pr{M = m, r R < r + dr} = Pr{M = m} Pr{r R < r + dr|S = sm}= Pr{M = m}pR (r|S = sm )dr. (2.6)

Also this reasoning leads to the definition of the decision variables below.

Definition 2.2 For a system based on a real scalar channel the products

Pr{M = m}pR(r|M = m) = Pr{M = m}pR (r|S = sm ) (2.7)which are, given the received outputr, indexed bym M, are called thedecision variables.The optimum decision rule f(

) is again based on these variables. It is possible that for certain

channel outputs r more decisions f(r) are optimum.

RESULT 2.1 (MAP) To minimize the error probability PE, the decision rule f() should besuch that for each received r a message m is chosen with the largest decision variable. Hence

for r that can be received, an optimum decision rule f() should satisfyf(r) = arg max

mMPr{M = m}pR(r|S = sm). (2.8)

Both sides of the inequality (2.5) can be divided by pR(r) =

mM Pr{M = m}pR (r|S = sm ),i.e. de density for the value of r that actually did occur. Then we obtain an equivalent formulation

of this optimum decision rule

f(r) = arg maxmM

Pr{M = m|R = r}, (2.9)

for r for which pR(r) > 0. Therefore this rule is again called maximum a-posteriori (MAP)

decision rule.

RESULT 2.2 (ML) When all messages have equal a-priori probabilities. i.e. when Pr{M =m} = 1/|M| for all m M, we observe from (2.8), that the optimum receiver has to choose

f(r) = arg maxmM

pR(r|S = sm ), (2.10)

for all r with pR(r) > 0. This rule is referred to as the maximum likelihood (ML) decision rule.

2.5 The Scalar Additive Gaussian Noise Channel

2.5.1 Introduction

Gaussian noise is probably the most important kind of impairment. Therefore we will investigate

a simple communication situation based on a channel that adds a Gaussian noise sample n to the

real scalar channel input signal s (see figure 2.3).


5/22

2.5. THE SCALAR ADDITIVE GAUSSIAN NOISE CHANNEL 19

E Ec

n

+'&$%s r = s + n

pN(n) = 12 2

exp( n222

)

Figure 2.3: Scalar additive Gaussian noise channel.

Definition 2.3 Thescalar additive Gaussian noise (AGN) channel adds Gaussian noise N to

the input signal S. This Gaussian noise N has variance2 and mean 0. The probability density

function of the noise is defined to be

pN(n)

=

1

2 2

exp(

n2

22

). (2.11)

The noise variable N is assumed to be independent of the signal S.

2.5.2 The MAP decision rule

We assume e.g. that |M| = 2, i.e. there are two messages and M can be either 1 or 2. Thecorresponding two signals s1 and s2 are the inputs of our channel. Without loss of generality let

s1 > s2.

The conditional probability density function of receiving R = r given the message m dependsonly on the value n that the noise variable N gets. In order to receive R = r when the signal iss

m, the noise variable N should have value r

s

m. The noise N is independent of the signal S

(and the message M). Therefore (assuming that dr is infinitely small)

pR (r|S = sm ) = Pr{r R < r+ dr|S = sm}/dr= Pr{r sm N < r sm + dr}/dr= pN(r sm )= 1

2 2exp((r sm )

2

22), for m = 1, 2. (2.12)

We obtain an optimum (maximum a-posteriori) receiver when m = 1 is chosen if

Pr{M

=1}

1

2 2exp(

(r

s1)

2

22)

Pr{M

=2}

1

2 2exp(

(r

s2)

2

22), (2.13)

and m = 2 otherwise. The following inequalities are all equivalent to (2.13):

lnPr{M = 1} (r s1)2

22 lnPr{M = 2} (r s2)

2

22,

22 lnPr{M = 1} (r s1)2 22 lnPr{M = 2} (r s2)2,22 lnPr{M = 1} + 2r s1 s21 22 lnPr{M = 2} + 2r s2 s22 ,

2r s1 2r s2 22 lnPr{M = 2}Pr{M = 1} + s

21 s22 . (2.14)


6/22


RESULT 2.3 (Optimum receiver for the scalar additive Gaussian noise channel) A receiver

that decides m = 1 if

r r

=2

s1 s2 lnPr

{M

=2}Pr{M = 1} +

s1+

s2

2 , (2.15)

andm = 2 otherwise, is optimum. When the a-priori probabilities Pr{M = 1} andPr{M = 2}are equal the optimum threshold is

r = s1 + s22

. (2.16)

5 4 3 2 1 0 1 2 3 4 50

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

Figure 2.4: Decision variables for equal a-priori probabilities as a function of r.

5 4 3 2 1 0 1 2 3 4 50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Figure 2.5: Decision variables as a function ofr for a-priori probabilities 1/4 and 3/4.

Example 2.1 Assume that 2 = 1 and s1 = +1 and s2 = 1. If the a-priori message probabilities areequal, i.e. when Pr{M = 1} = Pr{M = 2}, we obtain an optimum receiver if only for r r = s1+s2

2we


7/22

2.5. THE SCALAR ADDITIVE GAUSSIAN NOISE CHANNEL 21

decide for m = 1. The decision changes exactly halfway between s1 and s2. The intervals (, r) and(r, ) are called decision intervals. In our case, since s2 = s1, the threshold r = 0. This also can beseen from figure 2.4 where the decision variables

Pr{M = 1} 12 2

exp( (r s1)2

22), Pr{M = 2} 1

2 2exp( (r s2)

2

22) (2.17)

are plotted as a function ofr assuming that Pr{M = 1} = Pr{M = 2} = 1/2.Next assume that the a-priori probabilities are not equal but let Pr{M = 1} = 3/4 and Pr{M = 2} =

1/4. Now the decision variables change (see figure 2.5) and we must also change the decision rule. It

turns out that for r r = ln 32

0.5493 the optimum receiver should choose m = 1 (see again figure2.5). The threshold r has moved away from the more probable signal s1.

Note that, no matter how much the a-priori probabilities differ, there is always a value of r, which we

have called r, the threshold, before, for which (2.13) is satisfied with equality. For r > r the left side in

(2.13) is larger than the right side, for r < r the right side is the largest.

2.5.3 Probability of error

We now want to find an expression for the error probability of our scalar system with two mes-

sages. We write

PE = Pr{M = 1} Pr{R < r|M = 1} + Pr{M = 2} Pr{R r|M = 2}, (2.18)

where

Pr{R r|M = 2} =

r=r1

2 2exp((r s2)2

22)dr

=

r=r12

exp(((r/ ) s2/ )2

2)d(r/ )

=

=r/s2/

12

exp(2

2)d = Q(r/ s2/), (2.19)

and similarly

Pr{R < r|M = 1} = r=r

1

2 2 exp((r

s1)

2

22 )dr

=r=r

12

exp(((r/ ) s1/ )2

2)d(r/ )

==r/s1/

12

exp(2

2)d

= 1 Q(r/ s1/ ) ()= Q(s1/ r/). (2.20)

Note that the last equality (*) follows from (2.2).


8/22


RESULT 2.4 (Minimum probability of error) If we combine (2.18), (2.19), and (2.20), we ob-

tain

PE = Pr{M = 1}Q(s1

r

) + Pr{M = 2}Q(r

s2

). (2.21)

If the a-priori message probabilities Pr{M = 1} and Pr{M = 2} are equal then, according to(2.16), we get r = s1+s2

2and

s1 r

= s1 s1+s2

2

= s1 s2

2

r s2

=s1+s2

2 s2

= s1 s2

2(2.22)

hence for the minimum probability of error we obtain

PE = Q(s1 s2

2). (2.23)

Example 2.2 For s1 = +1, s2 = 1, and 2 = 1, we obtain if Pr{M = 1} = Pr{M = 2} that

PE = Q(1) 0.1587. (2.24)

For Pr{M = 1} = 3/4 and Pr{M = 2} = 1/4 we get that r = ln 32

0.5493 and

PE =3

4Q(1 + ln 3

2) + 1

4Q( ln 3

2+ 1)

0.75Q(1.5493) + 0.25Q(0.4507) 0.75 0.0607 + 0.25 0.3261 0.1270. (2.25)

Note that this is smaller than Q(1) 0.1587 which would be the error probability after ML-detection(which is suboptimum here).

2.6 Systems Based on Real Vector Channels

E

EE

E E

EE E

rN

r2

r1

sm N

sm2

sm1

......

vectorchannel

sm

source transmitter receiver destin.

mm

r

Figure 2.6: A communication system based on a vector channel.

In a communication system based on a vector channel (see figure 2.6) the channel input and

output are vectors with real-valued components.


9/22

2.7. DECISION VARIABLES, MAP DECODING 23

SOURCE The information source generates the message m M = {1, 2, , |M|} with a-priori probability Pr{M = m} for m M. M is the random variable associated with thismechanism.

TRANSMITTER The transmitter sends a vector-signal sm = (sm1, sm2, , sm N) consistingof N components if message m is to be transmitted. Each component assumes a value in

the range (, ). The random variable corresponding to the signal vector is denoted byS. The vector-signal is input of the channel. The set of used signals is {s1, s2, , s|M|}.

VECTOR CHANNEL The channel produces an output vector r = (r1, r2, , rN) consist-ing of N components. We assume here that all these components assume values from

(, ). The conditional probability density function of the channel output r when themessage M

=m is pR (r

|M

=m)

=pR(r

|S

=sm). Thus, when M

=m, the prob-

ability of receiving an N-dimensional channel output vector R with components Rn for

n = 1, N satisfying rn Rn < rn + drn is equal to pR(r|M = m)dr for an infinitelysmall dr = dr1dr2 drN.

RECEIVER The receiver forms m based on the received real-valued channel output vector r,

hence m = f(r). Mapping f() is the decision rule.

DESTINATION The destination accepts m.

2.7 Decision Variables, MAP Decoding

The optimum receiver upon receiving r determines which one of the possible messages has

maximum a-posteriori probability. It therefore chooses the decision rule f(r) such that for all r

that actually can be received

Pr{M = f(r)}pR (r|M = f(r)) Pr{M = m}pR (r|M = m), for all m M. (2.26)

In other words, upon receiving r, the optimum receiver produces an estimate m that corresponds

to a largest decision variable. The decision variables for the vector channel are

Pr{M = m}pR (r|M = m) for all m M. (2.27)

That this is MAP-decoding follows from Bayes rule which says that

Pr{M = m|R = r} = Pr{M = m}pR(r|M = m)pR (r)

(2.28)

for r with pR (r) > 0. Note that pR(r) > 0 for an output vector r that has actually been received.


10/22


2.8 Decision Regions

The optimum receiver, upon receiving r, determines the maximum of all decision variables which

are given in (2.27). To compute these decision variables the a-priori probabilities Pr{M = m}must be known to the receiver and also the conditional density functions pR (r|M = m) for allm M. This calculation can be carried out for every r in the observation space, i.e. the set ofall possible output vectors r. Each point r in the observation space is therefore assigned to one

of the possible messages m M. This results in a partitioning of the observation space in (atmost) |M| regions.

Definition 2.4 Given the decision rule f() we can write

Im= {r|f(r) = m}. (2.29)

where Im is called thedecision region that corresponds to messagem M.Note that in the example in subsection 2.5.2 of the previous chapter we considered decision

intervals. A decision region is an N-dimensional generalization of a (one-dimensional) decision

interval.

s3

e

e

dd

de

ee

dd

d

I1

I3

I2s

1

s2e

Figure 2.7: Three two-dimensional signal vectors and their decision regions.

Example 2.3 Suppose (see figure 2.7) that we have three messages i.e. M = {1, 2, 3}, and the corre-sponding signal-vectors are two-dimensional i.e. s1 = (1, 2), s2 = (2, 1), and s3 = (1, 3). A possiblepartitioning of the observation space into the three decision regions I1, I2, and I3 is shown in the figure.


11/22

2.9. ADDITIVE GAUSSIAN NOISE 25

s EE c r = s + n

n pN(n) = 1(2 2)N/2 exp(n222

)

+'&$%Figure 2.8: Additive Gaussian noise vector channel.

2.9 Additive Gaussian Noise

The actual shape of the decision regions is determined by the a-priori message probabilities

Pr{M = m}, the signals sm , and the conditional density functions pR(r|S = sm ) for all m M. A relatively simple but again quite important situation is the case where the channel adds

Gaussian noise to the signal components (see figure 2.8).

Definition 2.5 For the output of the additive Gaussian noise (AGN) vector channel we have

that

r = s + n, (2.30)

where n= (n1, n2, , nN) is an N-dimensional noise vector. We denote this random noise

vector by N. The noise vector N is assumed to be independent of the signal vector S. The N

components of the noise vector are also assumed to be independent of each other. Moreover all

noise components have mean 0 and the same variance 2. Therefore the joint density function

of the noise vector is given by

pN(n) =

i=1,N

12 2

exp( n2i

22) = 1

(2 2)N/2exp( 1

22

i=1,N

n2i ). (2.31)

This notation in the definition can be contracted by noting that

i=1,N

n2i = (n n) = n2, (2.32)

where (a b) = i=1,N ai bi is the dot product of the vectors a = (a1, a2, , aN) and b =(b1, b2, , bN). Thus

pN(n) =1

(2 2)N/2exp(n

2

22). (2.33)

If the channel output is r and its input was sm then the noise vector must have been n = r sm .This and the independence of the noise vector and the signal vector yields that

pR (r|M = m) = pR (r|S = sm ) = pN(r sm |S = sm ) = pN(r sm ). (2.34)


12/22


s3

e

e

dd

d

I1

I3

I2s1

s2e

Figure 2.9: Three two-dimensional signal vectors and the corresponding optimum decision re-

gions for the additive Gaussian noise channel.

Now we can easily determine the decision variables, one for each m M:Pr{M = m}pR (r|M = m) = Pr{M = m}pN(r sm )

= Pr{M = m} 1(2 2)N/2

exp(r sm2

22). (2.35)

Note that the factor (2 2)N/2 is independent of m. Hence maximizing over the decision vari-

ables in (2.35) is equivalent to minimizing

r sm2 22 lnPr{M = m} (2.36)over all m M. We get a very simple decision rule if all messages are equally likely.RESULT 2.5 (Minimum Euclidean distance decision rule) If the a-priori message probabili-

ties are all equal, the optimum receiver has to minimize the squared Euclidean distance

r sm2, for all m M. (2.37)In other words the receiver has to choose the message m with corresponding signal vector s mthat is closest in Euclidean sense to the received vector r.

Example 2.4 Consider again (see figure 2.9) three two dimensional signal vectors s1 = (1, 2), s2 =(2, 1), and s3 = (1, 3). The optimum decision regions can be found by drawing the perpendicularbisectors of the sides of the signal triangle. These are the boundaries of the decision regions I1, I2, and I3(see figure). Note that the three bisectors have a single point in common.


13/22

2.9. ADDITIVE GAUSSIAN NOISE 27

(a, b)

T

E

0

k

eee

ee

ee

eeu

)

0

k

0

u2

I

line

r2

r1

(r1, r2)

u1

Figure 2.10: A signal point and a line.

2.9.1 Error probabilities

The error probability of an additive Gaussian noise vector channel is determined by the locationof the signals s1, s2, , s|M|, and, more importantly, the hyperplanes that separate these signal-vectors. An error occurs if the noise pushes a signal-point to the wrong side of a hyperplane. In

general it is quite difficult to determine the exact error probability, however it is easy to compute

the probability that the received signal is on the wrong side of a hyperplane.

To study this behavior we investigate a simple example in two dimensions. i.e. N = 2.Consider a signal vector s = (a, b), see figure 2.10. This signal vector is corrupted by theadditive Gaussian noise vector n = (n1, n2) with independent components that both have meanzero and variance 2. What is now the probability PI that the received vector r = (r1, r2) =(a, b) + (n1, n2) is in region I, the region above the line, see the figure?

To solve this problem we have to change the coordinate system. Let

r1 = a + u1 cos u2 sin (2.38)r2 = b + u1 sin + u2 cos , (2.39)

where (a, b) is the center of a Cartesian system with coordinates u1 and u2, and the rotation-

angle. Coordinate u1 is perpendicular to and coordinate u2 is parallel to the line in the figure.

Note that

PI =I

1

2 2exp

((r1 a)

2 + (r2 b)222

)dr1dr2. (2.40)


14/22


Elementary calculus (see e.g. [26], p. 386, theorem 7.7.13) tells us that

I

f(r1, r2)dr1dr2= I f(r1(u1, u2), r2(u1, u2))|J|du1du2, (2.41)

where |J| is the determinant of the Jacobian J which is

J =

r1u1

r1u2

r2u1

r2u2

=(

cos sin sin cos

). (2.42)

Note that |J| = cos2 + sin2 = 1 and (r1 a)2 + (r2 b)2 = u21 + u22. Therefore

PI = I1

2 2exp

u

21 + u2222

du1du2

=

u1=

u2=

1

2 2exp

u

21 + u2222

du1du2

=

u1=

12 2

exp

u

21

22

du1

u2=

12 2

exp

u

22

22

du2

= Q(

) 1 = Q(

). (2.43)

Conclusion is that the probability depends only on the distance from the signal point (a, b) to

the line. This result carries over to more than two dimensions:

RESULT 2.6 For the additive Gaussian noise vector channel, the probability that the noise

pushes a signal to the wrong side of a hyperplane is

PI = Q(

), (2.44)

where is the distance from the signal-point to the hyperplane and 2 is the variance of each

noise component. All noise components are assumed to be zero-mean.

2.9.2 Upper bound on the error probability

In general, in the Gaussian case, the probability of error PE of an optimum receiver cannot be

computed easily. However we can use the union bound to obtain an upper bound for it. To see

how this works we assume that the a priori message probabilities are all equal. Then for the error

probability P1E

when message 1 is sent, we can write

P1E = Pr{

mM,m= 1(R sm R s1)|M = 1}

mM,m= 1Pr{R sm R s1|M = 1}. (2.45)


15/22

2.10. SYSTEMS BASED ON REAL MULTI-VECTOR CHANNELS 29

Note that in the last step we used the union bound Pr{iEi}

i Pr{Ei }, where Ei is an eventindexed by i .

Now the total error probability can upper bounded by

PE

mMPr{M = m}Pm

E

mM

1

|M|

mM,m = mPr{R sm R sm|M = m}. (2.46)

Also when the a priori probabilities are not all equal we can use the union bound to upper bound

the error probability.

2.10 Systems Based on Real Multi-Vector Channels2.10.1 System description

The output of a multi-vector channel consists of a pair (r1, r2) of vectors, often but not nec-

essarily of the same dimension as the input vector s. E.g. when the input vectors sm =

EEEEEEE

EEE

E destin.receiverchannel

multi-vectortransmittersource

sm m

r2

r1m

Figure 2.11: A multi-vector communication system.

(sm1, sm2, , sm N) are N-dimensional, the output vectors may be K- and L-dimensional, i.e.r1 = (r11, r12, , r1K) and r2 = (r21, r22, , r2L ). We sometimes call our system one withdiversity. The decision variables for a multi-vector channel are

Pr{M = m}pR1,R2 (r1, r2|S = sm ) for all m M, (2.47)and the optimum receiver chooses an m that maximizes (2.47).

2.10.2 Theorem of irrelevance

An important question is whether for a multi-vector channel one of the output vectors, say r2,

may be disregarded by the receiver without affecting the average error probability PE. We then

call the output r2 irrelevant.

To investigate this problem we rewrite the decision variables (2.47) as follows

Pr{M = m}pR1 (r1|S = sm )pR2 (r2|S = sm , R1 = r1) for all m M. (2.48)If the factor pR2 (r2|S = sm , R1 = r1) does not depend on the message m, it can be ignored.

From this observation we immediately get the following result.


16/22


THEOREM 2.7 (Theorem of irrelevance) The output r2 of a multi-vector channel is irrele-

vant if, for all r1 and r2 the value of pR2 (r2|S = sm , R1 = r1) does not depend on the messagem anymore.

To understand this theorem we study three examples. These examples all involve two additive

noise signals n1 and n2 that are independent of each other and the signal vector s.

E

EEE

Ec

+receiver

m

m '

&

$

%

transmitter

r1 = sm + n1sm

n1

r2 = n2Figure 2.12: The vector r2 is irrelevant.

Example 2.5 In the first example in figure 2.12 clearly

pR2 (r2|S = sm , R1 = r1) = pN2 (r2) (2.49)

which does not depend on m. Therefore r2 is irrelevant.

EEE

T

cE

E E

n1

+

m

receiver

m

+'

&

$

%

transmitter

'&$%

r1 = sm + n1sm

n2r2 = r1 + n2

Figure 2.13: The vector r2 is irrelevant.

Example 2.6 In the second example in figure 2.13 we get that

pR2 (r2|S = sm, R1 = r1) = pN2 (r2 r1). (2.50)

Since this does not depend on m again r2 is irrelevant.


17/22

2.10. SYSTEMS BASED ON REAL MULTI-VECTOR CHANNELS 31

E E

E

Tc

EEE

m

receiver

+r2 = n1 + n2

m

n2

n1

sm'

&

$

%

r1 = sm + n1

'&$%

transmitter +

Figure 2.14: Now the vector r2 need not be irrelevant.

2 1.5 1 0.5 0 0.5 1 1.5 22

1.5

1

0.5

0

0.5

1

1.5

2

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Figure 2.15: Contour plot of maxm Pr{M = m}pR1,R2 (r1, r2|S = sm}.

Example 2.7 In the last example in figure 2.14 clearly

pR2 (r2|S = sm, R1 = r1) = pN2 (r2 r1 + sm) (2.51)

and therefore r2 need not be irrelevant.

We will now analyse a simple but yet more specific case to see that r2 in general is not irrelevant in

our last example. Assume that all the vectors are actually one-dimensional. There are two messages eachhaving a-priori probability 1/2. The corresponding signals are s1 = +1 and s2 = 1. For the noisevariables, that are assumed to be Gaussian, the variances are 2n1 = 2n2 = 1. The means of the noisevariables are both 0. Now we can consider for 2 r1, r2 2 the decision variables

Pr{M = m}pR1,R2 (r1, r2|S = sm} for m = 1, 2. (2.52)

In figure 2.15 a contour plot can be found that shows for each coordinate pair (r1, r2) the maxm Pr{M =m}pR1,R2 (r1, r2|S = sm}. Simple calculus shows that the optimum boundary of the decision regions isgiven by the straight line r2 = 2r1 (see also exercise 7 of this chapter). Hence this boundary is not avertical line and r2 is not irrelevant.


18/22


Note that the condition in the theorem of irrelevance is sufficient but not always neces-

sary. One can think of situations where the channel output r2 is irrelevant but still pR2 (r2|S =s

m

, R1 =

r1

) does depend on m.

2.10.3 Theorem of reversibility

EE E

E

channel

r2

r1s r2G

Figure 2.16: IfG is reversible then r2 is irrelevant.

An important consequence of the theorem of irrelevance is the following result.

THEOREM 2.8 (Theorem of reversibility) The minimum attainable probability of error is not

affected by the introduction of a reversible operation at the output of a channel.

Proof:

E E Er1 r2 m

G1 receiverfor r2

optimum

Figure 2.17: An optimum receiver for r1

It can be easily seen (see figure 2.16) that

pR2 (r2|S = sm , R1 = r1) = pR2 (r2|R1 = r1) (2.53)which is independent of m since r2 = G1(r1) and therefore r2 is irrelevant. Thus an optimumdecision can be made from r1 only.

An alternative proof of the theorem of reversibility follows from observing that a receiver

for r1 can be built by first recovering r2 from r1 (see figure 2.17). This is possible since the

mapping G from r2 to r1 is reversible. Then an optimum receiver for r2 is used to determine

m. The receiver constructed in this way for r1 is optimum, thus a reversible operation does not

(necessarily) lead to an increase of PE. 2


19/22

2.11. EXERCISES 33

0

d

dddd

rr

pR (r|M = 1) pR (r|M = 2)

1 2 -1 1 2 30

1 1

1/4

Figure 2.18: Conditional probability density functions.

2.11 Exercises

1. A communication system is used to transmit one of two equally likely messages, 1 and 2.

The channel output is a real-valued random variable R, the conditional density functions

of which are shown in figure 2.18. Determine the optimum receiver decision rule andcompute the resulting probability of error.

(Exercise 2.23 from Wozencraft and Jacobs [25].)

2. The noise n in figure 2.19a is Gaussian with zero mean, i.e. E[N] = 0. If one of twoequally likely messages is transmitted, using the signals of 2.19b, an optimum receiver

yields PE = 0.01.

EE

c

-4 +4 +8+4-4

-2 +2r

=s

+n

n

s

s2s1

s1 s3s2 s1 s4

0 0

+

(a) (b)

(d)(c)

s2

s3

'&$%

Figure 2.19:

(a) What is the minimum attainable probability of error Pmi nE

when the channel of 2.19a

is used with three equally likely messages and the signals of 2.19c? And with four

equally likely messages and the signals of 2.19d?

(b) How do the answers to the previous questions change if it is known that E[N] = 1instead of 0?


20/22


(Exercise 4.1 from Wozencraft and Jacobs [25].)

3. Show that p(n)

=12

exp(

n2

2) for

< n s ,

see figure 2.20.

1/s

d

dd

dd

dr

pR (r|S = s)

s s

Figure 2.20: The probability density function pR(r|S = s).

There are two messages i.e. M = {1, 2} and the corresponding signals are s1 = 1/2 ands2 = 2. Let the a-priori probabilities Pr{M = 1} = Pr{M = 2} = 1/2.

(a) Sketch the decision variables as a function of the output r in a single figure. For what

values r does an optimum receiver decide M = 1?(b) Determine the corresponding error probability PE.

(c) If the a-priori probability Pr{M = 1} is small enough, an optimum receiver willchoose M = 2 for all r. What is the largest value of Pr{M = 1} for which thishappens? What is the error probability for this value of Pr{M = 1}?

(Exam Communication Theory, November 18, 2003)

5. Consider a communication channel with an input s that can only assume positive values.The probability density function of the output R when the input is s is given by

pR(r|S = s) =1

2 s2exp

( r

2

2s2

).

Note that conditionally on the input s, this R is Gaussian, with mean zero and variance s2.

In figure 2.21 this density is plotted for s = 1.There are two messages i.e. M = {1, 2} and the corresponding signals are s1 = 1 ands2 =

e. Let the a-priori probabilities Pr{M = 1} = Pr{M = 2} = 1/2.


21/22

2.11. EXERCISES 35

3 2 1 0 1 2 3

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

r

pR

(r|S=1)

Figure 2.21: The probability density function pR (r|S = 1).

(a) Sketch both probability density functions pR (r|S = s1) and pR (r|S = s2) in a singlefigure. For what values r does an optimum receiver decide M = 1?

(b) Determine the corresponding error probability PE. Express it in terms of the Q()function.

(c) If the a-priori probability Pr{M = 1} is small enough, an optimum receiver willchoose M = 2 for all r. What is the largest value of Pr{M = 1} for which thishappens? What is the error probability for this value of Pr{M = 1}?

(Exam Communication Theory, January 23, 2004)

6. One of four equally likely messages is to be communicated over a vector channel which

adds a (different) statistically independent zero-mean Gaussian random variable with vari-

ance N0/2 to each transmitted vector component. Assume that the transmitter uses the

s1

Es /2

s4s3

s2

Figure 2.22: Signal structure.

signal vectors shown in figure 2.22 and express the PE produced by an optimum receiver


22/22

decision rules for real channels

Documents