chapter 13c: random signals and noisebazuinb/ece3800/b_notes13c.pdf · 2020. 11. 18. · random...

Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet, Probability, Statistics, and

Random Signals, Oxford University Press, February 2016.

B.J. Bazuin, Fall 2020 1 of 25 ECE 3800

Charles Boncelet, “Probability, Statistics, and Random Signals," Oxford University Press, 2016. ISBN: 978-0-19-020051-0

Chapter 13c: RANDOM SIGNALS AND NOISE

Sections 13.1 Introduction to Random Signals 13.2 A Simple Random Process 13.3 Fourier Transforms 13.4 WSS Random Processes 13.5 WSS Signals and Linear Filters 13.6 Noise

13.6.1 Probabilistic Properties of Noise 13.6.2 Spectral Properties of Noise

13.7 Example: Amplitude Modulation 13.8 Example: Discrete Time Wiener Filter 13.9 The Sampling Theorem for WSS Random Processes

13.9.1 Discussion 13.9.2 Example: Figure 13.4 13.9.3 Proof of the Random Sampling Theorem

Summary Problems




Signal plus noise processing

System analysis with a noise input …

tx

tn

th ty tr

Where the signal of interest is x(t), n(t) is a noise or interfering process. The signal plus noise is r(t) and the received system output is y(t) which has been filtered.

We have tntxtr

Assuming WSS with x and n independent and n zero mean

tntxtntxEtrtrERRR

tntntxtntntxtxtxERRR

NNXXRR RtxtnEtntxERR

NNNXXXRR RRR 2

For 0 mean noise …

NNXXRR RRR

And then

* hhRRR NNXXYY

hhRhhRR NNXXYY

Design considerations

the filter should not “modify” the signal of interest (unity gain, no phase)

the filter should remove as much noise as possible.




Signal‐to‐Noise‐(Power)‐RatioSNR(alwaysdoneforpowers)

The signal-to-noise ratio is the power ratio of the signal power to the noise power.

The input SNR is defined as

0

02

2

NN

XX

Noise

Signal

R

R

tNE

tXE

P

P

The output SNR is defined as

0

02

2

hhR

hhR

thtNE

thtXE

P

P

NN

XX

Noise

Signal

For a white noise process and assuming the “filter” does not change the input signal (unity gain), but strictly reduces the noise power by the equivalent noise bandwidth of the filter.

We have

dhN

dwwHwSR XXYY202

22

10

With appropriate filtering with unity gain where the signal exists and bandwidth reduction for the noise

𝑅 01

2𝜋∙ 𝑆 𝑤 ∙ |𝐻 𝑤 | 𝑁 ∙ 𝐵

or 𝑅 0 𝑅 0 𝑁 ∙ 𝐵

We expect the output SNR is defined as

EQ

XX

Noise

Signal

BN

R

P

P

0

0

The narrower the filter applied prior to signal processing, the greater the SNR of the signal! Therefore, always apply an analog filter prior to processing the signal of interest!

From the definition of band-limited noise power, the equation for the equivalent noise bandwidth (performed as a previous example)

12

10

12

12

20 dh

NdhRthtNE NN

dffHdtthBEQ

22

2

1

2

1

Under the unity gain condition

dtth1




Otherwise, the equivalent noise bandwidth can be defined as

dffHfH

BEQ

2

2max

12

For a real, low pass filter this simplifies to

dffHH

BEQ

2

20

12

Using Parseval’s Theorem this can be also defined as

dffHdwwHdtth222

2

1

2

2

2

2

02

dtth

dtth

H

dtth

BEQ




Example Section 13.7 Amplitude Modulation.

𝑋 𝑡 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃 Where m(t) is a random message, typically WSS, zero mean and bounded by +/-1. A is a an amplitude, f is the center frequency and there is a random phase angle (uniform distribution around a circle). The R.V. are independent.

𝐸 𝑋 𝑡 𝐸 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃

𝐸 𝑋 𝑡 𝐴 ∙ 𝐸 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝐸 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃

𝐸 𝑋 𝑡 𝐴 ∙ 1 0 ∙ 0 0

Autocorrelation

𝐸 𝑋 𝑡 ∙ 𝑋 𝑡 𝜏 𝑅 𝑡, 𝑡 𝜏

𝑅 𝑡, 𝑡 𝜏 𝐸 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃

∙ 𝐴 ∙ 1 𝛽 ∙ 𝑚 𝑡 𝜏 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜏 𝜃

𝐸 𝑋 𝑡 ∙ 𝑋 𝑡 𝜏 𝐴 ∙ 𝐸1 𝛽 ∙ 𝑚 𝑡 𝛽 ∙ 𝑚 𝑡 𝜏 𝛽 ∙ 𝑚 𝑡 ∙ 𝑚 𝑡 𝜏∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜃 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝑡 𝜏 𝜃

𝑅 𝑡, 𝑡 𝜏 𝐴 ∙ 1 𝛽 ∙ 𝑅 𝜏 ∙12∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏

𝑅 𝜏𝐴2∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏

𝐴2∙ 𝛽 ∙ 𝑅 𝜏 ∙ 𝑐𝑜𝑠 2𝜋 ∙ 𝑓 ∙ 𝜏

Forming the PSD

𝑆 𝑤𝐴4∙ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓

𝐴4∙ 𝛽 ∙ 𝑆 𝑤 ∗ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓

Which after performing the convolution becomes

𝑆 𝑤𝐴4∙ 𝛿 𝑤 2𝜋 ∙ 𝑓 𝛿 𝑤 2𝜋 ∙ 𝑓

𝐴4∙ 𝛽 ∙ 𝑆 𝑤 2𝜋 ∙ 𝑓 𝑆 𝑤 2𝜋 ∙ 𝑓

Your textbook simplifies the problem a bit … dealing with only the message and not the additional carrier component.




ExamplesofLinearSystemFrequency‐DomainAnalysis

Noise in a linear feedback system loop.

sX sY

1

1 ssA

sN

Linear superposition of X to Y and N to Y.

sNsYsXss

AsY

1

sNsXss

A

ss

AsY

11

1

sNsXss

A

ss

AsssY

11

2

sNAss

sssX

Ass

AsY

2

2

2

There are effectively two filters, one applied to X and a second apply to N.

Ass

AsH X

2 and

Ass

sssH N

2

2

sNsHsXsHsY NX

Generic definition of output Power Spectral Density:

wSwHwSwHwS NNNXXXYY 22




Change in the input to output signal to noise ratio.

dwwS

dwwS

SNR

NN

XX

In

dwwHN

wSH

dwwSwH

dwwSwH

SNR

N

XXX

NNN

XXX

Out20

2

2

2

2

0

EQ

XX

X

N

XX

Out BN

wS

dwH

wHN

wS

SNR

0

2

2

0

02

Where for this special case … (HN not a low pass or band pass filter … bad example)

dwH

wHB

X

NEQ 2

2

0

If the noise is added at the x signal input, the expected definition of noise equivalent bandwidth results.




Systems that Maximize Signal-to-Noise Ratio – Advanced Concept

SNR is defined as

EQNoise

Signal

BN

tsE

P

P

0

2

Define for an input signal tnts

Define for a filtered output signal tnts oo

For a linear system, we have:

0

dtntshtnts oo

The input SNR can be describe as

2

2

tnE

tsE

P

PSNR

Noise

Signalin

The output SNR can be described as

EQo

o

o

o

Noise

Signalout BN

tsE

tnE

tsE

P

PSNR

2

2

2

0

2

2

0

2

1dtthN

dtshE

SNR

o

out

Using Schwartz’s Inequality the numerator becomes

0

2

0

2

2

0

dtsEdhEdtshE




Applying this inequality, an SNR inequality can be defined as

0

2

0

2

0

2

2

1dtthN

dtsEdh

SNR

o

out

Canceling the filter terms, we have

0

22 dtsEN

SNRo

out

To achieve the maximum SNR, the equality condition of Schwartz’s Inequality must hold, or

0

2

0

2

2

0

dtsdhdtsh

This condition can be met for utsKh

where K is an arbitrary gain constant.

The desired impulse response is simply the time inverse of the signal waveform at time t, a fixed moment chosen for optimality. If this is done, the maximum filter power (with K=1) can be computed as

tdsdtsdht

2

0

2

0

2

tN

SNRo

out 2

max

This filter concept is called a matched filter. – Advanced Concept

If you wanted to detect a burst waveform that has been transmitted, to maximize the received SNR in white noise, the receiving filter should be the time inverse of the signal transmitted!

Note and caution: when using such a filter, the received signal maximum SNR will occur when the signal and convolved filter perfectly overlap. This moment in time occurs when the “complete” burst has been received by the system. If measuring the time-of-flight of the burst, the moment is exactly the filter length longer than the time-of-flight. (Think about where the leading edge of the signal-of-interest is when transmitted, when first received, and when fully present in the filter).




TheMatchedFilter–AdvancedConcept

Wikipedia: https://en.wikipedia.org/wiki/Matched_filter

“In signal processing, a matched filter is obtained by correlating a known signal, or template, with an unknown signal to detect the presence of the template in the unknown signal.[1][2] This is equivalent to convolving the unknown signal with a conjugated time-reversed version of the template. The matched filter is the optimal linear filter for maximizing the signal to noise ratio (SNR) in the presence of additive stochastic noise. Matched filters are commonly used in radar, in which a known signal is sent out, and the reflected signal is examined for common elements of the out-going signal. Pulse compression is an example of matched filtering. It is so called because impulse response is matched to input pulse signals. Two-dimensional matched filters are commonly used in image processing, e.g., to improve SNR for X-ray. Matched filtering is a demodulation technique with LTI (linear time invariant) filters to maximize SNR.[3].”

Applications:

Radar Sonar Pulse Compression Digital Communications (Correlation detectors) GPS pseudo-random sequence correlation

If you are looking for a signal, maximize the output of the filter when the signal is input! You will have a matched filter!

See ChirpCorrelationReceiver.m




Cooper & McGillem 9-6 Another optimal solution: Systems that Minimize the Mean-Square Error between the desired output and actual output – Advanced Concept

The error function tYtXtErr

where

0

dtNtXhtY

Performed in the Laplace Domain

sFsFsHsFsFsFsF NXXYXE

sFsHsHsFsFsFsF NXYXE 1

Computing the error power

j

j

NNXX dssHsHsSsHsHsSj

ErrE 112

12

j

j XXXXXX

NNXX dssSsHsSsHsS

sHsHsSsS

jErrE

2

12

Defining the input PSD sSsSsFsF NNXXCC

j

j

CC

NNXX

C

XXC

C

XXC

ds

sFsF

sSsS

sF

sSsHsF

sF

sSsHsF

jErrE

2

12

We can not do much about the last term, but we can minimize the terms containing H(s). Therefore, we focus on making the following happen

0

sF

sSsHsF

C

XXC

Step 1:Note that for this filter 1s- F

1

s F

1

CC sSsS NNXX

This is called a whitening filter as it forces the signal plus noise PSD to unity (white noise).

Step 2: Letting sF

sHsHsHsHC

221




Minimizing the terms containing H2(s), now we must focus on

sF

sSsH

C

XX

2 and

sF

sSsH

C

XX2

Letting H2 be defined for the appropriate Left or Right half-plane poles

Let LHPC

XX

sF

sSsH

2 and RHPC

XX

sF

sSsH

2

The composite filter is then

LHPC

XX

C sF

sS

sFsHsHsH

1

21

This solution is often called a Wiener Filter and is widely applied when the signal and noise statistics are known a-priori!




13.8 Example: Discrete Time Wiener Filter – Advanced Concept

Your textbook works to derive a discrete form of the Wiener Filter.

In it, you must calculate the coefficients of a Finite-Impulse-Response (FIR) digital filter.




Cooper & McGillem - Eigenvalue Based Filters – Advanced Concept We can continue a derivation started in the previous class discussion about time-sample filters, matrices and eigenvalues (derived based on having a class in Linear Algebra & Matrix Theory).

kxkwky

knkskx The expected value

HH kxkwkxkwEkykyE

HXXH kwkRkwkykyE

For a WSS input

HXXH kwRkwkykyE

If the signal and noise are zero mean, this becomes

HNNSSH kwRRkwkykyE

How do we maximize the output SNR

HNN

HSS

Noise

Signal

kwRkw

kwRkw

P

P

If we assume that the noise is white, IR NNN 2

H

HSS

NH

HSS

NNoise

Signal

kwkw

kwRkw

kwIkw

kwRkw

P

P

22

11

Performing a cholesky factorization of the signal autocorrelation matrix generates the following. Here, the numerator should suggest that an eigenvalue computation could provide a degree of simplification.

H

HHSS

NNoise

Signal

kwkw

kwRRkw

P

P

2

1

Once formed, the eigenvalue equation to solve is kwRkw S

which result in solutions for the resulting eigenvalues and eigenvectors of the form

2

2

2

1

NH

H

NNoise

Signal

kwkw

kwkw

P

P

Selecting the maximum eigenvalue and it’s eigenvector for the weight that maximizes the SNR!

2

2max

NNoise

Signal

P

P




13.9 The sampling Theorem for WSS Random Processes – Advanced Concept

If you take ECE 4550, you will be dealing with the sampling theorem.

It involves discrete time sampling of a continuous signal and the conditions under which the continuous signal can be regenerated from the discrete time samples.




Advanced Topic Adaptive Filter – Advanced Concept

If a desired signal reference is available, we may wish to adapt a system to minimize the difference between the desired signal or signal characteristics and a filter input signal.

The following information is based on:

S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014.

There are four classes of Adaptive Filter Applications

Identification

Inverse Modeling

Prediction

Interference Cancellation

Identification

The mathematical Model of an “unknown plant”

In state space control system this is an adaptive observer of the Plant

Examples: Seismology predicting earth strata




InverseModeling

Providing an “Inverse Model” of the plant

For a transmission medium, the inverse model corrects non-ideal transmission characteristics.

An adaptive equalizer

Prediction

Based on past values, provide the best prediction possible of the present values.

Positioning/Navigation systems often need to predict where an object will be based on past observations




InterferenceCancellationExample

Cancellation of unknown interference that is present along with a desired signal of interest. Two sensors of signal + interference and just interference Reference signal (interference) is used to cancel the interference in the Primary signal

(noise + interference) Classic Examples: Fetal heart tone monitors, spatial beamforming, noise cancelling

headphones.

From: S. Haykin, Adaptive Filter Theory, 5th ed., Prentice-Hall, 2014

The “Reference signal” contains the unwanted interference. The goal of the adaptive filter is to match the reference signal with the “interference” in the “Primary signal and force the output “difference error” to be minimized in power. Since “interference” is the only thing available to work with, the “power minimum” solution would be one where the interference is completely removed!

These techniques are based on the Weiner filter solution. While the signal and interference statistics are not known a-priori (before the filter gets started), after a number of input samples they can be estimated and used to form the filter coefficients. Then, as time continues, there is a sense that the estimates should improve until the adaptive coefficients are equal to those that would be computed with a-priori information.

The advantage ... adaptive filter can work when the statistics are slowly time varying!

Note: An application using an LMS adaptive filter is not too difficult for a senior project! (e.g. noise cancelling headphones, remove 60 cycle hum, etc.)




Textbook:Cancellinganinterferingwaveform–AdvancedConcept

The example in your textbook p. 407-411.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-20

-10

0

10

20or

igin

al

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-10

-5

0

5

10

15

Filt

ered

Time (sec)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-8

-6

-4

-2

0

2

4

6Adaptive Weights in Time

Tim

e (s

ec)

Weights

a1

a2

a3

a4




Matlab % % Text Adaptive Filter Example % clear close all t=0:1/200:1; % Interfereing Signal - 60 Hz n=10*sin(2*pi*60*t+(pi/4)*ones(size(t))); % Signal of interest and S + I x1=1.1*(sin(2*pi*11*t)); x2=x1+n; % Reference signal to excise r=cos(2*pi*60*t); m=0.15; a=zeros(1,4); z=zeros(1,201); z(1:4)=x2(1:4); w(1,:)=a'; w(2,:)=a'; w(3,:)=a'; w(4,:)=a'; % Adaptive weight computation and application for k=4:200 a(1)=a(1)+2*m*z(k)*r(k); a(2)=a(2)+2*m*z(k)*r(k-1); a(3)=a(3)+2*m*z(k)*r(k-2); a(4)=a(4)+2*m*z(k)*r(k-3); z(k+1)=x2(k+1)-a(1)*r(k+1)-a(2)*r(k)-a(3)*r(k-1)-a(4)*r(k-2); w(k+1,:)=a'; end figure(1) subplot(2,1,1); plot(t,x2,'k') ylabel('original') subplot(2,1,2) plot(t,z,'k');grid; ylabel('Filtered'); xlabel('Time (sec)'); figure(2) plot(t,w);grid; title('Adaptive Weights in Time') ylabel('Time (sec)') xlabel('Weights') legend('a1','a2','a3','a4');




Review Skills 7 and Skills 8 from the solution web site. Exam based questions! Hopefully you have looked at them and potentially tried a few.

Highlights from Skills 8 include the following

36.1 Suppose the circuit shown below has input ttx 9sin6 , where is a random

variable uniformly distributed on [0,2pi]. Assuming the R=1 M and C=1uF,

a. If the output signal is y(t), find the transfer function of the circuit. (H(s) possible given)

sX

sCR

RsY

1

sCR

sCRsH

1 and

s

ssH

1

111 2

2

s

s

s

s

s

ssHsH

b. Find the spectral density Syy(w) of the output and simplify. (Need AC. of Rxx and PSD Sxx)

99sin69sin6 ttERXX

99sin9sin36 ttERXX

2918cos9cos2

36 tERXX

9cos18 XXR

dtjwttwS XX exp9cos18

dtjwttjtjwS XX exp9exp9exp9

999 wwwS XX

Now the PSD of the output can be computer

1

9992

2

w

wwwwSYY




9982

81999

19

99

2

2

wwwwwSYY

c. Sketch the spectral density Syy(w).

Syy(w) is almost identical to Sxx(w). The filter used is a high-pass filter with a relative cutoff frequency w0 of 1.

d. What would happen if the input term had a “DC” component? What would the filter output for signals at w=0 be?

36.5 Consider the following linear circuit, where x(t) is the input voltage signal and y(t) is the output voltage signal.

x(t)

C

y(t)

LR

a. Find the transfer function of this system. For R=25 ohms, L=5H and C=0.05F (Note: these are not realistic values!)

sXsLsCR

sCsY

1

1

05.052505.01

1

1

122

ssLCssCR

sH

41

4

125.01

1

25.025.11

12

ssssss

sH

161

16

41

4

41

422

ssssss

sHsH

b. Assume the input signal is 21 5cos42cos312 tttx , where 1 and 2 are independent random variables that are uniform on the interval [0,2pi]. From this compute (1) the autocorrelation of the input signal and (2) the power spectral density of the input signal.

21

21

55cos422cos312

5cos42cos312

tt

ttERXX




22

122

21

111

21

55cos5cos16

22cos5cos125cos48

55cos2cos12

22cos2cos92cos36

55cos4822cos36144

tt

ttt

tt

ttt

tt

ERXX

2

2121

1212

1

2510cos2

165cos

2

16

27cos2

1223cos

2

12

58cos2

1253cos

2

12

224cos2

92cos

2

9144

t

tt

tt

t

ERXX

5cos82cos2

9144 XXR

dtjwtwS XX exp5cos82cos

2

9144

558222

9288 wwwwwwS XX

c. Compute (1) the autocorrelation of the output signal and (2) the spectral density of the output signal. Simplify all answers.

Output Power Spectrum sHsHsSsS XXYY

161

1622

ww

wSwS XXYY

161

16

558222

9288

22

ww

wwwwwwSYY

161

16558

161

1622

2

9

161

16288

22

22

22

wwww

wwww

www

wSYY




16515

16558

16212

1622

2

9

16010

16288

22

22

22

ww

ww

w

wSYY

55533

64

22100

72

288

551066

168

22100

16

2

9

288

ww

ww

w

ww

ww

w

wSYY

dwjwwSR YYYY

exp2

1

dwjw

ww

wwwRYY

exp

55533

64

22100

72288

2

1

5cos533

642cos

100

72144 YYR

d. Compute the ratio of the output to input total average power of the signals.

5cos82cos2

9144

5cos533

642cos

100

72144

0

0

XX

YY

in

out

R

R

P

P

9255.0

5.156

84.144

82

9144

533

64

100

72144

0

0

XX

YY

in

out

R

R

P

P

e. Compute the ratio of the output to input dc average power of the two signals. 1

144

144

XX

YY

in

out

R

R

P

P

f. From the above, comment on the filtering effect of the original LCR circuit. Is it a low pass, band pass, or high pass filter?




This is a low-pass filter where dc is passed from the input to the output. For the selected input signal, with oscillation at w=2 and w=5, the filter first rolls off at w=1 and then continues at w=4.

The input to output power for w=3 is 16.025

4

9

72

100

2

2

9100

72

in

out

P

P

The input to output power for w=5 is 015.0533

8

8533

64

in

out

P

P

chapter 13c: random signals and noisebazuinb/ece3800/b_notes13c.pdf · 2020. 11. 18. · random...

Documents