chapter 6: random signals and noise• noise spectral densities and weiner-kinchinetheorem •...

ECE 407 – Spring 2009 – Farhan Rana – Cornell University

Chapter 6: Random Signals, Correlations, and Noise

In this lecture you will learn:

• Random Signals• Shot Noise• Correlated Noise: Bunching and Antibunching• Partition Noise• Langevin Equation• Noise Spectral Densities and Weiner-Kinchine Theorem • Brownian and Diffusion Noise


Fourier Transforms of Signals

2

)(

)()(

dxetx

dttxex

ti

ti


Random Signals

A random signal can be any signal from a set of signals )(tx }),(),(),({ 321 txtxtx

The set is the sample space}),(),(),({ 321 txtxtx

The probability that will equal is:)(tx )(txn )()( txP ntx

Mean:

nntxnx txPtxtxtm )()()()( )(

Auto-correlation:

n

ntxnnxx txPtxtxtxtxttR )()()()()(),( )(212121Cross-correlation:

n

nnm

tytxmnxy tytxPtytxtytxttR 21)()(212121 ),()()()()(),(


Stationary Signals and Ergodic SignalsStationary Signals: Are those whose characteristics do not depend upon the time origin

Stationary signals have the following properties:a) The mean values are independent of time, i.e.:

b) The auto- and cross-correlation functions are functions of the time difference only, i.e.,

xx mtxtm )()(

)()()(),( 212121 ttRtxtxttR xxxx )()()(),( 212121 ttRtytxttR xyxy

Ensemble Averages and Time Averages:

Averages of signals can be done in two ways,a) Ensemble Averages:

b) Time Averages:

nntxn txPtxtx

2

2)(1)(

T

TTdttx

Ttx limit


Stationary Signals and Ergodic SignalsErgodic Signals: When all ensemble averages equal the corresponding time averages the signal is called ergodic. Ergodicity implies stationarity but it is not the other way around

xT

TTmdttx

Ttxtx

2

2)(1)()( limit

dttytttxT

tytxtytxttRT

TTxy )()(

1)()()()()(2

221212121

limit

tx1 tx2 tx3

txN

Ergodicity implies that each signal in the sample set is representative of the whole set


Power Spectral Densities of Signals

The total “energy” of a signal can be infinite:

2|)(|)( 22 dxdttx

tx

So it is better to work with signal power and not with signal energies

otherwise022

)()(TtTtxtxT

Given , define a truncated signal as: tx

Signal Power:

Signal Energy:

2|)(|1

)(1)(1

2

22

2

2

dxT

dttxT

dttxT

T

T

T

T

The power in the signal is: tx


Power Spectral Densities of SignalsThe ensemble averaged signal power is:

.2

)(12

)(1 22

dx

Tdx

T TT

The power spectral density of the signal is defined as:2)(1limit)( TTxx

xT

S

Weiner-Kinchine Theorem:

dttReS xxti

xx )()(

The power spectral density of a stationary signal is the Fourier transform of the auto-correlation function:

tx

)()( txedtx tiA Useful Relation: Consider a stationary random signal : tx

xxSxx '2)('*

)('

2' *

xxdSxx


Shot Noise

Suppose we are looking at a process that consists of a set of discrete events happening randomly in time

Suppose the j-th event happens at time tj

Particle stream Detector

The rate of the events (i.e. the number of events happening per unit time) is:

j

jtttr )()(

)(tr

The times tj constitute the random part

We will assume the following:

i) The times tj are completely independent of each otherii) The probability that there is an event in a very short time interval dt is given

by dt where is called the average rate of the process


Shot NoiseEnsemble Average Rate:

)()()( tttdtttr

jj

Auto-Correlation:

)()(

)()()(

j mmj

rr

tttt

trtrR

)()()()(

terms diagonal-off )()(

terms diagonal )()()(

mjj mmj

mjj mmj

jjjrr

tttttttttd

tttt

ttttR

2)()( m

mj

j tttt


Shot Noise

2)()( rrRAuto-Correlation:

Spectral Density:

)(2)( 2 rrS

Noise: The noise in the signal r(t ) is:

)()()()( trtrtrtn

Noise Auto-Correlation and Spectral Density:

)(

)(2)()(

)()(

2

2

nn

rrnn

rrnn

SSS

RR

i) The noise spectral density is white (independent of frequency) or flat

ii) The noise spectral density is equal to the average rate

The noise is not present because the events are discreetThe noise is present because the events happen randomly in time

Shot Noise

)(nnS


Poisson Statistics and Shot Noise

Particle stream with shot noise Detector

Let be the probability of having n events in time t : ),( TnP

),1(),(),(

),(),1(),(),()1(),(),1(),(

TnPTnPTnPdTd

TnPTnPT

TnPTTnPTTnPTTnPTTnP

Solution given the boundary condition is:0)0,( nTnP

Tn

enTTnP

!)(),( Poisson statistics

The average events in time T is:

TTenT

enTnTnPnN

T

n

n

Tn

nnT

)()!1(

)(!)(),(

1

110

As expected


Poisson Statistics and Shot Noise

The standard deviation is:

2222 TTTTT NNNNN

)()()()!1(

)()11(

)()!1(

)(!)(

2

1

1

1

10

22

TTTenTn

TenTne

nTnN

n

Tn

Tn

n

T

n

nT

TTTT NTNNN )(222 Standard deviation is equal to the mean


Bunching and Antibunching

Particle stream DetectorShot Noise:

'"''|"'," dtdttPttPttP For :'" tt

Bunching Example:

'1"''|"'," '" dtedttPttPttP tt For :'" tt

22

21)(

nnS

Particles tend to arrive in bunches

Antibunching Example:

'1"''|"'," '" dtedttPttPttP tt For and :" 't t

22

21)(

nnS

Particles tend to arrive separately

)(nnS

)(nnS

2


0

0.5

1

1.5

0

0.5

1

1.5

Bunching and Antibunching

Shot noise signal(All frequencies are present in the noise)

Bunched noise signal(Lower frequencies are dominant in the noise)

Antibunched noise signal(Higher frequencies are dominant in the noise)

0 5 100

0.5

1

1.5


Partition Noise

Particle stream Detectorsplitter

ti

tf

tr

.)1()()1()()()(

)(

titrtitf

ti

Average Rates:

)1()()()()(

)()()()(

trtn

tftn

titititn

r

f

iNoise in the Streams:


j

jttti )()(

)()( j

jj ttftf

)()( j

jj ttrtr

Partition Noise


ti

tf

tr

10

11,0

j

j

f

f

j

P

Pf

0

111,0

j

j

r

r

j

P

Pr

)((()( tittttftfj

jj

jj

( ) ( 1 ( 1 ( ) 1j j jj j

r t r t t t t i t


Partition Noise


ti

tf

tr

Auto-Correlation:

mjmjmj

j mjjjjff

jmjmj

ff

ttttffttttfR

fftttt

tftfR

m

)()()()()(

)()(

)()()(

2

Note that:22

mjmjmjjjffffff


Partition Noise


ti

tf

tr

Auto-Correlation:

mjmj

j mjjj

jjj

mjmj

j mjjjff

tttttttt

tttt

ttttttttR

)()()()(

)()()1(

)()()()()(

2

2

)(iiR

)()()()1()( 2 iij

jjff RttttR


Partition Noise


ti

tf

tr

Auto-Correlation:

)()1()(

)()()1()(2

2

iiff

iiff

SS

RR

Noise: )()()()( tftftftnf

ii

ff

nn

iiffnn

R

RRR2

2222

1

1)()(

Noise Auto-Correlation:

Noise Spectral Density:

)()1()( 2 iiff nnnn SS


Partition Noise


ti

tf

tr

Noise Spectral Density:

)()1()( 2 iiff nnnn SS

Noise in the input that got attenuatedNoise added by the splitter

Case i: 1

)(ff nnSOutput stream has shot noise irrespective of the noise in the input stream

Case ii:

)(ff nnS Output stream has shot noise as well

)(iinnS (input stream has shot noise)

Case iii: 1

)()( iiff nnnn SS As expected


Langevin Equations: IntroductionVelocity of a Particle in Brownian Motion

)()( tvdt

tdv

Assume 1D problemSuppose we take velocity to be a random signal and write:

tetvtv )0()(tetvtv 222 )0()(

But in steady state (in 1D) statistical physics tells us:

mTKtvTKtmv BB )(2

1)(21 22


Langevin Equations: Introduction

)()()( tFtvmdt

tdvm

We need a better model for the kicks the particle velocity receives in collisions:

)()()(0)(

2121 ttAtFtFtF

We assume:

We don’t know A (yet)

t ttt dtetFm

etvtv0

1)(

1 1)(1)0()(

Solution:

t ttt

tttttt

etFdtm

etv

etFtFdtdtm

etvtv

0

)(11

)2(21

02

012

222

1

21

)()0(2

)()(1)0()(



t t ttt

t ttt

t

etFtFdtdtm

etFtvdtm

eetvtv

0 0

)2(21212

0

)(11

222

21

1

)()(1

)()0(2)0()(

In steady state, when ‘t’ can be assumed to be large, the only term that survives is the last term:

.22

)1(lim

lim

)(1lim)(lim

2

2

2

0

)(212

0 0

)2(21212

2

1

21

mA

re

mA

edtmA

ettAdtdtm

tv

t

t

t ttt

t t ttttt

Enforce the condition from statistical physics:

mTKtv B)(2 TKmA

mTK

mA

BB

2

2 2



mtFtv

dttdv )()()(

Langevin Equations for the Velocity of a Particle in Brownian Motion

)(2)()( 2121 ttTKmtFtF B 0)( tF

Spectral Density:

t ttt etFm

etvtv0

)(1 1)(

1)0()(

t tttrt ttt

t t tttttt

dtetFem

vdtetFem

v

eetFtFdtdtm

evtvtv

01

)(1

02

)(2

0 0

)()(21212

)(2

12

21

)()0()()0(

)()(1)0()()(

"

0

)()"'(212

'

01'

21

"

)(2lim)"()'(t tttttB

teettdtdt

mTKtvtv

t



|"'|)()(' ttBvv emTKtvtvttR

The correlation function points to a stationary process in steady state

)(

2)(

)(

22

||

mTK

S

eedm

TKS

B

vv

iBvv

Rvv()


Langevin Equations: IntroductionSpectral Density: Frequency Domain Technique

)()( tFedtF ti

mtFtv

dttdv )()()(

Note that:

0)()(

tFedtF ti

)(22)()(*

2

)(2

)()()()(*

2121

)(1

2121

212121

121

2211

2211

TKmFF

edtTKm

tteedtdtTKm

tFtFedtedtFF

B

tiB

titiB

titi

0)( tF)(2)()( 2121 ttTKmtFtF B



mtFtv

dttdv )()()(

Start from:

Fourier transform it:

][/)()(

)()()(

iimFv

Fvvi

22

2

2)(

2')'(2

]]['[

22

''

'1

2')()'(*)(

mTK

S

dii

mTK

dii

FF

m

dvvS

B

vv

B

vv

Compute the spectral density:


Stationary Processes, Correlation functions, and the Regression Theorem

The famous regression theorem

If holds for all signals in the sample set( )

( )d x t

A x tdt

Then:

Markov Process (no memory!)

( )

( )d x t x t

A x t x td

Regression of correlation functions obey the same dynamical equations as the mean values - also known as the Onsager’s Regression Hypothesis

( ) ( ) ( )x t x t x t x t x t x t

Microscopic reversibility Stationarity

If then 12

xx

x

1 1 1 1 21 2

2 2 1 2 2

x x x x xx x x x

x x x x x

Exterior product



Consider the Langevin equations of several coupled random variables (in vector notation):

( ) ( )dx t A x t F tdt

( )( )

d x tA x t

dt

, | ,d o oxx t d x x P x t x t

One can write the “conditional” average as:

If the random variable is stationary:

The solution subject to an initial condition would be:

( ) ( )o oA t t A t to ox t e x t e x

( )o ox t x

, | ,0d o oxx t d x x P x t t x

Markov Process (no memory!)


1 2 1 2 1 2

1 2 1 2 1 2 2

1 2 1 2 1

' ' , ; , '

, | , ' , '

,

d dx

d dx x

d dx

R t t x t x t d x d x x x P x t x t

d x d x x x P x t x t P x t

d x d x x x P x

2 2' | ,0 , 'xt t x P x t

Now consider the correlation function matrix:

Stationarity implies that the “unconditional” probability distribution cannot be a function of time, and if there is a steady state then:

2, 'xP x t

2 2, ' steadyx xstate

P x t P x

1 2 1 2 1 2 2' ' , ' | ,0d d steadyx xstate

R t t x t x t d x d x x x P x t t x P x

This implies:



1 2 1 2 1 2 2' ' , ' | ,0

' ' '

d d steadyx xstate

R t t x t x t d x d x x x P x t t x P x

d dR t t x t x t A x t x tdt dt

We get:

Which needs to be solved subject to the initial condition:

1 2 1 2 1 2 2

2 2 2 2

0 ' ' ,0 | ,0

' '

d d steadyx xstate

d steadysteady steadyxstatestate state

R x t x t d x d x x x P x x P x

d x x x P x x t x t x x

And the solution is:

'' ' ' 'A t t steadyxxstate

R t t x t x t e x t x t


This is the famous regression theorem!

1 2d x x


Stationary Processes, Correlation functions, and the Regression Theorem: An Example

mtFtv

dttdv )()()(

Consider the Brownian motion equation:

We know that in steady state:

20( ) ( )B

steadystate

K Tv t v t v t v t vm

Using the regression theorem:

( )

( )d v t v t

v t v td

Initial condition: 20vv steady

state

B

R v

K Tm

Solution for > 0 is:

( ) ( ) Bsteadystate

K Tv t v t e v t v t em

If then system has time reversal symmetry then:

( ) ( ) ( ) ( ) Bsteadystate

K Tv t v t v t v t v t v t e v t v t em


Particle Diffusion

)()( tWdt

tdx

The diffusion of a particle (in 1D) making random jumps is also described by a Langevin equation

0)( tW

)(2)()( 2121 ttDtWtW

The probability of the particle being at a particular location is given by the diffusion equation

2

2 ),(),(x

txPDt

txP

))((),( oo txxtxP If the initial condition is:

Solution is:

)](2[2

))((exp)(22

1)),(|,(2

o

o

ooo ttD

txxttD

ttxtxP


Particle Diffusion

)0()(

)()0()(0

xtx

dttWxtxt

)()( tWdt

tdx

Solution is:

Dtx

dtDx

ttDdtdtxtx

tWtWdtdtdttWxxtx

t

t t

t tt

2)0(

2)0(

)(2)0()(

)()()()0(2)0()(

20

12

0 02121

22

0 02121

0

22

Variance:

Dtxtx

Dtxtx

2)0()(

2)0()(2

22


Particle Diffusion

21221

21212

21

0 02121

2

0 02121

221

2)()(

andofsmaller),(min2)0()()(

)(2)0(

)()()0()()(

1 2

1 2

ttDtxtx

ttttDxtxtx

ttDdtdtx

twtwdtdtxtxtx

t t

t t

Auto-Correlation:

Not a stationary process!!

)](2[2

))((exp)(22

1)),(|,(2

o

o

ooo ttD

txxttD

ttxtxP

Diffusion Equation Result:

ooo ttttDtxtx for)(2)()( 2


Langevin Equations Vs Fokker-Planck Equations

For every Langevin equation of the form:

tFttaAdt

tda ),()(

)'(,)'()( ttttaBtFtF

0)( tF

there is a Fokker-Planck equation (a generalized diffusion equation) of the form:

),(,21),(,),( 2

2taPtaB

ataPtaA

attaP

which gives the full probability distribution of the random signal at any time

In most cases, one is never interested in the full probability distribution but only in the correlations which are measured experimentally

mtFtv

dttdv )()()(


Particle DiffusionSpectral Density (??):

22)(

')'(22

2'

')()'(*

2'

)()'(*2

')(

)()(

)()(

DS

Dd

WWd

xxdS

iWx

Wxi

xx

xx

Signature of diffusion!


Phasor Representation and Signal Quadratures

y()

2/

2/

2/

2/)(

2)(

2)(

)(2

)(

o

o

o

o

titi

ti

eydeydty

eydty

Consider a narrow band signal:

2/

2/

)(2/

2/

)( )()(2

)(o

o

ooo

o

oo titititi eyeeydety

2)(*

2)()( txetxety titi oo

Factor out the fast time dependence:



ti oetxty )(Re)(2

)(*2

)()( txetxety titi oo

)()()( 21 txitxtx

Let:

ttxttxty

etxitxty

oo

ti o

sin)(cos)()()()(Re)(

21

21

Then:

y()

Two quadratures of y(t)

x(t)x2(t)

x1(t) x1

x2



x(t) x+/ 2(t)

x(t)

i

ii

etxitx

etxetxtx

)()(

)()()(

2

22

Generalized Quadratures:

Example 1:

ioextx )(

sincos oo xixtx

txtyeex

etxty

oo

tiio

ti

o

o

cos)(Re

)(Re)(

x(t)


Phasor Representation and Signal QuadraturesExample 2:

x(t)

)()( 1 txtx o

ttxttxty

oo

o

cos))((

cos)()(1

oxt )(1

x(t)

1(t)

Amplitude Modulation

Example 3:

)()( 2 tixtx o

oxt )(2

ooo

xti

o

ooo

xttxty

ex

xtixtixtx

o

2

)(

22

cos

)(1)()(

2

2(t)

Phase Modulation



Example 4:

)()()( 21 titxtx o

oxtt )(),( 21

x(t)

Error region)(),( 21 tt are zero-mean random signals

x(t)

+ / 2(t)

(t)

)()( textx io Example 5:

oxt )(

iio eitextx 2)(

oxt

i

o

i

oo

etx

ex

titxtx

2

1)( 2

ooo x

tttxty 2cos)(

chapter 6: random signals and noise• noise spectral densities and weiner-kinchinetheorem •...

Documents