Correlation, Energy SpectralDensity and Power Spectral
Density
5/10/04 M. J. Roberts - All Rights Reserved 2
Introduction
• Relationships between signals can be just asimportant as characteristics of individualsignals
• The relationships between excitation and/orresponse signals in a system can indicate thenature of the system
5/10/04 M. J. Roberts - All Rights Reserved 3
Flow Velocity Measurement
The relative timing of the two signals, p(t) and T(t),and the distance, d, between the heater and thermometer together determine the flow velocity.
Heater
Thermometer
5/10/04 M. J. Roberts - All Rights Reserved 4
Correlograms
TwoCompletelyNegativelyCorrelatedDT Signals
Correlogram
Signals
5/10/04 M. J. Roberts - All Rights Reserved 5
Correlograms
TwoUncorrelatedCT Signals
Correlogram
Signals
5/10/04 M. J. Roberts - All Rights Reserved 6
Correlograms
Two PartiallyCorrelatedDT Signals
Correlogram
Signals
5/10/04 M. J. Roberts - All Rights Reserved 7
Correlograms
These two CTsignals are not
stronglycorrelated but
would be if onewere shifted intime the right
amount
Correlogram
Signals
5/10/04 M. J. Roberts - All Rights Reserved 8
Correlograms
DT Sinusoids Witha Time Delay
CT Sinusoids Witha Time Delay
Correlogram Correlogram
Signals Signals
5/10/04 M. J. Roberts - All Rights Reserved 9
Correlograms
Two Non-Linearly Related
DT Signals
Correlogram
Signals
5/10/04 M. J. Roberts - All Rights Reserved 10
The Correlation FunctionPositively CorrelatedDT Sinusoids with
Zero Mean
Uncorrelated DTSinusoids with
Zero Mean
Negatively CorrelatedDT Sinusoids with
Zero Mean
5/10/04 M. J. Roberts - All Rights Reserved 11
The Correlation FunctionPositively CorrelatedRandom CT Signals
with Zero Mean
Uncorrelated RandomCT Signals with
Zero Mean
Negatively CorrelatedRandom CT Signals
with Zero Mean
5/10/04 M. J. Roberts - All Rights Reserved 12
The Correlation FunctionPositively Correlated
CT Sinusoids withNon-zero Mean
Uncorrelated CTSinusoids withNon-zero Mean
Negatively CorrelatedCT Sinusoids with
Non-zero Mean
5/10/04 M. J. Roberts - All Rights Reserved 13
The Correlation FunctionPositively CorrelatedRandom DT Signalswith Non-zero Mean
Uncorrelated RandomDT Signals withNon-zero Mean
Negatively CorrelatedRandom DT Signalswith Non-zero Mean
5/10/04 M. J. Roberts - All Rights Reserved 14
Correlation of Energy SignalsThe correlation between two energy signals, x and y, is thearea under (for CT signals) or the sum of (for DT signals)the product of x and y*.
The correlation function between two energy signals, x andy, is the area under (CT) or the sum of (DT) that productas a function of how much y is shifted relative to x.
x y*t t dt( ) ( )−∞
∞
∫ x y*n nn
[ ] [ ]=−∞
∞
∑or
R x yxy*τ τ( ) = ( ) +( )
−∞
∞
∫ t t dt R x yxy*m n n m
n
[ ] = [ ] +[ ]=−∞
∞
∑or
In the very common case in which x and y are both real,
R x yxy τ τ( ) = ( ) +( )−∞
∞
∫ t t dt R x yxy m n n mn
[ ] = [ ] +[ ]=−∞
∞
∑or
5/10/04 M. J. Roberts - All Rights Reserved 15
Correlation of Energy Signals
The correlation function for two real energy signals is very similarto the convolution of two real energy signals.
Therefore it is possible to use convolution to find the correlation function.
It also follows that
x y x yt t t d( )∗ ( ) = −( ) ( )−∞
∞
∫ τ τ τ x y x yn n n m mm
[ ] ∗ [ ] = −[ ] [ ]=−∞
∞
∑or
R x yxy τ τ τ( ) = −( )∗ ( ) R x yxy m m m[ ] = −[ ] ∗ [ ]or
R X Yxy*τ( )← → ( ) ( )F f f R X Yxy
*m F F[ ]← → ( ) ( )For
5/10/04 M. J. Roberts - All Rights Reserved 16
Correlation of Power SignalsThe correlation function between two power signals, x andy, is the average value of the product of x and y* as afunction of how much y* is shifted relative to x.
If the two signals are both periodic and their fundamentalperiods have a finite least common period,
where T or N is any integer multiple of that least commonperiod. For real periodic signals these become
orR lim x yxy*τ τ( ) = ( ) +( )
→∞ ∫T TTt t dt
1R lim x yxy
*mN
n n mN
n N
[ ] = [ ] +[ ]→∞
=∑1
R x yxy*τ τ( ) = ( ) +( )∫
1
Tt t dt
TR x yxy
*mN
n n mn N
[ ] = [ ] +[ ]=∑1
or
R x yxy τ τ( ) = ( ) +( )∫1
Tt t dt
TR x yxy m
Nn n m
n N
[ ] = [ ] +[ ]=∑1
or
5/10/04 M. J. Roberts - All Rights Reserved 17
Correlation of Power Signals
Correlation of real periodic signals is very similar toperiodic convolution
where it is understood that the period of the periodicconvolution is any integer multiple of the least commonperiod of the two fundamental periods of x and y.
Rx y
xy τ τ τ( ) = −( ) ( )T
Rx y
xy mm m
N[ ] = −[ ] [ ]
orRx y
xy τ τ τ( ) = −( ) ( )T
Rx y
xy τ τ τ( ) = −( ) ( )T
R X Yxy*τ( )← → [ ] [ ]FS k k R X Yxy
*m k k[ ]← → [ ] [ ]FSor
5/10/04 M. J. Roberts - All Rights Reserved 18
Correlation of Power Signals
5/10/04 M. J. Roberts - All Rights Reserved 19
Correlation of Sinusoids
• The correlation function for two sinusoids ofdifferent frequencies is always zero. (pp.588-589)
5/10/04 M. J. Roberts - All Rights Reserved 20
AutocorrelationA very important special case of correlation is autocorrelation.Autocorrelation is the correlation of a function with a shiftedversion of itself. For energy signals,
At a shift, τ or m, of zero,
which is the signal energy of the signal. For power signals,
which is the average signal power of the signal.
R x xxx*τ τ( ) = ( ) +( )
−∞
∞
∫ t t dt R x xxx*m n n m
n
[ ] = [ ] +[ ]=−∞
∞
∑or
R xxx 02( ) = ( )
−∞
∞
∫ t dt R xxx 02[ ] = [ ]
=−∞
∞
∑ nn
or
R lim xxx 01 2( ) = ( )
→∞ ∫T TTt dt or R lim xxx 0
1 2[ ] = [ ]→∞
=∑
Nn NN
n
5/10/04 M. J. Roberts - All Rights Reserved 21
Properties of AutocorrelationFor real signals, autocorrelation is an even function.
Autocorrelation magnitude can never be larger than it is atzero shift.
If a signal is time shifted its autocorrelation does not change.
The autocorrelation of a sum of sinusoids of differentfrequencies is the sum of the autocorrelations of theindividual sinusoids.
R Rxx xxτ τ( ) = −( ) R Rxx xxm m[ ] = −[ ]or
R Rxx xx0( ) ≥ ( )τ R Rxx xx0[ ] ≥ [ ]mor
5/10/04 M. J. Roberts - All Rights Reserved 22
Autocorrelation Examples
Three differentrandom DTsignals and theirautocorrelations.Notice that, eventhough the signalsare different, theirautocorrelationsare quite similar,all peakingsharply at a shiftof zero.
5/10/04 M. J. Roberts - All Rights Reserved 23
Autocorrelation ExamplesAutocorrelations for a cosine “burst” and a sine “burst”.Notice that they are almost (but not quite) identical.
5/10/04 M. J. Roberts - All Rights Reserved 24
Autocorrelation Examples
5/10/04 M. J. Roberts - All Rights Reserved 25
Matched Filters
• A very useful technique for detecting thepresence of a signal of a certain shape in thepresence of noise is the matched filter.
• The matched filter uses correlation to detectthe signal so this filter is sometimes called acorrelation filter
• It is often used to detect 1’s and 0’s in abinary data stream
5/10/04 M. J. Roberts - All Rights Reserved 26
Matched FiltersIt has been shownthat the optimalfilter to detect anoisy signal is onewhose impulseresponse isproportional to thetime inverse of thesignal. Here aresome examples ofwaveshapesencoding 1’s and 0’sand the impulseresponses ofmatched filters.
5/10/04 M. J. Roberts - All Rights Reserved 27
Matched FiltersNoiseless Bits Noisy Bits
Even in the presence of alarge additive noisesignal this matched filterindicates with a highresponse level thepresence of a 1 and witha low response level thepresence of a 0. Sincethe 1 and 0 are encodedas the negatives of eachother, one matched filteroptimally detects both.
5/10/04 M. J. Roberts - All Rights Reserved 28
Autocorrelation Examples
Two familiar DTsignal shapes andtheir autocorrelations.
5/10/04 M. J. Roberts - All Rights Reserved 29
Autocorrelation Examples
Three random power signals with different frequency contentand their autocorrelations.
5/10/04 M. J. Roberts - All Rights Reserved 30
Autocorrelation ExamplesAutocorrelation functions for a cosine and a sine. Notice that the autocorrelation functions are identical even thoughthe signals are different.
5/10/04 M. J. Roberts - All Rights Reserved 31
Autocorrelation Examples
• One way to simulate a random signal is witha summation of sinusoids of differentfrequencies and random phases
• Since all the sinusoids have differentfrequencies the autocorrelation of the sum issimply the sum of the autocorrelations
• Also, since a time shift (phase shift) does notaffect the autocorrelation, when the phasesare randomized the signals change, but nottheir autocorrelations
5/10/04 M. J. Roberts - All Rights Reserved 32
Autocorrelation Examples
Let a random signal be described by
Since all the sinusoids are at different frequencies,
where is the autocorrelation of .
x cost A f tk k kk
N
( ) = +( )=
∑ 2 01
π θ
R Rx τ τ( ) = ( )=∑ kk
N
1
R k τ( ) A f tk k kcos 2 0π θ+( )
5/10/04 M. J. Roberts - All Rights Reserved 33
Autocorrelation Examples
Four DifferentRandom Signals
with IdenticalAutocorrelations
5/10/04 M. J. Roberts - All Rights Reserved 34
Autocorrelation Examples
Four DifferentRandom Signals
with IdenticalAutocorrelations
5/10/04 M. J. Roberts - All Rights Reserved 35
Autocorrelation Examples
Four DifferentRandom Signals
with IdenticalAutocorrelations
5/10/04 M. J. Roberts - All Rights Reserved 36
Autocorrelation Examples
Four DifferentRandom Signals
with IdenticalAutocorrelations
5/10/04 M. J. Roberts - All Rights Reserved 37
Cross CorrelationCross correlation is really just “correlation” in the cases inwhich the two signals being compared are different. Thename is commonly used to distinguish it from autocorrelation.
5/10/04 M. J. Roberts - All Rights Reserved 38
Cross Correlation
A comparison of x and y with y shifted for maximumcorrelation.
5/10/04 M. J. Roberts - All Rights Reserved 39
Cross CorrelationBelow, x and z are highly positively correlated and x and y areuncorrelated. All three signals have the same average signalpower. The signal power of x+z is greater than the signal powerof x+y.
5/10/04 M. J. Roberts - All Rights Reserved 40
Correlation and the Fourier Series
Calculating Fourier series harmonic functions can be thought ofas a process of correlation. Let
Then the trigonometric CTFS harmonic functions are
Also, let
then the complex CTFS harmonic function is
c cos and s sint kf t t kf t( ) = ( )( ) ( ) = ( )( )2 20 0π π
X R , X Rxsc sk k[ ] = ( ) [ ] = ( )2 0 2 0xc
z t e j kf t( ) = + ( )2 0π
X Rxzk[ ] = ( )0
5/10/04 M. J. Roberts - All Rights Reserved 41
Energy Spectral DensityThe total signal energy in an energy signal is
The quantity, , or , is called the energy spectraldensity (ESD) of the signal, x, and is conventionally given thesymbol, Ψ. That is,
It can be shown that if x is a real-valued signal that the ESD iseven, non-negative and real.
E t dt f dfx x X= ( ) = ( )−∞
∞
−∞
∞
∫ ∫2 2E n F dF
nx x X= [ ] = ( )
=−∞
∞
∑ ∫2 2
1or
X f( )2 X F( )2
Ψx Xf f( ) = ( )2Ψx XF F( ) = ( )2or
5/10/04 M. J. Roberts - All Rights Reserved 42
Energy Spectral DensityProbably the most important fact about ESD is the relationshipbetween the ESD of the excitation of an LTI system and theESD of the response of the system. It can be shown (pp. 606-607) that they are related by
Ψ Ψ Ψy x*
xH H Hf f f f f f( ) = ( ) ( ) = ( ) ( ) ( )2
Ψ Ψ Ψy x*
xH H HF F F F F F( ) = ( ) ( ) = ( ) ( ) ( )2
or
5/10/04 M. J. Roberts - All Rights Reserved 43
Energy Spectral Density
5/10/04 M. J. Roberts - All Rights Reserved 44
Energy Spectral Density
It can be shown (pp. 607-608) that, for an energy signal,ESD and autocorrelation form a Fourier transform pair.
Rx xt f( )← → ( )F Ψ Rx xn F[ ]← → ( )F Ψor
5/10/04 M. J. Roberts - All Rights Reserved 45
Power Spectral DensityPower spectral density (PSD) applies to power signals inthe same way that energy spectral density applies to energysignals. The PSD of a signal x is conventionally indicatedby the notation, or . In an LTI system,
Also, for a power signal, PSD and autocorrelation form aFourier transform pair.
Gx f( ) Gx F( )
G H G H H Gy x*
xf f f f f f( ) = ( ) ( ) = ( ) ( ) ( )2
G H G H H Gy x*
xF F F F F F( ) = ( ) ( ) = ( ) ( ) ( )2
or
R Gt f( )← → ( )[ ]F or R Gn F[ ]← → ( )F
5/10/04 M. J. Roberts - All Rights Reserved 46
PSD Concept
5/10/04 M. J. Roberts - All Rights Reserved 47
TypicalSignals in
PSDConcept