Download - Dct and adaptive filters
DEFINITION OF DCT
Due to its computational efficiency the discrete
Fourier transform (DFT) is very popular.
But it is very complex.
It has very poor energy compaction. Energy
compaction is the ability to pack the energy of the
spatial sequence into as few freq. coefficients as
possible. It helps in image compression.
When the input data contains only real nos from an
even function, the Sin component of DFT is ’0’, and
the DFT becomes DCT.
DCT is simply a computationally efficient version of
the DFT for the signal that are real and even.
In comparison, (DCT) is a real transform that
transforms a sequence of real data points into its
real spectrum and therefore avoids the problem of
redundancy
Also, as DCT is derived from DFT, all the
desirable properties of DFT (such as the fast
algorithm) are preserved.
One-dimensional DCT
Definition: Let n be a positive integer. The one-
dimensional DCT of order n is defined by an n
x n matrix C whose entries are
n
jiaC iij
2
)12(cos
To derive the DCT of an N-point real signal
sequence we first construct a new
sequence of points:
DFT and DCT basics
DFT AND DCT BASICS
1D-DFT1D-DCT
DCT (1D)
Discrete cosine transform
The strength of the ‘u’ sinusoid is given by C(u) Project f onto the basis function
All samples of f contribute the coefficient
C(0) is the zero-frequency component – the average value!
6
DCT (1D) Consider a digital image such that one row has the following
samples
There are 8 samples so N=8
u is in [0, N-1] or [0, 7]
Must compute 8 DCT coefficients: C(0), C(1), …, C(7)
Start with C(0)
7
Index 0 1 2 3 4 5 6 7
Value 20 12 18 56 83 10 104 114
DCT (1D)
8
DCT (1D) Repeating the computation for all u we obtain
the following coefficients
9
DCT MATRIX IS ORTHOGONAL:
DCT matrix is orthogonal i.e
and it is real
Now the DCT can be expressed in matrix form as:
Left multiplying both sides by C we will get
this is the inverse DCT: or in component form:
Example: When N=2 , we have ,
m,n=0,1.
and
For N=4,
Assume the signal is then its DCT transform is:
And the inverse transform is
DCT (2D)
The 2D DCT is given below where the definition for alpha is the same as before
For an MxN image there are MxN coefficients
Each image sample contributes to each coefficient
Each (u,v) pair corresponds to a ‘pattern’ or ‘basis function’
12
SEPARABILITY
The DCT is separable The coefficients can be obtained by computing the 1D coefficients for
each row
Using the row-coefficients to compute the coefficients of each column (using the 1D forward transform)
13
INVERTABILITY
The DCT is invertible Spatial samples can be recovered from the DCT coefficients
14
SUMMARY OF DCT
The DCT provides energy compaction
Low frequency coefficients have larger magnitude (typically)
High frequency coefficients have smaller magnitude (typically)
Most information is compacted into the lower frequency coefficients (those coefficients at the ‘upper-left’)
Compaction can be leveraged for compression
Use the DCT coefficients to store image data but discard a certain percentage of the high-frequency coefficients!
JPEG does this15
COMPARED WITH DFT, DCT HAS TWO MAIN
ADVANTAGES:
It is a real transform with better computational efficiency than DFT which by definition is a complex transform.
It does not introduce discontinuity while imposing periodicity in the time signal. In DFT, as the time signal is truncated and assumed periodic, discontinuity is introduced in time domain and some corresponding artifacts is introduced in frequency domain. But as even symmetry is assumed while truncating the time signal, no discontinuity and related artifacts are introduced in DCT.
18
INTRODUCTION TO ADAPTIVE FILTERS
Linear filters :
the filter output is a linear function of the filter input
Design methods:
1 The classical approach
frequency-selective filters such as
lowpass / bandpass / notch filters etc
2 Optimal filter design
Mostly based on minimizing the mean-square value
of the error signal
19
THE FILTERING PROBLEM
Filters may be used for three information-processing tasks
Filtering
Smoothing
Prediction
Adaptive filters are self-designing using a recursive
algorithm
Useful if complete knowledge of environment is not
available a priori
20
WIENER FILTER
work of Wiener in 1942 and Kolmogorov in 1939
it is based on a priori
statistical information
when such a priori
information is not available,
which is usually the case,
it is not possible to design
a Wiener filter in the first
place
21
ADAPTIVE FILTER
The signal and/or noise characteristics are often
nonstationary and the statistical parameters vary
with time
An adaptive filter has an adaptation algorithm, that is
meant to monitor the environment and vary the filter
transfer function accordingly
Based in the actual signals received, attempts to find
the optimum filter design
ADAPTIVE FILTER
The coefficients of an adaptive filter change
in time
Output
signal
Input
signal
Adaptive
algorithm
Criterion of
performance
Filter
structure
23
ADAPTIVE FILTER
In a stationary environment, the filter is expected to
converge, to the Wiener filter
In a nonstationary
environment,
the filter is expected to
track time variations and
vary its filter coefficients
accordingly
24
ADAPTIVE FILTER
The basic operation now involves two
processes :
1. a filtering process, which produces an output signal
in response to a given input signal.
2. an adaptation process, which aims to adjust the
filter parameters (filter transfer function) to the
(possibly time-varying) environment
Often, the (avarage) square value of the error signal
is used as the optimization criterion
BLOCK DIAGRAM OF ADAPTIVE SYSTEM
S(n)+No(n)No(n)
+
-
?
Primary
signal
d(n)
N1(n)
Reference
signaly(n)
output
e(n)
adaptive
26
ADAPTIVE FILTER
• Because of complexity of the optimizing algorithms most adaptive filters are digital filters that perform digital signal processing
When processing
analog signals,
the adaptive filter
is then preceded
by A/D and D/A
convertors.
[1]
27
ADAPTIVE FILTER
• The generalization to adaptive IIR filters leads to stability problems
• It’s common to use
a FIR digital filter
with adjustable
coefficients.
28
APPLICATIONS OF ADAPTIVE FILTERS: SYSTEM IDENTIFICATION
Used to provide a linear model of an unknown plant
Parameters u=input of adaptive filter=input to plant y=output of adaptive filter d=desired response=output of plant e=d-y=estimation error
30
APPLICATIONS OF ADAPTIVE FILTERS: INVERSE MODELING (ADAPTIVE
CHANNEL EQUALIZATION)
Used to provide an inverse model of an unknown plant
Parameters u=input of adaptive filter=output to plant y=output of adaptive filter d=desired response=delayed system input e=d-y=estimation error
Applications: Equalization
ADAPTIVE ARRAY ANTENNA
Adaptive Arrays
Linear Combiner
Interference
SMART ANTENNAS
Adaptive Array Antenna
APPLICATIONS OF ADAPTIVE FILTERS: INTERFERENCE CANCELLATION
Used to cancel unknown interference from a primary signal
Applications:
Echo / Noise cancellation
hands-free car phone, aircraft headphones etc
Parameters
u=input of adaptive filter=reference signal
y=output of adaptive filter
d=desired response=primary signal
e=d-y=estimation error=system output
EXAMPLE: ACOUSTIC ECHO CANCELLATION
Applications are manyDigital Communications (OFDM , MIMO , CDMA, and RFID)Channel EqualisationAdaptive noise cancellationAdaptive echo cancellationSystem identificationSmart antenna systemsBlind system equalisationAnd many, many others
ADAPTIVE ALGORITHM
An adaptive algorithm is used to estimate a time varying signal.
By adjusting the filter coefficients so as to minimize the error.
There are many adaptive algorithms like Recursive
Recursive Least Square (RLS), Kalmanfilter,
but the most commonly used is the Least Mean Square (LMS) algorithm.
LMS ALGORITHM Estimates the
solution to the Weiner-Hopfequations using gradient descent method which finds minima by estimating the gradient.
is the step size
X(n)
C(n)
Transversal
Filter
LMS
Y(n)
e(n)
d(n)
CONT..
Update the coefficients using
the following computation.
Filtering operation with the
previous version of the
coefficients. Compare the computed output
with the expected output.
e(n)
X(n) y(n)
d(n)
Adaptive
filter
Unknown
system
LMS ALGORITHM
• Most popular adaptation algorithm is LMS
Define cost function as mean-squared error
• Based on the method of steepest descent
Move towards the minimum on the error surface to get
to minimum gradient of the error surface estimated at
every iteration
LMS ADAPTIVE ALGORITHM
• Introduced by Widrow & Hoff in 1959
• Simple, no matrices calculation involved in the adaptation
• In the family of stochastic gradient algorithms
• Approximation of the steepest – descent method
• Based on the MMSE criterion.(Minimum Mean square Error)
• Adaptive process containing two input signals:
• 1.) Filtering process, producing output signal.
• 2.) Desired signal (Training sequence)
• Adaptive process: recursive adjustment of filter tap weights
50
The LMS Algorithm consists of two basic processes
Filtering process
Calculate the output of FIR filter by convolving
input&taps
Calculate estimation error by comparing the output to
desired signal
Adaptation process
Adjust tap weights based on the estimation error
LMS ALGORITHM STEPS
Filter output
Estimation error
Tap-weight adaptation
51
1
0
*M
k
k nwknuny
nyndne
neknunw1nw *kk
signal
error
vector
input
tap
parameter
rate
-learning
vector
weight-tap of
value old
vector
weigth-tap of
value update
STABILITY OF LMS
The LMS algorithm is convergent in the mean square
if and only if the step-size parameter satisfy
Here max is the largest eigenvalue of the correlation
matrix of the input data
More practical test for stability is
Larger values for step size
Increases adaptation rate (faster
adaptation)
Increases residual mean-squared error