Filter (Video)

Presented By:

Dr. S. K. SinghDepartment of Computer Engineering,Indian Institute of Technology (B.H.U.)

Varanasi-221005.

Filter (Video)

• A video filter is a software component that is used to decode audio and video.

• Multiple filters can be used in a filter chain, in which each filter receives input from its previous-in-line filter upstream, processes the input and outputs the processed video to its next-in-line filter downstream.

• Such a configuration can be visualized in a filter graph.

Applications• Applying filters to video clips : like Photoshop

filter to a video clip, image, or graphic.• Video Editor filters like de-interlace, brightness,

contrast : Automatic Filters: Automatic filters are balanced

filters that automatically improve the overall appearance of videos.

Professional Filters: Professional filters enable to manually improve the appearance of videos by selecting the required level and settings of the enhancement.

Visual Effects: Video Editor provides several special effects that can make your video more unique and professional-looking.

Filter Category

With regards to video encoding three categorized of filters can be distinguished:

•Pre-filters: used before encoding.•Intra-filters: used while encoding (and are thus an integral part of a video codec).•Post-filters: used after decoding.

Pre-filters

Common pre-filters include:

•De-noising•Resizing•Contrast Enhancement•De-interlacing •De-flicking

Intra-filters

Common intra-filters include

•De-blocking

Post-filters

Common post-filters include:

•De-interlacing

•De-blocking

•De-ringing

Video De-noising

Video de-noising is the process of removing noise from a video signal.

Before de-noising After de-noising

Cont…

Video de-noising methods can be divided into:• Spatial video de-noising methods, where image noise

reduction is applied to each frame individually.• Temporal video de-noising methods, where noise

between frames is reduced. Motion compensation may be used to avoid ghosting artefacts when blending together pixels from several frames.

• Spatial-Temporal video de-noising methods use a combination of spatial and temporal de-noising. This is often referred to as 3D de-noising.

Resizing • Video resizing can be an integral part of video

editing. Sometimes it is needed to re-size a video while editing so that it fits certain screen sizes.

• Resizing means up-sampling, down-sampling of signal/ video here.

Up-Sampling

• Up-sampling is interpolation, applied in the context of digital signal processing and sample rate conversion.

Cont.

Before Up-Sampling After Up-Sampling

Down-sampling

• Down-sampling is the process of reducing the sampling rate of a signal. This is usually done to reduce the data rate or the size of the data

Down-Sampling

Down-sampled image/video: used mainly in compression

De-interlacing• De-interlacing is the process of converting

interlaced video, such as common analog television signals, into a non-interla.

• In analog television, each frame is divided into two consecutive fields, one containing all even lines, another with the odd lines. The fields are captured in succession at a rate twice that of the nominal frame rate.

• This process of dividing frames into half-resolution fields at double the frame rate is known as interlacing.

Cont.

De-interlacing Methods

De-interlacing requires the display to buffer one or more fields and recombine them into full frames.De-interlacing methods are as follows : Field combination de-interlacing Field extension de-interlacing Motion detection

De-Flicking

• In video processing, de-flicking is a filtering operation applied to brightness flicker in video to improve visual quality.

• The main idea is to smooth image brightness between series of the same scene frames.

• The deflicking filter is usually used in video camera (for normalizing picture), used for postprocessing of captured video, and for restoration of video from old films.

Cont.

De-blocking

• A de-blocking filter is a video filter applied to decoded compressed video to improve visual quality and prediction performance by smoothing the sharp edges which can form between macro-blocks when block coding techniques are used.

• The filter aims to improve the appearance of decoded pictures.

H.264 De-blocking Filter

• It is a feature on both the decoding path and on the encoding path, so that the in-loop effects of the filter are taken into account in reference macro-blocks used for prediction.

• When a stream is encoded, the filter strength can be selected, or the filter can be switched off entirely.

• The filter strength is determined by coding modes of adjacent blocks, quantization step size, and the steepness of the luminance gradient between blocks.

De-blocking Filters as Post-processors

• The use of a de-blocking filter as a post-processing technique to improve the visual quality of decoded pictures was already a well-known technology.

• This was particularly true in the video conferencing industry, where the low bit rates used tended to produce significant blocking artifacts that could be substantially reduced by such a filter.

De-Ringing

• The blocking artifact is one major issue of the block based coding technique. It appears as intensity discontinuities in the flat areas of the decoded images.

• Ringing effect appears in images as oscillations near sharp edges. It is a result of a cut-off of high-frequency information. Ringing can appear as a result of image compression, image up-sampling and other applications.

Some Widely Used Video Filters :

• Kalman Filter

• Wiener Filter

• Bilateral Filter

Kalman Filter• A Kalman filter is an optimal estimator.-Infers parameters

of interest from indirect, inaccurate and uncertain observations.

• Linear Quadratic Estimation (LQE)

• It is an algorithm that uses a series of measurements observed over time, containing noise (random variations) and other inaccuracies, and produces estimates of unknown variables that tend to be more precise than those based on a single measurement alone.

• It operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state

Cont.

• If all noise is Gaussian, the Kalman filter minimises the mean square error of the estimated parameters.

• If noise is not Gaussian, Kalman filter is the best linear estimator. Non-linear estimators may be better.

Advantages of Kalman Filtering

• Good results in practice due to optimality and structure.

• Convenient form for online real time processing.

• Easy to formulate and implement given a basic understanding.

• Measurement equations need not be inverted

Applications of Kalman Filter

• Guidance, navigation and control of vehicles, particularly aircraft and spacecraft.

• Determination of planet orbit parameters from limited earth observations.

• Time series analysis used in fields such as signal processing and econometrics.

• Robot Localisation and Map building from range sensors/ beacons.

Steps of Kalman FilterThe algorithm works in a two-step process.• Prediction step, the Kalman filter produces estimates of

the current state variables, along with their uncertainties.

• The next measurement is observed, these estimates are updated using a weighted average, with more weight being given to estimates with higher certainty.

• it can run in real time using only the present input measurements and the previously calculated state and its uncertainty matrix; no additional past information is required.

Wiener Filter

• The Wiener filter is a filter used to produce an estimate of a desired or target random process by linear time-invariant filtering an observed noisy process, assuming known stationary signal and noise spectra, and additive noise.

• The Wiener filter minimizes the mean square error between the estimated random process and the desired process.

Theory

• The inverse filtering is a restoration technique for de-convolution.

• When the image is blurred by a low-pass filter, it is possible to recover the image by inverse filtering or generalized inverse filtering.

• But inverse filtering is very sensitive to additive noise.

• The approach of reducing one degradation at a time allows us to develop a restoration algorithm for each type of degradation and simply combine them.

Cont.

• The Wiener filtering executes an optimal trade-off between inverse filtering and noise smoothing.

• It removes the additive noise and inverts the blurring simultaneously.

• It minimizes the overall mean square error in the process of inverse filtering and noise smoothing and linear estimation of the original image.

• The approach is based on a stochastic framework.

Description of Wiener FilterWiener filters are characterized by the following:• Assumption: Signal and (additive) noise are

stationary linear stochastic processes with known spectral characteristics or known autocorrelation and cross-correlation.

• Requirement: The filter must be physically realizable/causal (this requirement can be dropped, resulting in a non-causal solution).

• Performance criterion: Minimum mean-square error (MMSE).

Applications of Wiener Filter

• The Wiener filter can be used in image processing to remove noise from a picture/video.

• De-noise audio signals.

• As a pre-processor before speech recognition.

Kalman Filtering Vs. Wiener Filtering• Wiener Filtering: Filtering, smoothing and

prediction (wide-sense stationary signals) in sequential LMMSE framework.

• Kalman Filtering: Generalization of Weiner filtering to (non- stationary signals) sequential MMSE estimator of a signal in noise, where signal characterized by a dynamical model.

Bilateral filter• A bilateral filter is a non-linear, edge-preserving and

noise-reducing smoothing filter for images.• The intensity value at each pixel in an image is

replaced by a weighted average of intensity values from nearby pixels.

• This weight can be based on a Gaussian distribution.• The weights depend not only on Euclidean distance of

pixels, but also on the radiometric differences.• This preserves sharp edges by systematically looping

through each pixel and adjusting weights to the adjacent pixels accordingly.

Applications of Bilateral filter• For the display of high-dynamic-range

images/video.

• For gray and colour images/video.

• For adaptive smoothing and the nonlinear diffusion equation.

• For mesh de-noising.

THANK YOU.