Download - IMAGE SCALING ALGORITHMS.pptx
IMAGE SCALING ALGORITHMSBy SUMEET SAURAV
INTRODUCTION: THE INTERPOLATION PROBLEM
Assume a unit square between the four nearest pixels.
The target pixel will then be some x fraction and y fraction into this unit square.
Interpolation problem then consists of finding suitable values for these fractions.
NO NEW DATA will be added by ANY interpolation scheme! The best you can do is minimize interpolation artifacts.
CATEGORIES OF INTERPOLATION METHOD
Interpolation is done by two methods: By interpolating the pixels based on the characteristics of
local features, such as edge information or neighboring pixel information.
Based on these differences interpolation method can be classified into two categories:
A)Adaptive
B)Non-adaptive
Non-adaptive is suitable for real time hardware implementation.
TYPES OF INTERPOLATION
Nearest Neighbor Bilinear Interpolation. Bicubic convolution Interpolation. Cubic convolution Interpolation Extended Linear Interpolation Win Scale Area Average Lanczos resampling Hqx(Pixel art scaling algorithm) Super scaling Vectorization
NEAREST NEIGHBOR INTERPOLATION Every pixel value in the output product is set
to the nearest input pixel value.Pros:1)Very simple and fast.2) No new values are calculated by interpolationCons:1) Some pixels get lost and others are
duplicated.2) Loss of sharpness.
BILINEAR INTERPOLATION Calculation of the new pixel value is performed by the
weight of the four surrounding pixels.
Pros:
1) Extremas are balanced
2)Image losses sharpness compared to Nearest Neighbor.
Cons:
1) Less contrast compared to Nearest Neighbor.
2) New values are calculated which are not present in the input product.
CUBIC CONVOLUTION Calculation of the new pixel value is performed by
weighting the 16 surrounding pixels.
Pros:
1. Extremas are balanced.
2. Image is sharper compared to Bi-linear Interpolation
Cons:
3. Less contrast compared to Nearest Neighbor.
4. New values are calculated which are not present in the input product
5. Slow, compared to Nearest Neighbor resampling
WINSCALE:AN IMAGE SCALING ALGORITHM
The winscale algorithm uses an area pixel model and a domain filtering method for image scaling.
The algorithm uses an area-model rather than a point-model for a pixel evaluating.
The pixel shapes of the original and the scaled image are treated as rectangles and pixel intensity is evenly distributed in rectangle area.
New pixel of the scaled image is obtained by weighted-averaging the pixel values of the original pixels with area coverage ratio.
A source image is the original image before scaling up/down. Target image refers to the scaled image. The region of the target pixel being calculated currently is
called a filter window.
IMAGE UP-DOWN SCALE
If the filter window is perfectly included in one source pixel region, sampling has the nearest neighbor property: the pixel value becomes equal to that of source pixel.
This property makes good high frequency response and reduces blurring effect.
If the filter window is spread over multiple source pixels, sampling has the linear property like bilinear: smoothness is added.
Therefore, winscale inherits two good properties of both nearest and linear: good high frequency response and good smoothness.
EDGE ORIENTED IMAGE SCALING Efficient area-pixel scaling technique is used. A simple edge catching technique is adopted to preserve
the image edge features.
When up-scaled the area of the target pixel is less than that of the source pixel.
A window is put on the current target pixel to calculate its estimated luminance value.
The number of source pixels overlapped by the current target pixel window is one, two, or a maximum of four.
Let the luminance values of the source pixels overlapped by the window of the current target pixel at (k,l) is given by
The estimated value of the current target pixel is given by
The areas of the overlapped regions can be calculated as:
Where
[A(m,n),A(m 1,n),A(m,n 1),A(m 1,n 1)]
[ ( , ) top(k, l), right(k, l) top(k, l), left(k, l) bottom(k,l),
right(k, l) bottom(k,l)]
left k l
( , ) (m,n)
top(k, l) heightA(m,n)
right(k, l) widthA(m 1,n 1)
bottom(k,l) height(m 1,n 1)
left k l widthA
The area-pixel scaling algorithm requires total of 13 addition,8 multiplications and one division to calculate one target pixel.
The proposed low-complexity algorithm uses the following set of equations:
where
[A'(m,n),A'(m 1,n),A'(m,n 1),A'(m 1,n 1)]
[left'(k, l) top'(k, l), right'(k, l) top'(k, l),
left'(k, l) bottom'(k, l), right'(k, l) bottom'(k, l)]
[ '( , ), top'(k, l), right'(k, l), bottom'(k, l)]
Appr[left(k, l), top(k, l), right(k, l), bottom(k,l)]
left k l
Here a low-cost edge-catching technique is employed to preserve the edge features to obtain better visual quality.
The final areas of the overlapped regions are given as
Where the tuning operator is used to tune the areas of four overlapped regions according to the edge features obtained by edge-catching technique.
[ ''( , ), ''( 1), ''( , 1), ''(m 1,n 1)]
([A'(m,n),A'(m 1,n),A'(m,n 1),A'(m 1,n 1)]
A m n A m A m n A
Assuming the width and height of the target pixel window are denoted as win(w) and win(h)
Here win(h) and win(w) are determined based on current magnification factors (mf_w for x-direction and mf_h for y-direction)
The value of mf_w and mf_h is given by
( ) ( )Asum win w win h
_ /
_ /
mf w TW SW
mf h TH SH
'( , ) ( ) '( , )
'( , ) ( ) '( , )
right k l win w lefr k l
bottom k l win h top k l
The value of left’(k,l) is determined from the fig
'( , ) min(SRCright(m,n) ( , ), ( ))left k l WINleft k l win w
DETERMINATION OF AND
Initially we have
Both Sw and Sh are rounded to integers.
(0,0) ( ) / 2
SRCright(0,0) Sw
WINtop(0,0) ( ) / 2
WINleft Sw WINw
Sh WINh
SRCbtm Sh
w h
OVERLAPPING METHOD FOR ROUNDING DOWN ERROR Let represent the regulating times
required for each row, thus it can be given asrw
OVERLAPPING METHOD FOR ROUNDING UP ERROR
THE LOW-COST EDGE-CATCHING TECHNIQUE Sigmoidal signal is taken as image-edge
model for image scaling. E(k) represents the luminance value of the
pixel at coordinate k. e
An Image-Edge Model Local Characteristics
The estimated value of the pixel to be interpolated is determined by using linear interpolation.
The interpolated value might not match greatly with E(k).
To solve this problem the distance s is modified to make approach E(k) for better estimation.
Here an evaluating parameter L is defined to estimate the local characteristics of the data in the neighborhood of k. It is given as
L=0 indicates symmetry, so s is unchanged. L>0 indicates that variation between E(m+1)
and E(m-1) is quicker than between E(m+2) and E(m).
It means that edge is more homogeneous to the right side.
L<0 indicates the edge is more homogeneous on the left side so s must be decreased to obtain a better estimation
The modified equation for estimating the interpolated pixel value is given by
Where s’ is calculated with the following equation
The same concept of 1-D edge-catching technique can be used in the proposed 2-D scaling processor.
Let be the evaluating parameter to estimate the local characteristics of the current pixel at the coordinate(k,l).
If top’(k,l) is greater than or equal to win(h)/2 it means that A’(m,n) is bigger than or equal to A’(m,n+1).
Hence the upper row(n) is more important than the lower row(n+1) to catch edge features.
Thus is given by
If =0 indicates symmetry, so A’(m,n) is unchanged.
If >0 indicates that the variation between E(m+1,n) and E(m-1,n) is quicker than between E(m+2,n) and E(m,n).
It means that edge is more homogeneous on the right hand side so we increase A’(m+1,n).
If <0 the edge is more homogeneous on the left-hand side ,thus we decrease A’(m+1,n) to obtain a better estimate.
Based on the above rules the final areas of the overlapped region is given by:
Where Ac=A’(m,n) if and Ac=A’(m+1,n) if If top’(k,l) is less than win(h)/2 ,it means that
A’(m,n) is smaller than A’(m,n+1).Hence the lower row(n+1) is more important than the upper row(n) to catch the edge features.
Thus is given by
0LA0LA
The final area of the overlapped regions are given by
Where Ac=A’(m,n+1) if and Ac=A’(m+1,n+1) if
0LA
0LA
VLSI ARCHITECTURE
Approximate module
Edge catcher Area
generator
Area tuner
Target generato
r
ARCHITECTURE OF REGISTER BANK
ARCHITECTURE OF AREA GENERATOR
ARCHITECTURE OF EDGE CATCHER
ARCHITECTURE OF AREA TUNER
ARCHITECTURE OF TARGET GENERATOR
AREA-AVERAGE ALGORITHM The algorithm can be done on horizontal and vertical
dimensions separately with minimal correlation which helps to exploit the parallel capabilities of the FPGA.
The scaling algorithm can be performed stepwise and without causing image quality loss.
If we need to reduce a image f(assume f =a/b>1) times, first we magnify each pixel in the original image b times and then shrink the enlarged image a times.
ALGORITHM IMPLEMENTATION
RESOURCE AND SYSTEM PERFORMANCE COMPARISON
BI-CUBIC INTERPOLATION ALGORITHM This algorithm requires 16 source pixels for getting the
interpolated value of the target pixel. It preserves fine details of the image .
Where
32
2
2
( )
1 (1 ) (1 )
2 (1 2 )
3 (1 )
4 (1 )
l round x a x l
t a a a
t a a
t a a a
t a a
32
2
2
( ) b=y-k
t5=-b (1-b) (1-b)*t1
t6=(1-2 )
7 *(1 )
8 *( 1)
k round y
b b
t b b b
t b b
HARDWARE ARCHITECTURE FOR BI-CUBIC INTERPOLATION
The main components of the architecture are: (1) Input Buffer Bank (2) Output Buffer Bank (3) Off-Chip RAM (4) Parallel Zoom
Processors (5) Control Unit (6) Interpolation Coefficient
Generator
The main components of the Zoom Processor are:
(1) Pixel demultiplexer (2) Pixel Registers (3) Coefficient Demultiplexer (4) Coefficient Registers (5) Temporal Registers (6) Output Pixel Register
ZOOM PROCESSOR ARCHITECTURE
THE CONTROL UNIT The unit synchronizes external memory,
input and output buffers bank, zoom processors and interpolation coefficient generator.
For each clock cycle, the Control Unit generates t1, t2, t3 and t4 (vertical coefficient) and t5, t6, t7 and t8 (Horizontal coefficients).
In the cycle L+4, the Zoom Processor 1 captures t1, t2, t3 and t4 and in cycle L+5 output the first result.
The control unit indicates which processor reads the Horizontal coefficients and when all the Zoom Processors has given one result.
PIPELINE SCHEME FOR ZOOM PROCESSOR
INTERPOLATION COEFFICIENT GENERATOR
The Interpolation Coefficient Generator generates dynamically the 4 by 4 input mask that will be used by the Zoom processors to obtain the output pixels.
The Interpolation Coefficient Generator gives a result each clock cycle after 5 latency cycles.
LINEAR SPLINE INTERPOLATION
Given , fit linear splines to the data.
(X 0, Y 0),(X1, Y1),(X 3, Y 3)..........(Xn, Yn)
),()()(
)()( 001
010 xx
xx
xfxfxfxf
),()()(
)( 112
121 xx
xx
xfxfxf
),()()(
)( 11
11
nnn
nnn xx
xx
xfxfxf
10 xxx
21 xxx
nn xxx 1
QUADRATIC SPLINE In these splines, a quadratic polynomial
approximates the data between two consecutive data points.
Given fit quadratic splines through the data.
nnnn yxyxyxyx ,,,,......,,,, 111100
,)( 112
1 cxbxaxf ,22
22 cxbxa
,2nnn cxbxa
10 xxx
21 xxx
nn xxx 1
TWO-POINT CUBIC SPLINE INTERPOLATION The most widely used scaling algorithms in usually
occasion are nearest neighbor, bilinear, and bicubic. Nearest neighbor and bilinear interpolation are simplest
and fastest, but the images come from them are either blocky or over-smoothed.
The bicubic can get a better image quality than them but it is more complicated because it needs a 4x4 neighbor.
The nearest neighbor has good high frequency response and can preserved more edge information of the original image. But it degrades image quality due to aliasing.
The weakness of bilinear and bicubic is blur effect causing bad high frequency response.
The proposed algorithm has a good high frequency response approximated to nearest neighbor and it is as simple and fast as the bilinear.
It can get an image quality as good as the bicubic.
Interpolation Kernel Curve
EXTENDED |LINEAR INTERPOLATION The kernel of the linear interpolation is built
up of first-order polynomials and the weighting coefficient is given by the relation:
The kernel of the cubic convolution interpolation is built up of third-order polynomials whose weighting coefficient is given by
KEYS’ AND BI-CUBIC WEIGHTING COEFFICIENT. Key determined the value of c=-0.5.
Bi-cubic interpolation uses the constant c=-1 and the weighting coefficient of the bi-cubic interpolation is given by
DECOMPOSITION OF THE 2-D BI-CUBIC
THE KERNEL OF EXTENDED LINEAR INTERPOLATION It has low complexity as well as it improves
the quality of the linear interpolation. Here the number of sampling points of the
third order polynomial interpolation is used. The weighting coefficients are given by
DERIVATION OF EL KERNEL FROM BI-CUBIC
Using the coefficient of the Bi-cubic weighting coefficient we have
After certain manipulation the above equation can be modified and the required weighting coefficient is given by
Where is called the sharpness factor.
SHARPNESS FACTOR DECISION
COORDINATE ORIENTATION AND THE INTERVAL CALCULATION Here the origin of a source pixel in source
coordinates is (0.5,0.5),although its location in the memory is at(1,1).
The coordinate of the interpolated pixel in the x-direction and y-direction is
The vertical coordinate of the interpolated pixel(Ym) gives the row address of the source pixel which is determined as follows:
Thereafter the address neighboring pixels are determined as follows:
Similarly column address can be calculated by using Xm of the interpolated pixel.
The distance in vertical and horizontal direction is given by:
REDUCTION OF THE COMPUTATION OF THE WEIGHTING COEFFICIENT
Vertical Weighting Coefficients
Horizontal Weighting Coefficients
DECOMPOSITION OF 2-D 4X4 INTERPOLATION
Vertical interpolation is performed first and four virtual pixels are calculated using four 1-D interpolation. The generated virtual pixel is given by:
Now from the vertically interpolated virtual pixels actual interpolated pixel is obtained by horizontal interpolation using one 1-D horizontal interpolation. The interpolated pixel is given by:
All the interpolated points within the same row of Q have the same vertical weighting coefficients.
These same vertical weighting coefficients are calculated only one time for each row and then proceeds the calculations of horizontal weighting coefficients of each interpolated pixel in that row.
The number of memory access time is given by
THE PROPOSED HARDWARE
THE COORDINATE CALCULATION UNIT
THE WEIGHTING COEFFICIENT GENERATOR
The most computation effort in extended linear interpolation is the calculation of interpolation weighting coefficients.
THE VERTICAL AND HORIZONTAL INTERPOLATION UNIT
THE VIRTUAL PIXEL BUFFER The scale ration determine both input and output data
rates of this buffer. While scaling up the data rate of vertical interpolator is
less than that of the horizontal interpolator in a period of time.
During scaling down the converse case happens.