IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 56, NO. 1, JANUARY 2009 51

A VLSI Implementation of Barrel DistortionCorrection for Wide-Angle Camera ImagesPei-Yin Chen, Member, IEEE, Chien-Chuan Huang, Yeu-Horng Shiau, and Yao-Tung Chen

Abstract—Wide-angle cameras are widely used in surveillanceand medical imaging applications nowadays. Images captured bywide-angle lens suffer from barrel distortion which means that theouter regions of the image are compressed more than the inner one.A low-cost high-speed VLSI implementation for barrel distortioncorrection is presented in this brief. In our simulation, the pro-posed circuit can achieve 200 MHz with 45 K gate counts by usingTSMC 0.18 m technology. Compared with the previous distor-tion correction design, our circuit requires less hardware cost andachieves faster working speed.

Index Terms—Barrel distortion correction, VLSI architecture,wide-angle.

I. INTRODUCTION

I N RECENT years, the wide-angle camera is widely used in

For example, in surveillance applications we can capture the en-tire region of interest with cameras as few as possible if cam-eras with large field of view are adopted. In clinical endoscopes,cameras with wide-angle lens (fish-eye lens) can provide moreinformation about the internal structure of the gastrointestinaltract in a single image. However, images captured by wide-anglelens suffer from barrel distortion. Barrel distortion shows non-linear changes in the image where the outer regions of the imageare compressed more than the regions near the distortion center.

Many researchers have proposed various techniques to cor-rect barrel distortion [1]–[9]. For real-time surveillance andmedical applications, high-speed solutions for barrel distortioncorrection are necessary and must be considered. Some VLSIarchitectures [10], [11] of barrel distortion correction havebeen presented recently. In [10], Asari proposed an efficientVLSI architecture for non-linear spatial warping of wide-anglecamera images. Using two efficient coordinate rotation digitalcomputer modules, Ngo et al. presented a pipelined architecturefor real-time correction of barrel distortion [11].

The distortion correction circuit may be included in end-usercamera equipments, so how to implement it with lower hardwarecost is an important issue. In this brief, we present a 21-stage

Manuscript received April 07, 2008; revised August 07, 2008. Current versionpublished January 16, 2009. This work was supported in part by the NationalScience Council, R.O.C, under Grant NSC-96-2622-E-006-018-CC3. Also, thiswork made use of Shared Facilities supported by the Program of Top 100 Uni-versities Advancement, Ministry of Education, Taiwan. This paper was recom-mended by Associate Editor A.-Y. Wu.

The authors are with the Digital IC Design Laboratory, Department of Com-puter Science and Information Engineering, National Cheng Kung University,Tainan 701, Taiwan (e-mail: pychen@csie.ncku.edu.tw).

Digital Object Identifier 10.1109/TCSII.2008.2010165

pipelined architecture for barrel distortion correction. The cor-rection technique implemented in the proposed architecture isbased on the least-square estimation method presented in [3],[11]. By eliminating the calculation of angle and adoptingthe odd-order back-mapping polynomial, our high-performancedistortion correction design can be implemented with very lowcost. As compared with [11], our implementation reduces about61% of logic cells for Altera EP20K600EBC652-1X FPGA,and achieves 133% throughput improvement. The proposed cor-rection circuit can be implemented with a dedicated chip orbe integrated with other image processing components, suchas noise removal and image enhancement, on a single chip forwide-angle camera applications.

The rest of this brief is organized as follows. The least-squareestimation method is introduced briefly in Section II. Section IIIdescribes the proposed VLSI architecture in detail. Section IVillustrates the VLSI implementation and comparisons. The con-clusion is provided in Section V.

II. DISTORTION CORRECTION TECHNIQUE

In this section, the distortion correction technique basedon the least-square estimation method [3], [11] is introducedbriefly. Let the distorted and corrected image spaces are rep-resented by DIS and CIS, respectively. The distortion centerin DIS is and the correction center in CIS is .

is the Cartesian and is the polar coordinates ofa pixel in DIS. Thus, the distance and the angle from thedistortion center to are given by

(1)

(2)

Let the same pixel be assigned a new location in CIS. Thedistance and the angle from the correction center to

are given by

(3)

(4)

Barrel distortion can be corrected by two main tasks: 1) backmapping of all pixels in CIS onto DIS, and 2) calculating theintensity of every pixel in CIS by linear interpolation [11]. Fig. 1shows block diagram of the distortion correction procedure. Thefirst step is to transform the Cartesian coordinate of the pixel tobe processed in CIS to its polar coordinate (see (3), (4)). Then,the back-mapping module will map the pixel in CIS to

1549-7747/$25.00 © 2009 IEEE

Authorized licensed use limited to: Hindusthan College of Engineering and Tech. Downloaded on August 24, 2009 at 02:39 from IEEE Xplore. Restrictions apply.

many fields, ranging from surveillance to medical imaging.

52 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 56, NO. 1, JANUARY 2009

TABLE IINPUT VALUES AND OUTPUT EQUATIONS OF DISTORTION CORRECTION PROCEDURE PROPOSED IN [11]

Fig. 1. Block diagram of distortion correction procedure proposed in [11].

the corresponding location in DIS. As mentioned in [3],[11], the back-mapping process can be described by a back-mapping expansion polynomial of degree as

(5)

(6)

where ’s are the back-mapping coefficients that can be ob-tained by using the least-square estimation, and angles andare the same since the distortion is assumed to be radial aboutthe distortion center. The third step is to perform the polar toCartesian coordinate conversion. The location in DIScan be calculated as

(7)

(8)

The pixel location need not be an integer location.Its intensity value can be calculated by linearly interpolatingwith the intensity values of the four neighboring pixels around

in DIS. Let and represent the integer parts of ,and be defined as

(9)

(10)

and represent the fractional parts of , and be definedas

(11)

(12)

Thus, the four neighboring pixels around are ,, and . The final intensity value of

is given by

(13)

where the intensities of the four neighboring pixels in DIS aredenoted as , , andrespectively.

III. THE PROPOSED VLSI ARCHITECTURE

The input values and output equations for the four-step cor-rection procedure, shown in Fig. 1, are summarized in Table Iwhere back-mapping coefficients , distortion center

and corrected center are given inputs, and thefinal output result is . In [11], Ngo et al. implementedthe four-step correction procedure with four dedicated hardwaremodules. In their design, the coordinate transformation is re-alized with one unrolled pipelined coordinate rotation digitalcomputer (CORDIC) module, which is a widely used methodfor calculating various trigonometric and transcendental func-tions by rotating vectors. In practical hardware implementation,the CORDIC module performs coordinate transformation byusing a series of shift-and-add operations [11], [12]. The draw-back is that the CORDIC module is usually implemented withmany iterations to obtain acceptable calculation precision at thecost of high hardware cost.

To achieve the goal of low-cost VLSI implementation, twoprocesses of reduction are considered in our design. First, weeliminate the calculation of . Since , and (inthe third step of Table I) can be calculated as

(14)

(15)

Thus, Table I can be reduced and rewritten as Table II. Obvi-ously, the calculation of angle is removed. Considering VLSIimplementation of Table II, we find that the hardware moduleused to calculate is the only one on which we have to focuson because other values ( , and ) can be calcu-lated with simple arithmetic operations ( ). Using thesquare-root approximation (SRA) technique suggested in [13],we can calculate as

(16)where and

. Hence, no complex CORDIC module is needed in our de-sign, so the whole hardware cost can be reduced largely.

Authorized licensed use limited to: Hindusthan College of Engineering and Tech. Downloaded on August 24, 2009 at 02:39 from IEEE Xplore. Restrictions apply.

CHEN et al.: VLSI IMPLEMENTATION OF BARREL DISTORTION CORRECTION FOR WIDE-ANGLE CAMERA IMAGES 53

TABLE IIINPUT VALUES AND OUTPUT EQUATIONS FOR DISTORTION CORRECTION AFTER REMOVING �

TABLE IIIINPUT VALUES AND OUTPUT EQUATIONS OF OUR

DISTORTION CORRECTION PROCEDURE

Secondly, we consider possible reduction of the back map-ping module to further reduce implementation cost. In some lit-eratures [7], [8], the back-mapping expansion polynomial forradial distortion is approximated by

(17)

(18)

where are the back-mapping coefficients of the odd-order polynomial. Based on the expression, the output equationsof the third step in Table II can be modified and given as

(19)

(20)

Observing (19) and (20), we find that no odd power of exists,so the square-root calculation for can be removed. We reduceTable II and propose a two-step distortion correction procedure,as illustrated in Table III. It can be seen that the first three steps inTable II are combined into one step in Table III, and the complexsquare-root calculation is eliminated because both and arecalculated with rather than with .

Fig. 2 shows the state flow chart of our two-step distortioncorrection procedure. We assume that no serial execution of twoor more operations in each state is allowed. In our current im-plementation, four coefficients for the odd-order back-mapping

polynomial are adopted. is determined in state , andis generated in . By em-

ploying pipeline scheduling approaches, we develop a low-cost21-stage pipelined VLSI architecture based on Fig. 2. Fig. 3shows the block diagram of our architecture. When the startsignal is enabled, the circuit will output the intensity value of thefirst pixel after 21 clock cycles. Then, it can process one pixelin CIS per clock cycle. Totally, it would take

cycles to process an image with 1024 1024 pixels. Ourdesign consists of four main modules: mapping unit, memorybank, linear interpolation unit and controller. The mapping unitperforms the calculation of as illustrated in Table III.The memory bank is composed of four duplicated RAMs usedto provide the intensity values of the four neighboring pixelsaround simultaneously. The size of each RAM is equalto that of input image which is set as 1024 1024 bytes in thecurrent implementation. The interpolation unit linearly interpo-lates the final intensity value. The controller provides the controlsignals of each state to other units and handles the whole cor-rection procedure. The detailed circuits of mapping and linearinterpolation units are described as follows.

A. Mapping Unit

For each pixel in CIS, the mapping unit performs theoperations needed to calculate (from state to statein Fig. 2). Most multipliers and adders are realized with 24-bitwidth as adopted in [11]. According to the simulation resultsobtained from DesignWare of SYNOPSYS [14], we found thatthe propagation delay of a 24-bit multiplier is quite long. Hence,we adopted the 24-bit two-stage pipelined multiplier in the de-sign to get better pipeline scheduling. Fig. 4 shows the 15-stagepipelined architecture of the mapping unit. In the figure, theshadowed rectangles are registers, the ellipses marked withare two-stage pipelined multipliers, and the variables are iden-tical to those shown in Fig. 2.

B. Linear Interpolation Unit

The linear interpolation unit obtains the intensity values ofthe four neighboring pixels around and calculates thefinal intensity value . Fig. 5 shows the 6-stage pipelinedarchitecture of the linear interpolation unit which performs theoperations needed to calculate (from state to state

in Fig. 2). At every clock cycle, four neighboring intensities

Authorized licensed use limited to: Hindusthan College of Engineering and Tech. Downloaded on August 24, 2009 at 02:39 from IEEE Xplore. Restrictions apply.

54 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 56, NO. 1, JANUARY 2009

Fig. 2. State flow chart of our two-step distortion correction procedure.

Fig. 3. Block diagram of our VLSI architecture.

are accessed simultaneously and one output intensity value isgenerated accordingly.

IV. VLSI IMPLEMENTATION AND COMPARISON

The VLSI architecture of our design was implemented byusing Verilog HDL. We used SYNOPSYS Design Vision to syn-

Fig. 4. 15-stage pipelined architecture of mapping unit.

Fig. 5. 6-stage pipelined architecture of linear interpolation unit.

TABLE IVSIMULATION RESULTS OF TWO DESIGNS USING

ALTERA EP20K600EBC652-1X FPGA

thesize the design with TSMC’s 0.18 m cell library. The layoutfor the design was generated with SYNOPSYS Astro (for autoplacement and routing), and verified by MENTOR GRAPHICCalibre (for DRC and LVS checks). Synthesis results show thatour design contains 44992 gate counts. It works with a clockperiod of 5 ns and operates at a clock rate of 200 MHz.

Furthermore, we have implemented and verified the ar-chitecture on a FPGA emulation board. Table IV lists theimplementation results of our design and the previous distor-tion correction design [11]. Obviously, our circuit requires lesshardware cost but achieves higher clock rate using the identical

Authorized licensed use limited to: Hindusthan College of Engineering and Tech. Downloaded on August 24, 2009 at 02:39 from IEEE Xplore. Restrictions apply.

CHEN et al.: VLSI IMPLEMENTATION OF BARREL DISTORTION CORRECTION FOR WIDE-ANGLE CAMERA IMAGES 55

Fig. 6. Simulation results for the barrel distorted images (1st row), the cor-rected images obtained from [11] (2nd row), and the corrected images obtainedfrom our design (3th row).

Altera EP20K600EBC652-1X FPGA. Fig. 6 shows the barreldistorted images (the first row) obtained from the wide-anglecamera (eMPIA 2750 of eGalax_eMPIA Technology Incor-poration, http://www.eeti.com.tw/), and the corrected imagesobtained both from [11] and the implementation of our architec-ture, respectively (the second and third rows). We can concludethat our design is a good candidate for low-cost high-speeddistortion correction circuit.

V. CONCLUSIONS

In this brief, we propose a low-cost real-time pipelined archi-tecture for barrel distortion correction based on the least-squareestimation method. Our architecture needs less hardware costbut achieves faster working speed as compared with the pre-vious design [11]. It proves to be a good candidate for low-costhigh-performance distortion correction circuit.

REFERENCES

[1] J. Weng, P. Cohen, and M. Herniou, “Camera calibration with distor-tion models and accuracy evaluation,” IEEE Trans.Pattern Anal. Mach.Intell., vol. 14, no. 10, pp. 965–980, Oct. 1992.

[2] Y. Nomura, M. Sagara, H. Naruse, and A. Ide, “Simple calibrationalgorithm for high-distortion-lens camera,” IEEE Trans.Pattern Anal.Mach. Intell., vol. 14, no. 11, pp. 1095–1099, Nov. 1992.

[3] V. K. Asari, S. Kumar, and D. Radhakrishnan, “A new approachfor nonlinear distortion correction in endoscopic images based onleast squares estimation,” IEEE Trans. Med. Imag., vol. 18, no. 4, pp.345–354, Apr. 1999.

[4] R. Swaminathan and S. K. Nayar, “Nonmetric calibration of wide-angle lenses and polycameras,” IEEE Trans. Pattern Anal. Mach. In-tell., vol. 22, no. 10, pp. 1172–1178, Oct. 2000.

[5] J. P. Helferty, C. Zhang, G. McLennan, and W. E. Higgins, “Videoen-doscopic distortion correction and its applications to virtual guidanceof endoscopy,” IEEE Trans. Med. Imag., vol. 20, no. 7, pp. 605–617,Jul. 2001.

[6] S. Thirthala and M. Pollefeys, “The radial trifocal tensor: A toolfor calibrating the radial distortion of wide-angle cameras,” in Proc.CVPR’05, 2005, pp. 321–328.

[7] M. T. El-Melegy and A. A. Farag, “Nonmetric lens distortion calibra-tion: Closed-form solutions, robust estimation and model selection,” inProc. IEEE ICCV2003, 2003, vol. 2, pp. 1554–1559.

[8] M. Marder, “Comparison of Calibration Algorithms for a Low-Reso-lution, Wide Angle, 3-D Camera,” M.Sc. thesis, Stockholm, Sweden,2005.

[9] J. Kannala and S. S. Brandt, “A generic camera model and calibrationmethod for conventional, wide-angle, and fish-eye lenses,” IEEE Trans.Pattern Anal. Mach. Intell., vol. 28, no. 8, pp. 1335–1340, Aug. 2006.

[10] V. K. Asari, “Design of an efficient VLSI architecture for non-linearspatial warping of wide-angle camera image,” J. Syst. Architec., vol.50, pp. 743–755, Aug. 2004.

[11] H. T. Ngo and V. K. Asari, “A pipelined architecture for real-time cor-rection of barrel distortion in wide-angle camera images,” IEEE Trans.Circuits Syst. Video Technol., vol. 15, no. 3, pp. 436–444, Mar. 2005.

[12] J. D. Bruguera, N. Guil, T. Lang, J. Villalba, and E. L. Zapata,“CORDIC based parallel/pipelined architecture for the Hough trans-form,” J. VLSI Signal Process., pp. 207–221, Jan. 2001.

[13] D. D. Gajski, Principles of Digital Design. Upper Saddle River, NJ:Prentice-Hall, 1997.

[14] Synopsys Inc., “DesignWare Building Block IP,” [Online]. Available:http://www.synopsys.com

Authorized licensed use limited to: Hindusthan College of Engineering and Tech. Downloaded on August 24, 2009 at 02:39 from IEEE Xplore. Restrictions apply.