Ap

PS

a

ARA

KDPJSB

1

s1dtsionrstouleabbd

uc

0d

Optik 122 (2011) 2044– 2049

Contents lists available at ScienceDirect

Optik

j o ur nal homepage: www.elsev ier .de / i j leo

nalysis of black-grating-effect on the displacement measurement based onhase-coded target joint transform correlators

eng Ge, Qi Li ∗, Huajun Feng, Zhihai Xutate Key Lab of Modern Optical Instrumentation, Zhejiang University, Hangzhou 310027, China

r t i c l e i n f o

rticle history:eceived 4 July 2010ccepted 2 October 2010

a b s t r a c t

Black-grating-effect of SLM on the displacement measurement based on phase-coded target jointtransform correlators(PCTJTC) is discussed. Displacement measurement technology based on PCTJTC isproposed. Black-grating-effect is explained and the impact of a pixel’s fill factor is discussed. Pixel seg-mentation is elaborated and simulation results are given. It is shown that black-grating-effect has brought

eywords:isplacement measurementhase-encodedoint transform correlatorpatial light modulator

errors to the detection precision. The optical efficiency is higher when the fill factor is bigger, and theintensity distribution of the spectrum moves closer toward the 0th fringe, which brings down the CCD’sneed of high resolution.

© 2011 Elsevier GmbH. All rights reserved.

lack-grating-effect

. Introduction

American scientist Jutamulia proposed a displacement mea-urement technology based on joint transform correlators (JTC) in992 [1]. Joint transform correlators have been widely applied inisplacement measurement [2], object discrimination [3,4], detec-ion of defects [5], tracking [6], etc. Janschek et al. applied JTC toatellite navigation, space monitoring, attitude determination andmage stabilization [2,7–11]. In Ref. [8] Janschek analyzed sourcesf errors, such as mechanical deformations of the optical system,oise, low percentage of current and reference images, geomet-ical distortions of the current and reference images and discretetructure of spatial light modulator (SLM) and CCD. In Ref. [9] theyook account of the rotation of the light source, inclination of theptical unit, inclination of the image sensor, rotation of the opticalnit and image sensor, longitudinal and transversal motion of the

ight source and image sensor as the sources of errors. Howeverffects caused by the discrete structure of SLM have never beennalyzed by Klaus or others in detail as far as I know. In this paperlack-grating-effect on the errors of displacement measurementased on PCTJTC due to the discrete structure of the SLM will beiscussed.

SLM is a linear or area array consisting of many fundamentalnits. Those units are called pixels. Those pixels can display opti-al or electronic input signals pixel by pixel. The amplitude, phase,

∗ Corresponding author.E-mail address: liqi@zju.edu.cn (Q. Li).

030-4026/$ – see front matter © 2011 Elsevier GmbH. All rights reserved.oi:10.1016/j.ijleo.2010.10.047

intensity, frequency and polarization of the written beam can bemodulated or transformed by the physical effect of the SLM.

In the electro-addressable SLM, there exists insulated electrodebetween adjacent pixels due to its circuit structure. The transmis-sion or reflectivity of light could be zero at these electrodes. Theseelectrodes form a black grating. It consists of two perpendicularcycle rectangle gratings [12]. Due to the two matrix gratings, thenoise generated by the electric circuit has been avoided, but atthe same time optical efficiency has been reduced. The diffractioncaused by the black-grating affects the quality of optical distri-bution and brings trouble to follow-up treatments. This is calledblack-grating-effect [13].

In order to display image seamlessly and modulate it in highquality, the SLM should have high transmission or reflectivity. Themost useful method is to improve the fill factor, which increases theeffective area of a pixel where light can be transmitted or reflected.Other methods include using micro-lens array, beam shaping ele-ments arrays, etc. [13–15].

2. Theory analysis

2.1. The theory of displacement measurement technology basedon phase-coded target JTC (PCTJTC)

Fig. 1 shows the principle of displacement detection technol-

ogy based on phase-coded target JTC (PCTJTC). SLM1 and SLM2 areplaced at front focal planes of Fourier lens 1 and lens 2, CCD1 andCCD2 are placed at their back focal plane correspondingly. First weemploy a vibration platform to simulate the scene with vibration in

P. Ge et al. / Optik 122 (2011) 2044– 2049 2045

aisainjitclti6tGftcfmp

ed

�

Fig. 1. The schematic of the system.

irborne remote sensing system, and a high-speed camera systems set up to grab the target to form a video sequence. In the firsttage, two adjacent frames of this video sequence will be selecteds the reference and target images to form input images displayedn SLM 1. Coherent light, generated by a laser, is used to illumi-ate the input. Processed by FT lens 1 and CCD 1 will capture the

oint power spectrum of the input image. Likewise, this spectrums sent to SLM 2 by computer 2. FFT lens 2 produces the correla-ion output at CCD 2. It can be seen from Fig. 1, Fourier transformalculation of the image is accomplished entirely by the Fourierens whose computing speed is as fast as light, and thus the detec-ion time of the system is greatly reduced. Fig. 2 shows our devicesn this experiment. The wavelength of Laser 1 and Laser 2 is both50 nm, the focal length of each Fourier lens is 300 mm. The spa-ial light modulator named FSLM-HD70-A/P manufactured by Xi’anongchuang Optoelectrinic Technology Co. Ltd., can work in the

orm of amplitude/phase modulation using a reflective liquid crys-al. In our experiment, amplitude modulation mode of the SLM ishosen, with resolution of 1920 × 1080, pixel size of 8.4 �m. Its fillactor is 90%. The top refresh rate is 200 Hz. CCD 1and 2 is A602f

ade by Basler Vision Technology, with resolution of 656 × 491,ixel size of 9.9 �m, fill factor of 90%.

The input to SLM 1 is made up of a reference and a phase-

ncoded target image. Consider a random phase function in Fourieromain ˚(u, v) [16–19], and define a phase mask �(u, v),

(u, v) = exp[−j˚(u, v)] (1)

Fig. 2. Experimental setup.

Fig. 3. Structure of SLM.

�(x, y) = ifft2[�(u, v)] (2)

In Eq. (1), ˚(u, v) is a random phase function normally dis-tributed between −2� and 2�. ifft2 is 2D reverse Fourier transformfunction. We use �(x, y) to encode the target image as followed

t′′ = t(x, y) ⊗ �(x, y) (3)

Because the target image has a displacement (xi, yi) with regardto the reference image r(x, y), the target image could be expressedas t(x + xi, y + yi), so the input images are given by

f (x, y) = r(x, y) + t′′(x + xi, y + yi) (4)

Its Fourier spectrum is expressed as followed

F(u, v) = fft2[r(x, y) + t′′(x + xi, y + yi)] = fft2[r(x, y)]

+ fft2[t(x + xi, y + yi) ⊗ �(x, y)] = R(u, v)

+ T(u, v)�(u, v) exp(−iuxi − ivyi) (5)

and its power spectrum is obtained by

P(u, v) = F(u, v)2 = R(u, v)2 + T(u, v)2 + R(u, v)T(u, v)∗�∗(u, v)

× exp(iuxi + ivyi) + R(u, v)∗T(u, v)�(u, v) exp(−iuxi − ivyi)

(6)

Then we multiply Eq. (6) with phase mask �(u, v), we get2 2 ∗

P(u, v)�(u, v) = R(u, v) �(u, v) + T(u, v) �(u, v) + R(u, v)T(u, v)

× exp(iuxi + ivyi) + T(u, v)�2(u, v)R(u, v)∗

× exp(−iuxi − ivyi) (7)

Fig. 4. Intensity distribution corresponding to different width of black-grating.

2046 P. Ge et al. / Optik 122 (2

Ft

F

c

tnpc

2

rbtyrbif

t

T

S

equal to 15 �m, 13.5 �m, 12 �m, 9 �m, 6 �m and 3 �m which cor-responding to the fill factor of 83.3%, 75%, 66.7%, 50%, 33.3% and16.7% are considered. The intensity distribution is shown as Fig. 4.In Fig. 4 X represents the effective aperture of the pixel. From Fig. 4

ig. 5. The schematic diagram of single pixel simulation: (a) a big pixel; (b) segmen-ation of a big pixel.

Finally we get the correlation output at the back focal plane ofourier lens 2

(x, y) = r ⊗ r∗ ⊗ � + t ⊗ t∗ ⊗ � + t ⊗ � ⊗ �∗ ⊗ r∗(x + xi, y + yi)

+ t∗(x − xi, y − yi) ⊗ r (8)

From Eq. (8), the former three items have a random phase func-ion. The random phase function scatters the DC term into systemoises, left only cross-correlation peak appearing in the outputlane, which is helpful for cross-correlation peak’s detection toalculate out the displacement (xi, yi).

.2. Theory of black-grating-effect

The grating structure of SLM is shown as Fig. 3. The white areaepresents transmitting parts where light can pass through, thelack area represents insulated electrode where light can not passhrough. Suppose the image needed modulating by the SLM is f(x,), the distance between adjacent pixels along the x, y axis is K, L,espectively. The effective aperture of a pixel along the x, y axis is a,. The total aperture of the SLM is W, H. The resolution of the SLMs E × F, so the transmittance of the SLM t(x, y) and the modulationunction T(u, v) are

(x, y) =[

1KL

rect(

x

a,

y

b

)∗comb

(x

K,

y

L

)]rect

(x

W,

y

H

)(9)

(u, v) = of −1{t(x, y)} = ab

KLsin c(au) sin c(bv)KLcomb(ku, Lv)∗

× (WH sin c(Wu) sin c(Hv)) = ab

KLsin c(au) sin c(bv)

×E∑

n=0

F∑m=0

(u − n

K, v − m

L

)∗[WH sin c(Wu) sin c(Hv)]

= ab

KL

E∑n=0

F∑m=0

sin c(au) sin c(bv)WH sin c[

W(

u − n

K

)]

× sin c[

H(

v − m

L

)](10)

So at the spectrum output plane, the spectrum will be

= F(u, v) ⊗ T(u, v) (11)

011) 2044– 2049

where F(u, v) is the Fourier transformation of f(x, y). It is easy tosee that the spectrum of the input image is modulated by the sincfunction.

3. Simulation and experiments

We take one type of SLM made by BNS corporation, ModelA256-�, resolution of which is 256 × 256 as an example. Take onedimension for simplicity. Its pixel size is 18 �m, so the physical sizeof the SLM is 4.61 mm × 4.61 mm. The effective aperture of a pixel

Fig. 6. Ideal input image and its corresponding spectrum, output: (a) referenceimage; (b) target image; (c) phase-encoded of (b); (d) ideal input image includ-ing reference and phase-encoded target; (e) its corresponding spectrum; (f) idealcorrelation output.

P. Ge et al. / Optik 122 (2

itttIesrs

st1ew

F

900 × 900 (equal to 150 × 6) shown in Fig. 7. In the new representa-tion a region of 6 × 6 pixels represent one pixel in images shown inFig. 6a. Displacement of the new representation images is (84.312,

Fig. 6. (Continued)

t is easy to see that optical energy distribution moves toward tohe 0th fringe when the effective size increases, which locates inhe middle area of a detection CCD. So the spectrum can be pho-ographed by a CCD with small field coverage and low resolution.t brings down the cost because high resolution CCDs are unnec-ssarily. In another view, due to the limit of bandwidth, the framepeed and the resolution of a CCD is often contradicted, as the highesolution is not needed, so the CCD with high frame rate is easy toatisfy with the same cost.

The simulation of the black-grating-effect of single pixel ishown as Fig. 5. The pixel size of Model A256-� is 18 �m, and fill fac-or 69.4%. We segment the big pixel into 6 × 6 small pixels, including1 black pixels and 25 white pixels where light can pass through. So

ach small pixel is 3 �m × 3 �m. The transmission is 25/36 = 69.4%,hich satisfies the real situation.

The reference and target images for simulation are shown asig. 6. The reference image is sub part of a picture in a video. The

011) 2044– 2049 2047

resolution is 150 × 150. The target frame is generated by moving thereference toward southwest with the step of (14, 1) pixels. In idealsituation, the input image including reference image and phase-coded target image is displayed seamlessly on the SLM as shownin Fig. 6d. The spectrum of Fig. 6d is shown as in Fig. 6e. Then thespectrum is transported to another SLM and the output of correla-tion is acquired as Fig. 6f. The displacement between the referenceand target image is (14, 1).

Then we simulate the single pixel with black-grating-effect andsegment each pixel into 6 × 6 pixels. The corresponding images areshown in Fig. 7. The resolution of reference and target image is

Fig. 7. Input images with black-matrix effect and its corresponding spectrum, out-put: (a) reference image; (b) target image; (c) phase-encoded of (b); (d) input imageincluding reference and phase-coded target; (e) its corresponding spectrum; (f)correlation output.

2048 P. Ge et al. / Optik 122 (2011) 2044– 2049

6a

ssatoei

Fig. 8. Displacement on x axis with black-grating-effect.

Fig. 9. Errors of displacement on x axis with black-matrix-effect of different fillfactor.

TR

Fig. 7. (Continued)

.126), equal to (14.052, 1.021) of the non-segmentation. The errorsre (0.052, 0.021) pixel.

Finally we use video sequence in our experiment. The videoequence is decomposed into consecutive frames. A sub region iselected. Every time two adjacent frames are selected as referencend target images. The video has 100 frames. The displacement of

his video along x axis and y axis is shown as Fig. 8, Fig. 10, errorsf displacement are shown as Fig. 9 and Fig. 11. Root mean squarerrors (RMSE) of displacement under different fill factor are shownn Table 1. Fig. 10. Displacement on y axis with black-grating-effect.

able 1MSE of displacement error under different fill factor.

RMSE on Fill factor = 25% Fill factor = 44.4% Fill factor = 56.25% Fill factor = 69.4% Fill factor = 73.47% Fill factor = 76.56%

x axis 0.0244 0.0241 0.0299 0.0190 0.0619 0.0374y axis 0.0356 0.0567 0.0529 0.0384 0.1867 0.0542

P. Ge et al. / Optik 122 (2

Ff

4

bT0ohtidAf

A

gN

[

[

[

[

[

[

[

[

ig. 11. Errors of displacement on y axis with black-matrix-effect of different fillactor.

. Conclusion

From Figs. 9 and 11 it is seen that the black-grating-effect hasrought in some errors to displacement detection precision. Fromable 1 it can conclude that in most cases RMSE could remain within.1 pixels. Black-matrix effect depends on the physical structuralf SLM. The method to reduce this effect is to select an SLM with aigher fill factor. The bigger the size of effective pixel is, the higherhe optical efficiency is, and the intensity distribution of the inputmage’s spectrum moves toward zero order’s spectrum. It bringsown the cost because higher resolution CCDs are unnecessarily.lso as the high resolution is not necessary, so the CCD with high

rame rate is easy to satisfy with the same cost.

cknowledgements

This research was supported by the National Basic Research Pro-ram (973) of China (Grant No. 2009CB724002) and the Nationalatural Science Foundation (Grant No.60977010).

[

[

011) 2044– 2049 2049

References

[1] S. Jutamulia, Joint transform correlators and their applications, Proc. SPIE 1812(1992) 233–243.

[2] K. Janschek, T. Boge, V. Tchernykh, S. Dyblenko, Image based attitude deter-mination using an optical correlator, in: Proceedings of 4th ESA InternationalConference on Spacecraft Guidance, Navigation and Control Systems, 1999, pp.487–492.

[3] G.S. Pati, K. Singh, Discrimination-sensitive rotation-invariant joint transformcorrelator using gradient preprocessing and circular harmonic components,Opt. Eng. 37 (1) (1998) 43–48.

[4] M.S. Alam, M.A. Karim, Multiple target detection using a modified fringe-adjusted joint transform correlator, Opt. Eng. 33 (5) (1994) 1610–1616.

[5] W. Liu, H. Kim, S. Lee, S. Lee, Joint transform correlator for the detection ofdefects in optical fibers, Opt. Eng. 37 (5) (1998) 1468–1474.

[6] E.C. Tam, D.A. Gregory, R.D. Juday, Autonomous real-time object tracking withan adaptive joint transform correlator, Opt. Eng. 29 (4) (1990) 314–320.

[7] K. Janschek, S. Kusimov, A. Sultanov, V. Tchernykh, S. Dyblenko, Optical corre-lator for onboard autonomous control of Earth observation camera pointing,in: Digest of the 2nd International Symposium of the International Academy ofAstronautics (IAA) Small Satellites for Earth Observation, 1999, pp. 231–234.

[8] K. Janschek, V. Tchernykh, S. Dyblenko, Design concept for a secondary-payloadEarth observation camera, Proc. SPIE 3870 (1999) 78–86.

[9] K. Janschek, V. Tchernykh, S. Dyblenko, Space application of a self-calibratingoptical processor for harsh mechanical environment, in: Proceedings of 1st IFAC– Conference on Mechatronic Systems, 2000, p. 309.

10] K. Janschek, V. Tchernykh, S. Dyblenko, Integrated camera motion compen-sation by real-time image motion tracking and image deconvolution, in:Proceedings of the 2005 IEEE/ASME International Conference on AdvancedIntelligent Mechatronics, 2005, pp. 1437–1444.

11] K. Janschek, V. Tchernykh, S. Dyblenko, Performance analysis of optomecha-tronic image stabilization for a compact space camera, Control Eng. Pract. 15(3) (2007) 333–347.

12] H.C. Huang, P.W. Claeng, H.S. Kwok, A high-resolution liquid crystal-on-siliconSXGA projection display, Adv. Display 4 (2000) 24–31.

13] Q.R. Chen, J.P. Zhou, Q. Li, et al., Diffraction of TFT-LCD pixel—structure in opticalinformation process, App. Laser 23 (1) (2003) 37–40.

14] S.H. Yan, Y.F. Dai, H.B. Lu, S.Y. Li, et al., Method of eliminating the black-matrixeffect of electrical-address spatial light modulator, Acta Photon. Sin. 31 (11)(2002) 1421–1424.

15] S.H. Yan, Y.F. Dai, H.B. Lu, S.Y. Li, Eliminating pixelilation effect in electro-addressable spatial light modulator using beam shaping elements arrays, Opt.Tech. 29 (2) (2003) 194–196.

16] P. Refregier, B. Javidi, Optical image encryption based on input plane and Fourierplane random encoding, Opt. Lett. 20 (7) (1995) 767–769.

17] B. Javidi, G.S. Zhang, J. Li, Experimental demonstration of the random phaseencoding technique for image encryption and security verification, Opt. Eng.

35 (9) (1996) 2506–2512.

18] T. Nomura, Phase-encoded joint transform correlator to reduce the influenceof extraneous signals, Appl. Opt. 37 (17) (1998) 3651–3655.

19] T. Nomura, Y. Yoshimura, K. Itoh, et al., Incoherent only joint transform corre-lator, Appl. Opt. 34 (8) (1995) 1420–1425.