tiled integral floating display without occlusion effect using an...

Tiled integral floating display without occlusioneffect using an offset lens array

and a perpendicular barrier

Hee-Jin Choi,1 Junkyu Yim,2 Youngmin Kim,2 Sung-Wook Min,2,* Joonku Hahn,3

Kwang-Mo Jung,4 and Byoungho Lee5

1Department of Physics, Sejong University, Seoul 143-747, South Korea2Department of Information Display, Kyung Hee University, Seoul 130-701, South Korea

3School of Electronics Engineering, Kyungpook National University, Daegu 702-701, South Korea4Korea Electronics Technology Institute, Seoul 121-835, South Korea

5School of Electrical Engineering, Seoul National University, Seoul 151-744, South Korea

*Corresponding author: [email protected]

Received 28 April 2014; accepted 4 July 2014;posted 24 July 2014 (Doc. ID 210793); published 27 August 2014

Low resolution and narrow viewing angle are common problems for most of the existing autostereoscopicdisplay systems. Although integral imaging is among the promising candidates for future autostereo-scopic display, it is not yet free from the above bottlenecks. In this paper, a new technique to enhancethe picture quality of a three-dimensional image using an offset lens array is proposed. The proposedsystem is suitable to realize a tiled integral floating display with high pixel density, while it also increasesthe viewing angle to match all of the optical devices in a single display unit. The proposed scheme wasalso verified with experimental results. © 2014 Optical Society of AmericaOCIS codes: (110.2990) Image formation theory; (100.6890) Three-dimensional image processing;

(220.2740) Geometric optical design.http://dx.doi.org/10.1364/AO.53.00G169

1. Introduction

The realization of a realistic three-dimensional (3D)image with the current flat panel display (FPD) tech-nologies has become an important issue. Since thelight rays from most of the real objects include 3Dinformation, it is desirable with a long history toreveal a way to display them as they are [1,2]. Withthe progress in FPD technologies during the lastdecade, it has become possible for consumers to en-counter stereoscopic 3D products that haveperformances and prices that are competitive withthe conventional two-dimensional (2D) displays.

However, the existing stereoscopic 3D productsrequire observers to wear special glasses, which mostconsumers are reluctant to do. As a result, there areincreasing needs of the autostereoscopic 3D displaytechnologies to make the observer free from wearingadditional devices. Among them, integral imaging(InIm) is regarded as one of the promising candidatesdue to its principle of providing a full parallax 3Dimage within the viewing angle [3–7]. Moreover,InIm is expected to be a way to resolve the conflictbetween accommodation and vergence by realizinga volumetric 3D image with integrated light raysfrom various directions [8].

Despite the advantages listed above, InIm hassome bottlenecks, such as low resolution and limitedviewing angle, which are common in autostereo-scopic 3D technologies. The problem of the limited

1559-128X/14/27G169-08$15.00/0© 2014 Optical Society of America

20 September 2014 / Vol. 53, No. 27 / APPLIED OPTICS G169

http://dx.doi.org/10.1364/AO.53.00G169

viewing angle even at the center viewing positionwith a close viewing distance system is shown inFig. 1. As shown in Fig. 1, the edge side of the 3Dimage is not integrated properly, and flipped imagesare observed since light rays through the neighbor-ing elemental lenses are also observed.

The integral floating display technique is one ofthe approaches to resolve those problems [9–12]. Itis a concept to combine the InIm display with a largefloating lens [13–18] to overcome the limitation ofviewing angle by forming a viewing window throughwhich the proper 3D image can be seen without theflipped images. However, even the integral floatingdisplay technique could not fully resolve the problemof low resolution, and we need to find another ap-proach to realize a high-resolution autostereoscopic3D imaging system using the existing FPD deviceto satisfy both the display manufacturers and theconsumers.

The fundamental way to increase the resolution ofthe 3D image from the autostereoscopic 3D display,including the integral floating technique, is to usea display device with higher pixel density. Basedon that approach some of the display manufacturershave demonstrated InIm 3D displays with an ultra-high-definition (UHD; composed of 4096 × 2160pixels) display device. However, although the UHDresolution is sufficiently high to be used in 2D con-sumer electronics, the InIm 3D display with a UHDdevice can provide 3D images with only high-definition (HD; 1280 × 720) resolution and horizontalparallax at just 8–9 viewpoints. Considering that afull HD (FHD; 1920 × 1080) resolution has alreadybecome common in 2D and stereoscopic 3D displays,even the InIm with UHD device has difficulty satis-fying the needs of consumers.

Another approach to increase the resolution andperformance of the InIm 3D image is to make a tiledstructure using several InIm displays. A tiling tech-nique is a common one both in 2D and 3D displays torealize a large size display with high pixel density.The most important point to realize the tiling system

is to remove or at least minimize the seam line be-tween the basic display units to cause the occlusioneffect. However, in case of a tiled integral floatingsystem, the floating lens becomes a problem in sat-isfying the above condition since the size of the float-ing lens is commonly different from the other opticaldevices of the integral floating system, such as thelens array and the display screen.

Although there is previous research to match thesize of the floating lens with that of the display areato resolve the problem, the viewing angle of thatmethod is restricted due to the size mismatch be-tween the intra-unit devices [19]. In this paper, animproved integral floating structure with enhancedviewing angle is proposed. It is suitable for the tilingstructure by matching the size and position of theviewing window with those of all of the other opticaldevices inside an integral floating unit.

2. Principles

In the integral floating system, only the light rayspassing through the viewing window can integrateproper 3D images. Figure 2 shows the upper andlower marginal rays that originate at the lower andupper edges of each elemental image. Therefore, thelight rays observed through the viewing windowinclude proper information of the integrated 3D im-age, and no flipped image will be seen through theviewing window.

However, there still exists a severe problem in theconventional integral floating system: abnormallight rays to integrate the flipped images [dashedlines in Fig. 3(a)] can also be seen, even at the centerviewpoint, due to the size mismatch between thefloating lens and the lens array, and additional bar-riers are needed around the viewing window to blockthem. Regarding the refraction of the floating lens,the barriers should be parallel to the planes of theother optics, such as the floating lens and the lensarray as shown in Fig. 3(b). Therefore, it is hard toprevent the barrier from being observed, and theviewing window is typically smaller than the size ofthe entire system.

Lens Array

Display Panel

Viewing Angle

Observer

Integrated Image

Lower Marginal Ray

Upper Marginal Ray

Flipped Image

Flipped Image

Proper 3DImage

Fig. 1. Structure of the basic integral imaging system and a prob-lem of flipped image at the edge sides of the integrated image.

Floating Lens

Lens Array

ViewingWindow

Floated Image

Viewing Angle

Lower Marginal Rays

Upper Marginal Rays

Observer

Display Panel(Elemental Images)

Fig. 2. Formation of the viewing window through an integralfloating system without the flipped image. The viewing windowcan be defined between the upper (thin dashed-dot lines) and lower(thick dashed-double dot lines) marginal rays.

G170 APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014

As a result, the integral floating system is not suit-able to realize a tiled structure due to the parallelbarrier. If the multiple integral floating system istiled to display large 3D images, only some part ofthose 3D images will be blocked by the barriers,and they can be seen only through the opened area(viewing window) of each integral floating system asshown in Fig. 4, and there will be a significantamount of occlusion effect in a tiled integral floatingstructure using the conventional principles. There-fore, we need to improve that problem to realize alarge autostereoscopic 3D display with superior res-olution using a tiled structure.

In this paper, a structure to satisfy the abovedemand is realized using a lens array and twofloating lenses. The structure of the proposed schemeis shown in Fig. 5. The difference between the

conventional one and the proposed technique is thatthe lens array is attached to one of the floating lensesto compose a nonuniform lens array, which has beenreported in Ref. [20] and called an “offset” lens array[21], although the possibility to extend the usageof the tiled integral floating display has not beenrevealed in the previous research.

The distance between the lens array and the dis-play panel (elemental images) is set to be the focallength f 1 of the lens array, and the distance betweenthe first and the second floating lenses is f 2—thefocal length of the floating lenses. In the proposedstructure shown in Fig. 5, the light rays from a singlepoint of the display panel will become parallelthrough the lens array and will be induced to the firstfloating lens. As a result, the image of those lightrays is formed at the location of the focal pointswhere the second floating lens exists. Therefore, thesecond floating lens has a function of floating the in-tegrated images. The other role of the second floatinglens is a natural viewing window of the 3D image

Fig. 3. (a) Upper and lower flipped images due to the light raysthrough neighboring elemental lens (dashed lines). (b) Parallelbarriers to block the flipped images in conventional integral float-ing system.

Integral floating unit 2

Integral floating unit 1

Barriers

Observer

Fig. 4. Problem of blocked 3D images in conventional tiled inte-gral floating system.

2nd Floating Lens(focal length: f2)

1st Floating Lens(focal length: f2)

Lens Array(focal length: f1)

f2f1

Display Panel

Dd

Offset Lens Array

Perpendicular Barrier

Fig. 5. Proposed system composed of an offset lens array, aperpendicular barrier, and the second floating lens.


since the light rays from the edges of the area of asingle elemental image will pass through the edgesof the second floating lens as shown in Fig. 6. In otherwords, the light rays within the viewing region orinside the viewing angle can pass only through thesecond floating lens.

Analyzing the paths of the abnormal rays thatpass through the neighboring elemental lenses andmake the flipped image (dashed lines in Fig. 7), theyare indicated to diverge and be blocked by theperpendicular barrier. In other words, there will beno flipped image in the proposed system withoutthe parallel barrier, and it is possible to realize a tiledstructure with unblocked 3D images.

The second floating lens also has another role: itallows the observer to see the light rays from a singlepoint on the offset lens array within the viewing an-gle and prevents the perpendicular barrier frombeing observed in the viewing zone as shown inFig. 8(a). If the second floating lens does not exist,the observer will see an area composed of the offsetlens array and the perpendicular barrier as shownin Fig. 8(b).

Moreover, the sizes of all optical devices in theproposed system are all the same, and it is ideal torealize a tiled integral floating system with aperpendicular barrier.

Considering the paths of the light rays to form the3D image at the position of the second floating lens,the viewing angle θ of the proposed system can beanalyzed as follows:

θ � 2 arctan�

D2f 2

�; (1)

where D is the size of the first and second floatinglenses. In order to meet the above condition, thereshould be the following relation between the sizeof the elemental lens d and that of the second floatinglens D:

df 1

� Df 2

: (2)

Considering the size matching between the opticalcomponents inside the system, the size of the lens

Fig. 6. Light rays to form the proper 3D images in the proposedsystem: (a) uppermarginal rays, (b) central light rays, and (c) lowermarginal rays.

Fig. 7. Paths of the abnormal rays to be blocked by theperpendicular barrier in the proposed system with tiled structure.


array should be matched with D, and the followingequations can be acquired:

D � nd; (3)

f 2 � nf 1: (4)

In the proposed pickup method, to pick up theelemental images on off-axis position (ith), the view-ing window (second floating lens) indicates a rangefor the proper pickup. After picking up a larger range(“picked-up range” in Fig. 9) than the viewing win-dow into Ppickup using the symmetric (conventional)CGII pickup method, the exceptional area (Ppickup-d)will be eliminated. For the above process, we used acriterion that was acquired from the principle inFig. 6 that the edges of the viewing window matchwith those of each elemental image. Regarding thesymmetry, the size of the picked-up range for theith elemental image will be as follows:

Si � 2 ×�D2� jijd

�: (5)

Since the size of the proper range for each elementalimage is the same for all elemental lenses, we can useEq. (3) to modify the above equation:

Si � �n� 2jij�d: (6)

Also, it is possible to recognize the relation belowfrom Fig. 5:

Si∶D � Ppickup∶d: (7)

Using Eq. (3) again, Eq. (7) can be rewritten asfollows:

Si∶nd � Ppickup∶d: (8)

As a result, the last equation can be acquired:

Ppickup � Si

n�

�1� 2jij

n

�d: (9)

From the equations above, it is possible to calculatethe size of the picked-up range for the ith elementalimage and also the magnification ratio to generatethe picked-up area (Ppickup). Also, for the elementalimages in opposite position (−ith), the same principleas above could be used.

3. Experimental Setup and Results

The pictures of the experimental setup are shown inFig. 10. In the experiments, two integral floatingunits using the offset lens array are used to verifythat the proposed scheme is appropriate for the tiledstructure. In a single unit, two lenses with size (D) of75 mm and focal length (f 2) of 150 mm have beenused as the first and second floating lenses. Besides,the lens array has a focal length (f 1) of 10 mm and iscomposed of 15 by 15 elemental lenses whose size(d) is 5mm. Therefore, the condition shown in Eqs. (3)and (4) could be satisfied. From the above para-meters, the expected viewing angle (θ) of the

Offset Lens Array



(Observed)

Area to be observed

Observer

Observer

Offset Lens Array


2nd Floating Lens

ObservedPoint

(a)

(b)

Fig. 8. Comparison of observed scene in cases in which theperpendicular barrier is (a) with the second floating lens and(b) without the second floating lens.

Elemental image

Offset lens array Picked up

rangefor -ith

elementalimage(S-i)

eliminated area

Picked uprangefor ith

elementalimage

(Si)

Fig. 9. Principles for generating the elemental images of offsetlens array system.


experimental system can be calculated to be about28 deg. As a display device, an LCD panel with diago-nal size of 9.7 in. and pixel size of 96 μm was used.

The elemental images are electrically generatedusing a computer-generated pickup method for theoffset lens array that was described in Fig. 11, andthey are to integrate two characters of “3” and “D”

at the positions of 130 and 90 mm from the first

floating lens, respectively, as shown in Fig. 12. There-fore, those two images have a depth difference of40 mm. Although those “3” and “D” characters them-selves are plane images, they are located in differentdepth positions and expected to show proper motionparallax.

The experimental results using the above setupand conditions are shown in Fig. 13. The 3D (“3”and “D”) images are taken into pictures at variouspositions. As expected in the theory, the realized sys-tem could provide the 3D images within a viewingangle of 28 deg. At the viewing positions exceedingthe expected viewing angle, the perpendicularbarrier blocks the light rays from the neighboring

Fig. 10. Pictures of experimental setup: (a) tiled structure com-posed of two integral floating units, (b) frontal side (second floatinglens) of integral floating units, and (c) rear side (lens array) of in-tegral floating units.

Integral floating unit 2 (Right)

Integral floating unit 1 (Left)Fig. 11. Generation of two 3D characters of “3” and “D” for theexperiment (plan view).

Fig. 12. Elemental images for two 3D characters of “3” and “D”

used in the experiment.


integral floating unit, and no image was observed. Inreviewing the experimental results, it can berecognized that the relative positions of “3” and“D” characters have been changed corresponding tothe view positions and therefore provided proper mo-tion parallax. Since the perpendicular barrier ismade from a black acrylic plate with thickness ofabout 1.2 mm, a thin seam line is observed betweentwo integral floating units. This problem is expectedto be resolved if we realize the perpendicular barrierwith thinner material such as a metal plate. In theexperimental setup, the acrylic plate was usedbecause it is easier to process.

In order to verify the floating and resamplingfunction of the second floating lens, the “3” and “D”

images were taken again with the same experimen-tal condition except one thing—the second floatinglens was removed from the setup. The experimentalresults using the modified setup are shown inFigs. 14(a) and 14(b). The images were taken at thesame horizontal view positions as in Fig. 13, but lookdifferent due to the lack of the second floating lens.At first, they are not integrated into an image (“3” or“D”) because they are not floated and therefore arenot at the designated positions. As a result, the im-ages are not displayed through two integral floatingunits, and each unit shows its wrong 3D images.Second, the other point to be reviewed is that thereis a gap between the images from each integral float-ing unit. The gap is actually an observed part ofthe perpendicular barrier. Since the barrier is madewith a black acrylic plate as shown in Fig. 14(a), theobserved part looks like a gap (nothing). Throughthe above analysis on the secondary experimentalresults shown in Fig. 14, the function of the secondfloating lens was proved as enhancing the picturequality of the 3D images.

4. Conclusions

In this paper, a tiled integral floating display with anenhanced viewing angle using an offset lens array isproposed. Considering the low resolution of an inte-gral floating display system, it could be a solution torealize a tiled structure with many integral floatingunits. In order to meet the above condition, the offsetlens array, which is composed of a lens array and afloating lens, is used. The role of the offset lens arrayis to refract the light rays that integrate the flippedimage (dashed lines in Fig. 4) in order to make thembe blocked by the perpendicular barrier as shown inFig. 7. In addition, another floating lens (secondfloating lens) is also attached at the exit pupil ofthe proposed structure to enhance the picture qualityby floating the 3D images and preventing theperpendicular barrier from being observed withinthe viewing angle. With the above combination, thesize of the viewing widow could be matched with thatof the other devices and the proposed integral float-ing unit was able to realize a tiled structure forhigher pixel density and enhanced 3D resolution.However, although the basic principles for realizinga tiled structure using the offset lens array and theperpendicular barrier are proved by the experimen-tal results with two integral floating units, the reali-zation of a large size display using multiple integralfloating units still requires us to solve new chal-lenges such as complex signal processing, opticalalignment between the integral floating units, andsynchronization of display signals for all units.

This research was supported by the Ministry ofKnowledge Economy (MKE), Korea, as a project,“Development of interactive user interface based3D system.”

Fig. 13. Pictures of two “3” and “D” images realized by the pro-posed scheme.

Fig. 14. Pictures of modified experimental setup and results:(a) modified experimental setup without the second floating lensand (b) experimental results using the modified setup.


References1. B. Lee, “Three-dimensional displays, past and present,” Phys.

Today 66(4), 36–41 (2013).2. T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68,

548–564 (1980).3. G. Lippmann, “Épreuves réversibles: photographies inté-

grals,” C. R. Acad. Sci. 146, 446–451 (1908).4. H. E. Ives, “Optical properties of a Lippmann lenticulated

sheet,” J. Opt. Soc. Am. 21, 171–176 (1931).5. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time

pickup method for a three-dimensional image based onintegral photography,” Appl. Opt. 36, 1598–1603 (1997).

6. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-indexlens-array method based on real-time integral photographyfor three-dimensional images,” Appl. Opt. 37, 2034–2045(1998).

7. S. Park, B.-S. Song, and S.-W.Min, “Analysis of image visibilityin projection-type integral imaging system without diffuser,”J. Opt. Soc. Korea 14, 121–126 (2010).

8. Y. Kim, J.-H. Jung, K. Hong, G. Park, and B. Lee, “Accommo-dation response in viewing integral imaging,” SID Int. Symp.Dig. Tech. Pap. 41, 530–532 (2010).

9. B. Lee, J. Hong, J. Kim, and J.-H. Park, “Analysis of theexpressible depth range of three-dimensional integral imag-ing system,” J. Opt. Soc. Korea 8, 65–71 (2004).

10. S.-W. Min, J. Kim, and B. Lee, “New characteristicequation of three-dimensional integral imaging systemand its applications,” Jpn. J. Appl. Phys. 44, L71–L74(2005).

11. J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imag-ing,” Appl. Opt. 48, H77–H94 (2009).

12. H. Choi, Y. Kim, J.-H. Park, S. Jung, and B. Lee, “Improvedanalysis on the viewing angle of integral imaging,” Appl.Opt. 44, 2311–2317 (2005).

13. S.-W. Min, M. Hahn, J. Kim, and B. Lee, “Three-dimensionalelectro-floating display system using an integral imagingmethod,” Opt. Express 13, 4358–4369 (2005).

14. H. Kakeya, “MOEVision: simple multiview display with clearfloating image,” Proc. SPIE 6490, 64900J (2007).

15. H. Kakeya, “Formulation of coarse integral imaging and itsapplications,” Proc. SPIE 6803, 680317 (2008).

16. S. Sawada andH. Kakeya, “Integral volumetric imaging usingdecentered elemental lenses,” Opt. Express 20, 25902–25913(2012).

17. H. Kakeya, T. Kurokawa, and Y. Mano, “Electronic realizationof coarse integral volumetric imaging with wide viewingangle,” Proc. SPIE 7524, 752411 (2010).

18. J. Hong, J. Yeom, and B. Lee, “Enhancing angular samplingrate of integral floating display using dynamically variableapertures,” Opt. Express 20, 10242–10255 (2012).

19. H.-J. Choi, Y. M. Kim, J. Jung, K.-M. Jung, and S.-W. Min,“Tiling integral floating display system with optimized view-ing window,” Appl. Opt. 51, 5453–5457 (2012).

20. D.-H. Shin, B. Lee, and E.-S. Kim, “Multidirectional curvedintegral imaging with large depth by additional use of alarge-aperture lens,” Appl. Opt. 45, 7375–7381 (2006).

21. M. S. Brennesholtz and E. H. Stupp, Projection Displays,2nd ed. (Wiley, 2008), Chap. 6.


tiled integral floating display without occlusion effect using an...

Documents