magic leaf _ head up _vrd

7/25/2019 Magic Leaf _ Head Up _vrd

1/47

VIRTUAL RETINA DISPLAY

REPORT

1


2/47

Abstract

The Virtual Retinal Display (VRD) is a personal display device under

development at the University of Washington's Human nterface Technology !a"oratory

in #eattle$ Washington U#%& The VRD scans light directly onto the vieer's retina& The

vieer perceives a ide field of vie image& ecause the VRD scans light directly on the

retina$ the VRD is not a screen "ased technology&

The VRD as invented at the University of Washington in the Human nterface

Technology !a" (HT) in 11& The development "egan in *ovem"er 1+& The aim as

to produce a full color$ ide field,of,vie$ high resolution$ high "rightness$ lo cost

virtual display& -icrovision nc& has the e.clusive license to commerciali/e the VRD

technology& This technology has many potential applications$ from head,mounted

displays (H-Ds) for military0aerospace applications to medical society&The VRD proects a modulated "eam of light (from an electronic source) directly

onto the retina of the eye producing a rasteri/ed image& The vieer has the illusion of

seeing the source image as if he0she stands to feet aay in front of a 12,inch monitor& n

reality$ the image is on the retina of its eye and not on a screen& The 3uality of the image

he0she sees is e.cellent ith stereo vie$ full color$ ide field of vie$ no flic4ering

characteristics&

5


3/47

Introduction

6ur indo into the digital universe has long "een a gloing screen perched on a

des4& t's called a computer monitor$ and as you stare at it$ light is focused into a dime,

si/ed image on the retina at the "ac4 of your eye"all& The retina converts the light into

signals that percolate into your "rain via the optic nerve&

Here's a "etter ay to connect ith that universe7 eliminate that "ul4y$ poer,

hungry monitor altogether "y painting the images themselves directly onto your retina& To

do so$ use tiny semiconductor lasers or special light,emitting diodes$ one each for the

three primary colors8red$ green$ and "lue8and scan their light onto the retina$ mi.ing

the colors to produce the entire palette of human vision& #hort of tapping into the optic

nerve$ there is no more efficient ay to get an image into your "rain& %nd they call it the

Virtual Retinal Display$ or generally a retinal scanning imaging system&

The Virtual Retinal Display presents video information "y scanning modulated

light in a raster pattern directly onto the vieer's retina& %s the light scans the eye$ it is

intensity modulated& 6n a "asic level$ as shon in the folloing figure$ the VRD

consists of a light source$ a modulator$ vertical and hori/ontal scanners$ and imaging

optics (to focus the light "eam and optically condition the scan)&

Fig1. Basic block diagram of the Virtual Retinal Display.

+


4/47

The resultant imaged formed on the retina is perceived as a ide field of vie

image originating from some vieing distance in space& The folloing figure illustrates

the light raster on the retina and the resultant image perceived in space&

Fig2.Illustration of light raster imaged onto the retina and the resultant perceived image.

n general$ a scanner (ith magnifying optics) scans a "eam of collimated light

through an angle& 9ach individual collimated "eam is focused to a point on the retina& %s

the angle of the scan changes over time$ the location of the corresponding focused spot

moves across the retina& The collection of intensity modulated spots forms the raster

image as shon a"ove

2


5/47

Potential Advantages of te Virtual Retinal Dis!la"

t is really interesting to note hy this family of imaging systems score "etter than

the conventional display systems&

Brightness

6ne pro"lem ith conventional helmet mounted display image sources is the lo

luminance levels they produce& -ost li3uid crystal array image sources have insufficient

luminance levels for operation in a see,through display& The VRD$ hoever$ does not

contain individual !am"ertian (or nearly !am"ertian) pi.el emitters (li3uid crystal cells

or phosphors) as do most !:D arrays and :RT's& The only light losses in the VRD result

from the optics (including the scanners and fi"er coupling optics)& There is no inherent

tradeoff$ hoever$ "eteen resolution and luminance as is true ith individual pi.el

emitters& n individual pi.el emitters$ a smaller physical si/e increases resolution "ut

decreases luminance& n the Virtual Retinal Display$ intensity of the "eam entering the

eye and resolution are independent of each other& :onse3uently$ the VRD represents a

maor step aay from the traditional limitations on display "rightness&

Resolution

%s mentioned in the previous section there is a tradeoff "eteen resolution and

"rightness in screen "ased displays& %s resolution re3uirements increase$ the num"er of

picture elements must increase in a screen "ased display& These greater pac4ing densities

"ecome increasingly difficult to manufacture successfully& The VRD overcomes this

pro"lem "ecause the resolution of the display is limited only "y the spot si/e on the

;


6/47

retina& The spot si/e on the retina is determined primarily "y the scanner speed$ light

modulation "andidth$ and imaging optics&

Yield

6ne limiting aspect in the manufacture of li3uid crystal array image generators is

the yield and relia"ility of the hundreds of thousands of individual li3uid crystal cells

present in these displays& &


7/47

#unda$entals of u$an e"e

The eye is a speciali/ed organ that is capa"le of light reception$ and in the case of

verte"rates$ is a"le to receive visual images and then carry it to the visual centre in the

"rain& The hori/ontal sectional vie of human eye is as follos (courtesy 9ncyclopedia

ritannica 5@@5)

Fig. !he cross sectional vie" of the human eye

A


8/47

The eye"all is generally descri"ed as a glo"e or a sphere$ "ut it is oval$ not circular& t

is a"out an inch in diameter$ transparent in front$ and composed of three layers&

1) The outer fi"rous$ the supporting layer

5) -iddle$ vascular$ and

+) nner nervous layer&

#i. muscles move the eye$ four straight and to o"li3ue& These lie inside the or"it

passing from the "ony alls of the or"it to "e attached to the sclerotic coat of the eye

"ehind the cornea& The movements of the eyes are com"ined$ "oth eyes move to right or

left$ up$ and don$ etc& *ormally the a.es of "oth the eyes converge simultaneously on

the same pointB hen oing to paralysis of one or more muscles$ they fail to do so s3uint

e.ists&

Te Sclera is the tough outer fi"rous coat& t forms the "hite of the eye and is

continuous in front ith the transparent indo mem"rane$ the cornea. The sclera

protects the delicate structures of the eye and helps to maintain the shape of the eye"all&

Te %oroid or middle vascular coat contains the "lood vessels$ hich are the

ramifications of the ophthalmic artery$ a "ranch of the internal carotid& The vascular coat

forms the iris ith the central opening orpupil of the eye& The pigmented layer "ehind

the iris gives its colour and determines hether the eye is "lue$ "ron$ grey etc& The

Choroids is continuous in the front ith the iris and ust "ehind the iris this coat is

thic4ened to form the ciliary body$ thus the ciliary "ody lies "eteen the choroids and the

iris& t contains circular muscle fi"res and radiating fi"resB contraction of the former

contracts the pupil of the eye&

Te Retina is the inner nervous coat of the eye$ composed of a num"er of layers

of fi"res$ nerve cells$ rods and cones$ all of hich are included in the construction of the

C


9/47

retina$ the delicate nerve tissue conducting the nerve impulses from ithout inards to

the optic disc$ the point here the optic nerve leaves the eye"all& This is the "lind spot$ as

it possesses no retina& The most acutely sensitive part of the retina is the macula# hich

lies ust e.ternal to the optic disc$ and e.actly opposite the centre of the pupil&

Fig$.!he layered vie" of retina sho"ing blood vessels

The retina is the nervous mechanism of sight& t contains the endings of the optic

nerves$ and is compara"le to a sensitive photographic plate&

When an image is perceived$ rays of light from the o"ect seen pass through the

cornea$ a3ueous humour$ lens$ and vitreous "ody to stimulate the nerve endings in the

retina& The stimuli received "y the retina pass along the optic tracts to the visual areas of

the "rain$ to "e interpreted& oth areas receive the message from "oth eyes$ thus giving

perspective and contour&

n ordinary camera one lens is provided& n the eye$ hilst the crystalline lens is

very important in focusing the image on the retina$ there are in all four structures acting

as lenses7 the cornea$ the a3ueous humour$ the crystalline lens$ and the vitreous "ody&


10/47

%s in all interpretations of sensation from the surface$ a num"er of relaying

stations are concerned ith the transmission of the senses hich in this case is the sight&

% num"er of these relaying stations are in the retina& nternal to the periphery of the retina

are layers of rods and cones hich are highly speciali/ed sight cells sensitive to light& The

circular interruptions in these are termed as granules& The pro.imal ends of the rods and

cones form the first synapse ith a layer if "ipolar cells$ still in the retina& The second

processes of these cells form the second nerve synapse ith large ganglion cells$ also in

the retina& The a.ons of these cells form the fi"res of the optic nerve& These pass

"ac4ards$ first reaching the loer centre in special "odies near the thalamus$ and finally

reaching the special visual centre in occipital lo"e of cere"ral hemisphere here sight is

interpreted&

Fig%. !he &uman visual path"ay

9ach retina includes multiple mosaics of neurons that separately represent the

visual field& mage transduction uses to systems of photoreceptors7 the rods and cones&

9ach system comprises a separate sampling mosaic of retinal image& The rods encode the

1@


11/47

data for a system ith lo spatial resolution "ut high 3uantum efficiency& The cones

encode the image data at much higher spatial resolution and loer 3uantum efficiency&

Rods and cones generally operate under different vieing conditions$ "ut there

are also many cases in hich multiple representations of the image are o"tained under a

single vieing condition&


12/47

hori/ontal scanning ithout the costs and other limitations of the %,6 devices& The -R#

as then utili/ed in full,color inclusive and see,through systems&

Virtual Retinal Dis!la"' A s"ste$ overvie(

The VRD can "e considered a porta"le system that creates the perception of an

image "y scanning a "eam of light directly into the eye& -ost displays directly address a

real image plane (typically a :RT or matri.,addressed !:D) hich might "e relayed to

form a larger$ more distant image for a head,mounted display (H-D)& The VRD uses a

scanned$ modulated light "eam to treat the retina as a proection screen$ much as a laser

light sho ould use the ceiling of a planetarium& The closest previously e.isting device

ould "e the scanning laser opthalmoscope (#!6) hich scans the retina to e.amine itB

the #!6 is designed to capture light returning from the eye hereas the VRD is designed

as a porta"le display&&

The VRD has several advantages over :RTs$ !:D$ and other addressa"le,screen

displays7

Resolution is limited "y "eam diffraction and optical a"errations$ not "y the si/e

of an addressa"le pi.el in a matri.& Very high resolution images are therefore

possi"le ithout e.tensive advances in micro,fa"rication technology& %lso$ the

VRD does not suffer from pi.el defects&

The display can "e made as "right as desired simply "y controlling the intensity of

the scanned "eam& This ma4es it much easier to use the display in see,though

configuration on a "right day&

15


13/47

The scanning technology in the current display re3uires only simple$ ell

understood manufacturing technology and can therefore "e manufactured

ine.pensively&

ecause the light is proected into the eye and the scanner is electro,mechanically

efficient$ the display uses very little poer&

n theory$ the VRD allos for accommodation to "e modulated pi.el "y pi.el as

the image is "eing scanned&

%ll components in the VRD are small and light$ ma4ing them ideal for use in a

porta"le display&

Te )asic S"ste$

n a conventional display a real image is produced& The real image is either

vieed directly or$ as in the case ith most head,mounted displays$ proected through an

optical system and the resulting virtual image is vieed& The proection moves the virtual

image to a distance that allos the eye to focus comforta"ly& *o real image is ever

produced ith the VRD& Rather$ an image is formed directly on the retina of the user's

eye& % "loc4 diagram of the VRD is shon in the figure "elo&

1+


14/47

Fig'. !he functional block diagram of a VRD system

To create an image ith the VRD a photon source (or three sources in the case of

a color display) is used to generate a coherent "eam of light& The use of a coherent source

(such as a laser diode) allos the system to dra a diffraction limited spot on the retina&

The light "eam is intensity modulated to match the intensity of the image "eing rendered&

The modulation can "e accomplished after the "eam is generated& f the source has

enough modulation "andidth$ as in the case of a laser diode$ the source can "e

modulated directly&

The resulting modulated "eam is then scanned to place each image point$ or pi.el$

at the proper position on the retina& % variety of scan patterns are possi"le& The scanner

could "e used in a calligraphic mode$ in hich the lines that form the image are dran

directly$ or in a raster mode$ much li4e standard computer monitors or television& 6ur

12


15/47

development focuses on the raster method of image scanning and allos the VRD to "e

driven "y standard video sources& To dra the raster$ a hori/ontal scanner moves the

"eam to dra a ro of pi.els& The vertical scanner then moves the "eam to the ne.t line

here another ro of pi.els is dran&

%fter scanning$ the optical "eam must "e properly proected into the eye& The goal

is for the e.it pupil of the VRD to "e coplanar ith the entrance pupil of the eye& The lens

and cornea of the eye ill then focus the "eam on the retina$ forming a spot& The position

on the retina here the eye focuses the spot is determined "y the angle at hich light

enters the eye& This angle is determined "y the scanners and is continually varying in a

raster pattern& The "rightness of the focused spot is determined "y the intensity

modulation of the light "eam& The intensity modulated moving spot$ focused through theeye$ dras an image on the retina& The eye's persistence allos the image to appear

continuous and sta"le&


16/47

The VRD does not re3uire an intermediate image on a screen as do systems using

!:D or :RT technology& The only re3uired components are the photon source

(prefera"ly one that is directly modulata"le)$ the scanners$ and the optical proection

system& #mall photon sources such as a laser diode can "e used& %s descri"ed "elo the

scanning can "e accomplished ith a small mechanical resonant device developed in the

HT!& The proection optics could "e incorporated as the front$ reflecting$ surface of a

pair of glasses in a head mount configuration or as a simple lens in a hand held

configuration& HT! engineers have e.perimented ith single piece


17/47

hori/ontal fields of vie have "een demonstrated& nclusive systems ith 1@@ degree

fields of vie are feasi"le& #ee through systems ill have somehat smaller fields of

vie& :urrent see through systems ith over 2@ degree hori/ontal fields of vie have

"een demonstrated&

Color and Intensity Resolution

:olor ill "e generated in a VRD "y using three photon sources$ a red$ a green$

and a "lue& The three colors ill "e com"ined such that they overlap in space& This ill

yield a single spot color pi.el$ as compared to the traditional method of closely spacing a

triad$ improving spatial resolution&

The intensity seen "y the vieer of the VRD is directly related to the intensity

emitted "y the photon source& ntensity of a photon source such as a laser diode is

controlled "y the current driving the device& Iroper control of the current ill allo

greater than ten "its of intensity resolution per color&

Brightness

rightness may "e the "iggest advantage of the VRD concept& The current

generations of personal displays do not perform ell in high illumination environments&

This can cause significant pro"lems hen the system is to "e used "y a soldier outdoors

or "y a doctor in a ell lit operating room& The common solution is to "loc4 out as much

am"ient light as possi"le& Unfortunately$ this does not or4 ell hen a see through

mode is re3uired&

The VRD creates an image "y scanning a light source directly on the retina& The

perceived "rightness is only limited "y the poer of the light source& Through

e.perimentation it has "een determined that a "right image can "e created ith under one

microatt of laser light& !aser diodes in the several milliatt range are common& %s a

result$ systems created ith laser diode sources ill operate at lo laser output levels or

ith significant "eam attenuation&

1A


18/47

Power Consumption

The VRD delivers light to the retina efficiently& The e.it pupil of the system can

"e made relatively small alloing most of the generated light to enter the eye& n

addition$ the scanning is done ith a resonant device hich is operating ith a high

figure of merit$ or J$ and is also very efficient& The result is a system that needs very little

poer to operate&

A rue Stereos!opi! "isplay

The traditional head,mounted display used for creating three dimensional vies

proects different images into each of the vieer's eyes& 9ach image is created from a

slightly different vie point creating a stereo pair& This method allos one important

depth cue to "e used$ "ut also creates a conflict& The human uses many different cues to

perceive depth& n addition to stereo vision$ accommodation is an important element in

udging depth& %ccommodation refers to the distance at hich the eye is focused to see a

clear image& The virtual imaging optics used in current head,mounted displays place the

image at a comforta"le$ and fi.ed$ focal distance& %s the image originates from a flat

screen$ everything in the virtual image$ in terms of accommodation$ is located at the same

focal distance& Therefore$ hile the stereo cues tell the vieer an o"ect is positioned atone distance$ the accommodation cue indicates it is positioned at a different distance&

With the VRD it is theoretically (this is currently in the development stage)

possi"le to generate a more natural three dimensional image& The VRD has an individual

avefront generated for each pi.el& t is possi"le to vary the curvature of the avefronts&

*ote that it is the avefront curvature hich determines the focus depth& This variation

of the image focus distance on a pi.el "y pi.el "asis$ com"ined ith the proection of

stereo images$ allos for the creation of a more natural three,dimensional environment&

In!lusi#e and See hrough

#ystems have "een produced that operate in "oth an inclusive and a see through

mode& The see through mode is generally a more difficult system to "uild as most

1C


19/47

displays are not "right enough to or4 in a see through mode hen used in a medium to

high illumination environment here the luminance can reach ten thousand candela per

meter s3uared& %s discussed a"ove$ this is not a pro"lem ith the VRD&

n the VRD a light source is modulated ith image information$ either "y direct

poer (internal) modulation or "y an e.ternal modulator& The light is passed through an

.,y scanning system$ currently the -R# and a galvanometer& !ight from the scanner pair

enters an optical system$ hich in present implementations of the VRD forms an aerial

image and then uses and eyepiece to magnify and relay this image to infinity&

%o$!onents of te Virtual Retinal Dis!la"

Video $le!troni!s

n its current form$ the video electronics of the VRD controls the light intensity

modulation$ scanner deflection$ and the synchroni/ation "eteen modulation and

scanning& The hori/ontal and vertical synchroni/ation signals in the video signal are used

1


20/47

to determine scanner synchroni/ation& % user selecta"le delay of up to one full line is

incorporated into the video electronics to allo for phase difference "eteen the

hori/ontal scanner position and the modulation timing& %lso$ the respective drive levels

for intensity modulation of each light source are output from the electronics&

The drive electronics control the acousto,optic modulators that encode the image

data into the pulse stream& The color com"iner multiple.es the individually,modulated

red$ green$ and "lue "eams to produce a serial stream of pi.els$ hich is launched into a

single mode optical fi"er to propagate to the scanner assem"ly& The drive electronics

receive and process an incoming video signal$ provide image compensation$ and control

image display&


21/47

future& ncidentally$ polychromatic sources can "e shon to form spots compara"le to

monochromatic ones of the same spatial e.tent& Therefore spatial coherence is

responsi"le for the small spot si/e hich leads to "oth high resolution and (given enough

poer) retinal ha/ard&

To achieve the desired resolution$ all current VRD prototypes have used lasers for

their superior spatial coherence characteristics& n order to use a point source such as an

!9D$ the image of the source should "e smaller than the diffraction limit of the scanner&

Using the lens magnification$ one can determine the ma.imum source si/e that can "e

used "efore degrading the diffraction limited spot si/e at the image plane& The angular

divergence of the source is effectively limited "y treating the scanner as a stop& !ight

hich does not hit the mirror does not contri"ute to the image plane spot si/e& ) analog modulation& The video

51


22/47

electronics regulate the voltage seen "y the laser current driver and it controls the current

passing through the laser hich in turn controls the light output poer from the laser& The

laser diode is operated "eteen amplitudes of @&@ and C@&@=m%>&

%cousto,optic (%,6) modulators intensity modulate the green and "lue laser

"eams& %cousto,optic modulators create a sound ave grating in a crystal through hich

a light "eam passes& The sound ave creates alternate regions of compression and

rarefaction inside the crystal& These alternating regions locally change the refractive

inde. of the material& %reas of compression correspond to higher refractive indices and

areas of rarefaction correspond to loer refractive indices& The alternating areas of

refractive inde. act as a grating and diffract the light& %s the sound ave traverses the

light "eam$ the diffracted "eam is intensity modulated according to the amplitudemodulated envelope on the carrier signal&

S!anners

The scanners of the VRD scan the raster pattern on the retina& The angular

deviation of the hori/ontal scanner com"ined ith the angular magnification of the

imaging optics determines the hori/ontal field of vie& The angular deviation of the

vertical scanner com"ined ith the angular magnification of the imaging opticsdetermines the vertical field of vie& The hori/ontal scanner speed and the frame rate

determine the num"er of hori/ontal lines in the display$

(umber of hori)ontal lines * hori)ontal scanner fre+uency , frame rate$

here frame rate is the num"er of times per second the entire picture (or frame)

is generated& The modulation rate and the hori/ontal scanner fre3uency determine the

num"er of pi.els per line in the display$

(umber of pi-els per line * modulation fre+uency , hori)ontal scanner fre+uency#

here the modulation fre3uency is the num"er of times per second the pi.els are created

(or modulated)&

55


23/47

The hori/ontal scanning mechanism of the VRD must "e capa"le of "oth

relatively high scan rates (1; 4H/,@L 4H/) and high resolution (;@@,5@@@L pi.els) for

*T#: to HDTV formats$ respectively& #VF% format systems (C@ 4H/) in

monochrome0greyscale using an %,6 scanner and +@ 4H/ in full,color ith a mechanical

resonant one have "een "uilt&

The scanning device consists of a mechanical resonant scanner and galvanometer

mirror configuration& The hori/ontal scanner is the mechanical resonant scanner (-R#)>&

The -R# has a flu. circuit induced "y coils hich are "eneath a spring plate& The flu.

circuit runs through the coils and the spring plate and alternately attracts opposite sides of

the spring plate and there"y moves the scanner mirror through an angle over time& n a

design developed at the HT! the vertical deflection mirror as chosen as the

galvanometer mirror& The galvanometer deflection can "e selected according to the aspect

ratio of the display and a typical ratio of 27+ can "e chosen& The galvanometer fre3uency

is controlled "y the video electronics to match the video frame rate&

The galvanometer and hori/ontal scanner are arranged in hat is "elieved to "e a

novel configuration such that the hori/ontal scan is multiplied& The scanners are arranged$

as shon in the folloing figures& #uch that the "eam entering the scanner assem"ly first

stri4es the hori/ontal scanner then stri4es the vertical scanner& The "eam is reflected "y

the vertical scanner "ac4 to the hori/ontal scanner "efore e.iting the scanner assem"ly&

The "eam therefore stri4es the hori/ontal scanner tice "efore e.iting the scanner

configuration& n such an arrangement$ the first scan (corresponding to the first "ounce or

reflection) is dou"led "y the second scan (corresponding to the second "ounce or

reflection)& The case shon is for M 2; =deg&> herein the e.it "eam returns parallel to

the hori/ontal incident "eam& n the first figure the -R# is undeflected and in the latter

the -R# is deflected "y =deg&>&

5+


24/47

Fig. /R0,alvanometer scanner assembly sho"ing incident and e-it beam paths for the

/R0 in an undeflected position.

Fig. /R0,alvanometer scanner assembly sho"ing incident and e-it beam paths for the

/R0 in a deflected position.

The result of arranging the scanners as in the a"ove figures is a dou"ling of the

hori/ontal optical scan angle& 6ther configurations have "een applied to this approach to

achieve a tripling in the hori/ontal direction and simultaneously a dou"ling in the verticaldirection&


25/47

scanner had "een developed& y etching thin layers from a sliver of silicon$ the

researchers ere a"le to "uild a scanner that eighs a mere ; grams and measures less

than 1 s3uare centimeter& The mirror$ too$ is much smaller at 1 millimeter across and is

mounted on the end of a thin$ fle.i"le$ "ar hich is anchored to the silicon& The mirror is

turned into one plate of a capacitor$ ith the other plate formed "y a small area of silicon

"eneath it& Iut a rapidly varying voltage across the to plates and then the mirror ill "e

first repelled and then attracted& The mirror can move up or don more than +@$@@@ times

each second&

Fig3. 4 /5/0 mirror

-icro,9lectro,-echanical #ystems (-9-#) is the integration of mechanical

elements$ sensors$ actuators$ and electronics on a common silicon su"strate through the

utili/ation of microfa"rication technology& The electronics are fa"ricated using integrated

circuit (:) process se3uences$ hile the micromechanical components are fa"ricated

using compati"le micromachining processes that selectively etch aay parts of the

silicon afer or add ne structural layers to form the mechanical and electromechanical

devices&

The electromagnetic actuation of the scanners yields more life to the system and

imparts more tor3ue& #uch designs have also "een developed for retinal scanning

displays&

5;


26/47

Pupil e'pander

*ominally the entire image ould "e contained in an area of 5 mm5& The e.it,

pupil e.pander is an optical device that increases the natural output angle of the image

and enlarges it up to 1C mm on a side for ease of vieing& The raster image created "y the

hori/ontal and vertical scanners passes through the pupil e.pander and on to the vieer

optics&


27/47

dependence is not severe& 6n first pass$ ?@N of the energy in the scan is transmitted

through the splitter0com"iner to a concave spherical mirror& The mirror is actually a

rectangular section of a spherical mirror ith radius of curvature ,1@@ =mm>& The

negative sign denotes concavity &

Fig16. !he vie"ing optics system of VRD

(olographi! )pti!al $lement

6ne of the pro"lems ith the VRD only "ecomes apparent hen you put it on& t

can "e li4ened to loo4ing through a pair of high, magnification "inoculars that one must

line his eyes precisely ith the "eam or the image disappears& #ince e rarely fi. our

eyes on a single point for more than fe seconds$ using VRD "ecomes difficult& #o en

eye,trac4ing system that follos the movements of the pupil "y monitoring the

5A


28/47

reflections from the cornea had to "e developed& The trac4er calculates here the eye is

loo4ing and moves the laser around to compensate& ut this system is comple. and

e.pensive&

% "etter solution may lie ith a special 4ind of lens 4non as a holographic

optical element& %n H69 is actually a diffraction grating made "y recording a hologram

inside a thin layer of polymer&

t or4s "y converting a single "eam of laser into a circular array of 1; "right

spots& Ilace the H69 "eteen the scanning mirrors and the eye$ and the array of "eams

that forms ill illuminate the region round your pupil& -ove your eyes slightly and one

of the "eams ill still stri4e the cornea and "e focused to form an image on the retina&

H69s have a "ig advantage over eye trac4ing systems7 "ecause they are made from a thin

layer of polymer$ they eigh ne.t to nothing& O%ll of the action ta4es place in a layer ust

a fraction of millimeter thic4P$ says a researcher&

Esti$ated Retinal Illu$inance

The relationship "eteen estimated retinal illuminance and scene luminance isimportant in understanding the display operating on this principle& %s the display in this

thesis contains no screen or real o"ect$ it is impossi"le to discuss the "rightness of the

display in terms of luminance& n terms of "rightness$ estimated retinal illuminance is a

5C


29/47

common denominator$ so to spea4$ of screen "ased display systems and retinal scanning

displays systems& The estimated retinal illuminance is =+?>7

I(trolands) MR. pupil area (mm5) . scene luminance (cd0m5)

hereIM retinal illuminance$ pupil area refers to the area of the pupil of the eye$ and

RM the effectivity ratio& The effectivity ratio$R$ allos for the #tiles,:raford effect and

is$

RM 1 , @&@1@?d5L @&@@@@21?d2&

here dM the eye's pupil diameter in millimeters& %s shon "y dimensional analysis on

the e3uation for I$ trolands reduce effectively to the units of optical poer per unitsteradian&

The #tiles,:raford effect descri"es the contri"ution to "rightness sensation of light

entering different points of the pupil (i&e& light entering the center of the pupil contri"utes

more to the sensation of "rightness than does light entering farther from the pupil center)&

#ome standard scene luminance values$ 7$ and their corresponding #tiles,:raford

corrected estimated retinal illuminance values$I$ are given in Ta"le &1 =+?$+A>&

Type of #cene%ppro.imate !uminance

=cd0m5>

9stimated Retinal

lluminance =trolands>

:lear day 1@2 +&@ . 1@2

6vercast day 1@+ 2&; . 1@+

Heavily overcast day 1@5 &; . 1@5

#unset$ overcast day 1@ 1&; . 1@5

102 hour after sunset$ clear 1 5@

105 hour after sunset$ clear 1@,1 5&@


30/47

!able1. 0tandard scene luminance values and corresponding estimated retinal

illuminance values.

ransmission Chara!teristi!s of the )!ular &edia

Transmission losses in the eye result from scattering and a"sorption in the cornea$ lens$

a3ueous humor$ and vitreous humor& The transmittance of the ocular media is a function

of the avelength of the light traveling through the media& &

Fig11. !ransmittance of the ocular media vs. "avelength.

I$age *ualit" as Related to te E"e

Introdu!tion

-easurements of display image 3uality depend heavily on to displaycharacteristics$ resolution and contrast (see su"se3uent sections)& t is virtually fruitless

to discuss image 3uality in terms of either resolution or contrast ithout including the

other& Definitions for display resolution$ contrast$ contrast ratio$ and modulation contrast

+@


31/47

are given in the folloing discussion& Whenever possi"le$ the meanings of the terms are

related to the effect or result at the retina&

"isplay Resolution and the $ye

The resolution of a display can "e defined as the angle su"tended "y each display

resolution element&


32/47

as in this thesis$ estimated retinal illuminance is a prefera"le measure of display

"rightness as there is no screen in the system&

"isplay Contrast Ratio and the $ye

The contrast ratio$ 8R$ of a display is the ratio of the ma.imum display intensity to the

minimum display intensity& n other terms =2@>$

8RM (7Dma-07Dmin)

here 7Dma- M the ma.imum display luminance and 7Dmin M the minimum display

luminance& 9.tending the definition of contrast in terms of estimated retinal illuminance

gives

8RM (IDma-,IDmin)

hereIDma-M the ma.imum estimated retinal illuminance due to the display and IDminM

the minimum estimated retinal illuminance due to the display& The values of IDma- and

IDmincorrespond to the estimated retinal illuminance values for displays ith luminance

values of7Dma-and7Dminrespectively&

"isplay &odulation Contrast and the $ye

The modulation contrast$ 8/$ of a display is the ratio of the difference "eteen the

ma.imum display intensity and the minimum display intensity divided "y the sum of the

minimum and ma.imum intensities& n other terms =2@>$

8/M (7Dma-,7Dmin), (7Dma-9 7Dmin)

here 7Dma- M the ma.imum display luminance and 7Dmin M the minimum display

luminance& 9.tending the definition of contrast in terms of estimated retinal illuminancegives

8/M (IDma-,IDmin), (IDma-LIDmin)

+5


33/47

hereIDma-M the ma.imum estimated retinal illuminance due to the display and IDminM

the minimum estimated retinal illuminance due to the display& n other ords$ the values

of IDma-and IDmincorrespond to the estimated retinal illuminance values of displays ith

luminance values of7Dma-and7Dminrespectively&

Stereogra!ic Dis!la"s using VRD

%s discussed previously hile treating the possi"ility of three,dimensional

imaging systems using VRD there are to cues "y hich the human "eings perceive the

real orld namely the accommodation cue and the stereo cue& There is a mismatch of the

information conveyed "y the to cues in proection systems so that prolonged vieing

can lead to some sort of psychological disorientation&

n VRD e can generate individual avefronts for each pi.el and hence it is

possi"le to vary the curvature of individual avefronts hich determines the focal depth$

so hat e get is a true stereographic vie&

The Virtual Retinal Display (VRD) developed at the University of Washington

Human nterface Technology !a" (HT !a") is "eing modified from a fi.ed plane of

focus display to a varia"le focus display&& y integrating a deforma"le mirror into the

VRD$ the avefront of light "eing scanned onto the retina can "e changed and various

fi.ation planes created depending on the divergence of the light entering the eye&

Irevious em"odiments of +D displays alloing for natural accommodation and vergence

responses include the use of a varifocal mylar mirror and the use of a li3uid,crystal

varifocal lens& n the former$ a reflective mylar surface as deformed "y air pressure

using a loudspea4er "ehind the mylar mirror frame& % :RT screen as positioned so thatthe vieers sa the reflection of the :RT in the mirror at various virtual image depths& n

the latter$ an electrically,controlla"le li3uid,crystal varifocal lens as synchroni/ed ith

a 5,D display to provide a +D image ith a display range of Q1&5 to L1&; diopters (10focal

++


34/47

length in meters)& %lthough these systems provided for a +D volumetric image alloing

for natural human eye response$ they are large and cum"ersome "enchtop systems&

"eforma*le &em*rane &irror

The deforma"le mem"rane mirror is a -9-# device that is used in adaptive

optics applications& The mirror is "ul4 micromachined and consists of a thin$ circular

mem"rane of silicon nitride coated ith aluminum and suspended over an electrode&

When a voltage is applied to the electrode$ the mirror mem"rane surface deforms in a

para"olic manner a"ove the electrode& The avefront of a "eam of light hitting the mirror

mem"rane surface can "e changed "y varying the voltage applied to the electrode& With

no voltage applied$ the mirror mem"rane surface remains flat& With a certain amount of

voltage applied$ the reflecting "eam ill "e made more converging& y integrating the

deforma"le mirror into the VRD scanning system$ a three,dimensional picture can "e

created "y 3uic4ly changing the scanned "eams degree of collimation entering the eye&

)pti!al "esign

The He*e laser "eam is spatially filtered and e.panded "efore stri4ing the

deforma"le mirror& When the mirror is grounded$ the "eam is at ma.imal divergence

hen entering the eye& :onversely hen the mirror voltage is at ma.imum$ the resultant

"eam is collimated hen entering the eye& The "eam is reflected off a scanning

galvanometer and through an ocular lens to form a vieing e.it pupil& % vieer putting

his eye at the e.it pupil ould see a 1,D image at a focal plane determined "y the amount

of "eam divergence& With no voltage on the mirror this image is located at close rangeB

ith ma.imum voltage on the mirror the image is at optical infinity& n this ay the

optical setup provides a range of focal planes from near to far hich can "e manipulated

"y changing the voltage on the mirror&

Evolution of VRD s"ste$s

+2


35/47

The proect's initial goal as to prove the via"ility of forming an image on the

retina using a scanned laser& %s a result of the or4$ a patent application as filed and the

technology licensed to a #eattle "ased start up company$ -icro Vision$ nc& Under terms

of the agreement$ -icro Vision is funding a four,year effort in the HT! to develop the

technologies that ill lead to a commercially via"le VRD product& This development

or4 "egan in *ovem"er 1+&

Prototype +,

The original prototype had very lo effective resolution$ a small field of vie$

limited gray scale$ and as difficult to align ith the eye& 6ne o"ective of the current

development effort as to 3uic4ly produce a "ench,mounted system ith improved

performance& Irototype S1 uses a directly modulated red laser diode at a ave length of

?+; nanometers as the light source& The re3uired hori/ontal scanning rate of A+$A5C Hert/

could not "e accomplished ith a simple galvanometer or similar commercially availa"le

moving mirror scanner& The use of a rotating polygon as deemed impractical "ecause of

the polygon si/e and rotational velocity re3uired& t as thus decided to perform the

hori/ontal scan ith an acousto,optical scanner& The vertical scanning rate of A5 Hert/ is

ithin the range of commercially availa"le moving mirrors and is accomplished ith a

galvanometer&

The use of the acousto,optical scanner comes ith a num"er of dra"ac4s7

t re3uires optics to shape the input "eam for deflection and then additional optics to

reform the output "eam to the desired shape&

t re3uires comple. drive electronics that operate at fre3uencies "eteen 1&5 FH/ and

1&C FH/&

ts total scan angle is 2 degrees& Thus$ additional optics are needed to increase the angle

to the desired field,of,vie& Due to the optical invariant$ this optical increase in angle

comes ith the penalty of decreased "eam diameter hich leads to a small e.it pupil& The

small e.it pupil necessitates precise alignment ith the eye for an image to "e visi"le&

+;


36/47

t is e.pensive and ill not$ in the foreseea"le future$ allo the producers to reach the

cost goals for a complete VRD system&

Prototype +-

To overcome the limitations of the acousto,optical scanner$ HT! engineers have

developed a miniature mechanical resonant scanner& This scanner$ in conunction ith a

conventional galvanometer$ provides "oth hori/ontal and vertical scanning ith large

scan angles$ in a compact pac4age& The estimated recurring cost of this scanner ill allo

the VRD system to "e priced competitively ith other displays& Irototype S5 of the VRD

uses the mechanical resonant scanner&& The system as "uilt and demonstrated during the

summer of 12& The VF% resolution images produced are sharp and spatially sta"le&

The mechanical resonant scanner is used in conunction ith a conventional

galvanometer in a com"ination hich allos for an increase in the optical scan angle&

When the mirrors of the to scanners are arranged in such a manner that a light "eam

undergoes multiple reflections off the mirrors$ then the optical scan is multiplied "y the

num"er of reflections off that mirror& 6ptical scan multiplication factors of 5K$ +K and

2K have "een reali/ed& Irototype S5 uses a system ith 5K scan multiplication in the

hori/ontal a.is&

Prototype +.

The third prototype system developed uses the same scanning hardare as Irototype S5

"ut uses three light sources to produce a full color image& n addition the eyepiece optics

have "een modified to allo for see through operation& n the see through mode the

image produced "y the VRD is overlaid on the e.ternal orld&

Present S!enario

n the current version$ a ireless computer ith a touch,pad control is orn on

the "elt& #uch units are largely used "y the production units of many industries$ most of

them automo"ile manufacturers& !i4e a high,tech monocle$ a clear$ flat indo angled in

+?


37/47

front of the technician's eye reflects scanned laser light to the eye& That lets the user vie

automo"ile diagnostics$ as ell as repair$ service$ and assem"ly instructions

superimposed onto the field of vision& The information that the device displays comes

from an automa4er's service,information We" site through a computer running -icrosoft

Windos #erver 5@@+ in the dealership or repair shop& The data gets to the display via an

ordinary 999 C@5&11" Wi,


38/47

%s an alternative$ small green laser are no "eing produced hich use a crystal to

fre3uency dou"le a neodymium %F laser& These devices are larger than desired and are

not directly modulata"le at the re3uired fre3uency& They do hoever$ offer a short term

solution& n the HT! researchers are investigating a num"er of alternatives to "lue and

green laser diodes& 6ne fre3uency dou"ling techni3ue "eing researched uses rare earth

doped fi"ers as the dou"ling medium& % second techni3ue uses ave guides placed in a

lithium nio"ate su"strate for the dou"ling&

The a"ove methods all utili/e a laser as the light source& %dditional or4 is

directed at using non,la/ing$ light,emitting diodes (!9Ds) as the light source& n order for

this to "e successful to primary issues are "eing addressed& The first issue is ho to

focus the !9D output to the desired spot si/e& The second issue is the development offa"rication techni3ues that ill allo us to directly modulate the !9Ds at the desired

fre3uency&

9nter the edge,emitting !9D& Unli4e conventional !9Ds$ hich emit light from

the surface of the chip$ an edge,emitting !9D has a sandich,li4e physical structure

similar to that of an inection,laser diode$ "ut it operates "elo the lasing threshold&

These !9Ds emit incoherent "eams of light that$ hile not so fine as a laser's "eam$

provide a tenfold increase in "rightness& We also use multiple ine.pensive surface,

emitting !9Ds$ each contri"uting a portion of the overall poer$ to achieve high

"rightness&


39/47

Laser safet" anal"sis

-a.imum Iermissi"le 9.posures (-I9) have "een calculated for the VRD in

"oth normal vieing and possi"le failure modes& The -I9 poer levels are compared to

the measured poer that enters the eye hile vieing images ith the VRD& The poer

levels indicate that the VRD is safe in normal operating mode and failure modes&

The scanned "eam is passed through a lens system hich forms an e.it pupil

a"out hich the scanned "eam pivots& The user places themselves such that their pupil is

positioned at the e.it pupil of the system& This is called a -a.ellian vie optical

system& The lens of the eye focuses the light "eam on the retina$ forming a pi.el image&

The folloing figure (fig&1@) compares the illumination of the retina "y a pi.el,

"ased display versus the VRD& nset figures sho schemati/ed light intensity over any

given retinal area in the image& Typical pi.el,"ased displays such as :RTs have

persistence of light emission over the frame refresh cycle$ hereas the VRD illuminates

in "rief e.posures&

+


40/47

Figure.12

Ireliminary tests and calculations of VRD images demonstrated that the system's

poer output ith typical images is "elo the ma.imum permissi"le e.posure (-I9)

limits esta"lished for various lighting schemes& -easures of poer output ith typicalimages indicate that the VRD generates poer on the order of 5@@ nanoatts during

normal operation& This is "elo the :lass 1 laser poer limit of 2@@ nanoatts& f failure

ere to occur$ i&e& if scanning ere to stop in one or "oth dimensions$ the poer limits

indicate the mechanism is still safe& To use the VRD in "righter light conditions$ such as

am"ient daylight$ higher poer levels ill "e needed&

The poer of the lasers in the VRD are ust a fe hundred nanoatts$ and it is

calculated that at these poers$ the laser ould need to continuously illuminate a single

spot in the retina continuously for eight hours "efore any damage occurred and that never

happens in this case&

2@


41/47

The tests ere underta4en for various prototypes assuming the laser source to "e

of pulsed nature$ continuous ave source or as e.tended sources&


42/47

through mode at high luminance levels$ it is an ideal display to replace the "ul4y video

monitor in a fluoroscopic e.amining room& The radiologist could see through the .,ray

display and see the patient as ell& 6ther features such as a display luminance control or

on0off sitch could easily "e included for this application&

Surgery

#urgery to remove a cancerous groth re3uires 4noledge of the groth's location&

:omputed tomographic or magnetic resonant images can locate a tumor inside a patient&

% high luminance see,through display$ such as the VRD$ in conunction ith head

trac4ing$ could indicate visually here a tumor lies in the "ody cavity& n the case that a

tumor lies hidden "ehind$ say$ an organ$ the tumor location and a depth indicator could "e

visually laid over the o"structing organ& %n application in surgery for any display ould

clearly re3uire accurate and relia"le head trac4ing&

&anufa!turing

The same characteristics that ma4e the VRD suita"le for medical applications$ high

luminance and high resolution$ ma4e it also very suita"le for a manufacturing

environment& n similar fashion to a surgery$ a factory or4er can use a high luminance

display$ in conunction ith head trac4ing$ to o"tain visual information on part or

placement locations& Draings and "lueprints could also "e more easily "rought to a

factory floor if done electronically to a Virtual Retinal Display (ith the option of see,

through mode)& 6perator interface terminals on factory floors relay information a"out

machines and processes to or4ers and engineers& Thermocouple temperatures$ alarms$

and valve positions are ust a fe e.amples of the 4ind of information displayed on

operator interface terminals& 9yeglass type see,through Virtual Retinal Displays could

replace operator interface terminals& % high luminance eyeglass display ould ma4e the

factory or4ers and engineers more mo"ile on the factory floor as they could "e

independent of the interface terminal location&

Communi!ations

25


43/47

The compact and light eight nature of the mechanical resonant scanner (-R#) ma4e an

-R# "ased VRD an e.cellent display for personal communication& % hand held

monochrome VRD could serve as a personal video pager or as a video the outside orld& #uch a

versatile display capa"ility is e.pected to provide a significant performance "oost to "oth

aircraft and pilot& When you can also ena"le a pilot to see the normally invisi"le '"loom'

of a radar signature$ or to proect a 'pathay in the s4y' in front of him$ and to

superimpose ireframe or +,D imagery onto the terrain$ it "ecomes even more poerful&

%rmy's vision of the virtual coc4pit also includes a hat you see depends on

here you loo4 concept& %s the pilot loo4s up and out of the coc4pit$ various types of

2+


44/47

targeting$ navigational or terrain overlays ould appear& When pilots loo4 in a donard

direction$ they may see virtual instruments proected onto the eye that literally replace

many of the e.isting dials and multifunction displays that are in coc4pits today&

Weara"le augmented reality displays ncorporated into eyeglasses$ goggles or

helmets$ VRD technology ill display an image that doesn't "loc4 the user's vie "ut ill

instead superimpose a high,contrast monochromatic or color image on top of it& This

a"ility can enhance the safety$ precision and productivity of professionals performing

comple. tas4s&

Te #uture of VRD Tecnolog"


45/47

%onclusion

Various strategic agencies have already started or4ing ith the VRD and ith so

much at sta4e$ status reports on progress are not readily availa"le& *evertheless e can

say that right no$ all those engineers$ fighter pilots and partially sighted people or4ing

ith VRD ill "e struggling ith different facets of the same pro"lem&

The proects of interest in the field are to study the "asic psychophysical

processes of image perception from scanned lasers including resolution$ contrast and

color perception$ to study the interaction of VRD images ith images from the real

orld to enhance the augmented reality applications of the technology$ to study VRD

image perception in partially sighted users$ to design VRD light scanning paradigms to

optimi/e image resolution$ contrast in lo,vision su"ects$ and to design te.t$ image and

computer icon representations for lo vision users and test speed&

f the VRD is capa"le of augmenting our real orld ith the e.tra information$

ho ill our minds handle and integrate it all -ight it fundamentally change the ay

e comprehend information&

6ne day ill e repeat the ords of :aesars Ha4 in utter perple.ity

O Veritas$ Jui est VeritasP

2;


46/47

i"liography

1) #cience X Technology$ The Hindu$ #eptem"er +@$1C&

5) 9ncyclopedia ritannica$ 5@@5&

+) O6ptical engineering challenges of the virtual retinal displayP$ "y oel # Eollin and

-ichael Tidell& HT! pu"lications&

2) O% virtual retinal display for augmenting am"ient visual environmentP$ a masters

thesis "y -ichael Tidell$ HT! pu"lications&

;) OThe virtual retinal display, a retinal scanning imaging systemP$ "y -ichael Tidell$

Richard # ohnston$ David -elville and Thomas %


47/47

1+) &hitl&ashington&edu$ &microvision&com$ &google&com

PS/ $le!troni! &ail Identity /0 ser#us1mariae2gmail1!om

magic leaf _ head up _vrd

Documents