augmented reality on the road v7

© Marianne Lindsell 2014. All rights reserved. Slide 1

Throwing Markers Outside the Car

Disclaimer:Information in these slides is representative of the True3D technology of Making Virtual Solid, LLC. There remains some flexibility in actual design to meet the requirements of individual clients. These slides also incorporate the research and opinions of the author at the time of writing. The author takes no responsibility for the accuracy or use of the information contained here.

Marianne Lindsell

Advisor to Making Virtual Solid, LLC

© Marianne Lindsell 2014. All rights reserved. Slide 217:54

1. My Background and approach to AR

2. The True3DTM HUD from Making Virtual Solid, LLC

3. Using it to improve Situational Awareness

4. A Model for Task-focussed AR?


Avatar-mediated Comms

Facial Recognition

Marker Triggered Visualise a

volumeRemote Skilling

Object / Image

Triggered

3D Maintenance Overlay

Earlier Structure

Future StructureVirtual Surroundingsfor the Airborne Camera Straw

See-through solid objects, e.g. walls (camera view or graphics

based on known data)

See against Bright LightSwap Senses

Show PulseSkin Circulation

Add Heat MapRemove Patterns Mark in

Space

Visualise

EnhanceContext

Work with

Faces

Enhance Senses

Guidance & Navigation

Markers & Tracking

Places

Things

People

Lines, Areas, Volumes


Visuo-spatial sketch pad

Phonological Loop

Central executive

Experimentsshow peoplecan recall 4 or 5items immediatelybut recall rate dropsvery quickly after a few seconds

Like a continuous

audio tape of 2 seconds

Assumed to be thesystem that

handlesattention

Based on:Alan D. Baddeley, Essentials of Human Memory, Psychology Press 1999


• Allow a clear view and not ‘get in the way’• Enhance surroundings rather than FoV• i.e. Conformal Representation

(as per NASA Ames/SJSU taxiing studies with untrained crew)

• Remove enhancements when no longer needed


Many planar cross-sections of a 3D volume are projected onto a spinning diffuser in this Perspecta volumetric 3D display. In this example it creates a volume-filling image of a scan for brain cancer.


18 degrees

20 degrees


NOTE: Objects not drawn to scale

1. Sun filter2. Eye lens group3. Folding mirror(s)4. Relay lens group5. Projection screen5. Path of laser spot6. Projection screen actuator7. Laser beam8. Scanner9. Beam focuser and profiler10. Laser11. Circuit board12. RGB raster projector

a.


Installation volume is around 7 litres, with

much flexibilityin the shape

Power consumption in region of 20W, increasing

or decreasing according to requirement and usage


But navigation is not top priority (for automotive OEMS):

1) ADAS (Advanced Driver Assistance Systems) – mainly safety

2) Navigation

3) Heads Up Vehicle Management (e.g. Controls, Aircon, Entertainment)

MVS won European Satellite Navigation Competition 2011

The Virtual CableTM is shown 8 deg above path being tracked, as per NASA Ames/SJSU studies

https://www.youtube.com/embed/VMTlN1TTWuQ

https://www.youtube.com/embed/VMTlN1TTWuQ


Looked-But-Failed-To-See Crashes

collisions in which a driver fails to detect another road user or object

despite looking at it

Why do LBFTS crashes occur?

Inattentional Blindness & Change Blindness

Third most frequently occurring driver error(Brown, 2005; Treat, 1980; White & Caird, 2010)

The failure to see highly visible objects we may be looking at directly when our attention is elsewhere.

(Mack & Rock, 1998)

(AKA cognitive tunnelling, cognitive capture,channelised attention)

After - Does Familiarity breed inattention? Why drivers crash on the roads they know bestSamuel G. Charlton, Traffic & Road Safety Research Group, Waikato University


Situational Awareness is quite important

1. Perceive items around youe.g. spotting a road sign

2. Comprehend their meaningi.e. understanding it is a warning, e.g. child symbol outside a school -so I can expect children crossing

3. Project their status into near futuree.g. I’m driving at 40 mph, so I’d better slow down in case a child runs out in front of me

Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32-64.Endsley, M. R. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37(1), 65-84.

*Mica Endsley’s 3 levels of Situational Awareness:* Mica Endsley is presently Chief Scientist of the U.S. Air Force


Already some OEMs are simulating warnings of shunt riskAs we realise V2V comms, could we show likely paths of other vehicles?

• Joining from a hidden entrance?• Cutting across lanes near us?• Approaching us at junctions?

Surprisingly, visual AR compares favourably with LIDAR and Radar

A clever scheme for Level 2 SA

Mark traffic features for Level 1 SA

Perceive items around you

Comprehend their meaning

Project status into near future

Show play-out markers for Level 3 SA

Traffic Vectors?


Embedded Mobile

• Compact• Available (e.g. driver’s

cellphone, with potential to restrict distracting SMS)

• High processing power• Extensive storage• Updateable online

• Distributed – allowing for preprocessors (e.g. camera DSPs)

• Deterministic RTOS• More reliable for ADAS

• QNX• Automotive Linux• Mentor Nucleus

But only for infotainment (IVI) and navigation

• Android Auto• Apple CarPlay• Mirrorlink• GENIVI®


Faster React to unexpected

More AccurateInterpret response in keeping withsocial principles

More ReliableRespond better to other humans

Condenses know-how ofmany humans

Will still be driving for a long time

Embedded ADAS tending toward autonomous controle.g. auto-brake

But once control ripped from driver, difficult to return itUnless driver SA can be sustained

Can AR help the driver stay aware of the automation?

Automation Human Driver


• A sort of early HUD

• AR was originally a work horse

Pic

ture

, Geo

ff A

dam

s,

NR

LCEO

fan

tasy

ru

gby

leag

ue

The WWII Reflector Sight

Pic

ture

, Wik

iped

ia

Fighter HUD <> Car HUD <> Smart/AR Glasses

But there is arguably a common thread:

• Minimise clutter & obstruction

• Maximise situational awareness

Moi?

http://en.wikipedia.org/wiki/File:Mark_III_free_gun_reflector_sight_mk_9_variant_reflex_sight_animation.gif

http://en.wikipedia.org/wiki/File:Mark_III_free_gun_reflector_sight_mk_9_variant_reflex_sight_animation.gif


AR Aerial Video Context Mapping System

The Churchill Augmented Reality mappingsystem, ARS, overlays street and otherinformation directly over live sensor videofrom aircraft-mounted gyro-stabilizedcameras such as the FLIR Star Safire and L-3 Wescam MX series gimbals. MGRS gridlines, radar targets and much more canalso be turned on or off at will --eliminating the need to take eyes off thetarget, and radically enhancing the abilityof sensor operators to quickly andaccurately direct ground assets to wherethey are needed. Get the most out of yourturret by interfacing it to an ARS system foran immediate improvement in utility andaccuracy.

Note registration between sensor video and VR map surround

Churchill Aerial AR.wmv




augmented reality on the road v7

Technology