state of augmented reality & hololens
TRANSCRIPT
PowerPoint Presentation
AUGMENTED REALITY
Hkan [email protected]@haqwin
1
Physical reality
Augmented reality
Virtual realityvirtuality continuum
Lets not hope our augmented reality will be this overwhelmingBut still the ability to be nudged with ongoing phone call informationAnd if you like messages and notificationMaybe just not all at once3
Lets not hope our augmented reality will be this overwhelmingBut still the ability to be nudged with ongoing phone call informationAnd if you like messages and notificationMaybe just not all at once4
HISTORY
But before we go what we can experience now and in the future we should start out by looking into some history
5
Ivan SutherlandFather of computer graphicsMITSketchpad
Ivan Sutherland Well know graphics workSketchpadLight pen UI and ability to sketch on screenPaper on a head mounted display
6
1968
University of UtahWith Bob Sproul he created the first HeadMountedDisplay as he figured that would be the best way to experience a 3D model
7
The ultimate displaySimple wireframes
8
Sword of Democles
As it was held in place with a large metal construct from above + to be able to track the movement9
1980
EyeTapSteve Mann
Steve Mann
11
Steve Mann with one of his inventions early iterations of EyeTapHeadMounted computerBut also a pioneer in AR
12
1990
Augmented RealityTom Caudell
The work with AR has been developed by Boeing as a way to help in aviation industryResearcher Tom Caudell coined the term AR
It was used to help tech + Handling the service+ but also more full exoskeleton devices to remote control system
14
NASA X-38 escape podThat was created to sit on the ISS and be able to safely get up to 7 people back to earthThis could be remote controlled and here we see an overlay of the landing site and obstacles on top of the thick cloud to be able to have a safe landingUnfortunately it was put on hold and the project was cancelled
15
NASA X-38 escape podThat was created to sit on the ISS and be able to safely get up to 7 people back to earthThis could be remote controlled and here we see an overlay of the landing site and obstacles on top of the thick cloud to be able to have a safe landingUnfortunately it was put on hold and the project was cancelled
16
2000
Up until this point most work has been done in aviation industry and on universities but around 2000 things changed
17
Hirokazu KatoARToolKit
Japanese Hiro ccretaed ARToolKitAnd now and it started to spread18
You were able to overlay an image through theThe printed media started to take some interest around nowThrough a web cam or laptop with cam19
both for the common developer and in more commercial and consumer use and we started to se ad campaigns with this techniques
20
21
2013
Google presented their glass23
Although very limited and actually nor much more than a head mounted display it show some of the future in head mounted displays and self containment
That brings us up till now
24
NOW
HUD displays and Aviation + medical are currently heavy users of AR already but for consumer and enterprise users there are not as many devices out yet25
Meta 2
The META 2
90-degree field of view.2560 x 1440 display resolution.Sensor array for positional tracking.
26
Project Tango
Googles system that uses depth cameras and acceelerometes to handle the sorrundings
27
Project Tango
Googles system that uses depth cameras and acceelerometes to handle the sorrundings
28
HoloLens
Physical reality
Augmented reality
Virtual realityMixed reality
Inside the machine
31
Output
See through lenses2 HD light engines Pupilar distance calibration
Surround speakersSpatial sound engine
32
Output
See through lenses2 HD light engines Pupilar distance calibration
Surround speakersSpatial sound engine
33
Output
See through lenses2 HD light engines Pupilar distance calibration
Surround speakersSpatial sound engine
34
Output
See through lenses2 HD light engines Pupilar distance calibration
Surround speakersSpatial sound engine
35
Sensors
4 environment sense cameras1 depth camera1 12MP HD video camera1 ambient light sensor1 inertia measurement unit4 microphones
36
Sensors
4 environment sense cameras1 depth camera1 12MP HD video camera1 ambient light sensor1 inertia measurement unit4 microphones
37
Sensors
4 environment sense cameras1 depth camera1 12MP HD video camera1 ambient light sensor1 inertia measurement unit4 microphones
38
Sensors
4 environment sense cameras1 depth camera1 12MP HD video camera1 ambient light sensor1 inertia measurement unit4 microphones
39
Sensors
4 environment sense cameras1 depth camera1 12MP HD video camera1 ambient light sensor1 inertia measurement unit4 microphones
40
Its the combination of stuff that makes it uniqueNo strings attached - only this device - no external computers 500g headheld device with everything in it
But that is of course not what makes it unique - its the mixed reality mode where the real world and the digital world meet
41
Processing
Just an ordinary Intel based Windows 10 machineWith an extra powerful HPU - 24 DSP cores, capable of 1 trillion calculations per second (using 1GB mem)+ Small laptop grade CPU for all the other stuff (the HPU is around 200 x as powerful as a normal CPU) (using 2GB mem)
And the usual PC related stuff.. WiFi, Bluetooth, 60G disk and all that
42
Processing
Just an ordinary Intel based Windows 10 machineWith an extra powerful HPU - 24 DSP cores, capable of 1 trillion calculations per second (using 1GB mem)+ Small laptop grade CPU for all the other stuff (the HPU is around 200 x as powerful as a normal CPU) (using 2GB mem)
And the usual PC related stuff.. WiFi, Bluetooth, 60G disk and all that
43
INTERACTION
InteractionGazeTapHoldBloomSpeech
Speach
Possibilities (add leap motion?)
InteractionGazeTapHoldBloomSpeech
Speach
Possibilities (add leap motion?)
InteractionGazeTapHoldBloomSpeech
Speach
Possibilities (add leap motion?)
InteractionGazeTapHoldBloomSpeech
Speach
Possibilities (add leap motion?)
Interaction
GazeTapHoldBloomSpeech
SelectPlace Face meBigger/smaller- Hey Cortana - record video, take picture, etc
Advantages
Self contained - Inside out - Windows holographicNo wires! - Knows its surroundings
The goal here is to push the tech. The usages can be as wide as ArchitectureMedicine Home furniture / decoratingManufacturing
50
HURDLES
What are the biggest hurdles for widespread use of AR/MR?
Price of course but that will change quicker than expectedINTERACTION
51
Finger gestures
On screen interaction
Simplified gesture set
Interaction
The interaction are quite like the Leap Motion sensor barHand and finger input, seems to be quite detailed but I havent been able to confirm that myself
Speech?
Consistency and StandardsDiscoverability FeedbackAffordance
Interaction
Consistency and StandardsThis is something we have to work on right now. How do you navigate, how do you interact. How do you interact with the digital part of a hologram. The gestures for the HoloLens is just the beginning.
DiscoverabilityThe discoverability need to be close to the real world. But you want to be able to manipulate and interact in ways that are not limited by the reality. This is the real challange.
FeedbackThe user should get feedback on actions. And in mixed reality this feedback need to work with real objects.Give virtual object some snapping/persistanse or visual queues when interacting with real objects
Affordance - Depends on context;What makes sense for different scenarios, in simulation, scaling do not make sense. But in analysis or some creative activities it makes perfectly sense. Is the object rooted in the real world?
FUTURE
Rony Abovitz
Magic Leap
The $1B vaporware?
Rony Abovitz
[T]he hardware projects an image directly onto your retina through an optics system built into a piece of semitransparent glass (the product wont fry your eyeballs; its replicating the way we naturally observe the world instead of forcing you to stare at a screen). The hardware also constantly gathers information, scanning the room for obstacles, listening for voices, tracking eye movements and watching hands.55
HoloLens 2
- Electrochromic dimming module- Better gesture and finger control
- FoV- More power / battery56
QUESTIONS?
Hkan [email protected]@haqwin
57
Hkan [email protected]@haqwinTHANKS!
58