extended reality (xr) with arcgis - esrimixed reality mr merging of information and real world...
TRANSCRIPT
![Page 1: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/1.jpg)
Stefan Arisona, Pascal Mueller, Adrien Meriaux, Rex Hansen
Extended Reality (XR) with ArcGIS
![Page 2: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/2.jpg)
Overview
![Page 3: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/3.jpg)
Being there
Oculus RiftOculus QuestHTC ViveHP Reverb
VR – Virtual Reality
![Page 4: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/4.jpg)
Interactingwith outside world
SmartphonesTablets
AR – Augmented Reality
![Page 5: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/5.jpg)
Mixed presence
HoloLensMagic Leap
MR – Mixed Reality© 2017 City of Zurich
![Page 6: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/6.jpg)
XR – Extended Reality
Extended Reality
Physical Reality
The real worldaround us
Augmented Reality AR
Overlay of informationonto the real world
Mixed Reality MR
Merging of informationand real world
Virtual Reality VR
Immersing in a digital world
![Page 7: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/7.jpg)
Positional Tracking for XR
![Page 8: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/8.jpg)
Inside-out vs Outside-in Tracking
Inside-out: Cameras on HMD, marker-less tracking of position changes in relation to environment
Outside-in: Cameras placed in stationary locations & markers on devices (HMD & controllers)
![Page 9: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/9.jpg)
Inside-out motiontracking, but needsinitial position
Positional Tracking for MR
![Page 10: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/10.jpg)
Global Positional Tracking for AR (and MR)
Accuracy:• GPS: ~10m• Wi-Fi: ~1m• RFID / Beacon: ~1m• UWB: ~0.1m
![Page 11: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/11.jpg)
XR Devices
![Page 12: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/12.jpg)
... and many more devices, each with
different features
![Page 13: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/13.jpg)
Premium VR• Inside-out or outside-in• 6 DoF• High cost, requires PC• Great graphics, but cables
• Inside-out• 6 DoF• Low cost, standalone• Limited graphics, no cables
Mobile VR
![Page 14: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/14.jpg)
Special Mention for VR: Oculus Quest• Standalone mobile VR device that optionally can be
connected to a PC with a USB cable (Oculus Link) Mobile and Premium VR in one
• Provides both controllers or hand tracking
![Page 15: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/15.jpg)
Special Mention for MR: Hololens 2• Standalone MR device with a larger FOV• Windows MR platform provides many features e.g. spatial
anchors, allowing for apps where VR & MR user can collaborate• Contents (pixels/video) can be streamed from Unreal Engine
without cable, allowing also for PC graphics
![Page 16: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/16.jpg)
XR User Interface Concepts
![Page 17: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/17.jpg)
• Common UX pattern for AR, MR & VR
• Collaborative • Intuitive, people can
relate to table• No motion sickness
Table-scale aka the “Tabletop” UX
© 2017 City of Zurich
![Page 18: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/18.jpg)
• Still the most used locomotion techniquein VR
• No motion sickness• May result in loss of
orientation
Teleporting
![Page 19: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/19.jpg)
• Fast and reliable• Complicated to use (like game pads)• Established UI concepts• Comfortable
Controllers Hand tracking
• Still a tiny bit laggy• Intuitive• UI concepts are in its infancy• Not comfortable (arms need to be up)
![Page 20: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/20.jpg)
![Page 21: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/21.jpg)
XR with ArcGIS
![Page 22: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/22.jpg)
GISdata
Unity / Unreal Engine for developing XR apps
ArcGIS Runtimefor developing AR apps
CityEngine CityEngine VR Experience Premium VR app
ArcGIS 360 VR Mobile VR app
ArcGIS plugin forUnity & Unreal
![Page 23: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/23.jpg)
GISdata
Unity / Unreal Engine for developing XR apps
ArcGIS Runtimefor developing AR apps
CityEngine CityEngine VR Experience Premium VR app
ArcGIS 360 VR Mobile VR app
ArcGIS plugin forUnity & Unreal
![Page 24: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/24.jpg)
Use CasesSimple creation of mobile VR demos for the public To showcase urban redevelopments to the public, CityEngine users like the City of Zurich are looking for a simple CE-scene-to-VR solution that is easy to use (= one-click publish) and easy to setup (= mobile VR that does not require high-end PC). Planned to be used on trade shows / booths, public show rooms / installation, and architectural competition viewing events.
Quick immersion into design to experience view impactCityEngine users interested in VR would like to quickly immerse into their 3D scenes to review the design. However they want a simple, iterate-able workflow and not a complicated 3D data pipeline to Unity. Also they want to share the VR experience (mainly for review by peers, not yet by the public). The typical design question that VR can answer better than every other visualization: “How does the new neighboring building impact the view from the balcony?”.
![Page 25: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/25.jpg)
GISdata
Unity / Unreal Engine for developing XR apps
ArcGIS Runtimefor developing AR apps
CityEngine CityEngine VR Experience Premium VR app
ArcGIS 360 VR Mobile VR app
ArcGIS plugin forUnity & Unreal
![Page 26: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/26.jpg)
![Page 27: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/27.jpg)
GISdata
Unity / Unreal Engine for developing XR apps
ArcGIS Runtimefor developing AR apps
CityEngine CityEngine VR Experience Premium VR app
ArcGIS 360 VR Mobile VR app
ArcGIS plugin forUnity & Unreal
![Page 28: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/28.jpg)
![Page 29: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/29.jpg)
![Page 30: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/30.jpg)
GISdata
Unity / Unreal Engine for developing XR apps
ArcGIS Runtimefor developing AR apps
CityEngine CityEngine VR Experience Premium VR app
ArcGIS 360 VR Mobile VR app
ArcGIS plugin forUnity & Unreal
![Page 31: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/31.jpg)
Augmented Reality with ArcGIS Runtime
• Available for production use- AR for mobile platforms
- iOS, Android devices- Not for VR or MR
• AR capabilities delivered with:- ArcGIS Runtime SDKs for Android, iOS, Qt, and .NET- Open source toolkits to work with hardware
![Page 32: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/32.jpg)
Supported Experiences and Use Cases
• Flyover- Use case: Education- Intuitive navigation of a scene
• Tabletop- Use case: Design- Lock scene to a surface
• World scale- Use case: Inspection- Overlay scene on the real world
![Page 33: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/33.jpg)
ArcGIS RuntimeAugmented Reality Demo
https://devpost.com/software/sonderfly
![Page 34: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/34.jpg)
Challenges and considerations
• Your user’s ability and expectations• Device capabilities
- AR sensors- Positional accuracy
• Data accuracy in 2D and 3D space
![Page 35: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/35.jpg)
GISdata
Unity / Unreal Engine for developing XR apps
ArcGIS Runtimefor developing AR apps
CityEngine CityEngine VR Experience Premium VR app
ArcGIS 360 VR Mobile VR app
ArcGIS plugin forUnity & Unreal
![Page 36: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/36.jpg)
Developing with Game Engines
• Premium rendering experience• Easy cross hardware development• Existing developer community• Physics, animation, other effects
![Page 37: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/37.jpg)
ArcGIS integration with Unity and Unreal Engine
• Need a developer solution that targets - All XR experiences- Any 3D “gaming app” that needs GIS
• Unity and Unreal Engine plug-ins - Provide APIs to access ArcGIS services and local data- Display in and honor real world/geographic coordinate space- Support local and global (open world) experiences- Integrate with game engine developer experience- Include SDK resources (doc, samples, API ref, demos)
![Page 38: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/38.jpg)
ArcGIS capabilities in Game Engines
• Raster tile layers - Display- Elevation
• Scene layers- 3D objects - Integrated mesh- Point cloud
• Feature layers- Points, lines, polygons
• Operations- Geocoding- Routing
![Page 39: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/39.jpg)
![Page 40: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/40.jpg)
ArcGIS for Game Engines Beta
• Beta program open for registration• First beta deliverable Q3 2020• First production release Q1 2021
Sign up now! http://esriurl.com/a4ge
![Page 41: Extended Reality (XR) with ArcGIS - ESRIMixed Reality MR Merging of information and real world Virtual Reality VR Immersing in a digital world Positional Tracking for XR Inside-out](https://reader031.vdocuments.us/reader031/viewer/2022012002/609e663c7d58d055a94cea99/html5/thumbnails/41.jpg)