the emerging trends of augmented reality and virtual ...€¦ · • for developing mobile,...

Eric Wittner

The Emerging Trends of

Augmented Reality and Virtual

Reality in Design

What are we talking about?

Introduction & Terminology

VR vs AR

VR - Virtual Reality

Being there

AR - Augmented Reality

Interacting with outside world

MR - Mixed Reality

Mixed presence

Immersive Computing

• Fundamental computer human interaction

that is better than anything else

• AR got breakthrough with Pokemon GO,

VR with Rift and Playstation VR

• Passive content / 360 experiences

lowest barrier entry for VR

• Terminologies are not yet clear

We are at the very beginning of this.

Terminology

“Experience”

The term “experience” is used for

apps or the actual content. Means it

is common industry jargon, e.g.

Oculus uses the term to list the

apps. As a consequence, people

refer to a VR Experience not only as

user experience but also as an

asset.

Mobile VR vs Console VR vs Desktop VR

• Low price

• Broad reach

• Graphics limited

• Bandwidth limited

• Only 3 DoF

• Easy to use

• Entertainment

content

• Walled garden

business model

• High cost

• Complex setup

• Free movement

• Advanced input

capabilities

Seated vs Standing vs Room-scale

Locomotion & Teleportation

Teleportation & Artificial Locomotion

• 45% of experiences avoid locomotion at all due to motion sickness

• 42% use teleportation / portals (= de-facto standard)

• 5% use artificial locomotion (game pads)

• 8% use other techniques

Source:

Teleportation and Locomotion from the Trenches: What Movement is Right for You. By Ram Ramakrishnan & Janet Brown. GDC 2017.

Input Devices

Applications and UsesWho is adopting

VR/AR Market Trends

VR- Education, training, planning

• Mobile• Easy, cheap to distribute

• Desktop• Cheaper than travelling to a site• Best VR experience possible

AR/MR- Field operations, resource management, planning

• Handheld• Use existing mobile devices

• Head-mounted• In the field, full scale; In the office, small scale• Hands-free for rapid response• Most immersive, collaborative 3D experience possible

Demo: Viewing a proposed building

ArcGIS 360 VRMobile focused VR Experience

ArcGIS 360o

VR - Vision

“Immersion made easy.”

“Putting the world in perspective,

past, present, and future

quickly and easily.”

(Currently an Esri Labs project, will evolve into a full offering)

Use Cases

Simple creation of mobile VR demos for

the public To showcase urban redevelopments to the public, CityEngine

users like the City of Zurich are looking for a simple CE-

scene-to-VR solution that is easy to use (= one-click publish)

and easy to setup (= mobile VR that does not require high-

end PC). Planned to be used on trade shows / booths, public

show rooms / installation, and architectural competition

viewing events.

Quick immersion into design to experience

view impactCityEngine users interested in VR would like to quickly

immerse into their 3D scenes to review the design. However

they want a simple, iterate-able workflow and not a

complicated 3D data pipeline to Unity. Also they want to

share the VR experience (mainly for review by peers, not yet

by the public). The typical design question that VR can

answer better than every other visualization: “How does the

new neighboring building impact the view from the balcony?”.

ArcGIS 360 VR Experience & 360 Viewer

• A new Esri supported format: .3VR

• Composed of multiple rendered photospheres

• Can store views from multiple locations

• Can switch between locations using visual bookmarks in scene

• Each location can store multiple states, such as design scenarios

• Can switch between scenarios

• How is it authored: Created from CityEngine scenes, with plans in the

future to support authoring from other applications, and consumption of

spherical photography

• How is it viewed: In a VR application for Samsung Galaxy (eventually

other devices). Additional support for viewing by a web app.

Roadmap

Web app for VR Experience

Web app for simple screen viewing

360 Photo Support

Refined view to view selection

Measurement

Pop-Ups

Depth

Video/Animation

Near Term

Mid and

Longer Term

ArcGIS 360 VR

Demo: BIM/IFC,

3D Utilities

VR/AR with ArcGIS RuntimePrivate beta in progress …

ArcGIS Runtime Beta to support Virtual and Augmented

Reality• Enhance existing ArcGIS Runtime SDKs

- VR: Add “stereo display” rendering mode

- AR: Transparent background to render on video/camera feed

- Private Beta began in November 2017

- .NET and iOS SDKs available today

➢ Email ArcGISRuntimeARVRBeta@esri.com to request invite to beta program

Research combining ArcGIS Runtime and Gaming engines

• Runtime can bring GIS content to existing gaming engines

• Good for VR/AR/MR but also any “gaming type” application that needs GIS

• Advantages:

- Easy cross hardware VR/AR development

- Mixed Reality (HoloLens) development

- Good VR/AR/MR UI design experience

- Integration with the existing VR/AR/MR community

- Physics and other 3D effects

• We need your feedback

- What type of apps would you build with this approach?

- How important is this architecture for your organization, your customers?

Unity and Unreal

Desktop VR with Unity and Unreal

Unity

• Cross-platform game engine

• For developing mobile, browser, console, and desktop games

• Affordable ($0/35/125 per month)

• (Relatively) Easy to use

• Used beyond entertainment e.g. for architectural visualization…

Entry point to develop VR apps for HTC Vive, Oculus Rift, Gear VR, Hololens, …

CityEngine: Select, «Export Models...»

Choose FBX, Tweak Settings, & Click Finish

Unity: Drag&Drop FBX to Assets & into scene

Move «Main Camera» to center

Enable VR

• Edit, Project Settings, Player

• Check «Virtual Reality Supported»

• Add OpenVR Support

- Click «+», «OpenVR»

- Drag above «Oculus»

Start in VR: Press «Play» and experience it in HTC Vive

Scene Tweaks: Scale

• Click mesh in «Assets»

• Scale Factor «100»

• Apply

Scene Tweaks: Tree Transparency

• In «Materials», select

all Tree Materials

• Set «Rendering Mode»

to «Cutout»

VR: Add Locomotion

• Window, «Asset Store»

• Search «SteamVR Plugin»

• «Import»

• «SteamVR_Settings» Dialog

- «Accept All»

Add «Player»

• Delete «Main Camera»

• Drag in

«SteamVr\

InteractionSystem\

Core\Prefabs\Player»

Add «teleport» prefab

• Drag in

«SteamVr\

InteractionSystem\

Teleport\

Prefabs\Teleporting»

Add Teleport Points

• Drag in multiple

«SteamVr\

InteractionSystem\

Teleport\

Prefabs\TeleportPoint»

• Optional: Increase

their scale

In VR, Teleport Using Click on Wheel

Teleport anywhere

• Click Mesh in Assets

• «Generate Colliders»

• «Apply»

Teleport anywhere

• Double-click street

• Copy/Paste

• Drag

«SteamVr\

InteractionSystem\

Teleport\

Scripts\TeleportArea»

on street copy

In VR, Teleport Anywhere on Street

CityEngine 2017.1 – Architectural Visualization with Unreal

Engine