a step-by-step guide to building a mobile outdoor ar application

A Step-by-step guide to

Building a Mobile Outdoor AR

Application

Gun Lee

21 Nov. 2013

SYMPOSIUM ON MOBILE GRAPHICS

AND INTERACTIVE APPLICATIONS

Outdoor AR Browsers

Wikitude Layar Junaio

Layar-Geoinfo.mp4

Wikitude-ARTravelGuide-part2.mp4



SW Dev. Tools & Frameworks

High-level content description

AR Browsers

Hard to customize app logic and UI

Low-level functional modules

Programming libraries

Lack of higher-level abstraction of AR scene

and content (BYO)



Outdoor AR Framework High-level abstraction and low-level

programming support in a seamless way.

Low-level functional modules

- Flexible and highly customizable

Ready-to-use high level components

- Focus on building the interface, behavior and

content of the application rather than system

integration

The high-level and low-level tools share the

common scene (or POI) data structure

- Both skilled and novice developers can work

together under the same framework.



Apps built with OAR Framework

[CityViewAR 2011] [CCDU 3D 2012]

[GeoBoids 3D 2012] [AntarcticAR 2012]

//VBOXSVR/data/_ARIntro/GeoBoids.m4v

//VBOXSVR/data/_ARIntro/CityViewAR.m4v

//VBOXSVR/data/_ARIntro/CCDU3D.m4v

//VBOXSVR/data/_Publication/ISMAR2013/AntarcticAR.m4v



Outdoor AR Framework

Covers full tier of ready-to-use

components, yet highly customizable

Mobile Device (Client)

Mobile OS

Mobile AR Application

Scene

data

Media

data

Mobile SW Library

Server

Server OS

Application Service

Scene

data

Media

data

Server SW Library Data

Communication

(Optional)



Server Software and Tools

Web Client

(Browser)

Server Mobile Client

(App)

HTML Web-based

Authoring Tool

RESTful API

Scene

Database

AR View

Data

Manager

Local

cache

Javascript



Server Software and Tools

Web-based Authoring Tool

Please sign up for closed beta – Jan 2014



Outdoor AR Library

For mobile app (client)

Android (v2.2 or above)

Google Maps API

Scenegraph based rendering engine

OpenGL ES (1.1 and 2.0)

Free download from our website:

http://www.hitlabnz.org/mobileAR



Outdoor AR Library Structure

Functional Component Layer

Service Component Layer

Map

Component

Scene Data Manager

AR

Component

List

Component

Tracking Data Manager

Scene Data

Structures

Tracking

Sensors

3D Graphics

Rendering UI Views

Tools &

Utilities

3D

Sound

Map View AR View List View Views

Controllers

Models



Model-View-Component

Controller

Model Scene & Tracking

Data Managers

View Visualization &

UI Elements

Application Logic &

Data Binding



Mobile App Software Library

Customization by extending components

Customized AR scenes

- Customize scene class data structure

Customized user interface

- Customize UI layout

- Customize UI behavior (e.g. onSceneSelected)

- Motion and touch gesture based interaction



Step-by-step Guide

Setup the development environment

Create an Android app

Using the Outdoor AR Library in your app

Adding outdoor AR scenes with 3D models

Customizing UI

Run your app on a Google Glass



Setup the dev environment

Prerequisite

Android Development Environment

- http://developer.android.com/sdk/index.html

- Eclipse, Android SDK

Google Play Services Library (for maps)

Download the Outdoor AR Library


- Download, Tutorials, Forum



Setup the dev environment

Import the library and sample projects

OutdoorARLibrary

SampleOAComponent

Dependency

Google (not Android) API 4.1.2 (API Level 16)

Google Play Services Library

- extras/google/google_play_services/libproject

Android Support Library 4

- extras/android/compatibility/v4/

android-support-v4.jar



Create an Android app

Create a new Android Application project

Blank Activity

- “An activity is a single, focused thing that the user

can do.” – Android SDK API Reference

Anatomy of an Android App project

src

res & gen

assets

AndroidManifest.xml

Add a “Hello MGIA” button



Using the OA Library in your app

Add the Outdoor AR Library to your

project’s build path

Add permissions in the manifest

Add a new AR view (activity)

Create a custom class inheriting

OAARComponentBase

Add activity to the manifest

- Full screen landscape style preferred

Wire a button to start the AR view



Adding scenes with 3D models

Preparing 3D model

OBJ/MTL with JPEG or PNG textures

OpenGL Coordinate frame

- X-right, Y-up, Z-out from screen

- North = -Z

Placing the 3D model in the real world

Latitude, Longitude

- http://itouchmap.com/latlong.html

Optionally, elevation from the ground plane

SketchUp – 3D model on Google Earth



Adding scenes with 3D models

Adding 3D models to the project

assets/OutdoorAR/3Dmodels(/name/model.obj)

Adding scenes to the data manager

Override onSetupScenes()

Handling the scene selection event

Override onSceneSelected()

Using a mock location

Override onCreate()

Sensor manager’s enableMockLocation()



Loading and managing scenes

Add scenes by code – difficult to manage

Load from assets or local file storage

OADataManagerAssets

OADataManagerLocal

SQLite or XML file

SQLite Database Browser

Default path

- (assets/)OutdoorAR/scenes.db

- (assets/)OutdoorAR/scenes.xml



Custom scene class

OAScene

Lat/Lon, 3D model, Name, Description,

Category

Custom scene classes extend OAScene

Override OADataManager’s loadScene()

to load in custom properties of your scene

Cast OAScene to your custom class in

onSceneSelected() or onTouchedScene()

Different scene classes can live together

- if(scene instanceof MyCustomScene)



Customizing UI

Add layout resource

Override setupUILayout()

LayoutInflater - read in the layout res file

addContentView() - overlay UI layout on top

of AR view

findViewById() - get reference to the UI

elements

onSceneSelected() vs. onTouchedScene()



Map and List views

OAMapComponentBase, OAListComponentBase

Similar class structure with OAARComponentBase

- Load scenes from a data manager

- setupXYZ() methods

- onSceneSelected() callback

Map components needs API key

Google Maps Android API v2

- OAGoogleMapsV2ComponentBase

https://developers.google.com/maps/documentation/an

droid/start



Run your app on Google Glass

Startup with AR

Set AR view as a launcher activity in the app

manifest file

Aim to select

Use Timer to trigger touch events at the

center of the screen

Override onResume() and onPause() to

start/stop the timer



Other tips for Google Glass

Touch gestures

onGenericMotionEvent()

Swipe forward/backward/up

Tap, double tap, tap and hold, etc.

Head motion

Add orientation listener to sensor manager

- getSensorManager().addSensorListener()

Avoid conflicts with AR scene viewing motions

- Time-outs



Other tips for Google Glass

GPS sensor blocked

Stream location from a phone through

Bluetooth

Mock location

Turning off camera background

Application dependent

Override setupOptions()

enableCameraBackground()



CityViewAR for Glass

Demo

Thank you!



More Information

Website


http://arforglass.org

Gun Lee

[email protected]

Mark Billinghurst

[email protected]

a step-by-step guide to building a mobile outdoor ar application

Technology