armedia sdk: new features and capabilities

INGLOBE TECHNOLOGIES SRL

WHY ARMEDIA SDK?

Not Just Another AR SDK

• 3D tracking of mobile device is a challenge in many application areas: Mixed reality, Location-aware services, Navigation, Robotics, User Interfaces

• No SDK Vendor is offering scalable 3D Object Recognition and Tracking methods

• The need of integrating different tracking methods in one solution is evident

• Expansion of Virtual Reality requires bridging the gap between AR and VR

What is Required

• Flexibility: provide a variety of methods that can be employed to deal with challenging application scenarios

• Modularity: availability of tracking module that developers can use in any combination depending on their needs

• Mixed Reality SDK: provide tools that enable a smooth transition between AR and VR visualization on mobiles and wearables

• Advanced CV Methods: to be able to identify and track real objects from any angle is the key, accurate estimation of 3D position and 3D orientation

ARMEDIA SDK

Overview of the SDK

• Develop Mobile Applications to recognize and track

• Build powerful Mixed Reality applications, all in one tool

• Bind your project to any available IoT SDK/API, use Beacons, NFC or any other Sensor or communication device

Object Planar Images Locations Device Movements

Features

Library iOS Android

Tracking Rendering Plugins (Unity3D) Tools Examples Developer Portal

Architecture

Application Layer

Tracking Module

OpenCV IMU

Rendering Module

Open Scene Graph

Unity3D

SDK Plugin

TRACKING MODULE

Overview of the Tracking Module

• Different tracking methods independent one of each other

• Tracking methods can be combined

– sequentially

– in parallel

• The Tracking API can be used in association with any other SDK through suitable matrix conversion

• The tracking module is independent of the Rendering Engine

Tracking Module

Tracking iOS Android

Object Planar Location Device Motion

Object Tracking

• Recognize and Track real world 3D objects and view them from any angle

• Provides accurate 3D position and 3D orientation with respect to a given 3D object

Planar Tracking

• Recognize and Track multiple planar images in the same application

• Provides accurate 3D position and 3D orientation with respect to a set of planar images

• Several images

Location Tracking

• Create applications that employ geo-location data from mobile devices

• Provides contextual access to location information

• Dedicated and simple API

Motion Tracking

• Use the Gyroscope and Sensors of the mobile device to interact with 3D contents

• No camera background is available with this method

• Useful for a range of MR applications

RENDERING MODULE

Overview of the Rendering Module

• A visualization layer that can be quickly used in your App

• The rendering components are associated with the tracking methods

• They also provide a number of nice VR visualization features

• Types of contents:

Video 3D files Images

Rendering Module

Rendering iOS Android

Object Planar Location VR Motion 3D Viewer VR Panorama

Rendering: Object

• Associate digital contents with a specific object that is tracked by the Object Tracker

+

Video

3D files

Images

Rendering: Planar

• Associate digital contents with any specific image that is tracked by the Planar Tracker

Video

3D files

Images

+

Rendering: Location

• Associate digital contents to any location defined in the Geo-location Tracker

Video

3D files

Images

+

Rendering: VR Motion

• Create interactive VR viewers using the gyroscope and magnetometer

Video

3D files

Images

+

Rendering: 3D Viewer

• A 3D viewer without background video.

Contents can be manipulated with user’s gestures

Video

3D files

Images

+

Rendering: VR Panorama

• Create interactive 360° Panoramas using equirectangular images and mobile sensors

Video

3D files

Images

+

PLUGINS

Unity3D Plugin

• All the tracking features of the SDK can be used in Unity3D applications.

• The Unity3D Plugin allows to manage Prefabs in the implementation of iOS/Android applications.

SDK TOOLS

ARmedia SDK Toolset

• The App is a set of tools to create and test 3D maps (targets) suitable for the ARMedia SDK and related products/plugins

• Available for iOS and Android

Developer Portal: Target Manager

The Target Manager is a web application that helps generate 3D targets to be used with the SDK

SAMPLE PROCESS

Using a Combination of SDK Tracking Methods

Estimate Approximate Position

•Use GPS or other location techniques to get approximate position

Improve estimation

•Use Camera View and image database to improve estimation

Load 3D Targets

•Load nearby point 3D Target for the tracker

Matching

•Find corresponding points between the camera view and the 3D Target

Tracking

•Track the features and update pose estimation accordingly

3D Targets Can be Generated Through Different Methods

• Reconstruct automatically using photos of the objects – The reconstruction is done by taking a number of

pictures usually 16, 24 or more depending on the actual object/scenario

• Reconstruct using depth camera images (e.g. Kinect)

• Use LIDAR to scan the environment

Tracker Properties

Property Notes

Very accurate • Pixel level precision

Fast • Running at more than 60 fps on iPhone

Wide operating range • Works close and far from target

Robust • Fast recovery from error situations • Tolerating shaking and fast movements of camera • Tolerating different and changing lighting conditions

APPLICATIONS

Some Possible Applications

• Mixed Reality Mobile Games

• Architecture, Engineering & Construction

• Service & Maintenance

• Marketing & Publishing

• Smart Environments

• Cultural Heritage

PLATFORMS

Supported Platforms

iOS version

Android version

Windows version

AVAILABLE

AVAILABLE

NOT PUBLIC

Planned Release

Samsung Gear VR

Zeiss VR

Epson Moverio

NOT PUBLIC

NOT PUBLIC

NOT PUBLIC