layar, the next mass medium
DESCRIPTION
Xuan Wang from Layar's developer support team explains how Layar works and how to start making your own layers.TRANSCRIPT
Creating Augmented Reality Experience with Layar Xuan WangDeveloper Support, Layar
Content• General Intro:
• What is Layar ?
• What does it offer ?
• How to create a layer ?
• Q&A, Discussion
• 3D modeling in Layar:
• What’s possible now?
• What are the best practices ?
• How to get 3d models in a layer ?
• Q&A, Discussion
GPSWhere am I?
CameraWhat do I see in the real world?
Compass In which direction am I looking?
AccelerometerHow is my phone oriented in the world?
Internet connectivityTo connect to the digital world
Gyroscope (optional)To provide a smoother experience
Information Service
Earthquake information
Experience - Berlin wall
WTC
Future - UAR
Brands/Agents: Audi A1, Moscow
Events
Social Service -Tweets Around
Foursquare
Games - ScavengAR Hunt
Uninvited Art Exhibition, MoMA
Layer Features Overview
• Support for icon/images/3D models
• Support for animated POIs
• Sharing to facebook, Twitter and Developer’s server
• Actions: call, email, share, web page, audio, video, etc
• Gaming elements: autotriggers, relative POIs, point to point
Open Platform
Brands, publishers Developers Layar End userContent Layers
Distribution
Search & Discovery
The Layar ecosystem
Layar Platform Components
Browser Player Publishing
Layar Connect - Layer Management System
• Publishing API for Third party layer management system
• End-to-end and one-stop-shop
Layar Platform Architecture
Layar server
Fixeddata
Layar publishing
environment
Layer service provider
Layer content sources
Get layersGet POIs
Layer definitio
ns
Layer Developer API
Get POIs
Create layer
Get POIsView POI information
View POI information
GPS data
Layar environment
Third-party environment
Legend
How to create layers ?
• Layers can be built using simple web technology
• You can use programming languages like PHP, MySQL, Java etc as long as the response is in JSON format
• 2 step process:
1. Create a layer definition in the Layar publishing environment (or using layar connect calls)
2. Create a layer service for delivering the POI content to the Layar app (Based on Developer API )
Step 1: Publishing environment
• http://layar.com/publishing
• Add all the static content and metadata of a layer
• Listing details
• Look & feel
• Service provider URL (connect to your web service)
Step 2: Layer service
• Create a layer service for delivering the POI content to the Layar app
• The following elements are needed to set up a layer service:
• A public web server
• A database
• The response should be in JSON format
Third party tools
• Instead of creating your own layer service, you can use one of the third party tools developed by the community, e.g.:
• PorpoiseOpen source server software, requires programming skills, more control
• Hoppala AugmentationHosted solution, easy to use, limited funtionality
• LMS using Layar Connect, such as BuildAR, Poistr, Poiz, VISAR, etc.
Developer Support
• The following support resources provide a wealth of information for developers:
• Layar developer wiki
• Layar developer support environment
• Discussion forums
• Support tickets
Launched in June 2009
Based in Amsterdam
VC funded
55 people
About Layar
3D modeling in Layar
• 3D capability in Layar
• Creating 3d models
• Converting 3D model (Layar3D Model Converter)
• Rendering 3d models
Layar Augmented Office
• Multiple materials with coloring and shininess
• Static/Animated texture
• Texture transparency
• Build 3D model on the fly
• Animation API (apply animation programmatically)
What’s possible now ?
General Flow
3D modeling Software, Blender, Google
Sketchup, 3ds Max, etc
wavefront file format(.obj/.mtl)
Layar3D Model Converter
.L3D file format
save as input output
• Coordinate system:
X → West to East
Y → South to North
Z → Ground to Sky
• Vertex: point in 3D space
• Face/polygons: area connecting 3 or more vertices.
Creating 3D models - Terminology (1)
• Normals:
• Indicate the direction of a face.
• Only front facing faces are rendered.
• have effect on the light shading
• If not present in your model, the vertex order is important. (front face is counter clock wise)
Creating 3D models - Terminology (2)
• Materials:
• Diffuse color: The main color of the material that is used when light is reflected on it.
• Ambient color: The color of the material that is used for environment light.
• Specular color: The color of the highlight in the model (often white)
• Shininess: controls the size of the highlight. High shininess gives a small highlight, low shininess makes the highlight area larger.
Creating 3D models - Terminology (3)
Limitations - Mobile devices
• Limited bandwidth (phone operator) - takes too long to load large files.
• Limited Memory/ processing power (low-end devices)
• The screen is small (usually 800X480), subtle details can not be seen.
• Limited GPU (Graphics processor) power
Limitations - Mobile devices
Keep the balance between complexity and efficiency !
• Limited bandwidth (phone operator) - takes too long to load large files.
• Limited Memory/ processing power (low-end devices)
• The screen is small (usually 800X480), subtle details can not be seen.
• Limited GPU (Graphics processor) power
• Polygon counts: recommended max. 10000 (after triangulation)
• File size: max 1mb
• Only support mesh based objects (polygonal modeling)
• The unit for the vertex coordinates is set to 1 meter.
• Keep the object centered on the grid (recommended)
• No transparency on the normal material colors (no alpha blending)
3D Model Requirements
• Supported Texture formats: PNG, JPEG, GIF (static/animated)
• Resized to be nearest power of 2 (width & height), 16x16, 32x64, etc.
• Use UV mapping
• Texture transparency is supported (PNG only)
• No blending (partial transparency not supported)
• No multiple textures supported for one material, e.g. bump mapping.
Texture Requirements
Texture UV Mapping
• Alpha values <0.1 are not rendered at all, causing the models behind the face to show. (cutout effect)
• Semi transparent color values (0.1 <alpha<1.0) are blended with the camera image, the models behind it are not shown.
Texture Transparency (1)
Texture Transparency (2)
Animated Texture
• Add frames:
• Using animated GIF texture, or,
• Add frames in Layar3D model converter
• Use good compression in image files (PNG, reduce number of colors)
• Use small images (recommended smaller than 256x256)
• Recommend to limit the number of frames to 10
Animated Texture
• Add frames:
• Using animated GIF texture, or,
• Add frames in Layar3D model converter
• Use good compression in image files (PNG, reduce number of colors)
• Use small images (recommended smaller than 256x256)
• Recommend to limit the number of frames to 10
Keep the balance between texture size and file size!
• Start with the basic model and add details later
• Keep the model complexity as low as possible
• use texture to add details, such as windows, doors, etc.
• prepare a simple version of the model ( 2 level of details supported in Layar )
• Keep the model as clean as possible
• no floating vertex points
• no overlapping faces/polygons
• remove invisible faces/polygons
• Check if the normals are facing out.
Guidelines & Best Practices - Model
• Keep the texture images as small as possible
• Use good compression in image files
• Reduce the amount of images used and combine them into a single texture file (UV-mapping).
• Create “see-through” parts using texture transparency.
• Optimize material group and organize faces based on material.
Guidelines & Best Practices - Textures
• Keep the texture images as small as possible
• Use good compression in image files
• Reduce the amount of images used and combine them into a single texture file (UV-mapping).
• Create “see-through” parts using texture transparency.
• Optimize material group and organize faces based on material.
Guidelines & Best Practices - Textures
Make sure your model is ready before export to .obj file!
• Convert Wavefront format to Layar3D (.l3d) format, which is based on Wavefront (.obj) but optimized for mobile phone devices.
• written in Java, requires Java v6.0
• Command line version is available
Layar 3D Model Converter
• Imported files
• .obj - contains the vertex and face data
• .mtl - contains the materials and references to texture files
• .png/.jpg/.gif - texture images
• Open existing .l3d model
Layar 3D Model Converter - Importing
Layar 3D Model Converter -Overview
• Model details
• Model dimensions (in meters)
• File size (bytes)
Layar 3D Model Converter -Materials
• Review materials and textures
• Edit colors and shininess values
• Replace texture
Layar 3D Model Converter - Animated Texture
• Change texture type:
• no texture
• static
• animated
• Add animated frames
Layar 3D Model Converter - Preview
• Imitating the Layar reality browser behavior in terms of 3D rendering
• The same look and feel can be expected in Layar reality Browser (v5.0 and above)
Layar 3D Model Converter - Edit model
• Basic functions for quick fix
• Will take effect after saving the model to .l3d format
• Still recommend to correct these while preparing the model in the modeling software
Layar 3D Model Converter - Placement
• Place the 3D model directly on the map ( bird view )
• Help find the accurate position of the model
• Save placement details in JSON format
Layar 3D Model Converter - Saving
• Save as .l3d model
• .l3d file embeds all textures and materials
• Approach 1:
• Create .obj file based on .obj file format
• Convert to .l3d model using command line version of Layar3D model converter.
• Approach 2:
• Generate .l3d file directly (in binary format, file format is available upon request)
Creating 3d model on the fly
• OpenGL ES API
• Simulated camera view in OpenGL
• position
• view
• up
• Object is rendered with perspective
Rendering 3D models in Layar
• Given: a 3d cube which is 500m away from user’s position and it is 50m in width, 40m in length and 30m in height.
• Question: How is it rendered in Layar ?
Rendering 3D models in Layar
{
"dimension": 3,
"object": {
"baseURL": “http://layar.example.com/”,
“full”: “full.l3d”,
“reduced”: “reduced.l3d”,
“icon”: “icon.png”,
“size”: 50
}
“transform”: {
“angle”: 45,
“rel”: false,
“scale”: 2
}
Define 3d Parameters in JSON Response
• 2 levels of detail for the 3d model : full, reduced
• “size” : determines which model to pick up to render. The length of the edge of the smallest cube in which the object can fit.
• “angle”: rotates the object around the z-axis (right hand rule)
• “rel”: if true, the rotation is calculated relative to the position of the user.
• “scale”: can be applied to alter the model size.
Which model to pick up ? (1)
• The rendered size in Layar is determined by:
• Distance: between the 3d model and the user, e.g. 500m
• “size”: the size parameter in “object”
• Scale factor: the “scale” parameter in “transform”.
{
"dimension": 3,
"object": {
"baseURL": “http://layar.example.com/”,
“full”: “full.l3d”,
“reduced”: “reduced.l3d”,
“icon”: “icon.png”,
“size”: 50
}
“transform”: {
“angle”: 45,
“rel”: false,
“scale”: 2
}
Which model to pick up ? (1)
In Layar, in the distance of 500m, there is a 3d object which is 100m (50m x 2) in width, length and height.
• The rendered size in Layar is determined by:
• Distance: between the 3d model and the user, e.g. 500m
• “size”: the size parameter in “object”
• Scale factor: the “scale” parameter in “transform”.
{
"dimension": 3,
"object": {
"baseURL": “http://layar.example.com/”,
“full”: “full.l3d”,
“reduced”: “reduced.l3d”,
“icon”: “icon.png”,
“size”: 50
}
“transform”: {
“angle”: 45,
“rel”: false,
“scale”: 2
}
• rendered size > 100 px, “full” model is picked up.
• 20 px < rendered size <100 px, “reduced” model is picked up.
• rendered size < 20 px, “icon” image is used and scaled down to 10 px threshold.
Which model to pick up ? (2){
"dimension": 3,
"object": {
"baseURL": “http://layar.example.com/”,
“full”: “full.l3d”,
“reduced”: “reduced.l3d”,
“icon”: “icon.png”,
“size”: 50
}
“transform”: {
“angle”: 45,
“rel”: false,
“scale”: 2
}
Based on the rendered size, layar client will determine which model (full/reduced/icon) should be downloaded and rendered.
• rendered size > 100 px, “full” model is picked up.
• 20 px < rendered size <100 px, “reduced” model is picked up.
• rendered size < 20 px, “icon” image is used and scaled down to 10 px threshold.
Which model to pick up ? (2){
"dimension": 3,
"object": {
"baseURL": “http://layar.example.com/”,
“full”: “full.l3d”,
“reduced”: “reduced.l3d”,
“icon”: “icon.png”,
“size”: 50
}
“transform”: {
“angle”: 45,
“rel”: false,
“scale”: 2
}
In Layar, in the distance of 500m, there is a 3d object which is 100m (50m x 2) in width, 80m in length and 60m height.
• The real size embedded in the 3d model will be used, instead of the “size” parameter.
• The “scale” factor will be applied to the real size of the model
The Actual Rendering{
"dimension": 3,
"object": {
"baseURL": “http://layar.example.com/”,
“full”: “full.l3d”,
“reduced”: “reduced.l3d”,
“icon”: “icon.png”,
“size”: 50
}
“transform”: {
“angle”: 45,
“rel”: false,
“scale”: 2
}
Animation API
• A collection of pre-defined animations on POIs.
• Simple appearance animation (drop, grow, spin)
• Full customizable animation: onClick, onUpdate, onFocus, etc
• Layer level/POI level
Animation Video
Useful links
• Creating 3D models (http://layar.pbworks.com/w/page/7783224/Creating-the-3D-objects)
• Download Layar3D model converter (http://public.layar.com/downloads/Layar3DModelConverter.jnlp)
• Using Layar3D model converter (http://layar.pbworks.com/w/page/32586555/3D-Model-Converter)
• Animation API (http://layar.pbworks.com/w/page/35910564/animation-for-API-5)
• Need Help ? Devsupport environment (http://devsupport.layar.com)
Discussion
Thank [email protected]