building usable ar interfaces
DESCRIPTION
Mark Billinghurst's talk on Building Usable AR Interfaces at the ARE 2012 conference, May 10th 2012TRANSCRIPT
![Page 1: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/1.jpg)
Building Usable AR Interfaces
Mark Billinghurst HIT Lab NZ
University of Canterbury
![Page 2: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/2.jpg)
The Vision of AR
![Page 3: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/3.jpg)
To Make the Vision Real.. Hardware/software requirements
Contact lens displays Free space hand/body tracking Speech/gesture recognition Etc..
Most importantly Usability
![Page 4: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/4.jpg)
Usability Issues Perception
How to make images appear part of real world?
Interaction How to pick content in mid air?
Social Do I look stupid?
Cultural Wave up or down?
Cognitive How do I remember where everything is?
![Page 5: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/5.jpg)
“The product is no longer the basis of value. The
experience is.”
Venkat Ramaswamy The Future of Competition.
![Page 6: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/6.jpg)
experiences
services
products/tools
components
Valu
e Gilmore + Pine: Experience Economy
Function
Emotion
![Page 7: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/7.jpg)
Build Experiences NOT Applications!
![Page 8: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/8.jpg)
experiences
applications
tools
components
Sony CSL © 2004
Building Compelling AR Experiences
Tracking, Display
Authoring
Interaction
Usability ??
![Page 9: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/9.jpg)
The Interaction Design Process
![Page 10: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/10.jpg)
Understand Your Users Workshops or focus groups
Group interviews/activities
Observations Spending time with users in day to day tasks
Questionnaires Looking for specific information
Interviews Good for exploring issues, using props
Documentation Procedures and rules written down in manuals
![Page 11: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/11.jpg)
Consider the Whole User
![Page 12: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/12.jpg)
AR as Perception Problem Goal of AR to fool human senses – create
illusion that real and virtual are merged Depth
Size Occlusion Shadows Relative motion Etc..
![Page 13: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/13.jpg)
![Page 14: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/14.jpg)
TAT Augmented ID
![Page 15: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/15.jpg)
![Page 16: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/16.jpg)
![Page 17: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/17.jpg)
AR Rapid Development
Prototyping and User Testing Low Fidelity Prototyping Sketches Paper Prototyping Post-It Prototyping PowerPoint Prototyping
High Fidelity Prototyping Wikitude, Junaio, Layar, BuildAR etc
![Page 18: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/18.jpg)
POST IT PROTOTYPING
First Dra)
Camera View with 3D
Second Dra) Third Dra)
• Selec8on highlighted in blue
• Home bu=on added for easy naviga8on to main menu
![Page 19: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/19.jpg)
POWERPOINT PROTOTYPING Benefits • Used for User Tes8ng • Interac8ve • Func8onali8es work when following the story of Scenario 1 • Quick • Easy arrangement of slides
User Tes8ng • Par8cipants found • 15 minute sessions screen captured
• ‘Talk Allowed’ technique used • Notes taken • Post-‐Interview
![Page 20: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/20.jpg)
BuildAR
http://www.buildar.org/ Stand alone application Visual interface for AR model viewing application Enables non-programmers to build AR scenes
![Page 21: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/21.jpg)
Interface Components Physical components Display elements
- Visual/audio
Interaction metaphor
Physical Elements
Display Elements
Interaction Metaphor Input Output
![Page 22: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/22.jpg)
AR Design Space
Reality Virtual Reality
Augmented Reality
Physical Design Virtual Design
![Page 23: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/23.jpg)
AR Lens Design Principles Physical Components
Lens handle - Virtual lens attached to real object
Display Elements Lens view
- Reveal layers in dataset
Interaction Metaphor Physically holding lens
![Page 24: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/24.jpg)
AR Chemistry (Fjeld 2002)
Tangible AR chemistry education
![Page 25: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/25.jpg)
Case Study: LevelHead
Block based game
![Page 26: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/26.jpg)
Case Study: LevelHead
Physical Components Real blocks
Display Elements Virtual person and rooms
Interaction Metaphor Blocks are rooms
![Page 27: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/27.jpg)
![Page 28: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/28.jpg)
Goal: An AR application to test molecular structure in chemistry
Physical Components Real book, rotation cube, scoop, tracking markers
Display Elements AR atoms and molecules
Interaction Metaphor Build your own molecule
![Page 29: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/29.jpg)
AR Chemistry Input Devices
![Page 30: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/30.jpg)
![Page 31: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/31.jpg)
Natural Hand Interaction
Using bare hands to interact with AR content MS Kinect depth sensing Real time hand tracking Physics based simulation model
![Page 32: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/32.jpg)
Evaluation Need for more evaluation
2008 -10% AR papers in IEEE, ACM had any evaluation
Informal Pilot, ‘quick and dirty’
Formal Lab studies, field studies, heuristic
![Page 33: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/33.jpg)
2D vs. AR Navigation?
VS
![Page 34: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/34.jpg)
HIT Lab NZ Test Platform – AR View
![Page 35: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/35.jpg)
HIT Lab NZ Platform – Map View
![Page 36: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/36.jpg)
Distance and Time
No significant differences
![Page 37: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/37.jpg)
Paths Travelled
Red – AR Blue – AR + Map Yellow - Map
![Page 38: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/38.jpg)
Navigation Behaviour Depends on interface
Map doesn’t show short cuts
![Page 39: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/39.jpg)
Survey Responses
![Page 40: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/40.jpg)
User Comments AR
“you don't know exactly where you are all of the time.” “using AR I found it difficult to see where I was going”
Map “you were able to get a sense of where you were” “you are actually able to see the physical objects around you”
AR+MAP “I used the map at the beginning to understand where the
buildings were and the AR between each point” “You can choose a direction with AR and find the shortest way
using the map.”
![Page 41: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/41.jpg)
Building Usable AR Interfaces Understand user needs
Consider whole user needs - Physical, emotional, cognitive, social, cultural
Perceptual issues
Design for those needs Rapid prototyping Virtual, physical elements, interaction metaphor
Test your design Formal, informal testing
![Page 42: Building Usable AR Interfaces](https://reader033.vdocuments.us/reader033/viewer/2022060110/555ae61dd8b42abe058b45c9/html5/thumbnails/42.jpg)
More Information
• Mark Billinghurst – [email protected]
• Website – www.hitlabnz.org – www.buildar.org