cosc 426 lect. 5 - mobile ar
DESCRIPTION
Lecture 5 of the COSC 426 graduate course on Augmented Reality. This lecture provides an overview of Mobile Augmented Reality. The Lecture is given by Mark Billinghurst of the HIT Lab NZ at the University of CanterburyTRANSCRIPT
![Page 1: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/1.jpg)
Lecture 5: Mobile ARLecture 5: Mobile AR
Mark Billinghurstk billi h t@hitl [email protected]
HIT Lab NZ University of Canterbury
![Page 2: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/2.jpg)
AR on mobile phonesLow cost, widely spread platform
Billions of phones deployedPeople know how to use themStrong demand from commercial sideHuge chance for AR!
Target practical applicationsEasy to useQuality graphicsRobust tracking15-30 Hz overall frame rate
![Page 3: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/3.jpg)
Why would you use phones?Robust and fool-proof
People know how to use their mobile phones…
Variety of supported devicesSelf-contained operationSelf contained operation
Enough processing power to do everything natively
Ultra mobileUltra-mobileLow cost devices
!Even better: people already own the target hardware!
A very unique chance for bringing AR to the masses
![Page 4: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/4.jpg)
Why would you not use phones?Compared to PC-based setups
Less processing power and memoryp g p yHarder to program, debug and deploy
Hardware difficult or impossible to extendHardware difficult or impossible to extendSmall number of available librariesT i ll l li l i i h Typically only little experience in research groupsSo many different devices and operating systems
![Page 5: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/5.jpg)
Other limitations of handheld ARUsual limitations in mobile HCI
Small screen- Less information possible to display- Less immersion
Limited inputLimited input…
Other limitationsOther limitationsYou see through the camera and not through the phoneSwitch attention between background and phoneSwitch attention between background and phoneStrain factor of holding the phone upSocial issues with pointing the phone at peoplep g p p p
![Page 6: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/6.jpg)
Mobile AR HistoryMobile AR History
![Page 7: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/7.jpg)
E l f M b l AREvolution of Mobile AR
Camera phoneCamera phone
Thin client ARWearable AR Camera phone
- Self contained AR
WearableComputers
- Thin client AR
Handheld AR Displays
Self contained ARPDAs-Thin client AR
p yPDAs-Self contained AR
1995 1997 2001 2003 20041995 1997 2001 2003 2004
![Page 8: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/8.jpg)
Handheld DisplaysTethered Applications
Fitzmaurice Chameleon (1994)( )Rekimoto’s Transvision (1995)Tethered LCDPC Processing and Tracking
![Page 9: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/9.jpg)
Handheld AR Display - Tethered
1995, 1996 Handheld ARARPad CameleonARPad, CameleonRekimoto’s NaviCam, TransvisionTethered LCDTethered LCDPC Processing and Tracking
![Page 10: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/10.jpg)
Mobile AR: Touring Machine (1997)Columbia University
Feiner, MacIntyre, Höllerer, Webster
Combines See through head mounted displayGPS t kiGPS trackingOrientation sensorBackpack PC (custom)p ( )Tablet input
![Page 11: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/11.jpg)
MARS View
Vi t l t l id th l ldVirtual tags overlaid on the real world“Information in place”
![Page 12: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/12.jpg)
Backpack/Wearable AR
1997 Backpack ARFeiner’s Touring MachineAR Quake (Thomas)Tinmith (Piekarski)MCAR (Reitmayr)Bulky, HMD based
![Page 13: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/13.jpg)
Mobile AR - Hardware
GPSAntenna
RTK correction AntennaRTK correction AntennaRTK correction AntennaRTK correction Antenna Example self-built workingsolution with PCI-based 3D graphics
PCI 3D Graphics BoardHMDHMD
ControllerControllerHMDHMD
ControllerController
solution with PCI based 3D graphics
PC104 Sound Card
TrackerController
DC to DCCPU
PC104 PCMCIA
DC to DCConverter
Battery
WearableComputer
Hard Drive
Serial
GPS RTK correction
Radio
Ports
Columbia Touring Machine
![Page 14: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/14.jpg)
1997 Philip Kahn invents camera phoneSharp J-SH04
1997 Philip Kahn invents camera phone1999 First commercial camera phone
![Page 15: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/15.jpg)
Millions of Camera Phones
1000
1200
800
1000
DSC
400
600DSCPhone
0
200
2002 2003 2004 2005 2006 2007 2008 2009 2010
![Page 16: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/16.jpg)
Handheld AR – Thin Client
2001 BatPortal (AT&T Cambridge)PDA used as I/O deviceWi l i k i Wireless connection to workstation Room-scale ultrasonic tracking (Bat)
2001 AR-PDA (C Lab)PDA thin graphics clientRemote image processingwww.ar-pda.com
![Page 17: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/17.jpg)
Mobile Phone AR – Thin Client
2003 ARphone (Univ. of Sydney)Transfer images via Bluetooth (slow – 30 sec/image)g ( g )Remote processing – AR Server
![Page 18: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/18.jpg)
Early Phone Computer Vision Apps2003 – Mozzies Game - Best mobile gameOptical motion flow detecting phone orientationSi SX1 S bi 120Mh VGA CSiemens SX1 – Symbian, 120Mhz, VGA Camera
2005 – Marble Revolution (Bit-Side GmbH)Wi f N ki ' S i 60 Ch ll 2005 Winner of Nokia's Series 60 Challenge 2005
2005 – SymBall (VTT)
![Page 19: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/19.jpg)
Computer Vision on Mobile PhoneCameras and Phone CPU sufficient for computer vision applications Pattern Recognition (Static Processing)g ( g)
QR CodeShotcode (http://www.shotcode.com/)
M i Fl (2D I P i )Motion Flow (2D Image Processing)GestureTek
- http://www.gesturetekmobile.com/
TinyMotion
3D Pose CalculationAugmented RealityAugmented Reality
![Page 20: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/20.jpg)
Handheld AR – Self Contained2003 PDA-based AR
ARToolKit port to PDAStudierstube ported to PDAStudierstube ported to PDAAR Kanji Educational App.Mr Virtuoso AR characterWagner’s Invisible Train
- Collaborative AR
![Page 21: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/21.jpg)
Mobile Phone AR – Self Contained2004 M bil Ph AR2004 Mobile Phone AR
Moehring, BimberHenrysson (ARToolKit)Henrysson (ARToolKit)Camera, processor, display together
![Page 22: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/22.jpg)
Location Aware Phones
Nokia NavigatorMotorola Droid
![Page 23: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/23.jpg)
Real World Information OverlayTag real world locations Tag real world locations
GPS + Compass inputOverlay graphics data on live videoOverlay graphics data on live video
Applications Travel guide, Advertising, etc
Eg: Mobilizy Wikitude (www.mobilizy.com)Android based, Public API released
Other companiesLayar, AcrossAir, Tochnidot, RobotVision, etc
![Page 24: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/24.jpg)
Layar – www.layar.com
![Page 25: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/25.jpg)
HIT Lab NZ Android AR PlatformArchitectural ApplicationLoads 3D models
a OBJ/MTL format
Positions content in spacepGPS, compass
Intuitive user interfaceIntuitive user interfacetoolkit to modify the model
Connects to back end model databaseConnects to back end model database
![Page 26: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/26.jpg)
HIT Lab NZ Mobile Outdoor AR
![Page 27: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/27.jpg)
![Page 28: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/28.jpg)
History of Handheld and Mobile AR1995 Handheld Display: NaviCam, AR-PAD, Transvision
1997 Wearable AR: Touring Machine, AR Quakeg
2001 Handheld AR – Thin Client: AR-PDA, Bat Portal
2003 Handheld AR Self contained: Invisible Train2003 Handheld AR – Self contained: Invisible Train
2003 Mobile Phone – 2D Vision: Mozzies, Symball
2003 Mobile Phone – Thin Client: ARphone
2004 Mobile Phone – Self contained: Moehring, Symbian
![Page 29: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/29.jpg)
Mobile AR by Weight1996
2003 2007
Scale it down:Vesp‘R [Kruijff ISMAR07]:
Scale it down more:S t h $500
Backpack+HMD:5 8k
Vesp R [Kruijff ISMAR07]:…Sony UMPC 1.1GHz…1.5kg
till >$5K
Smartphone…$500…All-in-one…0.1kg
…5-8kg …still >$5K …billions of units
![Page 30: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/30.jpg)
2011 S f h A2011 State of the ArtHandheld Hardware available
PDA, mobile phones, external camerasSensors: GPS, accelerometer, compass
Software Tools are AvailableTracking: ARToolKitPlus, QCARGraphics: OpenGL ESGraphics: OpenGL ESAuthoring: Studierstube, Layar, Wikitude, Unifye
What is needed:High level authoring toolsContent development toolsNovel interaction techniquesUser evaluation and usability
![Page 31: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/31.jpg)
Mobile AR CompaniesMobile AR
GPS + compass
M C iMany CompaniesLayarWikitudeWikitudeAcrossair PressLiteYelpRobot visionEEtc..
![Page 32: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/32.jpg)
$2 million USD in 2010$732 million USD in 2014
![Page 33: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/33.jpg)
Handset ManufacturersQualcommQualcomm
$100 million USD investment
N kiNokia25+ people in NRC
SamsungExploring the space
Apple586 AR Applications on App Store
GoogleGoogle goggles/Android AR Applicationsg g gg pp
![Page 34: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/34.jpg)
Qualcomm
Acquired Imaginationq gOctober 2010 - Releases free Android AR SDKComputer vision tracking - marker, markerless p gIntegrated with Unity 3D rendererhttp://developer.qualcomm.com/arp p q
![Page 35: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/35.jpg)
Rock-em Sock-em
Sh d AR DShared AR DemoMarkerless tracking
![Page 36: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/36.jpg)
Apple
iPhone 4 SDK supports direct camera accessppLaunches AR theme on App Store
![Page 37: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/37.jpg)
> 500 AR apps on App Store
![Page 38: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/38.jpg)
Developing AR applications
![Page 39: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/39.jpg)
Mobile AR TechnologyInvolves
Tracking gContent Loading
Rendering/3D graphicsg g pUser Interface Application Design Application Design Evaluation
![Page 40: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/40.jpg)
Scientific challengesAR requires (unlike related disciplines)AR requires (unlike related disciplines)
Strict real time operation (30Hz)- Unlike Ubicomp or mobile information systemsp y
High spatial precision (1cm, 1 degree)- Unlike location-based services
Robustness for operation by human user- Unlike many computer vision methods in automation etc.
M bil h AR i (i ddi i )Mobile phone AR requires (in addition)No thin client!S l l f f d k ARSame level of performance as desktop AR- New algorithms must be orders of magnitude more efficient
No unrealistic assumptions about HWNo unrealistic assumptions about HW
![Page 41: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/41.jpg)
How does a basic AR application work?Main loopMain loop
Get a video frame from the cameraEstimate the position and the orientation
- computer vision, sensor input (GPS, compass)p p ( p )
Render the augmented scene (video + virtual content))Render GUIProcess user inputProcess user inputUpdate application status
![Page 42: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/42.jpg)
Studierstube ES Framework
Typical AR application framework
Applications
Developed at TU Graz
StudierstGraz tube Software Staack
Platform
![Page 43: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/43.jpg)
End User Application
Programming Lib i
pp
Libraries
OS/Low Level API
Hardware
![Page 44: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/44.jpg)
Th S d b ES f kThe Studierstube ES frameworkA
pplicatioonsStudieContent
User Interface - Application
erstube Software S
Graphics
Content
StackTracking
Platform
Platform
![Page 45: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/45.jpg)
Platform
![Page 46: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/46.jpg)
What are the challenges?Experiences with embedded platforms requiredM l f ( i )Many platforms (operating systems)Slow CPUs (low clock rates, often no FPU)
Difficulties with trackingDifficulties with visualizations that require a lot of data
iprocessing
Slow memory accessNo or weak hardware 3D acceleration
Difficulties with graphics
![Page 47: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/47.jpg)
Processing power of mobile phones
Weak processing power~1GHz Single core~1GHz, Single coreOften no floating point unit- Floating point code ~40x slower than integer codeoat g po t co e 0 s owe t a tege co e- Fixed point problematic for many algorithms- Requires good math library
Code optimized for phones runs 5-10x slower on ahigh-end phone than on an average PCNot going to change dramatically due to battery power
![Page 48: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/48.jpg)
So what are the common problems?Bad camera quality under low lighting
Noise, motion blur,Strongly varies with different phones
Small memorySmall memoryKeeping large databases in memory is problematic
Slow memorySlow memoryLow clock rate- Processing large memory areas is prohibitiveg g y p- Typical CV building blocks (SVD, image processing) are too demanding
Slow data transfers between CPU and GPU
![Page 49: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/49.jpg)
Pl f f h dh ld ARPlatforms for handheld ARPros Cons
Windows Mobile
Easy to program and debugLots of devices
Camera drivers are not always good
SymbianLargest installed basisGood devices
Hard to program and debug, SDK changes often, usually slow CPUs
Very nice hardware Camera API only with OS4iPhone
Very nice hardwareHype factor
Camera API only with OS4Objective C as main language
AndroidIncreasing number of devices
Java as main languageAndroidHype factor
Java as main language
Linux (LiMo, M )
Full Linux support, limitedh d f AR
Large set of different librariesMaemo) handsets for AR
Large set of different libraries
RIM Blackberry
Widely spread in the US Java onlyBlackberry
Palm WebOS
Nice hardwareOnly very few devices so farNo native SDK so far
![Page 50: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/50.jpg)
Worldwide smartphone market share
1Q101Q09
Symbian
RIMRIM
iPhoneiPhone
Android
Source: Gartnerhttp://www.gartner.com
![Page 51: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/51.jpg)
2011 US Market Share
![Page 52: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/52.jpg)
What makes a device interesting for AR?
Open and easy to programGood cameraGood cameraFast CPU, FPU is a plusGood H/W 3D supportLarge installed basisg
Easy access to devices
GPS, accelerometer, compassGPS, accelerometer, compassEnough memory/storage
![Page 53: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/53.jpg)
Typical Smart Phone HardwareCPUCPU
300-800+ Mhz
GPUGPUNone, or Power VR Chip (OpenGL ES1.0/2.0)
InputpTouch screen, keyboard, keypad
SensorsGPU l (1 3 5 b )GPU, compass, accelerometer, camera (1.3-5mb+)
NetworkingBluetooth Wifi 3GBluetooth, Wifi, 3G
Screen320x240 up to 800x480
![Page 54: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/54.jpg)
HTC HD2HTC HD2Windows MobileWindows MobileFast CPU (1GHz)Big screen
4.3”, 800x480
GPS, compass and accelerometerGood cameraGood camera
Depends on lighting conditions
Hardware 3DHardware 3DSlow texture upload: slow video background renderingslow video-background rendering
![Page 55: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/55.jpg)
HTC DHTC DesireAndroidAndroidFast CPU (1GHz)Smaller screen
3.7”, 800x480
Multi-touchHardware 3DHardware 3DGPS, compass and accelerometeraccelerometer
![Page 56: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/56.jpg)
Ph 4iPhone 4Apple iOS 4 0Apple iOS 4.0Faster CPU (1.2GHz)Hi h l iHigh screen resolution
3.5”, 960x640
(Finally) camera APIMulti-touchHardware 3DGPS compass accelerometerGPS, compass, accelerometerand gyroscope
![Page 57: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/57.jpg)
Hardware SensorsC ( l i f )Camera (resolution, fps)
Maker based/markerless trackingVid lVideo overlap
GPS (resolution, update rate)Outdoor locationOutdoor location
Compass Indoor/outdoor orientationIndoor/outdoor orientation
AccelerometerMotion sensing relative tiltMotion sensing, relative tilt
![Page 58: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/58.jpg)
Programming Environments
![Page 59: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/59.jpg)
Mobile Development EnvironmentsNot like developing for desktops
Wide range of different OSLimited CPU, low memory, poor graphics, no floating point
Popular Mobile OSS bi (S bi C++ C bid IDE)Symbian (Symbian C++, Carbide IDE)iPhone (Objective C)Android (Java, Native NDK wrapper) (J pp )Windows Mobile (C#, C++, Visual Studio)
OtherPalm OS, Blackberry, Linux
![Page 60: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/60.jpg)
Programming Windows MobileVery similar to desktop Windows
Almost identical APIU d f lUnicode functions only
Development toolsEmbedded Visual C++ C#Embedded Visual C++, C#
- Deprecated, not suggested
Visual Studio 2005Visual Studio 2008
- Required for FPU usage
F i f b l k tFor overview of camera bugs look athttp://studierstube.org/handheld_ar/camera_phones.php
![Page 61: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/61.jpg)
Programming SymbianDevelopment tools
Carbide.c++Commercial version required for on-device debugging (important since emulator is bad…)SDK f dSDK appropriate for your device
Many quirksC i l d C++ tCrippled C++ supportWriting to static variables not allowed/recommendedCleanup StackpAPI includes ~1500 classes
Moving to Qtg
![Page 62: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/62.jpg)
Programming iPhoneHarsh restrictions from Apple
Apps have to go through the apps store
X d IDE f d lXcode IDE for developmentNice development tools
Objective CObjective-CRequired for application developmentCan call into C/C++ codeCan call into C/C code
Camera API support in iOS 4+Can overlay on live videoy
![Page 63: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/63.jpg)
AndroidHardware creatorsHardware creators
HTC, LG, Samsung, Motorola
Wid l il bl hWidely available phoneDifferent form factors – tablets, phones, PC, etc
Multiple versions and fragmentationAndroid 1.0, 1.6Android 2.0, 2.1
Free Tools Eclipse DevelopmentApp Integrator
![Page 64: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/64.jpg)
Mobile Graphics
![Page 65: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/65.jpg)
Computer Graphics on Mobile PhonesSmall screen, limited input optionsLimited support for accelerated graphics
Most phones have no GPU Most phones have no GPU
Mobile Graphics LibrariesOpenGL ES (1.0, 2.0)p ( , )
- Cross platform, subset of OpenGL- C/C++ low level library
Java M3GJava M3G- Mobile 3D graphics API for J2ME platform - Object importer, scene graph library- Support from all major phone manufacturerspp j p
![Page 66: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/66.jpg)
![Page 67: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/67.jpg)
O GL ESOpenGL ESSmall footprint subset of OpenGLSmall-footprint subset of OpenGL
OpenGL is too large for embedded devices!
P f l l l l API f ll f i li f 3D Powerful, low-level API, full functionality for 3D gamesCan do almost everything OpenGL can (but only one way)A l bl ll k l fAvailable on all key platformsSoftware and hardware implementations available
F ll iblFully extensibleExtensions like in OpenGL
No redundancy!Convenience functions removed
![Page 68: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/68.jpg)
OpenGL ESSLIDE 68
OpenGL ES vs. OpenGL (1.x)OpenGL OpenGL ES
glBegin/glEnd 11
Primitive Types all no quads & polygons
Data Types float double int etc float fixedData Types float, double, int, etc… float, fixed
glDraw/Read Pixels glReadPixels only
T tTextures 1D, 2D, 3D, cube 2D
Stencil optional
Window Bindings WGL, GLX, etc… EGL
1: Except for Security Critical profile
![Page 69: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/69.jpg)
OpenGL ESSLIDE 69
OpenGL ES on mobile devices
Java Applications C++ Applications
S h API G MiddlScenegraph APIM3G (JSR 184)
GameEngine
Middleware Engines
![Page 70: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/70.jpg)
OpenGL ES Performance
( )Faster graphics (esp. hardware accelerated)Longer batter performance (> 10%)
![Page 71: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/71.jpg)
VVersionsTwo major tracksTwo major tracks
Not compatible, parallel rather than competitive
OpenGL ES 1 xOpenGL ES 1.xFixed function pipelineSuitable for software implementationsSuitable for software implementationsAll 1.x are backwards compatible
OpenGL ES 2 xOpenGL ES 2.xVertex and pixel shaders using GLSL ESAll 2 x will be backwards compatibleAll 2.x will be backwards compatible
![Page 72: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/72.jpg)
F d F (1 )Fixed Function (1.x)
http://www.khronos.org/opengles/2_X/
![Page 73: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/73.jpg)
P bl (2 )Programmable (2.x)
http://www.khronos.org/opengles/2_X/
![Page 74: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/74.jpg)
OpenGL ES 1.x vs 2.0
![Page 75: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/75.jpg)
Tracking
![Page 76: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/76.jpg)
Mobile Augmented Reality’s goal
Create an affordable, massively multi-user, widespread platform
© Tinmith, U. of South Australia
![Page 77: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/77.jpg)
Tracking is…Estimating the device‘s pose (position and orientation) Strictly in real time (30Hz)Strictly in real time (30Hz)
With high spatial precision (1cm, 1 degree)Robustly for operation by human userRobustly for operation by human userNo unrealistic assumptions about HWLeaving enough power to other tasks (interaction graphics)Leaving enough power to other tasks (interaction, graphics)
![Page 78: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/78.jpg)
Tracking requirements for AR on phonesFast and efficientFast and efficientForm factor: light and robustTrack simultaneously
A large number of objectsBy a large number of users
Requires little or no …qDevice modificationManual calibration (targeting non-technical users)( g g )Instrumentation of the physical environment
Low costsLow costs
![Page 79: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/79.jpg)
Tracking on mobile phonesVision-based tracking
Marker-based trackingModel-based natural feature trackingNatural feature tracking in unknown environments
Sensor trackingGPS, inertial compass, gyroscope
![Page 80: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/80.jpg)
Tracking for Handheld ARSLIDE 80
Backpack-based 1.
Höllerer et al. (1999), Piekarski & Thomas al. (2001), Reitmayr & Schmalstieg (2003)( ), ( ), y g ( )
Laptop, HMDEnhanced GPS (DGPS / RTK) + inertial sensor for viewpoint trackingEnhanced GPS (DGPS / RTK) + inertial sensor for viewpoint trackingHand tracking w/ fiducial markers
![Page 81: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/81.jpg)
Tracking for Handheld ARSLIDE 81
Backpack-based 2.
Kalkusch et al., 2002Video see-through HMD w/ cameragViewpoint Tracking w/ inside-out computer vision using markersARToolKit markers on walls installed and surveyed manuallyy y
![Page 82: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/82.jpg)
Tablet PC / UMPC-based 1.
Schall et al., 2006Hybrid tracking on UMPC
Camera fiducial marker trackinggWhen no marker in view inertial sensor + UWB tracking
![Page 83: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/83.jpg)
Tablet PC / UMPC-based 2.CAMERA
LEDs
Klein & Drummond, 2004Combining outside in (LED tracking for low accuracy robust pose) & Combining outside-in (LED tracking for low accuracy, robust pose) & inside-out (edge-based tracking for high accuracy) computer vision
![Page 84: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/84.jpg)
PDA-based 1.
BatPortal (Newman et al., 2001) SHEEP (MacWilliams et al., 2003)( , )
PDA as thin client (rendering & tracking on server + VNC)
Tracking by ART (external IR cameras + retroreflective target)
Ultrasonic tracking Projection-based AR environ.
![Page 85: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/85.jpg)
PDA-based 2.
Signpost on PDA (Wagner & Schmalstieg, 2003)
First “truly” handheld AR platform: PDA + cameraStandalone self-contained AR systemStandalone, self contained AR systemOptimized fiducial marker tracking library
![Page 86: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/86.jpg)
History of non-AR Tracking on Phones (1)
AR-PDA (Gausemeier et al., 2003)
Model based tracking
Kick Real (Paelke et al., 2004)
Edge detection of real foot + collisionModel-based trackingPDA = thin client
tracking off-loaded to serverNot real-time
Edge detection of real foot + collision detection w/ virtual ball2D tracking and limited interaction(tailored to game)
![Page 87: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/87.jpg)
History of non-AR Tracking on Phones (2)
PhoneGuide (Bruns et al., 2005) LightSense (Olwal, 2006)
Neural network for recognizing visual features of museum artifactsCombined w/ BT “tracker”O
External camera tracks cell phone LEDSingle user, only coarse position tracking, no orientationBorder case of AROnly object recognition Border-case of AR
![Page 88: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/88.jpg)
History of non-AR Tracking on Phones (3)
Mosquito Hunt (Siemens, 2003)Marble Revolution (BitSide, 2004)Pi i (VTT 2006)Pingis (VTT, 2006)Game control w/ optical flow techniques
TinyMotion (Wang et al., 2006)GUI control & input on cell phones
w/ image differencing & block correlationflow techniques w/ image differencing & block correlation
![Page 89: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/89.jpg)
ARToolKit Tracking (Kato)
ARToolKit - Computer vision based marker tracking libraries
http://artoolkit.sourceforge.net/
![Page 90: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/90.jpg)
History of AR Tracking on Phones (1)
2003ARToolKit on PDAARToolKit on PDAWagner et at.
200420043D Marker on PhoneMöhring et al.g
2005ARToolKit on SymbianHenrysson et al.
![Page 91: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/91.jpg)
Tracking for Handheld ARSLIDE 91
Fiducial marker tracking on handhelds
Möhring et al., 2004 Henrysson et al., 2006Wagner et al., 2003
Rohs, 2006Bucolo et al., 2005
![Page 92: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/92.jpg)
History of AR Tracking on Phones (2)
20052005Visual CodesRohs et atRohs et at.
2008Advanced Marker TrackingAdvanced Marker TrackingWagner et al.
2008Natural Feature TrackingWagner et al.
![Page 93: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/93.jpg)
What can we do on today‘s mobile phones?
Typical specs600+ MHz~5MB of available RAM160x120 - 320x240 at 15-30 Hz camera
Possible to doMarker tracking in 5-15msNatural feature tracking in 20-50ms
![Page 94: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/94.jpg)
Handheld AR Interfaces
![Page 95: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/95.jpg)
Handheld HCI
Consider your userFollow good HCI principlesAdapt HCI guidelines for handheldsAdapt HCI guidelines for handheldsDesign to device constraintsRapid prototyping
User evaluation
![Page 96: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/96.jpg)
Consider Your User■ They are probably mobile■ They are probably mobile
able to use the interface with one hand
■ They want quick access to application content■ They want quick access to application content.Want enhanced interaction with the real world
Interaction with the real world is the main focusInteraction with the real world is the main focus
■ They expect to be able to multitaskstart phone call, use another application, etcp pp
![Page 97: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/97.jpg)
Norman’s Principles of Good PracticeEnsure a high degree of visibility
- allow the user to work out the current state of the system and the f ti ibl range of actions possible.
Provide feedback- continuous, clear information about the results of actions.continuous, clear information about the results of actions.
Present a good conceptual model- allow the user to build up a picture of the way the system holds
h h l i hi b i diff d h together, the relationships between its different parts and how to move from one state to the next.
Offer good mappingsg pp g- aim for clear, natural relationships between actions the user performs
and the results they achieve.
![Page 98: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/98.jpg)
Hi h L l D i G id liHigh Level Design GuidelinesFrom Shneiderman’s 8 desktop design guidelines:p g g
Enable Frequent Users to Use ShortcutsOffer Informative FeedbackDesign Dialogs to Yield Closure
G d T i h’ id liGong and Tarasewich’s guidelines:Design for Small Devices Design for Limited and Split AttentionDesign for speed and recovery Design for speed and recovery Allow for personalizationDesign for EnjoymentDesign for Enjoyment
![Page 99: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/99.jpg)
UI Device Constraints
Comparing Desktop to Handheld Interfaces
Screen Size Input Operation Multimedia Connectivity
Desktop > 1024 x 768 MouseKeyboard
Two handedStationary
Millions of coloursGraphics accel.5.1 Audio
WiredAlways On
Handheld < 640 x 480 StylusTouchButtons
One handedMobile
65K coloursNo graphics accel.Stereo audio
WirelessMaybe On
![Page 100: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/100.jpg)
E m l O2 A ti MExample: O2 Active Menu
Highly visualUse PDA buttons for inputLarge icons and easy to read textVisually indicate which tabs are scrollableApplication UI looks different from device UIApplication UI looks different from device UI
![Page 101: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/101.jpg)
iPhone Guidelines
Minimize required user input. Avoid unnecessary interactivity.y yProvide feedback when necessaryProvide fingertip sized target areasProvide fingertip-sized target areas.Avoid clutter and busy backgrounds. Express essential information succinctly.Make it obvious how to use your content.y
![Page 102: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/102.jpg)
iPhone Interface
![Page 103: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/103.jpg)
Designing for Device ConstraintsInput Device
Touch, stylus, keyboard, buttons, keypady y yp
ScreenSize, resolution,
SensorsCamera – frame rate image resolutionCamera frame rate, image resolutionGPS – resolution, coverageCompass - accuracyCo pass accu acy
![Page 104: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/104.jpg)
Sample Handheld AR InterfacesCleanLarge Video ViewLarge Icons Large Icons Text Overlay
![Page 105: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/105.jpg)
Twitter 360
www.twitter-360.comiPhone applicationppSee geo-located tweets in real worldTwitter com supports geo taggingTwitter.com supports geo tagging
![Page 106: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/106.jpg)
Wikitude – www.mobilizy.com
BlahBlahBl h
BlahBlah
Bl h
BlahBlahBlah
BlahBlah
Bl h
Blah
Blah Blah
Blah
BlahBlahBlah
Blah
BlahBlahBlah
BlahBlah
BlahBlah
![Page 107: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/107.jpg)
Information Filtering
![Page 108: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/108.jpg)
Information Filtering Information Filtering (Julier et al ’00)Information Filtering (Julier et al. ’00)
• Remove clutter by goal- and distance based filtering • User’s task is route finding: Sniper and relevant buildings are displayed; bj t hi h d t i d t b dobjects, which are determined to be unnecessary, removed
![Page 109: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/109.jpg)
W bl AR
HMD vs Handheld AR InterfaceHandHeld ARWearable AR
Output:Output:Display Input &
Output
Input
![Page 110: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/110.jpg)
Handheld Interface MetaphorsTangible AR Lens Viewing
Look through screen into AR scene Interact with screen to interact with AR Interact with screen to interact with AR content
- Eg Invisible Train
Tangible AR Lens ManipulationSelect AR object and attach to device Use the motion of the device as input
- Eg AR Lego
![Page 111: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/111.jpg)
![Page 112: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/112.jpg)
Handheld Display vs Fixed Display
Experiment comparing handheld moving, to handheld button input, small fixed display, desktop display, large plasmaUsers performed (1) navigation task, (2) selection taskUsers performed (1) navigation task, (2) selection taskMoving handheld display provided greater perceived FOV, higher degree of presence, faster completion time
J. Hwang, J. Jung, G. Kim. Hand-held Virtual Reality: A Feasibility Study. In proceedings of VRST 2006
![Page 113: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/113.jpg)
Search Task Completion Time
![Page 114: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/114.jpg)
FOV, Presence and ImmersionPerceived FOV and Actual FOV (deg. marked by subjects)
5852
64 606070
33 31
52
30 30 30
45
2030405060
Perceived FOVActual FOV
01020
Motionbased hh
Buttonbased hh
Smallscreen
17' screen 42' screen5.45.76
7
based hh based hh screen4.3 4.1 4.3
4.74.5 4.34.7 4.9
3
4
5
PresenceImmersion
0
1
2
0Motion
based hhButton
based hhSmall
screen17' screen 42' screen
![Page 115: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/115.jpg)
Rapid Prototyping
Speed development time by using quick hardware mockupsp p y g q phandheld device connected to PCLCD screenUSB phone keypadCamera
Can use PC development toolsCan use PC development toolsFlash, Visual Basic, etc
![Page 116: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/116.jpg)
Mobile Physical PrototypingBug Labshttp://www.buglabs.net/
Open source hardware modules, each producing one or more services. p g
Modules snap together physically and the i h l i ll services connect together logically to
enable users to easily build applications.
![Page 117: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/117.jpg)
Software PrototypingPython Symbian (HIT Lab NZ)
stbTracker wrapperstbTracker wrapperAccess to SMS, Bluetooth, GPSRapid developmentRapid development
![Page 118: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/118.jpg)
import e32import appuifwfrom gles import *
if e32.s60_version_info>=(3,0):import imp
t i l d d i ('M t' ' \\ \\bi \\M t d')magnet=imp.load_dynamic('Magnet', 'c:\\sys\\bin\\Magnet.pyd')else:
import Magnetfrom Magnet import
#Define Model
def frameback(num_markers):if (num markers > 1):if (num_markers > -1):
glMatrixMode(GL_PROJECTION)#Draw graphics…
appuifw.app.orientation = 'landscape‘ # Use full frameSetCameraCallback(frameback) # Register callbackcreateCamera() # Define cameraInitGLES() # Start Open GLInitGLES() # Start Open GLTrackerInit() # Start trackerInitCamera() # Start camera
![Page 119: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/119.jpg)
Design GuidelinesApply handheld HCI guidelines for on screen contentApply handheld HCI guidelines for on-screen content
- large buttons, little text input, etcDesign physical + virtual interface elementsPick appropriate interface metaphorpp p p
- “handheld lens” approach using handheld motionTangible AR for AR overlay- Tangible AR for AR overlay
Build prototypesContinuously evaluate application
![Page 120: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/120.jpg)
AR BrowsersAR Browsers
![Page 121: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/121.jpg)
AR BrowsersCommercial outdoor AR applications
Junaio, Layar, Wikitude, etc
All have their own language specificationsWikitude – ARMLJunaio - XML
Need for common standardBased on existing standards for geo-located content etcSupport for dynamic/interactive contentEasier to author mobile AR applicationsEasy to render on AR browsers
![Page 122: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/122.jpg)
Layar
![Page 123: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/123.jpg)
![Page 124: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/124.jpg)
![Page 125: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/125.jpg)
![Page 126: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/126.jpg)
![Page 127: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/127.jpg)
![Page 128: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/128.jpg)
![Page 129: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/129.jpg)
Junaio
![Page 130: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/130.jpg)
![Page 131: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/131.jpg)
![Page 132: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/132.jpg)
![Page 133: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/133.jpg)
Hello World Example" " " " "echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<results><poi id=\"1\" interactionfeedback=\"none\">
<name><![CDATA[Hello World POI]]></name><description><![CDATA[[This is my first POI.]]></description> <l>48.1385,11.5750,0</l>, ,<o>0,0,0</o>
<mime-type>text/plain</mime-type><thumbnail>http://www junaio com/publisherDownload/tutorial/icon jpg</<thumbnail>http://www.junaio.com/publisherDownload/tutorial/icon.jpg</thumbnail><icon>http://www.junaio.com/publisherDownload/tutorial/icon_map.png</icon> icon> </poi></results>"
![Page 134: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/134.jpg)
![Page 135: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/135.jpg)
![Page 136: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/136.jpg)
![Page 137: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/137.jpg)
![Page 138: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/138.jpg)
![Page 139: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/139.jpg)
![Page 140: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/140.jpg)
![Page 141: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/141.jpg)
![Page 142: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/142.jpg)
![Page 143: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/143.jpg)
![Page 144: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/144.jpg)
![Page 145: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/145.jpg)
KHARMA + Argon: A KML/HTML Architecture and Browser for AR Applications and Games
Bl i M I d Al HillBlair MacIntyre and Alex HillSchool of Interactive Computing,
145Georgia Institute of Technology 145
![Page 146: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/146.jpg)
KHARMAKHARMA: KML/HTML Augmented Reality Mobile Architecturemedia hackersresearch tools {
skilled computationalistsour past focus {savvy technical designers
general public
{{
breadth of adoption
general publiccurrent focus{Ygrasil IDE
Designers AR Toolkit
Unity AR Toolkit
146146Designers AR Toolkit
KHARMA
![Page 147: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/147.jpg)
KHARMA: KML/HTML Augmented Reality Mobile ArchitectureProblem: limited authoring tools for mobile ARProblem: limited authoring tools for mobile AR
limited expression (coord, name, desc, link) vs higher hurdle of 3Dno accepted standard and proprietary client protocollimited client-side interactivity is more akin to Web 1 0limited client-side interactivity is more akin to Web 1.0
Solution: HTML with KML combines What? and How? with Where?
• allows extensive client side (albeit 2D) interactivity and expressivityallows extensive client side (albeit 2D) interactivity and expressivity• two the most broadly adopted standards for presentation and geo location• HTTP server distribution, CSS and Javascript allow for true Web 2.0 content
++KML
147
HTML
![Page 148: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/148.jpg)
KHARMA Architecture with four componentsArchitecture with four components
Channel servers- delivering individual channels of AR content,
Tracking servers Tracking servers - providing content related to location
Tracking infrastructure serversTracking, infrastructure servers- delivering information about the physical environment,
Mobile client - for generating the resulting augmentations
![Page 149: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/149.jpg)
![Page 150: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/150.jpg)
KML already supports HTML• description tag accepts CDATA enclosed markup
- but no global styling of scripting support• no control over balloon styling• no control over balloon position and orientation• no relative positioning <Placemark id="culc_center">
<name>CULC Visualization</name><description><![CDATA[
<di id " l i ">G i T h><div id="culc_image">Georgia Tech><img src="http://www.culc.net/culc.png"></div>
</description><Balloon>
<location><location><latitude>34.0</latitude><longitude>--84.5</longitude>
</location><orientation><orientation>
<heading>31</heading></orientation>
</Balloon></Placemark></Placemark>
![Page 151: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/151.jpg)
<Placemark id="culc_center"><name>CULC Visualization</name><description><![CDATA[
KARML extension to KML
• added undecorated displayMode <description><![CDATA[<div id="culc_image"><img src="http://www.culc.net/culc.png"></div>
</description><Style>
<BalloonStyle>
• added undecorated displayMode
• added Balloon element<displayMode>undecorated</displayMode>
</BalloonStyle></Style><Balloon>
<locationMode targetId=”culc geospot”>relative</locationMode>
added Balloon element- similar in nature to Model element
g _g p<location>
<latitude units="meters">4.0</latitude><longitude units="meters">-2.5</longitude>
</location><orientation>
• relative locationMode- accepts “units” attribute
<orientation><heading>31</heading>
</orientation></Balloon>
</Placemark>
151151
![Page 152: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/152.jpg)
GPS Located Content<Placemark>
GeospotSurveyed location
<Placemark><name>GeoSpot</name><description>a surveyed GeoSpot </description><Camera> <!-- camera element for GeoSpot -->
Moves camera to fixed location
p<longitude>-84.394135</longitude> <!- GPS coordinates --><latitude>33.76083</latitude><altitude>0</altitude><TimeStamp> <! when GeoSpot was surveyed ><TimeStamp> <!-- when GeoSpot was surveyed -->
<when>1997-07-16T10:30:15+03:00</when></TimeStamp><Icon>
<href>http://geospot.imtc.gatech.edu/image/03_icon.png</href></Icon>
</Camera><Point> <! location displayed on map or within browser view ><Point> <!-- location displayed on map or within browser view -->
<coordinates>-84.394135,33.76083</coordinates></Point>
</Placemark>
![Page 153: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/153.jpg)
For more information, please visit:http://www argon gatech edu/http://www.argon.gatech.edu/
![Page 154: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/154.jpg)
Developing AR Experiences
Sony CSL © 2004
![Page 155: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/155.jpg)
![Page 156: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/156.jpg)
![Page 157: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/157.jpg)
![Page 158: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/158.jpg)
![Page 159: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/159.jpg)
![Page 160: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/160.jpg)
![Page 161: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/161.jpg)
![Page 162: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/162.jpg)
![Page 163: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/163.jpg)
![Page 164: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/164.jpg)
![Page 165: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/165.jpg)
![Page 166: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/166.jpg)
Game Case Study
![Page 167: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/167.jpg)
![Page 168: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/168.jpg)
![Page 169: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/169.jpg)
![Page 170: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/170.jpg)
![Page 171: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/171.jpg)
![Page 172: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/172.jpg)
![Page 173: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/173.jpg)
![Page 174: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/174.jpg)
![Page 175: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/175.jpg)
![Page 176: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/176.jpg)
![Page 177: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/177.jpg)
![Page 178: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/178.jpg)
![Page 179: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/179.jpg)
![Page 180: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/180.jpg)
Resources
![Page 181: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/181.jpg)
ResourcesBooksBooksMobile Interaction Design
M J d G M dMatt Jones and Gary MarsdenDesigning for Small Screens
Studio 7.5Handheld Usability
Scott WeissDesigning the Mobile User Experience
Barbara Ballard
![Page 182: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/182.jpg)
Developer Guidelines
Palmhttp://www.access-company.com/developers/documents/docs/ui/UI_Design.html
Zen of Palm guidelineshttp://www.access-company.com/developers/documents/docs/zenofpalm.pdf
Motorolahttp://developer.motorola.com/docstools/developerguides/
iPhone Human Interface Guidelineshttp://developer.apple.com/documentation/iPhone/Conceptual/iPhoneHIG/
![Page 183: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/183.jpg)
Handheld HCI Design WebsitesDo’s and Don’ts of PocketPC designhttp://www.pocketpcmag.com/_archives/Nov04/Commandements.aspx
U bili i l i h dh ld biliUsability special interest group – handheld usabilityhttp://www.stcsig.org/usability/topics/handheld.html
Usable Mobile websitehttp://www.smartgroups.com/groups/usablemobile
Mobile Coders Websitehttp://www mobilecoders com/Articles/mc-01 asphttp://www.mobilecoders.com/Articles/mc-01.asp
Univ of Waikato Handheld Grouphttp://www.cs.waikato.ac.nz/hci/pdas.html
Mobile Interaction Websitehttp://www.cs.waikato.ac.nz/~mattj/mwshop.html
![Page 184: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/184.jpg)
Platform – Recommended readingLots of low level information on the complete ARM familyp yValuable tool for driver andframework developersframework developersNot that important for pure application developersapplication developers
![Page 185: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/185.jpg)
Platform – Recommended readingVery low level and targeted for PCsMost information outdated on PCMost information outdated on PCEffective memory usage one of the most importantof the most importantoptimization strategies on mobile devices!on mobile devices!
![Page 186: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/186.jpg)
Tracking – Recommended readingLots of the basics on theComputer Vision you will p yneed for AR trackingSeveral code and pseudo-codeSeveral code and pseudo codesnippets
![Page 187: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/187.jpg)
Tracking – Recommended readingAll about the geometryyou will need for a tracking system
Camera modelsProjectionEpipolar geometryH hiHomographies…
![Page 188: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/188.jpg)
Graphics – Recommended reading
Mobile 3D Graphics( ll b t O GL ES 1 )(all about OpenGL ES 1.x)OpenGL ES 2.0
GProgramming GuideOpenGL ES 2.0 Man Pageshttp://www.khronos.org/opengles/sdk/docs/man/ShaderX7Chapter on “ Augmented Reality on Mobile Phones”
![Page 189: COSC 426 Lect. 5 - Mobile AR](https://reader031.vdocuments.us/reader031/viewer/2022013118/555136c4b4c905f1528b55d3/html5/thumbnails/189.jpg)
OpenGL ES ResourcesKhronos Group OpenGL ES Page
http://www.khronos.org/opengles/
OpenGL ES 2.0 Bookhttp://www.opengles-book.com/
AMDs OpenGL ES 2.0 Emulator