wien building the augmented classroom components for everyday mobile collaborative ar
TRANSCRIPT
WIENWIEN
Building theBuilding theAugmented ClassroomAugmented Classroom
Components forComponents for
everyday mobile collaborative AReveryday mobile collaborative AR
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 2/21
The Augmented Classroom - The Augmented Classroom - OverviewOverview
Spontaneous collaboration of mobile usersSpontaneous collaboration of mobile users Optical Tracking, use of markers as tangible Optical Tracking, use of markers as tangible
user interfacesuser interfaces
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 3/21
Standard User EquipmentStandard User Equipment
Mobile AR SystemMobile AR System Backpack with CPUBackpack with CPU
& connectors& connectors Optical See-throughOptical See-through
HMDHMD Head mountedHead mounted
cameracamera
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 4/21
User Equipment EnhancementsUser Equipment Enhancements
Pen is replaced byPen is replaced by„Pinch Gloves“„Pinch Gloves“
PIP is mounted to thePIP is mounted to thewrist of the left handwrist of the left hand
-> Both Hands are free-> Both Hands are freeto interact with theto interact with thereal worldreal world
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 5/21
User Equipment Details (1/6)User Equipment Details (1/6)Pinch Gloves: EvolutionPinch Gloves: Evolution First Generation:First Generation:
Markers sewed onto gloveMarkers sewed onto glove
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 6/21
User Equipment Details (2/6)User Equipment Details (2/6)Pinch Gloves: EvolutionPinch Gloves: Evolution Second Generation:Second Generation:
Rigid structure carrying markersRigid structure carrying markers
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 7/21
User Equipment Details (3/6)User Equipment Details (3/6)Pinch Gloves: EvolutionPinch Gloves: Evolution Second Generation:Second Generation:
Rigid structure carrying markersRigid structure carrying markers
Hypothesis:
Constant offset in space
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 8/21
User Equipment Details (4/6)User Equipment Details (4/6)Pinch Gloves: EvolutionPinch Gloves: Evolution Third Generation:Third Generation:
Finger-freeFinger-free Cables coveredCables covered Only one buttonOnly one button
Lightweight VersionLightweight Version 3 Buttons on thumb3 Buttons on thumb Markers attachedMarkers attached
seperatelyseperately
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 9/21
User Equipment Details (5/6)User Equipment Details (5/6)Modified PIPModified PIP
Wrist-attached to Wrist-attached to keep hands freekeep hands free
Due to inacurate Due to inacurate optical tracking, aoptical tracking, aTrack-pad is usedTrack-pad is usedfor 2D input.for 2D input.
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 10/21
User Equipment Details (6/6)User Equipment Details (6/6)HeadsetHeadset
Uses new Firefly FW Uses new Firefly FW camera with 4mm lens.camera with 4mm lens.
Nearly 90° FOVNearly 90° FOV Tracking area bigger Tracking area bigger
than HMD FOVthan HMD FOV
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 11/21
Table for overhead trackingTable for overhead tracking
Lightweight table with Lightweight table with camera for overhead trackingcamera for overhead tracking
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 12/21
Software EnhancementsSoftware Enhancements
Improvements in usability introduced byImprovements in usability introduced by One-shot quick calibration for HMD & glovesOne-shot quick calibration for HMD & gloves Shared tracking using multiple camerasShared tracking using multiple cameras
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 13/21
Software Details (1/9)Software Details (1/9)HMD CalibrationHMD Calibration Optical tracking must be calibrated in Optical tracking must be calibrated in
ARToolkitARToolkit Remaining problem: offset between Remaining problem: offset between
camera coordinate system and eye camera coordinate system and eye coordinate system.coordinate system.
ECS
CCS
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 14/21
Software Details (2/9)Software Details (2/9)HMD CalibrationHMD Calibration SoCameraCalibrationKit:SoCameraCalibrationKit:
Calibration Marker
Tracked position in overlay image
Calibration reference
• Align marker with calibrationreference
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 15/21
Software Details (3/9)Software Details (3/9)HMD CalibrationHMD Calibration SoCameraCalibrationKit:SoCameraCalibrationKit:
• Press Button
-> Offset is calculated and writtento OT node for camera offset
• Marker and tracked position arenow aligned correctly.
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 16/21
Software Details (4/9)Software Details (4/9)Glove CalibrationGlove Calibration
A Similar tool is provided to calibrate the A Similar tool is provided to calibrate the „Pinch Point“ of the gloves„Pinch Point“ of the gloves
The offset varies for every user, and can The offset varies for every user, and can be configured with a single clickbe configured with a single click
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 17/21
Software Details (5/9)Software Details (5/9)Shared TrackingShared Tracking Optical tracking fails:Optical tracking fails:
Marker (partially) out of FOV of cameraMarker (partially) out of FOV of camera Marker occludedMarker occluded Marker oriented away from cameraMarker oriented away from camera Marker not recognizedMarker not recognized
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 18/21
Software Details (6/9)Software Details (6/9)Shared TrackingShared Tracking Solution / Improvement:Solution / Improvement:
Use several cameras, sharing their tracking Use several cameras, sharing their tracking data over the networkdata over the network
Dynamically determine reference markers Dynamically determine reference markers (seen by 2 or more cameras)(seen by 2 or more cameras)
Calculate position ofCalculate position ofhidden marker fromhidden marker fromreference + remotereference + remotecoordinatescoordinates
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 19/21
Software Details (7/9)Software Details (7/9)Shared TrackingShared Tracking Opentracker implementation (1/2):Opentracker implementation (1/2):
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 20/21
Software Details (8/9)Software Details (8/9)Shared TrackingShared Tracking Opentracker implementation (2/2):Opentracker implementation (2/2):
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 21/21
Software Details (9/9)Software Details (9/9)Shared TrackingShared Tracking Results:Results:
Flo Ledermann
WIEN
Components for building the Augmented Classroom - 22/21
Thank you!Thank you!