wien building the augmented classroom components for everyday mobile collaborative ar

WIENWIEN

Building theBuilding theAugmented ClassroomAugmented Classroom

Components forComponents for

everyday mobile collaborative AReveryday mobile collaborative AR

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 2/21

The Augmented Classroom - The Augmented Classroom - OverviewOverview

Spontaneous collaboration of mobile usersSpontaneous collaboration of mobile users Optical Tracking, use of markers as tangible Optical Tracking, use of markers as tangible

user interfacesuser interfaces

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 3/21

Standard User EquipmentStandard User Equipment

Mobile AR SystemMobile AR System Backpack with CPUBackpack with CPU

& connectors& connectors Optical See-throughOptical See-through

HMDHMD Head mountedHead mounted

cameracamera

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 4/21

User Equipment EnhancementsUser Equipment Enhancements

Pen is replaced byPen is replaced by„Pinch Gloves“„Pinch Gloves“

PIP is mounted to thePIP is mounted to thewrist of the left handwrist of the left hand

-> Both Hands are free-> Both Hands are freeto interact with theto interact with thereal worldreal world

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 5/21

User Equipment Details (1/6)User Equipment Details (1/6)Pinch Gloves: EvolutionPinch Gloves: Evolution First Generation:First Generation:

Markers sewed onto gloveMarkers sewed onto glove

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 6/21

User Equipment Details (2/6)User Equipment Details (2/6)Pinch Gloves: EvolutionPinch Gloves: Evolution Second Generation:Second Generation:

Rigid structure carrying markersRigid structure carrying markers

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 7/21

User Equipment Details (3/6)User Equipment Details (3/6)Pinch Gloves: EvolutionPinch Gloves: Evolution Second Generation:Second Generation:

Rigid structure carrying markersRigid structure carrying markers

Hypothesis:

Constant offset in space

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 8/21

User Equipment Details (4/6)User Equipment Details (4/6)Pinch Gloves: EvolutionPinch Gloves: Evolution Third Generation:Third Generation:

Finger-freeFinger-free Cables coveredCables covered Only one buttonOnly one button

Lightweight VersionLightweight Version 3 Buttons on thumb3 Buttons on thumb Markers attachedMarkers attached

seperatelyseperately

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 9/21

User Equipment Details (5/6)User Equipment Details (5/6)Modified PIPModified PIP

Wrist-attached to Wrist-attached to keep hands freekeep hands free

Due to inacurate Due to inacurate optical tracking, aoptical tracking, aTrack-pad is usedTrack-pad is usedfor 2D input.for 2D input.

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 10/21

User Equipment Details (6/6)User Equipment Details (6/6)HeadsetHeadset

Uses new Firefly FW Uses new Firefly FW camera with 4mm lens.camera with 4mm lens.

Nearly 90° FOVNearly 90° FOV Tracking area bigger Tracking area bigger

than HMD FOVthan HMD FOV

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 11/21

Table for overhead trackingTable for overhead tracking

Lightweight table with Lightweight table with camera for overhead trackingcamera for overhead tracking

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 12/21

Software EnhancementsSoftware Enhancements

Improvements in usability introduced byImprovements in usability introduced by One-shot quick calibration for HMD & glovesOne-shot quick calibration for HMD & gloves Shared tracking using multiple camerasShared tracking using multiple cameras

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 13/21

Software Details (1/9)Software Details (1/9)HMD CalibrationHMD Calibration Optical tracking must be calibrated in Optical tracking must be calibrated in

ARToolkitARToolkit Remaining problem: offset between Remaining problem: offset between

camera coordinate system and eye camera coordinate system and eye coordinate system.coordinate system.

ECS

CCS

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 14/21

Software Details (2/9)Software Details (2/9)HMD CalibrationHMD Calibration SoCameraCalibrationKit:SoCameraCalibrationKit:

Calibration Marker

Tracked position in overlay image

Calibration reference

• Align marker with calibrationreference

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 15/21

Software Details (3/9)Software Details (3/9)HMD CalibrationHMD Calibration SoCameraCalibrationKit:SoCameraCalibrationKit:

• Press Button

-> Offset is calculated and writtento OT node for camera offset

• Marker and tracked position arenow aligned correctly.

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 16/21

Software Details (4/9)Software Details (4/9)Glove CalibrationGlove Calibration

A Similar tool is provided to calibrate the A Similar tool is provided to calibrate the „Pinch Point“ of the gloves„Pinch Point“ of the gloves

The offset varies for every user, and can The offset varies for every user, and can be configured with a single clickbe configured with a single click

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 17/21

Software Details (5/9)Software Details (5/9)Shared TrackingShared Tracking Optical tracking fails:Optical tracking fails:

Marker (partially) out of FOV of cameraMarker (partially) out of FOV of camera Marker occludedMarker occluded Marker oriented away from cameraMarker oriented away from camera Marker not recognizedMarker not recognized

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 18/21

Software Details (6/9)Software Details (6/9)Shared TrackingShared Tracking Solution / Improvement:Solution / Improvement:

Use several cameras, sharing their tracking Use several cameras, sharing their tracking data over the networkdata over the network

Dynamically determine reference markers Dynamically determine reference markers (seen by 2 or more cameras)(seen by 2 or more cameras)

Calculate position ofCalculate position ofhidden marker fromhidden marker fromreference + remotereference + remotecoordinatescoordinates

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 19/21

Software Details (7/9)Software Details (7/9)Shared TrackingShared Tracking Opentracker implementation (1/2):Opentracker implementation (1/2):

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 20/21

Software Details (8/9)Software Details (8/9)Shared TrackingShared Tracking Opentracker implementation (2/2):Opentracker implementation (2/2):

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 21/21

Software Details (9/9)Software Details (9/9)Shared TrackingShared Tracking Results:Results:

Flo Ledermann

WIEN

Components for building the Augmented Classroom - 22/21

Thank you!Thank you!