hrvoje benko, andrew d. wilson, and ravin balakrishnan microsoft research – university of...

Hrvoje Benko, Andrew D. Wilson, and Ravin Balakrishnan

Microsoft Research – University of Toronto

SphereMulti-touch Interactions on a Spherical Display

UIST '08 2

3

Spherical Displays

UIST '08

Pufferfish – PufferSphere

ARC Science – OmniGlobe

Kettner et al. ‘04 – ViBall

Global Imagination - MagicPlanet

Companje et al. ‘07 – Globe4D

NOAA – Science on a Sphere

Not directly interactive

Physical rotation only

4UIST '08

Interaction Metaphor Shift

UIST '08

Sphere Multi-touch sensitive Built on Global

Imagination’s MagicPlanet

Internal projection and vision-based sensing

5

GOAL: Explore interactive possibilities of spherical displays.

UIST '08 6

Video: Sphere

UIST '08 7

Sphere Video

UIST '08 8

Sphere Hardware

16” 24”

UIST '08 9

UIST '08 10

UIST '08 11

Reusing the Optical Path

UIST '08 12

Video: Touch-Sensing

UIST '08 13

Touch-Sensing Software Pipeline

UIST '08 14

Sensing and Projection Distortions

Vertex shader: Pre-distorts objects

into 2D disk image Objects appear

undistorted on the spherical surface

2D

3D

UIST '08 15

Video: Projection Distortions

UIST '08 16

Projection Distortions

UIST '08 17

Choosing a Coordinate System

2D (Disk)

Flat (Cylindrical)

3D(Spherical)

Hybrid(Tangential

Plane)

UIST '08 18

Application Prototypes Photo & Video Browser Omni-Directional Data Visualization

GlobeOmni-cameraPanoramic Walkthrough

Sphere Painting Sphere Pong

UIST '08 19

Design Space of Spherical Displays

Non-visible hemisphere Visibility changes with position

“Pseudo-private” and “public” areas No master user position / orientation Smooth transitions in depth and orientations

Near – farHorizontal – vertical

Borderless, but finite display Few natural landmarks: poles

UIST '08 20

Multi-Touch Interaction Techniques

Multi-touch basicsDrag, Rotation, Scale

Extend user’s reachFlicking, Send-to-Dark-Side

Minimize orientation difficultiesAuto-rotation, Tether

Universal circular menuOrb-like invocation gesture

UIST '08 21

Dragging

V

Flat Surface

12

Q

A

θ1

2

Spherical Surface

UIST '08 22

Send-to-Dark-Side Open-palm gesture Control the location where the object will

appear

UIST '08 24

Tether Allow unconstrained manipulation Preserve “natural” orientation of omni-

directional data upon release

UIST '08 25

Universal Circular Menu Orb-like Invocation

Easy to perform – Hard to do by accident

UIST '08 26

Video: Sphere Interactions

UIST '08 27

Sphere Multi-Touch Interactions

UIST '08 28

Video: Contact Area Interactions

UIST '08 29

Sphere Paint & Pong

UIST '08 30

User Observations

3 high-traffic demos>1000 people

Feedback:“Magical”, “Like a crystal ball”“What is it good for?”“Can I zoom?”

Interaction zones:Top – shared Middle – personal/working Bottom – storage

Logs show no position preference

UIST '08 31

Future Work Zooming Sphere as a large controller Sphere as a room projector (instant CAVE) Handheld Sphere

UIST '08 32

UIST '08 33

benko@microsoft.comhttp://research.microsoft.com/~benko/projects/sphere

Try our demo at the reception tonight!

UIST '08 34

Extra Slides

UIST '08 35

Applications Visualizing spherical data

Planets, stars Public information booths Remote omni-directional visualization

Sphere + RoundTable 360 deg. video-conferencing

Robot operation Ambient Displays Large displayable input device

TrackballSpace Mouse / JoystickPlanetarium Control

Games

UIST '08 36

Conclusions The first multi-touch sensitive spherical

displaySelf-enclosed solution without occlusion and

shadowing problems A set of multi-touch interactions that

facilitate data sharing around the display Omni-directional room projection

possibility Instant CAVE

37UIST '08

Uses

UIST '08 38

benko@microsoft.comhttp://research.microsoft.com/~benko/projects/sphere

Come to our demo at the reception

tonight!

UIST '08 39

M.C. Escher

benko@microsoft.comhttp://research.microsoft.com/~benko

UIST '08 40

Putting it all together…

UIST '08 41

Two Spheres (of Interest)

UIST '08 42

Sphere Pong

UIST '08 43

Sphere Painting

UIST '08 44

Extending User’s Reach

UIST '08 45

Auto-rotation & Menu

UIST '08 46

Tether Behavior