mark hancock, university of waterloo

mark hancock

university of waterloo

leveraging physical actions to interact with digital surfaces

downhill back-country (telemark)

downhill bindings attached at rear

telemark bindings detatched at rear

falling over is a good thing!

providing rich multi-touch manipulation together with the appropriate visual feedback can enable virtual objects to be used in a meaningful way, as tools

we can use these virtual tools to leverage physical actions to interact with digital surfaces

multi-touch manipulation visual feedback virtual tools + =

part i part ii part iii

embodied interaction

part iv

part i: multi-touch manipulation

how do I

how do I use them like real objects?

interact with these objects? pick them up? turn them over?

technique one: one-finger interaction

one finger = two DOF input

dedicated areas

can use on any one-touch display

technique two: two-finger interaction

two fingers = four DOF input

dedicated translate area

separate and simultaneous actions

technique three: three-finger interaction

three fingers = six DOF input

dedicated fingers

comparative study

twelve participants six male, six female

three tasks passing, docking, puzzle

results

people are faster with more touches

0

5

10

15

20

25

one-finger two-finger three-finger

Task

Co

mp

letio

n T

ime

(s) passing

docking

people prefer more touches

1

2

3

4

5

6

7D

isa

gre

e

Ag

ree

I found the

technique easy

to use

the object

reacted as I

expected it to

I could easily move

an object to where

I wanted

1

2

3

4

5

6

7

Dis

ag

ree

Ag

ree

one-finger two-finger three-finger

I found the

technique

difficult to control

I found it difficult

to turn objects in

the plane

I found it difficult to

roll objects over

1

2

3

4

5

6

7

Dis

ag

ree

Ag

ree

one-finger two-finger three-finger

I found the

technique

difficult to control

I found it difficult

to turn objects in

the plane

I found it difficult to

roll objects over

1

2

3

4

5

6

7D

isa

gre

e

Ag

ree

I found the

technique

difficult to control

I found it difficult

to turn objects in

the plane

I found it difficult to

roll objects over

people perform many simultaneous actions

0

2

4

6

8

10

12

one-finger two-finger three-finger

Tim

e S

pe

nt

Tou

ch

ing

(s)

0

2

4

6

8

10

12

one-finger two-finger three-finger

Tim

e S

pe

nt

Tou

ch

ing

(s)

0

2

4

6

8

10

12

one-finger two-finger three-finger

Tim

e S

pe

nt

Tou

ch

ing

(s) passing

docking

0

2

4

6

8

10

12

one-finger two-finger three-finger

Tim

e S

pe

nt

Tou

ch

ing

(s)

translations

planar rotations

spatial rotations

people are faster with more touches

people prefer more touches

people perform many simultaneous actions

observation: “why is the object turning when I just move it?”

part ii: perception

center of projection (COP)

rays picture plane

perspective projection

center of projection

point of view

perception study

factor: POV-COP discrepancy

no discrepancy medium discrepancy large discrepancy

factor: projection type

perspective

parallel

factor: motion parallax

factor: motion parallax

no parallax parallax

task

results

0

10

20

30

40

50

60

70

80M

ean

Err

or

(°)

motion parallax

discrepancy none med large none med large

parallel perspective

mean error up to 60°

part iii: virtual tools

the dawn of digital tables

application: sandtray therapy

cooperative design

face-to-face meetings

iterative design via distance collaboration

face-to-face feedback session interviews & mock therapy

design considerations

narrative potential

associative medium

digital extensions

facilitate interpretation

feedback

construction

storytelling

actions

arrangement

research in the Touchlab (and other labs)

part iv: embodied interaction

Urp (Underkoffler & Ishii, 1998)

stephanie mikulecky (ucalgary)

miguel nacenta (st. andrews, scotland)

sheelagh carpendale (ucalgary)

measuring embodiment

kimberly mikulecky (ucalgary)

john brosz (ucalgary)

sheelagh carpendale (ucalgary)

cloth interaction

joseph shum (uwaterloo)

collaborative multi-touch navigation

call of duty black ops lego star wars

Shear Shear

Ripple Ripple

dmitry pyryeskin (uwaterloo)

jesse hoey (uwaterloo)

above the surface

betty chang (uwaterloo)

stacey scott (uwaterloo)

understanding automation

adam bradley (uwaterloo)

sheelagh carpendale (ucalgary)

playing with data

downhill back-country (telemark)

with a better understanding of human perception together with interaction that uses our hands and bodies, we can leverage physical actions to better interact with digital surfaces

students: adam bradley

betty chang

arezoo irannejad

rebecca langer

dmitry pyryeskin

joseph shum

kimberly mikulecky (ucalgary)

stephanie mikulecky (ucalgary)

acknowledgements: NSERC

Games Institute

SurfNet

GRAND NCE

thank you mark.hancock@uwaterloo.ca