steerable interfaces for interactive environments
DESCRIPTION
Institut National de Recherche en Informatique et Automatique. Stanislaw Borkowski thesis director: James L. Crowley Jury:. Steerable Interfaces for Interactive Environments. Andreas Butz (UM), Jo ë lle Coutaz (UJF), Alex Pentland (MIT), Pierre Wellner (IDIAP). INRIA Rh ô ne-Alpes - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/1.jpg)
Steerable Interfaces for Interactive
EnvironmentsStanislaw Borkowski
thesis director: James L. CrowleyJury:
Institut National de Recherche en Informatique et Automatique
INRIA Rhône-AlpesJune 26, 2006
Andreas Butz (UM), Joëlle Coutaz (UJF), Alex Pentland (MIT), Pierre Wellner (IDIAP)
![Page 2: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/2.jpg)
2
User Interface (UI): aggregate of physical entities or information bound to these entities
What is a user interface?
A
![Page 3: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/3.jpg)
3
Steerable UI:can be relocated in spaceposition is mediated by
the computer system
Portable UI:can be relocatedposition is directly
controlled through physical contact
Mobile UI’s
Mobile UIs
Portable UIs
Steerable UIs
A
![Page 4: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/4.jpg)
4
Mobility in current IT Steerable interfaces
Conventional GUI (steerable output) X11 session teleporting [Richardson93]
Portable interfaces Wearable computers Cell phones Personal Digital Assistants Laptops ….
![Page 5: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/5.jpg)
5
Mobility in ambient computing Multiple displays embedded in
the environment Large size displays Mobile interaction resources,
both portable and steerable
[Pinhanez01] [Streitz99]
[Arias00]
![Page 6: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/6.jpg)
6
Why steerable?
Flexibility in resources usage
New forms of Human-computer interaction
New forms of Human-Human interaction
![Page 7: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/7.jpg)
7
Current situation – summary
Problem: Need for steerable UIs No predictive models
Solution: Provide enabling technology Explore interaction techniques Evaluate the value of steerable UIs
![Page 8: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/8.jpg)
8
Outline Mobility in IT Steerable UIs Mobile projected UI Mobile UIs for collaborative work Conclusions
![Page 9: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/9.jpg)
9
State of the art
EasyLiving [Brumitt00]
Tic-Tac-Toe [Pinhanez05]
![Page 10: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/10.jpg)
10
State of the art – limitations
UI is observable at standstill Limited spatial controllability Only predefined locations Planar surfaces only
Requirements for steerable UIs: Continuous observability and
controllability
![Page 11: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/11.jpg)
11
Outline Mobility in IT Steerable UIs Mobile projected UI
Prototype implementation[in collaboration with J. Letessier]
Evaluation – latency estimation
Mobile UIs for collaborative work Conclusions
![Page 12: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/12.jpg)
12
The Steerable Camera Projector
(2002)
Other steerable projection systems: The Everywhere Display (IBM 2000)
Fluid Beam (Fluidum consortium 2002)
SCP from Karlsruhe (UKA 2004)
![Page 13: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/13.jpg)
13
Steerable display (2002)
![Page 14: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/14.jpg)
14
User-centric approach
End-users: Latency limits < 50ms Easy setup, no maintenance Reliability / predictability
Developers: Abstraction: be relevant Isolation: allow integration Contract: offer quality of service
![Page 15: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/15.jpg)
15
Pragmatic approach
Black-box servicesBIP (Basic Interconnection Protocol)
BIP implementation ≈ SOA middleware
service/service and service/application communication
service discovery (standards-based)
![Page 16: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/16.jpg)
16
Interactive system
ApplicationSCP
software
Human and Environmen
t
Interaction events
Display orders
![Page 17: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/17.jpg)
17
Interactive system
Application
Human and Environmen
tSCPdisplay
SCPcontroller
Frame grabber
Interaction detector
SCPcalibrator
A
![Page 18: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/18.jpg)
18
Interactive system
Application
Human and Environmen
tSCPdisplay
SCPcontroller
Frame grabber
Interaction detector
SCPcalibrator
![Page 19: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/19.jpg)
19
Video projector
Light source
Screen
Source image
Projection on arbitrary oriented planar surfaces
User’s perception
![Page 20: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/20.jpg)
20
Video projector
Screen
Projection on arbitrary oriented planar surfaces
Light source
Image to project
User’s perception
Source image
SCPdisplay
![Page 21: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/21.jpg)
21
Projection on arbitrary oriented planar surfaces
Image to project
User’s view
![Page 22: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/22.jpg)
22
Interactive system
Application
Human and Environmen
tSCPdisplay
SCPcontroller
Frame grabber
Interaction detector
SCPcalibrator
![Page 23: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/23.jpg)
23
Sensor-centric environment model
1 2
3
123
![Page 24: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/24.jpg)
24
Display surface detection
Screen
![Page 25: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/25.jpg)
25
The Portable Display Surface
![Page 26: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/26.jpg)
26
Interactive system
Application
Human and Environmen
tSCPdisplay
SCPcontroller
Frame grabber
Interaction detector
SCPcalibrator
![Page 27: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/27.jpg)
27
Interactive widgets projected on a portable display surface
![Page 28: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/28.jpg)
28
Luminance-based button widget
![Page 29: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/29.jpg)
29
Locate widget in the camera image
Estimate occlusion
Update widget state
Touch detection
CIH
)()(:)( tLtLtL io
![Page 30: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/30.jpg)
30
Robustness to clutter
![Page 31: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/31.jpg)
31
Assembling occlusion detectors
![Page 32: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/32.jpg)
32
Striplet – the occlusion detector
x
y
0 R
![Page 33: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/33.jpg)
33
Striplet-based SPOD
SPOD – Simple-Pattern Occlusion Detector
![Page 34: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/34.jpg)
34
Striplet-based button
![Page 35: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/35.jpg)
35
SPOD-based calculator
Accelerated video
![Page 36: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/36.jpg)
36
Outline Mobility in IT Steerable UIs Mobile projected UI
Prototype implementationEvaluation – latency estimation
Mobile UIs for collaborative work Conclusions
![Page 37: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/37.jpg)
37
Latency estimation
0t
pt 0t
PCI A/D converter
Frame Grabber
CPU
Imalab shell
Image processing
Graphic Card
OpenGl render
0ttl p
![Page 38: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/38.jpg)
38
Latency estimation
Fan
PCI A/D converter
Frame Grabber
CPU
Imalab shell
Image processing
Graphic Card
OpenGl renderRegulated power
supply
Video sequence capture
Plastic bar
Projection of the bar
![Page 39: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/39.jpg)
39
Latency estimation – results
+ up to 51ms!!!
A~17ms
0tPCI A/D converter
Frame Grabber
CPU
Imalab shell
Image processing
Graphic Card
OpenGl render
~70ms
0tPCI A/D converterFrame Grabber
CPU
Imalab shell
Graphic Card~32ms
![Page 40: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/40.jpg)
40
Interactive system
Application
Human and Environmen
tSCPdisplay
SCPcontroller
Frame grabber
Interaction detector
SCPcalibrator
![Page 41: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/41.jpg)
41
Outline
Mobility in IT Steerable UIs Mobile projected UIs Mobile UIs for collaborative work
ContAct applicationUser study – comparison of different
take-over techniques
Conclusions
![Page 42: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/42.jpg)
42
ContAct – a system for authoring presentations
Collaboration through interface mobility
![Page 43: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/43.jpg)
43
ContAct application setup Wide angle camera Tabletop camera Steerable Camera Projector Portable Display Surface
![Page 44: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/44.jpg)
44
ContAct application GUI
![Page 45: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/45.jpg)
45
Outline Mobility in IT Steerable interface prototype Mobile UIs for collaborative work
ContAct applicationTaking control: a comparative user study
[in collaboration with J. Maisonnasse and J. Letessier]
Conclusions
![Page 46: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/46.jpg)
46
Evaluation of techniques for taking control
Objectives: Determine the preferred
control taking technique Evaluate the impact on the
task completion performance Evaluate user acceptance of
steerable interfaces
![Page 47: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/47.jpg)
47
Experimental setup
GUI
Users
Hardware:
Steerable Camera Projector
Microphone headsets
Portable Display Surface
Software:
Speech detector [D. Vaufreydaz]
Conversation modeling [J. Maisonnaisse]
Finger tracking [J. Letessier]
PDS tracking
Drawing application
![Page 48: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/48.jpg)
48
The User Interface
![Page 49: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/49.jpg)
49
The task
Collaborative reconstruction of a graph
![Page 50: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/50.jpg)
50
The task
Collaborative reconstruction of a graph
User 2 User 3User 1
![Page 51: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/51.jpg)
51
Experimental conditions
Proposed techniques for taking control: Baseline: fixed interface Portable: PDS Steerable: touch-based Steerable: voice-based steering
![Page 52: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/52.jpg)
52
Fixed interface
GUI
Users
![Page 53: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/53.jpg)
53
Explicit direct manipulation
![Page 54: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/54.jpg)
54
Explicit touch-based steering
![Page 55: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/55.jpg)
55
Implicit voice-based steering
Rules controlling the interface location: Interface is steered toward
the “main speaker” Interruptions are ignored Drawing inhibits vocal steering Conflicts result in loss of interface control
![Page 56: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/56.jpg)
56
Subjects
12 groups of 3 people 13 women, 23 men Average age 27.7 19 experts in IT 17 subjects familiar with IT
![Page 57: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/57.jpg)
57
1
1.5
2
2.5
3
3.5
fixed button pds voice
experts non-experts
Results – user preference
rank
Rank scale: 1 = most liked 4 = least liked
![Page 58: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/58.jpg)
58
Results – PDS
Fun to use Predictable
Less intuitive Less reactive Not well suited
for the task
Experts Non-experts
![Page 59: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/59.jpg)
59
Results – Voice-based control
Intimidating Limits
collaboration
Fun to use Enhances
collaboration
Experts Non-expertsModified their behaviour
Least predictable
![Page 60: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/60.jpg)
60
Example result
![Page 61: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/61.jpg)
61
User performance – ability to duplicate
0
10
20
30
40
50
60
70
80
90
100
fixed button pds voice
experts non-experts
% of remembered elements
![Page 62: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/62.jpg)
62
Outline Mobility in IT Steerable UIs Mobile projected UI Mobile UIs for collaborative work Conclusions
![Page 63: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/63.jpg)
63
Conclusions 1/2
Steerable camera-projector pair enables mobile UIs
Portable UIs (PDS)
Steerable UIs
![Page 64: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/64.jpg)
64
Conclusions 2/2
UI mobility can enhance the collaborative experience
Explicit control is preferred over implicit control
![Page 65: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/65.jpg)
65
Future directions 1/2
The SCP: Adapting to display surface texture
The PDS: Tracking and interaction with multiple PDS’ High frame-rate tracking
Vision-based projected widgets: Integration of multiple occlusion detectors
![Page 66: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/66.jpg)
66
Steerable interfaces: Other applications for steerable interfaces Alternative methods for controlling the
location Exploring links with plastic interfaces –
dynamic interface adaptation Creation of a “space manager”
Future directions 2/2
![Page 67: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/67.jpg)
67
Thank you for your attention
![Page 68: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/68.jpg)
68
![Page 69: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/69.jpg)
69
ResultsThe preference:#1 the PDS #2 button-based control #3 voice-based control#4 fixed interface
2
3
4
fixed button pds voice
0
2
4
6
8
10
12
14
16
1 2 3 4
![Page 70: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/70.jpg)
70
1 2
3
Sensor-centric environment model
![Page 71: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/71.jpg)
71
SPOD software components
Frame Grabber
Client Application
Calibration
GUI rendering
GUI
Striplets Engine
VEILS
P
O
D
![Page 72: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/72.jpg)
72
Striplet – the occlusion detector
dxdytyxyx t ),,L(),(f)(R gain
Gain
x
x
y
0),(fgain dxdyyx
![Page 73: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/73.jpg)
73
VEIL – Vision Events Interpretation Layer
Striplets Engine
VEILS
P
O
D
Inputs Widgets coordinates Scale and UI to camera mapping matrix Striplets occlusion events
Outputs Interaction events Striplets coordinates
![Page 74: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/74.jpg)
74
VEIL – Vision Events Interpretation Layer
Striplets Engine
VEILS
P
O
D
Inputs Widgets coordinates Scale and UI to camera mapping matrix Striplets occlusion events
Outputs Interaction events Striplets coordinates
![Page 75: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/75.jpg)
75
VEIL – Vision Events Interpretation Layer
Striplets Engine
VEILS
P
O
D
Inputs Widgets coordinates Scale and UI to camera mapping matrix Striplets occlusion events
Outputs Interaction events Striplets coordinates
![Page 76: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/76.jpg)
76
Inputs Striplets UI-coordinates UI to camera mapping matrix Images from camera service
Outputs Occlusion events
Striplets Engine Service
Striplets Engine
VEILS
P
O
D
![Page 77: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/77.jpg)
77
Inputs Striplets UI-coordinates UI to camera mapping matrix Images from camera service
Outputs Occlusion events
Striplets Engine Service
Striplets Engine
VEILS
P
O
D
![Page 78: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/78.jpg)
78
Inputs Striplets UI-coordinates UI to camera mapping matrix Images from camera service
Outputs Occlusion events
Striplets Engine Service
Striplets Engine
VEILS
P
O
D
![Page 79: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/79.jpg)
79
VEIL – Vision Events Interpretation Layer
Striplets Engine
VEILS
P
O
D
Inputs Widgets coordinates Scale and UI to camera mapping matrix Striplets occlusion events
Outputs Interaction events Striplets coordinates
![Page 80: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/80.jpg)
80
Striplet-based slider
![Page 81: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/81.jpg)
81
Tracking the PDS
Tracking edges in the Hough space
+ Naturally robust to partial occlusions
- High computation cost
Line-segments-based tracking
+ Efficient quadrilateral detection
- Difficulties in handling occlusions
![Page 82: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/82.jpg)
82
Pushing vs. pulling the UI
![Page 83: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/83.jpg)
83
Results
0
0.2
0.4
0.6
0.8
1
1 2 3 4
Time performance:
Trial number
Normalized trial time
sipi XHX
![Page 84: Steerable Interfaces for Interactive Environments](https://reader036.vdocuments.us/reader036/viewer/2022062517/56813557550346895d9cbdf4/html5/thumbnails/84.jpg)
84
Performance• +- 180 deg of pan
• 1600 discrete positions (resolution)• 90 deg/s max pan speed reached in 0.75 s
• 90 deg of tilt• 500 discrete positions• 80 deg/s max tilt speed reached in 0.60s
Video Projector
Camera
Pan Stepper-motor
Tilt Stepper-motor
Control and power supply