an architecture to enable spontaneous mobile spatial interaction with pervasive services
DESCRIPTION
TRANSCRIPT
![Page 1: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/1.jpg)
13.05.07
RelateGateways
An Architecture to Enable SpontaneousMobile Spatial Interaction with Pervasive ServicesMaster Thesis, Dominique Guinard, Fribourg 2007Supervisors: Hans Gellersen, Denis Lalanne, Rolf Ingold.
![Page 2: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/2.jpg)
RelateGateways: Dominique Guinard
2 / 35
Context
Lancaster, Northern UK:^ Pop: 30’000 ^ (100’000 when
including the sheeps).
Infolab 21:^ 250 IT researchers.^ Ubicomp Group.
![Page 3: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/3.jpg)
RelateGateways: Dominique Guinard
3 / 35
Today’s Menu…
Introduction, Aim E2ESD Model
^Spatial Discovery^Video^Network and Service Discovery^ Invocation and Interoperability
• Cross-device Interactions
Evaluation Conclusion
![Page 4: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/4.jpg)
RelateGateways: Dominique Guinard
4 / 35
Introduction: Bob’s Adventure
Bob, researcher in biology at Unifr talk at Lancs.
Bob needs to print his presentation.
Where is the printer?
?T - 30 minutes
![Page 5: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/5.jpg)
RelateGateways: Dominique Guinard
5 / 35
Introduction: Bob’s Adventure
Bob finds someone who knows someone who met someone who knows where the printer is located!
Hem, Bob feels less cool.
?T - 10 minutes
![Page 6: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/6.jpg)
RelateGateways: Dominique Guinard
6 / 35
Introduction: Bob’s Adventure
Bob now needs:^ To find the printer’s
drivers.^ Install the printer.^ Get the right to access it.^ Find out about the
printer’s properties and accepted formats.
^ Etc…^ … print the document!
Arg, Bob doesn’t feel cool anymore!T - 30 seconds
![Page 7: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/7.jpg)
RelateGateways: Dominique Guinard
7 / 35
Introduction: Summary
Mobile users can benefit from access to pervasive services.
Network and service discovery technologies facilitate spontaneous connections.
However, these approaches are not user centric:^ Difficult for users to identify services;^ Lack of simple and natural interaction techniques:^ => Lack of spontaneity in interaction…
Introduction
![Page 8: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/8.jpg)
RelateGateways: Dominique Guinard
8 / 35
Aim
Using the mobile device to:^ Identify services available in the user’s
immediate environment.^Consume the services in a natural and
standard manner.
Need to:^ Include users in the discovery process.^Address identification, discovery and
invocation.
Introduction
![Page 9: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/9.jpg)
RelateGateways: Dominique Guinard
9 / 35
Today’s Menu…
Introduction, Aim E2ESD Model
^Spatial Discovery^Video^Network and Service Discovery^ Invocation and Interoperability
• Cross-device Interactions
Evaluation Conclusion
![Page 10: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/10.jpg)
RelateGateways: Dominique Guinard
10 / 35
End-to-End Service Discovery Model (E2ESD)
Spatial Discovery
Invocation & Interoperability
Network & Service Discovery
Interaction Life-cycle
Mobile Code
Layer 1 Layer 2 Layer 3Mobile User Pervasive Services
Mo
del
Imp
lem
en
tati
on
$
E2ESD Model
![Page 11: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/11.jpg)
RelateGateways: Dominique Guinard
11 / 35
Layer 1: Spatial Discovery
Spatial Discovery
Invocation & Interoperability
Network & Service Discovery
Interaction Life-cycle
Mobile Code
Layer 1 Layer 2 Layer 3Mobile User Pervasive Services
Mo
del
Imp
lem
en
tati
on
$
![Page 12: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/12.jpg)
RelateGateways: Dominique Guinard
12 / 35
Gateways User Interface
Widgets as access points to the services:^Gateways.
Users visually discover the services:^Mapping the user’s
view of his environment on the mobile desktop.
^UI as a compass.
Keyboard
Beamer and Display
Printer
Spatial Discovery
![Page 13: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/13.jpg)
RelateGateways: Dominique Guinard
13 / 35
Implementation
Small Java Swing windows representing the service providers.
At the screen periphery, integrated to the desktop.
Two interaction modes:^ Drag-and-Drop.^ Click.
Spatial Discovery
![Page 14: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/14.jpg)
RelateGateways: Dominique Guinard
14 / 35
Spatial Context
Spatial context initialy delivered to the mobile client by a Wizard of Oz interface.
Introducing Relate:^ EU-founded project.^ Ad-hoc sensor network.^ Providing relative
positioning. Achieved: first extend to
provide real-time positioning data to the compass UI.
Spatial Discovery
![Page 15: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/15.jpg)
RelateGateways: Dominique Guinard
15 / 35
Spatial Context: Deployement
To provide the user interface with spatial context we need:^A USB Dongle/Brick on
the mobile device.^An autonomous Dot on
each service provider.
Spatial Discovery
![Page 16: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/16.jpg)
RelateGateways: Dominique Guinard
16 / 35
Video Demonstration
![Page 17: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/17.jpg)
RelateGateways: Dominique Guinard
17 / 35
Layer 2: Network & Service Discovery
Spatial Discovery
Invocation & Interoperability
Network & Service Discovery
Interaction Life-cycle
Mobile Code
Layer 1 Layer 2 Layer 3Mobile User Pervasive Services
Mo
del
Imp
lem
en
tati
on
$
![Page 18: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/18.jpg)
RelateGateways: Dominique Guinard
18 / 35
Extending Relate: BeforeNetwork Discovery
USB DONGLE
USB DONGLE
USB DONGLEUSB Connection
USB Connection
USB Connection
RF + Ultrasound
USB / Ethernet
![Page 19: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/19.jpg)
RelateGateways: Dominique Guinard
19 / 35
Extending Relate: After
DOT
DOT
DOT
DOTPublic
Display Server
Ethernet Connection
DOT
DOT
USB DONGLE
USB Connection
Position + IP to Service
Network Discovery
![Page 20: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/20.jpg)
RelateGateways: Dominique Guinard
20 / 35
Layer 3: Invocation and Interoperability
Spatial Discovery
Invocation & Interoperability
Network & Service Discovery
Interaction Life-cycle
Mobile Code
Layer 1 Layer 2 Layer 3Mobile User Pervasive Services
Mo
del
Imp
lem
en
tati
on
$
Interoperability
![Page 21: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/21.jpg)
RelateGateways: Dominique Guinard
21 / 35
Modeling the Services
A service is composed of:^ A ServiceProvider, enclosing the service logic.^ A ServiceRequester containing enough information to invoke the
service. 2 types of services:
^ Push service (can be invoked using a Universal Requester).^ Pull-and-Push service.
Interoperability
![Page 22: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/22.jpg)
RelateGateways: Dominique Guinard
22 / 35
« Plug and Play » Invocation
3: Load Mobile Code1: Discover Service
using Relate
Mobile Code
2: Get Mobile Code
4: Invoke Service
The semantics of Pull-and-Push Services is unknown to the mobile client before discovery.
Packets of Mobile Code (ServiceRequesters, descriptions, icon, etc.) are downloaded and dynamically loaded on the mobile device.
Interoperability
![Page 23: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/23.jpg)
RelateGateways: Dominique Guinard
23 / 35
Cross-Device Interactions
Using the computing power as a service.
Use-cases:^ Collaborative tasks^ Cross-device
interactions for single user.
Extending the EBL toolkit:^ Cooperation with UCL
(Université Catholique de Louvain la Neuve, HCI Lab).
Interoperability
![Page 24: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/24.jpg)
RelateGateways: Dominique Guinard
24 / 35
Today’s Menu…
Introduction, Aim E2ESD Model
^Spatial Discovery^Video^Network and Service Discovery^ Invocation and Interoperability
• Cross-device Interactions
Evaluation Conclusion
![Page 25: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/25.jpg)
RelateGateways: Dominique Guinard
25 / 35
Settings
Test run in Lancaster, formative user study in Munich.
Total of 20 users. 3 “service enabled”
devices within a large office.
Keyboard
Tablet PC
Handheld PCMobile PhoneUser
OR
OR
OR
Beamer and Display
Printer
Wireless/Wired Router
Evaluation
![Page 26: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/26.jpg)
RelateGateways: Dominique Guinard
26 / 35
Qualitative Results
Most cited benefits:1. No installation, no configuration: saves time.
2. Ease of interaction with the services: drag and drop.
3. Dynamic spatial arrangement of the gateways: making the UI more natural, especially useful in unknown places.
Suggested a number of UI improvements. And services to implement.
Evaluation
![Page 27: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/27.jpg)
RelateGateways: Dominique Guinard
27 / 35
Today’s Menu…
Introduction, Aim E2ESD Model
^Spatial Discovery^Video^Network and Service Discovery^ Invocation and Interoperability
• Cross-device Interactions
Evaluation Conclusion
![Page 28: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/28.jpg)
RelateGateways: Dominique Guinard
28 / 35Bootstraping Spontaneous Mobile Spatial Interactions
Implementation of the E2ESD bundled into a single, runnable application offering:^A spatial user interface « toolkit » (MVC based
+ contextual rules-engine).^A Network and Service Discovery system.^An Invocation and Interoperability system.^An architecture for prototyping pervasive
services.^Various simulation and debugging tools.
A framework (SOA) supporting the rapid prototyping of mobile spatial interactions.
Conclusion
![Page 29: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/29.jpg)
RelateGateways: Dominique Guinard
29 / 35
Quantitative Outputs
Dissemination^ 3 accepted workshop papers:
• MSI @ CHI 07, Permid @ Pervasive 07, SensorNet 07
^ 2 conference papers submited• Ubicomp 07 (Demo Paper), LoCA 07
Prototyping Framework:• ~16’000 lines of code.• ~160 classes (to consolidate!).• To few hours of sleep…
Conclusion
![Page 30: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/30.jpg)
RelateGateways: Dominique Guinard
30 / 35
Open Questions
Scalability of the application:^ In terms of user interface.^ In terms of prototyping framework.
User study is formative:^Need for a comparative study as well.^Need to evaluate the framework and its use for
the prototyping of mobile spatial applications.
Security concerns:^Spontaneity is nice but it leaves a number of
doors open for attackers.
Conclusion
![Page 31: An Architecture to Enable Spontaneous Mobile Spatial Interaction with Pervasive Services](https://reader034.vdocuments.us/reader034/viewer/2022051612/54c47eb44a7959876a8b45c9/html5/thumbnails/31.jpg)
RelateGateways: Dominique Guinard
31 / 35
Questions ?
Thanks for your attention…. Project’s homepage:
http://ubicomp.lancs.ac.uk/relategateways Contact: [email protected]
Conclusion