a generic framework for executable gestural interaction models
Post on 25-Dec-2014
206 Views
Preview:
DESCRIPTION
TRANSCRIPT
Behaviour and interaction modeling of 3D virtual objects
A Generic Framework for Executable
Gestural Interaction Models
Romuald Deshayes, Tom Mens, Philippe Palanque
Software Engineering Lab IRIT
2012-2013
Deshayes Romuald – UMONS 1 / 15
Behaviour and interaction modeling of 3D virtual objects
Table of Contents
1 IntroductionContextPaper Goal
2 The FrameworkModular and Generic FrameworkLayers and Gestures
3 Conclusion
4 Future Work
Deshayes Romuald – UMONS 2 / 15
Behaviour and interaction modeling of 3D virtual objects
Introduction
Context
Context
Different objects with different ways of interacting with them
Enhance computer with better insight in user interaction withreal and virtual objects
[Target Domain] Virtual and Augmented Reality applications suchas games, home automation and simulation
Deshayes Romuald – UMONS 3 / 15
Behaviour and interaction modeling of 3D virtual objects
Introduction
Context
Context
ContextDevelopment of Human-Computer Interaction (HCI) applicationsinvolving gestural interaction
High technical complexity (low level aspects)Low learning curve
Deshayes Romuald – UMONS 4 / 15
Behaviour and interaction modeling of 3D virtual objects
Introduction
Paper Goal
Paper Goal
GoalProvide a generic and efficient way of describing and executing softwareinvolving gestural interaction with real or virtual objects
Based on executable models
Domain Specific Modeling Language (DSML)
A DSML simplifies the creation of gesture based application
Generic framework for gestural interaction
Deshayes Romuald – UMONS 5 / 15
Behaviour and interaction modeling of 3D virtual objects
Introduction
Paper Goal
Why Modeling
Why is executable modeling appropriate for developing HCIapplications ?
Visual aspects allow to hide low level details by raising thelevel of abstraction (simplicity)Formal analysis can help reduce the amount of bugs incomplex systems (security)Behavior can be changed at runtime by editing the runningmodels (productivity)
Challengecounter critique from [SKF+94] that model-based approaches"model the previous generation of UI"
Deshayes Romuald – UMONS 6 / 15
Behaviour and interaction modeling of 3D virtual objects
Introduction
Paper Goal
Why Modeling
Why is executable modeling appropriate for developing HCIapplications ?
Visual aspects allow to hide low level details by raising thelevel of abstraction (simplicity)Formal analysis can help reduce the amount of bugs incomplex systems (security)Behavior can be changed at runtime by editing the runningmodels (productivity)
Challengecounter critique from [SKF+94] that model-based approaches"model the previous generation of UI"
Deshayes Romuald – UMONS 6 / 15
Behaviour and interaction modeling of 3D virtual objects
The Framework
Modular and Generic Framework
The framework
GoalProvide a generic and efficient way of describing and executingsoftware involving gestural interaction with real or virtual objects
How ?By creating a modular and generic framework that allows thespecification of executable models for manipulating objects withgestures
I/O independenceClient/server approach for modularityLayered architecture
Deshayes Romuald – UMONS 7 / 15
Behaviour and interaction modeling of 3D virtual objects
The Framework
Modular and Generic Framework
A Layered Architecture
A layered architectureEach layer is modeled as an ICO (formalism based onHigh-Level Petri nets)Layers communicate through events
Deshayes Romuald – UMONS 8 / 15
Behaviour and interaction modeling of 3D virtual objects
The Framework
Layers and Gestures
The Layers and the Gestures
The layersL1 detects new users, receives raw data and converts them topositions w.r.t head positionL2 create low-level gestures and detect hand state changes
MoveOpenClose
L3 combine low-level gestures with users’ state to createhigh-level gestures
SimpleDragColinearDragNonColinearDrag
Deshayes Romuald – UMONS 9 / 15
Behaviour and interaction modeling of 3D virtual objects
The Framework
Layers and Gestures
Modeling the state of the users
Layer 3 : modeling the state of the users
Deshayes Romuald – UMONS 10 / 15
Behaviour and interaction modeling of 3D virtual objects
The Framework
Layers and Gestures
Modeling the interaction with virtual objects
Layer 4 contains a differentmodel for each objectUsed to combine all existinggestures into a modeldescribing how to interactwith an object
Example : a book
Deshayes Romuald – UMONS 11 / 15
Behaviour and interaction modeling of 3D virtual objects
Conclusion
Conclusion
Advantages of resorting to executable modeling for HCI applicationdevelopment
Reduced technical complexity thanks to higher level ofabstractionDynamic execution particularly suitable for rapid prototypingof interactive applicationsCommunicating models allow to concurrently interact withmultiple objects and enable separation of concernsOur framework allows to easily specify gestural interactionwith real or virtual objects
Deshayes Romuald – UMONS 12 / 15
Behaviour and interaction modeling of 3D virtual objects
Future Work
Future Work
Current research and future workExplore heterogeneity in the layers to use the most appropriateformalism (MultiPM vs MonoPM)Develop a DSML for Layer 4Perform a controlled user study with CS students to evaluatethe productivity gain of using the framework and its usability.Code generation
Deshayes Romuald – UMONS 13 / 15
Behaviour and interaction modeling of 3D virtual objects
Future Work
Thank you
Questions ?
Deshayes Romuald – UMONS 14 / 15
Behaviour and interaction modeling of 3D virtual objects
Future Work
Bibliography I
P. N. Sukaviriya, S. Kovacevic, J. D. Foley, B. A. Myers, D. R.Olsen Jr., and M. Schneider-Hufschmidt.Model- based user interfaces: What are they and why shouldwe care?In Proc. UIST94, pages 133–135. ACM, November 1994.
Deshayes Romuald – UMONS 15 / 15
top related