![Page 1: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/1.jpg)
14. Juli 2009
LFE Medieninformatik •Diploma Thesis – intermediate report
Wolfgang Reithmeier
Complex Gestures for Mobile Interaction with Dynamic NFC-Displays
Betreuer: Dipl.Medieninf. Gregor Broll (LMU/DOCOMO)Dr. Matthias Wagner (DOCOMO)
Hochschullehrer: Prof. Hußmann
![Page 2: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/2.jpg)
Wolfgang Reithmeier14. Juli 2009 2
Motivation
• Dynamic NFC displays– Grid of ordered NFC tags– Projected dynamic display
• Touch & Interact: Applied to a Tourist Guide Prototype [Hardy et al., 2008] => winner at NFC Forum Global Competition
• Work on dynamic NFC-display in the context of the MULTITAG-project (DOCOMO, Lancaster University)
(Hardy et al., 2008)
![Page 3: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/3.jpg)
Wolfgang Reithmeier14. Juli 2009 3
Task Description
• Further development of the project thesis (NFC Display Framework)
• Partial integration with the MULTITAG-framework– For example Communication / RPC Module
• Focus on interaction modalities, gestures– Taking existing interaction modalities
as reference– Selection and adaption for dynamic
NFC displays– Chosen reference Scenario: Pinboard
![Page 4: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/4.jpg)
Wolfgang Reithmeier14. Juli 2009 4
Overview
• Related Work
• Analysis & Requirements
• Design of Gestures
• Status and Outlook
![Page 5: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/5.jpg)
Wolfgang Reithmeier14. Juli 2009 5
Selected Related Work
(Wilson, 2004)
Public Display Interaction• TouchLight [Wilson, 2004]
Gesture Techniques• Presense [Rekimoto. 2003]• HoverWidgets [Grossmann, Baudisch. 2006]
Physical Mobile Interaction• Hovering [Välkkynen. 2006]• Marked-up Maps [Reilly. 2004 / 05]
(Rekimoto. 2003)
(Välkkynen. 2006)
![Page 6: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/6.jpg)
Wolfgang Reithmeier14. Juli 2009 6
Analysis & Requirements 1/2Desktop Interaction Modalities (comp. WIMP)• Simple input gestures
– (Double-) Click, Right-Click / Context Menu
• Advance input gestures– Multi Select / Clear, Resize, Drag & Drop, Scroll (-wheel)
Mouse Gestures (ála Mozilla Firefox)
Touch & Pen based Interaction• Some special enhancements / adaptions
– Drag & Drop alternatives (e.g. Pick-and-Drop [Rekimoto, 97],… )
– Tap & Hold (e.g. used in Windows Mobile)
– Pen Gesture (characters, strokes, …)
– Mode switching techniques [Li et al., 2005]
(http://optimoz.mozdev.org/gestures/, 2009)
(© Wikipedia, 2009)
(Rekimoto, 97)
![Page 7: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/7.jpg)
Wolfgang Reithmeier14. Juli 2009 7
Analysis & Requirements 2/2
NFC touch based interaction[Reilly 2005, Vetter 2006, Hardy, Rukzio 2008/09]
• Simple input modalities– Touch Select / Hovering
– Click-Select
• Combined input modalities– Path-Select, Pick&Drop
– Multi-Select / Multi-Selection, Remote Clear
– Lasso-Select / Polygon-Select
– Bounding-Box-Select
![Page 8: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/8.jpg)
Wolfgang Reithmeier14. Juli 2009 8
Design of Gestures 1/6
The Pinboard Scenario
• Provides a rich environment fordifferent modalities
• Scenario Already made positiveexperiences due lecture exercises
• Expected functionalities:
– Create / Read / Edit / Delete Notes
– Place / move / transfer Message
– Scroll / Browse through the notes
– …
![Page 9: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/9.jpg)
Wolfgang Reithmeier14. Juli 2009 9
Design of Gestures 2/6
• Research Questions– Which interaction modality fits best in which program functionality?
– Which design / implementation is mostly accepted by the user?
– Are “mouse” gestures useful and where can they be applied?
• Attempt to formalize basic actions– touch_action := ((touch duration* release) interval*– key_action := (key_down duration* key_up) interval*– other_action := (tilt/rotation/”bumb”... duration*) interval*– basic_action := touch_action | key_action | other_action– action := basic_action+
![Page 10: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/10.jpg)
Wolfgang Reithmeier14. Juli 2009 10
Design of Gestures 3/6Interaction Modalities
Click / Double Click
• Double Click more explicitsingle click
• For example– View Message (Click)
– Transfer Message (Double Click)
• Sample Implementations:
(Double-) Touch/Click-Select, Press&Hold
![Page 11: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/11.jpg)
Wolfgang Reithmeier14. Juli 2009 11
Design of Gestures 4/6Interaction Modalities
Single- / Multiple Selection
• Preliminary grouping action for:– Transfer / remove (multiple) massage(s)
– Moving note(s) around
• Sample Implementations:
Via Multi-Select / Path-Select / Polygon-Select, …
![Page 12: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/12.jpg)
Wolfgang Reithmeier14. Juli 2009 12
Design of Gestures 5/6Interaction Modalities
Drag & Drop
• Moving / Placing notes
• Sample Implementations:
Pick&Drop, Path-Select,Multi-Touch-Select, ..
![Page 13: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/13.jpg)
Wolfgang Reithmeier14. Juli 2009 13
Design of Gestures 6/6Interaction Modalities
Right-Click / Context Menu• For example open Popup-Menu
• Sample Implementations:Press&Hold, Double-Click,alt. Click-Select
Cancel• Depends on actual gesture• Mostly via special mobile key
![Page 14: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/14.jpg)
Wolfgang Reithmeier14. Juli 2009 14
Status and OutlookDone• Analysis of existing interaction modalities• Integration of the frameworks• Design and implementation of the gesture rule set
In Progress• Implementation of the pinboard application• Planning a user study to evaluate different designs for interaction modalities and
gestures based on the pinboard application
Todo• Completion of the pinboard application• Conducting the user study• Evaluation of results, writing the thesis
![Page 15: Complex Gestures for Mobile Interaction with …...14. Juli 2009 3 Wolfgang Reithmeier Task Description • Further development of the project thesis (NFC Display Framework) • Partial](https://reader035.vdocuments.us/reader035/viewer/2022071014/5fcc3ac8dfb6e257c85248f8/html5/thumbnails/15.jpg)
Wolfgang Reithmeier14. Juli 2009 15
Thanks For Listening !
Questions ?!
NEWS
LINK