multi touch and gesture event interface and types
Post on 02-Dec-2014
7.936 Views
Preview:
DESCRIPTION
TRANSCRIPT
Multi Touch & Gesture - Event Interface and Types
June, 2010 TC Meeting in Minneapolis, MN
Eunil SEO / seoei2@tobesoft.com Siman KIM / smkim@tobesoft.com
Sangchual CHA / chasc@tobesoft.com
1
Agenda
• UI Event - Mouse Event in W3C
• Multi Touch and Gesture
• Event Issues
• W3C DOM Event Model Extension
2
Purpose of this presentation
• To collect current technical status on multi-touch and gesture event.
• To reveal issues about multi-touch and gesture event.
• To suggest multi-touch and gesture event model with architecture, interface and types.
3
Event Model in W3C
• Document Object Model (DOM) Level 3 Events Specification (Sep. 2009)
http://www.w3.org/TR/DOM-Level-3-Events/
• Event Architecture• Event Interface• Event Module
– Event Types– Event Module Definition
4
DOM Lvl3 Event Architecture
5
• Event Dispatch Mechanism• Event Flow
• Capture Phase• Bubble Phase• Target Phase
DOM Lvl3 Event Interface
6
http://www.w3.org/TR/DOM-Level-3-Events/#dom-event-architecture
DOM Lvl3 MouseEvent Interface
interface MouseEvent : UIEvent { readonly attribute long screenX; readonly attribute long screenY; readonly attribute long clientX; readonly attribute long clientY; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; … …};
7
DOM Lvl3 MouseEvent Interface Cont’d
interface MouseEvent : UIEvent { … … void initMouseEvent(in DOMString typeArg, in boolean canBubbleArg, in boolean cancelableArg, in views::AbstractView viewArg, in long detailArg, in long screenXArg, in long screenYArg, in long clientXArg, in long clientYArg, in boolean ctrlKeyArg, in boolean altKeyArg, in boolean shiftKeyArg, in boolean metaKeyArg, in unsigned short buttonArg, in EventTarget relatedTargetArg); };
8
DOM Lvl3 Event Types
9
Event Iterfaces• MouseEvent
• TextEvent
• KeyboardEvent
• CompositionEvent
• MutationName Event
• MouseWheelEvent
• WheelEvent
Event Iterfaces• MouseEvent
• TextEvent
• KeyboardEvent
• CompositionEvent
• MutationName Event
• MouseWheelEvent
• WheelEvent
Mouse Event Types• click event
• dblclick event
• mousedown event
• mouseenter event
• mouseleave event
• mousemove event
• mouseover event
• mouseout event
• mouseup event
• mousewheel Event
• wheel Event
Mouse Event Types• click event
• dblclick event
• mousedown event
• mouseenter event
• mouseleave event
• mousemove event
• mouseover event
• mouseout event
• mouseup event
• mousewheel Event
• wheel Event
RIA Event Spectrum
10
Device
O/S
RIA Platform
Application
Event Interpreter
Device Event
Device Event
Device Event
Event Interpreter
Event Interpreter
Interpreted Event
Interpreted Event
* Physical device with device driver
* With Application Framework
Device
* RIA Platform includes Application FrameworkRIA Platform
RIA Event Spectrum
• Flow through Device, O/S, RIA Platform, RIA Application Framework and RIA Application.
• Device Event– Events origin from device.– Keep its context through spectrum.
• Interpreted Event– Combination of device events and interpreted events.– Represent with State Diagram.
11
New Event Issues
• Single Touch - Click vs. Tap– Click with stylus vs. Tap with Fingers – Interval vs. Duration – Point(x, y) vs. Region(x, y, width, height)
• Array of Touch Evet– Sequence of touch vs. Set of touch
• Gesture – How to understand the intension of action.– Gesture in Space (Gesture with Glove, Camera, etc)
• Shake and rotate (Gyro Sensor)12
Multi-Touch Event
• Multi-touch is an enhancement to touch-screen technology, which provides the user with the ability to apply multiple finger gestures simultaneously onto the electronic visual display to send complex commands to the device. (Wikipedia)
• Multi Touch = Sequence of Touch (Down, Up, Move)
13
Gesture Event
• Any physical movement that can be sensed and responded to by a digital system without the aid of a traditional input device such as a mouse or stylus.
• To understand user intensions by recognizing multi-touch events, from the first finger down event to the last finger up event.
• Multi-touch gesture, Mouse gesture, Motion gesture, etc
14
Sample Gestures in Touch Device
15
Tap
Two finger zoom out Two finger zoom in
Double Tap
Touch Down
Touch Up
One finger drag
Pinch zoom
Two finger tap
One finger swipe Two finger swipe
Two finger double tap
Two finger horizontal scroll Two finger
vertical scroll
Gestureworks Open Souce Gesture Library (http://gestureworks.com/about/open-source-gesture-library/
)
Gestures in iPhone O/S
• Gesture Types– Tapping (any number of tabs)– Pinching in and out (for zooming a view)– Panning or dragging– Swing (in any direction)– Rotation (fingers moving in opposite
directions)– Long Press (aks touch and hold)
16
http://developer.apple.com/iphone/library/documentation/General/Conceptual/iPadProgrammingGuide/GestureSupport/GestureSupport.html
Gesture Recognizers
http://developer.apple.com/iphone/library/documentation/General/Conceptual/Devpedia-CocoaApp/EventHandlingiPhone.html Events (iPhone OS)
Gestures in iPhone O/S cont’d
17
• Device (O/S) detect touch event• Interpret Touch Event in Cocoa
Framework• UITouch Event Class
Gestures in iPhone O/S cont’d
18
Gestures in iPhone O/S cont’d
19
• touchesBegan:withEvent: is called for touch objects in the Began phase.• touchesMoved:withEvent: is called for touch objects in the Moved phase.• touchesEnded:withEvent: is called for touch objects in the Ended phase.• touchesCancelled:withEvent: is called when some external event (e.g. Incomming
Call).
Gestures in Android
• Event Listeners– OnClickListner– OnLongClickListner– OnTouchListener
• Touch Event– public abstract boolean onTouch (View v, MotionEvent event)
• TouchDown• TouchMove• TouchUp• TouchTap
Gestures in Android cont’d
• OnGestureListeners– OnGesture– OnOnGestureCancelled– OnGestureEnded– OnGestureStarted
OnGestureStartedOnGestureStarted
OnGestureCancelledOnGestureCancelled
ReadyReady
OnGestureEndedOnGestureEnded
OnGestureOnGesture
OnTouchDownOnTouchDownOnTouchUpOnTouchTap
OnTouchUpOnTouchTap
Cancelling EventEx) CallingEvent
Cancelling EventEx) CallingEvent
OnTouchDownOnTouchUp
OnTouchMoveOnTouchTap
OnTouchDownOnTouchUp
OnTouchMoveOnTouchTap
Gestures in Android cont’d
• GestureDetector (SimpleGestureDetector)– onDoubleTap– onDown– onFling– onLongPress– onScroll– onShowPress– onSingTapUp– onGestureRotate– onGestureScroll– onGestureScale
Reference : How to use Multi-touch in Android 2 in ZDNET
Gestures in MS Windows 7• System gesture Synthesized equivalent
message• Hover (when supported) >> Mouse hover• Tap (down and up) >> Mouse left-click• Double tap (down and up twice)>> Mouse double left-
click• Press and hold (down, pause, up)>>Mouse right-click
• Drag (down, move, up) >> Mouse left-drag• Press, hold, and drag (down, pause, move, up) >>
Mouse right-drag• Select (down, move over selectable objects, up)
>> Mouse select
23
MS Windows - Panning
• Entry state: One or two fingers in contact with the screen.
• Motion: Drag, with any additional fingers remaining in same position relative to each other.
• Exit state: Last finger up ends the gesture.• Effect: Move the underlying object directly
and immediately as the fingers move. Be sure to keep the contact point under the finger throughout the gesture.
24
MS Windows - Zoom
• Entry state: Two fingers in contact with the screen at the same time.
• Motion: Fingers move apart or together (pinch) along an axis.
• Exit state: Any finger up ends the gesture or the fingers break the axis.
• Effect: Zoom the underlying object in or out directly and immediately as the fingers separate or approach on the axis. Be sure to keep the contact points under the finger throughout the gesture.
25
MS Windows - Rotate
• Entry state: Two fingers in contact with the screen at the same time.
• Motion: One or both fingers rotate around the other, moving perpendicular to the line between them.
• Exit state: Any finger up ends the gesture. • Effect: Rotate the underlying object the
same amount as the fingers have rotated. Be sure to keep the contact points under the finger throughout the gesture.
26
MS Windows - Two-finger tap
• Entry state: Two fingers in contact with the screen at the same time.
• Motion: No motion. • Exit state: Any finger up ends the
gesture. • Effect: Alternatively zooms or restores
the default view for the object between the fingers.
27
MS Windows - Press and tap
• Entry state: One finger in contact with the screen, followed by a second finger.
• Motion: No motion. • Exit state: Second finger up ends the
gesture. • Effect: Performs a right click for the
object under the first finger
28
Related Issues
• Lack of standard : Different understanding in user actions.
• Event Sharing between components : Inter-component event dispatch. (e.g. Drag & Drop)
• We need RIA standard gesture event model.
• W3C DOM Level 3 Event Model doesn’t support Gesture event. Browser War !!
29
Reference
• UXGuide - http://msdn.microsoft.com/en-us/library/aa511258.aspx
30
top related