input (devices and models)€¦ · input • generally, input is somewhat harder than output –...
TRANSCRIPT
![Page 1: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/1.jpg)
Input (Devices and Models)
![Page 2: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/2.jpg)
2
Administration
• Assignment 3 (prelim) due today
• Assignment 4 set today
![Page 3: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/3.jpg)
3
Where we are...
• Two largest aspects of building interactive systems: output and input – Have looked at basics of output – Now look at input
![Page 4: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/4.jpg)
4
Input
• Generally, input is somewhat harder than output – Less uniformity, more of a moving
target – More affected by human properties – Not as mature
• Will start with low level (devices) and work up to higher level
![Page 5: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/5.jpg)
5
Input devices
• Keyboard – Ubiquitous, but somewhat
boring… – Quite mature design
• QWERTY key layout – Alternatives?
![Page 6: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/6.jpg)
6
Christopher Sholes 1868
QWERTY key layout
• Originally designed to spread out likely adjacent key presses to overcome jamming problem of very early mechanical typewriters – Often quoted as “intentionally
slowing down” typing, but that’s not true
![Page 7: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/7.jpg)
7
QWERTY keyboard layout
• Other layouts have been proposed – Dvorak is best known – Widely seen as better – Experimental and theoretical
evidence casts doubt on this • (Is only a little better) • Alternating hands of QWERTY are a win since fingers move in parallel
![Page 8: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/8.jpg)
• Whether or not Dvorak layout is better, it did not displace QWERTY
– Lesson: once there is sufficient critical mass for a standard it is nearly impossible to dislodge (even if there is an apparently good reason to)
8
Economic phenomenon of “lock-in” or “path dependence”
QWERTY keyboard layout
![Page 9: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/9.jpg)
9
Buttons
• Similar to keyboard, but not for typing letters – separate collection of keys with
typically same form but different purpose
– now seen as “function keys” that come standard w/ keyboards
– also show up on e.g., mouse
Ivan Sutherland, Sketchpad, 1963
![Page 10: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/10.jpg)
10
Buttons
• Buttons often bound to particular commands – e.g., function keys – Improved quite a bit with labels – Software changeable labels would
be ideal, but we don’t typically get this
![Page 11: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/11.jpg)
11
Valuators (e.g. Sliders)
• Returns a single value in range • Major implementation alternatives:
– Potentiometer (variable resistor) • Similar to typical
volume control – Shaft encoders
• Sense incremental movements
![Page 12: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/12.jpg)
12
Locators (AKA pointing devices)
• Returns a location (point) – two values in ranges – usually screen position
• Examples – Mice (current defacto standard) – Track balls, joysticks, tablets,
touch panels, etc.
![Page 13: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/13.jpg)
13
Locators
• Two major categories: – Absolute vs. Relative locators
![Page 14: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/14.jpg)
14
Absolute locators
• One-to-one mapping from device movement to input – e.g., tablet – Faster – Easier to develop motor skills – Doesn’t scale past fixed
distances • bounded input range
– less accurate (for same range of physical movement)
![Page 15: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/15.jpg)
15
Relative locators
• Relative or incremental mapping
• E.g., maps movement into rate of change of input – e.g., joystick
(or TrackPoint)
![Page 16: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/16.jpg)
16
Relative locators
– More accurate (for same range of movement)
– Harder to develop motor skills – Not bounded
(can handle infinite moves)
![Page 17: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/17.jpg)
17
Q: is a mouse a relative or absolute locator?
(Ignore mouse acceleration for a moment) Invented by Douglas Engelbart et al. ~1967
http://sloan.stanford.edu/MouseSite/Archive/AugmentingHumanIntellect62/Display1967.html
![Page 18: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/18.jpg)
18
Q: is a mouse a relative or absolute locator?
• Answer: Neither • Third major type:
“Clutched absolute” – Within a range it’s absolute – Can disengage movement (pick it
up) to extend beyond range • picking up == clutch mechanism
![Page 19: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/19.jpg)
19
Clutched absolute locators
• Very good compromise – Get one-to-one mapping when “in
range” (easy to learn, fast, etc.) – Clutch gives some of benefits of a
relative device (e.g., unbounded)
• Trackballs also fall into this category
![Page 20: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/20.jpg)
20
Mouse Acceleration
• Since mouse is unbounded we can play a clever trick
• Increase speed when mouse is moving fast – Middle of movement
• Normal when moving slow – Start and end of movement
• Interesting perceptual effect: people basically don’t notice this
Where we need
precision
![Page 21: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/21.jpg)
21
Device specifics: joysticks
– self centering – relative device – possible to have absolute
joysticks, but scaling is bad
![Page 22: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/22.jpg)
22
Trackballs
• (Typically large) ball which rolls over 2 wheels
![Page 23: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/23.jpg)
23
Trackballs
• Clutched absolute – but with small movement range
• Infinite input range, etc. • Properties vary quite a bit
– scaling of movements – mass of ball
![Page 24: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/24.jpg)
24
Touch panel
• What kind of a device?
![Page 25: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/25.jpg)
25
Touch panel / Tablet
• Absolute device • Possible to do input and output
together in one place – actually point at things on the screen
• Supports real drawing • Resolution limited by size of
finger (“digital input”)
![Page 26: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/26.jpg)
26
3D locators
• Can extend locators to 3 inputs • Some fun older devices
– 3D acoustic tablet – Wand on reels – Multi-axis joystick
![Page 27: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/27.jpg)
27
Lots of other devices
• Still mostly KB + mouse, but increasing diversity – Cameras!
• Lots of untapped potential in vision
– Microphones • speech as data • speech recognition
![Page 28: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/28.jpg)
28
Lots of other devices
• Any favorites?
![Page 29: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/29.jpg)
29
Some interesting ones
• Thumb Wheel • DataGlove • Motion detectors (and other sensors)
• Accelerometers
• Identification techniques – Magic apples
![Page 30: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/30.jpg)
30
Using identification as input
• If you have object identification and a network, you can appear to attach arbitrary amounts of information to an object with just identification – Use ID to store retrieve data in DB
• 64 bit ID will be unique • 96 or 128 bits better (don’t need more)
• Make assoc in place A, carry to place B, identify, get data over network ⇒ data appears to have moved w/ obj
![Page 31: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/31.jpg)
31
Specific identification technologies
• RFID tags • Bar codes • Fingerprint readers • Sub-gram resolution scales • Speech • Handwriting • Walking
![Page 32: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/32.jpg)
32
Sun Microsystems Starfire
• 1994-5, Vision of 2004 • Many different interaction styles • CHI’94 paper, video prototyping • Book
• Apologies for stereotypes
![Page 33: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/33.jpg)
33
Starfire Video
• http://www.idemployee.id.tue.nl/g.w.m.rauterberg/videos.html
![Page 34: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/34.jpg)
34
Prototyping These Visions
• Styles of input/output? • Differences/similarities with
1987 Knowledge Navigator
![Page 35: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/35.jpg)
• Break – 15 minutes
35
![Page 36: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/36.jpg)
Input (Part 2: Input Models)
![Page 37: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/37.jpg)
37
Dealing with diversity
• Saw lots of diversity in devices
– actual details of devices (e.g., device drivers) is a real pain
– how do we deal with the diversity?
• Need a model (abstraction) for input – like file systems abstract disks – higher level & device independent
![Page 38: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/38.jpg)
38
Input Models are Complex
• "One of the most complex aspects of Xlib programming is designing the event loop, which must take into account all of the possible events that can occur in a window.”
-- Nye & O'Reilly X Toolkit Intrinsics Programming Manual, vol. 4, 1990, p. 241.
• "The dispatching and handling of events is rather complicated.”
-- Galaxy Reference Manual, v1.2, p. 20-5.
![Page 39: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/39.jpg)
39
Logical device approach
• One approach: “logical devices” – A logical device is characterized
by its software interface (only) • the set of values it returns
– Rest of semantics (how it operates) fixed by category of device or left to the particular device
![Page 40: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/40.jpg)
40
Logical device approach
• Fixed set of categories – old “Core Graphics” standard had 6
• keyboard, locator, valuator, button • pick, stroke
• If actual device is missing, device is simulated in software – valuator => simulated slider – 3D locator => 3 knobs
• 1st step towards today’s widgets
![Page 41: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/41.jpg)
41
Logical device approach
• Abstraction provided by logical device model is good
• But… abstracts away too many details (some are important) – example:
mouse vs. pen on palm pilot • Both are locators • What’s the big difference?
![Page 42: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/42.jpg)
42
Not a success but..
• Still useful to think in terms of “what information is returned”
• Categorization of devices useful – Two broad classes emerged
• Event devices • Sampled devices
![Page 43: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/43.jpg)
43
Categorization of devices
• Event devices – Time of input is determined
by the user • Best example: button • When activated, creates an “event record” (record of significant action)
![Page 44: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/44.jpg)
44
Categorization of devices
• Sampled devices – Time of input is determined
by the program • Best example: valuator or locator • Value is constantly updated
– Might best think of as continuous
• Program asks for current value when it needs it
![Page 45: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/45.jpg)
45
A unified model
• Anybody see a way to do both major types of devices in one model?
![Page 46: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/46.jpg)
46
A unified model: the event model
• Model everything as events – Sampled devices are handled with
“incremental change” events – Each measurable change in value
produces an event containing the new value
– Program can keep track of the current value if it wants to sample
![Page 47: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/47.jpg)
47
Simulating sampling under the event model of input
• Can cause problems – lots of little events
• Can fall behind if doing a lot of computation/redraw for every event
– machines are fast, blah blah blah – but can get behind (sampling provided
built in throttling)
![Page 48: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/48.jpg)
48
The event input model
• Almost all systems now use this
• An “event” is an indication that “something potentially significant” has just happened – in our case user action on input device – but, can be generalized
![Page 49: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/49.jpg)
49
The event input model
• “Event records” are data structures that record relevant facts about an event – generally just called “events”
• Event records often passed to an “event handler” routine – sometimes (e.g., Flex) just encode
relevant facts in parameters instead of event record
![Page 50: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/50.jpg)
50
Relevant facts
• What do we need to know about each event?
![Page 51: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/51.jpg)
51
Relevant facts
• What • Where • When • Value • Additional Context
![Page 52: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/52.jpg)
52
What
• What (exactly) caused the event – e.g., left mouse button went down – for “method based” systems this
may be implicit in what handler gets called
![Page 53: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/53.jpg)
53
X-Windows defines 33 types of events:
1. buttonPress 2. buttonRelease 3. keyPress 4. keyRelease 5. motionNotify 6. enterNotify 7. leaveNotify 8. focusIn 9. focusOut 10. keymapNotify (change keymap) 11. expose 12. graphicsExpose 13. noExpose 14. colormapNotify 15. propertyNotify 16. visibilityNotify (become covered) 17. resizeRequest
18. circulateNotify 19. configureNotify 20. destroyNotify 21. gravityNotify 22. mapNotify (became visible) 23. createNotify 24. reparentNotify 25. unmapNotify (invisible) 26. circulateRequest 27. configureRequest 28. mapRequest 29. mappingNotify (kbd
mapping) 30. clientMessage 31. selectionClear (cut & paste) 32. selectionNotify 33. selectionRequest
![Page 54: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/54.jpg)
54
Where
• Where was the primary locator (mouse) when event happened – x,y position – also, inside what window, object,
etc. – this is specific to GUIs, but its
critical • e.g., can’t tell what mouse button down means without this
![Page 55: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/55.jpg)
55
When
• When did the event occur – Typically are dealing with events
from the (hopefully recent) past • queued until program can get to them
– In absolute time or relative to some start point
– Hopefully at resolution of 10s of ms • important for e.g., double-clicks
![Page 56: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/56.jpg)
56
Value
• Input value – e.g., ASCII value of key press – e.g., value of valuator – some inputs don’t have a value
• e.g. button press
![Page 57: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/57.jpg)
57
Additional context
• Status of important buttons – shift, control, and other modifiers – possibly the mouse buttons
![Page 58: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/58.jpg)
58
Extending the event model
• Events can extend past simple user inputs – Extra processing of raw events to get
“higher level” events • window / object enter & exit
– Can extend to other “things of significance” • arrival of network traffic • Low battery
• Generally event is a notification of the occurrence of a significant event and its convenient to use that abstraction
![Page 59: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/59.jpg)
59
Extending the event model
• Window systems typically introduce a number of events – window enter/exit region enter/
exit • system tracks mouse internally so code acts only at significant points
– Redraw / damage events – Resize & window move events
![Page 60: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/60.jpg)
60
Synchronization and events
• The user and the system inherently operate in parallel – asynchronously
• This is a producer consumer problem – user produces events – system consumes them
![Page 61: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/61.jpg)
61
Synchronization and events
• Need to deal with asynchrony – both parties need to operate
when they can – but can’t apply concurrency
control techniques to people
• How do we handle this?
![Page 62: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/62.jpg)
62
Synchronization and events
• Use a queue (buffer) between
– As long as buffer doesn’t overflow, producer does not need to block
– Consumer operates on events when it can
Producer Buffer Consumer
![Page 63: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/63.jpg)
63
Implications of queued events
• We are really operating on events from the past – hopefully the recent past
• But sampled input is from the present – mixing them can cause problems – e.g. inaccurate position at end of
drag
![Page 64: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/64.jpg)
64
Using events from an event queue • Two big questions:
– What object(s) gets the event? – What does it do with it?
• Interpret it based on what the event is, what the object is, and what state the object is in
![Page 65: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/65.jpg)
65
Two major ways to dispatch events • Positional dispatch
– Event goes to an object based on position of the event
• Focus-based dispatch – Event goes to a designated object
(the current focus) no matter where the mouse is pointing
![Page 66: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/66.jpg)
66
Question
• Would mouse move events be done by focus or positional dispatch?
![Page 67: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/67.jpg)
67
Question & answer
• Would mouse move events be done by focus or positional dispatch?
• It depends… – painting: use positional – dragging an object: need focus
(why?)
![Page 68: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/68.jpg)
68
Dragging an object needs focus dispatch
• Why? What if we have a big jump?
• Cursor now outside the object and it doesn’t get the next event!
Object
Previous mouse position
New mouse position
![Page 69: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/69.jpg)
69
Positional and focus based dispatch
• Will need both • Will need a flexible way to
decide which one is right – sometimes we need one,
sometimes another
![Page 70: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/70.jpg)
70
Positional dispatch
• If we are dispatching positionally, need a way to tell what object(s) are “under” a location
• “Picking”
![Page 71: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/71.jpg)
71
Pick ambiguity
• Classic problem, what if multiple things picked? – Two types – Hierarchical ambiguity
• are we picking the door knob, the door, the house, or the neighborhood?
![Page 72: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/72.jpg)
72
Pick ambiguity
– Spatial ambiguity • Which door are we picking?
![Page 73: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/73.jpg)
73
Solutions for pick ambiguity
• No “silver bullet”, but two possible solutions – “Strong typing” (use dialog state)
• Turn off “pickability” for unacceptable objects
» reject pick during traversal
![Page 74: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/74.jpg)
74
Solutions for pick ambiguity
– Get the user involved • direct choice
– typically slow and tedious • pick one, but let the user reject it and/or easily back out of it
– often better – feedback is critical – Need a way to get at the others
![Page 75: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/75.jpg)
75
Input Summary
• Lots of variety in input devices • Event model is good abstraction • Issues
– How to support user asynchrony – Who is each event dispatched to – What is done with the event
![Page 76: Input (Devices and Models)€¦ · Input • Generally, input is somewhat harder than output – Less uniformity, more of a moving target – More affected by human properties –](https://reader033.vdocuments.us/reader033/viewer/2022060411/5f10858e7e708231d44985a9/html5/thumbnails/76.jpg)
76
Questions?