1 ken hinckley microsoft research collaborators: mike sinclair, erik hanson, rick szeliski, matt...
Post on 20-Dec-2015
217 views
TRANSCRIPT
1
Ken HinckleyMicrosoft ResearchCollaborators: Mike Sinclair, Erik Hanson, Rick
Szeliski, Matt Conway
Disclaimer: • The devices demonstrated are research prototypes only and do not
represent commitments by Microsoft to future products.• Opinions are Ken’s only and may not reflect the views of Microsoft, or
anyone else for that matter
The Future of the Mouse: An Illustrated Tour of Commercial & Research Mouse-like Input Devices
2
The First Mouse
• 1968• Douglas Englebart & colleagues• Stanford Research Institute• Separate rollers for x, y
• Why has it endured for so long?
3
• Approx. as good as pointing with hand itself (Card)
• from Balakrishnan et al. CHI ’97:• Flexibility of grip – both power & precision
grasp• Stability – damps tremor, little overhead to
acquire • vs. stylus; slightly better for high-precision
tasks• Relative mode: limits footprint, no nulling
problem• Buttons integrated & orthogonal to sensed
plane• Dragging works well, no motion when you
click• Familiar, entrenched device; +10% not
enough
Some Desirable Properties of the Mouse
4
Illustrated Tour: Some Axes of Innovation in Computer Mice
• Strictly mouse-like form factors • other devices (tablet/pen, trackball, touchpad,
joystick) have advantageous properties, but not my focus today
• What has been explored for desktop mice?• What strategies are there to improve user
performance?• What has succeeded in the marketplace?• What are some interesting areas for the future?• What are some really silly ideas?
5
• Scrolling: save time to grab scrollbar; maintain visual focus on primary task• Wheel: MS IntelliMouse, Mouse Systems
• Isometric Joystick: IBM ScrollPoint I & II
• Touchpad: Fujitsu ScrollPad Mouse
• Better w/ left hand? (Buxton & Myers)• …
• PadMouse (Balakrishnan)• Gesture-based
command selection
Multi-Channel Inputfor Secondary Tasks
6
New Sensor & Connection Technologies
GyroPoint Microsoft IntelliEye /IntelliMouse Explorer
Wacom Intuos Pen ToolID
• Motion sensing via gyros, CCD camera• Cordless, unique ID • USB, higher DPI, sampling rate, …
Logitech Cordless MouseMan Wheel
7
Better Support for the Human Senses
Logitech WingMan Force Feedback
Tractile (Cambell, Zhai et. al)
Tactile nib in mouse (MacKenzie)
• Force, Tactile feedback• 10-20% faster pointing speed?• But, errors, multiple targets,
dependent on visual feedback• Imposes constraints on device (esp. force)
8
Better Support for the Human Senses
• Force, Tactile feedback• Visual feedback on device?
• Indicators / displays• (Static Labels, Industrial Design, …)
• Audio output on mouse?• “clicking” named after the sound it makes
9
Better Support for the Human Senses
• Force, Tactile feedback• Visual feedback on device• Audio output on mouse?• Smell: powerful cue for memory
• Sensorama “VR” experience
10
Better Support for the Human Senses
• Force, Tactile feedback• Visual feedback on device• Audio output on mouse?• Smell: powerful cue for memory
• Sensorama “VR” experience Intelli Smelli TM
11
Better Support for the Human Senses
• Force, Tactile feedback• Visual feedback on device• Audio output on mouse?• Smell: powerful cue for memory• Taste: Edible Mice?
• Candy Mouse (Amurol Confections Company)
12
Additional Sensed Degrees-of-Freedom
• Rotation• Two-ball mouse, MacKenzie• 4D Mouse, Wacom tablet (A|W)
• Tilting (4DOF)• Rockin’Mouse,
Balakrishnan
• Hand Proximity in X, Y, Z• FieldMouse, (Josh Smith, MIT)
13
• 2D camera mice already here!
• Expand software capabilities?
• Mouse-like 6dof device atconsumer price points?
• Built proto using grid• Gridless design feasible?
VideoMouse
IntelliMouse Explorer
14
Lightweight Scanning
• Camera has small fov, but useful for quick scanning tasks
• e.g. scan paper title, bring up electronic document
• Vision algorithm stitches together video images
• Camera senses if grid is present Scanning mode
• Registration mark on mouse, line up w/ paper doc
• OCR not impl. – typically assumes full page of text
15
Biometric Sensing
• Fingerprint scan for user ID / login• uMatch biolink mouse
• biolinkusa.com
• GSR, heart rate, temperature, …• EmotionMouse (IBM BlueEyes project)
• Pressure sensing• Tablet stylus for art software• Area-of-contact in touchpads• Mice: not as well-defined
point(s) of contact
16
• Sense hand contact via capacitance • New events: Touch, Release
• For Palm area • for button, wheel, thumb, …
• Goal: use touch as passive input modality for awareness
• Emulation from software...• Did user release mouse? Or just holding still?• Cannot emulate multiple sensors at all.• Cannot emulate on most non-mouse devices.
Touch-Sensing Devices
17
Integration with Keyboard
• IBM Trackpoint• Reduces device acquisition times• Tiny footprint• Less intuitive cursor control• Dragging problematic
• Other pointing devices in kbd• touchpad, trackball• e.g. Acer Ergo61 Ergonomic Keyboard
18
• Just having 2 cursors is a poor design• Mouse / tablet for scrolling +15%
(Buxton/Myers)• Camera Control & Docking in 3D
(Balakrishnan)• Camera control +20% faster• Supports epistemic actions
• Map Navigation (panning + zooming)
Use Both Hands: Two Mice are Better Than One?
19
Don’t Use Your Hands at All?
• Feet for input (Hunter Digital, I/O Test Inc.)
• Head mice, Eye gaze (Zhai: MAGIC pointing)• All of these have lower bandwidth than hand• Some utility for secondary tasks, disabled use
• Head tracking in games• Hands-busy situations
20
• “optimal” or nonlinear C:D ratio • Gain theoretically does not affect
performance by Fitts’ Law (Jellinek & Card)
• Reduces footprint of device & reclutching• Visual feedback (cursor, selection highlight,
dragging rectangle, …)• Snapping the cursor• Inferring intent by cursor location?
Better Software?
21
Other Strange Axes of Innovation
• “Interfaces should be transparent to the user”
22
• “Interfaces should be transparent to the user”
• The endless N-button debate
Other Strange Axes of Innovation
1: Too Few 5: Busy 10+: ?!?2-3: About Right
23
Other Strange Axes of Innovation
• “Interfaces should be transparent to the user”
• The endless N-button debate• Nail consumer product XYZ
to the mouse• TeleMouse
(Altra Technology Inc.)
24
Other Strange Axes of Innovation
• “Interfaces should be transparent to the user”
• The endless N-button debate• Nail consumer product XYZ
to the mouse• Gelatinous (soft / squishy)
• Fellowes Corp: GelMouse, SoftGrip Mouse
25
• “Interfaces should be transparent to the user”
• The endless N-button debate• Nail consumer product XYZ
to the mouse• Gelatinous (soft / squishy)
• Less Is More?
Other Strange Axes of Innovation
26
no-button mouse (0% error rate)
27
• A lot has been tried!• Some very clever ideas -- some very bad ones
too!
• There are still some promising areas that are yet unproven for consumer devices. Ken’s bets:• New sensors, multi-channel, kbd
integrated ()
• Added DOF, 2 hands beneficial; in research stage
• Biometric sensing: some obvious wins; others?
• Tactile feedback, maybe force feedback (games)
• There are still some undiscovered gold nuggets!
Illustrated Tour: Summary