Download - Tangible Interfaces
Readings
• Hiroshi Ishii, Brygg Ullmer, “Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms”, Proceedings of CHI ’97, March 1997, pp. 234-241.http://doi.acm.org/10.1145/258549.258715 Better figures here: http://www.acm.org/sigchi/chi97/proceedings/paper/hi.htm
• Roy Want, Kenneth Fishkin, Anuj Gujar, Beverly Harrison, “Bridging Physical and Virtual Worlds with Electronic Tags”, Proceedings of CHI ’99, May 1999, pp. 370-377.http://doi.acm.org/10.1145/302979.303111
More Readings
• Jun Rekimoto, Brygg Ullmer, Haruo Oba, “DataTiles: A Modular for Mixed Physical and Graphical Interactions”, Proceedings of CHI ’01, March 2001, pp. 269-276.http://doi.acm.org/10.1145/365024.365115
• Jun Rekimoto, Eduardo Sciammarella, “ToolStone: Effective Use of the Physical Manipulation Vocabularies of Input Devices”, Proceedings of UIST ’00, Nov. 2000, pp. 109-117.http://doi.acm.org/10.1145/354401.354421
Supplemental Readings
• John Underkoffler, Hiroshi Ishii, “Urp: A Luminous-Tangible Workbench for Urban Planning and Design”, Proceedings of CHI ’99, May 1999, pp. 386-393.http://doi.acm.org/10.1145/302979.303114
• Leo Tolstoy, War and Peace, 1869, English translation by Rosemary Edmonds, Viking Press, 1982. http://www.online-literature.com/tolstoy/war_and_peace/
Motivation
• Desire to remove divide between electronic and physical worlds– Bring benefits of computation (the virtual world)
beyond confines of the screen, to the rest of life• Ubiquitous computing (Weiser)• Augmented reality
– Making crossing between “bits and atoms” seamless (or at least a lot easier)
• Put information “in our hands” in a rather literal way
Motivation (cont)
• “Spatial thinking”– Concrete– Like sketching: fluid, natural, dexterous actions– (Graspable) physical objects provide strong
affordances (much closer to real affordances instead of virtual)
– Different sets of thinking / reasoning skills• Used to say “visual specification for visual things”
– E.g., UI layout“spatial specification for spatial things”
In “Tangible Bits” paperIshii Promotes Three Concepts• Interactive surfaces
– Transform (all?) surfaces into an interface• (Really nothing special about “surfaces” except that they are
convenient to work with at this stage)
• Coupling of bits to atoms– Couple information associated with (relevant to)
physical objects to the objects• Provide direct access to info from object• Right information is always “at hand”
• Ambient media– Ability to use the periphery of attention and move
between the periphery and center of attention
Prior Work Examples (From Ishii’s “Tangible Bits”)
• DigitalDesk– Wellner, EuroParc
– Desk with top-projected video– Camera tracking from above– Digitizing tablet– Microphone
• On surface
DigitalDesk (cont)
• Interact with real paper– Digitize with camera
(eventually) and recognize action gestures
– Augmented with video from above
– Used microphone to detect “taps”
• Difficult to detect contact via video alone
Prior Work Examples (From Ishii’s “Tangible Bits”)
• Marble Answering Machine– Durrell Bishop, RCA
– Marbles represent calls– Put marble in indentation to
play message– Another spot to call back
• Live Wire – Natalie Jeremijenko, Xerox PARC
– String turned slightly on each Ethernet packet
– Making invisible bits visible/tangible
Ishii’s work in “Tangible Bits” paper
• metaDesk– Rear projected video (the bits)– Physical objects tracked on top (the atoms)
• Manipulating physical objs affects the bits (not rev)
Applications of MetaDesk
• Tangible Geospace– Map manipulation via phicons, etc.
• Two phicons (representing particular places) allow very easy manipulations of position, orientation, and scale of the map
– Solid Plexiglas shape
• Active lens gives 3D views– LCD display on a tracked arm
• Passive lens gives augmented or filtered 2d views– Clear lens – Change the projected display under the lens!
MetaDesk Issues
• Good demo, but not a real application– Solid and compelling apps for tangible interaction still
somewhat of an issue, overall– Seems very compelling, but is their really a big benefit
• Seems somewhat domain specific (see URP & ToonTown later)
• Do I have to have a new physical object for every command/parameter/whatnot?– Loosing some of the extreme flexibility of the virtual
world– Big research issues: How do we find the right mix of
physical / virtual to get good properties of both?
AmbientRoom
• Interesting things here:– Ambient displays
• Aesthetics important• A start on user defined mappings
– Does this scale?
– What about all the “hidden state” (can’t see what you’ve spec’ed)?
– Physical containers for information
• Issues– Hamster awareness !?!
• Again, missing a compelling application
TransBoard
• Augmented whiteboard– Tracked pens, front projected image– Looking at “surfaces which absorb information
from the physical world”
• Collaborage we saw earlier (but done later) was similar but probably more compelling
Additional Apps for MetaDesk
• Illuminating Light– Optical bench simulation– “I/O bulb” concept– New color bar tracking (from above)
Video…
Additional Apps for MetaDesk
• URP – Urban planning app
– Physical objects cast virtual shadows and reflections
– Wind flow simulation– Tools for making
measurements
Finding a good mix of physical and virtual: DataTiles (Rekimoto)
• LCD with pen input– Transparent tiles
• May have preprinted information
• May have groves affording particular interactions
– Sit on pedestals • Serve as ID readers for tiles
DataTiles
• Appearance is combination of display and pre-printed
• Grooves afford particular actions (labeled as to purpose)
Interesting tiles
• Containers– Copy virtual objects, settings, etc. to a tile– Set aside until later
• Parameters– Search times with parameters– Dynamic query on (preprinted) map tile
• Grooves for sliders
Interesting tiles (cont)
• Actions– Print
• Generic print tile• Set next to object, join with gesture, fill in print dialog
– Time machine• Show previous states of another tile• “Clock” grooves with appropriate dynamic feedback
– Remote interaction tiles• Shared drawing surface with remote participants
Physical / virtual or generic / concrete balance
• Note that a range of interaction techniques are possible along scale of very concrete and pre-determined to very generic – Also static to dynamic and physical to virtual
• Composition mechanism allows fairly pre-determined tiles to have much more utility– General lesson: simple actions + composition
• Each small and simple
– Let combinatorics take over to give wide expressive range, but with (potentially) less overall complexity
Tagging Objects
• Several technologies (see below) for tagging objects to enable sensing of identity– Objects brought into proximity of special reader
• Range varies: room to a few cm
– Reader determines a unique identifier for object• Typically a unique number• Note that 64 bits is probably enough to uniquely tag
every individual object ever manufactured– every CD, ever can of soup, every page of paper – Even 32 bits goes a long way (4 billion unique IDs)– 96-128 bits is better (can partition ID space conveniently)
Fun with IDs
• A number of things you can do with an ID– If you have network connectivity (or can cache) at the
reader then any identifiable object can have an arbitrary amount of data attached to it
• Look up the ID in a database to retrieve “attached data”• Data can be general info, commands, parameters, anything, …
– Attached object can be static or dynamic
• Can also do this with a local database if info is local
– Object can now be a surrogate for the data• Arbitrary data (incl. arbitrary programs) very powerful
Identified Objects Can Act as• Container
– Can (appear to) put in, carry away, and (later/elsewhere) take out (arbitrary) data
• User ID– Can serve to represent / identify a person
• Context ID– Can serve to identify a place – Can indicate the presence of a piece of equipment
• Provide handle for accessing it• E.g., walk into a room with a printer and get access to its queue
• Actions to be performed• Set of attributes / parameters / settings
ID Technologies
• Printed Bar Codes– Read by optical scanner
• Laser or other• Usually close contact, but can be done at a distance• Typical read error rates: 1 per 10,000
– Most common: UPC on all retail items
• Most common: 10 decimal digits + prefix & check digits
– Enough for every kind of soup, but not every individual can
Bar Codes (cont)
• Can also store a lot more data– See: PA vehicle registration
UPS “maxicode”
• Advantages of bar codes– Very cheap (just print them with a printer)– No battery needed for tag– Reliable, readers relatively cheap
• Disadvantages– Requires contact (and/or careful alignment)– Only work on relatively flat/smooth surfaces– Ugly
Aside: Can get rid of Ugliness
• Xerox DataGlyph technology designed to look better – Appears as a textured gray region
• Relatively high density• Also survives bad faxes and other distortions well
IR Beacons
• Encode ID in pulses of (invisible) infrared light – Same technology as TV remotes (cheap & easy)
– ID sent periodically (typically every few seconds)
• Applications– Fixed beacons to indicate place
• Example: HP CoolTown project– Emits URL of web page giving access to services (e.g., print
queue for printer, or control of projector) in room
– Active Badges identify location of a person
Other Interesting ID Mechanisms
• Contact tags, magnetic stripe, and smart cards• For limited sets of objects: weight them
– 0.1 gram resolution scales distinguish most objects– Issues with “wear and tear”
• May want to augment with vision
• Fingerprints– User fingertips as data repositories (!)
• Virtually “pick up” arbitrary data, carry, and “drop” it.
– Another potential app: 10 actions for 1 button
RFID Tags
• Probably most generally useful technology
• Reader has a typically large coil– Emits RF at a particular frequency
• Tag has a typically small coil– Inductively couples with reader– Draws power from it (no batteries in tag)– Modulates its load on inductive coupling to send
information back to the reader
Different tag properties
• Embedded processors• Persistent memory• Single read vs. multiple
– Multiple requires special protocol with reader– Implies a processor
• Differing read ranges– Typical 10cm– With “active” tags (battery) can get longer (10m)
Advantages (from Want paper)
• Robustness– Designed to work in harsh environments
• Inside dogs, on pigs ears, nailed into wood,under labels at K-Mart
• No maintenance
• Post-hoc augmentation• Can typically be hidden (no aesthetic costs)
– Small and can be embedded in lots of things
• Easily sensed– Doesn’t require direct contact, line of sight, or alignment
Disadvantages (from Want paper)
• Need interface for associating meaning with objects– Not particular to RFID tags
• Knowing what is tagged– Hidden tags mean no affordance
A good application using object identification
• ToonTown access control interface in the SomeWire audio space Andrew Singer, Debby Hindus, Lisa Stifelman, and
Sean White, “Tangible Progress: Less Is More In Somewire Audio Spaces”, Proceedings of CHI ’99, May 1999, pp. 104-111, 625. http://doi.acm.org/10.1145/302979.303007
ToonTown
• Shared audio system with reciprocity– Needs good access control
• “Who can hear me” (awareness and easy control)
– Also supported localization of the audio• Can place participants left, right, etc.
• Place “cartoon characters” (small dolls) on trays to represent people– RF Tag in the bottom of each doll, readers in tray
ToonTown
• On tray can hear and be heard• Clear visibility, easy to manipulate• Downside: doesn’t scale (but not clear what would)
ToonTown
• Location on tray also controls apparent sound source location and volume
• Note spots at right for other UI functions
Toolstones
• An input device– Not exclusively tangible (used with GUIs), but a
lot to do with grasp and touch– Device designed for use by non-dominant hand
• Typically with a pen or other pointer in the dominant hand
– Note: In most GUIs if you aren’t typing you might as well sit on your other hand
• May be intermittent – Adjust, work with primary hand, adjust, etc.
About bimanual operations (cooperative between hands)
• Hands tend to take on different roles – A lot more than a “hand preference” or
“stronger hand”– Semantically different actions by dominant and
non-dominant hands
• Seminal work Yves Guiard, “Asymmetric Division of Labor in Human
Skilled Bimanual Action: The Kinematic Chain as a Model”, Journal of Motor Behavior, Vol. 19, No. 4, 1987, pp. 486-517.http://cogprints.soton.ac.uk/documents/disk0/00/00/06/25/
Three key observations from Guiard
(Assuming right-handed -- tendencies somewhat less clear left-handers)
• Left hand sets context for right– Left provides overall positioning and orientation
for detail work of right– E.g., left holds fabric, right stitches where fabric
held
Three key observations from Guiard (cont)
• Tendency for larger scale and lower frequency movements by the left hand– Larger, courser, slower motions by left– Smaller, more precise, faster motions by right
• Motion by left hand tend to precede those of the right– Position objects before acting on them– Pick up the cloth before sewing it
Working with these tendencies
• Taking proper advantage of different roles (rather than just subdividing the work) can provide better/smoother performance
• ToolStone device designed for non-dominant hand– Will see for tasks like selecting color from
palette instead of tasks like painting
ToolStones
• Rectangular device – Each of 6 faces – Distinguishable by touch alone (via size and/or marker)
• Works with a digitizing tablet– Uses components from particular type of tablet– Provides constrained six degree of freedom
(6 DOF) sensing• X,Y,Z position• 3 degrees of orientation (e.g., tilt, yaw roll)• constrained because stone must stay in contact with the
surface – typically one face must be on surface, but version supporting tilting
on edges also possible
Sensing
• Quantize possible positions and orientations out of full 6DOF space
• 8 Directions
• On 6 different faces
• Also track x,y position on tablet
Sensing
• One “pen” in contact (close proximity) with surface for any orientation
• Pens are passive– No battery, just resonant coil sensed from tablet
• Each pen allows sensing of– Contact (very close proximity, not “tip switch”)– Angles with respect to tablet– “Pressure” and ID of pen
• Actually location of an iron slug within the coil• Each unit initially tuned to slightly different frequency• Moving iron slug changes within a range• Tablet can sense at least 3 different pens at simultaneously
Interaction techniques
• Typically each face has a different task – Important that they be distinguishable by feel– But labeling also important / useful
• Palettes / menus– Orientation based menus (8 items max)
• Fitts’ law says…?
– Larger use motion to pick like a mouse• E.g. color palette
• Potential issue: What is it?…
…l…
Interaction techniques
• Typically each face has a different task – Important that they be distinguishable by feel– But labeling also important / useful
• Palettes / menus– Orientation based menus (8 items max)
• Fitts’ law says…?
– Larger use motion to pick like a mouse• E.g. color palette
• Potential issue: don’t have a button– Best for modal interactions such at tool palette or as
toolglass style interaction (will see this in a later lecture)– Basically piggyback off “primary” locator in dom hand
Interaction Techniques
• Also can do “Action + 1 scalar parameter”– E.g., Scale operation and magnitude of scaling– Action set by orientation
• Twist to get the one you want
– Parameter set by movement “up & down” (i.e, in direction of action selection)
• Absolute or clutched like mouse
• Could also do “Action + 2” but starting to get away from natural tendencies of non-dominant hand
• Can also do things like 3D object or camera or positioning that require more 2+ DOF
Other forms
• Doesn’t have to be rectangular or even have parallel sides– Although lots of flat surfaces seems
useful
• More tactile differentiation of faces might be useful
• Could also augment with buttons– But could easily get beyond spirit of
device for non-dominant hand