user interfaces: disappearing, dissolving, and evolving brown university july 2002 andries van dam
TRANSCRIPT
![Page 1: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/1.jpg)
User Interfaces: Disappearing, Dissolving, and Evolving
Brown University
July 2002
Andries van Dam
![Page 2: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/2.jpg)
• Why user interfaces are critical
• Computing environment trends:
ubiquitous, pervasive, invisible
• Immersive Virtual Reality and
Augmented Reality
• Research agenda
• Conclusion
Roadmap
![Page 3: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/3.jpg)
Computing Capacity
t
Use this power to increase bandwidth to the brain
Human Capacity
t
ComputeCompute GraphicsGraphics
Why Do We Need Dramatic Improvements in the User Interface?
![Page 4: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/4.jpg)
![Page 5: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/5.jpg)
GUIs are the “killer app” for graphics.
Even young children who can’t yet read and write can be productive computer users.
An Unanticipated and Revolutionary Result of GUIs
![Page 6: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/6.jpg)
But Are Today’s GUIs Good Enough?
![Page 7: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/7.jpg)
Dan Robbins
![Page 8: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/8.jpg)
Impedance-matchingLimitations of WIMP GUIs
Limited Vision (Flat, 2D)
No Speech
No Gestures
Limited Audio
One Hand Tied Behind Back
Limited Tactile
![Page 9: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/9.jpg)
![Page 10: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/10.jpg)
• None! UIs are a necessary evil
• Counterpoint: the aesthetics of a good UI
• Want to communicate and control as we do in and with the real world– objects
– tasks
– other participants (real and software agents)
• Models: Jeeves, HAL-9000
• Best today: transparency
• Future: Raj Reddy’s SILK, ultimately cogito ergo fac?
The Ultimate User Interface
![Page 11: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/11.jpg)
• Why user interfaces are critical
• Computing environment trends:
ubiquitous, pervasive, invisible
• Immersive Virtual Reality and
Augmented Reality
• Research agenda
• Conclusion
Roadmap
![Page 12: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/12.jpg)
![Page 13: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/13.jpg)
• Multimodal post-WIMP interfaces
– parallel sensory channels (sight, hearing, haptics)
• Ubiquitous, pervasive, invisible computing
– profusion of form factors - embedded in ordinary everyday devices, and even in/directed by our bodies
– smart appliances, furniture, rooms, vehicles, jewelry…
o MIT LCS’s Oxygen,
o MIT Media Lab’s Smart Rooms, Things that Think,…
Computing Environment Trends (1/2)
![Page 14: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/14.jpg)
![Page 15: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/15.jpg)
• Federation of devices mediating human-human interaction
Computing Environment Trends (2/2)
![Page 16: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/16.jpg)
![Page 17: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/17.jpg)
• User is video-tracked for identification, location, gaze, gesture (and in the future, affect?)
• Continuous speech recognition:
– + natural language understanding + intelligent information processing allows dialog with intelligent assistants (agents)
• Furniture: chair is instrumented to help detect posture, (affect?), adjust to the user’s preferred position
Smart Office Scenario
• Wall, displays + personal notepads to provide private space in addition to any shared space(s)
![Page 18: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/18.jpg)
![Page 19: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/19.jpg)
• Implanted delivery mechanisms
• Prostheses (today, heart pacemakers, hearing aids, cochlear implants, voice boxes, artificial joints and organs…)
• Avatars of– health-care providers– family and friends, support group
Medical Scenario
• Electro-chemical sensors/probes (increasingly less obtrusive), to monitor stress, body chemistry, etc.– smart toilet to monitor bodily wastes
• Scaled-down office computing environment
![Page 20: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/20.jpg)
• Why user interfaces are critical
• Computing environment trends:
ubiquitous, pervasive, invisible
• Immersive Virtual Reality and
Augmented Reality
• Research agenda
• Conclusion
Roadmap
![Page 21: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/21.jpg)
![Page 22: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/22.jpg)
Immersive Virtual Reality (IVR) & Augmented Reality (AR)
•Psychophysical sensation created by hardware and software
•Semi-immersive VR
Fishtank VR on a monitor
GMD’s Responsive workbench,Barco, Immersadesk™
Head-mounted displays (HMD’s)
CAVE™ and its derivatives
•Fully-immersive VR
•Augmented Reality (AR)Video or optics superimposes computer-generated data on real worlde.g., Columbia’s MARS
![Page 23: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/23.jpg)
![Page 24: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/24.jpg)
Uses of IVR and AR
• Workshops for today
– increasing use in design studios and laboratories
– artistic, therapeutic, and educational use increasing
– smart environments will make use of tools developed for IVR and AR
• Testbeds for the future– time machines that will become personally affordable, e.g.,
Elumens’ VisionStationCMU CUBE
![Page 25: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/25.jpg)
![Page 26: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/26.jpg)
• Thad Starner, formerly MIT Media Lab, Wearable Computing group, now at Georgia Tech
• Micro Optical Corporation, 320x240x8 LCD Display
• High- resolution Virtual Retinal Display (laser-based)at Dr. Furness’ HITLab@ U. of Washington
Augmented Reality: LCD Projection
![Page 27: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/27.jpg)
![Page 28: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/28.jpg)
CFD Scientific Visualization
• Blood flow through an arterial bypass graft
– fully 3D, time-dependent, viscous Navier-Stokes
– 3562 Spectral Elements: 1064 Prisms and 2498 Tetrahedra.
– only 50K triangles for artery wall, never came close to visualizing the LOD available from simulation results.
– this is a driving force for hierarchical, level-of-detail visualization
• Post-WIMP interaction
![Page 29: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/29.jpg)
![Page 30: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/30.jpg)
ARCHAVEVirtual Archeology
• Archeological research and analysis tool
• Walkthrough of excavation of Great Temple at Petra, Jordan– show excavation layers, trenches, to aid understanding of digging process
– abstract codings of multi- valued data– multi-scale navigation and visualization
• Analysis– 3D in situ presentation of artifacts
![Page 31: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/31.jpg)
• Why user interfaces are critical
• Computing environment trends:
ubiquitous, pervasive, invisible
• Immersive Virtual Reality and
Augmented Reality
• Research agenda
• Conclusion
Roadmap
![Page 32: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/32.jpg)
![Page 33: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/33.jpg)
Research Agenda (1/3)
• User interfaces to impedance-match all our senses
– component technologies are wholly inadequate
• User interfaces to seamlessly integrate a federation of devices and users, through different environments
• Interaction styles to be task, user, and computing device/environment specific, unlike desktop WIMP
![Page 34: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/34.jpg)
• Integration of direct and indirect (through agents) manipulation to be at a higher level of abstraction than point-and-click
• Knowledge representation techniques to deal with federations of devices and users
• Toolkits for these new styles of interfaces
Research Agenda (2/3)
![Page 35: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/35.jpg)
• Privacy, noise, sensory and cognitive overload issues
• “Universal design” principles to address needs of different cultures, physical, and mental capabilities
• User studies to show what is effective
– formal
– useful
– application-specific
Research Agenda (3/3)
![Page 36: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/36.jpg)
• Why user interfaces are critical
• Computing environment trends:
ubiquitous, pervasive, invisible
• Immersive Virtual Reality and
Augmented Reality
• Research agenda
• Conclusion
Roadmap
![Page 37: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/37.jpg)
![Page 38: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/38.jpg)
• Perceptual, cognitive, and social sciences and engineering
• Design disciplines: Industrial Design, User Interface Design, …
• Storytelling and communication arts: theater, film & video, advertising,…
• Computer science and engineering
• Early examples
– Pixar
– CMU’s Entertainment Technology Center
– USC’s Institute for Creative Technologies
– NPS’ MOVES (Modeling, Virtual Environments and Simulation) Institute
– UCF’s Digital Media program
Need a New Interdisciplinary Design Discipline
![Page 39: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/39.jpg)
“To Infinity and Beyond…”
![Page 40: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/40.jpg)
Recommended Reading
•“The Unifinished Revolution,” Michael Dertouzos, HarperCollins, 2001
•“The Next 1,000 Years,” Communications of the ACM, March 2001
•“Virtually There,” Jaron Lanier, Scientific American, April 2001
•“The Invisible Future,” Peter J. Denning, editor, McGraw Hill, 2001
![Page 41: User Interfaces: Disappearing, Dissolving, and Evolving Brown University July 2002 Andries van Dam](https://reader036.vdocuments.us/reader036/viewer/2022062801/56649e595503460f94b52982/html5/thumbnails/41.jpg)
The End