vub ai-ii 2001-2002 wvdv situated computing wearable computing - 1 vub ai-ii situated computing...

VUB AI-II2001-2002 WVdV

Situated ComputingWearable Computing - 1

VUB AI-II

Situated Computing

Module 2: Wearable Computing

Walter Van de Velde

[email protected]



Vision behind wearables• Bush

– Memex (1945)

• Licklider– Human-Computer symbiosis (1960)

• Engelbart (1962)– Intelligence augmentation, inventor of the

‘mouse’, collaborative computing

• Weiser– Ubiquitous computing (1991, scientific amer.)



Vision: Memex (Bush 1945)

• The Memex was a conceptual machine that can store vast amounts of information, in which users have the ability to create information trails, links of related texts and illustrations, which can be stored and used for future reference.

• Today some of us call this the World Wide Web



Vision: Human Computer Symbiosis (Licklider 1960)

“Man-computer symbiosis is an expected development in cooperative inter-action between men and electronic computers. It will involve very close coupling between the human and the electronic members of the partner-ship. The main aims are 1) to let computers facilitate formulative thinking as they now facilitate the solution of formulated problems, and 2) to enable men and computers to cooperate in making decisions and controlling complex situations without inflexible dependence on predetermined programs. In the anticipated symbiotic partnership, men will set the goals, formulate the hypothesis, determine the criteria, and perform the evaluations. Computing machines will do the routinizable work that must be done to prepare the way for insights and decisions in technical and scientific thinking. Preliminary analyses indicates that the symbiotic partnership will perform intellectual operations much more effectively than man alone can perform them. (…)”



‘Definitions’ of wearable systems

• Bradley Rhodes (MIT)– portable while operational

– needing minimal manual input

– aware of the environment

– always on and able to attract attention

• Steve Mann (Toronto)– part of the user

– controlled by the user

– negligible operational delays



Cybernetic approach• ‘The concept of the cyborg was to

allow man to optimize his internal regulation to suit the environment

he may seek’ – Manfred E. Cylnes, D. Sc. (1970, submitted to

Journal of Astronautics, they refused to publish it)

• Example: Getting a fish to live on land

• Problem: How will the fish breath; solution: take a small quantity of water along. Is it efficient? Maybe, however, there may exist much better ways of carrying out the functions of the respiratory system than by breathing, which becomes cumbersome in space (in the case of an astronaut for example)



Psycho-Physiological problems• ‘The purpose of the cyborg, as well as his own

homeostatic systems, is to provide an orgazinational system in which such robot-like problems are taken care of automatically and unconsciously, leaving man free to explore, to create, to think and to feel.’ -Clynes and Kline. “Drugs, space and cybernetics”, Psycophysiological aspects of Space Flight Symposium.

• Problems for long-term space flight: Wakefulness, radiation effects, metabolic problems and hypothermic controls, oxygenization and carbon dioxide removal, fluid intake and output, enzyme systems, verstibular function, cardiovascular control, muscular maintenance, perceptual problems, pressure, variations in external temperature, etc.



Idea of amplifying skills (body or mind related)

Here is an exoskeletonexample, designed asa muscle amplifier



Are we already Cyborgs?

• PDAs, watches, laptops, glasses, etc.

• Trends in broadband wireless networks in combination with advances in microelectronics technology create the infrastructure for on-body, where needed, when needed computing.



Enabling Technologies Overview

InputDevices

OutputDevices

Power Networks SoftwareEng.

Textual Textual Batteries Low-orbitsatellites

Jini fromSun

Audio Audio HumanBody

Bluetooth JavaSpacesfrom Sun

Video Video Low-powerarch.

IEEE 802.15PersonalArea Nets

InfernofromLucent

GPS/GIS Tactile T-Spacesfrom IBM

Context

Affectiveinputs



Textual input devices

Data Egg chording keyboard+: Doesn’t require acomputer to operate

Bat, chording keyboard-: requires a flat surface

Matias, Half-QWERTY keyb.+: Skill transfer from QWERTY

The twiddler, de facto standard in the wearables community



Audio input/output

• Speech recognition

– Can’t use it while talking with someone

– Continuous speech recognition, still a hard problem

– However, can support constraint vocabulary task such as procedural knowledge support (I.e., maintenance, inspection, etc.)

• Speech synthesis

• Spatial Audio interfaces (I.e., Nomadic Radio)



Nomadic Radio

•An audio only wearable

•3-D spatial audio output

•Voice input

•Access to multiple information streams

Nitin Sawhney, MIT Media Lab, wearing the audio vest (1998)



Nomadic Radio (MIT 1998)

Voice Message

UpcomingCalendar Event

Email

12:00 PM

6:00 PM

9:00 PM3:00 PM

2:30 PMhorizontal

plane

The Message Swatch, displayingvoice messages and email as

concentric beams and calendarevents as arcs.

Localization of an incoming messageon a chronological 12-hour spatial

audio display. The spatial andtemporal mapping is consistent with

the visual display.

Source: N

itin Saw

hney, MIT

Media L

ab



Video input• Computer vision

– requires a lot of computing power

– A lot can be inferred from cameras that see what we see in terms of immediate context

– State of the art vision research systems can recognize location, and sequences of actions such as crossing the street, in the supermarket, in a conversation, etc.

– Real-time face recognition could be an interesting input modality

– Statistical object recognition could be used to post notes on objects around us (‘virtual post-it’)



GPS/GIS• Location based context has immediate applications such as

the tourist guide systems

• -: GPS doesn’t work indoors

• Example prototype system from Columbia University:

Prototype tour guide Here’s what the user sees



Affective inputs

• There is a one-to-many mapping between our physiological state and our psychological state. Given that we can make some measurements about our physiological state (heart rate, skin conductance, pupil dilation, etc.), can we infer our psychological state?

• Such information can be useful in early detection of diseases, evaluation of machine performance, learning preferences over time



Gestural inputs• Eye tracking

– If we know where the user is (GPS/GIS), where their head is turned to (electronic compass), and where their eyes are focused at (eye tracking), a whole new application space opens up ranging from reminders to just-in-time training

• Finger tracking• Hand interfaces (american sign

language)• Motion tracking (accelerometers)

A recently announcedeye tracker basedon a CMOS cameraand a microcontroller.USB interface.



Visual Output Devices

• Head-Worn and Eye-glasses based Displays

1280x1024 displays in thesize of a postage stamp

Right-angle prism mold injectedinto glass



Optical principles behind Heads-up Displays

• Heads-up display technology is an appropriate output

device for wearable computing. Goal is to have physically small output devices which could provide the perception of large screens. First we will demonstrate the optical principles behind a heads-up display:



Augmented Reality• Registered Augmented Reality:

– Precise pixel/subpixel: Video Fudicials (Neumann, Klinker, etc.) Natural Video (Video orbits, etc.) Open-loop position trackers

– Field Oriented: Video based Position based

• Non-Registered Augmented Reality: • 2D Windowing Systems

• 3D Interfaces (Billinghurst)



Example AR application

Prototype apparatus User sees the instructionsoverlayed on top of thereal world

In this research, Feiner (‘91) demonstrated how AR couldsupport a procedural maintenance routine



Power generation and storage

Research status: 70-80mW from walking using PVDF piezos



Agents

• Remembrance Agent by Rhodes & Starner:

A context aware, pro-active agent running on Emacs



Cu

rren

t w

eara

ble

s

This can only be but a temporary state of development!



Power requirements <2W;Peripherals up to 300 mW (e.g. CMOS camera)



COMRIS Parrot Design



The COMRIS parrot



Wearables and Agents: COMRIS

• Comris = Co-Habited Mixed Reality Information Systems

• Based on a vision of two parallel worlds, a real one and a virtual one, coupled by wearable devices (the COMRIS parrots)

• Application envisaged in large-scale events



Applications and markets for wearable computing

• Vertical Markets• Maintenance

• Inspection and quality control

• Data collection (Health care, etc.)

• Training

• Process control

• Enterprise resource planning

• Warehousing

• Horizontal markets



U.S. numbers for mobile markets with applications to wearables

Distribution and transportation 7.27M

Health care 4.18M

Manufacturing 4.03M

Field Service 2.67M

Public Safety 1.37M

Utilities .19M

Total 19.71M

Source: IDC/LINK and Mobile Insights



Commercially available wearable systems

IBM Wearable– 233Mhz, 64Mb RAM

– 340Mb disk (Microdrive)

– 10 ounces

Via FlexiPC XybernautMobileAssistant



Privacy issues

• Feiner’s problem with everyone having the ability to capture/store information



Ethical Issues: Today’s Stories

• A EU project on Experimental School Environments

• Children 4-8 years

• Goal: reflection on small experiments inliving



Today’s Stories

“The mere replay of …behaviour is not sufficient to lead to reflection. In our approach the intertwined day’s histories, highlighting major events from different points of view (only) constitute the raw material for a subsequent reflective dialog involving children, educators and/or parents. (…)”



Today’s Stories: ethical issues

• Who owns the images?

• What is one allowed to do with it?

• Who should decide (design is unavoidable)?

• Ethics of research itself



Toward a Human Ecological Ethics (Flinders 92, Beach 99)

Utilitarian Deontological Relational Ecological

Recruitment InformedConsent

Reciprocity Collaboration CulturalSensitivity

Fieldwork Avoidance ofHarm

Avoidance ofWrong

Avoidance ofImposition

Avoidance ofDetachment

Reporting Confidentiality Fairness Confirmation ResponsiveCommunication

Table 1: Table of ethical guidelines for fieldwork, from Flinders (1992).

Technologies need to be viewed as elements of the larger natural/technological ecology that we are co-creating with nature. Wellbeing of the others (those involved or not) should be the primary concern, established whenever possible in a participatory fashion.



Future of wearables

• Direct neural interfaces in the form of implants?– Interval Research issued a patent for display

that resides under the skin and displays insulin levels. Same technology could be used as a programmable tattoo.

– Artificial Retinas wire some blind’s visual nerves so that ‘vision’ is recovered.

– Kevin Warick (Reading, UK) wears implanted electronic tags.



Future of Wearables

• A truly wearable computer in the form of textiles, powered up from the body and the environment? i-Wear

• True context awareness

• Embedding in Ubiquitous computing environments.

vub ai-ii 2001-2002 wvdv situated computing wearable computing - 1 vub ai-ii situated computing...

Documents