designing physical/digital objects for the internet of things: from sensors to user interaction
DESCRIPTION
TRANSCRIPT
DESIGNING PHYSICAL/DIGITAL OBJECTS FOR THE INTERNET OF THINGSFROM SENSORS TO USER INTERACTION
Gerd Kortuem
17 June 2009 Infolab21 | Lancaster [email protected] | @kortuem | www.kortuem.com
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
SOME TWITTER USERS ARE NOT HUMAN
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Andy Stanford Clark (IBM)
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Source: Cartoon by Peter Steiner. The New Yorker, July 5, 1993 issue (Vol.69 (LXIX) no. 20) page 61
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
ON THE INTERNET, NOBODY KNOWS YOU ARE A
THING
Source: http://www.botanicalls.com/kits/
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
• ATmega1280, 5V• Digital I/O Pins 54 (of which 14 provide PWM output)• Analog Input Pins 16• DC Current per I/O Pin 40 mA• DC Current for 3.3V Pin 50 mA• Flash Memory 128 KB of which 4 KB used by bootloader• SRAM 8 KB• EEPROM 4 KB• Clock Speed 16 MH
Arduino is an open source microcontroller board, plus a free software development environment, to make objects that can sense inputs from switches, sensors, and computers, and control motors, lights, and other physical outputs in the real world!
ARDUINO
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Source: Clive Thomas / Wired http://www.wired.com/techbiz/startups/magazine/16-11/ff_openmanufacturing
OPEN-SOURCE HARDWARE
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Source: Clive Thomas / Wired http://www.wired.com/techbiz/startups/magazine/16-11/ff_openmanufacturing
OPEN-SOURCE HARDWARE
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Blogjects track and trace where they are and where they’ve been
Blogjects have self-contained (embedded) histories of their encounters and experiences
Blogjects always have some form of agency — they can foment action and participate; they have an assertive voice within the social web.
BLOGJECTSA Manifesto for Networked Objects — Cohabiting with Pigeons, Arphids and Aibos in the Internet of ThingsBY JULIAN BLEEKER
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
THINGS AND PEOPLE
PEOPLE AND THINGS
An Internet of Things that is more than just sensors attached to things
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
1. CASE STUDIES2. HUMAN DIMENSION
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
BP Chemical storage facility, Saltend, Northeast England
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Road maintenance crew, South England
trial users anonymized
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Breaker Circular Saw & Wacker Plate
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Vibration White Finger
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Vibration Dosimeter
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Duration of use (measured)
Vibration characteristics (known for each equipment, measured by manufacturer)
Soil condition (assumed)
operating heavy vibrating machinery such as hydraulic drillsand breakers. Our experimental system comprises a collec-tion of tools, augmented with wireless sensor nodes, personaluser devices and mobile computers that collaborate in anad-hoc manner in order to collect HAV exposure informa-tion. This information is available in real-time to workersin the field and subsequently to management via a back-enddatabase. Our design allows health and safety regulationsspecified at the enterprise level to be embedded within thewireless sensor nodes, allowing them to operate without anyinfrastructural support. This is crucial for workers who oftenhave to operate in areas with limited or no wireless commu-nication infrastructure. The system was evaluated througha two week field trial that took place with the collaborationof a major UK construction and maintenance company.
2. DOMAIN ANALYSIS
2.1 The Hand Arm Vibration ProblemLong-term exposure to hand arm vibration can lead to
serious health conditions such as “vibration white finger”(VWF) and in extreme cases to life-long disability. VWF istriggered by excessive use of vibrating machinery such as hy-draulic drills and breakers, and causes the fingers to becomenumb and begin turning white. As the disease progresses itcan become irreversible; the person su!ers increasingly fre-quent painful attacks at any time and may even lose theirfingers (typically this only happens in extreme cases, for ex-ample, when people are working with vibrating machineryin very cold conditions, as is the case in the forestry industryamong people working with chainsaws).
Extensive health and safety regulations exist to limit work-ers’ exposure to HAV. For example, in 2005 the UK Parlia-ment introduced the Control of Vibration at Work Regula-tions [10]. These regulations set limits for daily exposureto vibration, define methods for their calculation and for-mulate guidelines for monitoring an employee’s exposure tovibration. The guidelines place a responsibility on employ-ers to assess each individual employee’s exposure risk and toconsider the specific working conditions of each employee.When it is not possible to eliminate a worker’s exposure tovibration then suitable health surveillance must be put inplace and immediate action be taken if specific vibrationlimits are exceeded.
The damage caused by exposure to vibrations is a combi-nation of both the frequency of the vibrating tool and theduration of the exposure. Using a tool that vibrates at alow frequency for a long time can be as damaging as usinga heavily vibrating tool for a short time. Thus regulationsspecify two figures for exposure duration, a limit to the over-all daily trigger time (Daily exposure limit), and a limit onshort-term exposure to very high levels of vibration (Dailyexposure action). These limits are defined in terms of theaverage daily exposure dose A(8):
• Daily exposure limit value = A(8) of 2.5m/s2
• Daily exposure action value = A(8) of 5.0m/s2
A(8) is defined as ahv
p
T/8 where
ahv = actual vibration acceleration rate expressed inm/s2
T = actual exposure duration expressed in hours
In turn, ahv is composed of the root–mean–square accel-eration magnitudes in three orthogonal directions, x, y andz, at the vibrating surface in contact with the hand. Wheredaily vibration exposure A(8) is above 2.5m/s2 but below5.0m/s2, the following steps should be taken: Inform work-ers of risk, carry out regular health surveillance of worker,record assessment. Where daily vibration exposure exceeds5.0m/s2, the following actions must be taken: Limit usageimmediately, rotate workers, introduce other working meth-ods to reduce HAV. Since in practice ahv is not known fora specific piece of work, equipment manufacturers publishestimated ahv values for each individual tool, which can beused to estimate an operator’s exposure.
There is a distinct lack of automated solutions for assistingwith HAV H&S rule compliance. For example, vibration ex-posure data is typically manually recorded by operatives onpaper sheets, which are then entered by hand into a healthand safety information system. Moreover, key data such astrigger time must be estimated by operatives, most oftenhours after work has been completed. This of course raisesserious concerns with respect to completeness, accuracy andconsistency of captured data. The current practice can beimproved using mobile data entry solutions based on hand-held wireless computers. Yet, while mobile solutions reducethe need for paper forms they still su!er from the fact thatthey rely on human information gathering and recordingin the field. In addition, current solutions are tailored foro!-line processing of data in the back o"ce, ignoring thepotential benefits for real-time information in the field.
2.2 Requirements CaptureIn close collaboration with a major UK company that car-
ries out road maintenance and construction operations, weset out to design, implement and evaluate a Hand Arm Vi-bration (HAV) monitoring system, based on our approach ofaugmenting work artefacts. Our team consisted of computerscientists, psychologists, ethnographers and organisationalmanagement experts. Together, we engaged with the com-pany on various levels, ranging from high-level managementto workers in the filed, to investigate current work practices,identify technical and usability requirements, and anticipatedeployment challenges. The following gives an outline of ourapproach and introduces the requirements that we gathered.
2.2.1 MethodologyTo inform the design and elicit requirements we followed
a two-pronged approach. On the one hand, we undertookextensive work place studies and interviews to understandcurrent work practices. On the other hand, we generateddesign sketches to elicit concrete feedback.
In total we conducted over twenty formal interviews withmanagers and employees and 16 full days of work obser-vation. Field observation days also involved informal dis-cussions with upwards of 25 operatives going about theirdaily work, with the fieldwork being conducted across threedi!erent regional sections of the company, each of whichwith its own safety culture and traditions. An importantresult with respect to risks from vibration and noise expo-sure (“invisible risks”) was that it emerged that these wereappropriately assessed and their mitigation documented atan organisational level, yet were often underestimated orviewed as insignificant by operatives due to the apparentlyloose linkage between the risk and its outcomes. For exam-
Actual Exposure
Maximum Legal Exposure
?
Hand-arm vibration syndrome (HAVS)
• Neurological component
– numbness, tingling, reduced sense of
touch & temperature
– main cause of disability in advanced
cases
• Vascular component
– “vibration white finger”
– episodes of finger blanching
– typically cold-induced attacks
• Musculoskeletal component
– pain, weak grip, etc.
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Static domain model+
Dynamic world model
Operator
IN-SITU ASSISTANCE
encode rules query
REAL-TIME ASSESSMENT
inform
CODIFICATION
Health & Safety Policies
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
PERSONAL HEALTH HISTORIES
0
37.5
75.0
112.5
150.0
13:50 14:00 14:10 14:20 14:30 14:40 14:50 15:00 15:20 15:30 14:20
Dai
ly E
xpos
ure
Act
ion
Valu
e
Time
0
10
20
30
40
10:50 11:00 11:10 11:20 11:30 11:40 11:50 12:00 12:20 12:30 12:40
Dai
ly E
xpos
ure
Lim
it Va
lue
Time
How dangerous is work for each individual?
Daily limit Intensity limit
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
The recorded usage time had an average error of 3.2% against the actual usage as recorded by the video footage.
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
SITE-SPECIFIC HEALTH LOGSWhere are safety hotspots?
Is health managed differently across work-sites?
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
ACTIVITY PATTERNSWhen do safety incidents occur?
When is a task finished?
1 5 9 13 17 21
Tool 1Tool 2MotionSoundLightTask
9:00 9:30 10:00 10:30 11:00 11:30
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Mod
el
Awarness
Interactivity
Activity-awa
re
Process-awar
ePolicy
-aware
Functions
Rules
Work-Flows
POLICY-AWARE OBJECTS
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Mod
el
Awarness
Interactivity
Activity-aware
Process-awar
ePolicy-
aware
Functions
Rules
Work-Flows
ACTIVITY-AWARE OBJECTS
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
RENTAL COMPANY CONSTRUCTION COMPANY
UseCost calculation
Recording usage
LeaseCost = (T "U) + Cost(m)ii=1
|M |
#
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
USAGE-BASED PRICINGRental equipment is charged per actual use, not per time
0
1
2
3
4
5
6
7
0 1 2 3 4 5 6 7
Tool
Worker
0
10
20
30
40
50
60
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Tool 3Tool 2Tool 1
d1
When, how, how often? By whom?
d1 d2 d3
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
THE SAFETY-AWARE BARREL
System software
Sensors
Display
Application software
Memory
Processor
Wireless Network
Storage
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
THE SAFETY-AWARE BARREL
Don’t shake me so much!I am too hot!
I shouldn’t be here!
There are too many of us!
I am standing too close to my neighbours
Don’t put me next to these barrels!
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
COOPERATIVE REASONING
Chemical A alone is ok Chemical A next to chemical B is not ok
Reactive chemicals may not be stored next to each other
m1 + m2 < safety limit
Total mass of a chemical must be below critical threshold
m1 + m2 + m3 > safety limit
©TECO Karlsruhe University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
COOPERATIVE REASONING
Chemical A alone is ok Chemical A next to chemical B is not ok
Reactive chemicals may not be stored next to each other
m1 + m2 < safety limit
Total mass of a chemical must be below critical threshold
m1 + m2 + m3 > safety limit
Chemical knowledge
©TECO Karlsruhe University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
COOPERATIVE REASONING
Chemical A alone is ok Chemical A next to chemical B is not ok
Reactive chemicals may not be stored next to each other
m1 + m2 < safety limit
Total mass of a chemical must be below critical threshold
m1 + m2 + m3 > safety limit
Chemical knowledge
Spatial awareness
©TECO Karlsruhe University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
DEVICE ARCHITECTUREDomain Knowledge
reactive(<chemical>,<chemical>)critical_mass(<chemical>,<number>)content(me,<chemical>)mass(me,<number>
Safety Ruleshazard:- content(me, CH1), content(C, CH2), reactive(CH1, CH2), min_dist(CH1, CH2, D1), distance(me, C, D2), D2 > D1.
Observational Knowledgedistance(<container>,<container>,<dist>)
Smart-Its Sensor Node
PIC18F6720 @ 20 MHz (5 MIPS)Program Memory: 128KB
Data Memory: 3.8KB RAM1KB internal EEPROM
Memory: 32 KB EEPROM
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Trial at BP’s chemical plant in Hull, UK (July 2006)Smart-its sensor node:
PIC18F6720 @ 20 MHz (5 MIPS)Program Memory: 128KB
Data Memory: 3.8KB RAM
©TECO Karlsruhe University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
1. CASE STUDIES2. HUMAN DIMENSION
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
DESIGN PROCESS
Google Maps http://maps.google.co.uk/maps/ms?f=q&hl=en&geocode=&ie=UTF8&om=1&msa=0&msid...
2 of 3 18/9/07 17:44
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
DESIGN PROCESS
Google Maps http://maps.google.co.uk/maps/ms?f=q&hl=en&geocode=&ie=UTF8&om=1&msa=0&msid...
2 of 3 18/9/07 17:44
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
INVISIBLE RISKSRisk perception is poor
Goal: increasing awareness
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
SELF-IMAGE AND SOCIAL PRESSURE Safety equipment often not used
Personal and social “invisibility” becomes key design goal
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
PAST EXPERIENCES & NARRATIVESGPS had been experienced as surveillance technology
Technology narrative must be part of the design
= or or ?
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
INFORMATION DISCLOSURE
Sensing-only Awareness display
Unit w/o display was perceived as surveillance technology
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
TECHNOLOGY ARCHETYPES
Sensor network(sensing + data)
Smart object(interaction)
Developers view problem in light of well-known system models
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
TECHNOLOGY ARCHETYPES
Management
Operatives' behavior
Data logs
Exception reports
Health & Safety Rules
Operatives
Feedback
Automatic data capture
Enterprise
System Personal Health & Safety Record
Operative'sbehavior
Context-sensitive notices
Sensor-network inspired H&S system
(top-down)
Smart-object inspired H&S system
(bottom-up)
Different system models imply different H&S solutions
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
BEHAVIOR-BASED SAFETY
Personal Health & Safety Record
Operative'sbehavior
Context-sensitive notices
Personal Health & Safety Record
Operative'sbehavior
Context-sensitive notices
Operatives as source of expertise and behavioral change
Collaborative safety
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
POINTS TO TAKE HOME
Internet of Things = Things + People
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
THANK YOUGerd Kortuem
www.comp.lancs.ac.uk/~kortuem
Gerd Kortuem | [email protected] | @kortuem | www.kortuem.com | Infolab21 | Lancaster University
PressSmart tags hail the web of things. BBC. http://news.bbc.co.uk/2/hi/technology/7207514.stmScientificDaniel Fitton, Fahim Kawsar and Gerd Kortuem. Exploring The Design of a Memory Model for Smart Objects. First International Workshop on Digital Object Memories. To appear.Daniel Fitton, Vasughi Sundramoorthy, Gerd Kortuem, James Brown, Christos Efstratiou, Joe Finney, Nigel Davies. Exploring the Design of Pay-Per-Use Objects in the Construction Domain. EuroSSC 2008: 192-205Kortuem et al. Sensor Networks or Smart Artifacts? An Exploration of Organizational Issues of an Industrial Health and Safety Monitoring System. Proceedings International Conference on Ubiquitous Computing (Ubicomp 2007), October 2007, Innsbruck, Austria Brown et al. Network Interrupts: Supporting Delay Sensitive Applications in Low Power Wireless Control Networks. Proceedings ACM MobiCom workshop on Challenged Networks (CHANTS 2007), September 2007, Montreal, CanadaEfstratiou et al. Experiences of Designing and Deploying Intelligent Sensor Nodes to Monitor Hand-Arm Vibrations in the Field. Proceedings of MobiSys 2007, San Juan, Puerto Rico Iszatt White, M. Catching Them At It? An Ethnography of Rule Violation. Symposium on Current Developments in Ethnographic Research in the Social and Management Sciences, 13th-14th September 2006, Liverpool, UK Busby, J. and Iszatt-White, M. Pushing the Boundaries of HRO Thinking: Non-complex and Uncoupled but still Deadly. SRA Annual Meeting - Risk Analysis in a Dynamic World: Making a Difference, 3rd-4th December 2006, Baltimore, Maryland Davies et al. Sensing Danger – Challenges in Supporting Compliance in the Field. 8th IEEE Workshop on Mobile Computing Systems and Applications (HotMobile 2007), February 2007, Tucson, Arizona Lowton M. and Finney, J. Finding NEMO: On the Accuracy of Inferring Location in IEEE 802.15.4 Networks. Proceedings Workshop on Real-World Wireless Sensor Networks (RealWSN 2006), June 2006, Uppsala, Sweden
PUBLICATIONS