january 2008 ucsb 1 the vision and reality of ubiquitous computing prof. henning schulzrinne dept....
Post on 15-Jan-2016
219 Views
Preview:
TRANSCRIPT
January 2008 UCSB 1
The Vision and Reality of Ubiquitous Computing
Prof. Henning Schulzrinne
Dept. of Computer Science
Columbia University(with Arezu Moghadam, Ron Shacham, Suman Srinivasan,
Xiaotao Wu and other IRT members as well as the IETF GEOPRIV and ECRIT WGs)
January 2008 UCSB 2
Overview
• The original vision of ubiquitous computing
• User challenges
• Beyond terminal mobility
• Location as new core service
• Beyond connectivity: 7DS
• CS technology into reality
January 2008 UCSB 3
Ubiquitous computing ubiquitous communications
• “It is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere.”
• Weiser’s original vision (“Nomadic Issues in Ubiquitous Computing”, 1996)– “one person, many computers”– many computers embedded in environment– dynamic ownership– PC phonebooth– “IR use will grow rapidly”
• Updated version, 2007– not physically invisible, but transparent– emphasis on communications, not computing– most devices are mobile (or nomadic)– cheap electronics personal devices– radio (channelized and UWB)
January 2008 UCSB 4
Overview
• The original vision of ubiquitous computing
• User challenges• Beyond terminal mobility• Location as new core service• Universality: 7DS• CS technology into reality
January 2008 UCSB 5
User challenges vs. research challenges
• Are we addressing real user needs?– Engineering vs. sports scoring
• My guesses
reliability
ease of use
cost
no manual
integration
limited risk
phishingdata loss
no re-entryno duplication
January 2008 UCSB 6
Example: Email configuration
• Application configuration for (mobile) devices painful
• SMTP port 25 vs. 587• IMAP vs. POP• TLS vs. SSL vs. “secure
authentication”• Worse for SIP...
January 2008 UCSB 7
Example: SIP configuration
• highly technical parameters, with differing names• inconsistent conventions for user and realm• made worse by limited end systems (configure by multi-tap)• usually fails with some cryptic error message and no
indication which parameter• out-of-box experience not good
partially explains
January 2008 UCSB 8
Mobile why’s
• Not research, but examples of real annoyances• Why does each mobile device need its own power supply?• Why do I have to adjust the clock on my camera each time I travel?• Why do I have to know what my IMAP server is and whether it uses
TLS or SSL?• Why do I have to type in my address book?• Why do I have to “synchronize” my PDA?• Why do I have to manually update software?• Why is connecting a laptop to a projector a gamble?• Why do we use USB memory sticks when all laptops have 802.11b?
January 2008 UCSB 9
Consumer wireless & mobile devices
MSN Direct weather
Prius key Garage door opener TAN display
Water leak alarm
wireless door bell
SPOT watch
January 2008 UCSB 10
Mobile systems - reality
• idea: special purpose (phone) --> universal communicator
– idea is easy...
• mobile equipment: laptop + phone– sufficiently different UI and capabilities
• we all know the ideal (converged) cell phone
• difficulty is not technology, but integration and programmability
– (almost) each phone has a different flavor of OS
– doesn’t implement all functionality in Java APIs
– no dominant vendor (see UNIX/Linux vs. Microsoft)
– external interfaces crippled or unavailable• e.g., phone book access
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
GPS
location data
January 2008 UCSB 11
Example: displays and speakers
January 2008 UCSB 13
The mobile ubiquitous challenge
Mobile phone
Mobile Internet access
Interconnected devices“Internet of things”
January 2008 UCSB 14
What do we need?
• Standards, not new technology• Radio connectivity
– 802.11a/b/g/n, 802.15.4 – better discovery of networks
• Location information everywhere• Discovery: devices & services
– network-local discovery via Bonjour (mDNS) – missing: location-based discovery
• Advanced mobility: session, personal, service• Event notification• Data formats
– location, sensor events, ...
January 2008 UCSB 15
Examples of “invisible” behavior
• MP3 player in car automatically picks up new files in home server• A new email with vcard attachment automatically updates my cell
phone address book• The display of my laptop appears on the local projector
– without cable or configuration
• I can call people I just met at UCSB– without exchanging business cards
• My car key opens my front door• My cell phone serves as a TAN (one-time password) generator• My cell phone automatically turns itself off during a lecture• My camera knows where the picture was taken
January 2008 UCSB 16
An interconnected system
any weather serviceschool closings
opens doors
incoming call
generates TAN
acoustic alerts
updates location
time, location
alert, events
address book
January 2008 UCSB 17
Thinking beyond 802.11 and UMTS
• Many interesting networks beyond those covered in conferences– ease of access by researchers vs. importance
– 90% of papers on 802.11b and maybe GPRS, BlueTooth
• New wireless networks– broadcast instead of unicast -- useful for many ubiquitous applications
– S5 for low-rate sensors (city scale)
– Zigbee (802.15.4) for local sensors (20 - 250 kb/s)
– FM subcarrier (not really new) -- MSN Direct
– FM Radio Data System -- TMC
– Sirius / XM
– HD radio
– paging
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
January 2008 UCSB 18
Overview
• The original vision of ubiquitous computing
• User challenges
• Beyond terminal mobility
• Location as new core service
• Universality: 7DS
• CS technology into reality
January 2008 UCSB 19
Application-layer mobility
• terminal mobility– one terminal, multiple network addresses
• Personal mobility– one person, multiple terminals– e.g., Grandcentral
• session mobility– one user, multiple terminals in sequence or in parallel
• service mobility– services move with user
January 2008 UCSB 20
Session mobility
• Walk into office, switch from cell phone to desk phone
– call transfer problem SIP REFER
• related problem: split session across end devices
– e.g., wall display + desk phone + PC for collaborative application
– assume devices (or stand-ins) are SIP-enabled
– third-party call control
January 2008 UCSB 21
How to find services?
• Two complementary developments:– smaller devices carried on user instead of stationary
devices– devices that can be time-shared
• large plasma displays• projector• hi-res cameras• echo-canceling speaker systems• wide-area network access
• Need to discover services in local environment– SLP (Service Location Protocol) allows querying for
services• “find all color displays with at least XGA resolution”• slp://example.com/SrvRqst?public?type=printer
– SLP in multicast mode– SLP in DA mode– Apple Bonjour
• Need to discover services before getting to environment
– “is there a camera in the meeting room?”– SLP extension: find remote DA via DNS SRV– LoST to find services by geographic location
January 2008 UCSB 22
Internet
CorrespondentNode (CN)
SIP UA
SLP UA
SIP SM
Local Devices
SLP SA SLP UA
SIP SM SIP UA
SLP DA
Mobile Node (MN)
SLPSIPRTP
SIP UA
Transcoder
Session mobility
January 2008 UCSB 23
Overview
• The original vision of ubiquitous computing
• User challenges• Beyond terminal mobility• Location as new core service• Universality: 7DS• CS technology into reality
January 2008 UCSB 24
Context-aware communication
• context = “the interrelated conditions in which something exists or occurs”
• anything known about the participants in the (potential) communication relationship
• both at caller and callee
time chronology, uniqueness
capabilities caller preferences
location location-based call routing
location events (emergency alerts)
security (access control)
service discovery
activity/availability presence
sensor data (mood, bio) privacy issues similar to location data
January 2008 UCSB 25
GEOPRIV and SIMPLE architectures
targetlocationserver
locationrecipient
rulemaker
presentity
caller
presenceagent
watcher
callee
GEOPRIV
SIPpresence
SIPcall
PUBLISHNOTIFY
SUBSCRIBE
INVITE
publicationinterface
notificationinterface
XCAP(rules)
INVITE
DHCP
January 2008 UCSB 26
Presence data architecture
rawpresencedocument
createview
(compose)
privacyfiltering
draft-ietf-simple-presence-data-model
compositionpolicy
privacypolicy
presence sources
XCAP XCAP
(not defined yet)
depends on watcherselect best sourceresolve contradictions
PUBLISH
January 2008 UCSB 27
Presence data architecture
candidatepresencedocument
watcherfilter
rawpresencedocument
post-processingcomposition(merging)
finalpresencedocument
differenceto previous notification
SUBSCRIBE
NOTIFY
remove data not of interest
watcher
January 2008 UCSB 28
Presence data model
“calendar” “cell” “manual”
alice@example.comaudio, video, text
r42@example.comvideo
person(presentity)
(views)
services
devices
January 2008 UCSB 29
RPID = rich presence
• Provide watchers with better information about the what, where, how of presentities
• facilitate appropriate communications:– “wait until end of meeting”– “use text messaging instead of phone call”– “make quick call before flight takes off”
• designed to be derivable from calendar information– or provided by sensors in the environment
• allow filtering by “sphere” – the parts of our life– don’t show recreation details to colleagues
January 2008 UCSB 30
Rich presence
• More information: for (authorized) people and applications• automatically derived from
– sensors: physical presence, movement– electronic activity: calendars
• Rich information:– multiple contacts per presentity
• device (cell, PDA, phone, …)• service (“audio”)
– activities, current and planned– sphere (home vs. work)– current user mood– surroundings (noise, privacy, vehicle, …)– contact information– composing (typing, recording audio/video IM, …)
January 2008 UCSB 31
Presence and privacy
• All presence data, particularly location, is highly sensitive
• Basic location object (PIDF-LO) describes
– distribution (binary)– retention duration
• Policy rules for more detailed access control
– who can subscribe to my presence
– who can see what when
<tuple id="sg89ae">
<status>
<gp:geopriv>
<gp:location-info>
<gml:location>
<gml:Point gml:id="point1“
srsName="epsg:4326">
<gml:coordinates>37:46:30N 122:25:10W
</gml:coordinates>
</gml:Point>
</gml:location>
</gp:location-info>
<gp:usage-rules>
<gp:retransmission-allowed>no
</gp:retransmission-allowed>
<gp:retention-expiry>2003-06-23T04:57:29Z
</gp:retention-expiry>
</gp:usage-rules>
</gp:geopriv>
</status>
<timestamp>2003-06-22T20:57:29Z</timestamp>
</tuple>
January 2008 UCSB 32
Events as missing Internet capability
• aka PUB/SUB• Used across applications, e.g.,
– email and voicemail notification
– presence
– replace RSS (= poll!)
– web service completion
– emergency alerts (“reverse 9-1-1”)
– network management
– home control
– data synchronization
• Rich research history– but too complex, optimize the wrong thing
• XMPP and SIP as likely transport candidates
January 2008 UCSB 33
Local Switch
Automatic Number Identification
Automatic Location Identification
Collaboration between local phone providers and local public safety agencies
January 2008 UCSB 34
911 technology failures
• NY Times (“An S O S for 911 Systems in Age of High-Tech”), 4/6/07:– “40% of … counties, most of them rural or small-town …, cannot yet pinpoint the
location of the cellphone callers, though the technology to do so has been available for at least five years.”“In … Okmulgee, Okla., last November, 4-year-old Graciella Mathews-Tiger died in a house fire after a 911 operator who lacked the technology to pinpoint the call misheard the address.”
• Phase II wireless; billions of dollars spent
• In Mississippi, only 1 of out 5 counties
– “As it ages, it is cracking, with problems like system overload, understaffing, misrouted calls and bug-ridden databases leading to unanswered calls and dangerous errors.”
• operator (CAMA) trunks, with 8-digit number delivery
• MSAG and ALI databases
January 2008 UCSB 35
Location delivery
DHCP
HTTP
GPS
HELD
LLDP-MED
wiremap
January 2008 UCSB 36
Location, location, location, ...
Voice Service Provider (VSP)sees emergency call
but does not know caller location
ISP/IAP knows user locationbut does not handle call
January 2008 UCSB 37
Options for location delivery
• Wireless– GPS– S5 wireless (active sensors + triangulation)– Skyhook (802.11 in urban areas)– HDTV
• L2: LLDP-MED (standardized version of CDP + location data)– periodic per-port broadcast of configuration information
• L3: DHCP for– geospatial (RFC 3825)– civic (RFC 4676)
• L7: proposals for retrievals: HELD, SIP, …– for own IP address or by third party (e.g., ISP to infrastructure provider)– by IP address– by MAC address– by identifier (conveyed by DHCP or PPP)– HTTP-based
January 2008 UCSB 38
Locating Caller using LLDP-MED
CALLER EQUIPMENT
LLDP-MED SWITCH
LLDP-MED stands for: *Link Layer Discovery Protocol “a vendor-neutral Layer 2 protocol that allows a network device to advertise its identity and capabilities on the local network.”Media Endpoint Discovery “an enhancement to the LLDP that allows discovery of other things including location “
“I am LLDP-MED Capable.I can process location information.”
“Your location is:500 W 120TH st. New York NY 10027”
* From Wikipedia
January 2008 UCSB 39
Location determination options
Method CDP or LLDP-MED
DHCP HELD GPS manual entry
Layer L2 L3 L7 (HTTP) - user
advantages • simple to implement
• built into switch• direct port/room
mapping
• simple to implement
• network locality
• traverses NATs
• can be operated by L2 provider
• accurate• mobile
devices• no carrier
cooperation
• no infrastructure changes
• no carrier cooperation
problems may be hard to automate for large enterprises
mapping MAC address to location?
mapping IP address to switch port?
• indoor coverage
• acquisition time
• fails for mobile devices
• unreliable for nomadic
Use Ethernet LANs Enterprise LANs
Some ISPs
DSL, cable mobile devices fall back
January 2008 UCSB 40
INVITE urn:service:sos SIP/2.0
To: urn:service:sosCall-ID: 763782461@192.168.1.106Via: SIP/2.0/TCP 192.168.1.106:4064;rportContent-Type: multipart/mixed; boundaryFrom: sip:caller@irt.cs.columbia.eduContact: <sip:eddie@160.39.54.70:5060>CSeq: 1 INVITEContent-Length: 1379
------ =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=MIME-Version: 1.0content-Type: application/sdpContent-Transfer-Encoding: 8bit
v=0o=eddie 1127764654 1127764654 IN IP4 192.168.1.106s=SIPC Callc=IN IP4 160.39.54.70t=0 0m=audio 10000 RTP/AVP 0 3m=video 20000 RTP 31
SDP
header fields
request line ------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=MIME-Version: 1.0Content-Type: application/pidf+xmlContent-Transfer-Encoding: 8bit
<?xml version="1.0" encoding="ISO-8859-1"?><presence xmlns="urn:ietf:params:xml:ns:pidf" xmlns:gp="urn:ietf:params:xml:ns:pidf:geopriv10" xmlns:cl=" urn:ietf:params:xml:ns:pidf:geopriv10:civilLoc" xmlns:gml="urn:opengis:specification:gml:schema-xsd:feature:v3.0" entity="sip:calltaker_ny2@irt.cs.columbia.edu"> <tuple id="28185"> <status> <gp:geopriv> <gp:location-info> <cl:civilAddress> <cl:country>us</cl:country> <cl:A1>ny</cl:A1> <cl:A2>new york</cl:A2> <cl:A3>new york</cl:A3> <cl:A6>amsterdam</cl:A6> <cl:HNO>1214</cl:HNO> </cl:civilAddress> </gp:location-info> <gp:method>Manual</gp:method> </gp:geopriv> </status> <contact priority="0.8">sip:eddie@160.39.54.70:5060</contact> <timestamp>2005-09-26T15:57:34-04:00</timestamp> </tuple></presence>------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=--
PIDF-LO
SIP message for Location Info.
January 2008 UCSB 41
January 2008 UCSB 42
Tracking
January 2008 UCSB 43
Data formats: location
• Civic (street)– jurisdictional & postal
• Geo (longitude & latitude)– point, polygon, circle, …
• see GeoRSS for simple example
January 2008 UCSB 44
ECRIT: LoST Functionality
• Civic as well as geospatial queries– civic address validation
• Recursive and iterative resolution• Fully distributed and hierarchical
deployment– can be split by any geographic or civic
boundary– same civic region can span multiple
LoST servers
<findService xmlns="urn:…:lost1"><location profile="basic-civic"> <civicAddress> <country>Germany</country> <A1>Bavaria</A1> <A3>Munich</A3> <A6>Neu Perlach</A6> <HNO>96</HNO> </civicAddress> </location> <service>urn:service:sos.police</service></findService>
• Indicates errors in civic location data debugging
– but provides best-effort resolution• Can be used for non-emergency services:
– directory and information services– pizza delivery services, towing companies, …
January 2008 UCSB 45
LoST: Location-to-URL Mapping
clusterserves VSP2
NYUS
NJUS
Bergen CountyNJ US
123 Broad AveLeoniaBergen CountyNJ US
cluster serving VSP1
replicateroot information
searchreferral
rootnodes
LeoniaNJ US
sip:psap@leonianj.gov
VSP1
LoST
January 2008 UCSB 46
LoST Architecture
T1
(.us)
T2
(.de) T3
(.dk)
G
G
GG
G broadcast (gossip)T1: .us
T2: .de
resolver
seeker313 Westview
Leonia, NJ US
Leonia, NJ sip:psap@leonianj.gov
tree guide
January 2008 UCSB 47
Left to do: event notification
• notify (small) group of users when something of interest happens– presence = change of communications state– email, voicemail alerts– environmental conditions– vehicle status– emergency alerts
• kludges– HTTP with pending response– inverse HTTP --> doesn’t work with NATs
• Lots of research (e.g., SIENA)
• IETF efforts starting– SIP-based– XMPP
January 2008 UCSB 48
Overview
• The original vision of ubiquitous computing
• User challenges• Beyond terminal mobility• Location as new core service• Universality: 7DS• CS technology into reality
January 2008 UCSB 49
Problems with Wide Area Wireless
• 802.11 currently hard to deploy across city or large area• Problem: How can mobile devices / gadgets get information
while on the move?• Use local peer-to-peer wireless networks to exchange
information– Peers can get information they do not have from another peer
Solution: 7DS!
January 2008 UCSB 50
How 7DS Works
1. When devices are in the same BSS (Basic service set) of 802.11 ad-hoc network, they discover each other using service discovery of Zeroconf
zeroconf
January 2008 UCSB 51
How 7DS Works
Internet
2. If there is no Internet connection, the devices can communicatewith each other to exchange information
January 2008 UCSB 52
Web Delivery Model
• 7DS core functionality: Emulation of web content access and e-mail delivery
January 2008 UCSB 53
Design
• Peer-to-peer network set up using zeroconf– Protocol enables devices to communicate with each other
without a DHCP server, a DNS server and a Directory server
• Proxy server serves content• Search engine searches for local data• MTA store and forward• In progress: File synchronization, BBS
January 2008 UCSB 54
Store and Forward
• Forwarding e-mail in the ad-hoc network• Acts as an MTA
January 2008 UCSB 55
Search Engine
• Provides ability to query self for results
• Searches the cache index using Swish-e library
• Presents results in any of three formats: HTML, XML and plain text
• Similar in concept to Google Desktop
January 2008 UCSB 56
Query Multicast Engine
• Used to actually exchange information among peers
• Requesting peer broadcasts a query to the network
• Responding peers reply if they have information
– Send encoded string with list of matching items
• Requesting peer retrieves suitable information
January 2008 UCSB 57
File Synchronization
SRV : 7ds-fs1.filesync._7ds._udp.local.
7ds-device1.local:2525
TXT : file1.xml
TXT : file2.xml
SRV : 7ds-fs2.filesync._7ds._udp.local.
7ds-device2.local:2525
TXT : word.doc
TXT : presentation.pptFile1.xmlFile2.xml
Word.docPresentation.ppt
“I wantWord.doc and presentation.ppt”
“I wantFile1.xml and file2.xml”
Word.docPresentation.ppt
File1.xmlFile2.xml
SERVICE ANNOUNCEMENTSERVICE RESOLUTIONFILE SYNCHRONIZATION USING RSYNC PROTOCOL
January 2008 UCSB 58
Overview
• The original vision of ubiquitous computing
• User challenges• Beyond terminal mobility• Location as new core service• Universality: 7DS• CS technology into reality
January 2008 UCSB 59
CS research to reality
CS as science
CS as engineering
January 2008 UCSB 60
CS tech transfer, mode 1
January 2008 UCSB 61
CS tech transfer, mode 2
January 2008 UCSB 62
The standards route
January 2008 UCSB 63
Role of standards
• Widespread use of technology requires standards– railroad gauges, nuts & bolts, Morse code, phone system
• industrial age = interchangeable parts
– CD ROM, DVD, HD-DVD/BlueRay– JPEG, MPEG, ...– SQL– C, Java
• Particularly for network technology– many mid-size actors– “network effect” = utility increases with number of users
• e.g., cars have inverse network effects
• chess game vs. email and word processor
– DECnet, SNA, ARCnet Ethernet, IP– “hypermedia” HTML, HTTP
January 2008 UCSB 64
Standards
• Tanenbaum: “The nice thing about standards is that you have so many to choose from.”
VHS + BetaEthernet + TokenringATM + IP
!
component technologies
January 2008 UCSB 65
Standards: technology translator
• Similar in some ways to text books• “accepted technology”
– lower/known risks (“vetted”)– infrastructure (“eco system”)
• requires expertise and broader training– many CS standards don’t have either
• example: HTTP/1.0, HTML 1.0
• thus, way to provide technology leadership
January 2008 UCSB 66
Examples of standards needed
• each with multi-B$ impact:– medical records– energy control– transportation
• inter-vehicle safety systems, traffic alerts, ...
– academic records– business transactions
• e.g., your travel expense report
• requires both CS and domain expertise
January 2008 UCSB 67
Conclusion
• Motivate mobile ubiquitous research by user problems• From stovepipe mobile & wireless systems to personal
and shareable wireless networks• Thinking beyond single applications
– presence event notification– 9-1-1 location time & location as infrastructure
• Need new models of creating services– domain-specific languages, not Java APIs
top related