mobile and location-based services jason i. hong [email protected] may 04 2007
TRANSCRIPT
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
inTouch: Coordination for Families
• Make it easier to coordinate with others while mobile– Better awareness and messaging
Target Users:• Small to med. groups of people• Fluid and demanding schedule• Multiple responsibilities
Examples:• Dual-career families• Work groups• Ad hoc (ex. conferences)• Carpools
Mobility
AwarenessMessaging
Dual-Career Families
• Coordination breakdowns inevitable– Children’s activities change without notice
– Parent’s meetings run over
– Impromptu appointments
– Unexpected traffic
• Result:– High levels of anxiety
– Some parents fear about “forgetting” their children
• Need support for awareness and improvisation
inTouch: Coordination for Families
• Two week field study with six dual-career families
Check, Double Check, Triple Check
0
20
40
60
80
100
120
140
160
180
Status Confirmation Reminder Planning Schedule Reschedule Recall
Message Type
Nu
mb
er o
f M
essa
ges
Key Transition Times
-2
0
2
4
6
8
10
12
14
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour of the Day (1AM, 2AM, etc)
Avg
Nu
mb
er o
f M
essa
ges
inTouch: Coordination for Families
• Make it easier to coordinate with others while mobile– Better awareness
– Contextual messaging
Combines:• Shared calendar• Shared todo lists• Reminders• Real-time location• Proximity
Project: InTouch
It’s 4:30pm and Mom is stuck in traffic
inTouch checks her calendar and sees she’s supposed to pick up Cindy from ballet
Project: InTouch
Mom’s phone senses that she is in a traffic jam, and automatically prepares a status message
Mom hits “send”, and Cindy sees that Mom is running late. Cindy decides to wait inside.
Contextual Messaging
• Using current context to: – Select a message template
– Fill in the blanks (like a MadLib)
– In most cases, can just hit “send”
• When is contextual messaging useful?– Calendar alarms “running late, will be there in <ETA>”
– Current activity “in a meeting now, done at <time>”
– Daily rhythms “picked up kid ok” at 3PM
– Messages received “where r u?” -> “I am at <place>”
Contextual Messaging
• Messaging can be linked to calendar or reminders– S: Can you get dinner tonight?
– J: Ok, I will pick up __________ on my way home
– Activate as a reminder when you leave work
Message easy to select around 4PM
Fill in the blank based on patternsand what’s near your home
Example Mockups
• Currently developing working prototypes
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
Whisper Mobile
• Goal: Make it easy to find, share, and coordinate friends going to social events
Whisper Mobile: Creating an Event
• Minimal text input– Use location
– Use audio
– Use camera
Continuing Work
• Developing working prototype of web site and mobile– Web crawler for finding social events
– Web site to coordinate on scale of weeks and days
• Link with inTouch– Coordinate friends
– See who’s late, where we’re going next
– Mobile to coordinate on scale of hours and minutes
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
Project: Hitchhiking
• Most location-based services about where you are• Hitchhiking is about the “busyness” of places
– “Is the café busy?”
– “How long are the lines at the airport?”
– “Where’s an empty room?”
– Is there any parking at the shopping district?
Project: Hitchhiking
• Estimate number of people in a place by counting the number of wireless devices there
• Periodically upload count + location to our servers• Other people can query our servers
Project: Hitchhiking
• How well does Hitchhiking work?
Project: Hitchhiking
• Privacy?– Upload anonymized counts only
– Upload from approved places only
– Our server shows “busyness” of a place only
• Advantages– Cheap, uses existing devices (everyone is a “sensor”)
– Deployable, don’t have to set up lots of new sensors
– Privacy
• What’s next?– Map visualizations
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
GurunGo
• Goal: Make it easy to access useful information while mobile
• Observation #1: People still tend to print out online maps, despite having mobile device. Why?– Found it via desktop, easier to print than to copy to mobile
– Slow or expensive wireless connections
– Inconvenient form factor on mobile device
• Observation #2: People don’t do the same kind of web browsing on mobile phones as on desktops– Don’t have to support all information finding tasks,
just ones more likely to be done when mobile
GurunGo Scenarios
• Idea: Tie mobile more closely with desktop
• You find an interesting product while browsing– Use GurunGo to copy-and-paste to mobile
– Augments with product reviews
– Copies to mobile
– Kept until explicitly deleted
• As you browse web on desktop:– GurunGo scans HTML for maps
– Generates speech-based directions
– Copies to mobile
– Directions eventually discarded after given time
GurunGo Usage
• Acquire– Let people explicitly copy-and-paste info to mobile
– Let people implicitly copy info via regular web browsing• GurunGo scans pages seen for potentially useful stuff
• Augment– Look for known data types, make mobile data more useful
– Ex. Augment maps with speech-based directions
• Copy (to mobile in the background)• Browse
– Organize data based on common data types
– Street addresses, product comparisons, phone #s
GurunGo: Speech-based Directions
Nice Features of GurunGo
• Reduces number of clicks to get to useful information– Can support specific information finding tasks while mobile
– Currently: Directions, products
– Future: Movies, phone #s, dates and times, recent emails
• Works even if you don’t have wide-area wireless– Works disconnected (no network or don’t want to pay)
– Only needs personal area network (Bluetooth)
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
The Problem
• Mobile devices becoming integrated into everyday life– Mobile communication
– Sharing location information with others
– Remote access to home
– Mobile e-commerce
• Managing security and privacy policies is hard– Preferences hard to articulate
– Policies hard to specify
– Limited input and output
• Leads to new sources of vulnerability and frustration
Our Goal
• Develop core set of technologies for managing privacy and security on mobile devices– Simple UIs for specifying policies
– Clear notifications and explanations of what happened
– Better visualizations to summarize results
– Machine learning for learning preferences
– Start with small evaluations, continue with large-scale ones
• Large multi-disciplinary team and project– Six faculty, 1.5 postdocs, six students
– Supported by NSF, CMU CyLab
– Roughly 1 year into project
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
Contextual Instant Messaging
• Facilitate coordination and communication by letting people request contextual information via IM– Interruptibility (via SUBTLE toolkit)
– Location (via Place Lab WiFi positioning)
– Active window
• Developed a custom client and robot on top of AIM– Client (Trillian plugin) captures and sends context to robot
– People can query imbuddy411 robot for info• “howbusyis username”
– Robot also contains privacy rules governing disclosure
Contextual Instant MessagingPrivacy Mechanisms
• Web-based specification of privacy preferences– Users can create groups and
put screennames into groups
– Users can specify what each group can see
Contextual Instant MessagingPrivacy Mechanisms
• Notifications of requests
Contextual Instant MessagingPrivacy Mechanisms
• Social translucency
Contextual Instant MessagingPrivacy Mechanisms
• Audit logs
Contextual Instant MessagingEvaluation
• Recruited ten people for two weeks– Selected people highly active in IM (ie undergrads )
– Each participant had ~90 buddies and 1300 incoming and outgoing messages per week
• Notified other parties of imbuddy411 service– Update AIM profile to advertise
– Would notify other parties at start of conversation
Contextual Instant MessagingResults
• Total of 242 requests for contextual information– 53 distinct screen names, 13 repeat users
0
20
40
60
80
100
120
Interruptibility Location Active Window
Contextual Instant MessagingResults
• 43 privacy groups, ~4 per participant– Groups organized as class, major, clubs,
gender, work, location, ethnicity, family
– 6 groups revealed no information
– 7 groups disclosed all information
• Only two instances of changes to rules– In both cases, friend asked participant to
increase level of disclosure
Contextual Instant MessagingResults
• Likert scale survey at end – 1 is strongly disagree, 5 is strongly agree
– All participants agreed contextual information sensitive• Interruptibility 3.6, location 4.1, window 4.9
– Participants were comfortable using our controls (4.1)
– Easy to understand (4.4) and modify (4.2)
– Good sense of who had seen what (3.9)
• Participants also suggested improvements– Notification of offline requests
– Better notifications to reduce interruptions (abnormal use)
– Better summaries (“User x asked for location 5 times today”)
Contextual Instant MessagingCurrent Status
• Preparing for another round of deployment– Larger group of people
– A few more kinds of contextual information
• Developing privacy controls that scale better– More people, more kinds of information
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
People Finder
• Location useful for micro-coordination– Meeting up
– Okayness checking
• Developed phone-based client– GSM localization (Intel)
• Conducted studies to see how people specify rules (& how well)
• See how well machine learning can learn preferences
People FinderMachine Learning
• Using case-based reasoning (CBR)– “My colleagues can only see my location on
weekdays and only between 8am and 6pm”
– It’s now 6:15pm, so the CBR might allow, or interactively ask
• Chose CBR over other machine learning– Better dialogs with users (ie more understandable)
– Can be done as you go (rather than accumulating large corpus and doing post-hoc)
People FinderCurrent Work
• Small-scale deployment of phone-based People Finder with a group of friends– Still needs more value, people finder by itself not sufficient
– Trying to understand pain points on next iteration
• Need more accurate location– GSM localization accuracy haphazard
• Integration with imbuddy411– Smart phones expensive, IM vastly increases user base
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
Grey – Access Control to Resources
• Distributed smartphone-based access control system – physical resources like office doors,
computers, and coke machines
– electronic ones like computer accounts and electronic files
– currently only physical doors
• Proofs assembled from credentials– No central access control list
– End-users can create flexible policies
GreyCreating Policies
• Proactive policies– Manually create a policy beforehand
– “Alice can always enter my office”
• Reactive policies– Create a policy based on a request
– “Can I get into your office?”
– Grey sees who is responsible for resource, and forwards• Might select from multiple people (owner, secretary, etc)
– Can add the user, add time limits too
GreyDeployment at CMU
• 25 participants (9 part of the Grey team)• Floor plan with Grey-enabled Bluetooth doors
GreyEvaluation
• Monitored Grey usage over several months• Interviews with each participant every 4-8 weeks• Time on task in using a shared kitchen door
GreySurprises
• Grey policies did not mirror physical keys– Grey more flexible and easier to change
• Lots of non-research obstacles– user perception that the system was slow
– system failures causing users to get locked out
– need network effects to study some interesting issues
• Security is about unauthorized users out, our users more concerned with how easy for them to get in– never mentioned security concerns when interviewed
GreyCurrent work in Visualizations
The Big Picture
• Mobile social computing– inTouch: Coordination for Families and Small Groups– Whisper Mobile: Coordinating groups for social events
• Large-scale mobile collaboration– Hitchhiking: Estimating “busyness” of places
• Mobile data– Gurungo: linking desktop and mobile devices
• Usable privacy and security– Contextual Instant Messaging– People Finder– Grey: Access control to resources
• Memory support– Memory Karaoke
Memory Karaoke
• Phone-based system for preventing cognitive decline– Take pictures with camera phone
– Tag with location, time
– Tell stories about them
Summary
• Mobile social computing• Large-scale mobile
collaboration• Mobile data• Usable privacy and
security• Memory support
Jason I. Hong
• NSF• DARPA• Microsoft SenseMap• Motorola• Nokia
Lots of Large-Scale Mobile Apps
• Gawker Stalker
Lots of Large-Scale Mobile Apps
• One-way Matchmaking
GreyResults of Time on Task of a Shared Kitchen Door
GreyResults of Time on Task of a Shared Kitchen Door
GreyResults of Time on Task of a Shared Kitchen Door