university of notre dame college of engineering lessons learned from the netsense smartphone study...
TRANSCRIPT
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Lessons Learned from theNetSense Smartphone Study
Aaron Striegel1, Shu Liu1, Lei Meng1, Christian Poellabauer1, David Hachen2, Omar Lizardo2
1Department of Computer Science and Engineering2Department of SociologyUniversity of Notre Dame
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Problem Space
Wireless Data Tsunami
Small Cells
Indoor Coverage
IoT QoE / Analytics
Segmented Industry
Standards Bound
How do we get good data at a reasonable scale?
! 5x Grad Students + 1-3 Friends
NewDevices
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Why is it Hard?
PrivacyInvasive – Phone Agent
Great Data – Social, Location
ExpensiveRaw Cost - $60+ / month
Data Scale - Granularity
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Talk Premise
Provide a retrospective of theNetSense study and futuredirections / study evolution
PhoneLab – Geoffrey ChallenSUNY Buffalo
LiveLabs – Archan MisraSingapore Management University
On-Going Efforts
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
NetSense Cell Phone Study
Study how “always-on” networking (i.e. smartphones)impacts how we make / keep friends
Provide 200 smart devices to incomingfreshmen at Notre Dame (Aug 2011)
Sociological
Proximity, Facebook,Phone Calls
Technical
Impact on Usage
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Funding “No bucks, no Buck Rodgers”
NSF SoCS grant - $750k
Three years – 1 year prep + 2 years data200 users via “smart” devices
$75k fordevices
+200x Nexus S
4G200 anytime
minsUnlimited data,
text
=
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Approvals
Human Subjects Institutional Review Board (IRB)
Consent Forms Deception vs. Transparency
What is being gathered? How often?
Data Security / Access
Clear Policies – Overages, Breakages, Repairs
Collaborate / Educate Yourself Early
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
NetSense Agent – Year 1
• Reflashed with Cyanogenmod• Gingerbread / ICS (2.4+)• “Full” root - monitoring• Proximity via Bluetooth• Mandated w/WiFi on
Agent application Local SQLite database Threads for “instruments” Periodically phones home
Communications SMS, Call, E-MailUsage Apps, TonnageEnvironment Battery, WiFi APs, B/T
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Study Mechanics
Distribute / manage 200 phones
Reflash 200 phones w/CyanogenMod
Distribute 200 phones to the student
Week before classes
Cohort selection
Notification, response
ND Dorm Assignment - July
Weekly meetings for 1x year
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Check-In Architecture
Next-generation Play Store basedXML Configurability, Rendezvous Server
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Active Devices Over Time(Mar 2012) Criterion
Must report w/ sufficient data (> 100 reports, on avg per day)
Threshold for evaluating “good” users who are fully compliant
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Active Devices (Mar 2012)
Coverage (indoor)plays a huge role
in battery life
Battery life istricky to balance
Provider upgradeswere
“challenging”
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Phone Repairs / EffortSD Card (Green) Peaked at end of 1st year Nexus S did not have removable SD card
Critical to havealways-on student / liaison to keep tabs
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Quiz Response RateMultiple choice quizzes Do you use WiFi? A. Yes B. No
Much worse response rates for long-style tie / behavioral quizzes
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
So what did you learn?
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Lessons Learned
Phone reimbursement / reward for participation
Purchased phone (subsidized) + free plan
$100 (Galaxy S3), $200 (Galaxy S4)
Hardship cases?
Subsidy for compliance via pre-paid cards$40 / month for arm band, phone agent, laptop agent
Non-compliantusers?
Free phone + plan
University-owned phonesCustom ROM?
Coercive?
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Lessons Learned
Points of Contact are Key
Undergraduate liaisons S. Pentland @ MIT
Paid, < 10 hours per week
Configurable Sampling is Essential
Probabilistic, Triggered SamplingDon’t Log It – GoneHow to Measure User QoE
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Lessons LearnedCohort Effects
How big of a sample do you need?
Is your population biased?
Can you quantify the extent of bias?
Device BreadthIs a phone enough?
How fine of data do you need?
Per-packet RSSI? PHY layer? Tower info?
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Collaborating
Can I get access to your data?
Yes*Depending on what data you want
Can I run experiments on your phones?
Probably not
SUNY PhoneLab
Singapore LiveLabs Urban Lifestyle Innovation Program
Cohort through May 2015 Cohort of +500 in August 2015*
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Potential for RelayingTo Appear at MobiCom 2013
UNIVERSITY of NOTRE DAMECOLLEGE of ENGINEERING
Overview
• Problem Space• Overview of NetSense Study Goals• Study Process• Funding• IRB Approval• Architecture
• Response / Compliance• Lessons Learned / Looking Forward