timecard : controlling user-perceived delays in server-based mobile applications
DESCRIPTION
Timecard : Controlling User-Perceived Delays in Server-Based Mobile Applications. Lenin Ravindranath, Jitendra Padhye, Ratul Mahajan, Hari Balakrishnan. MIT, Microsoft Research. Cloud Services. Mobile Apps. Response Time Matters. Users are impatient. Rendering. User interaction. - PowerPoint PPT PresentationTRANSCRIPT
Timecard:Controlling User-Perceived Delays in
Server-Based Mobile Applications
MIT, Microsoft Research
Lenin Ravindranath, Jitendra Padhye, Ratul Mahajan, Hari Balakrishnan
Cloud Services
Mobile Apps
Response Time MattersUsers are impatient
User interaction Rendering
User-perceived delay
Tightly control
App App
ServerUplink Downlink
User interaction Rendering
Server processing delay
App processing delay
Appprocessing delayRequest
transfer delayResponse
transfer delay
User-perceived delay
Tightly control
ServerUplink Downlink
Server delayControl
App AppUser interaction Rendering
Server processing delay
User-perceived delay
Tightly control
Control Server Delay
Request Response
Deadline
Server processing
Controlling the Server Delay
• Trade-off quality of result for processing time– More time to process, better quality results
Server
Request Response
Deadline
Controlling the Server Delay
WorkerWorker
Worker
Worker
Worker
Server
Request Response
Deadline
Controlling the Server Delay
WorkerWorker
Worker
Worker
Worker
Better result
Server
Deadline
Controlling the Server Delay
Server
Server processing
• Servers keep fixed deadlines– No consideration about external delays– Assume constant external delays
Assumed external delay
Assumed external delay
Device type Data size
TCP stateTCP state
Device type
Significant Variability in External Delays
App App
Server
User interaction Rendering
Server processing
Reading sensors
Radio state
Network(3G, WiFi, LTE, ..)
RTT, tput
Highly variable[AppInsight – OSDI ’12]
Significant Variability in External Delays
Server
• Client with high external delays– Poor end-to-end response time
• Client with low external delays– Do not produce the best quality result
Deadline
Significant Variability in External Delays
Server
Servers should adapt to external delaysto control the user-perceived delay
Significant Variability in External Delays
Server
Deadline
Significant Variability in External Delays
Server
Adapt
Significant Variability in External Delays
Server
User interaction Rendering
End-to-end deadline
Adapt
App App
Timecard
App App
Server
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Time elapsed since user interaction
Predicted downlink & app processing delay
Adapt
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Desired end-to-end delay
Adapt Processing Time
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Adapt Processing Time
Trade-off on quality of the result
Desired end-to-end delay
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Adapt Response
Desired end-to-end delay
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Adapt Response
Desired end-to-end delay
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Better quality result
Desired end-to-end delay
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Time elapsed since user interaction
Predicted downlink & app processing delay
Challenges
Server
App App
Challenges
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Server
App
UI dispatcherWeb callback
AppHighly asynchronous
Server Threads
Challenges
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
Transaction
Highly asynchronous
Challenges
Server
App App
GetElapsedTime();
No single reference clock
Variable network delaysand processing delays
PredictRemainingTime(responseSize);
Transaction
Highly asynchronous
Timecard
Server
App App
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Transaction
Transaction Tracking
TimeSync Delay Predictors
Timecard
App
App Instrument
er
InstrumentedApp
Service
Developer
Timecard.dll
GetElapsedTime();PredictRemainingTime
(responseSize);
Desired end-to-end delay
config
App Store
Timecard
Server
App App
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Transaction
Transaction Tracking
TimeSync Delay Predictors
Timecard
Server
App App
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Transaction
Transaction Tracking
TimeSync Delay Predictors
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
Transaction
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Track across thread boundaries o at the app and at the server
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
Track between app and server
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
Transaction
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
TC
TC
TC
TC
TC
TC
TC
TC
TC
TC
Transaction context (TC) attached to every thread– Carry along timestamps, transaction information
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
TC
TC
TC
TC
• Event handler – create new Transaction context (TC)– Timestamp and attach to thread
• Asynchronous call – Pass TC from current thread to callback thread– Pass TC by detouring callbacks [AppInsight – OSDI ‘12]
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
TC
TC
TC
TC
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
TC
TC
TC
TC
TC
HTTP[“x-Timecard-Request”]
TC
• Web Request – pass TC from current thread to server– Encode TC in a special HTTP header
• Request handler at server – Parse TC and attach to server thread
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
TC
TC
TC
TC
TC
TC
Server Threads
Transaction Tracking
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
Spawn workers
Send response
UI dispatcherWeb callback
Server
App App
Request handler
TC
TC
TC
TC
TC
TC
TC
TC
TC
TC
TC
GetElapsedTime();
Timecard
Server
App App
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Transaction
Transaction Tracking
TimeSync Delay Predictors
TimeSync
GPS
• Sync with Cell Tower– Off my several seconds to minutes
• GPS– Does not work indoors
• Probing
Probe
TimeSync
Instrument app to automatically send probes
Probe
Symmetric Asymmetric
Clock offsetClock drift
Smartphone clock Server clock
TimeSync
Probe
Radio wakeup delay
with large extra delay
Active
IdleIdle
Network Radio States
TimeSync
Queuing delay
with large extra delay
Probe
Network traffic
Active
Idle
Active
TimeSync
Network busy
Radio idle
> 100ms error
min RTT
TimeSync
Optimal time to send the probes Radio is active No network traffic from the phone
Need to be radio-aware and network-aware
Probe
TimeSync
Server
App App
TimeSync
UI Thread
Background Thread
Background Thread
Thread start
GPS start
Web requestGPS callback
Event handler
App
Probe
TimeSync
~5ms error
Network busy
Radio idleTimecard
min RTT
Timecard
App App
Server
GetElapsedTime();
Time elapsed since user interaction
Predicting Remaining Time
App App
Server
PredictRemainingTime(responseSize);
Downlink delay App processing delay
Predicting Downlink Delay
Response sizeLatencyThroughput
Loss rate
TCP window state• Most transfers are short– Median – 3KB– 90% percentile – 37KB
Analysis of 4000 apps
• Cellular networks– High-bandwidth, rare-loss,
high-latency,
• 99% transfers over HTTP– TCP window state matters -> multiple RTTs
Predicting Downlink Delay
TCP Window
1 extra RTT
Predicting Downlink Delay
Response sizeLatency
TCP parameters
Determined by RTT Number of round trips
Predicting Downlink Delay
• Use recent estimate of RTT– Use time sync probes
Read TCP window state
• Read TCP window state at serverMiddlebo
x• Do not work with middleboxes
– Split-TCP connection
• TCP window state at the middlebox matters– No easy way to estimate
TCP window state
Build an empirical data-driven model
RTT
Number of round trips
Predicting Downlink DelayPredictRemainingTime(responseSize);
Learn
• Decision tree model– Response size– Recent RTT– Network provider– Bytes already transferred in
the TCP connection o Proxy for TCP window state at the
middlebox
Middlebox
Downlink Delay Predictor
• 20 users + controlled experiments– 1 month, two cities
• 250,000 downloads– 1 KB to 500KB
• AT&T, T-Mobile, Sprint, Verizon, Wi-Fi– 3G, 4G (HSPA+), LTE
• Indoors, outdoors, static, walking, driving
Predicting Downlink Delay
• Split the data into training and testing
Wi-Fi Median error - 12 ms, 90th percentile - 31ms
Cellular Median error - 17 ms, 90th percentile - 86 ms
Predicting App Processing Delay
App App
Server
PredictRemainingTime(responseSize);
Predicting Remaining Time
App App
Server
PredictRemainingTime(responseSize);
Processing Delay Predictor
Downlink Delay Predictor
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
D
e rd = D – e – r
Desired end-to-end delay
• Mobile ads service• Twitter analysis service
Deployment
Mobile Ads ServiceContextual ads to mobile apps [Mobisys ’13]
Mobile Ads Service
Fetch and process ads for keywords
Extract keywords Render adSend keywordsBest Ad
Mobile Ads Service
Extract keywords Render adSend keywordsAd
Ad providerAd provider
Ad provider
Ad provider
Ad provider
A
Keywords
Mobile Ads Service
Extract keywords Render adSend keywordsAd
Mobile Ads Service
Extract keywords Render adSend keywordsAd
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Adapt Processing Time
Mobile Ads Service• ~200,000 transactions• Significant variability in external delay
Within 50ms of the target delay 90% of the time
With Timecard
Without Timecard
Target delay
• Mobile ads service• Twitter analysis service
Deployment
Twitter Analysis Service
Fetch from Twitter and analyze
Get keyword Parse and renderSend keywordScore and tweets
Twitter Analysis Service
Fetch from Twitter and analyze
Get keyword Parse and renderSend keywordScore and tweets
Twitter Analysis ServiceGetElapsedTime(
); PredictRemainingTime(responseSize);
Adapt Processing Time
Adapt Response
Get keyword Parse and renderSend keywordScore and tweets
10KB to 50 KB
Twitter Analysis Service• ~150,000 transactions• Adapt server processing time in steps of 150ms
With Timecard
Without Timecard
Target delay
Tightly control end-to-end delay with Timecard
• 0.1% runtime overhead
• Less than 1% memory overhead
• Less than 1% network overhead
• Negligible battery overhead
Timecard Overhead
Limitations
• Bootstrapping– Training the downlink and app processing delay predictors– Learn app processing delay for every transaction type– Turn off adaptation till enough data is collected
• Complex transactions– Multiple parallel or serial requests– Elapsed time will work!– Need to model complex interactions or need developer help
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Adapt Processing Time
Adapt Response
End-to-end deadline
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Adapt Resources Used
End-to-end deadline
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);Request Prioritization
End-to-end deadline
Timecard
GetElapsedTime();
PredictRemainingTime(responseSize);
Deadlines
Scheduler
• Controlling end-to-end delays– Elapsed time– Prediction on remaining time
• Can be applied in several context
Take-Away
Adapt quality of the paper
SOSP Deadline
GetElapsedTime();
PredictRemainingTime(paper);
Timecard
App App
Server
GetElapsedTime(); PredictRemainingTi
me(responseSize);
Time elapsed since user interaction
Predicted downlink & app processing delay
Backup
Predicting App Processing Delay
AppResponse size
• Correlated with response size
Analysis of AppInsight data 1653 types of transactions
Response size (KB)
Proc
essin
g de
lay
(ms)
Predicting App Processing Delay
AppResponse size• Parsing delay
– Correlated with data size• Rendering delay– Correlated with data size
• Dependent on device type– Processing speed
• Correlated with response size
Analysis of AppInsight data 1653 types of transactions
Predicting App Processing Delay
App
PredictRemainingTime(responseSize);
Processing Delay Predictor
Learn
• Decision tree model– Response size– Device type
Analysis of AppInsight data 1653 types of transactions
Median error - 8ms90th percentile error - 100ms