a provider-side view of web search response time yingying chen, ratul mahajan, baskar sridharan,...
TRANSCRIPT
![Page 1: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/1.jpg)
A Provider-side View ofWeb Search Response Time
YINGYING CHEN, RATUL MAHAJAN,
BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA)
MICROSOFT
![Page 2: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/2.jpg)
Web services are the dominant way to find and access information
![Page 3: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/3.jpg)
Web service latency is critical to service providers as well
Bing
revenue-20%
Latency+2 sec
revenue-4.3%
Latency+0.5 sec
![Page 4: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/4.jpg)
Understanding SRT behavior is challenging
t
300+tS
RT
(m
s)
M T W Th F S Su
peak off-peak
200+t
t
SR
T (
ms)
![Page 5: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/5.jpg)
Our work
Explaining systemic SRT variation
Identify SRT anomalies
Root cause localization
![Page 6: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/6.jpg)
Client- and server-side instrumentation
HTML header
Brand header
BoP scriptsQuery results
Embedded images
query
𝑇 𝑓𝑠 𝑇 𝑓𝑐
𝑇 h𝑒𝑎𝑑
𝑇 𝑏𝑟𝑎𝑛𝑑
𝑇 h𝑖𝑛𝑡𝑐 𝑘1
𝑇 𝑟𝑒𝑠𝐻𝑇𝑀𝐿
𝑇 𝐵𝑂𝑃
𝑇 h𝑖𝑛𝑡𝑐 𝑘2
𝑇 𝑒𝑚𝑏𝑒𝑑
𝑇 𝑟𝑒𝑓
𝑇 𝑠𝑐𝑟𝑖𝑝𝑡
𝑇 𝑠𝑐
𝑇 𝑡𝑐
on-load
Referenced content
![Page 7: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/7.jpg)
Impact Factors of SRT
𝑇 𝑓𝑠
network browser queryserver
𝑇 h𝑒𝑎𝑑𝑇 𝑏𝑟𝑎𝑛𝑑𝑇 h𝑖𝑛𝑡𝑐 𝑘1𝑇 𝑟𝑒𝑠𝐻𝑇𝑀𝐿𝑇 𝐵𝑂𝑃𝑇 h𝑖𝑛𝑡𝑐 𝑘2𝑇 𝑟𝑒𝑓𝑇 𝑠𝑐𝑟𝑖𝑝𝑡𝑇 𝑛𝑒𝑡𝑇 𝑠𝑐𝑇 𝑓𝑐𝑇 𝑒𝑚𝑏𝑒𝑑 𝑇 𝑡𝑐
![Page 8: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/8.jpg)
Primary factors of SRT variation
Apply Analysis of Variance (ANOVA) on the time intervals
ƞ
SRT variance
Variance explained by time interval k
Unexplainedvariance
![Page 9: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/9.jpg)
Primary factors: network characteristics, browser speed, query type Server-side processing time has a relatively small impact
network browser queryserver
𝑇 h𝑒𝑎𝑑𝑇 𝑟𝑒𝑠𝐻𝑇𝑀𝐿𝑇 𝐵𝑂𝑃𝑇 𝑟𝑒𝑓 𝑇 𝑠𝑐𝑟𝑖𝑝𝑡𝑇 𝑛𝑒𝑡 𝑇 𝑠𝑐𝑇 𝑓𝑐 𝑇 𝑡𝑐
Exp
lain
ed
vari
an
ce (
%) 6
0
40
20
0
![Page 10: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/10.jpg)
Variation in network characteristics
RT
T
![Page 11: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/11.jpg)
Explaining network variations
Residential networks send a higher fraction of queries during off-peak hours than peak hours
Residential networks are slower
![Page 12: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/12.jpg)
residential enterprise
RTT
(ms)
25%
1.25t
t
Residential networks are slower
Residential networks send a higher fraction of queries during off-peak hours than peak hours
residential unknownenterprise
![Page 13: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/13.jpg)
Variation in query type
Impact of query on SRT Server processing timeRichness of response page
Measure: number of image
![Page 14: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/14.jpg)
Explaining query type variation
Peak hours Off-peak hours
![Page 15: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/15.jpg)
Browser variations
Two most popular browsers: X(35%), Y(40%) Browser-Y sends a higher fraction of queries during off-peak hours Browser-Y has better performance
Browser-X Browser-Y
Javascript exec time
82%
1.82t
t
![Page 16: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/16.jpg)
Summarizing systemic SRT variation Server: Little impact
Network: Poorer during off-peak hours
Query: Richer during off-peak hours
Browser: Faster during off-peak hours
![Page 17: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/17.jpg)
Detecting anomalous SRT variations
Challenge: interference from systemic variations
![Page 18: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/18.jpg)
Week-over-Week (WoW) approach
+ Seasonality + Noise
![Page 19: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/19.jpg)
![Page 20: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/20.jpg)
![Page 21: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/21.jpg)
![Page 22: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/22.jpg)
Comparison with approaches that do not account for systemic variations
WoW One Gaussian model of
SRT
Change point
detection
False negative 10% 35% 40%
False positive 7% 17% 19%
![Page 23: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/23.jpg)
Conclusions
Understanding SRT is challengingChanges in user demographics lead to systemic
variations in SRT
Debugging SRT is challenging Must factor out systemic variations
![Page 24: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/24.jpg)
Implications
Performance monitoringShould understand performance-equivalent classes
Performance managementShould consider the impact of network, browser, and
query
Performance debugging End-to-end measures are tainted by user behavior
changes
![Page 25: A Provider-side View of Web Search Response Time YINGYING CHEN, RATUL MAHAJAN, BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA) MICROSOFT](https://reader035.vdocuments.us/reader035/viewer/2022081602/5517710b5503460e6e8b4f5e/html5/thumbnails/25.jpg)
Questions?