analyzing throughput and stability in cellular networks...the comparison across mnos shows a large...
Post on 31-Mar-2020
1 Views
Preview:
TRANSCRIPT
Analyzing Throughput and Stability in CellularNetworks
Ermias Walelgne 1 Jukka Manner 1 Vaibhav Bajpai 2 Jorg Ott 2
April 25, 2018 — NOMS’18, Taipei1(ermias.walelgne | jukka.manner)@aalto.fi,Aalto University, Finland
2(bajpaiv | ott)@in.tum.de, Technical University of Munich, Germany
April 25, 2018 — NOMS’18, Taipei
Introduction
Introduction
Introduction — Motivation
Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].Quality & performance of the cellular network depends:
radio technology,limitations of device hardwarewireless link characteristics (e.g interference, fading, etc.)mobility, location and time of the dayInfrastructure of Mobile Network Operators (MNO)
About 30% of cellular measurements from netradar [3] experience suddendrops to zero bitrate for(≥ 200 ms).
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20
Introduction
Introduction — Motivation
Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]
Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].Quality & performance of the cellular network depends:
radio technology,limitations of device hardwarewireless link characteristics (e.g interference, fading, etc.)mobility, location and time of the dayInfrastructure of Mobile Network Operators (MNO)
About 30% of cellular measurements from netradar [3] experience suddendrops to zero bitrate for(≥ 200 ms).
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20
Introduction
Introduction — Motivation
Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].
Quality & performance of the cellular network depends:radio technology,limitations of device hardwarewireless link characteristics (e.g interference, fading, etc.)mobility, location and time of the dayInfrastructure of Mobile Network Operators (MNO)
About 30% of cellular measurements from netradar [3] experience suddendrops to zero bitrate for(≥ 200 ms).
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20
Introduction
Introduction — Motivation
Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].Quality & performance of the cellular network depends:
radio technology,limitations of device hardwarewireless link characteristics (e.g interference, fading, etc.)mobility, location and time of the dayInfrastructure of Mobile Network Operators (MNO)
About 30% of cellular measurements from netradar [3] experience suddendrops to zero bitrate for(≥ 200 ms).
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20
Introduction
Introduction — Motivation
Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].Quality & performance of the cellular network depends:
radio technology,limitations of device hardwarewireless link characteristics (e.g interference, fading, etc.)mobility, location and time of the dayInfrastructure of Mobile Network Operators (MNO)
About 30% of cellular measurements from netradar [3] experience suddendrops to zero bitrate for(≥ 200 ms).
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20
Introduction
Introduction — Research Question
What are the factors affecting the throughput and stability of cellular networks?
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 2 / 20
Introduction
Introduction — Contribution
1 Time of a day and the location affect the throughput:Throughput drops during peak hours in cities as congestion increases.Radio technology switches from legacy (UMTS) to advanced (HSDPA) duringthe course of a day.
2 Predict the probability sudden drops in bit rate in a cellular network with90% accuracy
relying on easily accessible information (e.g device model, location, networktechnology).
3 Predict the average TCP download speed based on the first 5 seconds ofmedian bit rate value of throughput.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 3 / 20
Introduction
Introduction — Contribution
1 Time of a day and the location affect the throughput:Throughput drops during peak hours in cities as congestion increases.Radio technology switches from legacy (UMTS) to advanced (HSDPA) duringthe course of a day.
2 Predict the probability sudden drops in bit rate in a cellular network with90% accuracy
relying on easily accessible information (e.g device model, location, networktechnology).
3 Predict the average TCP download speed based on the first 5 seconds ofmedian bit rate value of throughput.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 3 / 20
Introduction
Introduction — Contribution
1 Time of a day and the location affect the throughput:Throughput drops during peak hours in cities as congestion increases.Radio technology switches from legacy (UMTS) to advanced (HSDPA) duringthe course of a day.
2 Predict the probability sudden drops in bit rate in a cellular network with90% accuracy
relying on easily accessible information (e.g device model, location, networktechnology).
3 Predict the average TCP download speed based on the first 5 seconds ofmedian bit rate value of throughput.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 3 / 20
Introduction
Introduction — Contribution
1 Time of a day and the location affect the throughput:Throughput drops during peak hours in cities as congestion increases.Radio technology switches from legacy (UMTS) to advanced (HSDPA) duringthe course of a day.
2 Predict the probability sudden drops in bit rate in a cellular network with90% accuracy
relying on easily accessible information (e.g device model, location, networktechnology).
3 Predict the average TCP download speed based on the first 5 seconds ofmedian bit rate value of throughput.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 3 / 20
Methodology
Methodology
Methodology — Measurement platform
0
1
2
3
4
5
0 2.5 7.5 105Time (Sec.)
TCP
down
lik sp
eed (
Mbps
)
It measures & collects information including throughput (TCP), signalstrength, radio technology type, RTT( UDP ) etc. towards Amazon Cloudinstances & Aalto University servers.The measurement server tests the download speed by sending a random dataover TCP for 10 seconds.During the measurement session, both the client and the server record thenumber of bytes transferred every 50 ms.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 4 / 20
Methodology
Methodology — Data Set and Measurement Trials
Based on a longitudinal dataset collected using the netradar measurementplatform [3].Focused on stationary nodes during a ten second measurement session — tominimize the variability that might arise from mobility.A year-long measurement data (∼750K ) from 3 Finnish MNO.
Network Total # Of LTE HSPA HSPA+ HSDPA UMTS OthersOperator Measurements
Elisa 373K 45.75% 8.49 % 14.70% 1.12% 25.43% 5%DNA 235K 63.30% 7.52% 8.17% 10.69% 0.9% 9.42%
TeliaSonera 140K 34.74% 1.04% 14.27% 1.29% 33.60% 15.06%
Table: Measurement Distribution per radio Technology for each MNO
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 5 / 20
Data Analysis
Data Analysis
Data Analysis — Device Model
EDGE GPRS HSDPA HSPA HSPA+ LTE UMTS
20112012
20132014
20152011
20122013
20142015
20112012
20132014
20152011
20122013
20142015
20112012
20132014
20152011
20122013
20142015
20112012
20132014
2015
10−3
10−2
10−1
100
101
102
Release year of the device
Dow
nloa
d sp
eed
(Mbp
s)
Figure 1: TCP download speed of different device models per network technology.
The release year of the device models does not correlate to TCP downloadspeed.It is not always the newest device model whose TCP download performanceis best.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 6 / 20
Data Analysis
Data Analysis — Mobile Network Operator
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80TCP Download Average (MBps)
CD
F
DNAElisaTeliaSonera
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50TCP Upload Average (MBps)
CD
F
DNAElisaTeliaSonera
Figure 2: Mean TCP throughput for LTE networks of 3 MNO downlink (left) and uplink(right) speed.
Clear variation between MNOs on mean uploading speed for LTE.MNO’s are significant for network performance variation.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 7 / 20
Data Analysis
Data Analysis — Subscribers Location
Elisa (Network Operator With LTE)
10 Mbps
15 Mbps
20 Mbps
25 Mbps
30 Mbps
35 Mbps
40 Mbps
DNA (Network Operator With LTE)
10 Mbps
20 Mbps
30 Mbps
40 Mbps
50 Mbps
60 Mbps
70 Mbps
80 Mbps
90 Mbps
100 Mbps
110 Mbps
Sonera (Network Operator With LTE)
10 Mbps
20 Mbps
30 Mbps
40 Mbps
50 Mbps
60 Mbps
70 Mbps
80 Mbps
90 Mbps
Figure 3: Mean TCP throughput distribution by area in Finland for LTE networks ofthree MNOs: Elisa (left), DNA (middle), TeliaSonera (right).
The comparison across MNOs shows a large variation in throughput perlocations.Users in a metropolitan area & (subscribed to DNA or Elisa) get betterthroughput than urban areas.
Better infrastructure provisioning — base station density — sufficient corenetwork capacity.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 8 / 20
Data Analysis
Data Analysis — Radio Technology Changes and Time of DayHSPA to HSPA+
UMTS to HSDPA
0 5 10 15 20
0%
2%
4%
6%
0%2%4%6%8%
Time of a day
Hand
over
freq
uenc
y
Figure 4: Frequency of radio technology switches over time of day, for the TeliaSoneranetwork (similar to other MNOs).
The occurrence of switches (from legacy to more advanced technology e.gUMTS to HSDPA) increases during peak hours.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 9 / 20
Data Analysis
Data Analysis — Network Stability
0
1
2
3
4
5
0 2.5 7.5 105Time (Sec.)
TCP d
ownli
k spe
ed (M
bps)
Classified the data into two groups:
dropped: measurement sessions that experience a sudden dropout >200 ms.non-dropped: sessions without this phenomenon.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 10 / 20
Data Analysis
Data Analysis — Received Data per Recording Interval
After sudden dropout Before sudden dropout
'O 200 +
Q) 175
Q) 150 o.-
tll tll 125 •
'O .0 100 <11:S:
75 o- D rl 50 •
25
0
0.0 0.51.0l.52.02.53.03.54.04.5 5.0 0.0 0.51.0l.52.02.53.03.54.04.5 5.0
Sudden dropout duration (sec)
oa
s
HSPA
LTE
UMTS
Do
wn
lo
ad
sp
ee
d
(M
bp
s)
Figure 5: TCP maximum download rate observed before and after a sudden dropout ofa certain duration (≥ 200 ms) per radio technology.
A sudden dropout duration that stayed for at least 200 ms does have animpact on download bit rate.The impact is visible especially after the dropout period is over (left side ofthe figure).
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 11 / 20
Data Analysis
Data Analysis — Received Data per Recording Interval
mode
modedropout
dropout
Bytes downloaded at every 50 ms sample time span
Bytes downloaded at every 50 ms sample time span
Measurements with NO sudden dropout
Measurements with sudden dropout
Mean & median of dropped diverge with a relatively higher standarddeviation than the non-dropped measurements.Network inconsistency and jitter is present in the dropped measurementsthan in non-dropped ones.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 12 / 20
Data Analysis
Data Analysis — Received Data per Recording Interval
10 1 101 1021000.0
0.2
0.4
0.6
0.8
1.0
CDF
Average of the first 3 bit-rates samples before and after dropout
Average of the first 3 bit-rates before dropout Average of the first 3 bit-rates after dropout
Download Speed (Mbps)
Figure 6: Average of the first 3 bit rates samples before and after a sudden dropout.
The effect of sudden dropout is reflected even after the sudden dropout (zerobit rate) is over.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 13 / 20
Data Analysis
Data Analysis — Sudden Dropout by Network TechnologyEDGE
GPRSHSDPA
HSPAHSPA+
LTEUMTS
0 5 10 15 20
0%2%4%6%
0%2%4%6%
0%2%4%6%
0%2%4%6%8%
0%2%4%6%
0%2%4%6%
0%2%4%6%
Time of a day
Sudd
en dr
opou
t freq
uenc
y
The sudden dropouts is distributed in all radio technology that has been used.Some technologies such as UMTS and HSPA show frequent sudden dropoutsduring daytime.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 14 / 20
Data Analysis
Data Analysis — Switching Radio Technology
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50Download speed (Mbps)
CD
F
Unknown to LTEUnknown to UMTSHSPAHSPA to HSPA+LTEHSPA+UMTS to HSPA+UMTSHSDPA
Figure 7: Impact of radio technology switches for download speed.
When sudden dropout happens a switch in radio technology causes asignificant variation in TCP download speed.
E.g. a change from UMTS to HSPA+ has better download speed than fromHSPA to HSPA+.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 15 / 20
Data Analysis
Data Analysis — Feature Importance and Selections
Downlink inter-arrival(variance)
Average throughput(TCP)
Device Model
Area province
Radio type at the beginning
Latency(UDP)
Radio type at the end
Uplink speed
Signal strengths
Mobile Network Operator
Time of a day(hours)
Battery level
Handover
Cell ID Changes
Day of week
Frequency of handover
Platform
Byte inter-arrival(mean)
0 250 500 750
Mean Decrease Accuracy
Downlink (TCP)
Downlink inter-arrival(variance)
Area province
Uplink speed
Device Model
Latency(UDP)
Radio type at the beginning
Day of week
Battery level
Time of a day(hours)
Signal strengths
Radio type at the end
Handover
Mobile Network Operator
Cell ID Changes
Frequency of handover
Byte inter-arrival(mean)
Platform
0 500 1000 1500 2000
Mean Decrease Gini
(a) with TCP throughput
RadioHandoverStartEndStartingNetTechNames
EndingNetTechNameradioEvolution
RadioHandoverdeviceVendor
CellIDChangedbattery_level
weekdayDModel
hourssignalStrengths
carrierlatency
0 25 50 75Mean Decrease Accuracy
radioEvolutionStartingNetTechNames
EndingNetTechNameCellIDChangedRadioHandover
carrierdeviceVendor
RadioHandoverStartEndDModel
weekdayhours
signalStrengthsbattery_level
latency
0 2000 4000 6000Mean Decrease Gini
(b) without TCP throughput
Latency, carrier network, signal strength radio network technology, time ofthe day and device model found to be important predictive variables forclassification.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 16 / 20
Data Analysis
Data Analysis — Classification model
0.0 0.2
ROC
0.4 0.6 0.8 1.0
False positive rate
True
pos
itive
rate
Models:Conditional Inference TreeBaggingRandomForest
o.o
0.2
0.4
0.6
0.8
1.0
Figure 9: True positive and false positive rates of the three classification models;random forest shows the best Receiver Operating Characteristic (ROC) curve.
Random forest based classification produces better prediction with accuracyof 90% & error rate of 10.2% on the testing dataset.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 17 / 20
Data Analysis
Data Analysis — Predicting the Average Throughput
Prediction of througput is useful (eg. to improve video performance incellular networks [4]).How to predict the overall mean throughput only using the first five secondsof TCP bit rate measurement?Train a model using Randome Forest algorithm from caret [5] package.The model predict average throughput: using only the first 5 sec. downloadrates (which would be easily available right after startup) withRoot-Mean-Square Error (RMSE) ( 0.003 Mbps) & 98% R-squared.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 18 / 20
Conclusion
Conclusion
Conclusion
1 Throughput observed by a cellular network user depends on various factors.E.g location and time of the day where metropolitan areas during peak hoursshowed more drops in throughput.
2 Network stability:TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate -could create performance degradations.classify stability based on sudden dropouts only relying on easily accessibleinformation.
3 Predicting the average throughput:useful to anticipate future performance & to adjust application demands.trained a model that predicts average throughput based on throughput of TCPslow start phase.
4 Future work extending — our predictive approach :Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network.
ermias.walelgne@aalto.fi
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20
Conclusion
Conclusion
1 Throughput observed by a cellular network user depends on various factors.E.g location and time of the day where metropolitan areas during peak hoursshowed more drops in throughput.
2 Network stability:TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate -could create performance degradations.classify stability based on sudden dropouts only relying on easily accessibleinformation.
3 Predicting the average throughput:useful to anticipate future performance & to adjust application demands.trained a model that predicts average throughput based on throughput of TCPslow start phase.
4 Future work extending — our predictive approach :Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network.
ermias.walelgne@aalto.fi
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20
Conclusion
Conclusion
1 Throughput observed by a cellular network user depends on various factors.E.g location and time of the day where metropolitan areas during peak hoursshowed more drops in throughput.
2 Network stability:TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate -could create performance degradations.classify stability based on sudden dropouts only relying on easily accessibleinformation.
3 Predicting the average throughput:useful to anticipate future performance & to adjust application demands.trained a model that predicts average throughput based on throughput of TCPslow start phase.
4 Future work extending — our predictive approach :Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network.
ermias.walelgne@aalto.fi
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20
Conclusion
Conclusion
1 Throughput observed by a cellular network user depends on various factors.E.g location and time of the day where metropolitan areas during peak hoursshowed more drops in throughput.
2 Network stability:TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate -could create performance degradations.classify stability based on sudden dropouts only relying on easily accessibleinformation.
3 Predicting the average throughput:useful to anticipate future performance & to adjust application demands.trained a model that predicts average throughput based on throughput of TCPslow start phase.
4 Future work extending — our predictive approach :Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network.
ermias.walelgne@aalto.fi
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20
Conclusion
Conclusion
1 Throughput observed by a cellular network user depends on various factors.E.g location and time of the day where metropolitan areas during peak hoursshowed more drops in throughput.
2 Network stability:TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate -could create performance degradations.classify stability based on sudden dropouts only relying on easily accessibleinformation.
3 Predicting the average throughput:useful to anticipate future performance & to adjust application demands.trained a model that predicts average throughput based on throughput of TCPslow start phase.
4 Future work extending — our predictive approach :Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network.
ermias.walelgne@aalto.fiAnalyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20
Conclusion
References
ITU, “ITU releases 2017 global information and communication technologyfacts and figures.”
IDC, “IDC idc 50th anniversary transformation everywhere,” 2017.
“Netradar measurement platform.” http://www.netradar.org/.
X. K. Zou, J. Erman, V. Gopalakrishnan, E. Halepovic, R. Jana, X. Jin,J. Rexford, and R. K. Sinha, “Can Accurate Predictions Improve VideoStreaming in Cellular Networks?,” HotMobile, pp. 57–62, 2015.
M. Kuhn, “Building Predictive Models in R Using the caret Package,”Journal of Statistical Software, vol. 28, no. 5, pp. 1–26, 2008.
Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 20 / 20
top related