network breaking point- bruce peterson panel
Post on 14-Jun-2015
266 Views
Preview:
DESCRIPTION
TRANSCRIPT
Focus: reliable KPIs
Aggregation and filtering.
?? ??!
Internet, S
Client!
Server,ApplicationRealtime
1Measuring E2E Quality Dr.Stefan Rugel 15.2.2011 slide
Measuring E2E throughput as provided by the network
Checking efficiency of time sampling method:
20 M t P b l d t diff t it ithi T 2 N t k• 20 Measurement Probes placed at different sites within To2 Network
• 7.2 Mbit/s max throughput, good coverage placement
• Testcase: 2 Mbyte http downloads
Probe Measurements Network-based measurements
2Measuring E2E Quality Dr.Stefan Rugel 15.2.2011 slide
Measuring E2E throughput as provided by the network
=> Perfect match with true throughput measured by the probe.
1600
Final Improvement
1200
1400
sec]
800
1000
hput [kbit / s
400
600
Through
0
200
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
Measurement Sample
"Effective Throughput": considering the type of the idle times Probe Measurements
3Measuring E2E Quality Dr.Stefan Rugel 15.2..2011 slide
User Experience vs Network Monitoring
Service Flow Measures User Experience Monitoring
RAN Core IPUser Data
User Data
FlRAN Core IPFlows Flows
RANStats
CoreStats
RouterStats
NetworkMonitoring = + +
Quick Wins (vague specific)
F l i t S ifi D t il• For a complaint – can now see Specific Details• Which network elements involved (name of GGSN, SGSN, etc)• PDP Context Set-Up records (Accept / Reject)PDP Context Set-Up records (Accept / Reject)• Data Flow
Network Initial RNC GGSN SGSN Sig Name SGSN SigSGSN DatStart Time End Time APN Create/Update Closure DL Vol (kB) UL Vol (kB)g g p ( ) ( )
Hutchison 3RMHB13 BM0FXG01 HH0SGN07 217.171.1217.171.1424/09/2010 02:55 24/09/2010 03:55 234.20.three.co.uk Hourly Update 3.88 2.06
Hutchison 3RMHB13 BM0FXG01 HH0SGN07 217.171.1217.171.1424/09/2010 01:55 24/09/2010 02:55 234.20.three.co.uk Hourly Update 1.89 1.03
Hutchison 3RMHB13 BM0FXG01 HH0SGN07 217.171.1217.171.1424/09/2010 00:55 24/09/2010 01:55 234.20.three.co.uk Hourly Update 3.78 2.06
Hutchison 3RMHB13 BM0FXG01 HH0SGN07 217.171.1217.171.1423/09/2010 23:55 24/09/2010 00:55 234.20.three.co.uk Hourly Update 27.99 3.37
Hutchison 3RMHB13 BM0FXG01 HH0SGN07 217.171.1217.171.1423/09/2010 22:57 23/09/2010 22:57 234.20.three.co.uk Hourly Update 0 0
Hutchison 3RMHB13 BM0FXG01 HH0SGN07 217.171.1217.171.1423/09/2010 22:57 23/09/2010 23:55 234.20.three.co.uk Hourly Update 39.77 18.68
RMHB13 HH0FXG02 HH0SGN07 23/09/2010 22 57 23/09/2010 22 57 234 20 th k R j t d b GGSN 0 0Hutchison 3RMHB13 HH0FXG02 HH0SGN07 217.171.1217.171.1423/09/2010 22:57 23/09/2010 22:57 234.20.three.co.uk Rejected by GGSN 0 0
Hutchison 3RMHB13 HH0FXG01 HH0SGN07 217.171.1217.171.1423/09/2010 20:55 23/09/2010 21:01 234.20.three.co.uk Success Coverage Lost 0 0
Hutchison 3RMHB13 HH0FXG01 HH0SGN07 217.171.1217.171.1423/09/2010 19:55 23/09/2010 20:55 234.20.three.co.uk Hourly Update 25.4 1.15
Hutchison 3RMHB13 HH0FXG01 HH0SGN07 217.171.1217.171.1423/09/2010 19:04 23/09/2010 19:55 234.20.three.co.uk Hourly Update 97.07 26.01
Orange UK Roaming HH0FXG01 194.33.x.x 194.33.24194.33.24.23/09/2010 18:51 23/09/2010 18:51 234.20.three.co.uk Success Success 4.04 1.12
Orange UK Roaming HH0FXG01 194.33.x.x 194.33.24194.33.24.23/09/2010 18:51 23/09/2010 18:51 234.20.three.co.uk Success Success 3.39 1.1
Hutchison 3RMBS06 HH0FXG02 BM0SGN05 217.171.1217.171.1423/09/2010 16:55 23/09/2010 17:49 234.20.three.co.uk Success Coverage Lost 0 0
Hutchison 3RMBS06 HH0FXG02 BM0SGN05 217.171.1217.171.1423/09/2010 16:49 23/09/2010 16:55 234.20.three.co.uk Hourly Update 0 0
Orange UK Roaming HH0FXG01 194.33.x.x 194.33.24194.33.24.23/09/2010 16:43 23/09/2010 16:49 234.20.three.co.uk Success Success 578.77 37.44
Hutchison 3RMBS06 BM0FXG01 BM0SGN05 217.171.1217.171.1423/09/2010 16:17 23/09/2010 16:42 234.20.three.co.uk Success Success 144.02 48.14
Peak Speed Metric Calibration (591 Customers Surveyed 10 = Excellent, 1 = Poor)( y , )
gePe
rcen
tag
ed M
etric
Customer Score Customer ScoreSpee
• Good Correlation between Customers’ Scores and Speed Metric• Wide disagreement about what speed is good – perhaps due to g p g p p
different expectations and uses of Internet.• Vote Good – Actual speeds measured are also good• Vote Poor – Actual speeds measured may be good or bad.
Call Centre Use
• Per Session Per Customer Data flow Metrics
S d i tti
• Per-Session, Per-Customer Data-flow Metrics has resulted in a 8% reduction in Call Handling Times.
Speed Metric(last 30 days)
Speed is getting worse in last
fortnight
Which devices customer has used and data volumes
Shouldn’t You Be Scoring Your Customers’Customers Experience?
Per Session, Per User QoE Scoring
Experience Latency Packet Error EffectiveExperienceScore
Latency Packet Error Effective Throughput
5. Excellent4. Good3. Fair2. Poor1. Bad > 600 ms >6% < 400 Kbps
2220016142216790
2220016142247380
Impacted PDP Sessionsby Cell ID
2220016142216790
2220016142247380
2220016142211010
2220016142216860
2220016142204710
2220016142247400
2220016142216790
0 200 400 600 800 1000 1200 1400
2220016142216770
2220016142216790
Real-Time Measurements & Analytics Feed Business ProcessesBusiness Processes
End-to-End SScoring
+Analytics & Reporting
+g
Customer Experience
Resolution
Trouble Ticket
Resolution Isolation
Proactive Problem Isolation Effectiveness
Capacity Planning
Effectiveness
SolutionsValidate experience issues and quickly
Isolate problems with detailed session data
Aggregate scores of actual user experience
Verify planned capacity expansions based onissues and quickly
develop corrective actions
detailed session data, network element
reports
actual user experience and alarm about
problems
expansions based on user experience
scoring
BackupBackup
The Challenge:M i Q li f M bil IManaging Quality for Mobile Internet
iatio
niff
eren
tias
ing
Di
Incr
ea
12Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
Monitoring E2E-Quality: Network based Approach
Advantage: Full network coverage, real time availability.Problem: Away from Client.Challenge: Drawing reliable conclusion from observations.
?
Internet, S
Client:Throughput,Latency ServerLatency
13Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
E2E-Protocols for E2E KPIs
Packet Inspection:• E2E Protocols • Flow specific• Gn-If
Move from Network PM counter evaluation to E2E Protocol analysis.
A li ti
• Gn-If• Real Time
y
PDCP PDCPIPTCP
Application
GTP GTP
RLC
MAC-d
MAC-c/sh
RLC
MAC-d
MAC-c/sh IP / MPLS(ATM) IP
UDP
IP / MPLS(ATM) IP
UDP
L3 L3
PHY
MAC-hs
PHY
MAC-hs
PHY
L2 / L3
HS_DSCHFP
PHY PHY
L2
(ATM)
PHYL2
PHYL2
IP
PHYL2L2 / L3
HS_DSCHFP
(ATM) IP
PHYL2
PHYL2
UE Nb RNC SGSN GGSN Server
IU-PSIub GnGi /
InternetUu
14Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
UE Nb RNC SGSN GGSN Server
Protocols providing E2E Quality information
E il Web Vid St iService AEmail Web
Browsing Video Streaming Access, Topology2nd Step:
• Classification• Detailed KQI
• Internet Access• Location• Network Topology
POP3,SMTP,
Containers (.flv, .f4v, .mpg, .mp4, .avi, .swf ...)
CodecsHTTP GTP
DNS
• Network Topology• Device Type• Radio Access Type
RTMPIMAPRTCP RTP
DNS
UDP(<1% of traffc)
HTTP
TCP (> 99% of Traffic) UDP(<1% of traffc)
IPTCP (> 99% of Traffic)
1st Step:• Throughput• RTT (Round Trip Time)• Packet Retransmissions• Service Access
Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide 15
Agenda
1 Introduction
g
1. Introduction2. TCP Throughput Measurements3. Conclusion
M t i & Q EMetrics & QoEUser perceived Quality (QoE)User perceived Quality (QoE)
QoE
M
appi
ng
Mapping
VoIP, Life Streaming
File Download
Web VideoM Web Browsing
BufferinTime to loadApplication
ifi ty
P k t L
gApplication
Met
rics
Time to load
k t lThroughput
specific
mpl
exit
Packet LossRound Trip Time
TCP
M
RTCP
packet loss jitter
utQoS C
o
Network, L2, IP, UDP
TCPRTCP
Which kind of Throughput?
• Network provided ThroughputNetwork provided Throughput.
• Realistic download scenario, no peak throughput.
Usecases:
• Strategic Marketing.g g
• Customer Care
• Capacity Engineering
• Traffic ManagementTraffic Management
• ……….
18Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
Measuring E2E throughput as provided by the network
Basic Approach: Data VolumeConnection Time
Download Throughput =Connection Time
Problem: several downloads per connection, separated by idle times.
TCP Setup TCP TeardownC
Idle Idle Idle Idle
Connection Time
DownlinkUplink
Ti
TCP Control Packet
C
Time
TCP Packet with Payload19Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
Measuring E2E throughput as provided by the network
Improved approach: sampling fixed time intervals (e.g. 0.5 sec)
Data VolumeDownload Throughput =Active Time
Download Throughput
Active Time
Idle Idle Idle Idle
DownlinkUplink
Ti
TCP Control Packet
C
Time
TCP Packet with Payload20Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
Measuring E2E throughput as provided by the network
Checking efficiency of time sampling method:
20 M t P b l d t diff t it ithi T 2 N t k• 20 Measurement Probes placed at different sites within To2 Network
• 7.2 Mbit/s max throughput, good coverage placement
• Testcase: 2 Mbyte http downloads
• Correlation with network based time filtering taken from inside the
network
Probe MeasurementsNetwork Based:Time sampling
21Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
Measuring E2E throughput as provided by the network
6000
Probe Throughput Measurement versus Active ThroughputMeasurement Results (Time Filter 0.5sec)
5000Over‐Estimation due to neglecting idle periodst [
kbps
]
4000
kbps]
Over Estimation due to neglecting idle periods
mt R
esul
t
3000
red Throughp
ut [k
Active Throughput
ProbeThroughptused
Msm
!!!2000
Measur ProbeThroughptu
wor
k ba
s !!!1000
Under‐Estimation due to sampling‐intervalmissmatch at Download Start and ‐End
Net
w
0
0 1000 2000 3000 4000 5000 6000
Throughput [kbps]Probe Measurement[kbps]O fOver Estimation for bad throughput!
22Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
Measuring E2E throughput as provided by the network
Final Correction: considering the type of idle times.
• Netw. BufferingTCP Timeout Bad“ Idle Times
Idle Times: due to• TCP Timeout
Time between two
„Bad Idle Times
Good“ Idle Times
TCP Setup TCP TeardownC
separate Downloads „Good Idle Times
Idle Idle Idle Idle
Connection Time
DownlinkUplink
Ti
TCP Control Packet
C
Time
TCP Packet with Payload23Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
Measuring E2E throughput as provided by the network
=> Perfect match with true throughput measured by the probe.
1600
Final Improvement
1200
1400
sec]
800
1000
hput [kbit / s
400
600
Through
0
200
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
Measurement Sample
"Effective Throughput": considering the type of the idle times Probe Measurements
24Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
TCP: E2E property
Client ServerMSMTTCP handshake enables PING-like
round trip time measurements.
Client ServerMSMT
RTTAcknowledged data transfer shows
packet retransmissions and delivered
RTT
data volume.
Various methods for throughput Duration
measurement.Duration,Volume
25Measuring E2E Quality Dr.Stefan Rugel 17.11.2010 slide
P i t f tPoint of measurement
Measurement at the Clients Side.
V l t Q E
Network based measurement.
F ll t k/tiVery close to QoE
No full network/time coverage
Full network/time coverage
Real time capable
Expensive Only part of information available
RTT+ k t l > Th h t?RTT+packet loss => Throughput?2500
Life Network Measurement, static probes, 2Mbyte file download
2000
e [m
sec]
1500
d Trip Tim
e
< 1000kbps
< 1500kbps
Throughput
1000
erage Ro
und
< 2000kbps
< 2500kbps
>= 2500kbps
500
Ave
0
0 5 10 15 20 25 30 35
Packet Retransmission Rate [%]
A dAgenda
• Introduction
• Network based measurement on QoE in mobile data networksmobile data networks.– Why do we need it
– Approach
– Example: Web Video– Example: Web Video
– Conclusion
W b VidWeb VideoApproach:
Buffer simulation: estimate buffered time by correlating packet arrival‐ and playoutBuffer simulation: estimate buffered time by correlating packet arrival and playout time.
ViewingViewing Time Buffered
Time
1 t ti i i f ti1. get timing informationInternet Video Server
TCP Acknowledgements
IP Packets of VideostreamFirst Step:
Measure packet arrivalPoint of Measurement
Get:TCP-Sequence Number,Timing Info of last FrameCalculate:Playout Time
ptimes.Extract correspondingframe playout times (flv, Playout Time
Save:TCP-Sequence Number,Playout Time
Table
frame playout times (flv, mp4)
Seq-Nr. Playout Timen1 x1
.. ..nm xm
nm+1 xm+1
Get:TCP-Ack Number,Timestamp of ArrivalLookup:TCP-Ack NumberSave:Timestamp of Arrival
Ack Timey1
..ym
.. ..
Calculate QoE – KPI
KPI = f(xi yi)Codec
Protocols used( i,yi)
Video File Format
Codec
TCP
HTTP
30Video Client
TCP
2 B ff i l ti2. Buffer simulationBuffering Mode = “ON“Buffering Mode = ON
BufferedTime = 0Init TB
i=2Get x1, y1
Second Step: Simulate Buffered Time
BufferingMode =
“OFF“
BufferedTime =
Yes
No
BufferedTime = BufferedTimei =BufferedTimei-1+ (xi-xi-1) - (yi-yi-1)
BufferedTimei>
BufferedTimei≤
BufferedTimei =BufferedTimei-1+
(xi-xi-1)
KPIs per Video Flow: Total re-bufferred playtime fe
rin
Playo
> TB
BufferingMode =“OFF“
≤0
YesYes
NoBufferingMode =
“ON“
Total re bufferred playtime.Number of rebufferingevents.
Input Parameter:
Buf
fg
out
Last iEnd Yes
BufferedTime =0
Input Parameter: Buffering threshold TB.
Last iEnd
Next i
Yes
No
31
Test setupTest setup
Compare packet trace Analysis with client measurements
Network Based:• Initial Buffering TimeR b ff i Ti
Client based (manually):• Initial Buffering TimeR b ff i Ti • Rebuffering Time
• Number of Rebuffering Events• Rebuffering Time• Number of Rebuffering Events
32
Fi t R lt (1)First Results (1)
16
18
12
14
ec]
8
10
ered
Tim
e [se
4
6
Buff
0
2
0
0 20 40 60 80 100 120
Viewing Time [sec]Buffering Period:Good agreement with
33
Client Measurement
Fi t R lt (2)First Results (2)2.1Mbit/s required < 2.3Mbit/s achieved
34
C ll C t UCall Centre Use• Per Session Per Customer Data flow Metrics has resulted in a
S d i tti
• Per-Session, Per-Customer Data-flow Metrics has resulted in a 8% reduction in Call Handling Times.
Speed Metric(last 30 days)
Speed is getting worse in last
fortnight
35
Which devices customer has used and data volumes
E i i I ti tiEngineering Investigation Site Throughput Problem detectedSite Throughput Problem detected
Site flat-lined from 9am!
36
Not all user activity uses the available speedNot all user activity uses the available speed(Unloaded Cell with 7.2 HSPA dongle)
Download 3 5MBLarge Web Page Asset
Download 2MB Photo(unloaded website)
Download 3.5MB Application (popular website)
BBC iPlayer
5.8 Mbits/sec
Browsing Web Pages Flash Video
(Speed limited by External Site to ~1.6Mbits/s)
37
top related