Mobile Network Testing
Arnd Sibila
Technology Marketing MNT
(Dr. Jens Berger)
December 2019
ENSURING QOE IN 5G NETWORKS
EVOLVING APPLICATIONS AND MEASUREMENTS
AGENDA► Mobile Network Testing
► Technical improvements and who uses 5G
► Traditional and new applications and how
they will change
► Interactivity and how to test
► New QoE concepts and QoE models
► Conclusion
Rohde & Schwarz
5G TECHNICAL IMPROVEMENTSWHAT DO PEOPLE EXPECT AND WHAT ARE 5G PROMISES?
► More data-speed! Technically it means more transport capacity
► More interactivity! Technically it means very short transport latency and seamless transport
► More flexibility! Technically addressed by ‘network slices’ serving different needs and QoS requirements
► New applications and use cases! (Finally)
Dec 2019 Ensuring QoE in 5G3
Rohde & Schwarz
5G DEPLOYMENT – FIRST STEP IS DONE IN FIELD
► First 5G real field deployments are done!
► What does it mean in a first step?
5G EN-DC 5G non-standalone mode
Device is connected by LTE (4G), one or more 5G carriers are added for supporting transfer capacity
Requires dual connectivity (4G + 5G)
► What is next?
5G standalone access (no dual connectivity required)
URLLC
(mMTC)
It is not as much benefit for a normal customer…
Download is going a bit faster,
but none of the 5G promises, yet.
Dec 2019 Ensuring QoE in 5G4
eMBB
mMTC URLLC
Rohde & Schwarz
WHO USES 5G AND HOW?
► Todays networks are optimized for humans using smartphones
► In 5G new classes of users will take advantage of the infrastructure
Humans (smartphone use case)
Automotive (connected, autonomous driving, Vehicle-to-X)
Industry 4.0 (Smart Manufacturing, private 5G networks)
IoT, mMTC (Smart City, Connected Energy,...)
Each class causes individual traffic patterns and has individual requirements to the network!
A network optimized for human users may not deliver best performance for cars or industry
Requires
5G standalone
Dec 2019 Ensuring QoE in 5G5
Rohde & Schwarz
WHAT TO EXPECT FROM 5G?
► Human users will be one user class amongst others
► Main popular applications will remain, but evolve
► New application areas and use cases will be
launched and used
► It is not a ‘switch-on’ rather a ‘phase-in’
Applications will use what is made available
There will be a transition phase
6
Source:
Dec 2019 Ensuring QoE in 5G
Rohde & Schwarz
WHAT HAPPENS BEHIND THE SCENES TECHNICALLY? APART FROM 5G
► Simple Video Download is replaced by DASH
► HTTP almost disappeared and is replaced by HTTPs
► FTP almost disappeared
► TCP is more and more replaced by QUIC
► Today’s remaining UDP traffic could be replaced by a QUIC derivate to make it reliable
► New applications will create and use new types of protocols (e.g. AR, VR, XR)
► Far most data traffic is handled by proprietary and encrypted protocols
Dec 2019 Ensuring QoE in 5G7
DASH: Dynamic Adaptive Streaming over HTTP
QUIC: Quick UDP Internet Connections
Rohde & Schwarz
WHAT DOES 3GPP SAY ABOUT QOS?
Dec 2019 Ensuring QoE in 5G8
3GPP TS23.501: System Architecture for the 5G System (5GS); Stage 2 (Rel16, draft 09/19)
5G QoS characteristics (chap 5.7.3.) ► Priority Level
► Packet Delay Budget (including Core Network Packet Delay Budget)
► Packet Error Rate
► Maximum Data Burst Volume (for Delay-critical GBR resource type only)
► Averaging window (for GBR and Delay-critical GBR resource type only)
Non-GBR
AGENDA► Mobile Network Testing
► Technical improvements and who uses 5G
► Traditional and new applications and how
they will change
► Interactivity and how to test
► New QoE concepts and QoE models
► Conclusion
Rohde & Schwarz
LET’S START SIMPLE: HOW TELEPHONY WILL EVOLVE IN 5G?
► 3GPP Telephony (telephony as primary service)
QoE makes no significant difference to 4G VoLTE
Very short call setup time, EVS 24.4 high quality coding
Accessibility and sustainability will further increase
Video Telephony may increase but strong competition by OTT
► OTT Telephony
Today (4G) OTT telephony applications are still behind VoLTE
Lower accessibility and sustainability (more failed and dropped calls)
Voice quality can be lower due to time-warping and packet loss (unreliable channels), not by coding
Significant improvement in 5G (more reliable channels, short latency)
OTT services will become fully equivalent or even superior to VoLTE (because of wider functionality)
Dec 2019 Ensuring QoE in 5G10
Rohde & Schwarz
DATA APPLICATIONS IN 5GTHE VAST MAJORITY
► Existing applications and use cases
Today’s use cases will remain popular
(HTTP-Browsing, Video streaming, OTT Messaging, posting to social media,…)
► Evolving applications and use cases (based on existing use cases)
4K/8K video, HDR Similar to today, ‘just’ more throughput
360° video, Virtual Reality Similar to today, ‘just’ more throughput
Rendering will move from device to core (in interaction with device)
Live video feed, video upstream Similar to today, more throughput, more reliable
Real time gaming More throughput, shorter latency, more reliable
Rendering will move from device to core (cloud gaming)
► New applications and use cases
Applications: Augmented reality, Virtual Reality vs. Use Cases: AR gaming, remote control, VR retail shopping,…
Highly adaptive to network conditions by Machine Learning and AI techniques
Dec 2019 Ensuring QoE in 5G11
Rohde & Schwarz
WILL WE HAVE AN AR OR VR ‘SERVICE’?
► “Service” is UDP (or QUIC)
► Individual use cases will use
same underlying libraries and techniques
and may lead to similar data patterns
in the network
► Differences will be in the objectives and
the requirements in performance
► Individual QoE models for individual
use cases
(an AR retail shopping has different
objectives than an AR assisted surgery)
Source: Dec 2019 Ensuring QoE in 5G12
Application Use Cases
Rohde & Schwarz
► Let’s go back to the integral QoE concept and its dimensions
HOW TO MEASURE QOE – WHAT WILL HAPPEN?
Connectivity will be
‘always’ given, how to
measure very rare ‘fails’?
Instead chasing individual
‘failed access’,
a new concept could be a
‘network stability score’
(‘probability to fail’)
This is the idea of
‘ultra reliable’ in URLLC
Dec 2019 Ensuring QoE in 5G13
5G
Rohde & Schwarz
► Let’s go back to the integral QoE concept and its dimensions
HOW TO MEASURE QOE – WHAT WILL HAPPEN?
Access time will become
very short
Only minor influence on QoE
in the future
Dec 2019 Ensuring QoE in 5G14
5G
Rohde & Schwarz
► Let’s go back to the integral QoE concept and its dimensions
HOW TO MEASURE QOE – WHAT WILL HAPPEN?
This is the key and the
challenge at the same time!
How to measure? What are the key parameters?
Dec 2019 Ensuring QoE in 5G15
5G
AGENDA► Mobile Network Testing
► Technical improvements and who uses 5G
► Traditional and new applications and how
they will change
► Interactivity and how to test
► New QoE concepts and QoE models
► Conclusion
Rohde & Schwarz
TESTING INTERACTIVITY – THE DOOR INTO 5G
Dec 2019 Ensuring QoE in 5G17
Interactivity
► The future will be interactive and real-time
► There will be tons of new interactive applications
► What does it mean: Interactivity?
More than Bitrate
Latency and Continuity are key
Latency stands for transport and also response time
Continuity covers interruptions, undercuts
of required data rate and indirectly packet loss
Rohde & Schwarz
HOW TO TEST INTERACTIVITY?
18
► Interactivity is a perceptual metric, depends on application and use case
► Let’s start with: How to measure
Bitrate
Round-trip latency / response time
Continuity
of a data connection between a device in the field
and a server in the network
Dec 2019 Ensuring QoE in 5G
Rohde & Schwarz
THE NEW INTERACTIVITY TEST
19
► The device sends a stream of unique packets to an (active) server
► The server sends back packets as responses
Data-rate controlled by frequency and size of packets
Round-trip latency, packet delay variation and packet error rate / corruption ratio measured
► Based on UDP (…is the future anyway)
► Packet rates of 100 to 1500 per second.
This is a continuous packet flow!
Dec 2019 Ensuring QoE in 5G
Rohde & Schwarz
THE NEW INTERACTIVITY TEST – THE TECHNIQUE BEHIND
20
► The protocol and basic implementation is taken from TWAMP
► TWAMP: Two-Way Active Measurement Protocol – specified by IETF, open source
► Implemented under Android native (R&S MNT) to minimize OS influence
Ready for URLLC
► Server location and implementation:
Industry target: TWAMP responders in firewalls, routers, infrastructure
R&S MNT will deploy own server installations
Server package available by R&S MNT for private installations
Dec 2019 Ensuring QoE in 5G
Rohde & Schwarz
THE NEW INTERACTIVITY TEST – PARAMETERS & RESULTS
21
► Control parameters (on technical level) are:
Packet rate
Packet size
Delay budget (…if exceeded,
packet is counted
as dropped)
Dec 2019 Ensuring QoE in 5G
► Results:
Data rate TX and RX
Round-trip latency and
packet delay variation
Packet error rate
low
high
small
large
accepted delay
Rohde & Schwarz
THE NEW INTERACTIVITY TEST – IN PRACTICE
22
► Pre-defined ‘Traffic Pattern’ to emulate a certain application, e.g. e-Gaming
► Constant packet flow (low, medium, high)
► Composed patterns (e.g. emulating a 10s ‘e-Gaming’ phase)
Initial phase
Highly interactive phase
Sustainable phase
Trailing phase
are emulated by different packet flow rates
► Achieved bitrate, round-trip latency and packet
delay variation for each phase and for ‘overall’
duration
► Interactivity score ‘e-Gaming’
Dec 2019 Ensuring QoE in 5G
Rohde & Schwarz
► e-Gaming emulation in 4G / LTE
THE NEW INTERACTIVITY TEST – IN PRACTICE
23 Dec 2019 Ensuring QoE in 5G
Round-trip latency and
packet delay variationPacket error rate
Interactivity Score Per packet
round-trip latency
Achieved bitrate
Round-trip latency
(Median and 10th percentile)
► Detailed measurement results available (even per packet)
Rohde & Schwarz
THE NEW INTERACTIVITY TEST – IN PRACTICE
24
► e-Gaming emulation in 4G / LTE Bad channels
Dec 2019 Ensuring QoE in 5G
Delay variation
Packet loss
Rohde & Schwarz
► e-Gaming emulation in 4G / LTE
INTERACTIVITY TEST SMART ANALYTICS
25 Dec 2019
Round-trip latency
Packet delay variation
Packet error rate
Achieved bitrate
Ensuring QoE in 5G
Interactivity Score
AGENDA► Mobile Network Testing
► Technical improvements and who uses 5G
► Traditional and new applications and how
they will change
► Interactivity and how to test
► New QoE concepts and QoE models
► Conclusion
Rohde & Schwarz
INTERACTIVITY SCORE – THE QOE MODEL
27
► For emulating ‘real’ applications we will apply a generic QoE model
► The QoE model produces a ‘synthetic’ MOS based on QoS and technical KPIs
► Interactivity Score (~ ‘synthetic MOS’)
Interactivity Score = f1 (latency, time) * f2 (delay variation) * f3 (packet loss)
Dec 2019 Ensuring QoE in 5G
Rohde & Schwarz
INTERACTIVITY SCORE – E-GAMING
28
► Interactivity Score
𝐼𝑛𝑡𝑒𝑟𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦 𝑆𝑐𝑜𝑟𝑒 = 𝑆𝑐𝑜𝑟𝑒𝐿𝐴𝑇𝐸𝑁𝐶𝑌 ∗ 𝑆𝑐𝑜𝑟𝑒𝑃𝐷𝑉 ∗ 𝑆𝑐𝑜𝑟𝑒𝑃𝐿 ∗ 100%
Dec 2019 Ensuring QoE in 5G
Factors scaling linearly
the Interactivity Score
Rohde & Schwarz
► This sort of generic QoE model is fully scalable
► Based on a common set of parameters
INTERACTIVITY TEST CONCEPT – THE QOE MODEL
Dec 2019 Ensuring QoE in 5G29
‘Drone control’
► Different QoE models for different
Interactivity use cases
Rohde & Schwarz
SUMMARY
30
► 5G allows for ‘always available’ services and very short access times
focus more on quality of a running use case
► Interactivity and continuity become crucial for real-time applications under 5G
► Interactivity Test: combines testing round-trip latency, packet delay variation,
packet error rate and proofing bitrate in one single test
► Full set of detailed results (down to per-packet)
► Smart integrative scoring: Interactivity Score for given use cases (e.g. e-Gaming)
► Interactivity is key for future applications!
Interactivity Test released in R&S network test solutions
► We are 5G URLLC ready!
Dec 2019 Ensuring QoE in 5G
THANK YOU!
…AND KEEP BEING REAL AND INTERACTIVE!