spirent: improving the volte experience
TRANSCRIPT
Improving the VoLTE Experience: Best Practices from Early Launches
An Infonetics Research Webinar
Co-produced with Spirent Communications
#VoLTE
Today’s Speakers
2
JoAnne Emery
Event Director
Infonetics Research
(Moderator)
Rich McNally
Director
Service Experience Programs
Spirent Communications
Stéphane Téral
Principal Analyst,
Mobile Infrastructure and
Carrier Economics
Infonetics Research
Eric Sinclair
Manager
Service Experience Programs
Spirent Communications
#VoLTE
Agenda
3
Market Trends
3
4
Challenges
Best Practices
Case Studies
Sponsor Background
Conclusions
Audience Q&A
1
2
3
4
5
6
7
3
#VoLTE
Mobile Broadband Rules but Voice Stays
‣ Global mobile revenue forecast:
• US$815.8 billion by year end (+2%
over last year)
• US$852.5 billion in 2017
‣ Broadband is the next growth
engine, surpassing SMS/MMS
revenue this year
• We forecast US$290 billion in 2018,
up 75% from 2013
‣ Voice will still account for >48% of
total mobile service revenue in 2018
$0
$200
$400
$600
$800
$1,000
CY12 CY13 CY14 CY15 CY16 CY17 CY18
Reve
nu
e (
US
$ B
illi
on
s)
Voice SMS/MMS Broadband
© Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014 4
‣ 1.5 billion active users
worldwide of OTT mVoIP
in 2013…
• Mobile operators using OTT for
subscribers travelling out
of country
• Activity continues to expand in
Japan, China, and South Korea
‣ …that will taper as users
whittle choices down and
over-penetration settles out
‣ Proliferation of smartphones
fueling growth
• We expect OTT mVoIP subscribers to more than double from 2013 to 2018, to 3.1 billion
0
500
1,000
1,500
2,000
2,500
3,000
3,500
CY11 CY12 CY13 CY14 CY15 CY16 CY17 CY18
Su
bsc
rib
ers
(M
illi
on
s)
OTT mVoIP VoLTE
5 Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014
Although OTT Dominates mVoIP…
…the Revenue Model Remains Challenging
‣ In 2013, 1.5 billion mVoIP OTT
subscribers contributed
$10 billion in revenue
• Successful providers are using
the app as a platform to upsell
• Revenue declines as users
eliminate multiple OTT apps
‣ VoLTE is starting to have an
impact but values are small; only
12 operators have commercial
service so far
‣ We forecast combined native and OTT
mobile VoIP to grow to $24.7 billion in 2018
• Based on 3.4 billion mVoIP subscribers, 90% of which are OTT
$0
$5
$10
$15
$20
CY11 CY12 CY13 CY14 CY15 CY16 CY17 CY18
Reve
nu
e
(US
$ B
illi
on
s)
OTT mVoIP Native mVoIP
6 © Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014
0
20
40
60
80
100
120
140
2009 2010 2011 2012 2013 2014
Cumulative W-AMR HD Voice Networks
HSPA commercial VoLTE commercial
‣ 119 mobile operators offer HD
voice with W-AMR* as key
HD voice enabler
• 95 on HSPA
• 12 on LTE
(VoLTE + IR.94 video calling)
• 8 on GSM/HSPA
• 2 on HSPA/LTE
• 2 on GSM
‣ 92 smartphones (including
carrier and frequency variants)
support VoLTE
• Including products by Asus, Huawei, LG, Pantech, Samsung, and Sony Mobile
• The new Apple iPhone 6 & 6 Plus models support VoLTE
Source: GSA, September 23, 2014
7 *W-AMR: Wideband Adaptive Multi-Rate codec
VoLTE Taking Off, Marketed as HD Voice…
…and SK Telecom Fuels the Momentum
‣ 20 commercial VoLTE networks and 51 million VoLTE subscribers this year
‣ Worldwide revenue of $1.2 billion at the current voice ARPU
• In spite of SK Telecom’s VoLTE adoption (8M+ subscribers in May 2014)…
• …no evidence VoLTE is increasing ARPU (despite SKT charging a premium!)
‣ By 2018 we expect ~325 million VoLTE subscribers and $7 billion in revenue
8
0
50
100
150
200
250
300
350
$0
$2
$4
$6
$8
CY12 CY13 CY14 CY15 CY16 CY17 CY18
Su
bsc
rib
ers
(M
illi
on
s)
Reve
nu
e (
US
$ B
illi
on
s)
VoLTE Market Opportunity
© Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014
The Feedback Is Extremely Positive so Far
‣ Connection is super fast, ringtone sound quality exceptional
‣ Virtually no background noise; voice is crystal clear
‣ Video call quality is crisp and sound
• Superior to Skype and other OTT apps
9
The ultimate goal is to keep every subscriber on your network
with the carrier-grade service features that beat OTTs
There Is Little Surprise!
‣ VoLTE is reliable
• Offers guaranteed QoS
‣ VoLTE provides high quality
• Uses W-AMR codecs
(12.65kbps or 23.85kpbs)
and QoS Class Identifier
(QCI) SIP signaling
‣ VoLTE is spectrally efficient
• Due to LTE’s all-IP architecture and new features in 3GPP releases —
such as MIMO antenna technology — voice requires less bandwidth in
LTE spectrum than it does in 2G/3G networks
10
Source: Nokia Networks, 2012
Agenda
11
Market Trends
3
4
Challenges
Best Practices
Case Studies
Sponsor Background
Conclusions
Audience Q&A
1
2
3
4
5
6
7
11
#VoLTE
The Experience Matters
13
‣ VoLTE needs to deliver a better user
experience than OTT and legacy services
• There is only one way to truly know if new devices
and services are ready to launch
• You need to measure the experience at the
device, using actual consumer devices
‣ Spirent has been in the trenches, rolling out
VoLTE with leading operators for 2 years
• Following are some best practices for evaluating
and improving VoLTE…
• …to ensure a successful launch of new devices
and services
Agenda
14
Market Trends
3
4
Challenges
Best Practices
Case Studies
Sponsor Background
Conclusions
Audience Q&A
1
2
3
4
5
6
7
14
#VoLTE
Practice #1: Measure What Matters
15
‣ Our experience:
• Functional testing is increasingly insufficient for
assuring user experience
• Combined live network and lab evaluation of user
experience (UX) is a better approach
• If you don’t measure everything that matters to
users, problems will emerge
‣ Best practices:
• Focus on objective assessment of what the
user experiences
• Measure all key factors impacting user experience
• Measure quality and consistency
MOS (wideband
mobile-to-mobile and
narrowband landline)
Mobile Originated and
Terminated Block/Drop
Rates
Conversational Audio
Delay
Video Delivery
Audio / Video Sync
Battery Life
VoLTE UX Metrics
Practice #2: Stress the Device Like a User
16
‣ Our experience:
• Measurements of UX are only valid if they
reflect real user behavior
• Multi-service voice and data usage
stresses the device
• Dropped calls and other UX elements are
impacted by multi-service usage
‣ Best practices:
• Implement multi-service use cases for
pre-launch device evaluation
• For example, set up continuous push
emails during test calls
1.1% 0.9%
1.1%
Manufacturer 1 Manufacturer 2 Manufacturer 3
Voice only Multi-service
Dropped Call Rate by
Device Manufacturer
Voice Only Vs. Multi-Service
1.3% 1.4%
1.9%
Practice #3: Rank Devices (and Services)
17
‣ Our experience:
• There is a wide variance in UX across
device models
• It’s a competitive marketplace - showing
rank by UX category is a powerful motivator
‣ Best practices:
• Consider the relative performance of
devices and not just the absolute
• Rank and compare all pre-launch devices
by UX category
• Set thresholds based on population
performance (and raise over time)
No one wants to be in this range
Speech Quality
Downlink Uplink
Device A
Device A
Device B Device B
Device D
Device D
Device C
Device C
Device E
Device E
Practice #4: Assess from the Top Down
18
‣ Our experience:
• You can spend a lot of time and money
gathering stack-loads of data you don’t use
• Broad statistical assessments with focused
drilldown into problems saves time
• RTP and RF tracing can provide key insight into
root causes of poor experience
‣ Best practices:
• Only collect RTP/IP and RF DM logs at UE
(and IMS) where problems are identified or in
focused pre-planned instances
• Use the data to accelerate triage/isolation of
UX issues to network/device/service
0
10
20
30
40
50
60
200 300 400 500 600
Packets
/s
Received Packets/s Missing Packets/s
HO HO
Isolating the Root Causes of
Poor VoLTE Speech Quality
0
1
2
3
4
5
0 100 200 300 400 500 600 700 800
Do
wn
lin
k M
OS
RTP Metrics Help Identify Cell-Specific Issue
Agenda
19
Market Trends
3
4
Challenges
Best Practices
Case Studies
Sponsor Background
Conclusions
Audience Q&A
1
2
3
4
5
6
7
19
#VoLTE
Case Study: VoLTE Speech Quality
20
‣ Drive test of live US VoLTE
network with varying RF
‣ MOS very flat across good, fair
and poor RF environments
• Average speech quality mean
opinion score (MOS) of 3.5
• In very poor RF conditions, MOS
tends to degrade rapidly
‣ RTP jitter and delay also flat
across varying RF conditions
• In very poor RF, RTP performance
drops off rapidly
22.1
17.2 16.8 16.2 16.6 16.2 15.9
-105 -100 -95 -90 -85 -80 -75
Average RTP Jitter (ms) by RSRP (dBm)
3.55 3.53 3.46 3.53 3.48 3.52
Good Fair Poor
Device 1
Device 2
VoLTE MOS (POLQA) by RF Environment
Case Study: Typical Causes of Poor MOS
21
‣ Week-long drive test of VoLTE in
major US city
‣ RF issues are most common and
fairly easy to detect
• Distant or no dominant server
• Interference / weak server
• Dragging handover
‣ IP (RTP) issues also common
• Configuration / IP connectivity
‣ Other issues
• No apparent RF or IP issues
• Device issue related to buffering,
packet re-assembly or codec
RF 79%
IP 16%
Other 5%
Root cause of VoLTE MOS < 2.5 by category
for week long drive test of major US city
‣ Are problems due to:
device, network, or interaction?
‣ Comparative scorecards help
identify the source of issues
‣ Scorecard on right comes from
tests on a live VoLTE network
• Network RF conditions are good
• Device 2 is less efficient at using
air-interface resources, but…
• RTP performance is okay
• Ideal MOS from RTP stream shows
speech quality should be good
• Device 2 jitter buffer or codec is
not working properly!
Case Study: Isolating Root Causes
22
VoLTE Analytics Scorecard
Overall Rank Device 1 Device 2
User Experience
Speech Quality
Network – RF Conditions
Received Power (RSRP)
Signal Quality (RSRQ)
Interference (CINR)
Device-Network Interaction
Resource Allocation
Modulation Efficiency
RTP – Packet Jitter
RTP – Packet Delay
RTP – Packet Loss
Device Buffering and Decoding
Ideal MOS from RTP
Actual - Ideal MOS
Case Study: Isolating Root Causes
23
‣ Used RTP analysis to isolate issue
with service infrastructure / IP layer
‣ Collected IP sniffer data via device and
server-side logs in area with low MOS
• RF is good
• RTP jitter and delay is okay
• RTP packet loss rate very high
‣ Diagnosis
• The network is working and packets are
getting through but…
• There is a major IP connectivity issue
leading to dropped packets
-86.3
-98.7
Serving Cell Neighbor Cell
RSRP
One location: VoLTE MOS < 2.5 with
good RF but packet loss rate > 9%!
RTP Jitter: 14 ms
RTP Delay: 20 ms
RTP PLR: 9.3%
RF
RTP
RSRQ: -12.3
CINR: 4.6
Case Study: Making Improvements
24
‣ Results from live network testing
of VoLTE and IR.94 video
‣ Key factors that matter to users:
• Speech quality (MOS)
• Frozen or impaired frame rate
‣ Prior to optimization, VoLTE
performs poorly
‣ After optimization VoLTE is 1st
or 2nd best in all UX categories
3.4%
13.2%
19.5%
25.3%
Frozen/Impaired Frames
IR.94 (Optimized)
OTT 2
OTT 1
IR.94 (Pre-optimization)
Frozen/Impaired Frames for IR.94 Video vs. OTT
2.4
3.0
3.3
3.6 3.8
Speech Quality (POLQA MOS)
OTT 1
VoLTE (Pre-optimization)
Circuit-Switched
VoLTE (Optimized)
OTT 2
VoLTE vs. OTT and CS Speech Quality (HD Codec)
Optimization
improvement
Optimization
improvement
Agenda
25
Market Trends
3
4
Challenges
Best Practices
Case Studies
Sponsor Background
Conclusions
Audience Q&A
1
2
3
4
5
6
7
25
#VoLTE
26
Partnering with service providers and device manufacturers to
improve the user experience of devices and services
Over 2 years F4L program
led to decreased variance
between devices, more
consistent speech quality
F4L program also led to
better average speech
quality across portfolio
Effect of Fit4Launch on Portfolio Speech
Quality over first 2 years
Fit4Launch programs for
evaluating user experience
of pre-launch devices are
deployed at 3 US operators
Improving User Experience
27
Inventing systems and methodologies for measuring and analyzing
the user experience of mobile devices and services
Inventing Systems and Methods
Location
2003 2008 2010 2012 2013 2014
Live Lab
Spirent User Experience Analytics timeline
Speech File Transfer &
Web Browsing Call Battery Life
Voice & Video
Calling
2011
Video
Agenda
28
Market Trends
3
4
Challenges
Best Practices
Case Studies
Sponsor Background
Conclusions
Audience Q&A
1
2
3
4
5
6
7
28
#VoLTE
‣ With the RF in order, beat the OTT with voice and video calling
over LTE!
• Video calling over LTE is very challenging because it requires low
latency, low error rate and high bit rate
• With the GSMA toolbox, IR.92 and IR.94, when the LTE network is
congested, VoLTE and video calling services can use LTE QoS
features to prioritize packets to ensure service quality
- Pure OTT can’t
‣ With 71 operators currently investing in VoLTE studies, trials or
deployments, it’s only a matter of time for mobile operators to
retake their destiny
29
The VoLTE Advantage
Thank you for your interest!
30
Watch the full webinar on the link given below:
http://www.spirent.com/OnDemand/Service_Exp
erience/Improving-the-VoLTE-Experience