T-Mobile Technology Meetup - E2E Mobile Device and Network Performance

QoE Lab| Device Development |TPD

August 25, 2014

T-Mobile Confidential 1

Contributors: Jie Hui, Kranthi Sontineni, Zach McPherson, Peter Myron

Agenda

• Introduction to QoE lab

• E2E Performance - Network Side – How YouTube Performance is Improved in T-Mobile Network

• E2E Performance - Device Side – QoE Analyzer demo

– Hands-on trial

T-Mobile Confidential 2

Introduction to QoE Lab

T-Mobile Confidential 3

Why

Make the customer experience more enjoyable!

QoE is measurement of the enjoyment of the customer experience.

Mobile QoE 2.0

Network

Experience Real User Speed

Experience Real User Application

Experience

Crowd-sourced from devices Network-instrumented

1.0 1.5 2.0

Figure originally from : I. Grigorik, “High Performance Browser Networking: What every web developer should know about networking and web performance”. O’Reilley Media, 2013

What

T-Mobile Confidential 6

Measure QoE

Identify Problem

Root Cause Analysis

Optimize

Scaling Through Partnerships and Automation Development

T-Mobile Confidential

Operations

NW Engineering

“How YouTube Performance is Improved in T-Mobile Network”, Velocity 2014

[Info] [Video]

T-Mobile Confidential 8

QoE Analyzer

T-Mobile Confidential 9

QoE Analyzer

Turns data into

something which is

meaningful in Seconds

10

Motivation to build QoE Analyzer

Test QoE Analysis

Automated log parser and charting tool to assist in root cause analysis for QoE issues.

QoE Analyzer TCP log QXDM log App log

TCP analysis Radio analysis

Note: QoE Analyzer implements part of T-Mobile patent “Network Device Trace Correlation”, Jie Hui, Himesh Bagley, Jeff Smith, US20140119196

QoE Analyzer - Design Modem log (.isf)

Packet trace (.pcap)

PDCP/RRC/RLC– Modem QXDM (.isf)

IP/Transport– OS (.pcap)

App/ e.g YouTube (logcat)

UE

Server

IP/Transport (.pcap)

IP/Transport (.pcap)

GGSN/PGW

12

RUM performance

Device QoE

QoE Analyzer

Note: QoE Analyzer implements part of T-Mobile patent “Network Device Trace Correlation”, Jie Hui, Himesh Bagley, Jeff Smith, US20140119196

QoE Analyzer

What Analysis can QoE analyzer do?

• TCP Connection times

• DNS Lookup time

• Throughput graphs

• TCP Failure Analysis

• TCP retransmission analysis per Stream/IP

• TCP sequence Analysis

• IP filtering.

• E2E analysis-Correlate Packets between UE, SGW/PGW and server

• Supports IPV4/IPV6

T-Mobile Confidential 14

PCAP Analysis Radio Analysis Logcat Analysis

• 3G/4G/LTE Radio state

Transition summary

• 3G/4G/Radio bearer

Reconfiguration

response time.

• 3G/4G/RLC UL & DL

PDU Retransmissions.

• LTE PDCP and RLC

throughput graphs

• 3G/4G/ IP packet and

RLC PDU Correlation.

YouTube analysis

• Video Startup time

• Video buffer count

• Buffer Ratio

PCAP Analysis

T-Mobile Confidential 15

Input Output

1 DNS Lookup Time Average: 0.070 Min: 0.019 Max: 0.122

2 TCP -SYN to SYN/ACK RTT Average: 0.069 Min: 0.025 Max: 0.124

3 TCP -SYN/ACK to ACK RTT Average: 0.002 Min: 0.001 Max: 0.006

4 TCP Connection time( TCP SYN -ACK) Average: 0.071 Min: 0.028 Max: 0.125

6 Total Number of TCP retranmissions Total: 2774

7 Total Number of TCP Duplicates Total: 3282

8 Total number of TCP Reset's Total: 6

9 Total number of TCP Packets Total: 19456

10 Total Size of TCP packets MB: 13.686

KPI's Analyzed from wireshark Trace Totals:

TCP Connection Summary

TCP Failure Analysis

T-Mobile Confidential 17

Input Output

TCP Packet Sequence Analysis

T-Mobile Confidential 18

Input-choose specific IP

Output

TCP - Cross IP-Radio and Cross-Node Analysis

T-Mobile Confidential 19

Input-Pcap file from UE, SGW,Server

Output

Radio Analysis

T-Mobile Confidential 20

Radio State Transition Analysis – 3G/4G

T-Mobile Confidential 21

Input Output

RLC UL & DL PDU Retransmission analysis- 3G/4G

T-Mobile Confidential 22

Input Output

Radio State Graph with IP Packet Overlay---3G/4G

T-Mobile Confidential 23

Input Output

Radio State Transition Analysis - LTE

T-Mobile Confidential 24

Input Output

LTE Cross Layer Throughput – PDCP & RLC

T-Mobile Confidential 25

Input Output

Logcat Analysis

T-Mobile Confidential 26

YouTube Video Metrics – Logcat Analysis

T-Mobile Confidential 27

Input Output

Log File StartTime Buffer Count Avg Buffer Buffer Time Rebuff Count MTBR Rebuffer Time Elapsed Time Buffer Ratio

Youtube_4G_2014_03_07_12_57_03.txt 00:02.510 0 00:00.000 00:00.000 1 02:36.220 02:36.220 02:38.730 0.00%

Youtube_4G_2014_03_07_14_50_42.txt 00:01.950 0 00:00.000 00:00.000 1 02:36.120 02:36.120 02:38.070 0.00%

Youtube_4G_2014_03_07_18_38_03.txt 00:28.620 12 00:10.849 02:10.190 13 00:11.872 02:34.330 05:13.140 45.76%

Youtube_4G_2014_03_07_19_49_12.txt 00:29.970 11 00:11.468 02:06.150 12 00:12.879 02:34.550 05:10.670 44.94%

Youtube_4G_2014_03_07_20_19_25.txt 00:30.020 13 00:10.172 02:12.230 14 00:11.016 02:34.230 05:16.480 46.16%

Youtube_4G_2014_03_07_22_01_58.txt 00:28.320 12 00:10.708 02:08.490 13 00:11.858 02:34.150 05:10.960 45.46%

Youtube_4G_2014_03_07_23_56_42.txt 00:28.700 13 00:10.085 02:11.100 14 00:11.031 02:34.440 05:14.240 45.91%

Youtube_4G_2014_03_08_00_28_07.txt 00:28.150 13 00:10.007 02:10.090 14 00:11.041 02:34.570 05:12.810 45.70%

Contact Us

Please send any question or request to get trial access for QoE Anlayzer to DeviceQoELab@T-Mobile.com.

T-Mobile Confidential 28