mobile benchmark report mwc 2015
DESCRIPTION
Mobile Experience Benchmark - MWC 2015 EditionTRANSCRIPT
Mobile Experience Benchmark - MWC 2015 Edition
Introduction
2
This report provides an overview of three key aspects to consider when developing mobile apps:
1. State of the mobile ecosystem, across operating systems and devices
2. The digital customer experience for mobile apps, across app store categories
3. Differences in key metrics for wireless networks, across different markets & geos
The data in this report is aggregated from Crittercism’s leading mobile application performance management (mAPM) solution that processes over 50,000 requests per second from 1 billion mobile users in over 120 countries, every day.
What’s the state of the mobile ecosystem?
3
Why the state of the mobile ecosystem matters
4
“The smarts of Android on screens big & small” “iOS 8 is the biggest iOS release ever — for developers and everyone else” Knowledge about the installed base of devices & OS inform design, feature and architecture decisions.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% Apr/14 May/14 Jun/14 Jul/14 Aug/14 Sep/14 Oct/14 Nov/14 Dec/14 Jan/15 Feb/15
5
iOS 7
iOS 8
iOS 6
Other
iOS 8 passing 50% of iOS app load traffic in 50 days Apple ships a new OS bundled with a new flagship device, and pushes the OS update immediately to the entire installed base of devices, resulting in quick uptake.
Apple iOS, Share of App Load Traffic by Version, April 2014 – January 2015
50% share of app load
traffic
Nov 5th Sept 17th
50 days
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Apr/14 May/14 Jun/14 Jul/14 Aug/14 Sep/14 Oct/14 Nov/14 Dec/14 Jan/15 Feb/15
iPhone 6 Plus
iPhone 6
iPhone 5S
iPhone 5C
iPhone 5
iPhone 4S
iPhone 4
Other
6
iPhone 6 / 6 Plus reaching ~20% app load traffic within ~20 weeks iPhone 5S as the previous flagship device achieved largest share of a single iOS device, indicating how good Apple is at providing latest computing technology to consumers.
Apple iPhone, Share of App Load Traffic by Device Generation, April 2014 – January 2015
19% share of app load
traffic
Feb1st Sept 17th ~20 weeks / 4.5 months
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Apr/14 May/14 Jun/14 Jul/14 Aug/14 Sep/14 Oct/14 Nov/14 Dec/14 Jan/15 Feb/15
Lollipop 5.0
KitKat 4.4
Jelly Bean 4.3
Jelly Bean 4.2
Jelly Bean 4.1
Ice Cream Sandwich 4.0
Other
7 Google Android, Share of App Load Traffic by Version, April 2014 – January 2015
Android Lollipop <2% share of app load traffic, KitKat still growing Google, device OEMs, carriers provide phased upgrades to the existing installed base of Android devices, resulting in a uptake of a new Android version that’s slower by an order of magnitude compared to iOS.
<2% share of app load traffic Feb1st Nov 12th 81days
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Apr/14 May/14 Jun/14 Jul/14 Aug/14 Sep/14 Oct/14 Nov/14 Dec/14 Jan/15 Feb/15
Droid Razr HD
Motorola Moto G
Samsung Galaxy Note 2
Samsung Galaxy S3
Samsung Galaxy Note 3
Samsung Galaxy S3 Mini
Samsung Galaxy S4
Samsung Galaxy Tab 2
Samsung Galaxy S5
8
Samsung dominates Android, with S5 supplanting S3 Data indicates that owners of Samsung devices follow a 2-year upgrade cycle (e.g. from S3 to S5) along with carrier subsidies.
Android, Share of App Load Traffic by Device Type, April 2014 – January 2015
20% share of app load traffic
Nov 1st Apr 1st ~30 weeks / 7 months
OS updates can cause unforeseen problems
9
Despite extensive testing & preparation by OEMs and carriers, things can still go wrong.
Source: https://twitter.com/azkop2010/status/569312645527908352; https://twitter.com/dianaturner/status/569971859573358592
What’s the digital customer experience of my mobile apps ?
10
Why the digital customer experience for mobile apps matters
11
Badly functioning apps are the source for customer frustration: “Your app has one job: online check in, so how come it’s never available?”
Source: https://twitter.com/British_Airways/status/560238312540364800; https://twitter.com/British_Airways/status/560055280604835840
12
iOS 8 crash rates stabilized within a single quarter Initial iOS 8 crash rate of 3.3%, dropping to 2.2% and converging with crash rates of iOS 6 (2.2%) and iOS 7 (2.0%). Improvements due to Apple quickly providing OS updates and developers fixing errors & bugs.
1.5%
1.7%
1.9%
2.1%
2.3%
2.5%
2.7%
2.9%
3.1%
3.3%
3.5%
Apr/14 May/14 Jun/14 Jul/14 Aug/14 Sep/14 Oct/14 Nov/14 Dec/14 Jan/15 Feb/15
iOS 8
iOS 6
iOS 7
iOS Crash Rates by Version, April 2014 – February 2015
Crash rate declines from 3.3% to 2.2%
Dec 14th Sept 17th 12 Weeks
1.2%
1.4%
1.6%
1.8%
2.0%
2.2%
2.4%
Apr/14 May/14 Jun/14 Jul/14 Aug/14 Sep/14 Oct/14 Nov/14 Dec/14 Jan/15 Feb/15
13
Anatomy of an iOS release cycle: Maturity of “old” OS Crash rates are continuously declining as developers get more acquainted with the installed OS, continuously improving the quality of their apps.
iPhone 4
iOS 7 maturing, with crash rates continuously going down and reversely the quality of the
in-app experience goes up
iPhone 6 plus
iPhone 5S
iPhone 5C
iPhone 6
iPhone 5
iOS Crash Rates by Device, April 2014 – February 2015
iPhone 4S
1.2%
1.4%
1.6%
1.8%
2.0%
2.2%
2.4%
Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14 Jan-15 Feb-15
14
Crash rates spike following the introduction of a new OS version New OS features – while tested - are breaking apps, as end points are hit in a new way. Developers need at least two new builds to fix errors, implying two App Store review cycles with ~20 days of wait time.
iPhone 4
iOS 8 launch on Sept 2014
iPhone 4S
iPhone 6 plus
iPhone 5S
iPhone 5C
iPhone 6
iPhone 5
iOS Crash Rates by Device, April 2014 – February 2015
1.2%
1.4%
1.6%
1.8%
2.0%
2.2%
2.4%
Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14 Jan-15 Feb-15
15
New cycle of developer productivity starts, driving down crash rates Quality of apps goes back up as Apple stabilizes the OS and developers continuously fix errors and optimize the delivery of their apps. Dev efforts are also shifting away from older devices such as iPhone 4.
iPhone 4
iPhone 6 plus
iPhone 5S
iPhone 5C
iPhone 6
iPhone 5
Developers getting more acquainted with the new OS, creating higher quality apps, but also moving on from older hardware platforms in
the case of iPhone 4
iOS Crash Rates by Device, April 2014 – February 2015
Devs moving on from iPhone 4
iPhone 4S
16
App Store category crash rates Apps that consume a lot of resources across the stack (e.g. CPU, GPU, bandwidth) like Photo & Video and Games tend to crash more frequently than other categories.
2.5% 2.4%
2.1%
1.8% 1.8% 1.8% 1.6%
1.5%
Photo & Video Games Social Networking
Health & Fitness
Business Entertainment Lifestyle News
0.05% 0.02%
Best-in-class apps operate within a crash rate band of 0.02-0.05% on both iOS and Android
AppStore Crash Rates by Category – December 2014
5.3%
3.4% 3.3% 3.3% 2.8%
2.5%
1.8% 1.7%
Entertainment Business Photography Shopping Games News & Magazines
Social Health & Fitness
17
Google Play category crash rates On average higher crash rates per category than for the App Store, same phenomenon for resource-intensive apps like Entertainment and Photography.
0.05% 0.02%
Best-in-class apps operate within a crash rate band of 0.02-0.05% on both iOS and Android
Google Play Crash Rates by Category – December 2014
What are key differences for wireless networks across different markets & geos?
18
Source: https://twitter.com/wuhelp/status/539823712766144512
Why networks & geographies matter for mobile apps
19
Carrier-enforced firmware for Android devices can cause an app to drain battery life; network settings can lead to unexpected consequences for the availability of mobile apps: “Yes, it does work on wifi but does not work over AT&T’s network.”
20
France
€26
€17
€23
€23
Orange
SFR
Free
Bouygues
274ms
293ms
262ms
250ms
Wireless Carriers Ranked by Speed
ARPU Carrier Avg. Latency
Relative Split between iOS & Android
iOS Android
1.
2.
3.
4.
37%
35%
41%
40%
63%
65%
59%
60%
Bouygues scores best on latency among all French carriers, with the highest ARPU. France has the fastest networks and with it the highest ARPUs in the Eurozone countries.
= highest ARPU in a market Split between iOS and Android calculated based on app usage / traffic.
21
Germany
€15
€14
€11
€9
O2
E-Plus
T-Mobile
Vodafone
320ms
448ms
317ms
300ms
Wireless Carriers Ranked by Speed
ARPU Carrier Avg. Latency
Relative Split between iOS & Android
iOS Android
1.
2.
3.
4.
45%
44%
69%
77%
55%
56%
31%
23%
Vodafone with the fastest network and the highest ARPU; E-Plus by far the slowest, with lowest ARPU
= highest ARPU in a market Split between iOS and Android calculated based on app usage / traffic.
22
United Kingdom
Wireless Carriers Ranked by Speed
ARPU Carrier Avg. Latency
Relative Split between iOS & Android
iOS Android
£19 / ~€26
£18 / ~€24
£19 / €26 EE
Vodafone
O2 1.
2.
3. 347ms
344ms
327ms 82%
87%
87%
18%
13%
13%
O2 with the fastest network and the highest ARPU. All carriers with highly similar OS split and latency metrics, leading to similar ARPUs.
= highest ARPU in a market Split between iOS and Android calculated based on app usage / traffic.
23
Italy
€11
€12
€13 Vodafone
TIM
WIND
343ms
339ms
282ms
Wireless Carriers Ranked by Speed
ARPU Carrier Avg. Latency
Relative Split between iOS & Android
iOS Android
1.
2.
3.
7%
55%
55%
93%
45%
45%
No clear correlation between low relative latency / high relative ARPU.
= highest ARPU in a market Split between iOS and Android calculated based on app usage / traffic.
24
Spain
€15
€13
€18
€15
Vodafone
Movistar
Yoigo
Orange
358ms
431ms
334ms
314ms
Wireless Carriers Ranked by Speed
ARPU Carrier Avg. Latency
Relative Split between iOS & Android
iOS Android
1.
2.
3.
4.
8%
23%
32%
26%
92%
77%
68%
74%
No clear correlation between low relative latency / high relative ARPU; Vodafone with highest ARPU and highest share of iOS usage.
= highest ARPU in a market Split between iOS and Android calculated based on app usage / traffic.
25
United States
$44
$49
$55
$42
45%
44%
69%
77%
55%
56%
31%
23%
Verizon
AT&T
Sprint
T-Mobile
356ms
362ms
340ms
327ms
Wireless Carriers Ranked by Speed
ARPU Carrier Avg. Latency
Relative Split between iOS & Android
iOS Android
1.
2.
3.
4.
AT&T with the slowest network and the lowest ARPU in the market.
= highest ARPU in a market Split between iOS and Android calculated based on app usage / traffic.
Mobile Experience Benchmark - MWC 2015 Edition