T H E M O B I L E A P P

F O R M O B I L E P R O D U C T &

ANALYTICS GUIDEANALYTICS GUIDEM A R K E T I N G T E A M S

INTRODUCTION 3

THE DATA NEEDS OF PRODUCT AND MARKETING TEAMS 4

IN-APP ANALYTICS 6

A/B TESTING 9

CRASH REPORTING AND ANALYTICS 12

APP STORE ANALYTICS 15

VOICE OF CUSTOMER (VoC) ANALYTICS 18

PUSH ANALYTICS 21

MOBILE TESTING 24

BUILDING A COMPREHENSIVE MOBILE APP ANALYTICS TECHNOLOGY STACK 27

AppAssurance™ 28

ABOUT THE AUTHOR 29

Table of Contents

3

Introduction

To some extent your mobile app users can tell you what about your app works well for them, and what does not.

But when it comes to delightful mobile app experiences, a non-UX-oriented customer base may have a hard time putting into words the je ne sais quoi aspect of your app that makes their experience delightful.

Translating je ne sais quoi experiences into insights-driven strategy relies on integrating the right technologies into your mobile app, as well as involving the correct stakeholders from your product and marketing teams.

This guide discusses the data needs of key roles in your mobile app development and marketing teams, including:

• Product• UX/UI Designers• Developers• Quality Assurance Engineers• Marketers

as well as the different categories of mobile app analytics technologies, including:

• In-App Analytics• A/B Testing• Crash Reporting and Analytics• App Store Analytics• Voice of Customer (VoC) Analytics• Push Analytics• Mobile Testing

You will also find links to sample platforms, as well as additional resources.

Note: While it could be argued that some boutique analytics platforms may fit nicely into only one of these catego-ries, there is often overlap. The categories and corresponding descriptions of each analytics technology given here are more for the sake of building a comprehensive mobile app product/marketing strategy and technology stack, so not every category will require its own vendor.

4

The Data Needs of Product and Marketing Teams

As in all aspects of mobile app development and marketing, you need to establish the “why” for inte-grating mobile app analytics into both your mobile app technology stack as well as your strategy.

Doing so on a role-by-role basis will keep all stakeholders invested in maintaining analytics technology and heeding data-driven, evidence-based strategy.

“Measuring mobile presents an entirely unique set of challenges”

PRODUCT

As the chief strategists in mobile app design, product managers are also the chief advocates for mobile app analytics.

Responsible for developing evidence-based product strategy, product teams use data to both build and defend their product roadmaps.

Data saves product strategists from falling prey to arbitrary feature development or design sourced from their own gut feelings or eager executives/sales reps.

By formulating hypotheses and then testing the effectiveness of those hypotheses, product teams can refine their products to be exactly what the consumer needs.

UX/UI DESIGNERS

UX/UI designers make the mobile app experience beautiful and pragmatic, engaging and seamless.

While designers rely on their sense of taste in determining how to build beautiful mobile app interfaces and user flows, using behavioral data from mobile app analytics to form a feedback loop allows them to see how their creativity has played out.

Data also allows designers to optimize creative assets through iterative A/B testing.

-Eric Peterson, Analytics Demystified

5

DEVELOPERS

Developers can use analytics technologies like crash reporting platforms to identify remedies for points of failure, as well as use performance-based metrics to suggest courses of action to optimize performance on mobile devices.

QUALITY ASSURANCE ENGINEERS

Similar to developers, QA engineers can use analytics tools to monitor mobile app functionality, identi-fying the small percentage of use cases that fall outside of their testing portfolio and suggesting reme-dies for app crashes, bugs or bottlenecks.

MARKETERS

While marketers may be on the outside of product teams, they are expected to be just as agile and consistent with product’s messaging efforts.

There is a large analytical overlap between product managers and product marketers, both relying heav-ily on user behavior data and customer feedback.

The application of data, however, will be different for marketers, as they are not so much concerned with strategizing around feature development and design, but rather how to foster loyalty among users by engaging with them across all channels.

6

In-App Analytics

“Because most users only open five applications per day, creating great customer experi-ences differentiates your app from those that end up in the trash.”

-Arun Bhattacharya, Adobe

MEASURE + IDENTIFY

Personalization is the new watchword for mobile app marketers and product teams. Forrester Research shows that “77% of consumers have recommended or paid more for a brand that provides personal-ized experiences or chosen that company because of those services”—and because app monetization, acquisition and retention are key concerns, that’s not a statistic to be ignored.

Offering personalized experiences to targeted cohorts or markets-of-one requires data, and lots of it. That’s where in-app analytics come in.

WHAT IS IN-APP ANALYTICS?

In-app analytics is the principal form of mobile app analytics deployed by product and marketing teams. For app owners trying to get more than just a pulse on their customer base—we’re talking blood pressure, vitals, respiration, temperature, etc.—in-app mobile app analytics are their mainstay.

In-app analytics provide mobile app teams with the who, what, where, when and how of user behavior—invaluable insights for understanding user activity:

Who: In-app analytics allow you to build and refine your ideal customer persona, using data such as age, gender, location, device type, interests and activity level. This allows app owners to segment users into different groups to perform cohort analysis for targeted marketing campaigns.

What: Identifying what your users are viewing in your app will give you context as to what features or screens are most important. Digging into data will help you see the frequency of feature usage, which screens are viewed and more.

7

Potential use cases

While there are a lot of pre-built measurement functions for in-app analytics platforms, one of the key benefits is the extensive customizability—app owners can measure almost anything a user does within the app, helping them to identify whether or not users are accomplishing the “why” behind their prod-uct.

In-app analytics technologies can also capture key revenue data, aggregating segmentable purchase behavior data.

Where: Knowing where your users are located will help you initiate key marketing efforts during the best possible engagement windows. Users can be targeted by relevant geographical locations or at different time periods depending on time zone.

When: Knowing at what time users engage with your app will help you know how your app fits into a user’s daily life cycle. Do users check your app in the morning before they start their day, or do they engage during the course of their daily activities?

Frequency is another key indicator of user behavior—understanding how sticky your app is will show you how dedicated to or reliant upon your app users are.

How: Device type and OS version give product managers, developers and QA engineers insight into how they should prioritize and structure their testing and development processes. If an app is not performing well on a certain device, they can use this data to understand what may be negatively affecting user experience for these users.

Another “how” aspect to consider is whether or not users mainly use your app with WiFi or cellular data, providing additional context into usage patterns.

• Analyze historical data for user behavior before app uninstalls to predict and safeguard against potential churn

• Create an onboarding funnel of first-time users to see conversion rates from one step to the next, allowing you to identify the principal dropping-off points

• Identify at what times your most active users engage with your app, and use this data to help low-activity or inactive users to build up a pattern of continuous engagement

8

Key value benefits of in-app analytics:• Product: Observe user behavior and identify usage patterns and opportunities

• UX/UI Designers: Observe user flow and identify bottlenecks or experience spoilers

• Developers: Identify high-traffic time periods to optimize server performance/product updates

• Quality Assurance: Observe dropping off points and identify potential points of failure. Identify most popular device types for testing purposes

• Marketers: Observe user growth and retention, and identify opportunities to reconnect and increase loyalty. Identify engagement windows for nurturing app users

Sample platforms:

Adobe Analytics, Firebase Analytics, Flurry Analytics, LocalyticsMixpanel,Countly

Additional resources:

Measure the Success of Your Mobile Application Using these 17 Detailed Mobile App Analytics Strat-egies9 Mobile App KPIs to KnowEven Better Strategies for Measuring Mobile App Success

9

A/B Testing

“As human beings, we’re naturally wired to try and find the best outcome. A/B testing is an invaluable tool to help achieve this aim.”

-Bill Schneider, Urban Airship

EXPERIMENT + OPTIMIZE

Building an engaging mobile app isn’t a matter of bloating your app with an insanely large amount of features—it’s about refining experiences and paying attention to the details.

A portion of user behavior is driven by intuition rather than analytical thought, and appealing to this portion of the consumer psyche should be a top priority. Businesses need to tap into human intuition to discover how to build experiences that not only make functional sense but also invoke emotional responses.

This is the essence of mobile A/B testing.

WHAT IS MOBILE A/B TESTING?

Mobile A/B testing is a complement to optimization through in-app analytics—using A/B testing, poten-tial variations of an asset are tested on a small audience before being adopted. This allows companies to both refine their mobile apps over time, as well as test potential feature or design changes to verify how they affect user behavior.

According to Forrester Research, “over half (54%) of mobile programs now use effective customer ex-perience optimization methods such as A/B testing” (Forrester’s H2 2016 Global Mobile Executive On-line Survey).

Potential use cases• When “priming for push,” test the messaging and design of different custom opt-in prompts to

increase push opt-in rates

• Test various onboarding screen flows to optimize the number of active users

• Test various techniques for inviting your users to leave a rating in the app store

This branch of mobile app analytics begins with an evidence-based hypothesis where a product manag-er, designer or developer makes a prediction of how changing a feature, design or process could result in a more positive user experience. The new variation is tested against the existing format and applied to a small test group of mobile app users to discover whether the hypothesis proves true or false. The results are then evaluated and the highest performer is adopted.

This has significant benefits in the mobile world, as app users have a high likelihood of churning if an experience doesn’t match up to their expectations. And by testing a change on a small group, the risk of offending a large cohort of your user base is minimized.

A/B tests start off testing relatively large differences in design, layout and flow. But if your user base is large enough to justify testing minute details—adding a little more color here, changing a phrase there—these small changes could result in a large enough revenue boost to cover the cost of testing and increase your bottom line.

As you iteratively test changes to your mobile app, you can come to understand better what drives cus-tomers and how to create an intuitive, seamless experience for your user base.

Sample platforms:

Optimizely Mixpanel Apptimize

Additional resources:

5 Reasons why App Developers fail in their A/B testing strategyEmpathic A/B testing

Key value benefits of A/B testing for mobile apps:• Product and UX/UI Designers: Experiment with messaging, color schemes, screen flows, layouts

and more to optimize usability

• Developers: Make changes to an app without having to go through the entire app store approv-al process

• Marketers: Experiment with marketing tools like push notifications to optimize user engage-ment or pricing to identify an optimal price point

12

Crash Reporting and Analytics

“Feedback that comes early and often is the best way to improve your app experience.”

-Hemal Shah, Instagram

MONITOR + ALERT

Product managers, developers and QA personnel are constantly on the prowl to identify potential points of failure that could compromise the usability of their app or frustrate their users.

The goal is for an app to be as performant as possible in a variety of circumstances, so development teams rely on unit, alpha and beta testing to pinpoint as many issues as possible before moving into production.

Still, only so many different use cases can be tested—and there will almost inevitably be an untested use case wherein the app becomes less than performant. And when app failures occur, it affects reten-tion rates—as an example, nearly 34% of mobile users will uninstall an app if it keeps freezing.

When bugs plague the mobile user experience, it’s certainly something mobile app owners want to know about. Crash reporting platforms help monitor app performance and alert owners to distinct use cases and patterns in crashes.

WHAT IS CRASH REPORTING AND ANALYTICS?

Crash reporting and analytics tools provide a suite of diagnostic tools to help you identify broken use cases and the underlying causes of fatal and non-fatal errors, and in some cases helping mitigate cus-tomer dissatisfaction by providing immediate customer support.

The overall goal of crash reporting tools is to provide context into what circumstances led up to a crash and what functions were in effect when the failure occurred. This context comes by means of several different features in crash reporting tools, such as:

13

Log records are output by the product during runtime and outline functional processes. The most fundamental debugging resource, log records allow developers, QA engineers and product teams to break down the circumstances under which their product failed. Crash analytics tools help associate these records with errors.

Log Records

Breadcrumbs help provide context into what led up to a crash. They, in essence, help product stake-holders “retrace their steps” when a crash occurs in order to understand the factors that may have caused the failure.

Breadcrumbs

Network connectivity is a key determinant of app performance, and therefore a key factor to consider when evaluating the conditions under which an app must be performant.

Network Analytics

Heatmaps are a visual representation of how users interact with app screens, showing frequently touched areas as being “hot.” Heatmaps can provide insight into the factors leading up to an app error.

Heatmaps

Available in some crash reporting tools, crash video is a capture of user behavior preceding a crash, and when paired with log records and network data, allows QA personnel to replicate bugs.

Crash Video

Potential use cases• Identify points of failure causing app crashes

• Optimize performance by monitoring app functionality under various conditions

14

Key value benefits of crash reporting for mobile apps:• Product: Monitor feature performance and receive alerts when crashes occur

• Developers: Monitor app performance with detailed reports of crashes. Find and resolve broken use cases

• Marketers: Monitor app performance with detailed reports of crashes

Sample platforms:

Crashlytics

Additional resources:

iOS Crash Reporting Tools – 2017 Update

15

App Store Analytics

GROW + ATTRIBUTE

Talking about the app marketplace, John Koetsier, Mobile Economist at TUNE, recently said:

“You’ve worked hard and spent significant sums to evangelize your app, build it, and market it. You’ve probably paid for some app install ads, and you’ve invested in so-cial media marketing. Here is where it all pays off…or where it all dies.”

The rapid influx of mobile apps into the marketplace has presented significant challenges in visibility and discoverability for mobile app owners. Thousands of apps are added to the two principal app stores on a daily basis. There are currently more than 5 million apps in Apple’s App Store and the Google Play Store, with the App Store alone projected to have 5 million of its own apps by 2020.

If you have an app, participating in the mobile app marketplace is not optional—app stores are the gatekeepers to user downloads, whether users discover your app organically through app store search or your own campaigns pointing back to your install page.

This essential role of the app store has sparked the discipline of app store optimization (ASO), where marketers and product teams work together to cut through the noise and ensure downloads lead to en-gaged users. As with other digital marketing disciplines, ASO is fueled by data, and this data comes from a branch of mobile app analytics called app store analytics.

16

Potential use cases• Segment out which marketing campaigns result in the most engaged users

• Experiment with various app screenshots, descriptions or logos on your app store page to iden-tify and adopt the highest performer

• Use in-app purchase data from your principal analytics platform in conjunction with app store keyword data to target revenue-driving prospects

While much of this information is available to developers through the app store itself (Apple has its own proprietary App Store Analytics platform), there are also third-party enterprise platforms who can extract that data from your app store and provide even more capabilities.

WHAT IS APP STORE ANALYTICS?

App store analytics tools provide key data about the mobile customer journey. Much of this data focuses on optimizing the acquisition phase, but well-rounded ASO platforms also provide intelligence concern-ing late-stage user experience and retention metrics, such as uninstalls, ratings and reviews. Some of the most valuable features in app store analytics platforms include:

• Key acquisition metrics such as downloads, revenue and uninstalls

• App store session data (impressions, clicks, devices, etc.)

• Audience segmentation

• Campaign attribution

• A/B testing (for app store page)

• Ratings/Reviews/Sentiment analytics

• Keyword research

• Market analysis

17

Sample platforms:

TUNEAppfiguresApp Annie

Additional resources:

Apple’s App Analytics

Key value benefits of app store analytics:• Product: Check user engagement data against in-app metrics for any large discrepancies

• Marketers:

• Observe user growth and retention, perform campaign attribution and segment users based on their download data

• Attribute downloads to specific marketing campaigns

• Experiment with App Store messaging and graphics

18

Voice of Customer Analytics

LISTEN + SUPPORT

Rate of defection, or churn, in the mobile app world is rampant. Being able to address concerns quick-ly—even preemptively—will mitigate the risk of losing customers that cost you real money to acquire.

Consumers often voice their opinions concerning products in the form of social media posts, help fo-rums, survey responses, online reviews and more. But because these unstructured data formats do not fall into cut-and-dry data models, analyzing a large volume of these highly relevant conversations can be a challenge.

Understanding these insights at scale becomes possible by utilizing a Voice of Customer (VoC) platform to sift through and organize unstructured data into indexed, query-enabled information.

According to a report by Aberdeen Group, best-in-class VoC users enjoy 55% greater customer retention rates and grow annual revenue by 48.2% year-over-year. Those definitely aren’t numbers to balk at.

“Every great relationship is a two-way street. Unfortunately, too many companies really shy away from this approach to their customer relationships – they talk at, but don’t lis-ten to customers, leading to huge communications gaps. When companies truly invest in listening, across the entire organization, they are able to glean insights and learn more about where they should be going next with their product, service and business.”

-Robi Ganguly, Apptentive

19

Potential use cases• Segment out users that gave a high satisfaction rating in a survey and look at their behavior in

order to form potential models for successful customer journeys

• Monitor for social advocates and prompt these users to provide a review in the app store

• Optimize screen flows by segmenting user behavior data for users who had trouble finding desired content and voiced their concerns on review sites, forums or in customer support calls

WHAT IS VOICE OF CUSTOMER ANALYTICS?

VoC platforms contain a suite of data-parsing algorithms, including natural language processing, text recognition, speech recognition, sentiment analysis and more. Data can be parsed based on predefined models (e.g. predefined phrases), as well as be indexed for later queries. Combining algorithms with data collection methods like surveys and social listening allows companies to keep their eyes and ears open on a vast digital plain.

VoC platforms can also be linked with your primary in-app analytics vendor, allowing you to segment user data based on VoC data.

20

Sample platforms:

ApptentiveUser Hook

Additional resources:

Use Voice of Customer Data to Improve Customer Experience AnalyticsGet the Most from your Voice of Customer Data

Key value benefits of VoC mobile app analytics:• Product:

• Listen to customer feedback to refine product roadmaps

• Support new features to cater to customer’s needs and wants

• Create additional documentation to answer consumers’ questions

• Quality Assurance:

• Identify broken use cases affecting the quality of the mobile app expe-rience

• Marketers:

• Listen to customers’ needs and interests for more effective marketing messaging

• Segment users based on VoC data for targeted marketing campaigns

21

Push Analytics

“The good thing about notifications is they remind your users that your app is installed. A bad thing about notifications is they remind your users that your app is installed.”

-Sam Jarman, Sailthru

NOTIFY + ENGAGE

Push notifications are a dual-edged sword—depending on how you use them they can be a powerful tool to re-engage users, or a source of annoyance that results in an uninstall.

According to Chris Maddern, co-founder and CPO at Button, “71% of all app uninstalls are triggered by a push notification.” Consequently, it’s imperative that you optimize your notifications using robust mo-bile app analytics so that push messaging is personalized, timely and impactful.

WHAT IS PUSH ANALYTICS?

Push analytics provide insight into how users engage with your notifications.

Primary metrics include notifications delivered, opens and open rate, which give you a high-level under-standing of how push efforts affect user engagement.

You can go even deeper, recording data such as revenue, app uninstalls and notification opt-outs, which allow you to see how push notifications can affect long-term customer value. This will help you to find the sweet spot for notification cadence.

You can also segment mobile app users based on push engagement. Similar to the way an advertiser can set a cap on how many times a viewer sees an advertisement, you can segment users who never engage with notifications and target these consumers at a different cadence or with a different style of notifica-tion, or reach out to them on other channels.

22

Potential use cases• eCommerce/Retail: Use historical data like purchases and wishlist items to notify users when

relevant items go on sale

• eCommerce/Retail: Use geographic data to prompt in-store visits when users are in close prox-imity of a store or sale

• Finance: Notify users when stock they browsed has an earnings call or increases in value

• Finance: Prompt users to view the details of how they exceeded a budget for the month

• Entertainment: Use viewing behavior combined with recommendation engines to suggest new-ly released content

• News & Media: Use average time spent on screen paired with article/video metadata to suggest newly published content

• Overall: Use A/B testing to identify which type of push messaging best resonates with your users

As with all efforts to personalize experiences for audiences of one, automation is required. When per-formed properly, push analytics will allow you to personalize beyond just inserting the user’s name into the notification. You will be able to use user behavior data to fuel push messaging and notification cadence, keeping loyal users active and re-engaging passive users.

23

Sample platforms:

OneSignalUrban AirshipLeanplumCarnival

Additional resources:

Why Push Messaging Demands Powerful AnalyticsHow To Send More Personalized Push NotificationsThe Must-Follow Best Practices for Your Push Notifications

Key value benefits of push mobile app analytics:• Product:

• Notify users of new versions, features and content

• Increase retention rates by re-engaging users

• Marketers:

• Promote customer re-engagement and retention

• Evaluate and understand optimal user engagement windows

24

Mobile App Testing

“User demands are ever changing, and the integrity of your analytics strategy (and the insight you glean from your tech stack) holds critical value in keeping your app highly competitive. Mobile app analysis is equally as important to mobile app development. Through quality app analytics data you can identify your application’s strengths and weaknesses, anticipate user behavior, and deliver a better experience. But how can you know your data is accurate? You need to know what to measure and how to test your data collection.”

OBSERVE + VALIDATE

As can be gathered from the explanations above, there is a vast array of mobile app analytics technolo-gies available on the market. Making robust data collection via these mobile analytics vendors possible requires the use of mobile SDKs.

A mobile software development kit (SDK) is a collection of resources provided by a vendor to assist a developer in integrating the vendor’s technology into the developer’s mobile application. At a minimum these kits contain libraries of code that perform key functions to make the technology useful, without requiring the developer to write an excessive amount—or any—custom code on an app.

Deploying mobile SDKs is an integral part of data-driven app design and innovation. It is the key techni-cal component of mobile app analytics. As such, planning which SDKs to use and how they will be used should be part of the pre-development strategy and early business requirements documentation.

Developers build their apps to comply with business requirements to include all the necessary SDKs. But following deployment, they need to continuously monitor how device type, operating system and app version affect vendor performance.

Mobile testing can help you ensure your mobile app analytics vendors work properly on multiple devic-es, OSes and app versions.

-Sun Sneed, ObservePoint

25

WHAT IS MOBILE APP TESTING?

There are a variety of mobile testing categories, including unit testing, alpha/beta testing, device testing and others. Because this text is primarily concerned with mobile app analytics, the type of mobile testing explained here may be better referred to as mobile SDK validation.

Mobile SDK validation does a number of things, including:

• Ensure your mobile SDKs function under various conditions beyond the environmental factors stated above, such as:

• What happens when your app runs in the background

• What happens when the user has multiple apps open

• How analytics technologies respond to various screen flows

• Validate custom analytics events and variables are set and recorded as expected

• Foster data integrity by protecting against data loss or inflation. If SDKs are deployed twice on the same app screen, or not at all, it will eschew your app data, resulting in false insights

• Promote app security by checking that no unauthorized vendors or accounts are collecting and transmitting data

Mobile SDK implementation may be easy, but maintenance is a key issue. Keeping your data clean should be a top priority, and thus so should mobile testing.

Potential use cases• Ensure certain categories of screens all have the same technologies

• Validate that the proper variables are being captured on specific pages

• Receive a notification any time a validation rule fails

26

Key value benefits of testing your mobile app analytics:• Product/UX Designers/Marketing: Maintain clean product analytics data for accurate deci-

sion-making and evidence-based strategy

• Developers: Test that all deployed technologies function on early-stage prototypes of their applications

• Quality Assurance: Validate that business requirements are continuously met as new features or technologies are integrated into the stack

Sample platforms:

ObservePointTestmunkAppium

Additional resources:

7 Ways to Win at Testing Your Mobile App Analytics (Webinar)Meeting Your Mobile Analytics Challenges Through Automation (Webinar)Mobile App Testing Strategy: Ensure Mobile App Success in 7 Steps

27

Building a Comprehensive Mobile AppAnalytics Technology Stack

Bringing a top-notch app to the marketplace is an iterative process. Knowing what to iterate on requires behavioral data from your customers, showing you what delights them, what works well for them and what aggravates them.

Building the optimal mobile analytics technology stack is a long-game effort—as you apply both breadth and depth in your mobile analytics strategy, you will be able to better understand your user base and how they inter-act with your app.

The end result: an exceptional user experience.

28

AppAssurance™ is ObservePoint’s patented technology for scanning mobile apps, capturingdata SDK functionality, and validating your implementation against pre-defined compliance rules.

Request a free sample audit of your app to see how automation can streamline testing processes.

REQUEST A SAMPLE AUDIT

29

ABOUT THE AUTHORSUN SNEED

Sun is currently Director of Product Management at Ob-servePoint. She is passionate about internet products and marketing. Sun conceptualizes and drives change in an impactful and sustainable way. Before becoming Director of PM, Sun oversaw product innovation for AppAssurance™, ObservePoint’s mobile app tag and data quality platform. In past roles, Sun has contributed to the product innovation of Deutsche Telekom, T-Online International AG, and Fast Multimedia AG.