Web analytics is dead! Long live event analytics

How data scientists and big data tech are killing one industry and creating another

What role Snowplow plays

Web analytics is a big industry

• Spend in the just the US on web analytics software (Adobe Sitecatalyst, Webtrends, Google Analytics Premium etc.) estimated at $500m and growing 17 – 20% p.a. in 2011*

• Likely that at that amount is spent again on consulting services related to the use of web analytics data• Whole industry of web consultants e.g.:

• Semphonic in the USA (bought by Ernst and Young)• Logan Tod in the UK (bought by PwC)• Big 4 accounting firms only buy businesses they can sell into (tens of) thousands of

companies

• Whole ecosystem around web analytics• “Digital analytics professionals” – it is a career path (retailers, media agencies)• Events, books, organisations geared towards web analysts

*Source: Quora http://www.quora.com/Web-Analytics-what-is-the-size-of-the-web-analytics-market

Web analytics is an old industry, predating the recent wave in big data technology

Web analytics Big data

1990

1993

1996

1997

2004

2006

2008

2010

Web is born

Log file based web analytics

Javascript tagging

publishes MapReduce paper

Hadoop project split out of Nutch

Facebook develops Hive

publishes Dremel paper

2011 open sources Storm

…

Two problems with web analytics, that stem from the fact web analytics came of age in the 1990sThe web was static, hyperlinked documents Tech to handle massive data sets was

prohibitively expensive

• The entities and events that web analytics programmes understand is limited

• Page views, link clicks, transactions, goals, sessions, visitors

Hard to model the rich interactions in today’s interactive webapps

• Web analytics programmes aggregate raw data to reduce data volumes

• This requires specifying in advance how data can be analysed, so that the data can be ‘pre-cut’

Web analytics reporting is very inflexible

In particular, web analytics insistence on aggregating data is an anathema to data scientistsData scientist approach Web analytics approach

Give me the data and I’ll figure out how to answer the question

You can’t get your answer from one of our pre-canned reports? Have a go with our “advanced report-builder”

What if I want to: build a model? Understand underlying causality? Use the data in my web application? Dynamically optimize spend / content?

We built Snowplow to address the two weaknesses in the web analytics approachDescribe web events in much richer grammar and vocabulary

Liberate your data• Where you store your data has a big

impact on what types of analyses you can quickly run on it

Snowplow is an event data collection and warehousing platform

Website / webapp

Mobile apps

Other applications (e.g. on games consoles, connected TVs, desktops,

connected devices)

Snowplow data pipeline

CollectTransform

and enrich

Amazon Redshift /

PostgreSQL

Other (Neo4J,

BigQuery…)

Amazon S3

Snowplow delivers your complete, granular event data in your own data warehouse(s), so you can

plugin any tool to analyse it

Snowplow is composed of a set of loosely coupled subsystems, architected to be robust and scalable

1. Trackers 2. Collectors 3. Enrich 4. Storage 5. AnalyticsA B C D

A D Standardised data protocols

Generate event data

Examples:• Javascript

tracker• Ruby / Lua /

No-JS / Arduino tracker

Receive data from trackers and put it in a queue

Examples:• Cloudfront

collector• Clojure

collector for Amazon EB

Clean and enrich raw data

Built on Scalding / Cascading / Hadoop and powered by Amazon EMR

Store data ready for analysis

Examples:• Amazon

Redshift• PostgreSQL• Amazon S3

Snowplow is open source and cloud-based

• Open source but easy to deploy via integration with Amazon Web Services (cloud infrastructure)

• Our technology is free!

• Collecting massive quantities of digital event data should be easy and cheap…

• … so that we can focus time and effort on using the data productively

• We charge for Professional Services on top of our platform

• More value in how you use the data, than in collecting / storing it

• Lots of scope to build applications on top of our platform going forwards

Our users…

…use our tech to solve some of their most intractable problems• What is the impact of different ad campaigns and creative on the way users

behave, subsequently? What is the return on that ad spend?

• How do visitors use social channels (Facebook / Twitter) to interact around video content? How can we predict which content will “go viral”?

• How do updates to our product change the “stickiness” of our service? ARPU? Does that vary by customer segment?

We believe that event data is one of the most exciting data sources to work with, today

We are only at the beginning of figuring out how to use this data…• How do we represent different types of event sequence?

• What makes journeys similar and what makes them different? How can we cluster them?

• How can we “spot” those events that are predictive of future events? Of consumer value? Of consumer interest?

• How can we unpick the effects of marketing / digital products and user’s predisposition to the way sequences of events unfold?

• How best should we model different users at different points on different types of journeys?

We hope people like you will use our tech to do amazing things with the data!

More information

• Snowplow repo: https://github.com/snowplow/snowplow

• Twitter: @SnowPlowData

• Website: http://snowplowanaltyics.com

• My LinkedIn:

• My Twitter:

Questions?