byo/diy analytics platform (measurecamp presentation by clancy childs)

Post on 29-Nov-2014

352 Views

Category:

Data & Analytics

0 Downloads

Preview:

Click to see full reader

DESCRIPTION

The slides accompanying Clancy Childs' talk at Measurecamp V (2014) in London. Might be missing a lot if you weren't at the session, but basically covering some of the design decisions, pitfalls, technology choices and requirements when choosing to build your own analytics / eventing platform and data warehouse.

TRANSCRIPT

BYO/DIY Analy,cs  Pla1orm

Clancy  Childs  Founding  Partner,  Inderus  (Google  Analy,cs  Refugee)  

!clancy@inderus.com  @cchilds  -­‐  @inderus

My  Past  Life

What  is  in  the  Big  Mac  Sauce?

hLp://inder.us/big-­‐mac-­‐recipe

WHY  would  you  ever  WANT  to  build  your  own  Analy,cs  Solu,on?  

“See all that stuff in there, Homer? That’s why your robot never worked.”

Tools  Are  Generally  Good  Things

The  Right  Tool  for  the  Job

…but  eagerness  to  use  a  tool  can  be  at  the  cost  of  geWng  the  job  done  correctly

Why  Some  Companies “Roll  Their  Own”  Analy,cs

• End-­‐to-­‐end  customisa,on  (collec,on,  processing  and  repor,ng)  

• Complete  control  and  governance  of  data  

• Integra,ons  with  other  opera,onal  and  repor,ng  systems  

• Ability  to  make  their  own  mistakes  on  their  own  terms

An  Insanely  Simplified  Analy,cs  Processing  Model

Collection

Processing

Enrichment

Extraction VisualisationStorage

An  Insanely  Simplified  Analy,cs  Processing  Model

Collection

Processing

Enrichment

Extraction VisualisationStorage

Collec,on

• First  vs.  Third  Party  beacons  

• Client  vs.  Server  Side  collec,on (over  GTM’d?)  

• Opera,onal  By-­‐Products  (Logging)  

• Snowplow,  Logstash,  Kinesis,  Kaaa(?)

An  Insanely  Simplified  Analy,cs  Processing  Model

Collection

Processing

Enrichment

Extraction VisualisationStorage

Processing  and  Enrichment• Sessioniza,on  and  User  S,tching  

• Processing  Schedules  and  Goals(Lambda  Architecture?)  

• Reprocessing  vs  “Golden”  Immutability  

• Captured  Foreign  Keys  (gclid,  IP  address,  etc.)  

• MapReduce,  Storm,  Flume…

An  Insanely  Simplified  Analy,cs  Processing  Model

Collection

Processing

Enrichment

Extraction VisualisationStorage

Storage

An  Insanely  Simplified  Analy,cs  Processing  Model

Collection

Processing

Enrichment

Extraction VisualisationStorage

Query/Extract  and  Visualize

• SQL  is  generally  much  easier  than  anything  else.  

• Prototyping  with  Pandas  /  R  

• Produc,on  Dashboarding  with  Visualiza,on  tools  

• Char,o,  Looker,  Tableau,  Klipfolio,  Legronic,  others?

Thank  You!Clancy Childs - @cchilds

Inderus - @inderus clancy@inderus.com

top related