![Page 1: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/1.jpg)
Analytics from 330 million
smartphonesSean Byrnes
CTO & Co-founder
![Page 2: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/2.jpg)
Flurry Overview
60,000
160,000
App Developers:
Live Applications:
Flurry Analytics Better apps on iOS, Android, BB, WP, HTML5
480MDevices per month:
33BSessions per month:
AppCircle NetworkAcquisition & Monetization: iOS, Android
6,200App Developers:
200MDevices per month:
300BEvents per month:
3MDaily Completed Views
![Page 3: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/3.jpg)
How Flurry Works
![Page 4: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/4.jpg)
Flurry’s Scale
1.2 Billions Sessions / Day
900 Servers
1.56 PB
![Page 5: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/5.jpg)
Topics
1. Big Data Collection (HDFS)
2. Big Data Processing (Hadoop)
3. Data Mining at Scale (Hbase)
![Page 6: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/6.jpg)
BIG DATA COLLECTION
![Page 7: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/7.jpg)
Incoming Data
Peak Connections per Second: 25,000 Data per day: 1.5 TB
![Page 8: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/8.jpg)
Data Collection
Reports
Load BalancerLoad Balancer
Load BalancerData Collector Load BalancerData Collector Load BalancerData Collector
File File File
HDFS
![Page 9: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/9.jpg)
Data Collection
Reports Reports
HDFS HDFS
Location A Location B
![Page 10: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/10.jpg)
BIG DATA PROCESSING
![Page 11: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/11.jpg)
11
Normalization
Data Correction
Metrics Computation
Agent Report
De-duplication
Portfolio Analysis
Benchmarking Clustering
Identify Device, Country,Carrier, etc.
Bad Phone ClocksPartial Session Reports
Handle duplicate reports
Flexible calculationConfigurable Dimensions
Data mining and analysis
Audience Segmentation
Industry Trends Application Analytics
MerchandisingAnalytics
Analytics Processing
![Page 12: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/12.jpg)
Large-scale Data Processing
Input Data
NoSQL DataStore
Real-Time Batch
Collectors
Consumer/ ProducerSystems
MapReduce(jobs)
External Action
External Action
![Page 13: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/13.jpg)
Map/Reduce Management
• Challenge: Task Starvation
• Challenge: Task Roadblocking
• Challenge: Network Connection Waiting
![Page 14: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/14.jpg)
Network Topology: Chained
Rack 1 Rack 2
Switch 1 Switch 2
Rack 3
Switch 3
![Page 15: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/15.jpg)
Network Topology: Star
Rack 3 Rack 2
Switch 3 Switch 4
Switch 1 Switch 2
Trunk
Rack 1 Rack 2
![Page 16: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/16.jpg)
DATA MINING AT SCALE
![Page 17: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/17.jpg)
Stages of Data
Normalized OLAP CubeRaw Data
80 Billion Rows160 Billion Rows500 Billion Records
![Page 18: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/18.jpg)
NoSQL Tables
111111111 Data Data
Index Column Family A Column Family B
222222222 Data Data
333333333 Data Data
![Page 19: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/19.jpg)
NoSQL OLAP
metric.dimension
Index Column Family A
#
metric.dimensionA
metric.dimensionB
metric.dimensionC
metric.dimensionA.dimensionB.dimensionC
metric.dimensionA.dimensionB
metric.dimensionA.dimensionC
...
![Page 20: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/20.jpg)
Lexicographical Ordering
metric dimensionA dimensionB index
3 1 1 3113 1 11 31113 11 1 3111
metric.dimensionA.dimensionB
![Page 21: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/21.jpg)
Lexicographical Ordering
metric dimensionA dimensionB index
3 001 001 30010013 001 011 30010113 011 001 3011001
metric.dimensionA.dimensionB
![Page 22: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/22.jpg)
NoSQL OLAP
metric.dimension.date
metric.dimension.1_1_12metric.dimension.3_1_12
Index
Row Scan
metric1/1/12
3/1/12
![Page 23: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/23.jpg)
blog.flurry.com
![Page 24: Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649f1e5503460f94c362e8/html5/thumbnails/24.jpg)
Sean [email protected]
Flurry, Inc.
282 2nd St. Suite 202
San Francisco, CA 94105
http://www.flurry.com