1. 1. Mastering the New Matrix: Big Data
2. 2. Agenda The Matrix Data & Data Everywhere Big Data & Four Vs Seeing Big Data Are you Neo? 2014 BizKnowlogy - Philip Topham
3. 3. The Matrix NEO - Agent Smith 2014 BizKnowlogy - Philip Topham
4. 4. Thomas A. Anderson is a man living two lives. By day he is an average computer programmer and by night a hacker known as Neo. Neo has always questioned his reality, but the truth is far beyond his imagination. Neo finds himself targeted by the police when he is contacted by Morpheus, a legendary computer hacker branded a terrorist by the government. Morpheus awakens Neo to the real world, a ravaged wasteland where most of humanity have been captured by a race of machines that live off of the humans' body heat and electrochemical energy and who imprison their minds within an artificial reality known as the Matrix. As a rebel against the machines, Neo must return to the Matrix and confront the agents: super-powerful computer programs devoted to snuffing out Neo and the entire human rebellion. 2014 BizKnowlogy - Philip Topham
5. 5. Agent Smith Keeps us Trapped In Data 2014 BizKnowlogy - Philip Topham
6. 6. Trapped by Data What is Data? 2014 BizKnowlogy - Philip Topham
7. 7. Taste Data and the Human Senses Touch Sight SoundSmell How would you describe a Caribbean island? 2014 BizKnowlogy - Philip Topham
8. 8. Number? Spreadsheet? Software? Smart phone? What is Data? 2014 BizKnowlogy - Philip Topham
9. 9. An abstract representation of something Five. 5. (go). Feet. Miles. Pennies. Dollars Atomic number1 Speed. Lightyears. Miles Per Gallon. Acceleration. Meters/Second2 Data is.. 1 Boron 2014 BizKnowlogy - Philip Topham
10. 10. Five. 5. (go). Five Story Building Data is An abstract representation of something made meaningful in context 2014 BizKnowlogy - Philip Topham
11. 11. Wind Speed Data Goes Beyond the Five Senses Temperature GPS Salinity How would you describe a Caribbean island? 2014 BizKnowlogy - Philip Topham
12. 12. Microsensor Data TasteTouch Sight SoundSmell Touch screen Digitizer Motion sensor Accelerometer Ambient light sensor Proximity sensor Digital cameras Gyroscope Moisture Sensor Cellular, WiFi, Bluetooth 2014 BizKnowlogy - Philip Topham
13. 13. Sensors, Machines, People The Internet of Things Data is Everywhere 2014 BizKnowlogy - Philip Topham
14. 14. Is Big Data The Modern Matrix - Are we Trapped? 2014 BizKnowlogy - Philip Topham
15. 15. Humungous Gargantuan Immense Enormous Colossal Tremendous Monumental Titanic Elephantine Brobdingnagian Mammoth Extensive Sizable Huge Great Vast Voluminous Spacious Whopping Astronomical What is ??? 2014 BizKnowlogy - Philip Topham
16. 16. Junk Drawer by Marc Miller https://www.flickr.com/photos/markmarkmark/4551476025/ Velocity Variety Volume Veracity video voice text sensors many truths real-time Four Vs What is Big Data? 2014 BizKnowlogy - Philip Topham
17. 17. Recent History Hourly Who attacked our website? Realtime Realtime watching Who is attacking our website now?? Weekly / Monthly What did you buy this week? Realtime spending alerts Are you over spending now? Preferences What web add do I show you? Realtime buying Make instant suggestions Daily fraud alerts Did any fraud happen today? Realtime fraud detection Block fraudulent use now. Velocity 2014 BizKnowlogy - Philip Topham
18. 18. Online in 60 Seconds is an infographic that was produced by qmee.com (2013)
19. 19. From the dawn of time civilization to 2003, humankind generated five exabytes of data. Now we product five exabytes every two daysand the pace is accelerating Eric Schmidt, Executive Director Google Volume 2014 BizKnowlogy - Philip Topham
20. 20. Whats an Exabyte? One non-stop DVD film, starting 50,000 years ago, when Homo Sapiens, first arrived in North America equals one Exabyte! Whats an Exabyte? 2014 BizKnowlogy - Philip Topham
21. 21. Structured Highly organized data, saved in repeatable ways that allows for easy access, manageability, and use between computer systems. Unstructured Recorded data, saved without much regard for interoperability across computer systems (a specific program must be used to use or search the data) http://www-i6.informatik.rwth-aachen.de/web/Research/speech_recog.html Speech recognition waveformData entry form Variety 2014 BizKnowlogy - Philip Topham
22. 22. Structured Databases Data Entry forms Sensor data (at a point in time) eg. GPS Unstructured Freeform text, speech, movies Sensor data Chemical sensors Heat probes Wave detectors (sound, light, X- rays, etc.) Motion (accelerometers, gyroscopes) Magnetometers Pressure Human data curation Little to no curation Variety 2014 BizKnowlogy - Philip Topham
23. 23. Structured Known source Data curated (check for errors) Unstructured Uncertain source Unclear trail of custody Occasionally purposeful untruths or half-truths Hidden meaning (sarcasm) Unclear encoding or changing encoding Human data curation Little to no curation Veracity 2014 BizKnowlogy - Philip Topham
24. 24. Learning to See Big Data 2014 BizKnowlogy - Philip Topham
25. 25. Standard Computing The Old Way 2014 BizKnowlogy - Philip Topham
26. 26. Big Data Does NOT FIT into One computer But60 Trillion Web pages!! http://www.google.com/insidesearch/howsearchworks/thestory/ 2014 BizKnowlogy - Philip Topham
27. 27. Thousands of Computers Parallel Computing How did Google Do it? 2014 BizKnowlogy - Philip Topham
28. 28. How did Google Do it? Divide and Conquer Page Rank 2014 BizKnowlogy - Philip Topham
29. 29. How did Google Do it? Divide and Conquer Count Words 2014 BizKnowlogy - Philip Topham
30. 30. How did Google Do it? Divide and Conquer Count Words frequency Book 1: The quick brown fox Book 2: Bill was as quick as a fox 2014 BizKnowlogy - Philip Topham
31. 31. How did Google Do it? Divide and Conquer: Mapping Data The quick brown fox Bill was as quick as a fox Brown: 1 Fox: 1 Quick: 1 Bill: 1 Fox: 1 Quick: 1 Was: 1 Ignore little words 2014 BizKnowlogy - Philip Topham
32. 32. How did Google Do it? Divide and Conquer: Reducing Results Brown: 1 Fox: 1 Quick: 1 Bill: 1 Fox: 1 Quick: 1 Was: 1 Bill: 1 Brown: 1 Fox: 2 Quick: 2 Was: 1 A-M N-Z 2014 BizKnowlogy - Philip Topham
33. 33. How did Google Do it? If they Count Words why Page 17 2014 BizKnowlogy - Philip Topham
34. 34. How did Google Do it? Divide and Conquer Add Location Page 17Page 1 2014 BizKnowlogy - Philip Topham
35. 35. How did Google Do it? Divide and Conquer Apply Information Theory Dress Carrot Lemon Cupcake + 2014 BizKnowlogy - Philip Topham
36. 36. How did Google Do it? Divide and Conquer Count words Word Importance Title, Heading, Font Size Geolocation Related Importance 2014 BizKnowlogy - Philip Topham
37. 37. How did Amazon Do it? Divide and Conquer Count BehaviorsWesterns 2014 BizKnowlogy - Philip Topham
38. 38. How did Amazon Do it? Divide and Conquer Count Behaviors Did I browse (click)? buy? 2014 BizKnowlogy - Philip Topham
39. 39. How did Amazon Do it? Divide and Conquer Count Behaviors AMAZON PATENT US7113917 B22014 BizKnowlogy - Philip Topham
40. 40. National League Baseball 2014 BizKnowlogy - Philip Topham
41. 41. Im not Amazon or Google? Now What?
42. 42. Are you Neo? Will you look beyond the data? Will you create a new story? 2014 BizKnowlogy - Philip Topham
43. 43. Big Data Thinking DATA - Unimportant alone CONTEXT - Creates meaning BIG DATA - Is overwhelming. It has no context BUSINESS - Our world, always has context DIVIDE and CONQUER - Split into small parts Five. 5. (go). Five Feet Five Gazillion ??? 5% Increased Sales 1 + 2 + 3. 2014 BizKnowlogy - Philip Topham
44. 44. Big Data Skills Base Skills: Mathematics, Algorithms & Data structures Filter&Mine Acquire&Clean Represent Visualize Domain Expertise Data Skills divide and conquer 2014 BizKnowlogy - Philip Topham
45. 45. Big Data Thinking Patterns Emerge 2014 BizKnowlogy - Philip Topham
46. 46. Our Reflected Selves = Aggregated Data in Context The Modern Matrix 2014 BizKnowlogy - Philip Topham
47. 47. ONENEO 2014 BizKnowlogy - Philip Topham