1. Making Big DataSmall:Big Data for the Rest of UsNovember 20121

2. 2000Google announced it had released the largestsearch engine on the InternetGoogles new index comprised more than 1 billionURLsBIG!!!2 3. 2008Our indexing system for processing links indicatesthat we now count 1 trillion unique URLs(and the number of individual web pages out thereis growing by several billion pages per day)BIGGER!!!!3 4. An unprecedentedamount of data is beingcreated and is accessible4 5. 5 6. 6 7. Applies to more than just CPUs Summary version? Things double at regularintervals Its exponential growthand applies to Big DataBBC: Your current PC is more powerful than the computer they had on board the first flight to the moon.7 8. 9,000900067504,40045002,15022501,000500 55 120 250 1 4 10 24 02000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 20118 9. 9 10. 10 11. 11 12. Source: Silicon Angle, 201212 13. Source: Silicon Angle, 201213 14. Source: Silicon Angle, 201214 15. Storage/OperationsProcessing/Analytics15 16. 16 17. In 1998 Google won the search race throughcustom software and infrastructure In 2002 Amazon again wrote custom andproprietary software to handle their BIG Dataneeds In 2006 Facebook started with off the shelfsoftware, but quickly turned to developing theirown custom-built solutionsWhat do these have in common? Big Data was critical to making them win 17 18. 18 19. 19 20. MS Office -> OpenOffice Oracle DB ->PostgreSQL Unix -> Linux Weblogic -> JBoss Documentum -> Alfresco Cognos ->Pentaho/Jasper Salesforce ->SugarCRM Informatica -> Talend iOS -> Android (?) Etc.20 21. Web Innovation OSS companiesVendor-sponsoredIndividual developers21 22. 22 23. The best minds of my generation are thinking about how to make people click ads. (Jeff Hammerbacher)23 23 24. Where Do We Go from Here?24 25. Agile Development Iterative & continuous New and emerging appsVolume and Typeof Data Trillions of records 10s of millions of New Architecturesqueries per second Systems scaling horizontally, Volume of data not vertically Semi-structured and Commodity serversunstructured data Cloud Computing 25 26. stormApache Drill 26 27. 27 28. Worlds Most Popular Big Data Sources, 2012Source: JasperSoft, 2012 28 29. The future ishu MONGOus 29 30. 5,900 companies evaluated. 10gen is #1 in Software and #9 overall30 31. Relational databases[dont] necessarily match the way we see ourdata. mongoDB gave us the flexibility to store data in the way thatwe understand it as opposed to somebodys theoretical view.Its friendly. By friendly, I mean that coming from a relationalbackground, specifically a MySQL background, a lot of the conceptscarry over.... It makes it very easy to get started.Selecting MongoDB as our database platform was a no brainer as thetechnology offered us the flexibility and scalability that we knewwed need for Priority Moments. 31 32. 32 33. 33 34. 34 35. 35 36. 36 37. @mjasay37