tools and tech for big data success
Post on 19-Oct-2014
1.305 views
DESCRIPTION
What do you need to succeed in working with Big Data? RedMonk analyst Donnie Berkholz will present quantitative research on the state of the field, covering the breadth of languages, tools, and infrastructure, to show you which choices to make today and which ones you'll need to get ready for, soon.TRANSCRIPT
![Page 1: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/1.jpg)
Tools and tech for Big Data success
Donnie Berkholz, Ph.D.IT Industry Analyst@dberkholz
[Imagine cheesy clipart of hammers, silicon, etc.]
![Page 2: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/2.jpg)
Tools and tech — huh?
● Languages● Infrastructure● Tooling● It's not about the what, it's about the how
![Page 3: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/3.jpg)
Thesis: Technology adoption is increasingly bottom-up
![Page 4: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/4.jpg)
The new kingmakers
![Page 5: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/5.jpg)
The best DX wins
![Page 6: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/6.jpg)
It's all about barriers to entry
![Page 7: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/7.jpg)
Ecosystems matter
![Page 8: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/8.jpg)
The challenges
IT pros delivering data solutions60/36 IT/biz
Talend, summer 2012
![Page 9: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/9.jpg)
The skills gap
NVP survey of execs, summer 2012
![Page 10: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/10.jpg)
What can we learn from book sales?
![Page 11: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/11.jpg)
![Page 12: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/12.jpg)
What about the data analysts?
KDNuggets, summer 2012
![Page 13: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/13.jpg)
What about the data analysts?
KDNuggets, summer 2012
![Page 14: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/14.jpg)
It's not all about popularity...
![Page 15: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/15.jpg)
The real growth isn't in SQL interfaces
![Page 16: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/16.jpg)
Developers vs. marketers
Google Trends
![Page 17: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/17.jpg)
IT pros delivering data solutions
60/36 IT/bizTalend, summer 2012
![Page 18: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/18.jpg)
Hadoop distributions: popularity
Google Trends: $VENDOR hadoop
![Page 19: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/19.jpg)
![Page 20: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/20.jpg)
Developers choose what's easiest
![Page 21: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/21.jpg)
![Page 22: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/22.jpg)
How to run: bare metal / DC, private cloud, public cloud
![Page 23: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/23.jpg)
Storing Data: HDFS, Ceph, Gluster
![Page 24: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/24.jpg)
Open Building Blocks but not Open Source: EMR on AWS
![Page 25: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/25.jpg)
Realtime: Impala, Druid
![Page 26: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/26.jpg)
In-memory: Redis, Memcached
![Page 27: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/27.jpg)
Streaming: Storm, S4
![Page 28: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/28.jpg)
Other Options: Hadoop YARN, HPCC, Cassandra, Mongo, Riak
![Page 29: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/29.jpg)
Other Options: Spark/Shark/Mesos
![Page 30: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/30.jpg)
Abstraction: Mortar, Continuuity, Qubole, Concurrent Lingual, etc.
![Page 31: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/31.jpg)
Conclusion: KISS
![Page 32: Tools and Tech for Big Data Success](https://reader034.vdocuments.us/reader034/viewer/2022051816/54445b90b1af9f600a8b489c/html5/thumbnails/32.jpg)
Donnie Berkholz, Ph.D.Analyst, RedMonk@dberkholz
Disclosure: 10gen, Amazon, Basho, Cloudera, Continuuity, IBM, MapR, Microsoft, and VMware are clients.