hw09 welcome to hadoop world

Welcome to Hadoop World: NYC 2009Hadoop is Everywhere

Christophe BiscigliaFounder [email protected]

Presents:

Hadoop World Details and Event UpdatesToo Late to Print

▪ WiFi Details▪ SSID: HadoopWorld▪ Password: hadoop09

▪ Twitter: #hadoopworld

▪ Break Out Sessions▪ Applications (This Room)▪ Dev / Admin: Terrace Ballroom (Across Lobby)▪ Extensions: Vanderbilt Suite (One Floor Up)

▪ UI BOF▪ Lead: Philip Zeyliger, Cloudera▪ Vanderbilt Suite, Afternoon Break

▪ HBase BOF▪ Lead: Michael Stack, Microsoft▪ Terrace Ballroom, Afternoon Break

Hadoop World SponsorsThanks!

Why Hadoop World?Time to Upgrade Your Data Management Strategy

▪ Hadoop isn’t just for Web Companies anymore▪ Terabytes are common place▪ Enables consumption of all enterprise data▪ Wide adoption across verticals

▪ Hadoop is driven by the Community▪ Most registrants are new to Hadoop▪ Sharing experience is critical - and incredibly valuable▪ Users and Developers exchanging needs and ideas

Growing Up with HadoopYou’ve come a long way baby...

▪ Early Days▪ 2004: Google Publishes MapReduce/GFS▪ 2005: Hadoop Prototype▪ Doug Cutting and Mike Cafarella

▪ 2006: Hadoop Running on 20 nodes▪ Internet Archive and UW


Doug CuttingPhoto Credit: New York Times

▪ Formative Years▪ 2006: Yahoo! Begins Major Investment▪ 2007: Yahoo! Runs Hadoop on 2000 nodes▪ 2008: Yahoo! uses Hadoop to claim Terasort

Benchmark



▪ 5 Major Releases for Hadoop in last year▪ More Reliable▪ More Scalable▪ More Manageable

▪ New Sub-Projects Embrace New Users▪ Hive: SQL Data Warehouse for Hadoop▪ Pig: Data Analysis Language


▪ Sqoop: Database import for Hadoop▪ Developer by Aaron Kimball, Cloudera▪ Works over JDBC▪ Extensible for better pefromance


▪ RDBMS Vendors Embrace Hadoop▪ MapReduce is great for Analytics▪ Hadoop is the MapReduce Standard▪ integrates directly with Hadoop



▪ Adoption Spanning Globe▪ HUGs outside the US▪ Over 10x Companies “PoweredBy”▪ Not Just for Web Companies Anymore

Cloudera’s Distribution for HadoopDelivering Hadoop to a Larger Community


Hadoop Community


Latest Stable Hadoop Release

Stable Upcoming Features (by customer request) Distribution for Hadoop

Hadoop Community

Source Code Powering Y!

Improvements for EC2 and S3

New Features from Cloudera




Hadoop Community

Source Code Powering Y!

Improvements for EC2 and S3

New Features from Cloudera

Cloudera EnhancementsBug Fixes

Contributed to Apache




Hadoop Community


Distribution for Hadoop

Cross-Platform Packaging,Integration and Testing

Hive, Pig, Sqoop, ...

Support





Support

Packages

Private Cloud

Public Cloud

Images





Support

Packages

Private Cloud

Comparing Growth Rates since March 2009Standard Packaging Drives Adoption

March 2009 May 2009 July 09 Aug 09 Sept 09

95%97%93%133%95%96%100%

1,835%

1,392%

1,026%

762%

384%

238%

100%

Cloudera DownloadsApache Downloads

▪ Consistent Downloads from Apache

▪ Cloudera Packages Drive New Usage

▪ Enables New Hadoop Applications

Normalized by unique users accessing hadoop.apache.org/core/releases.html and Cloudera Package Repositories in March 2009

Cloudera’s Business to DateSupport, Training and Professional Services

▪ Dozens of Support Customers▪ Using Hadoop for real enterprise workloads

▪ Training and Certification▪ 100’s of engineers trained▪ Sysadmin and Manager programs launched at Hadoop World

▪ Professional Services

hw09 welcome to hadoop world

Technology