AmbariFutures

April 2, 2013

Jeff Sposetti @ Hortonworks

Ambari Futures

2

Extend core capabilities to include the critical tasks

associated with provisioning and operating Hadoop clusters.

Cluster Operations

Enable insight into job performance and reduce the

burden on specialized Hadoop skills and knowledge.

Job Diagnostics

Expose integration and customization points so Hadoop

can interoperate with existing operational tooling.

Extensible Platform

Make Hadoop easier to manage

What’s Planned for 1.3.0

• Improved Configuration Mgmt with host-level overrides• HBase Multi-Master + Heatmaps• Multi-tenancy support via Capacity Scheduler• More Databases (MySQL and Oracle)• Centralized Stack Upgrade• Manage Kerberos Secure Cluster• External group mappings (LDAP/AD)• Job Diagnostics Improvements

– New Visualizations (swimlane, scatter plots)

Configuration Mgmt + Host Exceptions

Page 4

• Allows you to handle “heterogeneous” clusters– For example: “Not all my DataNodes are the same”

• Set configuration property exceptions for one or more hosts

Capacity Scheduler UI

Page 5

• Basic UI for adding and configuring queues• Set scheduler properties and see capacity allocations

More Databases

Page 6

• Ambari to support Postgres, MySQL or Oracle• Configure Hive and Oozie to use MySQL or Oracle

HBase

Page 7

• New Heatmaps specific to HBase Region Servers• Ability to designate Multiple Hbase Masters

Other Goodies

Page 8

• Add slaves components to hosts

• Stop/Start All Services

• Re-assign Master Components

• Host status filtering

Job Diagnostics

Page 9

• Enhanced swimlane visualizations• See job DAG with task overlay• See task scatter plot across jobs

10

Beyond 1.3.0

• Rack awareness• Log Aggregation• HDFS Rebalance• HBase Compaction• Manage Full Stack HA• Finer-grain Ambari user roles• Customizable Metric Graphs and Heatmaps• Capacity Scheduler Usage Reporting• HDFS Mirroring• Ambari Server HA

Stack Definitions

11

• Design Goals– Ambari should be able to support choice of Hadoop stacks– Ambari should enable adding new components to a stack

• Define which Services are available (services)• Define where to get the packages (repos)

S S S S

S S S S

S

Stack B

repos

services

Stack A

repos

services

Stack Inheritance

12

• Provide ability to “extend a stack”• Define clear wrapper around services to have consistent mgmt interface and

configuration controls

Stack CextendsStack A

repos

services S S S SSS +

APIs APIs and More APIs

13

• Consistent front-end REST API• Service Provider plugin architecture

“ZERO TOUCH” INSTALLS

“LIGHTS OUT” WORKFLOWS

“BRING MY OWN” SCENARIOS

Blueprint ClusterAmbari

Alarm Decom Inform

Custom Provider

Cluster Blueprints

14

• Perform “Headless Install”• Export blueprint from cluster• Boot wizard with blueprint• Save wizard as blueprint

BLUEPRINT<stack><host>

<service><component><config>

Ambari Server

MANIFEST<host><meta>

<configs><configs><config>

Blueprint

Thank Youhttp://incubator.apache.org/ambari/