ambari meetup: 2nd april 2013: teradata viewpoint hadoop integration with ambari

Steve RatayChief Architect, Teradata ViewpointApril 2, 2013

TERADATA VIEWPOINTHADOOP INTEGRATION WITH AMBARI

2 Copyright Teradata 2012

• Combination of Teradata Database, Aster, and Hadoop in a single appliance• Connectors between the different systems• Monitor the entire appliance with Teradata Viewpoint

Teradata Unified Data Architecture

DISCOVERY PLATFORM

CAPTURE | STORE | REFINE

INTEGRATED DATA WAREHOUSE

Aster Connector for Hadoop Teradata Connector for Hadoop

Aster Teradata Connector

SQL-H SQL-H

3 Copyright Teradata 2012

Viewpoint Architecture

TD 1

DataCollectionService

PostgresDatabase

ViewpointPortal

Viewpoint Managed Server

ActiveMQ

AlertService

TD n

Aster 1

Aster n

HDP 1

HDP nEmail SNMP Portlet

4 Copyright Teradata 2012

• Different technologies and formats:> Ganglia> Nagios> JMX> Screen scraping HTML

• Connection to multiple nodes:> Connectivity to all nodes> Access to several ports> Location of each service, especially when using high availability

Monitoring Hadoop without Ambari

5 Copyright Teradata 2012

• RESTful APIs• JSON response format• Single server and port

Ambari to the Rescue!

6 Copyright Teradata 2012

• Viewpoint uses Java and Spring• Used Spring’s RestTemplate class:

> Makes HTTP call to specified URL> Binds JSON results to Java model objects

• Data is collected every minute and stored in Viewpoint’s Postgres database> Use of Rewind to view previous state of system> Easy generation of trending charts and graphs

Accessing Ambari APIs

7 Copyright Teradata 2012

System Health

8 Copyright Teradata 2012

Metric Analysis

9 Copyright Teradata 2012

Node Monitor

10 Copyright Teradata 2012

Hadoop Services

11 Copyright Teradata 2012

Hadoop Alerts

12 Copyright Teradata 2012

• Viewpoint delivered comprehensive Hadoop monitoring solution in a couple of months• Java and web developers focused on collecting data and

building UI• Didn’t waste time dealing with intricacies of Hadoop and

Hadoop monitoring

Benefits of Using Ambari