CLOUMON

The powerful Hadoop open-source software stack requires careful integration,

calibration, and monitoring, which is why Gruter has developed its own in-house cloud

management solution, Cloumon. With a user-friendly interface and management

console, Cloumon enables system administrators to optimize the Hadoop ecosystem

and take control of the cloud across the entire data lifecycle.

ENJOY HADOOP

Hadoop, Hive and Hadoop Ecosystem

Monitoring and Management System

CLOUMON CH (Core Hadoop)

Hadoop (HDFS, MapReduce) and Hive

CLOUMON PA (Power Analytics)

Advanced Analysis Rule Manager, Streaming Data Processing

Manager, and Interactive Analysis Query Manager

CLOUMON EPs (Extension Packs)

Oozie, HBase, ZooKeeper, and Flume

R

R

R

R

CLOUMON KEY FEATURES

DATA STORAGE

HDFS File

Manager

Hive Table

Manager

HBase Data

Manager

MapReduce Job

Manager

Job Workflow Manager

Hive Query

Manager

MANAGEMENT

ZooKeeper Node

Manager

Real-time Analysis Manager

Flume Data Flow Manager

ZooKeeper

Esper Query

Manager Esper

DATA COLLECTION

HDFS File

Hive Table

HBase

Collector Data Agent

•

DATA ANALYSIS

MapReduce

Hive Query

Pig Query

Flume Data Flow

Manager

STREAMING DATA PROCESSING

Collect and graph metrics from target daemon servers including NameNode, DataNode, JobTracker and TaskTracker

Create alerts by setting thresholds on target metrics and servers

Construct highly visible log data management views

Monitor system resource usage

MONITORING

•

•

•

•

Manage integrated configurations for server groups

Conveniently access optimized Hive and Oozie functionality

Remotely control servers to perform stop-start maintenance routines

Run various Hadoop distributions including Apache Hadoop 1.0.x, Apache Hadoop 2.0.x and CDH Hadoop 4.2.x

Control multiple Hadoop clusters

CLUSTER MANAGEMENT

•

•

•

•

•

Manage entire data lifecycle from data collection to storage, batch analysis and real-time analysis

Browse files with Hadoop File Browser; create and execute queries with Hive Query Workbench

Design and schedule workflows with Oozie Workflow Designer; manage ZNode with ZooKeeper Manager

DATA MANAGEMENT

•

•

1 Enjoy Connecting GRUTER

•

* Key Cloumon management zones in orange

R

R

Cloumon CH provides a streamlined environment for the operation of Hadoop and Hive, the core components of

advanced Big Data platforms. Through enhanced component visibility and task management features, Cloumon CH

gives unprecedented access to and control over Big Data systems.

2 Enjoy Connecting GRUTER

CLOUMON CH (Core Hadoop)

HDFS Manager

HDFS Cluster Manager

HDFS daemon status monitoring

Remote server control

Server group configuration

Comprehensive metric monitoring

Integrated log view creation and

management

Multiple cluster commissioning and

management

User-configured server threshold

alerts

· Monitor status and failures on NameNode, JournalNode, SecondaryNameNode,

DataNode and DFSZKFailoverController

· Use simple pre-configured wizards to add new servers to running clusters

(coming release — Q2 2013)

· Start and stop servers remotely

KEY FEATURES

· Manage configurations in server groups

· Detect servers with asymmetric configurations automatically

· Apply configurations to all clusters or specific target servers

· Collect HDFS metrics at single minute intervals

· Track performance history and graph server metrics for thorough system analysis

· Set disk usage thresholds by server and partition

· Set alert thresholds on all HDFS metrics

· Set SMS alerts for critical metrics via Alert Plugin

Major HDFS distribution compatibility · Compatible with major distributions including Apache Hadoop 0.20.x, Apache

Hadoop 1.0.x, Apache Hadoop 2.0.x, CHD 4.1.x, CDH 4.2.x

HDFS commands

List sorting

· Execute commands including list, mkdir, delete, chown and chmod

· Sort lists by name, size, date and owner to improve search speed

And more: Directory tree views; file block information; file data views; file download/upload capabilities

HDFS File Browser

KEY FEATURES

· Create one-stop views of logs from across the distributed system

· Commission and manage multiple clusters as system scales out

R

MapReduce Manager

MapReduce daemon status monitoring

Remote server control

Server group configuration

Comprehensive metric monitoring and

configurable server threshold alerts

Integrated log view creation and

management

Multiple cluster commissioning and

management

· Monitor status and failures on JobTracker and TaskTracker

· Use simple pre-configured wizards to add new servers to running clusters

(coming release — Q2 2013)

· Start and stop servers remotely

· Manage configurations in server groups

· Detect servers with asymmetric configurations automatically

· Apply configurations to all clusters or specific target servers

· Set disk usage thresholds by server and partition

· Set alert thresholds on all HDFS metrics

· Set SMS alerts for critical metrics via Alert Plugin

MapReduce Cluster Manager

KEY FEATURES

MapReduce Job Manager

KEY FEATURES

Job management

Job status monitoring

· Manage current job information and track job history

· Filter job lists by status and period

· Monitor task status and job counter

· Track full execution history

Task profiling · Profile task execution progress and elapsed execution time

Task control · Abort processes through stop task functionality

Scheduler monitoring · Monitor fair scheduler mode queue status

· Manage queues

Hive and Oozie integration · Monitor Hive query executions

3 Enjoy Connecting GRUTER

· Create one-stop views of logs from across the distributed system

· Commission and manage multiple clusters as system scales out

Hive Query and Hive Configuration

KEY FEATURES

Hive connection management

HDFS

Apache Hadoop 0.20.x

Apache Hadoop 1.0.x

Apache Hadoop 2.0.x

CDH 4.1.x

CDH 4.2.x

MapReduce

Apache Hadoop 0.20.x

Apache Hadoop 1.0.x

CDH 4.x-mr1

Hive

Apache Hive 0.8.x

Apache Hive 0.9.x

Apache Hive 0.10.x

CDH 4.1.x Hive

CDH 4.2.x Hive

· Support multiple connections with built-in Hive delegator (Hive installation not

required)

Hive session management · Manage driver sessions and track query execution status

Table meta viewer and table viewer · Generate detailed table description views and data tables

Multiple query executor · Execute multiple simultaneous queries

User-defined jar and script

management · Upload/delete/apply UDF and Custom M/R

Progress viewer and query status

inquiry · Check query execution progress and track execution history

Query management · Generate saved query and Hive function description views

Table and query wizard · Use simple pre-configured wizards to create tables and queries

Configuration management · Edit and dynamically deploy Hive and Hadoop client configurations

· Access comprehensive storage usage, partitioning and bucket information.

OS

WebServer

DataBase

Java Virtual Machine

Linux, Windows

Tomcat 6.x

MySQL 5.x

JDK6

4 Enjoy Connecting GRUTER

Hive Manager

Versions Supported System Requirements

Web-based support

8x5

24x7

Phone support

2-24 hour initial response time

Service SLA

5 Enjoy Connecting GRUTER

CLOUMON PACKAGE CLOUMON PA (Power Analytics)

Cloumon PA is a high-performance Big Data system which brings together a powerful set of cutting-edge

technologies and tools to help you perform advanced analytics on Hadoop and Hive.

Smart query building processes and intuitive execution flows generate sophisticated outputs in just a few clicks

without the need for complex query syntax.

Stream Processing Rule Manager

Console for streaming data

processing

KEY FEATURES

Interactive Analytics (Impala, Tajo)

Impala

Tajo

KEY FEATURES

· Manage metadata such as table schemas for integration with Hive

· Use Impala query workbench

· Monitor status of Impala clusters

· Manage metadata such as table schemas

· Use Tajo query workbench

· Monitor status of Tajo clusters

· Manage entire lifecycle of streaming data processing by registering data type,

configuring parser, managing EPL queries for analysis, and storing and querying

results, among other functionalities

Data type management and

configuration · Define type name, column, record parser and result table

· Use built-in storage interfaces such as HBase and MySQL

· Extend interface to add and select user-defined storage Analysis result storage management

· Manage EPL queries

· Add and delete queries dynamically in a running environment

· Have results stored automatically in selected storage

Analysis query manager

· Visualize results using various charts and graphs according to data type Analysis output visualization

· Manage Hive/Tajo/Impala queries in an integrated fashion and choose optimal

execution platform

· Manage queries in concert with Advanced Analysis Rule Manager

R

Advanced Analysis Rule Manager

Hive Query Based Analysis Rule Management

Analysis target object management

KEY FEATURES

· Select analysis targets such as Hive table and existing queries, among others

· Manage aliases down to fields and rules via user-friendly UI

· Build complex queries with multiple “join”, “group by”, and “order by”

functions/clauses simply and quickly

· Employ variables for high re-usability and productivity

· Define materialized views as analysis targets without burden of generating

actual views

· Visualize usage of individual rules and their interrelationships through charting

and graphing tools

· Manage execution start time

· Track execution history

· Create fresh results at each execution or conveniently reuse previous results at point of execution

6 Enjoy Connecting GRUTER

KEY FEATURES

Hive query builder

Analysis target querying

Rule charting

Execution and Result Management

Scheduling

Query optimization

Dynamic variable binding at execution

Multiple storage options · Use Hive Table, HDFS Directory and HBase Table

Powerful viewer and APIs to access analysis outputs

· Bind actual values to variables dynamically at point of execution

· Create fresh results at each execution or conveniently reuse previous results at point of execution

Cloumon EPs provide additional monitoring and management capabilities for other key components of the Hadoop

ecosystem including Oozie, HBase, Flume and Zookeeper, granting comprehensive control of the entire data lifecycle

from data collection, storage and workflow design to task scheduling and distributed system role management.

7 Enjoy Connecting GRUTER

CLOUMON PACKAGE CLOUMON EPs (Extension Packs)

HBase Manager

HBase Cluster Management

Table data scanning

KEY FEATURES

Oozie Workflow Manager

Wysiwyg job designer

Library file management

Job execution management

· Upload jar files

· Manage mapper, reducer, and writable classes

· Manage job libraries (distributed cache)

KEY FEATURES

· Schedule job execution

· Track job execution history

· Monitor jobs via integrated Cloumon MapReduce Job Manager

· Collect information at single minute intervals

· Track history of metric changes over time and chart in time-series

· Fetch lists of tables and look up table schemas

· Manage table region lists and region lists on RegionServers

· Monitor detailed region metrics

· Create and drop table (Q3 2013)

· Perform Region compaction, split, and merge

· Execute and schedule jobs according to user-configured rules

HMaster and RegionServer alerts

Single server metric monitoring

Table and Region status monitoring

Manage Region (Q3 2013)

HBase Data Management

Column data fetching by row query

Long type transformation

Web-based HBase shell (Q3 2013)

· Automatically convert byte array long to numeric long for readability

R

8 Enjoy Connecting GRUTER

ZooKeeper Manager

ZooKeeper Cluster Management

Monitor server status and set alerts

KEY FEATURES

Flume Manager for Flume-OG (v0.9.4)

Data flow management

Powerful configuration tool

KEY FEATURES

· Inspect data flow between agent and collector

· Monitor workloads of each node using workload indicators

· Design data processing flows via powerful tool which allocates source, deco

and sink

· Easily set parameters with pre-configured forms and help tips

· Reuse and edit existing configurations

Physical/logical node status

monitoring

· Check overview of node status and drill down to analyze specific details

· List logical nodes on specific physical nodes

Map/unmap/decommission/purgeAll · Control the entire lifecycle of logical nodes with minimal clicks

· Use smart proxies to complete complex jobs in a single click

Multiple cluster management · Manage multiple clusters

· Collect metrics at single minute intervals

· Track metric change history

View detailed ZooKeeper server

metrics

Monitor ZooKeeper connections · Monitor and inspect all connections to ZooKeeper servers

Manage multiple clusters simultaneously

Easily manage zNodes by accessing detailed information and manipulating data through convenient file browser

interface

Manage ACLs for each zNode

Manage zNode watcher registration

ZooKeeper Node Management

· Create integrated views by combining data from Flume masters and ZooKeeper

Phone: +82-2-508-5911

Fax: +82-2-508-5912

E-mail: inquiries@gruter.com

Web: www.gruter.com

For demo videos, please visit: www.gruter.com/products/cloumon#video

GRUTER, INC.

5F Sehwa Office Building 889-70 Daechi-dong, Gangnam-gu, Seoul, South Korea 135-839

Gruter: Your Partner in the Big Data Revolution