hortonworks technical workshop - build a yarn ready application with apache slider feb. 26 -...

© Hortonworks Inc. 2015

Build a YARN Ready Application with Apache Slider February 2015

Agenda

Introduction to Slider Steve Loughran

Authoring Application Packages Sumit Mohanty

Secure HBase deployment Jon Maron


Introduction to Slider Steve Loughran


log

Background: Modern applications

front end

web

phones

devices

feeds log

stream processing

front end

front end

log

database

analytics


Fun operational problems

Placement: where to run?

Installation

Configuration & Binding

Client configuration

Lifecycle

Failure handing and recovery

Logging

Upgrading

Metrics & Monitoring

Start/Stop

Reconfigure

Scale up/down

Rolling-restart

Decommission/Recommission


Getting your code to work across a set of fairly reliable machines


Apache Slider Deploying and managing applications on Apache Hadoop YARN

http://slider.incubator.apache.org/


HDFS

YARN Node Manager

HDFS

YARN Node Manager

HDFS

YARN Resource Manager “The RM”

HDFS

YARN Node Manager

•  Servers run YARN Node Managers (NM) •  NM's heartbeat to Resource Manager (RM) •  RM schedules work over cluster •  RM allocates containers to apps •  NMs start containers •  NMs report container health

YARN schedules work


Client creates Slider App Master

HDFS

YARN Node Manager

HDFS

YARN Node Manager

HDFS

YARN Resource Manager “The RM”

HDFS

YARN Node Manager

Client Slider Application Master


Slider App Master runs the application

HDFS

YARN Node Manager

HDFS

YARN Node Manager

HDFS

YARN Resource Manager

HDFS

YARN Node Manager

Slider Application Master

HBase Master

HBase Region Server

HBase Region Server

© Hortonworks Inc.

An application consists of

“Application Package”

JSON configuration files: YARN resources + config

Data persisted in HDFS

Placement history in HDFS

© Hortonworks Inc

resources.json { "schema": "http://example.org/specification/v2.0.0", "global": { "yarn.memory": "512" }, "components": { "HBASE_MASTER": { "yarn.role.priority": "1", "yarn.component.instances": "1", "yarn.vcores": "1", }, "HBASE_REGIONSERVER": { "yarn.role.priority": "2", "yarn.component.instances": "1" } } }

configure: app_config.json

{ "application.def": "/slider/hbase_v096.zip", "site.global.app_log_dir": "${AGENT_LOG_ROOT}/app/log", "site.global.app_pid_dir": "${AGENT_WORK_ROOT}/app/run", "site.global.hbase_master_heapsize": "1024m", "site.global.ganglia_server_host": "${NN_HOST}", "site.global.ganglia_server_port": "8667", "site.global.ganglia_server_id": "Application1", "site.hbase-‐site.hbase.tmp.dir": "${AGENT_WORK_ROOT}/work/app/tmp", "site.hbase-‐site.hbase.master.info.port": "${HBASE_MASTER.ALLOCATED_PORT}", "site.hbase-‐site.hbase.regionserver.port": "0", "site.hbase-‐site.hbase.zookeeper.quorum": "${ZK_HOST}", "site.core-‐site.fs.defaultFS": "${NN_URI}", }

Configurations needed by Slider

Named variables

Site variables for application

Named variables for cluster details

Allocate and advertise

Variables for the application scripts

© Hortonworks Inc

create, start, stop, destroy

$ slider create hbase1 -‐-‐resources resources.json -‐-‐template config.json $ slider list $ slider status hbase1 $ slider stop hbase1 $ slider start hbase1 $ slider destroy hbase1


Dynamic application resize

slider flex hbase1 -‐-‐component HBASE_REGIONSERVER 2


Failures

HDFS

YARN Node Manager

HDFS

YARN Node Manager

HDFS


HDFS

YARN Node Manager

Slider App Master

HBase Master

HBase Region Server

HBase Region Server


YARN notifies AM

HDFS

YARN Node Manager

HDFS

YARN Node Manager

HDFS


HDFS

YARN Node Manager

Slider App Master

HBase Master

HBase Region Server


AM requests replacement

HDFS

YARN Node Manager

HDFS

YARN Node Manager

HDFS


HDFS

YARN Node Manager

Slider App Master

HBase Master

HBase Region Server

HBase Region Server


Slider View for Apache Ambari

http://ambari.apache.org/

© Hortonworks Inc.

Project Status

• In ASF incubation • 0.70-incubating release this month • Growing set of application packages • Working in simplifying packaging • HDP 2.2 added long-lived service support for Slider and other services


ü  Labelled nodes & queues

ü  Log aggregation

ü Container retention over Application Master restart

ü  Service registration & discovery

ü  Kerberos token renewal

ü Windowed Application failure tracking

HDP2.2 features for Slider

—essential for all long-lived YARN services


http://slider.incubator.apache.org/

© Hortonworks Inc 2015

Meetup March 4

http://www.meetup.com/HSquared-Hadoop-Hortonworks-User-Group/events/220422211/


Authoring Applications

Sumit Mohanty smohanty at hortonworks.com Feb 25, 2015

© Hortonworks Inc.

Agenda

• A Simple Program • Simple Program that Advertises • Bring your own binaries • A Minimal Application Package

© Hortonworks Inc.

A Simple Program

• All dependencies installed • Default configuration • Well-known endpoints • Long running

© Hortonworks Inc.

A Simple Program : Recap

• create – slider create memcachedmin -‐-‐metainfo metainfo.json -‐-‐resources resources.json

• metainfo: { "schemaVersion": "2.1", "application": { "name": "MEMCACHED", "components": [ { "name": "MEMCACHED", "commands": [ { "exec": "/usr/jdk64/jdk1.7.0_67/bin/java …", "name": "START", "type": "SHELL", } ...

© Hortonworks Inc.

A Simple Program : Recap

• resources: { "schema" : "http://example.org/specification/v2.0.0", "metadata" : { }, "global" : { }, "components": { "MEMCACHED": { "yarn.role.priority": "1", "yarn.component.instances": "1", "yarn.memory": "256" } } }

© Hortonworks Inc.

Simple Program that Advertises

• All dependencies installed • Portable – advertises the endpoints • Long running

© Hortonworks Inc.

Simple Program that Advertises : Recap

• command – slider create memcachedexp -‐-‐metainfo metainfo.json -‐-‐resources resources.json -‐-‐template appConfig.json

• metainfo: "application": { "name": "MEMCACHED", "exportGroups": [ { "name": "Servers", "exports": [ { "name": "host_port", "value": "${MEMCACHED_HOST}:${site.global.port}” ..., "components": [ { "compExports": "Servers-‐host_port", "exec": "... -‐-‐port={$conf:@//site/global/port}",

© Hortonworks Inc.


• appConfig

{ "schema": "http://example.org/specification/v2.0.0", "metadata": { }, "global": { "site.global.port": "${MEMCACHED.ALLOCATED_PORT}{PER_CONTAINER}” } }

© Hortonworks Inc.


• from registry

http://RM-‐HOST:8088/proxy/app_id/ws/v1/slider/publisher/exports/servers { "description": "Servers", "updatedTime": "Thu Feb 26 06:11:49 UTC 2015", "entries": { "host_port": [ { "value": "c6403.ambari.apache.org:43292", "containerId": "container_1424910595794_0008_01_000002", "tag": "1", "level": "component", "updatedTime": "Thu Feb 26 06:11:49 UTC 2015" ...

© Hortonworks Inc.

Bring your own Binaries

• Dependencies deployed by the app instance • Portable – advertises the endpoints • Long running

© Hortonworks Inc.

Bring your own Binaries: Recap

• command – slider create memcachedjar –appdef /usr/work/memcached-‐jar -‐-‐resources resources.json -‐-‐template appConfig.json

• metainfo: "application": { "name": "MEMCACHED", "exportGroups": [...], "components": [ { "exec": "java -‐classpath {$conf:@//site/global/app_root}/*:..." ..., "packages": [ { "type": "folder", "name": "files/jmemcached-‐1.0.0" } ] ...

© Hortonworks Inc.

Bring your own Binaries: Recap

• folder contents: –  ./package –  ./package/files –  ./package/files/jmemcached-‐1.0.0 –  ./package/files/jmemcached-‐1.0.0/jmemcached-‐cli-‐1.0.0.jar –  ./package/files/jmemcached-‐1.0.0/jmemcached-‐core-‐1.0.0.jar –  ./metainfo.json

© Hortonworks Inc.

Minimal Application Package

• Why might I need a package? – Easier to share – Versioned – Default configs – Easier to upgrade

– admins may install blessed versions

– May package a client –  slider client install --package … --client-config …

© Hortonworks Inc.

Minimal Application Package: Recap

• Package content Archive: jmemcached-‐1.0.0.zip Length Date Time Name -‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐ 472 02-‐26-‐2015 02:49 appConfig-‐default.json 2029 11-‐23-‐2014 14:11 metainfo.xml 0 08-‐27-‐2014 20:54 package/ 0 01-‐09-‐2015 01:34 package/files/ 118272 01-‐09-‐2015 01:33 package/files/jmemcached-‐1.0.0.tar 0 11-‐23-‐2014 14:11 package/scripts/ 1659 11-‐23-‐2014 14:11 package/scripts/memcached.py 1346 11-‐23-‐2014 14:11 package/scripts/params.py 1823 11-‐23-‐2014 14:11 README.txt 280 11-‐23-‐2014 14:11 resources-‐default.json 1100 02-‐26-‐2015 02:59 metainfo.json -‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐-‐-‐ 126981 11 files


Secure Hbase Deployment

Jonathan Maron jmaron at hortonworks.com 2015

© Hortonworks Inc.

Agenda

• HBase Application Package • Application Deployment • Secure Deployment • Application Management • Demo • Debugging/Diagnosis • Q&A

© Hortonworks Inc.

HBase Application Package - Structure

appConfig-default.json appConfig-secured-default.json metainfo.xml package/files/hbase-0.98.4.2.2.1.0-2085-hadoop2.tar.gz jmx_metrics.json package/templates/... package/scripts/. . . resources-default.json ganglia_metrics.json get-hbase-site.sh get-hbase-site.py

© Hortonworks Inc.

HBase Application Package - metainfo

<metainfo> <schemaVersion>2.0</schemaVersion> <application> <name>HBASE</name> <version>0.98.4.2.2.1.0-2085-hadoop2</version> <type>YARN-APP</type> <minHadoopVersion>2.1.0</minHadoopVersion> <exportedConfigs>hbase-site</exportedConfigs> <exportGroups> <exportGroup> <name>QuickLinks</name> <exports> <export> <name>org.apache.slider.jmx</name> <value>http://${HBASE_MASTER_HOST}:${site.hbase-site.hbase.master.info.port}/jmx</value> </export> . . . </exports> </exportGroup> </exportGroups> <components> <component> <name>HBASE_MASTER</name> <category>MASTER</category> <minInstanceCount>1</minInstanceCount> <appExports>QuickLinks-org.apache.slider.jmx</appExports> <componentExports> <componentExport> <name>org.apache.slider.jmx</name> <value>${THIS_HOST}:${site.hbase-site.hbase.master.info.port}/jmx</value> </componentExport> </componentExports> <commandScript> <script>scripts/hbase_master.py</script> <scriptType>PYTHON</scriptType> <timeout>600</timeout> </commandScript> </component> . . . </components> <configFiles> <configFile> <type>xml</type> <fileName>hbase-site.xml</fileName> <dictionaryName>hbase-site</dictionaryName> </configFile> . . . </configFiles> </application> </metainfo>

© Hortonworks Inc.

HBase Application Package - Configuration

{ "schema": "http://example.org/specification/v2.0.0", "metadata": { }, "global": { "application.def": ".slider/package/HBASE/slider-hbase-app-package-0.98.4.2.2.1.0-2085-hadoop2.zip", "create.default.zookeeper.node": "true", "java_home": "/usr/jdk64/jdk1.7.0_67", "system_configs": "core-site,hdfs-site", "site.global.app_user": "${USER}", "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/hbase-0.98.4.2.2.1.0-2085-hadoop2", . . . "site.hbase-site.hbase.security.authentication": "kerberos", "site.hbase-site.hbase.security.authorization": "true", "site.hbase-site.hbase.security.access.early_out": "true", "site.hbase-site.hbase.coprocessor.master.classes": ”. . .", "site.hbase-site.hbase.coprocessor.region.classes": ”. . .", "site.hbase-site.hbase.regionserver.kerberos.principal": "${USER}/[email protected]", "site.hbase-site.hbase.regionserver.keytab.file": "/etc/security/keytabs/hbase.keytab", "site.hbase-site.hbase.master.kerberos.principal": "${USER}/[email protected]", "site.hbase-site.hbase.master.keytab.file": "/etc/security/keytabs/hbase.keytab", "site.hbase-site.hbase.rest.kerberos.principal": "${USER}/[email protected]", "site.hbase-site.hbase.rest.keytab.file": "/etc/security/keytabs/hbase.keytab", "site.hbase-site.hbase.thrift.kerberos.principal": "${USER}/[email protected]", "site.hbase-site.hbase.thrift.keytab.file": "/etc/security/keytabs/hbase.keytab", "site.hbase-site.hbase.master.keystore.password": "myPasswordForTheKeystore", "site.hbase-site.hbase.regionserver.truststore.password": "myPasswordForTheTruststore" }, "components": { "slider-appmaster": { "jvm.heapsize": "1024M", "slider.hdfs.keytab.dir": ".slider/keytabs/hbase", "slider.am.login.keytab.name": "hbase.headless.keytab", "slider.keytab.principal.name": "hbase" } } }

© Hortonworks Inc.

Application Deployment - CLI

• Package Installation – slider package –install –package package.zip –name HBASE

• Application creation – slider create myApp –template appConfig.json –resources

resources.json

© Hortonworks Inc.

Secure Deployment •  Login as application owner •  Keytab required for application owner

–  E.g. hbase.headless.keytab –  Referenced by configuration properties:

–  Manually distributed

–  Configuration: -  slider.am.keytab.local.path -  slider.keytab.principal.name

–  Application keytab properties reference the local path on NM hosts –  Distributed by Slider

–  Configuration: -  slider.hdfs.keytab.dir -  slider.am.login.keytab.name -  slider.keytab.principal.name

–  Application keytab properties reference the localization path on NM hosts -  E.g. “${AGENT_WORK_ROOT}/keytabs/hbase.service.keytab”

•  More information at http://slider.incubator.apache.org/docs/security.html –  Keystore/Truststore generation

© Hortonworks Inc.

Application Management

• Flex – Increase or decrease number of application component instances

• Stop – Halt an application instance

• Start – Start an instance of a defined application

• Destroy – Remove an application definition

© Hortonworks Inc.

Demo

• Application Package Contents • Cluster description • Review security setup

– Login – Keytab availability

• Install package • Deploy Application • Flex • Stop • Destroy

© Hortonworks Inc.

Debugging/Diagnosis

• Log aggregation – Provides a mechanism for retrieving all log files associated with

application instance – yarn logs –applicationId <application ID>

• Other mechanisms to leverage are provided on Slider incubator web site – http://slider.incubator.apache.org/developing/debugging.html

© Hortonworks Inc.

Additional Information

• Meetup – Long Running Services on YARN Using Slider

– Wednesday, March 4th, 6:30 to 8:30 PM @ HWX HQ

– http://www.meetup.com/HSquared-Hadoop-Hortonworks-User-Group/events/220422211/

• http://slider.incubator.apache.org/index.html


Questions?

hortonworks.com


Meetup March 4

http://www.meetup.com/HSquared-Hadoop-Hortonworks-User-Group/events/220422211/

hortonworks technical workshop - build a yarn ready application with apache slider feb. 26 -...

Technology