emc world 2016 - deep dive with mesos and persistent storage for applications

DEEP DIVE WITH MESOS & PERSISTENT STORAGE FOR APPLICATIONSDAVID VONTHENEN, EMC {code}

• David vonThenen, Developer Advocate - EMC {code}– Twitter: @dvonthenen– Blog: http://dvonthenen.com– Github: https://github.com/dvonthenen

PRESENTER

• Quick Review of Mesos• Frameworks and Two Layer Scheduling• Production Applications• Demo• Q&A

AGENDA

QUICK REVIEW OF MESOS

• From a resource perspective, it’s a cluster manager:– Pools resources to be centrally managed as a single unit

• From an application perspective, it’s a scheduler:– Dispatches workloads to consume pooled resources

• Often described as a Data Center Operating System (DCOS)– Static versus Dynamic Partition of Resource– Efficient utilization of CPU, Memory, Disk

• Proven resource scheduler for production workloads

QUICK REVIEW OF MESOS

SCHEDULERS ARE DIFFICULT TO BUILD

• Master– Manages Mesos Agents– Enables fine-grained sharing of resources (CPU, RAM, …)

across frameworks in the form of resource offers.• Zookeeper

– Enables highly reliable distributed coordination (cluster state)

• Agent (aka Slave)– The entity that will be running the task

MESOS COMPONENTS

MESOS ARCHITECTURE

• Generic– Anything that can be run on a standard Linux shell– Provides a mechanism for deploying applications packages– Supports Docker workloads

• Framework– A plug-in to Mesos that allows you to implement a sub-

scheduler– specialize a workload by providing custom placement and

resource allocation

APPLICATION SUPPORT

GENERIC APPLICATIONS

FRAMEWORKS ANDTWO LAYER SCHEDULING

• On the surface…• Allows specialization of

workload by providing custom placement and resource allocation• Provides monitoring of

these workloads to protect and react when things fail

MESOS FRAMEWORK

• Scheduler– a scheduler that registers with the master to be offered

resources• Executor

– an executor process that is launched on slave nodes to run the framework’s tasks

FRAMEWORK COMPONENTS

FRAMEWORK ARCHITECTURE

• Odds are you have been using at least 1 framework without realizing it…

AS IT TURNS OUT…

• Marathon is a framework– A framework for performing container

orchestration for long running tasks• Chronos is a framework

– A framework for performing batch or scheduled tasks

• Zooming into the “Marathon” box…• Marathon is a Scheduler• Launching a Marathon task, the “generic”

applications are created inside a “generic” (or default) executor• Dogfooding!

MARATHON DISSECTED

MESOS FRAMEWORKS

• 2 layer scheduling or sub-scheduling of resources• Manages the lifecycle of your

workload through application specific monitoring• What does this mean

specifically?

DIGGING DEEPER

BUILDING HADOOP CLUSTERS

MANAGING HADOOP CLUSTERS

APPLICATIONS USING HADOOP

THIS SEEMS SILLY

• Allows you to subsume the default scheduler such that:• You can install and configure your application• Resources can be allocated in a way that optimizes

your applications• Allows operation real-time automated improvements

for your applications– Scaling up for load balancing peak times– Scaling back to free up resources

2 LAYER SCHEDULING

• The details of your applications are hidden from the end user!– Deployment– Operations– Growth– Disaster recovery– Performance

• Just a consumer of the application as a service!

THE NON-TECHNICAL VERSION

• CouchBase Framework (doesn’t exist, let’s imagine it does)– CouchBase is a NoSQL DB

• Monitor the CouchBase instances, when nearing full– Dynamically add an node using “server-add”– Then call “rebalance”

• Reactive and Proactive but fully automated• I think we had all seen this movie…

FOR EXAMPLE

SKYNET

PRODUCTION APPLICATIONS

• Containers are ephemeral therefore stateless• Quickly found the need for state

– Configuration– Data– Logging / Auditing

• Containerizers like Mesos and Docker introduced exporting to local storage• Failure of a node? hardware? A better way?

EVOLUTION OF STATEFUL

• Persistent– Containers can come and go,

but continue where they left off– Application Data, Logs, Audit

Trail, etc• External

– Storage lives outside the Mesos Agent node

– High Availability– Tolerates node failure

PERSISTENT EXTERNAL STORAGE

• REX-Ray– Provides a vendor agnostic storage orchestration engine– AWS, GCE, ScaleIO, XtremIO, many many more– Github: https://github.com/emccode/rexray

• mesos-module-dvdi– Provides a hook to Mesos agent nodes to provision, attach,

and detach external storage– Github: https://github.com/emccode/mesos-module-dvdi

HOW WE ACHIEVE THIS?

• We can run “real world” applications• Persistence – DBs• Highly Available, Fault Tolerant• Audit trail – Transaction Logs• Not worry about the “2am call”

STATELESS TO STATEFUL

LET’S FACE IT…

• Trouble is… this still represents the “old way” of thinking• PREVENT THE CRASH OR FAILURE!• Problem is you can’t• What is the alternative?

STATEFUL 2.0?

• Mesos frameworks can proactively monitor for thing going sideways• Leveraging external persistent storage and

frameworks fail fast and recover quicker!• Skynet!

FRAMEWORKS + STATEFUL 2.0

• Running on Amazon EC2– Mesos– Marathon– Mesos DNS

• Production HA Cluster– 3 x Master + Zookeeper Nodes (t2.micros)– 5 x Slave Nodes with Docker Support (t2.medium)

DEMO CONFIGURATION

• Elastic Search Framework– Create and mount external volume– Pour some data into Elastic Search– Github: https://github.com/mesos/elasticsearch

• Elastic Search Instances backed by Persistent External Storage– Using REX-Ray– Using mesos-module-dvdi

DEMO: FRAMEWORK + EXTERNAL STORAGE

• Deploy Elastic Search Framework using REST API– Observe Scheduler creation– Using browser, log into the Elastic Search Framework UI– Observe Executor nodes get created and attaching to EBS

volumes• Simulate Failure

– Kill EC2 Instance– Watch executor recreation and reattach to EBS volumes

DEMO: FRAMEWORK + EXTERNAL STORAGE

DEMO: AUTO DEPLOY ELASTIC SEARCH

Data Persistence in the New Container WorldWednesday 3PM

Joshua BernsteinVP of Technology for ETD

Tobi Knaup CEO & Co-Founder of Mesosphere

Guru Session

@EMCcode@dvonthenen

emccode.comcommunity.emccode.com

Come visit us at Booth #1044 or in the vLab

Questions?

emc world 2016 - deep dive with mesos and persistent storage for applications

Technology

introduction to apache mesos

resource management - mesos and yarn · mesos yarn 4/46....

mesos study report 03v1.2

cs 744: mesos

nvidia gpu support on mesos: bridging mesos … gpu support...

mesos tech report

networking & security for mesos -...

spark on mesos-a deep dive-(dean wampler and tim chen,...

growing the mesos ecosystem

mesos introduction

emc world 2016 - code.14 deep dive with mesos and persistent...

a travel through mesos

mesos and yarn

a whirlwind tour of apache mesos - yow! conferences ·...

apache flink meets apache mesos and dc/os @ mesos meetup...

monedes i bitllets d’euro...1 trimestre = 3 mesos 1 segon...

apache mesos

introduction to mesos bay

mastering mesos - sample chapter

mesos vs kubernetes comparison