services on dc/os (v2.0) running database - percona · 2018-04-27 · open source technologies with...

Gabriel Ciciliani - April 24th, 2018

Running database services on DC/OS (v2.0)

AGENDA

© 2017 Pythian. Confidential 2

- A fairly brief introduction to DC/OS

- Stateful services

- What we did

- Q&A


A fairly brief introduction to DC/OS

DC/OS is basically a distributed operating system based on Apache Mesos



SchedulerMesos master process

Mesos agent Executor “Task”


The scheduler will not only take care of launching service’s tasks, but also restarting, migrating and terminating them



● Custom Scheduler and/or executor (framework)

● Service-specific scheduling needs

● Custom executors will launch app-specific processes

● Netflix’s Titus, Apple’s J.A.R.V.I.S., Uber’s cassandra framework

● Default Scheduler and executors




SchedulerMesos master process

Mesos agent Executor “Task”

Mesos master process

Mesos agent

Docker executor


● Apps could be deployed from:

● Public package repository (Mesosphere Universe)

● Private package repository (Local Universe)

● Public docker registry (ie. Docker Hub)

● Private registry


13© The Pythian Group Inc., 2018

13

Some of this reminds me to Kubernetes..


DC/OS and Kubernetes



DC/OS K8s

Package repository ✓ ✓(Lower integration)

VIPs and load balancing ✓ ✓

Persistent internal and external volumes ✓ ✓

Web GUI ✓ ✓

Container agnostic ✓ ✓

Pods ✓ ✓


DC/OS and KubernetesDC/OS K8s

Secrets store ✓(Enterprise only)

✓

Jobs ✓ ✓

Autoscaling ✓(Lower integration)

✓


Ok, but how’s DC/OS different from Kubernetes

-Kubernetes is also container agnostic, although most people uses docker-Kubernetes also have node agents (kubelet)-Not using rexray apparently (for the GCE deployment)

DC/OS offers:- Secret store passing to containers- Pods- cron containers (jobs)- load balancing- auto-scaling (both containers and underlying infra)- RBAC for deployments/UI- private container image repo- built in REX-ray to transfer container data across slaves.

-Is there a way to create an auto-scaling policy on dc/os?Yes, and zero downtime upgrade policies

kube has this `- built in REX-ray to transfer container data across slaves.`not sure if it's rexray though but I tested a MySQL service using external volumes and it was transparently migrated upon node failure

Bill Fraser [12:36 PM]k8s has secrets and pods and load balancing and scaling and RBACalso DNS

John Laham [12:37 PM]DNS < == > Service Discovery

Bill Fraser [12:38 PM]there is kube-dns for container resolution within cluster, and there are mechanisms for integrating with external DNS (like route53) as well (edited)


Stateful services


● Storage needs to be persistent

● Data needs to be available on any slave node

● Automatic scaling (up or down)

Stateful services - challenges


● Non-persistent storage (local and ephemeral)

● Persistent storage (local)

● External persistent storage (non-local)

● Non-persistent storage (local and ephemeral)

● Persistent storage (local)

● External persistent storage (non-local)

Stateful services - storage options in DC/OS


● Resource requirements cannot be changed (cpu, memory, storage)

● External persistent volumes are not removed when a service is scaled down or removed. Also, Local persistent volumes may need to be wiped out before reusing them

● Absolute or relative mount points with “/” require a workaround

Stateful services - DC/OS limitations


What we did


What we did

Requirements:

● Highly available MySQL services on DC/OS

● InnoDB only

● Ability to scale up

● Ability to scale out too


What we did

Three DC/OS services:

● PXC, running three tasks/containers/mysqld instances

● Marathon load balancer (aka HAProxy)

● etcd for PXC metadata *

* based on Vadim Tkachenko PXC docker image in DockerHub


● No Universe app available to deploy PXC on DC/OS nativelyWe created a customized private image in DockerHub

● Storage needs to be persistentExternal persistent volumes

● Data needs to be replicated across slave nodes when scalingPXC SST

What we did


● Data needs to follow you container REXRay + EBS

● Resource requirements cannot be changed (cpu, memory, storage)Spawn a new service (cluster) and manipulate etcd metadata to replicate from the existing cluster

What we did


Conclusions


● DC/OS is an innovative solution to distribute apps and services across your infrastructure (cloud or on-premises)

● DC/OS is complex under the hood, and it integrates several different open source technologies with Apache Mesos.

● It’s feasible to run database services on DC/OS but a few things need to be considered first, specially with RDBMS

● PXC integrates very well with this model, as it handles replication, cluster membership and recovery.

Conclusions


THANK YOU!Gabriel CicilianiInternal Principal [email protected]@gabocic

mailto:[email protected]

services on dc/os (v2.0) running database - percona · 2018-04-27 · open source technologies with...

Documents