pattern-based orchestration - iaas · pattern-based orchestration university of stuttgart cloud...

© 2015 IBM Corporation

Pattern-based Orchestration University of Stuttgart Cloud Course, November 2015

Thomas Spatzier ([email protected])

© 2015 IBM Corporation 2

Agenda!

•  Introduction

• Pattern Modeling Concepts

• OpenStack Heat

• Container Orchestration

• Hybrid Orchestration


What is Orchestration?!

Orchestration describes the automated arrangement, coordination, and management of complex computer systems, middleware, and services.

[…] orchestration is often discussed in the context of service-oriented architecture, virtualization, provisioning, Converged Infrastructure and dynamic datacenter topics. Orchestration […] defines the policies and service levels through automated workflows, provisioning, and change management. This creates an application-aligned infrastructure that can be scaled up or down based on the needs of each application. […]

From Wikipedia, the free encyclopedia. h6p://en.wikipedia.org/wiki/Orchestra=on_(compu=ng)


What is a pattern?!

•  A structural model of components of an application-aligned infrastructure, incl. relationships between those components (applications, middleware, virtualized infrastructure, …)

•  The definition of automation behavior for those components to control their lifecycle state (scripts, executables, …)

•  The definition of policies and SLAs to scale the infrastructure according to application needs

•  The definition of semantics for component lifecycle states, relationships and policies to derive orchestration flows from the structural model

Server1 Server2

Web Applica0on Database

Storage Volume

Private Network

Load Balancer

Public Network

2..10


Agenda!

•  Introduction






Contribu)ng Members

  provides the Interoperable Descrip)on and Solu)on Portability for:   Applica)ons, their component Services and Ar)facts, including Rela)onships of these services

  is an approved OASIS Standard since Nov. 25th 2013: hFp://docs.oasis-‐open.org/tosca/TOSCA/v1.0/os/TOSCA-‐v1.0-‐os.pdf

The TOSCA standard…

  PlaJorm and Infrastructure services

  Management and Opera)onal behavior of these services

------------

------------

Standards perspective on patterns – OASIS TOSCA!


Contents of a TOSCA service package!

Defini'on of building blocks for services

… along with the implementa'on ar'facts for manageability opera'ons

… and the defini'on of deployment ar'facts for components

… op'onal orchestra'on flows

Service Template

Prop

er0e

s Interfaces

Node Types

Requirements

Capabili0es Topology Template

Rela0onship Template

Node Template

type for

Cloud Service ARchive (CSAR)

-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐

-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐

Scripts

Installables

Images

Defini'on of Service Templates …

… including a Topology Template describing the structure of a service

Interfaces Prop

er0e

s Rela'onship Types Plans

Defini'on of possible links between components

type for


Thought process when defining patterns with TOSCA!

Deployment Artifact Node Type

R

C

Implementation Artifact

Met

a-M

odel

Rela0onship Type

…

“Orchestrator” Deployed Instances

SugarCRMDB [MySQLDatabase]

R

C

MySQL [MySQL]

R

C

SugarCRMApp [WebApplication]

R

R

Tem

plat

es

Proper'es: DB Name: “mysqldb” …

Proper'es: Admin User: myadmin …

Proper'es: Context root: /mycrm …

Proper'es: HTTP Port: 8080 …

Apache [ApacheWebServer]

R

C

start.sh

MySQL Database

R

C start.sh

RPMs: mysql

mysql-server

configure.sh

MySQLDBConnec0on Type

s

MySQLDBContainer Req. LinuxOS Req.

MySQL

R

C

Primary focus for users


Processing of patterns – declarative vs. imperative!

•  Declarative processing – “what is the desired state?” •  Processing logic is derived from the topology model • Requires clear specification of node and relationship semantics

•  Imperative processing – “how to reach a desired state?” •  Processing logic is implemented in orchestration workflows (“plans”) • Requires explicit modeling of those workflows • Requires a standardized API to make workflows portable across

implementations


Role of relationships in declarative processing!

NodeType “Database” C

NodeType “Application” R

Lifecycle operations (create, configure, start, …) are used by an orchestrator

to create and manage components

Connector concept Relationship Types declare

what endpoints they can connect, and which

processing logic they inject to establish a link at runtime

Base Relationship Types HostedOn, DependsOn and ConnectsTo define the base

semantics for processing topology models

Rela0onshipType “DBConnec0on”

NodeTemplate “MyDB” C NodeTemplate

“MyApp” R

is of type is of type is of type

•  Use defined lifecycle opera0ons to deploy and manage each component •  Use base rela0onship types to derive processing order

•  Process a host before a hosted component (HostedOn) •  Process a provider before a client (DependsOn, ConnectsTo)

•  If a component is related to another component, see if rela0onship injects addi0onal processing logic (e.g. pre-‐configure endpoint)

… a preFy mechanical process


How is a topology processed?!•  Use base rela0onship types to derive component processing order •  First process a host, then process hosted component •  First process a component that another component depends on, then process the dependent component

•  First process a component that another component connects to, then process the connec0ng component

•  For each component •  Deploy its Deployment Ar0facts •  Invoke lifecycle opera0ons in right sequence (create, configure, start …); there can be no-‐ops

•  If a rela0onship contributes logic, inject it into component opera0on invoca0ons

Web 1

Web 2

Web 3

Web 4

Web 5

DB 1

DB 2

DB 3

DB 4

rela'onship injects logic

DB 1

DB 2

DB 3

DB 4

Web 1

Web 2

Web 3

Web 4

create VM

perform base OS config

install and configure hFpd

install and configure php run0me

create VM

perform base OS config

install and configure MySQL

create and configure SugarCRM database

Web 5 install and configure SugarCRM app, configure database endpoint proper0es

run in parallel

wait for step “DB 4” to complete


Some Application

Com

pute

Sto

rage

arch = x86_64 cpus = 2

mem >= 4GB OS = Linux

size >= 10GB

Database

provider = MySQL version >= 5.5

Requirements & Capabilities!

Database Provider

Database

Com

pute

Sto

rage

Requirements can be fulfilled explicitly by other components in the model

Requirements can be fulfilled by the run0me

Requirements/Capabili'es are base for subs'tutability

Some Component

R

Provider A

C

Provider B

C

Provider C

C

OR OR

Some Application

Com

pute

Sto

rage

arch = x86_64 cpus = 2

mem >= 4GB OS = Linux

size >= 10GB

Database

provider = MySQL version >= 5.5


Model composition!

Node Template

Node Template

Service Template 1

Service Template 2

can subs0

tute

boundary defini0ons

Subsystems can be abstracted in some models.

Other models can define details of subsystems.

à separa0on of concern, re-‐use


Usage scenarios for model composition & substitution!

Load Balancer

VM

Web Server

VM

1..*

Web Server

VM

Single node web 'er Scalable web 'er

WebApp

WebTier

App DB

Tomcat MySQL

VM VM

Network Storage

Applica'on Layer

PlaSorm Layer

Infrastructure Layer

Custom workload

Mul0ple op0ons of

middleware deployments

Mul0ple op0ons of

infrastructure configura0ons

Varying deployment op'ons Layering of models


Agenda!

•  Introduction






OpenStack Heat!

•  A pattern-based orchestration engine on-top of OpenStack APIs

•  Consumes templates that declaratively describe a workload consisting of a set of OpenStack resources (e.g. servers, storage volumes, networks)

•  Instantiates and manages instances of those workloads (“stacks”) by orchestrating over the respective OpenStack APIs

Server

Network Storage Volume

Server

Network Storage Volume

Template Stack


Heat Orchestration Template (HOT) example!

heat_template_version: 2013-‐05-‐23 parameters: key_name: type: string description: Name of keypair to assign to servers. image: type: string description: Name of image to use for servers. resources: private_net: type: OS::Neutron::Net properties: # properties of private_net ... private_subnet: type: OS::Neutron::Subnet properties: # properties of private_subnet ... server1: type: OS::Nova::Server properties: # properties of server1 ... outputs: server1_private_ip: description: IP address of server1 in private network value: { get_attr: [ server1, networks, private, 0 ] }

configurable parameters for each stack deployment

the set of resources to deploy for each stack

observable outputs of each stack deployment

resource type to be created


Heat software orchestration!

•  New concept introduced in OpenStack Icehouse release, enhanced in Juno release •  Up to then, only base OpenStack infrastructure

resource support

•  SoftwareComponent or SoftwareConfig: •  Definition of re-usable orchestration metadata,

incl. reference to automation for a specific piece of software

•  SoftwareDeployment: •  Instantiation of a software component on a

specific server •  Similar to an association class •  Includes concrete input values for deployment

and produces output •  Can define dependencies on other deployments

Automation (Chef, Puppet,

script, …) Automation (Chef, Puppet,

script, …)

Server “server1”

So`ware Deployment “app_depl”

So`ware Component “app_comp”


So`ware Deployment “db_depl”

So`ware Component “db_comp”

“Nod

e Type

” “N

ode Template”


Heat software orchestration example!

heat_template_version: 2013-‐05-‐23 parameters: db_name: type: string description: Name of database to create. resources: db_comp: type: OS::Heat::SoftwareComponent properties: inputs: -‐ name: database_name default: my_db configs: -‐ actions: [ CREATE ] tool: script config: { get_file: install_db.sh } -‐ actions: [ UPDATE ] tool: script config: { get_file: reconfigure_db.sh } db_depl: type: OS::Heat::SoftwareDeployment properties: config: { get_resource: db_comp } server: { get_resource: server } input_values: database_name: { get_param: db_name } server: type: OS::Nova::Server # ... details ...


script, …)

Server “server”

So`ware Deployment “db_depl”

So`ware Component “db_comp”


script, …)


Heat Provider Templates!

heat_template_version: 2013-‐05-‐23 parameters: database_name: type: string description: Name of database to create. database_admin_user: type: string description: Username to set for database admin. resources: db_server: type: My::DB::Server properties: db_name: { get_param: database_name } db_admin_user: { get_param: database_admin_user } outputs: db_server_ip: description: IP address of database server. value: { get_attr: [ db_server, db_server_ip ] }

heat_template_version: 2013-‐05-‐23 parameters: db_name: type: string description: Name of database to create. db_admin_user: type: string description: Name of database admin user. resources: mysql_config: type: OS::Heat::SoftwareConfig properties: # software configuration properties mysql_deployment: type: OS::Heat::SoftwareDeployment properties: config: { get_resource: mysql_config } server: { get_resource: server } server: type: OS::Nova::Server properties: # server properties outputs: db_server_ip: description: IP address of database server. value: { get_attr: [ server, networks, private, 0 ] }

end user template

“provider template”

“implements”

# Heat environment file # mapping of resource type names to provider templates resource_registry: My::DB::Server: file:///my_provider_template.yaml


Heat resource model!

•  Resource types used in templates are implemented as Python classes that support a well-defined lifecycle model

•  Out-of-the-box resource type implementations for almost all OpenStack resources (OS::Nova::Server, OS::Cinder::Volume, OS::Neutron::Net, …)

•  Additional, custom resource type implementations can be installed as plug-ins

•  HOT templates (“provider templates”) can serve as implementation of resources types, however finally must resolve to python-implemented resource types

•  SoftwareConfig/SoftwareComponent as a “meta resource type” pointing to actual implementation in external automation assets (scripts, …)

class MyResourceType(resources.Resource): properties_schema = { 'foo': properties.Schema( properties.Schema.STRING, _('foo prop description'), default='foo', required=True ), } attributes_schema = { ‘attr_1': 'The first attribute', } def handle_create(self): # creation logic def check_create_complete(self, token): # check creation status for longer running tasks def handle_update(self): # update logic def check_update_complete(self, token): # check update status for longer running tasks # handle and check methods for suspend/resume ... def handle_delete(self): # deletion logic def check_delete_complete(self, token): # check deletion status for longer running tasks


Heat declarative processing!

•  Build dependency graph of all resources •  Resources can depend on each other

explicitly: depends_on •  Resources can depend on each other

through data flow: •  get_attr •  get_resource

•  Process all resources that do not have any dependency – i.e. there can be a lot of parallel processing •  Per resource, invoke handle_create

and wait until resource is in state CREATE COMPLETE (check_create_complete)

•  Process all resources according to dependencies

heat_template_version: 2013-‐05-‐23 resources: db_comp: type: OS::Heat::SoftwareComponent properties: outputs: -‐ name: db_port configs: # config actions and scripts ... app_comp: type: OS::Heat::SoftwareComponent properties: inputs: -‐ name: db_port configs: # config actions and scripts ... db_depl: type: OS::Heat::SoftwareDeployment properties: config: { get_resource: db_comp } server: { get_resource: server } app_depl: type: OS::Heat::SoftwareDeployment properties: config: { get_resource: app_comp } server: { get_resource: server } input_values: db_port: { get_attr: [ db_depl, db_port ] } server: type: OS::Nova::Server # ... details ...


Heat stack update!

•  Heat supports update of already deployed stacks in a declarative way •  Create a stack using original template •  Get a template describing the created stack (includes input data provided during

stack creation) •  Modify current template (add, remove or updates resources) •  Invoke heat stack-update with modified template

•  Heat compares the updated template against the currently active template and derives the set of resources to be updated, added or removed

•  Changes get applied according to the dependency graph among resources •  A change to one resource might trigger changes in dependent resources!

•  Resources may support in-place update, or may need replacement •  Example for in-place update: change maximum size of an AutoScalingGroup

resource •  Example for replacement: use of a different image for a server


Heat stack update example!


So`wareDeployment “my_app”

db_pw: get_aFr: [ my_db, pw ]


So`wareDeployment “my_db”

pw: mypassw0rd






pw: otherpassw0rd


192.168.100.10




192.168.100.11


pw: mypassw0rd

template stack

2So`wareDeployment

“my_db”

pw: otherpassw0rd

1stack-‐create

template-‐show

stack-‐update


Agenda!

•  Introduction






Containers and container orchestration overview!

•  Docker is an open technology platform for building and shipping applications as self-contained container images, portable across any Docker runtime

•  Containers provide an operating system environment to applications, but use core operating system capabilities of the host system – containers run as sandboxed processes within the host OS

•  Container images can be customized towards the specific needs of each application

•  Container-based applications (e.g. in micro-services architectures) typically consist of multiple containers with inter-dependencies

Build Ship Run

https://www.docker.com

container dependencies scaling dependencies on

other resources


docker-compose!

•  A tool for defining and running multi-container applications with Docker

•  Multi-container pattern defined in a docker-compose YAML configuration file

•  Ability to create, start, stop, delete complete multi-container deployment

•  Ability to operate containers individually

•  Ability to scale containers defined in the template

https://docs.docker.com/compose/

db: image: db_image:1.0 environment: some_var: some_value volumes: -‐ /var/my_files:/opt/my_db/data app: image: app_image:2.0 ports: -‐ "80:8080" links: -‐ db:database

link to other service "db" using alias "database"

externally exposed ports


Kubernetes!

•  Open-source platform for automating deployment, scaling, and operations of container-based application across clusters of hosts

•  Initiated by Google and donated to Cloud Native Computing Foundation in July 2015

•  Definition of Pods consisting of one or more containers •  Pod is a definition of a desired

state (all containers running) that is ensured by the kubelet engine

•  Automatic placement and scaling of Pods across nodes ("minions") in a cluster

http://kubernetes.io

apiVersion: v1 kind: Pod metadata: name: www spec: containers: -‐ name: nginx image: nginx volumeMounts: -‐ mountPath: /srv/www name: www-‐data readOnly: true -‐ name: git-‐monitor image: kubernetes/git-‐monitor env: -‐ name: GIT_REPO value: http://github.com/some/repo.git volumeMounts: -‐ mountPath: /data name: www-‐data volumes: -‐ name: www-‐data emptyDir: {}

Pod definition


Agenda!

•  Introduction






Orchestration of hybrid workloads!

app app

middle-ware

middle-ware

VM VM VM

Docker

app app *aaS

containerized components full-stack deployments as-a-service components

VM VM

MW MW

MW MW

MW container

Service


Hybrid pattern orchestration in OpenStack Heat!

•  In case of mixed deployments (containers, full stack, external services) a generic orchestration engine with pluggable support for different target platforms and technologies is required

•  Heat allows for using resource type plug-ins (see Heat resource model) for orchestrating any API-based resources • Out-of-the-box support for OpenStack Magnum resources to manage

containers through the Magnum container service as part of a Heat stack

•  Resource type plug-ins can be implemented for any *aaS service to be included in a hybrid Heat stack


Summary!

•  Orchestration describes the automated arrangement, coordination, and management of complex computer systems, middleware, and services •  Orchestration can be done based on patterns as structural models of application-

and infrastructure components and relationships between those components •  Patterns are processed (primarily) in a declarative way, deriving orchestration flows

from the structural model of patterns

•  OASIS TOSCA describes pattern elements and concepts in a standardized way

•  OpenStack Heat is a pattern-based orchestration engine on-top of base OpenStack Service, and has evolved as the de-facto open pattern technology

•  Container technologies (e.g. Docker) are rapidly gaining traction for deploying and managing micro services based applications •  Orchestration technologies for Docker containers have evolved that are based on

simple multi-container environment patterns

•  Orchestration of hybrid environments (e.g. full stack + container + *aaS) requires use of generic orchestration technology (e.g. Heat) able to cover the superset of pattern components

pattern-based orchestration - iaas · pattern-based orchestration university of stuttgart cloud...

Documents