distributed data - centralized policy - openstack-tage.de

Leif Berntsson, DC Systems Engineer

Bastian Offergeld, DC Sales Specialist

June 2016

Distributed Data - Centralized Policy

Apps Applications Micro

ServicesMonolithic Client/

Server

3-Tier

Web/App/DB

SOA Cloud

AccessUser

Interface

Pervasive

DevicesTerminals Desktops GUI Mobile NUI

DevDevelopment

to Production Operators: Periodic

Releases

Continuous

Delivery

Developers: Months Weeks Days

DevOps

IT Consumption Via Cloud is a Complete Paradigm Shift

And, an Evolution of Workloads

Existing IT

App

Web

Servers

App

ServersDatabase

Physical Infrastructure

App App

Cloud-Enabled

Service

Web

Servers

App

ServersDatabase

Local Dedicated Shared

App App Service

Cloud-Native(Containers)

AppApp

Runtime Micro-services

on Containers

.rb .py.go Java

Service

ServiceApp

Scale out infrastructure

Lightweight Linux kernel (e.g. CoreOS)

LX Containers (e.g. Docker, Rocket, Lattice, Flockport)HDFS

Cluster Managers – Orchestration (e.g. Kubernetes, Mesosphere)

API layer

Data Integration Framework / Cisco Data Virtualization

YARN Hadoop MPI Storm Spark Cisco CSA

Apache Tez Jenkins

Data Apps / Services

Impala Hive Shark Kafka Druid MySQL ParStream

Apps 1 Apps 2 Apps 3 …

File, Block, Object

Hypertable

Cassandra

Elastic Search

?Apps

Data Svcs.

Programmable

Infrastructure

Next-Gen Infrastructure Stack + Predictive Data Platform

• OpenStack is NOT a single software package

• There is no 1-800-OpenStack number

• NO clean upgrade path when moving to a newer version

• Deployments are highly customizable, if the installer leaves your company, you are compromised

• Scaling OpenStack is very hard

OpenStack is Not Simple

Networking

Hardware

OpenStack and AWS APIs

OpenStack Unified CLI

HA Service Orchestration

StorageCompute IdentityNetworking

Enhanced

Dashboard

Advanced Operational Support

• 24x7 Cloud Operations and Supports

• Infrastructure Capacity Planning

• Monitoring and Error Detection

• SLA Guarantees

• Platform and Security Updates

• Cloud Design and Deployment

In Your Data Center, on Your Hardware, Delivered as a Service

Cisco Metapod

http://pivotal.io/cisco

Most distributions are

community supported

Support is message

boards and email

No single point of contact

Other OpenStack

ancillary projects

Which distribution?

Which deployment

system?

Many deployment

methods

Many package / update

systems

Best practices on specific

architectures?

IT Challenges of Implementing OpenStack

Support Deployment Complexity

Problems at Scale

Right Tool for the Job

Product Innovation Built on OpenStack

Cisco UCS OpenStack

Optimized OpenStack

Computing

Rich OpenStack Plugins

Wide range of plugins

optimizing both virtual

and physical

infrastructure

Nexus

Application Centric

Infrastructure (ACI)

Group Based Policy

(GBP)

Neutron Pros & Cons

● Powerful API ● Enables More Complex Project

Network Topologies ● Plugin Support Capable to enable other

Network services

● Full software approach doesn’t work● Scaling and performance issues

o Linux based routing

o All L3 traffic flows through the

controllers

o Require 2 additional control plane

servers

● Limited HA capabilites

● L3 fail over requires rebuilding networks on

new controller (active/passive)

Hardware Assisted Neutron

Faster time to production and improved service consistency (SLAs)

Hardware monitoring of controller environment with TAC escalation

Now let’s imagine a network switch … … at the moment, largely configured on the CLI

Cisco ACI solves the problem …

Interfaces, protocols, TCAM, etc … all represented in an object model, and

ALL accessible through an XML/JSON API and CLI

APIC becomes single point of management for the entire fabric … with a policy-based model

What’s Wrong with OpenStackNetworking Today?

Service B Service C

Service A

• No broadcast or multicast

• Resilient and fault tolerant

• Scalable tiers

• Built around loosely coupled services

• Does not care about IP addresses

• Layer 2 and broadcast is the base API

• Network, routers, and subnets

• Based on existing networking models

• No concept of dependency

mapping or intent

External NetworkRouter

Network

and

subnet

Network

and

subnet

Cloud Application Model Neutron Model

MySQL MySQL

Group-Based Policy Model

Policy group: Set of endpoints with the same properties;

often a tier of an application

Policy rule set: Set of classifiers and actions describing

how policy groups communicate

Policy classifier: Traffic filter including protocol, port,

and direction

Policy action: Behavior to take as a result of a match;

supported actions include allow and redirect

Service chains: Set of ordered network services

between groups

Layer 2 policy: Specification of the boundaries of a

switching domain; broadcast is an optional parameter

Layer 3 policy: An isolated address space containing Layer

2 policies and subnets

Policy

Rule SetPolicy

GroupPolicy

Group

Policy Target

Policy Target

Policy Target

Policy Target

Policy Target

Policy Target

Policy Rule

Policy Rule

ConsumeProvide

Classifier Action

Classifier Action

Layer 2 PolicyLayer 2 Policy

Service Chain

Node Node

Layer 3 Policy

Why Cisco ACI and OpenStack?

Distributed, Scalable

Virtual Networking

• Fully distributed Layer 2, anycast

gateway, DHCP, and metadata

• Distributed NAT and floating

IP address

• Choice of group policy or Neutron API

Hardware-Accelerated

Performance

• Automatic VXLAN tunnels at top of

rack (ToR)

• No wasted CPU cycles for tunneling

Operations and

Telemetry

• Troubleshooting across physical and

virtual environments

• Health scores, atomic counters, and

capacity planning per tenant network

Integrated Overlay

and Underlay

• Fully managed underlay network

through Cisco® APIC

• Capability to connect physical servers

and multiple hypervisors to overlay

networks

Service Chaining

• Support for Layer 3 or Layer 2 service

insertion and chaining

• Device package ecosystem for third-

party devices or group-based policy

(GBP) service chaining

Secure Multitenancy

• Virtual network isolation maintained

even when a hypervisor is

compromised

How do I do this with containers?

The Status Quo

Variety of users: cars,

trucks, ambulances, buses,

pedestrians, two-wheelers,

etc.

No Policy: No Lights, No

Lanes, No Rules, No

Governance, No

Enforcement, Best Effort

Meskel Square [ Source: Reddit.com ]

Status Quo: Deploying Applications on Shared Infrastructure

Container Orchestration needs ability to leverage infrastructure differentiation better for Application Performance, Security and Visibility.

Container Stacks

Infrastructure

Unified | Integrated | Automated

Scheduling | Allocation

Visibility

Application Awareness

Infrastructure Capabilities Nexus 2k-9k

Contiv: Making Infrastructure/Solutions Ideal for containers

• Container industry is focused on creating ability to define applications through Docker Compose, Kubernetes Pod definition etc.

• As applications move from development to production, there is need to able to define and enforce infrastructure operational policies

• Contiv is creating industry thought leadership around need for infrastructure policies for containerized applications in a shared infrastructure

• Contiv provides framework and implementation to address operation intent for Infrastructure.

Contiv Addressing Enabling Infrastructure to Run Production Containerized Applications Better

Where does Contiv Fit in the Container Stack ?

Optimized Infrastructure/ Cisco Integrated Infrastructure

Cisco Hardware: UCS Compute, Nexus 9k, ACI

Ops Orchestration/PaaS (Provides Roles/Multi-tenancy/Visibility/GUI), Contiv Plugins

Container Optimized OS

Container Cluster Scheduler | Contiv Cluster-wide Intent Manager

Container Image

Store

Container Runtime (Docker, etc.)

Contiv Networking/Volume Agents

Developer

DevOps

SysAdmin

Host-1 Host-n

Contiv– Best Choice for Enterprise Containerized Application Deployments

• Best integration with existing infrastructure install-base, any network

topology• No topology/connectivity/feature changes to get started with containers

• Best leverage of infrastructure hardware (UCS, Nexus)• Integrated with Cisco ACI for container applications for highly scalable

solutions

• Consistent behavior with variety of workloads (VM, Container, Bare-metal)• Native visibility of container workloads in network

• Value added features• Scalable Policies based approach, Multi-tenancy with telemetry and fully

automated cluster maintenance

• Feature Rich Integration with Container eco-system - Docker,

Kubernetes/Mesos

How do we put all this together?

Pets vs. Cows

IT treats the servers as a Pet. A lot

of care and time is spent to ensure

the server is running.

IT treats the servers as Cows. Even if a

Cow dies is not important. They will be

replaced, the important thing is that the

“heard” survives.

If your servers have names … you are treating them as Pets!

N-Tiered Apps vs. Micro-services

Presentation Presentation

Logic Logic

Persistence

Database

Logic Logic

API Gateway

Microservice Microservice

Microservice Microservice

Microservice

Database Database

Queue

Web client IoT Mobile client

http

http

http http

http publish

subscribe

http, json, notifications, webhooks

Application Complexity is shifted to the Network

http http

In Microservices application complexity is running through the network

The world’s largest taxi company owns no

vehicles.

The world’s most

popular media

company creates no

content.

The world’s most

valuable retailer has no

inventory.

The world’s largest

accommodation provider

owns no real estate.

The Digital Disruption Era

All the above companies have adopted Micro services

The world’s largest movie rental company owns no

movies.

NETFLIX

PaaS dilemma

• PaaS is great for application deployment.

• But, still creates separate silos for statefull services like databases and message buses.

• Need a more unified way of deploying stateless micro-services and statefull services.

PaaS Cluster Statefull Services Storage Services

App App App

App App App

DB DB

DB DB

Orchestration

Containers

Storage

Compute

Networking

Virtual B/M

DB LB

Integration

Data

Applications

…

Storage

Compute

Networking

Virtualization

O/S

Databases

Integration

Data

Applications

Traditional

Storage

Compute

Networking

Virtualization

O/S

Databases

Integration

Data

Applications

IaaS

Storage

Compute

Networking

Virtualization

O/S

Databases

Integration

Data

Applications

PaaS New Breed of PaaS

Container Stack Components

http://www.eightypercent.net/post/layers-in-the-stack.html

Stripped OS

Infrastructure

as Code

Container Engine

Container

Image

Registry/

Repository

Orchestration

Persistent Storage

Networking

MANTL

• Cisco’s answer to an open container stack.

• Open source, end to end, integrated stack for running container workloads. Including Deployment automation & assurance.

• Pluggable, designed to grow into a platform for application and data services.

https://mantl.io

CNDP: Cloud Native DevOps Platform

Infrastructure (Private, Public, Managed)

Unified Orchestration

Application Intelligence:

Management, Networking Security and Compliance

Mantl

CNDP

IT Risk Management

Po

licy

https://cncf.io/ https://www.opencontainers.org/