Building Scalable, Open, Programmable and Application Centric Data Center with

Cisco ACI

林瑝錦 / Jerry Lin

Cisco Systems

2015 July

Data Center Demands For the Cloud-Era

VM Density and Server I/0

10G LAN on

Motherboard2

Big Data

IP Traffic 25%

CAGR4

“Bare Metal”

75% physical

servers1

Multi-Cloud

~45% of DC

Multi-Hypervisor3

Lower TCO | Workload Flexibility | Agility | Compliance/Security

Impact of Server Virtualization on Network Complexity

3

$0

$50

$100

$150

$200

$250

96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13

Cu

sto

me

r S

pe

nd

ing

$B

Server Spending

Standalone Servers - Mgnt & Admin

Virtual Servers - Mgnt & Admin

Power & Cooling Expense

Source: IDC, 2011 “New Economic Model for the Datacenter”

Server utilization

improved by almost 4x

Server CapEx

spend dropped

However VM related OpEx

significantly increased

“Increased OpEx is

attributed to network

optimization to VM’s to

deliver application SLA’s”

Source: ZK Research

Orchestration in the IT world

Overloaded Network Constructs

VLAN VLAN VLAN

Subnet Subnet Subnet

Basic Network Policy

SLAs L4-7 Services

Network constructs are overloaded with unintended functionality.

Application Language Barriers

Developers

Application

Tiers

Provider /

Consumer

Relationships

Infrastructure Teams

VLANs

Subnets

Protocols

Ports

Developer and infrastructure teams must translate between disparate languages.

Industry Standards and Forums

Initiatives:

Quantum (Folsom release)

Donabe

Overlay Working Groups:

NVO3, L2VPN, TRILL, L3VPN, LISP, PWE3

API Working Groups/BOFs

NETCONF, ALTO, CDNI, XMPP, SDNP, I2AEX

Controller Working Groups:

PCE, FORCES

Protocol Working Groups:

IDR, IS-IS, OSPF, MPLS, CCAMP, BFD

New working group:

I2RS – Interface to the Routing System

Technical Advisory

Group, Working Groups:

Config, Extensibility,

Futures/FPMOD/OF2.0

ETSI SGI on “Network

Function Virtualization”

Open Source Cloud

Computing project

802.1 Overlay Networking ProjectsSDN WG

Open Network Research

Center at Stanford University

Network Programmability Models

Control Plane

Data Plane

Controller

Data Plane

Applications

Vendor-

specific APIs

OpenFlow,

PCEP,

I2RS

2a Pure SDN

Vendor

Specific

(e.g. onePK)

Controller

Data Plane

Applications

Vendor-

specific APIs

OpenFlow,

PCEP,

I2RS

Control Plane

2b Hybrid SDN

Applications

Virtual Switch

Overlays

Overlay

Protocols

(e.g. VXLAN)

Vendor-

specific APIs

3 Overlays Networks

Control Plane

Data Plane

Overlays

Vendor-

specific APIs

Applications

1 Programmable APIs

Control Plane

Data Plane

Vendor

Specific

(e.g. onePK)Vendor

Specific

(e.g. onePK)

Openstack and Network Overlays Apply to All Models (Physical/Virtual)Custom Features Can Be Built

CLI,

SNMP,

Netflow,

…

Applications(Network Mgmt,

Monitoring, …)

40G Aggregation

Scale and PerformanceLow latency, High

Transaction Processing

Capabilities

Scalable

multi-tenancy

Availability with

Multi-tenant scale

End-host scale

for v4/v6

Secure Multi-

tenant cloud

Solution based on

Openness, Service Agility

Open, Extensible

Framework,

Multi-hypervisor Support

Agility, Automation

CTC

Programmability,

Telemetry,

Troubleshooting

We Listened To You!

Future Proof Investment in

SDN, Open Solution, 10/40G

Scale

10/40/100GOpen Policy

AutomationMulti-Tenant

Security

Telemetry Investment

Protection

Physical AND Virtual

Application Centric Infrastructure

ACI

GROUP-BASED

POLICIESCONTROLLER

Best SDN Controller

Interop 2015

Rapid Deployment of Applications onto Networks with Scale, Security and Full Visibility

ACI FABRIC

Integrated GBP VXLAN Overlay

Embracing SDN and Going Beyond

http://www.interop.com/lasvegas/special-events/best-of-interop-awards.php

ACI Fabric

• Industry’s most efficient fabric

• 1G/10G/40G edge - High density 40G spine (100G capable)

• Routed fabric – Optimal IP Forwarding

• Bridging (L2) and Routing (L3) of VXLAN/NVGRE/VLAN at scale

• No x86 GW’s – Physical & Virtual

• Full visibility into virtual and physical

• Common operations from Hypervisor to Compute, To Fabric, to WAN

Virtual Overlay Networks Drive Cloud Readiness• Unprecedented Infrastructure Flexibility

L2 / VLAN MOBILITY CONSTRAINTS

POD

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

POD

• Without planning, physical

networks can introduce

obstacles to VM migration

• Resource sharing over

larger resource pools can

optimize costs

• Goal is to reduce

management complexity

and integrate physical and

virtual (consistent

management, visibility,

policies, etc..)

POD POD

Virtual Overlay Networks Drive Cloud Readiness• Unprecedented Infrastructure Flexibility

SECURITY POLICIES ENFORCED INDEPENDENT OF LOCATION

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

VIRTUAL

NETWORK

OVERLAY

• Virtual Network Overlays

remove network

complexity, increase scale

• VXLAN tunnels provide

logical isolation of network

traffic

• vPath-enabled services

provide location

independence of services

and consistency for apps

independent of location

“Users”“Files”

ACI Fabric

Logical Endpoint Groups by

Role

Heterogeneous clients, servers,

external clouds; fabric controls

communication

Every device is one hop away, microsecond

latency, no power or port availability

constraints, ease of scaling

Flexible Insertion

ACI Controller manages all

participating devices, change control

and audit capabilities

Unified Management and Visibility

Fabric Port Services

Hardware filtering and bridging; default

gateway; seamless service insertion,

“service farm” aggregation

Flat Hardware Accelerated

Network

Full abstraction, de-coupled from

VLANs and Dynamic Routing, low

latency, built-in QoS

Application Centric Infrastructure Fabric

VXLAN

VNID = 5789VXLAN

VNID = 11348

NVGRE

VSID = 7456

Any to Any

802.1Q

VLAN 50

Normalized

Encapsulation

Localized

Encapsulation

IP Fabric Using

VXLAN Tagging

PayloadIPVXLANVTEP

• All traffic within the ACI Fabric is encapsulated with an extended VXLAN header

• External VLAN, VXLAN, NVGRE tags are mapped at ingress to an internal VXLAN tag

• Forwarding is not limited to, nor constrained within, the encapsulation type or

encapsulation ‘overlay’ network

• External identifies are localized to the Leaf or Leaf port, allowing re-use and/or translation

if required

Payload

Payload

Payload

Payload

Payload

Eth

IPVXLAN

Outer

IP

IPNVGREOuter

IP

IP802.1Q

Eth

IP

Eth

MAC

Normalization of Ingress

Encapsulation

ACI Fabric – Integrated OverlayData Path - Encapsulation Normalization

APPLICATION

SECURITY

Web

Tier

App

Tier

DB

Tier

Trusted

ZoneDB

Tier

DMZ

External

Zone

Cloud

Application Admin

Security Admin

Network Admin

Cloud Admin

ACI Goal: Common Policy and Operations Framework

16

Application Admin

Security Admin

Network Admin

SECURITY

Trusted

ZoneDB

Tier

DMZ

External Zone

APPLICATION

COMMON POOL OF RESOURCES

Cloud Admin

Cloud

ACI Goal: Common Policy and Operations Framework

17

Application Policy Model and Instantiation

All forwarding in the fabric is managed through the application network profile

• IP addresses are fully portable anywhere within the fabric

• Security and forwarding are fully decoupled from any physical or virtual network attributes

• Devices autonomously update the state of the network based on configured policy requirements

Policy instantiation: Each device

dynamically instantiates the required

changes based on the policies

VM VMVM

10.2.4.7

VM

10.9.3.37

VM

10.32.3.7

VMVM

Application policy model: Defines

the application requirements

(application network profile)

Application

Client

App Tier DB Tier

Storage Storage

Web Tier

Defining Application Logic Through PolicyApplications and Conversations

• Application communication can be defined as who is allowed to talk to whom.

DB Farm

App ServersWeb

FarmUsers

• Communication between objects on the network can be thought of as one or two way conversations (monologue/dialogue.)

Building ACI Contracts

Subjects are a combination of A filter, an action and a label

Contracts are groups of subjects which define communication between source and destination EPGs.

Filter | Action | LabelSubject

TCP Port 80

Filter

Permit

Action

Web Access

Label

Actions are policy options: Permit the traffic Block the traffic Redirect the traffic Log the traffic Copy the traffic Mark the traffic (DSCP/CoS)

The defined policy encompasses traffic handling, quality of service, security monitoring and logging.

Subject 1

Subject 2

Subject 3

Contract 1

Cisco ACI Layer 4-7 Service Integration

EXTERNAL

Application Profile

APP APP APP

APP APPDBAPPPolicyPolicy Policy

WEB WEB WEB

WEB DBDB DB DB

Func:

Firewall

Func:

Load Balancer

Service Graph: “WebGraph”

Func:

Load Balancer

Service Graph:

“appGraph”

Terminal: Input1Terminal: Output1

Policies Policies Policies

ACI Fabric Powered with Group-Based Policies

ACI Fabric

Scale-Out Penalty-Free Overlay

AppWeb DB

QoS

Filter

Filter

LB

APIC APIC APIC

Firewall

Filter

Connectivit

y

Outside

(Tenant VRF)

Application Network Profile

“Users”“Files”

ACI Fabric

Define Endpoint Groups

Any endpoints anywhere within the

fabric, virtual or physical

Enforce Ingress Fabric Rules

Hardware rules on each port, security in

depth, embedded QoS

Single Point of Orchestration

Different administrative groups

use same interface, high level

of object sharing

Application Policy Infrastructure

Controller (APIC)Create Contracts Between Endpoint

Groups

Port-level rules: drop, prioritize, push to

service chain; reusable templates

Service Graph

Single Pass Services

Security administrator defines

generic templates in APIC,

availed to contract creation

All TCP/UDP: Accept, Redirect

UDP/16384-32767: Prioritize

All Other: Drop

Policy Contract “Users → Files”

ACI is a network Fabric which provides a new communication abstraction model

L/B

EPG

APP

EPG

DBF/W

EPG

WEB

Application Network Profile

VM VM VM

WEB PORT GROUP APP PORT GROUP DB PORT GROUP

Hypervisor Integration with ACI

APIC Relationship is formed between APIC and

Virtual Machine Manager (VMM)

ACI Fabric implements policy on Virtual

Networks by mapping Endpoints to EPGs

Endpoints in a Virtualized environment are

represented as the vNICs

VMM applies network configuration by

placement of vNICs into:

Port Groups (VMWare),

VM Networks (Hyper-V)

Networks (OpenStack)

EPGs are exposed to the VMM as a 1:1

mapping to Port Groups, VM Networks or

OpenStack Networking.

ACI Hypervisor Integration: VMWare DVS

VIRTUAL

Virtual Distributed Switch

APIC Admin

HYPERVISOR HYPERVISOR

WEB APP DB WEB APP DB

1 Cisco APIC and VMware vCenter Initial Handshake2 Create VDS3 Attach Hypervisor to VDS4 Learn location of ESX Host through LLDP5 Create Application Policy6 Automatically Map EPG To Port Groups7 Create Port Groups8 Instantiate VMs, Assign to Port Groups9 Push Policy (Lazy)

vCenter

WEB PORT GROUP

APP PORT GROUP

DB PORT GROUP

VM Admin

Open: APIC Programming Interfaces

APIC

Open REST APIs Support Integration With Any Software

OpFlex: Open Fabric Attached Device API Supports Integration with

Any Network Device

AutomationEnterprise Monitoring

SystemsManageme

ntOrchestrationFrameworks

OVM

Hypervisor Management Applications

NORTHBOUND PROGRAMMABILITY LAYER

SOUTHBOUND PROGRAMMABILITY LAYER

OpFlex – A Flexible, Extensible Policy ProtocolOPFLEX is a new extensible policy resolution protocol designed for declarative management of any datacenter infrastructure. Unlike legacy protocols such as OVSDB, OPFLEX was designed to offer: APIC

Opflex Agent Opflex Agent Opflex Agent Opflex Agent

Opflex Proxy

Hypervisor

Switch

Opflex

Agent

Firewall

Opflex

Agent

ADC

Opflex

Agent

Declarative resolution – Push + Pull API support

Abstract policies rather than device-specific

configuration

Flexible, extensible definition of using XML / JSON

Support for any device – vswitch, physical switch,

network services, servers, etc.Legacy API

Policies

Who can talk to whom

What about

Topology control

Ops stuff

http://tools.ietf.org/html/draft-smith-opflex-00

Cisco Virtual Networks Support Multiple Cloud Stacks

Physical Network

vSphere Hyper-V

Unified Fabric (Nexus 2000 – 7000)

UCSComputing Platform

HypervisorMultiple (Hyper-V,

KVM,…)

vCloud

Director/

DynamicOps

System

Center

Cloud Portal

and Orchestration

Storage Platform

Cisco UCS Director

OpenStack

and Partners

Virtual Network

Infrastructure

L4-7

L2-3

vPath

Nexus 1000V

Cloud Network ServicesWAAS NAMASA 1000V NetScaler ImpervaVSG

ACI Summary

Physical

Virtualization

Networking

APP DBPOLICY WEB

HYPERVISORHYPERVISOR HYPERVISOR

APICApplication

External Network POLICY POLICY

Polic

y D

riven

Why ACI?

The data center is both Virtual and Physical

Enterprise Scale and Performance requires hardware acceleration

A SINGLE architecture to deliver performance, programmability, agility and reduced complexity

An Application Centric Policy Model that dynamically defines the network fabric by means of the application requirements

An AUTOMATED network fabric for both virtual and physical workloads and services

Reduce Network

Provisioning

58%Reduce

Management

Costs

21%Reduce

Power and

Cooling Costs

45%CAPEX

Reduction

25%Compute and

Storage

Optimization

10–20%

GreaterBusiness

AgilityLower Capital

Expenses

Reduced Costs/

Complexity

Lower Operating

CostResource

Optimization

Delivering Business OutcomesExample: Cisco IT with ACI (Based On Projections)

*Based on Cisco IT Projections

Cisco IT has already gained cost efficiencies through UCS. These are incremental savings with ACI.

SECURITY

INHERENT Security and INTEGRATION

TELEMETRY

Rich TELEMETRY & Application HEALTH

SCORE

SPEED

NETWORK and SERVICES Delivered in

minutes

POLICY

Policy-based deployment/governance

Physical & virtual

Cisco ACI TakeawaysCisco Application Centric Infrastructure

Fixed Workloads Variable Workloads

OPEN and AGNOSTIC