cisco's cloud services extending the...

Cisco's Cloud Services Router (CSR 1000V):

Extending the Enterprise Network to the Cloud

Ray Wong, Technical Marketing Engineer

BRKVIR-2016

Housekeeping

• We value your feedback – don’t forget to complete your online session evaluations after each session

• Visit the World of Solutions and Meet the Engineer

• Please switch off your mobile phones

• Follow us on Twitter for real time updates of the event: @CiscoLive, #CLUS

• CSR 1000V Overview and Architecture

• Licensing

• Use Cases

• CSR 1000V in Public Cloud

• Deployment and Management

• Performance and Scale

• Q&A

Agenda

CSR 1000VOverview & Architecture

ASR 1K Architecture

• RP

• Handles control plane traffic

• Manages system

• ESP

• Handles forwarding plane traffic

• SPA Interface Processor

• Shared Port Adaptors provide interface connectivity

• Centralized Forwarding Architecture

• Traffic flows through the active ESP

• Standby is synchronized with all flow state

• Distributed Control Architecture

• Dedicated control processors for major systems components

Route Processor (standby)

RP

Interconn.

Embedded ServicesProcessor(active)

FECP

Interconn.

QFP subsys-

temCrypto assist

Embedded ServicesProcessor(standby)

FECP

Interconn.

QFP subsystemCrypto

assist

SPASPA

IOCPSPA

Agg.

…

Interconn.

SPASPA

IOCPSPA

Agg.

…

Interconn.

SPASPA

IOCPSPA

Agg.

…

Interconn.

Passive Midplane

Route Processor

(active)

RP

Interconn.

RP CPU

CSR 1000V: Take ASR 1001 and Remove Hardware

ESP FECP

Interconn.

Crypto assist

Chassis Mgr.

Forwarding Mgr.

Chassis Mgr.

Forwarding Mgr.QFP Client / Driver

Interconn.

Interconn.

SIP

SPASPA

IOCP

SPA Agg.

Interconn.

Kernel (incl. utilities)

Chassis Mgr.SPA driver

SPA driver

SPA driver

SPA driver

IOS



QFP subsys-tem

QFP code

VSR 1000 (virtual IOS XE)

ConsoleMgmt ENET Ethernet vNICsFlash / DiskMemoryCPU

CSR 1000V: Embed the Resulting Software in a VM

RPChassis Mgr.

Forwarding Mgr.IOS


ESP

Chassis Mgr.

Forwarding Mgr.FFP Client / Driver

FFPcode

• No crypto ASIC

CSR 1000V leverages AES-NI

• No QFP

Lower forwarding performance

• No hardware accelerators

Less efficient feature processing

Cisco Cloud Services Router (CSR 1000V)

Virtualized Networking with Rapid Deployment and Flexibility

Cisco IOS XE Software in Virtualized Form-Factor

IOS XE Cloud Edition

• Selected IOS XE features based on use cases

Infrastructure Agnostic

• Supports any x86 server or vSwitch

• Runs on ESXi, KVM, Hyper-V, Xen, Amazon AWS, Microsoft Azure*

Throughput Elasticity

• Delivers 10Mbps to 20 Gbps throughput

Multiple Licensing Models

• Term, Perpetual, Usage**

Programmability

• RESTful APIs for automated management

Server

Hypervisor

Virtual Switch

OS

App

OS

App

CSR 1000V

* Available from June 2015

** Available on AWS, Smart Licensing (CA)

Supported Hypervisors and vNICs (IOS XE 3.15)

VMWare ESXi KVM Microsoft

Hyper-V

Citrix

XenServer

Supported

Versions

5.0, 5.1, 5.5 RHEL 6.6,

Ubuntu Server

14.04 LTS

Windows Server

2012 R2

6.2

Supported NIC

Types

VMXNET3ixgbevf/ixgbe

VirtIOixgbevf/ixgbe

HV NETVSC VIFixgbevf/ixgbe

Max. Number of

vNICs per VM

10 26 3 7

vNIC Hot

Add/Remove

Support

Yes Yes No No

SR-IOV Support Yes(since XE 3.13)

Yes (since XE 3.12.1)

Yes(since XE 3.13)

Yes(since XE 3.12.1)

CSR 1000V Architecture – Virtualized IOS XE

Virtualized IOS XE

• Generalized to work on any x86 system

• Hardware specifics abstracted through a virtualization layer

• Forwarding (ESP) and Control (RP) mapped to vCPUs

• Bootflash / NVRAM are mapped into memory from hard disk

• No dedicated crypto engine – leveraged the Intel AES-NI instruction set to provide hardware crypto assist

• Boot loader functions implemented by GRUB

Control PlaneForwarding Plane

vNICvCPU vMemory vDisk

Physical Hardware

CPU Memory Disk NIC

Hypervisor (VMware / Citrix / KVM)

Chassis Mgr.

Forwarding Mgr.

IOS

Chassis Mgr.

Forwarding Mgr.

FFP Client / Driver

FFP code Linux Container

CSR 1000V Architecture – IOSd

• Runs as a process under the Guest Linux Kernel

• IOS timing is governed by Linux Kernel scheduling

• Provides virtualized management ports

• Managed by their respective software processes

• No direct hardware component access

• Runs Control plane features

• CLI and configuration processing

• SNMP handling

• Running routing protocols & computing routes

• Interfaces, tunnels and sessions management

• Processing of punted features (legacy protocols)

Control PlaneForwarding Plane

vNICvCPU vMemory vDisk

Physical Hardware

CPU Memory Disk NIC

Hypervisor (VMware / Citrix / KVM)

Chassis Mgr.

Forwarding Mgr.Chassis Mgr.

Forwarding Mgr.

FFP Client / Driver

FFP code

IOS

CSR 1000V Architecture – Hypervisor Interaction

• Hypervisor abstracts and shares physical hardware resources from / among multiple VMs

• Scheduling of vCPUs onto physical cores can create non-deterministic behavior

• Scheduling of vNICs onto physical ports can lead to packet losses / jitter

• ESXi Scheduler spreads the load across all physical cores intelligently according to a proportional share-based algorithm

UCS

Blade

Blade

Phy i/f Phy i/f

CPU

Core Core

Hypervisor

VMCSR

vCPU

CPU

Core Core

vCPUvCPUvCPU

Scheduler

Vswitchport port

Memory

vMem

Tables

VNIC

VMCSR

VNIC

vMem

Tables

x86 machine

Host-OS /

KVM

Qemu /

v-Host

tap

vSwitch (OVS) / Linux bridge

NIC driver

Guest-OS

Virtio-net

Guest-OS

Virtio-net

Qemu /

v-Host

tap

CSR CSR

CSR 1000V Architecture – KVM Example

• Hypervisor virtualizes the NIC hardware to the multiple VMs

• Hypervisor scheduler responsible for ensuring that I/O processes are served

• One vHost/VirtIO thread used per configured interface (vNIC)

• Each VM appears as a regular Linux process to the Host OS

• Linux schedulers generally time-share between processes

NIC port

CSR 1000V Architecture - vCPU allocation

# vCPUs Virtual Route

Processor

Virtual Forwarding

Processor

1 1 1

2 1 2

3 1 2-3

4 1 2-4

5 1 2-5

6 1 2-6

7 1 2-7

8 1 2-8

Control Plane Data Plane

CSR 1000V

Separation of control plane and data plane

vCPU allocation is static and done during boot-up

CSR 1000V Architecture – Network I/O

Method Driver Performance Pros/Cons Supported

Emulated E1000 Low Wide compatibility

Worst performance

NO

Para-virtualized VMXNET3

VirtIO

Excellent “Virtualization aware”

High degree of

interaction between

guest OS and

hypervisor – para APIs

YES - default

Pass-through Depends on

NIC type

Best Direct access to HW –

high I/O

Lose virtualization

features such as

vMotion

YES – only Intel

NICs (ixgbevf /

ixgbe

drivers)

I/O Optimizations: SR-IOV with PCIe Pass-Through

• Allows a single PCIe device to appear to be multiple separate PCIe devices

• NIC supports virtualization

• Enables network traffic to bypass software switch layers

• Creates physical and virtual functions (PF/VF)

• PF: Controls sorter

• VF: Passes packets

• Requires support in BIOS/Hypervisor

• Intel VT-D / AMD IOMMU

x86 machine

NIC

Host-OS / KVM

Guest-OS Guest-OS Guest-OS

layer-2 sorter / switch / classifier

VF VF VF PF

VF driver VF driver VF driver

AppAppAppAppAppApp

AppAppApp

SR-IOV

Master

Driver

I/O Optimizations: UCS VM-FEX

• UCS VM-FEX provides dedicated hardware resources to each VM

• vSwitch and hypervisor virtualization layers are bypassed

• Virtualization performed in hardware

• One-to-one relationship between VM

• Can run in DirectPath or emulated mode

• Support for vMotion

• Requires dedicated cards (e.g. VIC1280)

Licensing

CSR 1000V Licensing Overview

• Since IOS XE 3.13, CSR 1000V package names are now: IPBase, Security, AppX and AX

• ‘license boot level’ command adjusted accordingly

• Old CLI commands are hidden but still accepted (‘[premium | advanced | standard]’)

• Smart Licensing available since 3.14

• Evaluation licenses can be generated for 60 days using the demo portal (www.cisco.com/go/license)

• Requires CSR 1000V UDI – “show license udi”

• After evaluation period expires, throughput will be throttled to 100Kbps

• Refer to CSR SW Config Guide for license management details http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/licensing.html

http://www.cisco.com/go/license

http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/licensing.html

CSR 1000V Licensing Key Concepts

• CSR license is tied to the UDI (Unique Device Identifier)

• UDI = Product ID (CSR1000V) + Serial Number. CSR internally generates its own random serial number on its first boot and stores it persistently in the image

• UDI will change when CSR is cloned, invalidating the license

• UDI will not change during vMotion or similar operations. License will remain valid

CSR 1000V Licensing StructurePick one option from each column…

Technology Package(See next slide for details)

Throughput License Type

Example:

IP Base

250 Mbps

1-Year

IP Base10 Mbps

50 Mbps

100 Mbps

250 Mbps

500 Mbps

1 Gbps

2.5 Gbps

5 Gbps

Perpetual

Subscription

(1-year or 3-year)

Usage

(target date CY15)10 Gbps

SEC

AppX

AX

* CSR add-on license options not shown above

CSR 1000V Technology Package FeaturesTechnology Package IOS-XE features

IP Base(formerly Standard)

Basic Networking: BGP, OSPF, EIGRP, RIP, ISIS, IPv6, GRE, VRF-LITE, NTP, QoS, PBR

Multicast: IGMP, PIM

High Availability: HSRP, VRRP, GLBP

Addressing: 802.1Q VLAN, EVC, NAT, DHCP, DNS

Basic Security: ACL, AAA, RADIUS, TACACS+, SGT/TrustSec, VASI

Management: IOS-XE CLI, SSH, Flexible NetFlow, SNMP, EEM, NETCONF

SEC(formerly Advanced)

IP Base plus…

Advanced Security: Zone Based Firewall, IPSec VPN, EZVPN, DMVPN, FlexVPN,

SSLVPN, GETVPN

High Availability: Box-to-box HA for FW and NAT

AppX / APP

IP Base plus…

Advanced Networking: L2TPv3, BFD, MPLS, VXLAN

Unified Communications: CUBE-ENT

Application Experience: WCCPv2, AppNav, NBAR2 / AVC, IP SLA

Hybrid Cloud Connectivity: LISP, OTV, VPLS, EoMPLS

Subscriber Management: PTA, LNS, ISG

AX(formerly Premium)

ALL FEATURES

CSR 1000V Performance-to-Footprint in XE 3.15

• For each throughput/technology-package combination, the minimum required vCPU and RAM is listed

• Performance results based on 1500 Byte packets and VMWare ESXi

Throughput IP Base SEC AppX AX

10 Mbps 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB





1 Gbps 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB 2vCPU/4GB

2.5 Gbps 1vCPU/4GB 1vCPU/4GB 4vCPU/4GB 4vCPU/4GB

5 Gbps 1vCPU/4GB 2vCPU/4GB 8vCPU/4GB NA

10 Gbps 2vCPU/4GB NA NA NA

SHAPER

(50)

G1

G2

G3

G4

15 Mbps

20 Mbps

10 Mbps

15 Mbps

ESP

10Mbps (60-50)

G1->G3: 15

G2->G4: 20

G3->G2: 10

G4->G3: 15

Total: 60 Mbps

CSR 1000V License Throughput Enforcement• A shaper is implemented in the ESP data path at the root of the QoS hierarchy

• All egress traffic is subjected to the shaper

• Max. rate parameter (derived from license) is programmed into the shaper

• Throughput limits are checked “globally”, not on per-interface basis

• Without any interface QoS Configuration, each interface gets an equal available bandwidth share

• Shaper does not distinguish between different types of traffic

• To ensure high-priority traffic is not dropped by the license shaper, configure QoS

• E.g. LLQ on interfaces (leveraging priority propagation of the QoS Scheduler)

• Note that Control Plane Policing can be applied to also mark control plane packets!

CSR1000V Smart Licensing: Pooling

Advanced Security Licenses

Use any compatible licenses from pool with devices

London

Device 1

Brisbane

Device 2

Tokyo

Device 3

Smart Account (Pool)

London

Device 1

Brisbane

Device 2

Tokyo

Device 3

Associate licenses with virtual accounts

• Full visibility to all assets across the company

• Central repository for all licenses

• Licenses are company account specific and can be

used with any compatible device in your company

• License associated with specific device

• No easy means to move licenses from one

device to another

Traditional Node-Lock Pooling

Smart Software Licensing Overview How it works…

Collectors

*License Pooling is handled through the Cisco Smart Software Manager

Dis

trib

ute

d

De

plo

ym

en

tC

en

tra

l D

ep

loym

en

t

Cisco Commerce

Workspace

Cisco

Smart

Software

Manager

Customer

Places Order

Customer

Manages

Licenses

Usage

Cisco Smart SoftwareManager

Distribution

Entitlement

Router

Switch

Firewall

Unified Communications

CSR 1000V

Router

Switch

Firewall

Video

Unified Communications

1

3

Customer

Activates

and Uses

Software

2

Annuity Platform

Cisco Smart Software Manager

Warning and Notifications -25Insufficient licenses – 25 needed to return to compliance

License Quantity In

Use

Surplus /

Shortage

50 Mbps SEC 300 325 -25

100 Mbps AX 500 425 +75

CSR1000V Smart Licensing: Out-Compliance Scenarios

Smart Licensing workflow

Could not connect to smart licensing portal or collector

after first install

Operate in default mode (100Kbps, CSR-AX)

Was able to register with

smart licensing & activated

CSR performs with configured feature set

and performance

Not able to report to smart portal or

collector for 90 days in a row

Operate in default mode

CSR configured more than purchased

feature set & performance

CSR reports out of compliance for 90

days

Use Cases

CSR 1000V Secure VPN Gateway

CSR

1000V

ISR

Virtual Private Cloud

Distribution

and ToR

Switches

Servers

Virtual Private Cloud

Data

Center

ASR

CSR

1000V

Cloud Provider Data Center

Enterprise

VPN Challenges Integrating Enterprise & Cloud VPN

policies

Backhaul to data center increases

latency

Each cloud imposes different VPN

type and scale limits

VPN Solutions

Common VPN Types: IPSec,

DMVPN, GETVPN, EZVPN,

FlexVPN

Routing based VPNs and private

addressing

Firewall, ACLs, AAA

CSR Benefits

Direct, secure access. Avoids

backhaul to data center.

Familiar, reliable, and scalable VPN

Compatible with existing

management tools

Internet

Branch

Location

WAN

RouterBranch

Location

ISR

SSL VPN on CSR 1000V

• IPv4 available since IOS XE 3.12.1 and IOS-XE 3.13

• IPv6 available since IOS XE 3.15

• Supports Full Tunnel (Thick Client)

• AnyConnect client

• Clientless (browser based) and Thin Client ( port forwarding) modes not supported

• Amazon/AWS support

• IPsec and SSL can co-exist

CSR

1000V

vCE

PE

WAN

Router

VPC/ vDC

Cloud CE/PE Router

MPLS

Servers

Segment A

Segment B

DC

Fabric

Tenant Scale

CSR

1000V

vPE

PE

WAN

Router

VPC/ vDC

MPLS

Servers

Segment A

Segment B

DC

Fabric

VLAN

MPLS

IPoVLAN, IPoIP, MPLSoVLAN, MPLSoIP (IP=GRE, VXLAN, etc.)

MP-BGP

Benefits

• More Tenants per Physical Infrastructure

• End-to-end Managed Connectivity and SLAs

Challenges

• Mapping tenant traffic from VRFs to VLANs

• Maximum 4,096 VLANs limits scalability

VxLAN on CSR 1000V

Destination is in another segment.

Packet is routed to the new segment

VXLANORANGE VXLANBLUE

Ingress VXLAN packet on

Orange segment

VXLAN

Router

BDI

Egress interface chosen

VLAN100 VLAN200

CSR as VXLAN

L3 Gateway

Uses EVC (Ethernet Virtual Circuits): BD (Bridge Domain – L2) and BDI (Bridge Domain Interface – L3)

Unicast or Multicast (bidir-PM) control plane

Supports VxLAN routing – unique to CSR and ASR1K! Not yet available on merchant silicon HW platforms

Supports VRF Aware VxLAN (multiple VTEP support)

Network Function Virtualization with CSR 1000V

Edge

Corporate

Business

CPERR

ISP

VOD TV SIP

Content Farm

BNG

PE

Access & Aggregation

OLT

xPON

xDSL

DSLAM

WirelessWiFi

Wireline

CableDOCSIS

LNS

IP/MPLS CoreA

Peering

HGW

Residence

Mobile Subscriber

• High Speed CPE • WiFi Access Gateway• BNG-LAC, PTA• PE (L3VPN and L3VPN)

• LNS• Route Reflector• Internet Peering

CGN

vCPE

vBNGvRR

vLNS

vCGN

vPE

CSR 1000V in Public Cloud

IOS XE Coverage for All Deployment Types

Enterprise Data Center

ISR 4000 ASR 1000

Hypervisor

CSR 1000V

Cloud Platform

CSR 1000V

The Benefits of Bringing IOS XE into Public Clouds

Extends Existing Routing

Topology

Integrates With Existing VPN Topology (Eg.

DMVPN)

Shares Existing Zone Based

Firewall Policies

Network Logging to

Existing Tools

Identifies Cloud Performance

Problems

IOS XE Supportable by Existing IT Staff

Existing Monitoring Tools

Existing Troubleshooting

Steps

Q: Where can I find the CSR on AWS?A: In the AWS marketplace!

1. Search for “Cisco”

2. Pick a flavor

What are all the different CSR 1000V types listed?

• Cloud Services Router 1000V BYOL

• Can be any tech package and throughput level depending on license purchased from Cisco and installed on CSR (not all throughputs supported)

• Cloud Services Router 1000V Security Tech Package

• Includes features from the Security technology package. Performance based on AWS instance type selected (more or less vCPU/vMemory)

• Cloud Services Router 1000V AX Tech Package

• Includes features from the AX technology package. Performance based on AWS instance type selected (more or less vCPU/vMemory)

• “Maximum Performance” versions of the above three

• Enables SR-IOV enhanced networking for higher performance

• CSR Direct Connect 1 Gig and Multi-Gig

• Instances used for securing AWS Direct Connect circuits

CSR 1000V Licensing for AWSTwo Options…

AWS Marketplace Billing

• Provision hourly or annually billed CSR instances from AWS Marketplace

• Pay AWS for basic instance-type usage AND fees for CSR usage

• AWS pays Cisco for CSR usage fees they collect. You pay Cisco nothing directly.

• No license file to manage or install

• Choose EC2 instance type based on performance requirement

Bring Your Own License “BYOL”

• Provision “BYOL” CSR instances from AWS Marketplace

• Only pay AWS for basic instance-type fees

• Purchase desired license from Cisco or Cisco Partner

• Install purchased license onto “BYOL” version of CSR you provisioned from the AWS Marketplace

• Scalable from 10 Mbps up to 2.5 Gbps (AWS has a 2 Gbps throughput limit)

CSR 1000V Features Availability on AWS

Features in Green will work only over a Tunnel interface

Features in Red will not work in Amazon – limitations of AWS infrastructure (lack of L2 support, Multicast not supported)

Technology Package IOS-XE Features

IP Base

(formerly Standard)

Basic Networking: BGP, OSPF, EIGRP, RIP, ISIS, IPv6, GRE, VRF-LITE, NTP, QoS, PBR

Multicast: IGMP, PIM

High Availability: HSRP, VRRP, GLBP

Addressing: 802.1Q VLAN, EVC, NAT, DHCP, DNS

Basic Security: ACL, AAA, RADIUS, TACACS+

Management: IOS-XE CLI, SSH, Flexible NetFlow, SNMP, EEM, NETCONF

SEC

(formerly Advanced)

IP Base Plus…

Advanced Security: Zone Based Firewall, IPSec VPN, EZVPN, DMVPN, FlexVPN,

SSLVPN, GETVPN

High Availability: Box-to-box HA for FW and NAT

AppX

IP Base Plus…

Advanced Networking: L2TPv3, BFD, MPLS, VRF, VXLAN

Application Experience: WCCPv2, AppXNAV, NBAR2, AVC, IP SLA

Hybrid Cloud Connectivity: LISP, OTV, VPLS, EoMPLS

Subscriber Management: PTA, LNS, ISG

AX

(formerly Premium)ALL FEATURES

AWS VPC Networking 101

• VPC = Logically isolated network own IP range, routes, security policies, etc.

• VPCs’ IP ranges can overlap

• AWS Internet Gateway provides external access in/out of VPC

• Public IP NAT or Overload NAT for outbound traffic (No true public IPs)

• AWS VPC Peering can route between VPCs (with limitations)

• Security Options:

• Network ACLs protect subnets

• Security Groups protect instances

• AWS Route Tables route within the VPC (always first IP in subnet)

• All VPC subnets ALWAYS have a route to all other VPC subnets!

Internet Gateway

VPC1

CIDR 10.2.0.0/16

Subnet A

10.2.1.0/24

Subnet B

10.2.2.0/24

CSR 1000V Placement in the AWS Network

• NAT at the Internet GW

• Will break services that do not work over NAT, such as GET-VPN

• Tunnel source will be a private address

• Tunnel destination from the perspective of VPN peers will be a public address

• Assign EC2 elastic IP address so that address does not change if the CSR1K is shutdown

• Other VPCs see Elastic IP address unless using VPC peering

• CSR should be the default gateway for the application VMs

10.1.1.10

10.1.1.11

10.1.2.10

Gi2 Gi1

Maps to AWS Elastic IP

Internet IP 54.x.x.x

10.2.1.10

10.2.1.11

10.2.2.10

Gi2 Gi1

Interconnecting AWS VPCs Using the CSR 1000V

• No native AWS ability to connect two VPCs together, in same or different regions

• Even VPC Peering in AWS cannot span multiple regions

• Easily integrate multiple AWS regions into existing VPN topology as new sites

• Distribute applications across the globe, and keep the network simple

virtual private cloud

AWS cloud

US west region

virtual private cloud

US east region

Securing AWS Direct Connect Circuits

• Encrypts Direct Connect traffic, for corporate security policy or regulatory compliance

• Powered the Test Drive area at Amazon re:Invent 2014 Las Vegas

AWS CloudVirtual Private Cloud (VPC)

Corporate Data Center

VPC Public Subnet

VPC Private Subnets

Virtual

Private

Gateway

CSR 1000V

Enterprise Subnets

Cisco

ISR/ASR

AWS Direct

Connect

IPSec

CSR 1000V High Availability in AWS

• No virtual IP as with HSRP, since AWS doesn’t allow multicast

• AWS Route Tables for app subnets are re-pointed to opposite CSR

• Failure detection is automatic

• CSR itself calls AWS API to adjust AWS Route Table routes

VPC

CSR Subnet

App Subnet A

App Subnet B

Before HA Failover

After HA FailoverAWS EC2 Query API

CSR 1000V on Microsoft Azure Availability Timeline

• May 4th-8th 2015: Official solution launch at Microsoft Ignite conference

• Early June 2015: Early Field Trials with selected customers

• Late June 2015: CSR 1000V available on Azure Marketplace with Bring-Your-Own-License (BYOL)

• 2nd Half 2015: Launch of hourly billing in Azure Marketplace

Deployment & Management

CSR 1000V VM Instantiation Overview

• CSR 1000V VM Instances can be instantiated using the following methods (with possible hypervisor dependencies)

• VMWare ESXi: vSphere

• KVM: OpenStack

• Public cloud: Amazon Marketplace, Microsoft Azure

• Image Management

• VMWare ESXi: vCloud Director

• KVM: OpenStack Glance

• Public cloud: Amazon Marketplace, MS System Center

• An new Configuration OVF Tool (COT) is also provided for Cisco VMs

• License management

• Smart licensing

CSR 1000V VMWare ESXi VM Deployment

• CSR 1000V can be installed and edited under VMWare ESXi using the vSphere tools

• Deploying an OVA in vSphere involves several steps to navigate through the vSphere GUI

• Deployment using an .ISO format is also supported in vSphere

• Editing the properties of a VM can be done using vSphere vApp

• For more details, refer to http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/installesxi.html

http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/installesxi.html

vCloud Director Integration – CSR 1000V Bring-Up

Install CSR OVA and create template

in vCenter

Import into vCloudDirector Catalog

and create vApp

template

Build new vApp from template

and deploy

Add vAppto the

Catalog

MASTER TEMPLATE CREATED

vCloud Director Integration: Automated Deployment

Deploy vApp on

vDC using data file.

Map vCloud org networks to CSR

interfaces

Obtain “Location” of created instance

Modify “productSections” with

IOSparameters

Power on the CSR instance

Use CSR REST API

for additional

config

CSR READY

Scale out with automated deployment of multiple customized CSR instances using the vApp

Leverage vCloud Director REST API to configure IOS bootstrap parameters (IP address, credentials, etc)

CSR now ready to talk to “outside” world – network connectivity, credentials

Further per-tenant CSR customization using REST API calls – CSR REST API guide available

CSR 1000V KVM VM Deployment

• CSR 1000V is supported under KVM with RHEL, RHEV and Ubuntu

• Deploying a CSR 1000V manually in KVM involves going through several steps in the console

• Based on the VM Manager

• Installation can be done using OpenStack (XE 3.12+)

• Based on Horizon GUI

• Based on the Openstack CLI tool by• Creating a Nova flavor

• Creating a Glance image

• Using the Nova ‘boot’ command

• For more details, refer to http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/installkvm.html

http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/installkvm.html

OpenStack Conceptual Architecture

CSR 1000V and OpenStack

• CSR 1000V as Instance VM

• CSR 1000V replaces the default Neutron Router

• Need a CSR 1000V router service plugin and a cfg agent

• Multiple Plugins and using Service type framework for features

• Plugins for OpenStack Kilo

• Router-aaS

• FWaaS

• VPNaaS

REST Follows a Familiar Model

• REST = Representational State Transfer

• Stateless client-server model

• Uses URIs to identify resources of interest

• Uses JSON (JavaScript Object Notation)

• A light-weight, open standard, human readable data interchange format

• A more compact alternative to XML

• Benefits:

• Human readable

• Software friendly

• Large developer base

• Client libraries in many languages

HTTP GET

JSON/XML

Describes data in a

format applications

can understand

{"ids":[303776224, 19449911, 607032789,

86544242, 2506725913, 17631389],

"next_cursor":0, "next_cursor_str":"0",

"previous_cursor":0,

"previous_cursor_str":"0"}

REST API

REST API• Client Authentication

• Global

• Banner, Hostname, Domain name, User name / password, Logging, Import / export running config, SNM, etc…

• Licensing

• Call-home

• Smart Licensing

• DHCP server / relay

• Routing Protocols

• BGP / OSPF / EIGRP / static

• Display routing tables

• ACLs

• VRF-awareness: DNS, OSPF, BGP, EIGRP, Routing tables, NAT, DHCP, VPN

• QoS

• DNS

• NTP

• LISP

• Interface IP

• NAT (Static / dynamic)

• Zone-based firewall

• System Usage (Memory / CPU)

• VPN: sVTI, EzVPN

• See http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/restapi/restapi.html for details

• REST support typically lags behind official feature support on CSR 1000v

http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/restapi/restapi.html

CSR 1000V RESTful API Architecture

REST

APIOneP

SDK

OneP Python App

Client 1

Client 2

Client 3

C1 REST API calls

C2 REST API calls

C3 REST API calls

C2 SessionTIPC

(Transparent

Inter-Process

Commn.)

IOSd

OneP

AL

LXC Container

Web

Server

HTTPS

CSR 1000V

Performance & Scale

Factors Affecting CSR 1000V Performance• Hypervisor Type (VMware, KVM, Hyper-V, Citrix XenServer)

• Numbers of cores / vCPU allocated to a CSR instance

• Features (CEF, IPsec, NAT, FW, Features combination)

• CPU type and settings

• Host processor clock speeds (GHz); Processor/chipset cache sizes – L1, L2, L3

• Hyper-Threading

• Processor Affinity

• BIOS settings (power mgmt.)

• I/O model and settings

• Para-virtualized drivers (default)

• Cisco VM-FEX; SR-IOV (Single Root I/O Virtualization)

• Definition of Non-drop (NDR) rate

• ‘0 packet loss’ or ‘5 packets lost’ or ‘0.01% packet loss’

• VM Oversubscription

Loss Rate Interpretation – Background

• Performance results vary depending on what acceptable frame loss is defined. Typical definitions for loss rates (FLR) range from

• Absolutely 0 packets lost -> Non-drop Rate

• 5 packets lost

• 0.01% of PPS lost

• Small relaxation of FLR definition can lead to significant higher throughput

• Typically FLR Test data reported for 5 packet loss (to account for warm up) with multiple consecutive 2 minute runs

• Unless stated otherwise

2vCPU: throughput

of 670 Mbps at

0.01% acceptable

traffic loss

2vCPU: throughput

of 384 Mbps at 0%

acceptable traffic

loss

Sample Data only!

Number of Packets Lost in Perspective• At high-speed link rates, number of packets that may be lost may be substantial

while still meeting the FLR Loss Tolerance

Maximum

Throughput at Line Rate

Total Dropped Packets during Trial Duration

Allowed by Loss Tolerance of 0.01%

Dropped Packet Rate allowed by Loss

Tolerance (PPS)

Physical Media

10 Mbps Ethernet 179 1

100 Mbps 1,786 15

GE 17,857 149

10GE 178,571 1,488

Maximum

Throughput at Line Rate

Total Dropped Packets during Trial Duration

Allowed by Loss Tolerance of 0.1%

Dropped Packet Rate allowed by Loss

Tolerance (PPS)

Physical Media

10 Mbps Ethernet 1,786 15

100 Mbps 17,857 149

GE 178,571 1,488

10GE 1,785,714 14,881

OversubscriptionHV

Sample Performance with Multiple VMs with VMFEX

1 vCPU VMs

Ag

gre

ga

te T

hro

ug

hp

ut M

bps

• B200 M2

• 12 Cores, 2.67 Ghz

• VM/FEX & Direct Path

• ESXI 5.1

• 1VM standalone ~ 220 Mbps

• IP Packets CEF IMIX

Hypervisor CPU

contention VM

Oversubscription

Near linear

performance

increase as

VMs are added

due to VM-Fex

with Direct Path

0%

100%

200%

300%

400%

500%

600%

700%

800%

1 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32

KVM Performance Tuning Recommendations

• Use a Direct path I/O technology (SR-IOV w/ PCIe pass-through) with CPU tuning below! Otherwise the following configurations are recommended:

Tuning

Recommendation

Details / Commands Tuning

Disable Hyperthreading Can be done in BIOS CPU

Pin vCPUs ‘sudo virsh vcpupin test 0 6’ CPU

Pin vHost processes ‘sudo taskset -pc 4 <process Number>’,

Where <process Number> is found using ‘ps -ef | grep vhost’

I/O

Change vnet txqueue

length to 4000

Default tx queue length is 500

‘sudo ifconfig vnet1 txqueuelen 4000’

I/O

Turn off TSO, GSO,

RSO,

‘ethtool -K vnet1 tso off gso off gro off’ I/O

NOTE: these settings may impact the number of VMs that can be instantiated on a server / blade

VM-FEX performance (ESXi – IOS-XE 3.15, IMIX)

Test ParametersVM-FEX / VM Direct Path enabled. Hardware: Cisco UCSC-C240-M3S CPU: Intel Xeon E5-2643 v2 @ 3.5 GHz

20 Gbps+

Performance with

Large Packets

0

1

2

3

4

5

6

7

8

CEF ACL NAT FW Ipsec

High Performance with VM-FEX

1vCPU 2vCPU 4vCPU 8vCPU

CSR Performance and Scale (IOS-XE 3.15)

Feature

Throughput (Mbps)

IMIX

2 vCPU 4 vCPU

CEF 2929 2216

Firewall 2681 2219

IPSec (SHA, AES) 587 839

FW + NAT 1417 1377

FW + HQoS + NAT 1213 1271

FW + NAT + IPSec + QoS 361 511

Test parameters

0.01 % pkt. loss, vSwitch, UCS server with Intel Xeon CPU @ 3.5 GHz, ESXi 5.5

Feature Scale

NAT 44 450K

Firewall 256K

IPsec 1200

IPv4 routes 600K

IPv4 ACEs / system 60000

vRR25M IPv4 Routes (w/

16GB)

Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.

• Complete your session surveys though the Cisco Live mobile app or your computer on Cisco Live Connect.

Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Table Topics

• Meet the Engineer 1:1 meetings

• Related sessions

Thank you

cisco's cloud services extending the...

Documents