cloudstack collab talk

Making a case for distributed overlay-

based network virtualization

Ben CherianChief Strategy Officer@bencherianMidokura

So, you’re building a cloud?

Requirements

Horizontal scaling

1 2 3 4 5

1 New1

vs

Building blocks of an IaaS cloud

Cloud management system

Compute

Storage

Networking

Traditional networking devices scale up

Service interruptions

High churn, micro granularity

Limitations of VLANs

Traffic trombones

Human costs don’t scale

Additional Requirements

• ACLs• Stateful (L4) Firewall

Security Groups

• VPN IPSec

• BGP gateway• REST API• Integration with CMS

CloudStack OpenStack

• Multi-tenancy• L2 isolation• L3 routing

isolation VPC Like VRF (virtual

routing and forwarding)

• Scalable control plane

ARP, DHCP, ICMP

• NAT (Floating IP)

IaaS Cloud Networking Requirements

Typical Network Topology

IaaS Cloud Networking Requirements

Solution: Distributed overlay-based network virtualization

Use encapsulation to build a virtual network

Handle network intelligence / network state at the edge

Require less of the physical network

• Isolation not using VLANs IP encapsulation

• Decouple from physical network• Provisioning VM doesn’t change underlay

state• Underlay delivers to destination host IP• Use scalable IGP (iBGP, OSPF) to build multi-

path underlay• Inspired by VL2 from MSR

Edge to Edge IP Overlays

• Packet processing on x86 CPUs (at edge)– Intel DPDK facilitates packet processing– Number of cores in servers increasing fast

• Clos Networks (for underlay)– Spine and Leaf architecture with IP– Economical and high E-W bandwidth

• Merchant silicon (cheap IP switches)– Broadcom, Intel (Fulcrum Micro), Marvell– ODMs (Quanta, Accton) starting to sell directly– Switches are becoming just like Linux servers

• Optical intra-DC Networks

Market trends supporting overlay model

• Virtual L2 Distributed Switching

• Virtual L2 Isolation

• Virtual L3 Distributed Routing

• Virtual L3 Isolation

• L4 Services (Load Balancing, Firewall)

• NAT

• Access Control Lists (ACLs)

• Virtual port and device monitoring

• Restful API

• Web based management control panel

The MidoNet Solution

Logical Topology

Private IP Network

MN

MN

MN

Internet

BGPMulti

Homing

Physical Topology

MNVM

VM

MNVM

VM

MNVM

VM

BGPTo ISP3

BGPTo ISP2

BGPTo ISP1

vPort

ProviderVirtualRouter

Tenant AVirtualRouter

Tenant BVirtualRouter

VirtualSwitch A1

VirtualSwitch A2

VirtualSwitch B1

vPort

vPort

vPort

vPort

vPort

Network State Database

MN MN MN

Tunnel

The MidoNet Solution

• Distributed and scalable control plane Handle all control packets at local MidoNet agent

adjacent to VM

• Scalable and fault tolerant central database Stores virtual network configuration Dynamic network state

MAC learning, ARP cache, etc Cached at edges on demand

• All packet modifications at ingress One virtual hop

No travel through middle boxes Drop at ingress

MN

Packet

Encapsulated

Tunnel

Drop/Block

Ingress

The MidoNet Solution

Scale out model

• Scalable edge gateway interface to external networks– Multihomed BGP to ISP

• REST API and GUI

• Integration with popular open source cloud stacks– OpenStack

• Removes SPOF of network node• Scalable and fault tolerant NAT for floating IP• Implements security groups efficiently

– CloudStack (in progress)

The MidoNet Solution

• Currently have L2 integration

• Full integration is slated for Q1, 2013– L3 isolation (without VM / appliance)– Security groups (stateful firewall)– Floating IP (NAT)– Load balancing (L4)

CloudStack integration

Questions?

Slides: http://www.slideshare.net/midokura

Backup slides

Candidate Models

• Traditional network

• Centrally controlled OpenFlow based hop-by-hop switching fabric

• Edge to edge overlays

Traditional Netowrk

• Ethernet VLANs for L2 isolation 4096 limit VLANs will have large spanning trees terminating on many hosts High churn in switch control planes doing MAC learning non-stop Need MLAG for L2 multi-path

Vendor specific

• MPLS VPN?• VRFs for L3 isolation

Not scalable to cloud scale Expensive hardware Not fault tolerant

OpenFlow Fabric

• State in switches Proportional to virtual network state Need to update all switches in path when

provisioning Not scalable, not fast enough to update, no

atomicity of updates

• Not good for IaaS cloud virtual networking

Spine and Leaf Network Architecture

Deep OpenStack Integration• Quantum Plugin

– L2 isolation, of course

• Also…– L3 isolation (without VM / appliance)– Security groups (stateful firewall)– Floating IP (NAT)– Load balancing (L4)

37