net1191bu multisite networking and security with cross · pdf filecentralized management of...

Humair Ahmed, VMware NSBUTwitter: @Humair_Ahmed

Kent Munson, F5 NetworksTwitter: @Kent_Munson

NET1191BU

#VMworld #NET1191BU

Multisite Networking and Security with Cross-VC NSX - Part 2

VMworld 2017 Content: Not fo

r publication or distri

bution

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

2#NET1191BU CONFIDENTIAL



bution

1 Multisite Design Fundamentals with Cross-VC NSX

2 Cross-VC NSX Networking Design

3 Cross-VC NSX Security Design

4 3rd Party Services Design

5 NSX Component / Site Recovery with Demo

6 Demo – Cross-VC NSX and 3rd Party Services

7 Summary / Q&A

Agenda

#NET1191BU CONFIDENTIAL 3

Multisite Networking and Security with

Cross-VC NSX – Part 1NET1190BU

Disaster Recovery Solutions with NSXNET1188BU



bution

Multi-site Options with NSX

4

https://communities.vmware.com/docs/DOC-32552

When to use a particular solution depends on

the following factors:

• Datacenter distances: Geo / Metro

• Administrative boundaries: are both

sites managed by common organization

• Infrastructure considerations:

Network MTU

Latency

Storage type

vSphere version



bution

https://communities.vmware.com/docs/DOC-32552

General Enhancements

Multiple UDFW Sections

ApplyTo can use Universal SGs

* CDO Mode

New Support for Active-Standby Use Cases (DR)

Universal Security Tags

Universal Security Groups using Universal Security Tags

Universal Security Groups using VM Name

Benefits:

- Benefits of Logical Networking and Security

- Local Egress

- Metro Storage Cluster Deployments Possible

- Future-proofing - planning for NSX Logical

Networking and Security across multiple vCenters

Benefits:

- NSX Logical Networking and Security Across

vCenter Domains:

* Workload Mobility across vCenters

* Resource Pooling across vCenters

* Multi-site Security across vCenters

* Disaster Recovery

- Local Egress

Cross-VC NSX Multi-site Deployment Models

#NET1191BU CONFIDENTIAL



bution

Multisite Design Fundamentals

ULS - VNI 7000

ULS - VNI 8000

UDLR

ULS - VNI 9000

Mgmt ClusterCompute Cluster

Edge Cluster

Mgmt Cluster Compute Cluster

Edge Cluster

Local Storage Local Storage<=150ms

Edge

VDS

Compute

VDS

Mgmt

VDSMgmt

VDS

Compute

VDS

Edge

VDS

Site 1 Site 2

Primary SecondaryUniversalControl VM

UniversalControl VM

ESGs ESGs

UDFW

Universal Transport Zone



bution

Multisite Design Fundamentals

7

ULS - VNI 7000

ULS - VNI 8000

UDLR

ULS - VNI 9000

Mgmt–Edge–Compute Cluster


Mgmt-Edge-Compute VDS

Site 1 Site 2

Primary SecondaryUniversalControl VM

UniversalControl VM

ESGs ESGs


Mgmt–Edge–Compute Cluster

Mgmt-Edge-Compute VDSUDFW



bution

Platform Services Controller (PSC)

8

Although VMware NSX will work with both the embedded

and external PSC, it is recommended to use the external

PSC as it allows for:

1.) Centralized management of NSX with Enhanced Link

Mode

2.) Cross-VC vMotion from vSphere Web Client UI

The embedded PSC used across multiple vCenter

servers design model is deprecated and will be

unsupported in the future



bution

From Local Deployments to Multi-site with Cross-VC NSX

9

LS - VNI 7000LS - VNI 8000

DLR

LS - VNI 9000

Mgmt ClusterCompute Cluster

Edge Cluster

Mgmt Cluster Compute Cluster

Edge Cluster


Edge

VDS

Compute

VDS

Mgmt

VDSMgmt

VDS

Compute

VDS

Edge

VDS

Site 1 Site 2

Standalone Standalone

DLRControl VM

DLRControl VM

ESGs ESGs

DFW

Local Transport Zone Local Transport Zone

Primary Secondary

UniversalControl VM

DLR

LS - VNI 7000LS - VNI 8000LS - VNI 9000

LS - VNI 20000LS - VNI 30000LS - VNI 40000

DFW


UDLR

UDFW

ULS - VNI 70000ULS - VNI 80000ULS - VNI 90000

UniversalControl VM



bution

Leveraging NSX for Multisite to Cloud


DR to a cloud environment

Active - stand-by model

WEBAPPDB

APPAPP APP

Stand-by

L2 over L3 via

Cross-VC NSX

Active

Direct Connectivity

WEBAPPDB

VMware Cloud Provider

APPAPP APP



bution





5 NSX Component / Site Recovery


7 Summary / Q&A

Agenda




bution

Leveraging Cross-VC NSX for DR


DR to another data center

APP

Active Stand-byActive - stand-by model

ULS - VNI 7000

ULS - VNI 8000

UDLR

APP

UDFW

ULS - VNI 9000

APP

APP

APP

APP

WEB

APP

DB

APP

APP

APP



bution

Multi-site, Multi-vCenter, Active-Passive Site Egress

Route Updates




bution

Active - Standby Model: Bi-directional DR


Active Stand-by

Active - stand-by model

UDLR

UDFW

APPAPP

WEBAPPDB

APPAPP APP APP

WEBAPPDB APP APP APPAPPAPP APP

UDLR

ActiveStand-by



bution

Multisite with NSX: Active - Active Model


Application active on both sides

APP

Active ActiveActive - Active Model

ULS - VNI 7000

ULS - VNI 8000

UDLR

APP

UDFW

ULS - VNI 9000

WEB

APP

DB

APP

APP

APPAPP



bution

Route Updateswith Locale ID


Peering - OSPF / BGP Peering – BGP / OSPF


Route Updates

with Locale ID


Multi-site, Multi-vCenter, Active-Active Site Egress



bution

Multi-site, Multi-vCenter, Active-Active Site Egress

17

GSLB GSLB/32 Host Route Injection /32 Host Route Injection

SLB SLB

Application Pool Application Pool




bution

Flexibility with Multi-site with Cross-VC

18

Tenant 1: Active/Passive Site Egress via Dynamic Routing


Tenant 3: Active/Active Site Egress via Local Egress



bution

Example Deployment

19



bution

Example Deployment




bution

Communicating with Workloads on the Physical Network

21



bution







7 Summary / Q&A

Agenda




bution

Cross-VC NSX – Multi-site Security Design

The Universal section of the DFW supports the following network and security objects:

• Universal IP Sets

• Universal Mac Sets

• Universal Security Groups

• Universal Services

• Universal Service Groups

• Universal Security Tags (Static Inclusion)

• VM Name (Dynamic Inclusion)


Universal Network and Security Grouping Objects



bution

24



Tenant 3: Active/Active Site Egress via Local Egress

Entire application must be at site 1

Or entire application must be at site 2

General Enhancements

Multiple UDFW Sections

ApplyTo can use Universal SGs

* CDO Mode

New Support for Active-Standby Use Cases (DR)

Universal Security Tags

Universal Security Groups using Universal Security Tags

Universal Security Groups using VM Name

Cross-VC NSX – Multi-site Security Design



bution

In NSX 6.3, ApplyTo now also supports Universal Security Groups with new matching criteria:

Since application always occurs locally, can be used in Active/Standby or Active/Active Scenario

- VM Name (Dynamic)

- Security Tag (Static)

If Active/Standby deployment (No Cross-VC Traffic Flows):

- Source and Destination fields can use Universal Security Groups with matching criteria of Universal Security Tags and VM Name

If Active/Active deployment (Cross-VC Traffic Flows):

- Source and Destination fields must use Universal IP Sets or Universal Security Groups with Universal IP Sets membership

Apply to Design Considerations




bution

26

A Universal Security Tag is universal in the sense that it is synced to and exists on the secondary NSX Managers.

It’s automatically applied to a VM on the secondary site based on the VM the tag is applied to on the primary site and the unique selection criteria selected.

VMs with same Instance UUID vMotioned or recovered at Site 2 via SRM

VM Instance UUID Used to Apply Tags to Correct VMs

VMs with WEB Security Tag Cannot Communicate with VMs with DB Security Tags

https://[vCenter IP Address]/mob

Universal Security Tags Design Considerations



bution







7 Summary / Q&A

Agenda




bution

VMware NSX + F5 Networks for Active/Active Designs




bution

More Sites, More problems


SECURITY AVAILABILITY PERFORMANCE

ADC

FirewallApplication Security

Identity and Access

DDoSProtection

Local LoadBalancing

Global Load Balancing

Application Performance

Secure Web

Gateway

Application

Proxies

Global Load Balancing



bution

30

Site1–PaloAlto,CA Site2–SanJose,CA

Site1NSXManager1

Primary

Site2NSXManager2

Secondary

vCenter1 vCenter2

Universal

Controller

Cluster

CompueCluster1 CompueCluster2 EdgeCluster

MgmtvCenter


UniversalTransportZone

UniversalDistributedFirewall(UDFW)

ComputeVDS EdgeVDS ComputeVDS EdgeVDS

UniversalDistributedLogicalRouter(UDLR)

UniversalTransit:172.39.39.0/28

.1 .2

Universal

ControlVM

.14

VLAN279

10.100.9.2/28VLAN280

10.100.11.2/28

VLAN379

10.200.9.2/28VLAN380

10.200.11.2/28

.1 .1.1 .1

ESXi1-1:10.100.0.50/24

ESXi1-2:10.100.0.51/24ESXi1-3:10.100.0.52/24 ESXi1-4:10.100.1.51/24

ESXi1-5:10.100.1.52/24

ESXi1-6:10.100.1.53/24

ESXi2-1:10.200.0.50/24

ESXi2-2:10.200.0.51/24ESX2-3:10.200.0.52/24 ESXi2-4:10.200.1.51/24

ESXi2-5:10.200.1.52/24

ESXi1-6:10.200.1.53/24

UniversalWeb2:172.20.8.0/24

.1 .2

UniversalApp2:172.20.9.0/24

UniversalDB2:172.20.10.0/24

.1

.1

UniversalWeb:172.20.1.0/24

UniversalApp:172.20.2.0/24

UniversalDB:172.20.3.0/24

.254 .254 .254.254 .254.254

.1

.1

.1

SummaryRoute:

172.20.0.0/20

10.100.1.71/2410.100.1.72/24 10.200.1.71/2410.200.1.72/24

10.100.1.73-74/24

Cluster1 Cluster2

iBGP

BGPWeight:60

iBGP

BGPWeight:30

eBGPeBGP

Laptop

1. DNS Request

2. DNS Return IP in PA or SJ

3. Client Connects to ESG VIP

4. ESG LBs to application



bution

31

Site1–PaloAlto,CA Site2–SanJose,CA

Site1NSXManager1

Primary

Site2NSXManager2

Secondary

vCenter1 vCenter2

Universal

Controller

Cluster


MgmtvCenter




ComputeVDS EdgeVDS ComputeVDS EdgeVDS

UniversalDistributedLogicalRouter(UDLR)

UniversalTransit:172.39.39.0/28

.1 .2

Universal

ControlVM

.14

VLAN279

10.100.9.2/28VLAN280

10.100.11.2/28

VLAN379

10.200.9.2/28VLAN380

10.200.11.2/28

.1 .1.1 .1

ESXi1-1:10.100.0.50/24

ESXi1-2:10.100.0.51/24ESXi1-3:10.100.0.52/24 ESXi1-4:10.100.1.51/24

ESXi1-5:10.100.1.52/24

ESXi1-6:10.100.1.53/24

ESXi2-1:10.200.0.50/24

ESXi2-2:10.200.0.51/24ESX2-3:10.200.0.52/24 ESXi2-4:10.200.1.51/24

ESXi2-5:10.200.1.52/24

ESXi1-6:10.200.1.53/24

UniversalWeb2:172.20.8.0/24

.1 .2

UniversalApp2:172.20.9.0/24

UniversalDB2:172.20.10.0/24

.1

.1

UniversalWeb:172.20.1.0/24

UniversalApp:172.20.2.0/24

UniversalDB:172.20.3.0/24

.254 .254 .254.254 .254.254

.1

.1

.1

SummaryRoute:

172.20.0.0/20

10.100.1.71/2410.100.1.72/24 10.200.1.71/2410.200.1.72/24

10.100.1.73-74/24

Cluster1 Cluster2

iBGP

BGPWeight:60

iBGP

BGPWeight:30

eBGPeBGP

Mgmt:10.200.1.80 Mgmt:10.200.1.81Internal(Web):172.20.8.248 Internal(Web):172.20.8.249

HA:172.90.90.2/30

InternalFloa?ngIP(Web):

172.20.8.250

ExternalFloa?ngIP(Web):

10.200.9.14

External(Edge):10.200.9.12 External(Edge):10.200.9.13

Mgmt:10.100.1.80/24 Mgmt:10.100.1.81Internal(Web):172.20.8.251 Internal(Web):172.20.8.252

HA:172.80.80.1/30 HA:172.80.80.2/30

InternalFloa?ngIP(Web):

172.20.8.253

ExternalFloa?ngIP(Web):

10.100.9.14

External(Edge):10.100.9.12 External(Edge):10.100.9.13

[BIG-IP DNS VE]

Mgmt:10.114.223.75 Dataplane:10.100.1.190

[BIG-IP DNS VE]

Mgmt:10.114.223.78 Dataplane:10.200.1.190

Laptop

1. DNS Request

2. Intelligent DNS response

3. Client Connects to LTM VIP

4. LB to local application



bution



Laptop

[BIG-IP Local Traffic Manager VE] [BIG-IP Local Traffic Manager VE]

[BIG-IP DNS VE] [BIG-IP DNS VE]

Servers Servers Servers Servers




bution



Laptop

[BIG-IP DNS VE] [BIG-IP DNS VE]

Servers Servers Servers Servers


NSX ESG Load BalancerNSX ESG Load Balancer



bution

VMware NSX + Palo Alto Network for Advanced Multisite Security


3

4



bution

35

Multi-site Security Policy

Security Policy Management Layer

HA

Active Standby



bution







7 Summary / Q&A

Agenda




bution

37

Site Failure at Primary Site

1. Promote NSX Manager at Site 2

from Secondary to Primary

2. Redeploy Universal Controller Cluster

at Site 2

3. Update Controller State

4. Redeploy DLR Control VM at Site 2

Site Failure: NSX Component Recovery



bution

NSX Control Plane Resiliency


Controller Disconnected Mode (CDO Mode)



bution

• VTEP table for VNI 5000 includes Hosts 1, 2

and 3

• vMotion from Host 2 to Host 3: no change

• Powering on a new VM on Host 4 or

vMotioning a VM to Host 4 results in a table

add (Host 4)

Scenario



Normal Operation

Transport ZoneTransport Zone

Host 1 Host 2 Host 3 Host 4

Logical Switch 5000

NSX Controller Cluster VTEP Table – VNI 5000

Host 1

Host 2

Host 3

Host 4

39



bution


and 3

• Hosts become disconnected from Controllers

• vMotion from Host 2 to Host 3 works as

destination host is in VTEP Table

• Powering on a new VM on Host 4 or

vMotioning a VM to Host 4 results in data

plane disruption due to stale VTEP table

Scenario



Pre-NSX 6.3.2 Behavior on Control Plane Outage



Logical Switch 5000


Host 1

Host 2

Host 3



bution


and 3

• Hosts become disconnected from Controllers

• CDO mode: BUM traffic replicated to the

entire TZ via CDO Logical Switch

• No issues when powering on a VM on Host 4

or vMotioning a VM to Host 4 (Global VTEP

list used)

Scenario



NSX 6.3.2: Controller Disconnected Operation (CDO Mode)



Logical Switch 5000


Host 1

Host 2

Host 3

CDOCDO CDO CDO

BUM



bution







7 Summary / Q&A

Agenda






bution

43

Demo Placeholder




bution





5 NSX Component / Site Recovery with Demo


7 Summary / Q&A

Agenda






bution



bution

Humair Ahmed, VMware NSBU

Twitter: @Humair_Ahmed

Personal Website: HumairAhmed.com

Kent Munson, F5 Networks

Twitter: @Kent_Munson



bution

net1191bu multisite networking and security with cross · pdf filecentralized management of...

Documents