file based storage block storage vhdx guest clustering guest clustering with commodity storage...

The Cloud Service Provider ArchitecturePhilip MossFounding Partner NTTX

Allen Stewart Principal Group Program ManagerWindows Server and System Center Group

ATC-B212

Session Agenda• Focus on the Service• Service Provider Architecture Principles • Lets look at:• Compute• Storage• Network

• Making it Real (NTTX)• Cloud Service Provider in Action

Focus on the Service• Everyone is a Service Provider!!!!

• Building a virtualization environment for all servers• Do you host workloads or services• Manage and operate infrastructure on behalf of a business unit internally or

external customer• Charge, Showback, provide a service for a fee

• Beyond Virtualization to building a compelling IAAS service

• Service provider IAAS service requirements drove Windows Server 2012 development:• Storage Spaces • SMB 3.0• SMB Direct• Network Virtualization• Automation 2400 PowerShell cmdlets• Fabric – Consumer principles• Hyper-V Replica• 64 node clusters• CAU

Service Provider Architecture Principles • How do I build a shared multi-tenant IAAS service:

• Provides the best service at the Lowest Cost per VM• Scale – allow consumers of the service to scale up and down and run the most

demanding workloads• Leverages flexible hardware choices – commodity components• Super Tenant Experience – signup, galleries of services, billing, ease of managing

resources and consuming additional services• Automation-Automation-Automation – RunBooks, Recipes, Task automation,

repeatable processes, orchestration• Building Compelling Services – Additional monetization of infrastructure with

services:• Cloud Storage• Management• Disaster Recovery• Backup and Archiving• SAAS applications

• Interoperability – Run the workloads well that customers care about (Windows, Linux, etc)

• LiveSite Operations – Management tooling, visibility, capacity management, service operations

Compute

Service Provider Compute Challenges• How much compute do I need and have today vs.

tomorrow• Capacity management jujitsu

• Noisy neighbor management• Just in time compute vs. over allocation of compute

resources• Running demanding workloads without application

visibility• Windows – Linux support parity• Consumption based billing vs. monolithic (t-shirt

sized billing)

In Place Virtual Machine Migration• Live migrate virtual machines from

Windows Server 2012 to Windows Server 2012 R2

Faster Live Migration• Compression enables over 2x

improvement in live migration time• SMB Direct support allows for high-speed

live migrations over high-end networks

Online VHDX resize• Increase and decrease the size of virtual

hard disks – while the virtual machine is running

Live virtual machine export / clone• Export a running virtual machine, while it

is running• Export a snapshot of a virtual machine,

while the virtual machine continues to run

Storage QoS• Can restrict disk throughput for

overactive / disruptive virtual machines• Dynamically configurable while the

virtual machine is running

Linux Guest Support• Dynamic Memory - Linux VMs are now able to take

advantage of the increased density and resource usage efficiency of Dynamic Memory.  Memory ballooning and hot add are supported.

• Support for online snapshots - Linux VMs can be backed up live, with consistent memory and file systems. 

• Online resizing of VHDs - Linux VMs can have attached VHDs expanded while the VM is running.

• Synthetic 2D frame buffer driver - Improves display performance within graphical apps.

File Based StorageBlock Storage

Guest Clustering with Shared Virtual Disks

VHDX VHDX

Guest Clustering

Guest Clustering with commodity storage

Sharing VHDX files provides shared storage for Hyper-V Failover Clustering

Maintains separation between infrastructure and tenants

Virtual SAS

VM presented a shared virtual SAS disk

Appears as shared SAS disk to VM

Used for data disk only

Cluster Shared Volumes (CSV) on block storage

Scale-Out File Server for file based storage

Disaster Recovery

DR JourneyDR Platform

Enterprise Scale DR

2012 2012 R2

Design Principles for Enterprise scale DR Management Service

Simple Reliable

ExtensibleConsistent User

Experience

Hyper-V Replica Windows Server 2012 R2

• Variable replication frequency going as low as 30 sec (Near-sync)

• Multi-node replication support • Improved performance and scale

OffsiteDC1 DC2

Storage

Service Provider Storage Challenges• I need to delivery IOPS to any workload my

customers run• Does that cost a lot $$$$$$$

• Multi-Tenant aware storage • Manage the IOPS killer applications• Leverage flexible storage hardware options• Is this all DIY without clear architecture guidance and integration

• Build a storage service that hosts virtual machines:• Build a scalable storage service that supports backup, archiving and

hosting applications

• Does that file based storage stuff scale to service level

• How do I management this file based storage stuff

Storage ObjectivesBuilding upon the foundation in

Windows Server 2012 to further deliver:

Minimized Opex

Continuous Availability

Minimize $/TB & Capex

Maximize IOPS/$

Server 2012 R2 Feature Integration

Tiered Mirror Space

Tiered Mirror Space

Tiered Mirror Space

Dual Parity Space

Physical or Virtualized Workloads

Hyper-V Compute NodesHigh Speed Network

(10GbE/InfiniBand)

Unified Cluster Shared Volume

Namespace\\SRV\VDI_Mrktg \\SRV\Archive\\SRV\DB\\SRV\VDI_Dev

SMB Direct

60-baySAS Array

60-baySAS Array

60-baySAS Array

60-baySAS Array

Clustered File Servers with 10GbE/InfiniBand

9 6 G b p s S h a r e d S A S L i n k s 9 6 G b p s S h a r e d S A S L i n k s

SMB Direct

Clustered Storage Pool & Tiered

Storage Spaces

60-bay Shared SAS Arrays with mix of

SSDs and HDDs

Tiered Storage: Maximizing IOPS/$

• Utilize best characteristics of SSDs and HDDs in single storage space

• Provides long-term performance improvement

• Admins can assign files to specific storage tiers

Hyper-V Compute Nodes

Storage Space

HDD Tier

Cold Data

SSD Tier

Hot Data

400GB EMLC SAS SSD

4TB 7200RPM SAS HDD

Reads/Writes Accumulates Data Activity

Storage Service Management Pillars

Allocation and Assignment

Standards Based Management

VM, Host, and Cluster Storage Management

Extensive Device Support

Scalable Provisioning

Storage Monitoringand Capacity Trending

Scale-out File ServerBare Metal Provisioning

Pool, Volume, and File Share Classification

Rapid Provisioning Copy File, Diff Disks

Block and FileProvisioning

End to End Discovery and Mapping

Insight Flexibility Automation

Disaster Recovery

Storage Mangement API (SM-API)• New architecture with 10x faster enumerations• Added remoting and cluster-awareness for

managing Storage Spaces• Added support for new Storage Spaces features

(write-back cache, tiering, etc.) • VMM manages Storage Spaces and Scale-Out

File Server through SM-API

Storage Spaces compatible JBOD

SMI-S compliant subsystem

SMP based subsystem

Storage Spaces

Storage Management API (WMI)

Windows Server 2012Server Manager

ISV or Storage Vendor Applications

Windows Server now has a single standardized management

interface to manage storage

SMI-S compliant NAS

SMI-S compliant Fibre Channel switch

CIM Pass Through

System Center Virtual Machine Manager

VMM + Scale-out File Server and Storage Spaces

Scale-Out File Server ClustersStorage Spaces Virtualization and Resiliency

Hyper-V Clusters

SMB

Shared JBODStorage

Capacity Manageme

nt

Pool/volume/file share classification

File share ACL management

VM workload deployment to file shares

Scale-out File Server

Deployment

Bare metal deployment of file server

Creation of scale-out file server cluster

Adding/removing file server nodes

File share management

Spaces Provisionin

g

Discovery of physical spindles

Storage pool creation and deletion

Mirror and Parity Spaces creation and deletion

End-to-end management of both file and block based

storage with System Center Virtual Machine Manager

Storage and Fabric Classification

Storage classificatio

n

Classification of pools, volumes, and file shares

Classification aware Placement

Deployment of VMs to classification

SAN or Shared JBOD

Storage Pool

LUN (GOLD)

Hyper-V Clusters

Volume (GOLD)

Disk (GOLD)

GOLD

LUN (GOLD)

Volume (GOLD)

Disk (GOLD)

Storage and Fabric Classification

Storage classificatio

n

Classification of pools, volumes, and file shares

Classification aware Placement

Deployment of VMs to classification

Hyper-V Clusters

Volume (GOLD)

Disk

SILVER

Volume (SILVER)

Disk

GOLD

SAN or Shared JBOD

Storage Pool

LUN

Hyper-V Clusters

GOLD

LUN

Scale-out File Server

Volume (SILVER)

Disk

Volume

Disk

File Share (SILVER) File Share (GOLD)

SILVER

Storage and Fabric Classification

Storage classificatio

n

Classification of pools, volumes, and file shares

Classification aware Placement

Deployment of VMs to classification

Storage and Fabric Classification

Storage classificatio

n

Classification of pools, volumes, and file shares

Classification aware Placement

Deployment of VMs to classification

FC Fabric Classificatio

n

Identify fabric using friendly name

Classification aware Placement

FC SwitchesFC Switches

Fabric (SILVER)

Zone

Hyper-V Clusters

Virtual HBA

Virtual SAN

GOLD

Zone

Virtual HBA

Virtual SAN

Fabric (GOLD)SILVE

R

Networking

Service Provider Network Challenges• How can I make the promise of Network

virtualization real• Hardware gateway dependency

• One gateway per tenant (two for HA)• Limited routing capabilities• Manual steps for end-to-end provisioning• Lack of Internet NAT

Hybrid Cloud with WS2012Blue

Corp Site 1

Blue Corp Site

2

Green Corp Site

Orange Corp site1

Orange Corp site2

S2S Tunnel

S2S Tunnel

S2S Tunnel

S2S Tunnel

S2S Tunnel

Blue Virtual Network

Green Virtual Network

Orange Virtual

Network

BlueGW

GreenGW

OrangeGW

With Hyper-V Network Virtualization and on-premises Site-to-Site VPN on-premise resources seamlessly extended to the cloud

Internet Hoster’s datacenterRemote sites

Hybrid Cloud with WS2012 R2Blue

Corp Site 1

Blue Corp Site

2

Green Corp Site

Orange Corp site1

Orange Corp site2

Internet

Blue Virtual Network

Orange Virtual

Network

Hoster’s datacenterRemote sites

Green Virtual Network

Hybrid Cloud with WS2012 R2Blue

Corp Site 1

Blue Corp Site

2

Green Corp Site

Orange Corp site1

Orange Corp site2

S2S TunnelS2S Tunnel

S2S Tunnel

S2S Tunnel

S2S Tunnel

Internet

Blue Virtual Network

Green Virtual Network

Orange Virtual

Network

Hoster’s datacenterRemote sites

MultitenantGW

• Multitenant HNV/S2S gateway

Hybrid Cloud with WS2012 R2Blue

Corp Site 1

Blue Corp Site

2

Green Corp Site

Orange Corp site1

Orange Corp site2

S2S TunnelS2S Tunnel

S2S Tunnel

S2S Tunnel

S2S Tunnel

Internet

Blue Virtual Network

Green Virtual Network

Orange Virtual

Network

Hoster’s datacenterRemote sites

MultitenantGW

• Multitenant HNV/S2S gateway

• Guest clustering for HA

MultitenantGW

Active-Standby

Hybrid Cloud with WS2012 R2Blue

Corp Site 1

Blue Corp Site

2

Green Corp Site

Orange Corp site1

Orange Corp site2

S2S TunnelS2S Tunnel

S2S Tunnel

S2S Tunnel

S2S Tunnel

Internet

Blue Virtual Network

Green Virtual Network

Orange Virtual

Network

Hoster’s datacenterRemote sites

MultitenantGW

• Multitenant HNV/S2S gateway

• Guest clustering for HA

• BGP for dynamic routes update

MultitenantGW

Active-Standby

BGP

Hybrid Cloud with WS2012 R2Blue

Corp Site 1

Blue Corp Site

2

Green Corp Site

Orange Corp site1

Orange Corp site2

S2S TunnelS2S Tunnel

S2S Tunnel

S2S Tunnel

S2S Tunnel

Internet

Blue Virtual Network

Green Virtual Network

Orange Virtual

Network

Hoster’s datacenterRemote sites

MultitenantGW

• Multitenant HNV/S2S gateway

• Guest clustering for HA

• BGP for dynamic routes update

• Multitenant aware NAT for Internet access

MultitenantGW

Active-Standby

BGP

Hybrid Cloud with WS2012 R2Blue

Corp Site 1

Blue Corp Site

2

Green Corp Site

Orange Corp site1

Orange Corp site2

S2S TunnelS2S Tunnel

S2S Tunnel

S2S Tunnel

S2S Tunnel

Internet

Blue Virtual Network

Green Virtual Network

Orange Virtual

Network

Hoster’s datacenterRemote sites

MultitenantGW

• Multitenant HNV/S2S gateway

• Guest clustering for HA

• BGP for dynamic routes update

• Multitenant aware NAT for Internet access

• Multitenant remote access VPN dial in

MultitenantGW

Active-Standby

BGP

Making it RealCloud Service Provider in Action (NTTX)

Key Service Provider Drivers

Reducing cost to service.Providing a feature set

that meets our customers need.

Make Money

NTTX

Service Provider

Next generation managed services

Public Cloud (true multi-

tenant)Hybrid Cloud

Delivering the highest levels of user experience

Full multi-point High-Availability

Geo-location agnostic access

NTTX System Engineering Goals• 100% virtualisation• 100% automation• Support for multiple diverse workloads• Full end-to-end high-availability• Sub-system scale-out• Storage• Networking• Compute

• Cost to serve reduction• Removal of middleware• Hardware platform agnostic• Just in time hardware provisioning

Logical Architecture

Storage Spaces

Scale-out CA file-server

SMB Transport

Hyper- V Cluster – General Workloads Hyper-V Cluster - PVM #͛s (WARP)

Hyper-V Cluster - PVM #͛s (virtual GPU)

DC #͛s Exchange Lync RDSH

SQL DPM DHCP

RDS SharePoint WDS

DNS

Storage

Networking

Compute

DataCentre A

Fault-tolerant data storage

Highly-available data delivery platform

Data transport fabric

Virtualisation Compute Fabric(Hyper-Visor clusters)

Perimeter Security

DataCentre B

Fault-tolerant data storage

Highly-available data delivery platform

Data transport fabric

Virtualisation Compute Fabric(Hyper-Visor clusters)

Perimeter Security

Data replication

Datacentre Architecture

NTTX Architecture Choices

Doing Storage at NTTX• Storage Spaces• Windows Server as the storage controller• Cheap generic JBOD’s

• Continuous Availability• Full scale out• Removes the requirement for the SAN• Cheap but reliable

Doing Networking at NTTX• Let Windows do the work• Cheap networking (Ethernet)• Hyper-V over SMB• De-couple compute for storage• No iSCSI• No Fibre Channel

• SMB multi-channel • Teaming (switch agnostic)• QoS

Doing Compute at NTTX• Hyper-V 3.0• 64 node clusters• Inter-cluster live migration• SMB 3.0 support• Dynamic RAM• vGPU support• CSV Read Cache

NTTX Road AheadWindows Server 2012 R2

Big WinsPerformance and Scale Cost Reduction Service Improvements

Storage – R2• Tiering• Mix SSD and HDD• Dynamic movement of hot data into SSD “fast” tire

• De-Dup• CSV Support• “Hot” on open files

• Support for clusters on parity space• Storage QoS

Networking - Scale Out File Server / Clustering

• Intelligent connection redirection• SMB connections

• Mitigation of requirement for dedicated CSV network

• Intelligent node majority management

Compute - Hyper-V• Gen 2 VM’s• No more legacy NIC

• Shared VHDx guest clusters• No more software iSCSI

• Live Migration between 2012 and 2012 R2• Migration performance increase

Services – Remote Desktop Servcies• Remote App usability improvements• Transparency• Better disconnection experience

• vGPU performance and scale enhancements

Services - Hybrid Cloud• Multi-tenant RRAS gateway• Storage as a service• VM usage billing• BYOD support• DR as a service

Questions

Trustworthy Computing ResourcesTrustworthy Computing (TwC) is a long-term, collaborative effort to deliver more secure, private, and reliable computing experiences for everyone. Learn more at:http://microsoft.com/twc

Cloud Security Readiness ToolPass the Hash GuidanceData, Insights and Guidance (Security Intelligence Report, volume 14)

and more…

msdn

Resources for Developers

http://microsoft.com/msdn

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources

Sessions on Demand

http://channel9.msdn.com/Events/TechEd

Resources for IT Professionals

http://microsoft.com/technet

Evaluate this session

Scan this QR code to evaluate this session.

© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.