Building a Dynamic and Flexible Exchange Architecture

B S Nagarajan

Senior Technology Consultant

6th November, 2008

Agenda

� What is new in Exchange 2007?

� Why Virtualize Exchange?

� Sizing guidelines

� Eat your own Dog Food

� Customer references

State of the Art – Exchange Architectural Changes

Exchange 2003

32-bit Windows

900MB database cache

4Kb block size

Requires high-end storage

Storage is common pain pointSmall, fast, expensive drives to provide required IOPS.

Primitive roles

Front-End, Back-End

Exchange 2007

64-bit Windows

Multi-GB database cache

8Kb block size

50% reduction in disk I/O

Affordable storage

iSCSI, DAS

Eliminates storage pain point

More granular roles

More flexible deployment topology

���������� ������

������������� �������������

• Server role is a logical concept used to organize Exchange 2007 services and features across one or more servers.

• While Exchange 2003 provided primitive server roles called Back End server and Front End server, Exchange 2007 has more granular divisions.

• Dividing Exchange features among several server roles has advantages:

• More flexible deployment topology

• Better hardware utilization and scalability

• Easy to maintain

���������� �����

������������� �������������

• Mailbox (MB): Host mailbox and public folder data. Provides MAPI access for Outlook clients. If Clustered Mailbox role is selected, other server roles cannot be combined on the same physical server.

• Client Access (CA): Provides other mailbox server protocol access apart from MAPI. Similar to Exchange 2003 FrontEnd server, it enables user to use an Internet browser (OWA), 3rd party mail client (POP3/IMAP4) and mobile device (ActiveSync) to access their mailbox.

• Unified Message (UM): This role enables end users to access their mailbox, address book, and calendar using telephone and voice. IP-PBX or VoIP gateway needs to be installed and configured to facilitate much of the functionality of this server role.

• Hub Transport (HT): The Hub Transport role handles mails by routing them to next hop: another Hub Transport server, Edge server or mailbox server.

• Edge Transport (ET): The last hop of outgoing mail and first hop of incoming mail, acting as a "smart host" and usually deployed in a perimeter network, Uses ADAM with one-way sync from AD, does not require Active Directory to function once synced, can function with limited access to the corporate network for increased security. Cannot mix with other roles in same box.

���������� ���������������������� ������

Exchange 2007 Server RolesExchange 2007 Server Roles

Enterprise NetworkOtherSMTP

Servers

Mailbox

Routing Hygiene Routing Policy

Voice Messaging

Client Access

PBX or

VoIP

Public Folders

Fax

ApplicationsOWA

ProtocolsActiveSync, POP,

IMAP, RPC / HTTP …

ProgrammabilityWeb services,

Web parts

Unified Messaging

EdgeTransport

HubTransport

Mailbox

INTERNET

State of the Art – Server Hardware Changes

Exchange 2007 requires 64-bit hardware

256GB+ physical RAM

Multi-core (2/4/6 cores)

Hardware-assisted virtualization

Exchange 2007 workload more efficient

Virtualization to unlock full power of this hardware

State of the Art – ESX Server Changes

Increased guest OS memory (64GB)Increased physical RAM on ESX (256GB)Network improvements lower CPU utilizationNUMA optimizations improve multiple VM performanceImproved storage efficiencyAnd many more improvementsESX Server 3.5 is ready for Exchange

Exchange 2007 Performance on VI3

PerformanceMyth: Exchange performs poorly in a VM especially in terms of disk I/O.

Reality: Numerous performance studies have been conducted with our server and storage partners:

HP Performance Testing -http://www.vmware.com/files/pdf/exchange_hp_performance_wp.pdf

Dell EMC Performance Testing -http://www.vmware.com/files/pdf/exchange_ms_virtualizing_wp.pdf

IBM Performance Testing -http://www.vmware.com/company/news/releases/ibm_exchange_vmworld.html

Summary of findingsNegligible I/O differences running in a VM…some tests were slightly better than physical.

Slightly higher processor utilization (5 – 10%) in a virtual machine.

8K mailboxes

Native VMware Infrastructure

8 VM

16K mailboxes

16 core

128 GB 16 core

128 GB

Record Capacity for Exchange 2007

How many mailboxes can ESX Server support on a single server?

Is storage the limit?No. Exchange 2007 on Windows Server makes excellent use of correctly configured storage.

Is CPU the limit?Possibly. The Mailbox Server’s recommended maximum is eight cores.

http://technet.microsoft.com/en-us/library/aa998874(EXCHG.80).aspx

Is memory the limit?Possibly. The maximum recommended memory allocation is 32G per server

http://www.microsoft.com/technet/prodtechnol/exchange/2007/plan/hardware.mspx?wt.svl=sysreqs

Maximum mailboxes: 8,000Eight cores at recommended 1,000 mailboxes per core

This results in providing 3.75MB/user at the server; this is in the middle of the 2-5MB/user recommendation

���������� �������

������������� ��������� ���������

Exchange 2007 server role Minimum Recommended Maximum

Edge Transport 1 x processor core

2 x processor cores

4 x processor cores

Hub Transport 1 x processor core

4 x processor cores

8 x processor cores

Client Access 1 x processor core

4 x processor cores

4 x processor cores

Unified Messaging 1 x processor core

4 x processor cores

4 x processor cores

Mailbox 1 x processor core

4 x processor cores

8 x processor cores

All-in-one server roles (except ET)

1 x processor core

4 x processor cores

4 x processor cores

���������� �������

������������� ��������� ������

Exchange 2007 server role Minimum per

server

Recommended Maximum per

server

Edge Transport 2 GB 1 GB per core (2 GB minimum)

16 GB

Hub Transport 2 GB 1 GB per core (2 GB minimum)

16 GB

Client Access 2 GB 1 GB per core (2 GB minimum)

8 GB

Unified Messaging 2 GB 1 GB per core

(2 GB minimum)

4 GB

Mailbox 2 + n GB 2 + n GB 32 GB

All-in-one server roles (except ET)

2 + n GB 4 + n GB 8 GB

Building Block Performance

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

1 VM 2 VMs 3 VMs 4 VMs 5 VMs 6 VMs 7 VMs 8 VMs

Virtual Machines

Mai

lbox

es

0

100

200

300

400

500

600

700

800

900

1000

Late

ncy

(ms)

Mailboxes

SendMailLatency

Record-setting Server Performance

Quick Sizing Discussion

Processor:

Light/Average: 1,000 users/core

Heavy/Very heavy: 500 users/core

Memory:

2GB base

2MB per “light” mailbox

3.5MB per “average” mailbox

5MB per “heavy/very heavy” mailbox

Sizing Examples

500 “heavy” users:1vCPU

2GB base + (500 x 5MB) = 5GB RAM

1,000 “heavy” users:2 vCPU

2GB base + (1,000 x 5MB) = 7GB

2,000 “heavy” users:4 vCPU

2GB base + (2,000 x 5MB) = 12GB

�������������������������������

• Single Copy Clusters (SCC) uses at least two servers and a shared disk storage to house the database, logfiles, and quorum.

• Local Continuous Replication (LCR) keeps an additional copy of the database and log files on the same server.

• Cluster Continuous Replication (CCR) keeps an additional copy of the database and log files on a different server. Both node must be in same subnet. Auto failover possible.

• Standby Continuous Replication (SCR) replicate Storage Group to a number of remote servers. Need SP1. Source can be an CCR,SCC, or Stand alone mailbox server. Manual failover only.

Native

Mailbox Mailbox

MSCCR

VMware Infrastructure

Network Load Balancing

Edge Hub Client Mailbox

Simpler, Cost-Efficient Availability for Exchange 2007

Edge Hub Client

MSLCR

VMware HA & VMotion

SimpleEliminate complexity of MS clusteringOne HA solution for all Exchange roles

Cost-effectiveWindows and Exchange Standard EditionNo dedicated standby servers

ReliableVMware HA for unplanned downtimeVMotion for planned downtimeLCR for data replication

Improve Testing Efficiency with Snapshots and Clones

Exchange ExchangeBlackBerry

ServerBlackBerry

DB

ProductionTest

Archive for Fast Roll-back

Exact copy of production

12

Run more tests faster

3

Move changes into production

4

> Faster testing> More accurate testing on exact production copy> Lower cost testing infrastructure

VMware Exchange Deployment

Business and User Characteristics

� Business Type: Software Company

� Typically 1 million internal / 100k external / 40k spam messages per day� Primary datacenter in Palo Alto

� No Business Continuity or Disaster Recovery capabilities

� Single Active Directory site

� Production Mailboxes: ~9000

� Mailbox profiles� Unlimited mailbox sizes for users (largest ~54GB)

� Total mailbox data: 10TB

� Average mailbox size: ~1GB

� Average IOP rating per user: 2.25

Exchange 2007 Architecture

� 3 physical locations (Multi-Site configuration)� Primary datacenters in Palo Alto and Santa Clara

� Fail-over between primary datacenters

� Second Palo Alto site for file share witness

� 1 Gig connection between facilities

� Mailboxes (9,000 and rapidly growing)� Extra-large or non-quota’d mailboxes (>2 GB): ~2000

� Standard mailboxes (1-2 GB): ~7000

� Mailbox profiles� 1 and 2 GB mailbox quotas for regular mailboxes

� 3, 4 and 5 GB mailbox quotas for extra-large mailboxes

� Average 1,200 mailboxes per VM

� No mailbox sizes greater than 5GB

Exchange 2007 ArchitectureSystem-Wide� Active Directory Windows Server 2003 R2 x64 Enterprise Edition (Multi-Site configuration)

� Exchange 2007 Enterprise Edition

� Clustered Continuous Replication (CCR) over stretched VLAN used for database replication

� 1 GB connection between sites (Bandwidth is tunable depending on requirements)

� 11 application-aware (2 –node) High Availability VMs using CCR

� 9000 mailboxes transitioned to the Exchange Server 2007 environment

� Project completion date: 30-June-2008

� Entire Exchange infrastructure is virtualized using VMware Infrastructure 3

Palo Alto Datacenter• 17 Exchange Server VMs

• 6 Hub/CAS

• 11 Mailbox VMs

• Fiber Channel SAN

• 2 EMC Networker storage node VMs

Santa Clara Datacenter• 14 Exchange Server VMs

• 3 Hub/CAS (proxy and transport only)

• 11 Mailbox VMs

• Fiber Channel SAN

• 2 EMC Networker storage node VMs

• 2 Windows 2003 Std 32 bit DC/GC VMs

Other Virtualized Exchange Customers

� University of Plymouth, 50K mailboxes

� University of Texas, Brownsville, 20K mailboxes

� VIA Health / Rochester General, 5K mailboxes

� Branch of US govt, 750K mailboxes

� Atrenta, Delhi

� Chitale diary, Pune

� Macawber Beekay, Delhi

� Helix Financials, Delhi

“The university has virtualized 50,000 Exchange 2007 mailboxes on VMware Infrastructure. We not only have a more manageable and flexible Exchange

environment, but we have replaced Microsoft clustering with VMware’s built-in high availability solutions such as HA and VMotion. We couldn’t be happier with the uptime

and performance of our Exchange implementation on VMware. VMware technology works for small companies all the way up to massive financial institutions. And clearly,

it has worked for us.”

Adrian JaneHead of Faculty Support and StrategyUniversity of Plymouth

University of Plymouth

Performance

Cost

Exchange Performs Well on VMwareIO is not an issueScales out better on shared hardware

Reduce infrastructure costsConsolidate Exchange server rolesEliminate need for dedicated standby servers

ManagementDynamic Management

Streamline testing & troubleshooting with snapshots and clonesProvision in minutesScale dynamically and right-size infrastructureAchieve high availability without complexity of clustering

Summary - Why Run Exchange on VMware

Microsoft SVVP Support and LicensingSupport

Questions and, possibly, answers

bsn@vmware.com