storage, san and business continuity overview
TRANSCRIPT
Storage Systems and Business Continuity Overview
Alan McSweeney
April 11, 2023 2
Objectives
• To information on SAN storage options• To provide details on business continuity and
disaster recovery options
April 11, 2023 3
Agenda
• Types of Storage• Enabling Greater Resource Utilisation Through Storage System
Virtualisation• Business Continuity and Disaster Recovery• Systems Center Operations Manager (SCOM)• Managing Disk Based Backup Through Storage Virtualisation Single
Instance Storage (Deduplication)• Enabling greater Data Management Through Storage System SnapShots• Enabling Greater Application Resilience Through SnapShot Technologies• Enabling Greater Data Resilience Through Storage System Mirroring• Easing the Pain of Development Through SnapShot Cloning• Rapid Microsoft Exchange Recovery through Storage Systems
Technologies• Rapid Microsoft SQL Recovery through Storage Systems Technologies• Rapid Recovery of Oracle DB Through Storage Systems Technologies• Server Virtualisation and Storage• Storage Management and Business Continuity/Disaster Recovery• Storage Management and WAN
April 11, 2023 4
Types of Storage
• DAS• NAS• SAN
April 11, 2023 5
Direct Attached Storage (DAS)
• Directly attached to server• Internal or External• Cannot be shared with other servers
April 11, 2023 6
Network Attached Storage (NAS)
• Storage devices connected to Ethernet network• Can be shared among servers and users• Usually used in places of dedicated file servers• Not for database use (In the Microsoft World)
April 11, 2023 7
Storage Attached Network (SAN)
• Hosts attached via Fibre Channel Host Bus Adaptors
• Connect to storage system via Fibre Channel Switches
• Sees pre assigned storage as dedicated free space
• Desktops access storage on local server as normal
April 11, 2023 8
Storage Attached Network
April 11, 2023 9
What Differentiates NAS and SAN?
April 11, 2023 10
What Differentiates NAS and SAN?
Storage Protocols
April 11, 2023 11
What Differentiates NAS and SAN?
Storage Protocols
• File Level – NAS− Windows File System Share (With no Windows
Servers)− \\ServerName\ShareName
April 11, 2023 12
What Differentiates NAS and SAN?
Storage Protocols
• File Level – NAS− Windows File System Share (With no Windows
Servers)− \\ServerName\ShareName
• Block Level – SAN− Sees provisioned disk as its own drives and formats
accordingly. E.g. NTFS, EXT3− F:\Directory Structure
April 11, 2023 13
File Level
April 11, 2023 14
File Level
• CIFS− Common Internet File System− Predominantly Windows Environments
April 11, 2023 15
File Level
• CIFS− Common Internet File System− Predominantly Windows Environments
• NFS− Network File System− Non Windows Environments
• Unix, Linux, NetWare, VMware
April 11, 2023 16
Block Level
April 11, 2023 17
Block Level
• Fibre Channel− Uses Fibre Channel Switches
• FC-AL• 1Gb, 2Gb, 4Gb
April 11, 2023 18
Block Level
• Fibre Channel− Uses Fibre Channel Switches
• FC-AL• 1Gb, 2Gb, 4Gb
• iSCSI− Uses Ethernet Switches
• 1GB• 10Gb
April 11, 2023 19
Storage Options – Advantages and Disadvantages
April 11, 2023 20
DAS - Pros
• Inexpensive− Use of large capacity SCSI and SATA drives− No added expense for controllers
April 11, 2023 21
DAS - Pros
• Inexpensive− Use of large capacity SCSI and SATA drives− No added expense for controllers
• Performance− Dedicated disk array with various cache options
April 11, 2023 22
DAS - Pros
• Inexpensive− Use of large capacity SCSI and SATA drives− No added expense for controllers
• Performance− Dedicated disk array with various cache options
• Skill Levels− No new skill levels required to mange storage
April 11, 2023 23
DAS - Cons
• Captive Storage− Storage can only be used by one server
April 11, 2023 24
DAS - Cons
• Captive Storage− Storage can only be used by one server
• Performance− Disk Arrays may be limited to the number of drives
that can be used
April 11, 2023 25
DAS - Cons
• Captive Storage− Storage can only be used by one server
• Performance− Disk Arrays may be limited to the number of drives
that can be used− Backups can be slow and inconsistent
• Expense− Can be expensive in terms of wasted disk space.
April 11, 2023 26
NAS - Pros
April 11, 2023 27
NAS - Pros
• Can replace file servers and introduce enterprise resilience− Windows, Unix
April 11, 2023 28
NAS - Pros
• Can replace file servers and introduce enterprise resilience− Windows, Unix
• Easily expandable− From 36GB to over 0.5PB
April 11, 2023 29
NAS - Pros
• Can replace file servers and introduce enterprise resilience− Windows, Unix
• Easily expandable− From 36GB to over 0.5PB
• Cost Effective− Single Appliance replace multiple servers
April 11, 2023 30
NAS - Pros
• Can replace file servers and introduce enterprise resilience− Windows, Unix
• Easily expandable− From 36GB to over 0.5PB
• Cost Effective− Single Appliance replace multiple servers
• Ease of backup− Can backup all shares from NAS appliance
April 11, 2023 31
NAS - Cons
April 11, 2023 32
NAS - Cons
• Expense− Can be expensive relative to cost of single server
April 11, 2023 33
NAS - Cons
• Expense− Can be expensive relative to cost of single server
• Performance− Depending on protocol
April 11, 2023 34
NAS - Cons
• Expense− Can be expensive relative to cost of single server
• Performance− Depending on protocol
• Database Support− No support for MS SQL or MS Exchange
April 11, 2023 35
NAS - Cons
• Expense− Can be expensive relative to cost of single server
• Performance− Depending on protocol
• Database Support− No support for MS SQL or MS Exchange
• Skill Levels− May require new skill sets
April 11, 2023 36
SAN - Pros
April 11, 2023 37
SAN - Pros
• High Performance− IO/s− Disk Utilisation
April 11, 2023 38
SAN - Pros
• High Performance− IO/s− Disk Utilisation
• Resilience− SnapShots− Mirroring− Replication
April 11, 2023 39
SAN - Pros
• High Performance− IO/s− Disk Utilisation
• Resilience− SnapShots− Mirroring− Replication
• Scalability− Scales to PB
April 11, 2023 40
SAN - Cons
April 11, 2023 41
SAN - Cons
• Costs− Initial Capital Cost− Running Costs− Maintenance
April 11, 2023 42
SAN - Cons
• Costs− Initial Capital Cost− Running Costs− Maintenance
• Skill Sets− New skill sets will be required
April 11, 2023 43
SAN - Cons
• Costs− Initial Capital Cost− Running Costs− Maintenance
• Skill Sets− New skill sets will be required
• Compatibility− Most vendors require ‘Fork Lift’ upgrades
April 11, 2023 44
SAN - Cons
• Costs− Initial Capital Cost− Running Costs− Maintenance
• Skill Sets− New skill sets will be required
• Compatibility− Most vendors require ‘Fork Lift’ upgrades
• Business Risk− Lose the SAN and lose data from many servers− Maximum resilience is a must
April 11, 2023 45
Which Storage Solution is Right for Me?
April 11, 2023 46
NAS or SAN?
• Depends on Application requirements• Depends on User Requirements• Depends on Skill Budget
April 11, 2023 47
Why Not Both NAS and SAN
• Most organisations will benefit from both NAS and SAN
• NAS for file serving and low end applications• SAN for greater application performance, OLTP,
Exchange, SQL, Oracle• Can be expensive
− Use multiprotocol storage systems
April 11, 2023 48
Multiprotocol Storage
Windows ServerUNIX Server
GbE switch
Windows Server
CIFS NFS
iSCSI
FC fabric
FCP
April 11, 2023 49
Multiprotocol Storage Systems
• No physical boundaries between NAS and SAN• NAS protocols for file serving• SAN protocols for Application Performance• Bring enterprise functionality to NAS
environment− NAS data is no less important than SAN data
• Greater return on investment
April 11, 2023 50
SAN Basics
• SAN infrastructure (also called “fabric”) comprises the hardware, cabling and software components that allows data to move into and within the SAN − Server network cards (fibre channel HBAs or Ethernet
NICs) and switches
• A disk array is a centralised storage pool for servers
• Data from multiple servers is stored in dedicated areas called logical unit number (LUNs)
• Data can be protected against data loss in the event of multiple disk failures using RAID
April 11, 2023 51
What is RAID
April 11, 2023 52
What is RAID
• Redundant Array of Inexpensive Disks• Allows for single or multiple drive failure• Can increase read and write performance
− Depending on environment
• Can have an adverse affect on performance− Depending on environment
• Dependant on RAID controller
April 11, 2023 53
Multiple RAID Levels
April 11, 2023 54
Multiple RAID Levels
• RAID 0− No fault tolerance
April 11, 2023 55
Multiple RAID Levels
• RAID 1− Hardware Mirror
April 11, 2023 56
Multiple RAID Levels
• RAID 4− Single dedicated parity drive
April 11, 2023 57
Multiple RAID Levels
• RAID 5− Distributed parity
April 11, 2023 58
Multiple RAID Levels
RAID 6 (As it should be)− As RAID 4 but with two parity drives with separate
parity calculations. Also known as RAID Diagonal Parity, RAID DP
April 11, 2023 59
RAID 6 Overview (RAID DP)
• Description− Diagonal-Parity RAID – two parity drives per RAID group
• Benefits− 2000~4000X data protection compared to RAID 4 or 5− Protects against 3 modes of double disk failure
• Concurrent failure of any 2 disks (very rare)• 2 simultaneous disk uncorrectable errors (also very rare)• A failed disk and an uncorrectable error (most likely)
− Comparable operational cost to RAID 4• Equivalent performance for nearly all workloads• Equally low parity capacity overhead supported
− Less system impact during RAID reconstruction
April 11, 2023 60
Why is RAID-DP Needed?
• ‘Traditional’ single-parity-drive RAID group no longer provides enough protection− Reasonably-sized RAID groups (e.g. 8 drives) are exposed to data loss
during reconstruction• Larger disk drives• Disk drive uncorrectable (hard) error rate
• RAID 1 is too costly for widespread use− Mirroring doubles the cost of storage− Not affordable for all data
April 11, 2023 61
Six Disk “RAID-6” Array
{D D D D P DP
April 11, 2023 62
Simple RAID 4 Parity
3 1 2 3 9
{D D D D P DP
April 11, 2023 63
Add “Diagonal Parity”
31
2
1
11
3
1
22
1
3
31
22
95
8
7
7
1212
11{D D D D P DP
April 11, 2023 64
Fail One Drive
31
2
1
11
3
1
22
1
3
31
22
95
8
7
7
1212
11{D D D D P DP
7
April 11, 2023 65
Fail Second Drive
31
2
1
11
3
1
22
1
3
31
22
95
8
7
7
1212
11{D D D D P DP
7
April 11, 2023 66
Recalculate from Diagonal Parity
31
2
1
11
3
1
22
1
3
31
22
95
8
7
7
1212
11{D D D D P DP
7
April 11, 2023 67
Recalculate from Row Parity
31
2
1
11
3
1
22
1
3
31
22
95
8
7
7
1212
11{D D D D P DP
7
April 11, 2023 68
The rest of the block … diagonals everywhere
3121
1131
2213
3122
9587
7121211{
D D D D P DP
April 11, 2023 69
Business Continuity and Disaster Recovery
April 11, 2023 70
Specific Business Continuity and Disaster Recovery Requirements
• RTO – Recovery Time Objective− How quickly should critical services be restored
• RPO – Recovery Point Objective− From what point before system loss should data be available− How much data loss can be accommodated
1
2RTO
Systems Restored
System Loss
3Last System Backup/Copy
RPO
April 11, 2023 71
Options and Issues
• Virtualised infrastructure− Virtualise secondary and/or primary server
infrastructure
• Data replication software− DoubleTake− WANSync
• Hardware replication
April 11, 2023 72
Possible Core Architecture 1
April 11, 2023 73
Possible Core Architecture 2
1. Core server infrastructure virtualised for resilience and fault tolerance
2. Centralised server management and backup
3. SAN for primary data storage
4. Backup to disk for speed
5. Tape backup to LTO3 autoloader for high capacity
6. Two-way data replication
April 11, 2023 74
Data Backup and Recovery
1. Servers backed-up to low cost disk - fast backup and reduced backup window
2. Disk backup copied to tape - tape backup to LTO3 autoloader for high capacity and reduced manual intervention
3. Move tapes offsite
April 11, 2023 75
Resilience
• Virtual infrastructure in VMware HA (High Availability) Cluster
• Fault tolerant primary infrastructure
• Failing virtual servers automatically restarted
• Dynamic reallocation of resources
April 11, 2023 76
Disaster Recovery
• Failing servers can be recovered on other site
• Virtualised infrastructure will allow critical servers to run without the need for physical servers
• Virtualisation makes recovery easier – removes any hardware dependencies
April 11, 2023 77
Data Replication Options
• Option 1 – Direct server replication− Each server replicates to a backup server in the other
site
• Option 2 – Consolidated virtual server backup and replication of server images for recovery− Copies of virtual servers replicated to other site for
recovery
• Option 3 – Data replication− Replication of SAN data to other site
• Option 4 – Backup data replication− Replication of backup data to other site
• Each option has advantages and disadvantages
April 11, 2023 78
Option 1 – Direct Server Replication
• Install replication software (DoubleTake, Replistor, WANSync) on each server for replication
• Continuous replication of changed data
• Need active servers to receive replicated data
• Active servers can be virtual to reduce resource requirements
• Replication software cost of €3,500 per server
• Failing servers can be restored
• Minimal data loss
April 11, 2023 79
Option 2 – Consolidated Virtual Server Backup
• Use VCB feature of VMware to capture images of virtual machines
• Replicate image copies
• Recovery to last image copy
• Low bandwidth requirements
April 11, 2023 80
Option 3 – SAN Hardware Replication
• SAN replication at hardware level
• Very high bandwidth requirements - > 1 Gbps each way
• Not all SANs support hardware replication
• Very fast recovery• Can be an
expensive option
April 11, 2023 81
Option 4 – Replication of Backup Data
• Scripted replication of disk backup data
• Recovery to last backup
• Low bandwidth requirements
• Low cost option
April 11, 2023 82
Business Focus on Disaster Recovery
• Every year one out of 500 data centres will experience a severe disaster
• 43% of companies experiencing disasters never
re-open, and 29% close within two years• 93% of business that lost their data centre for
10 days went bankrupt within one year• 81% of CEOs indicated their company plans
would not be able to cope with a catastrophic event
April 11, 2023 83
DR Recovery Facility
Primary InfrastructureDesigned for
Resilience and Recoverability
ProcessesAnd
Procedures
OperationalDisaster
RecoveryAnd Business
ContinuityPlan
Components of Effective DR
April 11, 2023 84
Components of Effective DR
• DR Recovery Facility – this will be the second McNamara site
• Primary Infrastructure Designed for Recoverability – this will consist of virtualised infrastructure and backup and recovery tools
• Processes And Procedures – this is a set of housekeeping tasks that are followed to ensure recovery is possible
• Operational Disaster Recovery And Business Continuity Plan – this is a tested plan to achieve recovery at the DR site
April 11, 2023 85
Server Virtualisation and Disaster Recovery
• Server Virtualisation assists recovery from disaster
• Changing disaster recovery requirements− Higher standards are required− More reliability is expected− Faster pace of business generates more critical
change− Intense competitive environment requires high
service levels
April 11, 2023 86
Challenges of Testing Recovery
• Hardware bottlenecks− Need a separate target recovery server for each of the primary
servers under test− If doing “bare metal” restore, need to locate target recovery
hardware matching exactly the primary server configurations
• Lengthy process with manual interventions− Configure hardware and partition drives− Install Windows and adjust Registry entries− Install backup agent− Before recovering automatically with the backup server
• Personnel not trained− Complex processes and limited equipment availability make it
difficult to train personnel
April 11, 2023 87
Successful Disaster Recovery
• Ensure successful recovery − Diligent use of a reliable backup tool− Regular testing of recovery procedures
• Meet the TTR/RTO (Time To Recover/Recovery Time Objective) objectives− Target recovery hardware available− Alternate site available− Processes documented and automated
• Put personnel plan in place− Primary and backup DR coordinators designated
and trained− Dry runs are conducted regularly
April 11, 2023 88
Why Virtual Infrastructure for DR?
• Hardware Independence− Flexibility to restore to any hardware
• Hardware Consolidation / Pooling / Oversubscription− Test recovery of all systems to one physical server
• Speed up recovery− Use pre-configured templates with pre-installed OS & backup
agent
• Single-step simplified capture and recovery− Different purposes – same procedures – Staging, Deployment,
Disaster Recovery− One step system and application recovery− No additional licensing requirements for bare metal restore
tools− More trained personnel available
April 11, 2023 89
Disaster Recovery at Lower Cost
• Hardware / System/ Application independence− No need to worry about the exact hardware
configuration− Flexibility to restore to any hardware− Application independent capture and recovery
processes
• Less hardware required at “hot” failover site• Support for all capture / replication
technologies− Tape / Media− Disk-based Back up− Synchronous or Asynchronous Data Replication
April 11, 2023 90
Simplified Processes for Recovery
• Restore system and application data in one step• Single-step simplified capture and recovery
− One step system and application recovery− No Windows registry issues− Easy-to-automate recovery
• No need for 3rd party ‘bare metal’ restore tools− Reduce learning and ramp-up− Reduce software licensing expense
• Use the same methodology through application lifecycle− Staging /Deployment/ DR
• Test once – recover anything− Application independent recovery means simplified testing
April 11, 2023 91
Virtual Hardware for Real Recovery
• Follow the usual procedure for data backup• For recovery
− Find ONE physical server − Install VMware ESX Server− Copy from a template library a virtual machine with
the appropriate Windows OS service packs and the Backup Agent pre-installed
− Register and start VM, edit IP addresses− Restore from tape into VM using backup server
April 11, 2023 92
Compare Recovery Steps
Find hardware
Configure hardware /
partition drives etc.
Install Operating System
Adjust Registry entries,
permissions, accounts
Install backup agent
Find hardware
Install VMware with Templates
“Single-step automatic recovery” from backup server
“single-step automatic recovery” from backup server
Physi
cal to
Physi
cal
Do O
nce
Repeat
for
each
boxP
hysi
cal to
Vir
tual
Repeat
for
each
box
April 11, 2023 93
Customer Options for Recovery
• 1 - Physical to Physical
• 2 - Physical to Virtual
• 3 - Virtual to Virtual
April 11, 2023 94
Disaster Recovery with SAN Replication
• Speed up recovery in solutions based on storage replication− No need to upgrade secondary site server hardware in lock-
step with the primary site− Easy to automate and no need for bare metal recovery tools
April 11, 2023 95
SAN Replication Issues
• Hardware− Synchronous – data is written simultaneously to both SANs.
The write operation is not completed until both individual writes are completed. This will require a communications link between both sites operating at least 1 Gbps.
− Asynchronous – data is not written real-time to the backup unit. Data is buffered and written in blocks. This will require a communications link between both sites operating at least 2 Mbps.
• Software− CommVault QiNetix ContinuousDataReplicator− DoubleTake− RepliStor− WANSync
April 11, 2023 96
Virtualisation Resource Allocation and Configuration Analysis
• How much resources to leave free to cater for server failure?
VM1 VM2 VM3 VM4
Limit Threshold
Reservation Threshold
Actual Usage
VM5 VM6 VM7 VM8
Server 1 Server 2HA Cluster
April 11, 2023 97
Virtualisation Resource Allocation and Configuration Analysis
• Critical (or all virtual servers) will be restarted on other physical server(s)
VM1 VM2 VM3 VM4 VM5 VM6 VM7 VM8
Server 1 Server 2
VM1 VM2 VM3 VM4
HA Cluster
April 11, 2023 98
VMware Platforms and Options
• VMware Infrastructure 3 Starter NAS or local storage− No HA, DRS, VCB− Restrictions
• 4 processors• 8 GB RAM
• VMware Infrastructure 3 Standard− HA, DRS, VCB available as separate options
• VMware Infrastructure 3 Enterprise− Includes virtual SMP, VMFS, VMotion, HA, DRS,
Consolidated Backup
• VirtualCentre
April 11, 2023 99
VMware Sample Costs
Product Rough Cost Annual Software Subscription and
Support
Year 1 Total Year 2
VMware Infrastructure 3 Starter for 2 processors €781.25 €697.27 €1,478.52 €697.27
VMware Infrastructure 3 Standard for 2 processors
€2,929.69 €615.23 €3,544.92 €615.23
VMware Infrastructure 3 Enterprise for 2 processors
€4,492.19 €943.36 €5,435.55 €943.36
VMware VirtualCenter Management Server 2 €3,906.25 €625.00 €4,531.25 €625.00
VMWare Enterprise for two 2-processor servers and VirtualCentre
€12,890.63 €2,511.72 €15,402.34 €2,511.72
VMWare Enterprise for four 2-processor servers and VirtualCentre
€21,875.00 €4,398.44 €26,273.44 €4,398.44
VMWare Enterprise for four 4-processor servers and VirtualCentre
€39,843.75 €8,171.88 €48,015.63 €8,171.88
April 11, 2023 100
Sample Configurations
• Two ESX Servers, VirtualCentre, Backup to Disk, Tape Backup
• Two ESX Servers, VirtualCentre, Backup to Disk, Tape Backup, Virtualised DR Facility with Replication
• Very Large Scale Implementation
April 11, 2023 101
Two ESX Servers, VirtualCentre, Backup to Disk, Tape Backup
1. Two servers running ESX Server – provides resilience in the event of server failure
2. SAN to store data
3. VirtualCentre to administer and manage virtual infrastructure
4. Backup to disk using low cost disk
5. Tape backup unit
April 11, 2023 102
Two ESX Servers, VirtualCentre, Backup to Disk, Tape Backup
1. Primary SAN data copied to inexpensive disk – fast backup
2. Disk backup copied to tape/autoloader
April 11, 2023 103
Two ESX Servers, VirtualCentre, Backup to Disk, Tape Backup, Virtualised DR Facility with Replication
1. Two servers running ESX Server – provides resilience in the event of server failure
2. SAN to store data
3. VirtualCentre to administer and manage virtual infrastructure
4. Backup to disk using low cost disk
5. Tape backup unit
6. Link for data replication
7. Backup virtual infrastructure for recovery
April 11, 2023 104
Two ESX Servers, VirtualCentre, Backup to Disk, Tape Backup, Virtualised DR Facility with Replication
1. Primary SAN data copied to inexpensive disk – fast backup
2. Disk backup copied to tape/autoloader
3. Disk to disk copy to DR location
4. Move tapes to backup location
April 11, 2023 105
Two ESX Servers, VirtualCentre, Backup to Disk, Tape Backup, Virtualised DR Facility with Replication
April 11, 2023 106
Very Large Scale Implementation
April 11, 2023 107
Very Large Scale Implementation
April 11, 2023 108
Cost Benefit Analysis
• Tangible savings− Server purchases− Operational costs− Administration costs− Power, HVAC− Deferred cost
• Intangible savings− Faster server provisioning− Better utilisation− Reduced floorspace− Improved business continuity and disaster recovery
April 11, 2023 109
Server Operation AssumptionsServer Environmental DetailsServer Watts/Hour 600UPS Watt/Hour 25Server BTU/Hour 2000Server Operational Hours 8760kWh Cost €0.10Total kWh/Server/Year 7227Total Electricity Cost (Server, UPS, HVAC) €722.70Maintenance/Server €350.00Operation Costs Per Server/Year €1,072.70
Server Tasks - Per Server Hours Before Virtualisation
Hours After Virtualisation
New Server Deployment 16 2Build / Installs 40 10Change / Upgrade 12 3Configuration Changes 2 0.1Problem Resolution 2 0.1Rebuilding Test Servers 2 0.1Installing Software 2 0.1Rebooting System 2 0.1Testing 10 0.5Recovery 8 1
April 11, 2023 110
Sample Project Costs and Savings 1
• 16 servers to be virtualised• Avoid 4 new servers a year
Virtualisation Project Initial Year 1 Year 2 Year 3 TotalSoftware €21,900.00 €6,100.00 €6,100.00 €6,100.00 €6,100.00Hardware €16,000.00Procurement €800.00Project Costs €25,000.00Server Operation €3,489.40 €3,489.40 €3,489.40Maintenance and Support
€12,000.00 €12,000.00 €12,000.00
Server Administration €573.73 €573.73 €573.73Total €63,700.00 €22,163.13 €22,163.13 €22,163.13 €130,189.38Saving €120,171.68
Existing Servers Initial Year 1 Year 2 Year 3 TotalNew Server Purchases €32,000.00 €32,000.00 €32,000.00Procurement €1,600.00 €1,600.00 €1,600.00Server Operation €22,798.00 €22,798.00 €22,798.00Server Administration €27,055.69 €27,055.69 €27,055.69Total €83,453.69 €83,453.69 €83,453.69 €250,361.06
Return on Investment 39 Months
April 11, 2023 111
Sample Project Costs and Savings 2
• 32 servers to be virtualised• Avoid 6 new servers a year
Virtualisation Project Initial Year 1 Year 2 Year 3 TotalSoftware €29,900.00 €8,300.00 €8,300.00 €8,300.00 €8,300.00Hardware €32,000.00Procurement €1,600.00Project Costs €50,000.00Server Operation €6,978.80 €6,978.80 €6,978.80Maintenance and Support
€20,000.00 €20,000.00 €20,000.00
Server Administration €1,147.45 €1,147.45 €1,147.45Total €113,500.00 €36,426.25 €36,426.25 €36,426.25 €222,778.75Saving €221,107.16
Existing Servers Initial Year 1 Year 2 Year 3 TotalNew Server Purchases €48,000.00 €48,000.00 €48,000.00Procurement €2,400.00 €2,400.00 €2,400.00Server Operation €43,450.60 €43,450.60 €43,450.60Server Administration €54,111.37 €54,111.37 €54,111.37Total €147,961.97 €147,961.97 €147,961.97 €443,885.92
Return on Investment 36 Months
April 11, 2023 112
Sample Project Costs and Savings 2
• 64 servers to be virtualised• Avoid 8 new servers a year
Virtualisation Project Initial Year 1 Year 2 Year 3 TotalSoftware €45,900.00 €12,700.00 €12,700.00 €12,700.00 €12,700.00Hardware €64,000.00Procurement €3,200.00Project Costs €75,000.00Server Operation €13,957.60 €13,957.60 €13,957.60Maintenance and Support
€25,000.00 €25,000.00 €25,000.00
Server Administration €2,294.90 €2,294.90 €2,294.90Total €188,100.00 €53,952.50 €53,952.50 €53,952.50 €349,957.51Saving €424,141.93
Existing Servers Initial Year 1 Year 2 Year 3 TotalNew Server Purchases €64,000.00 €64,000.00 €64,000.00Procurement €3,200.00 €3,200.00 €3,200.00Server Operation €82,610.40 €82,610.40 €82,610.40Server Administration €108,222.75 €108,222.75 €108,222.75Total €258,033.15 €258,033.15 €258,033.15 €774,099.44
Return on Investment 30 Months
April 11, 2023 113
SAN Options and Vendors
April 11, 2023 114
SAN Vendors
• Dell/EMC− AXnnn - iSCSI− NSxxx – IP − CXnnn – Fibre Channel− DMX− Centera
• IBM− DS series− N Series – multi-protocol
• HP− MSA− EVA− XP
April 11, 2023 115
System Center Operations Manager (SCOM)
April 11, 2023 116
SCOM Configuration
April 11, 2023 117
SCOM ComponentsComponent Description
SCOM Database A Microsoft SQL Server database that stores configuration information and operations data that is produced by the monitoring process.
SCOM Management Server A computer that is responsible for monitoring and managing other computers. The SCOM Management Server consists of the Data Access Server, and the SCOM Server and SCOM Agent components. The SCOM Management Server is an essential part of a management group.
Data Access Server (DAS) A COM+ application that manages access to the SCOM database.
SCOM Server A component that manages the SCOM Agents that monitor computers in a MOM environment.
SCOM Agent A component that monitors and collects data from a managed computer.
SCOM Reporting Database A SQL Server database that collects and stores the operations data contained in the SCOM Database.
User interfaces The Administrator console and Operator console installed by default when you install SCOM.
Management Pack A specific extension that provides for the monitoring of a given service/application
April 11, 2023 118
SCOM Deployment Options
• Agentless Monitoring− SCOM monitors agentless servers. This is aimed at IT
environments where agents could not be installed on a few exception nodes. Agentless monitoring is limited to status monitoring only.
• Agent Support− Agents are installed on servers. SCOM lets you
manage applications running on servers.
• Server Discovery Wizard− Allows for server lists to be imported from Active
Directory, from a file, or from a typed list. It also allows the list to be filtered using LDAP queries, as well as name– and domain name–based wildcards.
April 11, 2023 119
Architecture
ProductConnector
ThirdParty Agent
ManagedComputer
AgentlessManagedComputer
DASSDK
Reporting Console
Administrator Console
Operator Console
Web Console
MOM MANAGEMENT SERVER
SQL views:Configuration data
Operations data
MOM OPERATIONS DATABASE
Archival dataFor analysis
SQL ReportingServices
SYSTEM CENTER DATA WAREHOUSE
MOM AgentMOM Server
MOM SERVICE
RPC/DCOMSecureTCP/IP
Channel
OLEDB
OLEDB
DTS
April 11, 2023 120
SCOM Rule: Unit Of Instruction/Policy
• Event Rules− Collection rules− Filtering rules− Missing event rules− Consolidation rules− Duplicate Alert
Suppression
• Performance Rules− Measuring− Threshold
• Alert Rules
Rule
Provider
NT event log Perfmon
data WMI SNMP Log files Syslog
Criteria Response
Alert Script SNMP trap Pager E-Mail Task Managed
Code File Transfer
•Wheresource=DCOM and Event ID=1006
Knowledge
• Product Knowledge
• Links to Vendor
• Company Knowledge
• Links to Centralised Company knowledge
April 11, 2023 121
SCOM Database
• The SCOM database is a single authoritative source of all Configuration in a Management Group
− Rules, Overrides− Scripts− Computer attributes− Views− SCOM Server and Agent Configurations− Nested Computer Groups− Extensible schema for classes, attributes and
associations
MOM DatabaseSQL views:
Configuration dataOperations data
April 11, 2023 122
UI Consoles • Operator Console
− To create and display view instances, Update Alerts• User Customizable Views • Views can be organized in a
folder hierarchy• Context Sensitive tasks
− Multipane View
• Administrator Console− One MMC Snapin per
management group− Rules Node – To author, view,
modify, Export/Import rules− Config Node – To configure
SCOM
• Web Console
April 11, 2023 123
SCOM Console Views
• State View - Provides you with a real-time, consolidated look at the health of the computers within the managed environment by server role, such as Active Directory domain controllers, highlighting the systems that require attention.
• Diagram View - Gives you a variety of topological views where the existence of servers and relationships are defined by management packs. The Diagram View allows you to see the status of the servers, access other views, and launch context-sensitive actions, helping you navigate quickly to the root of the problem.
• Alerts View - Provides a list of issues requiring action and the current state and severity of each alert. It indicates whether the alerts have been acknowledged, escalated, or resolved, and whether a Service Level Agreement has been breached.
• Performance View - Allows you to select and display one or more performance metrics from multiple systems over a period of time.
• Events View - Provides a list of events that have occurred on managed servers, a description of each event, and the source of the problem.
• Computers and Groups View - Allows you to see the groups to which a computer belongs, the processing rule groups with which it is associated, as well as the attributes of the computer.
April 11, 2023 124
SCOM and SQL
April 11, 2023 125
The SCOM Administrator Console
April 11, 2023 126
SCOM Management Packs
• SCOM management packs provide built-in, product-specific operations knowledge for a wide variety of server applications
• Management packs contain rules for monitoring an array of server health indicators and creating alerts when problems are detected or reasonable thresholds are exceeded
• Monitoring capability is extended by knowledge base content, prescriptive guidance, and actionable tasks that can be associated directly with the relevant alerts included in the management packs
• Administrators can then act to prevent or correct situations, such as degraded performance or service interruption, maintaining service availability with greater ease and reliability
April 11, 2023 127
SCOM 2005 Management Packs• Standard Management
Packs− Exchange 2000 and 2003
Server− Internet Information Services − SCOM 2005 and SCOM 2000
Transition − Security (MBSA)− SQL Server 2000− Windows Active Directory− Windows Server Cluster− Windows DNS− Windows Server (2000, 2003,
NT4)
• Tier 2 Management Packs− Windows Update Services− Virtual Server 2005− Web Services− Application Center 2000− Terminal Services− DHCP− Remote File Systems− Print Server
April 11, 2023 128
Management Packs
• Management Pack imported via SCOM Server• Discovery finds computers in need of a given
Management Pack • SCOM deploys appropriate Management Packs
− No need to touch managed nodes to install Management Packs
• Rules: Implement all SCOM monitoring behavior− Watch for indicators of problems− Verify key elements of functionality
• Management Packs provide a definition of server health
April 11, 2023 129
Management Pack Features
• Alerts: Calls attention to critical events that require administrator intervention − Product Knowledge: Provides guidance for administrators to resolve
outstanding alerts
• Views: Provide targeted drill down details about server health− Performance plots, collections of specific events/alerts, groups of servers ,
topology, etc.
• State Monitoring: At a glance view of the state of my servers and applications by server role− Detail to component level
• Tasks: Enable administrators to investigate and repair issues from the SCOM console− Context sensitive diagnostics and remediation
• Reports: Historical data analytics− Assess operations performance and capacity planning
April 11, 2023 130
Alert Handing and Viewing
• When a new alert is identified it will appear in the Alert Pane with a resolution state of “New”
• If you highlight that alert its details will appear in the Alert detail Pane
• Clicking on the “Properties” tab in the Alert Detail Pane will give you the description (and other details) of the alert
• The alert can be classified as:
− False Negative− Hardware Issue− Non Hardware Issue
April 11, 2023 131
Alert Handling
April 11, 2023 132
SCOM VMware Management Pack Integration
April 11, 2023 133
SCOM and nWorks Management Pack
• nworks Collector is referred to as VEM (Virtual Enterprise Monitor)
• The VEM server can be a virtual server to reduce cost
April 11, 2023 134
Enabling Greater Resource Utilisation Through Storage System Virtualisation
April 11, 2023 135
What is “Storage Virtualisation”?
• Abstracted Physical Storage• Storage Pools Created from Physical Blocks of
Storage• Virtual Disks created from Storage Pool• Physical Devices and Capacity Distribution
Transparent to Servers and Applications
April 11, 2023 136
Why Is Storage Virtualisation so Critical?
April 11, 2023 137
Opposing Forces on Volume Size
Bigger Gives EfficiencySmaller Gives Control
Different classes of data
Different management requirements
Tools work on volumes(Snapshots, etc)
Disks growing
ATA growing faster
More disks for performance
RAID-DP
April 11, 2023 138
The Problem: Volumes Tied to Disks
What we’ve got today:• Small volumes are impractical• Large volumes are hard to manage
What we’d like:• Manage volumes separately from physical disks• Volumes for data; aggregates for disks
April 11, 2023 139
14 x 72 GB disks = 1 TB capacity
Virtualisation Improve Utilisation
Vol 0
Data Parity
Database
Data Data Data Data Data Data Data Parity Spare
Home Directories
Data Data Parity
Logical Drive 1 = 2 Disks
Logical Drive 2 = 8 Disks
Logical Drive 3 = 3 Disks
1 Hot spare
140 GB 370 GB40 GB
550 GB of wasted space
April 11, 2023 140
The Solution: Flexible Volumes (FlexVol)
• Aggregate contains the physical storage
• FlexVol: no longer tied to physical storage
• FlexVol: multiple per aggregate
• Storage space can be easily reallocated
Storage Pool
Disks Disks Disks
Flexible Volumes
April 11, 2023 141
RG1 RG2 RG3
Storage Blocks
Storage PoolStorage Pool
RG1 RG2 RG3
Storage Pools and Flexible VolumesHow Do They Work?
• Create RAID groups
• Create Storage Pool
• Create and populate each flexible volume
• No pre allocation of blocks to a specific volume
• Storage System allocates space from pool as data is written
Flexible Volume 1
Flexible Volume 2
Flexible Volume 3
vol1vol1 vol2vol2
vol3vol3
April 11, 2023 142
14 x 72 GB disks = 1 TB capacity
Flexible Volumes Improve Utilisation
Logical Drive 1 = 144GB
Logical Drive 2 = 576GB
Logical Drive 3 = 216GB
1 Hot spare
SpareData Data Data Data Data Data Data Data Data Data Data Parity Parity
Aggregate
Database Home DirsVol0
400 GB used
600 GB of Free Space!
April 11, 2023 143
Flexible Volume Data Management Benefits
• Distinct containers (volumes) for distinct datasets• Flexible Volumes resize to meet space requirements,
simple command to adjust size (grow / shrink)• Soft allocation of volumes and LUNs• Free space flows among all Flexible Volumes in a
storage pool; space reallocation without any overhead• Flexible Volumes can be:
− SnapManaged independently− Backed up independently− Restored without affecting other Flexible Volumes
April 11, 2023 144
Compare Benefits
Flexible Volumes Legacy SAN
Space Allocation Flexible and dynamic Volumes can be grown
and shrunk
Management
SpindleSharing
Preallocated and static Space is preallocated
during configuration Space can’t be shrunk
Simple Complex
Automatic sharing of spindles among all volumes, including newly added disks
New spindles are only used when volumes are expanded
Optimal configuration is a daunting task(sliced, striped, etc.)
April 11, 2023 145
Compare Benefits
Granularity Volumes can be grown and shrunk in small increments (1MB) without performance or management impact
Disruption
Rapid Replication
More granularity comes at the expense of performance or management
Growing and shrinking are nondisruptive and instantaneous operations
Shrinking is not possible; growth involves reshuffling of data
Often involves downtime and data copying
FlexClone™ is immediate No performance implications Large space savings for
similar volumes
Business continuance volumes involve physical replication of the data
No space savings
Flexible Volumes Legacy SAN
April 11, 2023 146
LUNs
Application-levelsoft allocation
10 TB
800 GB
Flexible Volumes: Enabling Thin Provisioning
Flexible Volumes: Container level:
flexible provisioning Better utilisation
Physical Storage: 1 TB
FlexVols: 2TB
Container-levelsoft allocation
1 TB
300GB
200GB
200GB
50GB
150GB
100GB
Application-level: Higher granularity Application over-
allocation containment
Separates physical allocation from space visible to users
Increases control of space allocation
April 11, 2023 147
Managing Complexity through Storage Virtualisation
April 11, 2023 148
Unified Management
• Storage management and administration is very vendor specific
• Most vendors require different skills for different storage systems
• Hardware is not cross compatible
April 11, 2023 149
The Unified Storage Architecture Advantage
Incompatible silos Compatible family
Platforms
HP, EMC, DELL, IBM Storage Virtualisation
Software &Processes
Incompatible software;different processes
Unified software;Same processes
Experts &IntegrationServicesLots of experts and
integration services Reduced training & service requirements
April 11, 2023 150
DMX SeriesCX3-20 CX3-40AX150/S
EMC FCEMC FC
CX3-80CX3-10
Virtualisation:Virtualisation:Architectural Simplicity
Multiple Concurrent ProtocolsIntegrated Mgmnt, DR, BC, ILM, D2D,
…
NS40G NSXNS80G
Centera
CelerraSymmetrix / DMX and CX ONLY
Virtual GatewaysHP, IBM, HDS, SUN
CX300iAX150iiSCSI OnlyiSCSI Only
The EMC Effect? - ComplexityThe EMC Effect? - Complexity• 8 Dissimilar Operating Systems• 8 Dissimilar Mgmnt GUI’s
• Dissimilar DR, BC, …• ILM required
CentraStar - 6
1 - FLARE OE
5 - Enginuity
2 - FLARE
EMC IPEMC IP
NS80NS40NS350
8 - MS Win
3 - Dart
4 - RHEL
2 - FLARE
8 - MS Win
Virtual Storage Environment / EMC – Comparison
External server w/MS Win and CLARalert required to support CX dial/email home support (compare to AutoSupport).
Virtual Gateway
Limited iSCSI Support
April 11, 2023 151
Managing Disk Based Backup Through Storage Virtualisation Single Instance Storage (Deduplication)
April 11, 2023 152
Snapshot
and Snapshot Restore
Snapshot
and Snapshot Restore
Backup Integration
Backup and Recovery SoftwareBackup and Recovery Software
Disk Based Target Disk Based Target
Secondary
Storage
Secondary
Storage
Primary Data
9AM
12PM
3PM
Snapshot
Snapshot
Snapshot
Primary StoragePrimary Storage
InstantRecovery
Short-TermLocal Snapshot Copies
Mid- to Long-Term Disk to Disk
Block-Level Backups
Client Drag-and-Drop Restores
Changed Blocks
April 11, 2023 153
Advanced Single Instance Storage
User1 presentation.ppt
20 x 4K blocks
User2 presentation.ppt
Identical file 20 x 4K blocks
User 3presentation.ppt
Edited, 10 x 4K
User4 job-cv.doc
Different file 8 new 4K blocks
= Identical blocks
Data Written to Disk:
With ASIS: 38 blocks
Without ASIS: 75 blocks
April 11, 2023 154
Enabling greater Data Management Through Storage System SnapShots
April 11, 2023 155
Snapshots Defined
• A Snapshot is a reference to a complete point-in-time image of the volume’s file system, “frozen” as read-only.
• Taken automatically on a schedule or manually
• Readily accessible via “special” subdirectories
• Multiple snapshots concurrently for each file system, with no performance degradation.
• Snapshots replace a large portion of the “oops!” reasons that backups are normally relied upon for:
− Accidental data deletion− Accidental data corruption
• Snapshots use minimal disk space (~1% per Snap)
April 11, 2023 156
Snapshot Internals - As They Should Be
C’
Snapshot
File: FILE.DAT
• Client modifies data at end of file• Data actually resided in block C on disk
System writes modified data block to new location on disk (C’)
A B C
Active File System
File: FILE.DAT
Disk blocks
April 11, 2023 157
Snapshot Internals
Active file system version of FILE.DAT is now composed of disk blocks A, B & C’.Snapshot file system version of FILE.DAT is still composed of blocks A, B & C
C’
Snapshot
File: FILE.DAT
A B C
Active File System
File: FILE.DAT
Disk blocks
April 11, 2023 158
User is offered this most recent previous version (and up to 255 older versions)
User may drag any of these read-only files back into active service
Snapshot-Based Data Recovery
April 11, 2023 159
Snapshots are State-of-the-Art Data Protection
Snapshots should be near instantaneous!
To create a point-in-time Snapshot copyrequires copying a simple data structure,not copying the entire data volume
Additional storage is expended incrementally only for changed blocks only as data changes, not at Snapshot creation time
Avoids the significant costs associated with the I/O bandwidth, downtime, CPU cycles dedicated to copying and managing entire volumes
April 11, 2023 160
Not all Snapshots Are Equal
• What is the disk storage requirement to maintain online data copies? • Will a planned or unplanned or "dirty" system shutdown lose existing data
copies? • What is the overall performance impact with snapshots enabled? • How many data copies can be maintained online? • Is the reserve area fixed? Can this "save area" be re-sized on the fly? • Are data copies automatically deleted once the save area is full?• What is the answer to file system recovery? Do they feature a
SnapRestore-like capability? • Are snapshots a chargeable item? How much? What is the pricing
model? • Is this snapshot method supported across the vendor's entire product
line?
Questions to ask regarding storage system data copy techniques:
April 11, 2023 161
Enabling Greater Application Resilience Through SnapShot Technologies
April 11, 2023 162
Snapshot Active File System
SnapRestore Recovery
2 N
Active File SystemsnapX restore
1 … 2’ N’1’ …
Marked as free blocksafter Snapshot Restore
April 11, 2023 163
Database Recovery
9am 5pm10:00 11:00 12:00 13:00 14:00 15:00 16:00
Snapshots
1 2 3 4 5 6 7 8 9
15:22
Corruption !
Snapshot restore
April 11, 2023 164
Enabling Greater Data Resilience Through Storage System Mirroring
April 11, 2023 165
Storage Mirroring
• Storage Mirroring− Synchronous− Semi Synchronous− Asynchronous
April 11, 2023 166
Storage Mirroring Defined
• Replicates a filesystem on one storage system to a read-only copy on another storage system (or within the same storage system)
• Based on Snapshot technology, only changed blocks are copied once initial mirror is established
• Asynchronous or synchronous operation
• Runs over IP or FC
• Data is accessible read-only at remote site
• Replication is volume based
April 11, 2023 167
SnapMirror Function
…...
SAN or NAS Attached hosts
Source
of source volume(s)
Baseline copy
…...
Source
of changed blocks
Periodic updates
Step 1: Baseline
Step 2: Updates
Target
LAN/WAN
Target
LAN/WANSAN or NAS Attached hosts
ORImmediate Write Acknowledgement
Immediate Write Acknowledgement
April 11, 2023 168
Snap A
Storage Mirroring Internals
Baseline Transfer
Source Volume Target Volume
April 11, 2023 169
Snap A
Storage Mirroring Internals
Baseline Transfer
Source Volume Target Volume
Completed
Target file system is now consistent, and a mirror of the
Snapshot A file system
Source file system continues to change
during transfer
Commonsnapshot
April 11, 2023 170
Storage Mirroring Internals
Incremental Transfer
Source Volume Target Volume
Snap B
Target volume is now consistent, and a mirror of the
Snapshot B file system
Completed
Snap A
April 11, 2023 171
Storage Mirroring Internals
Source Volume Target Volume
Snap CIncremental
TransferCompleted
Target volume is now consistent, and a mirror of the
Snap C file system
April 11, 2023 172
Storage Mirroring Applications
• Data replication for local read access at remote sites
− Slow access to corporate data is eliminated− Offload tape backup CPU cycles to mirror
• Isolate testing from production volume− ERP testing, Offline Reporting
• Cascading Mirrors− Replicated mirrors on a larger scale
• Disaster recovery− Replication to “hot site” for mirror failover and
eventual recovery
April 11, 2023 173
Data Replication for Warm Backup/Offload
• For Corporations with a warm backup site, or need to offload backups from production servers
• For generating queries and reports on near-production data
MAN/WAN
Backup Site
Production Sites
Tape Library
April 11, 2023 174
& WRITEREAD
Isolate Testing from Production
• Target can temporarily be made read-write for app testing, etc.− Source continues to run online− Resync forward after re-establishing the mirror relationship
SnapMirror
Production Backup/Test
READ & WRITE
XSnap C Incremental
Transfer
SnapMirror Resync
(Resync backward works similarly in opposite direction)
April 11, 2023 175
Cascading Mirrors
• Allows a target volume to be a source to other targets• Each target operates on an independent schedule• Replicate data up to 30 destinations
Source NS
Source Volume
(read + write)
SnapMirror
Target NS
Target Volume
(read only)
SnapMirror
Target NS
Target Volume
(read only)
SnapMirror
Target NS
Target Volume
(read only)
April 11, 2023 176
Cascading Replication - Example
• Replicate to multiple locations (30) across the continent− Send data only once across the expensive WAN− Reduces resource utilisation on source NS
WAN
Office 1 Office 2
Office 5
Office 4
Office 3
April 11, 2023 177
Disaster Recovery
LAN/WAN
• For any corporation that cannot afford the downtime of a full restore
from tape. (days)
• Data Centric Environments
• Reduces “Mean Time To Recovery” when a disaster occurs.
Production Site Disaster Recovery Site
(redirect)
(resync backwards after source restoration)
X
April 11, 2023 178
Easing the Pain of Development Through SnapShot Cloning
April 11, 2023 179
Cloning SnapShots
• Write enables SnapShots
• Enables multiple, instant data set clones with no storage overhead
• Provides dramatic improvement for application test and development environments
• Renders alternative methods archaic
April 11, 2023 180
Cloned SnapShot Volumes: Ideal for Managing Production Data Sets
• Error containment− Bug fixing
• Platform upgrades− ERP− CRM
• Multiple simulations against a large data set
April 11, 2023 181
Start with a volumeVolume 1
Volume 2(Clone)
Create a clone(a new volume based on the Snapshot copy)
Volume Cloning: How It Works
Data Writtento Disk:
Snapshot Copy
Snapshot™Copy of
Volume 1
Create a Snapshot copy
Result: Independent volume copies, efficiently stored
Modify the cloned vol
Cloned VolumeChanged Blocks
Volume 1 Changed Blocks
Modify the original vol
April 11, 2023 182
Volume Splitting
Split volumes when most data is not shared
Volume 1
Snapshot™Copy of
Volume 1 Replicate shared blocks in
the background
Volume 2
Result: Easily create new permanent volume for forking project data
April 11, 2023 183
The Pain of Development
Prod Volume (200gb)
Pre-Prod Volume (200gb)
QA Volume (200gb)
Dev Volume (200gb)
Test Volume (200gb)
Sand Box Volume (200gb)
1.4 TB Storage Solution
200 GB Free
Create copies of the volume
Requires processor time and Physical storage
April 11, 2023 184
Clones Remove the Pain
Prod Volume (200gb)
Pre-Prod Volume
QA VolumeDev Volume
Test Volume
Sand Box Volume
1.4 TB Storage Solution
Create Clones of the Volume – no additional space required
Start working on Prod Volume and Cloned Volume
Only changed blocks get written to disk!
1 Tb Free
April 11, 2023 185
Ideally…
Primary Production
Array
Secondary
Array
Mirror
Create Clones from the Read Only mirrored volume
Removes development workload from Production Storage!
April 11, 2023 186
Rapid Microsoft Exchange Recovery through Storage Systems Technologies
April 11, 2023 187
Why use Storage Systems Series for Exchange Data?
Just a few off the top…
Snapshot copies “snapshots” Data and snapshot management, replication Flexible and easy, dynamic provisioning Performance iSCSI, cost effective and gaining on Fibre Channel Excellent high-end FCP, clustering and MPIO options Tight Windows OS (incl. MSCS) and Exchange 5.5., 2000, 2003 and
2007 Server integration (SME, VSS on Windows 2003, etc.)
April 11, 2023 188
Required Storage Software for Exchange
• SnapShot Management− Rapid online backups and restores—integrates with
Exchange backup API; runs ESEFILE verification; automates log replay
− Intuitive GUI and wizards for configuration, backup, and restore
• Server Based Connection Manager− Dynamic disk and volume expansion− Supports both Ethernet and Fibre Channel environments− Supports MSCS and NS Series CFO for high availability
• Single mailbox recovery software− Restores single message, mailbox, or folder from a
Snapshot™ backup to a live Exchange server or a .pst file
April 11, 2023 189
Effective SnapShot Management with Exchange
• Manages the entire snapshot backup process• Backup and restore Exchange storage groups • Backups may be scheduled• Each backup is a “full” Exchange backup and is
verified using MS provided software, which is integrated into the storage system
April 11, 2023 190
SnapShot Management with Exchange Overview
• Interacts with Exchange using Exchange backup APIs• interacts with VSS
− SnapShot Management is VSS requestor− Exchange is VSS writer− Storage System is VSS hardware provider
• Provides point-in-time and up-to-the-minute recovery using snapshots and Exchange database transaction logs
April 11, 2023 191
SnapShot Mirroring
• SnapShot Mirroring− Automatic mirroring of Exchange data to
remote site − Volume based mirroring− Occurs immediately following a Exchange
backup and is initiated by Exchange Server− Can replicate over LAN or WAN− Only changed blocks since previous mirror are
replicated− Rate of replication can be throttled to
minimize impact on network
April 11, 2023 192
Single Mailbox Recovery
• Allows restores of individual items form Exchange backups in minutes compared to hours or days
• Single mailbox recovery is the most requested feature by Exchange customers
April 11, 2023 193
Single Mailbox Restore (Exchange)
• PowerControls Software− Quickly access Exchange data already stored in
the online snapshot backups− Select any data, down to a single message− Restore the data to one of two locations:
• An offline mail file (.PST personal storage file) which can be opened in MS Outlook
• Connect to a live Exchange server and copy data directly into the users mailbox, making it instantly available
April 11, 2023 194
Exchange Single Mailbox Restore (SMBR)
April 11, 2023 195
Current Alternatives: Inadequate
• Perform daily brick level backups− Pros
• Allows quicker recovery of a single mailbox− Cons
• Backs up each mailbox separately; one message sent to a 100 people will be copied 100 times
• Very time and disk intensive• Impractical to have frequent backups • Brick level backup software is expensive
• Have a dedicated recovery server infrastructure− Pros
• Reduces the time to recover a single mailbox by eliminating the need to setup a recovery server each time
• Eliminates brick level backups− Cons
• Still very time and labor intensive (many hours)• Requires additional hardware investments
April 11, 2023 196
SMBR and SnapShot Management
• SnapShot backs up Exchange in seconds with snapshots
• SMBR restores individual mailboxes from snapshots in minutes
Primary Data Center
Single MailboxRecovery Software
Time to restore: minutes
Restore mail box
April 11, 2023 197
SMBR: Features
• Reads contents of Exchange Information Store without an Exchange server
• Extracts mail items at any granularity from an offline copy of the Exchange Information Store (E5.5, E2K, & E2K3)− Folder− Single mailbox− Single message− Single attachment
• Restores single mail items to a production Exchange server, alternate server or to an Outlook PST file.
• Advanced search and retrieval− Search subject or message body; keyword, user, or date
April 11, 2023 198
SMBR: Benefits
• Dramatically reduces the time required for single mailbox and single message recovery− From hours or days to just minutes − Simplifies the most dreaded task by Exchange
administrators
• Eliminates the need for expensive, cumbersome and disk-intensive daily brick level backups
• Eliminates the need for recovery server infrastructure
• Allows easy search and discovery of email messages and attachments
April 11, 2023 199
Rapid Microsoft SQL Recovery through Storage Systems Technologies
April 11, 2023 200
SnapShot Management with SQL Server
Application consistent data management
April 11, 2023 201
SnapShot Management with SQL Server
• Provides integrated data management for SQL Server 2000 and SQL Server 2005 databases− Automated, fast, and space-efficient backups using
Snapshots− Automated, fast, and granular restore and recovery
using SnapShot restore technologies− Integrated with storage system Mirroring for database
replication
• Provides tight integration with Microsoft technologies such as MSCS, Volume Mount Points.
April 11, 2023 202
SnapShot Management with SQL Server – Required Features
Features BenefitsRapid hot backup and restore times
• Maximizes SQL database availability and helps meet stringent SLAs
• Helps organizations recover from accidental user induced errors or application misbehavior
• Minimizes SQL downtime and thus reduces cost
• Increases the ability of SQL Servers to handle large number of databases and/or higher workloads.
Hot backups to Snapshot copies • No performance degradation during backups
Configuration, Backup, and Restore wizards with standard Windows GUIs
• Ease of use• Virtually no training costs • Cost savings
MSCS Support • High availability and enhanced reliability of SQL Server environment
Clustered Failover • Further enhances availability of SQL Server
Storage Mirroring Integration • Increases SQL Server’s availability – can replicate the database to a secondary storage system for faster recovery in case of a disaster
April 11, 2023 203
SnapShot Management with SQL Server – Required Features
Features BenefitsOnline disk addition (storage expansion)
• Increases SQL Server’s availability -- additional storage can be added without bringing the SQL Server down
Volume Mount Point Support • Support for Volume Mount Points in order to eliminate the limitation with drive letters
Native x64 support • Supports 64bit natively on AMD64/EM64T
April 11, 2023 204
SnapShot Management for SQL Server (SMSQL)
DBA:• Ability to backup DB faster with fewer resources and
without any storage knowledge• Reduces Mean Time to Recovery on failure
− Quick Restores− More frequent backups Less logs to replay Faster
Recovery
Storage Admin:• Ability to backup and restore DB without any DB
knowledge• Space, time & infrastructure efficient backups, restores
and clones• Increased productivity and storage utilization
April 11, 2023 205
iSCSI or FCP
1
Primary Data Center
Benefits:
• Simplified, centralized management
• Shared storage for improved utilization
• Better system availability
SQL Server
Consolidate SQL Server storage on storage system1
2
2 Add disks and expand volumes on the fly without downtime
3
3 Cluster for higher availability
Technical Details – Consolidated SQL Server Storage
April 11, 2023 206
Primary Data Center
iSCSI or FCP
SQL Server
• Eliminate backup windows• Automation reduces manual
errors• More frequent backups reduce
data loss• No performance degradations
Benefits:
SnapManager automates data management for SQL Server
1
1
Time to backup: seconds
Snapshots
2
2 Snapshots for near-instantaneous backups
3
3 Backup multiple databases simultaneously
Technical Details – Simplified Backup » More Frequent Backups
April 11, 2023 207
Primary Data Center
Time to restore: minutes
• Fast and accurate restoration of SQL Server
• Reduce downtime from outages
• Automation saves administrative time
Benefits:
Near-instant restore from online snapshot
Snapshot
1
1
iSCSI or FCP
SQL Server
Roll transaction logs 2
2 Automated log replay for current image
3
3 Restore single or multiple databases
Standby Server
4
4 Rapid failover to standby server
Technical Details – Rapid Restores » Less Downtime
April 11, 2023 208
Technical Details – Simple & Robust Disaster RecoveryPrimary Data Center DR Site
• Ensures business continuance• Minimizes length of outages• Cost effective – efficient use of
existing IP network
Benefits:iSCSI or
FCPiSCSI or
FCP
System Mirroring
1
Storage Mirroring replicates SQL Server data to remote location
1
Replicate over existing IP networks
2 2 Failover to DR site
After Failure
Failover DB Server
IP network
3 Rebuild primary site from DR site3
April 11, 2023 209
Technical Details – Volume Mount Point (VMP) Support
• Drive letter limitations in SMSQL− Only 26 available drive letters in a system.− Minimum for 2 LUNs required for database migration.
• Limitation for customers who have hundreds of databases.• The customer might not want to have multiple databases on
one/two LUN.• Again one database might span multiple LUNs.
− LUN restore is performed on whole disk.• To support individual database restore, each database will require
its own LUN and drive letter.− Verification will fail on Local server if free drive letter exhausts.
April 11, 2023 210
Technical Details – VMP Storing Database Files
• All SQL SnapShot related files can reside on a mounted volume, same as that of a Standard Volume:− SQL user databases− SQL system databases − SQL Server transaction log file− SnapInfo directory
• Configuration wizard can be used to migrate database files to a mounted volume, same as that of a Standard Volume.− The rules applicable for migrating databases to
Standard Volume will apply for Volume Mount Point also.
April 11, 2023 211
Technical Details – VMP Rules For Mount Point Root
• Database file cannot reside on a LUN which is the root of a mount point:− After LUN restore, all the mount points residing in the
LUN will be overwritten. − For example, db1 resides on G:\mnt1
• Take backup of the database db1 with SMSQL• Now create a mount point G:\mnt1\mnt2• Create a second database db2 in G:\mnt1\mnt2• On restoring the backup set for db1, taken before, G:\mnt1\
mnt2 will go off and hence db2 will become inaccessible
April 11, 2023 212
Technical Details – VMP Rules
• Mounted volumes should not be treated differently from standard volumes.
• Configuration rule for multiple databases on one or two LUNs apply for volume mount point also.
• Backup, restore and other SQL SnapShot operations will have no difference between mounted volume and standard volume, just longer path for mounted volume.
April 11, 2023 213
Technical Details – Backup of Read-Only Databases
• Storage System SQL SnapShots now allows backup of Read-Only database
• In previous release, read-only databases were not displayed in the list of databases in Configuration Wizard
• Now all read-only databases are listed in Configuration wizard, just as normal databases
April 11, 2023 214
Technical Details – Resource Database Management
• Each instance of SQL Server has one and only one associated mssqlsystemresource.mdf file − Instances do not share this file
• The Resource database depends on the location of the master database− If you move the master database, you should also
move the Resource database to the same location
April 11, 2023 215
Technical Details – Resource Database Management
• SMSQL migrates Resource database along with master database− Resource database will not be listed in the
Configuration Wizard− Internally SMSQL migrates it while it migrates master
database− It will be migrated to the same location as master
database
• This is supported only for SQL Server 2005
April 11, 2023 216
SnapShot Management with SQL Server – Summary
• SnapShot Management with SQL Server:− Helps consolidate SQL Server on highly scalable and
reliable storage− Efficient, Predictable, Reliable Backup, Restore and
Recovery for SQL Server databases− Allows dynamic provisioning of storage for
databases− Allows DBAs to efficiently perform database
backup, restore, recovery, clone operations with minimum storage knowledge
− Facilitates Disaster Recovery and Archiving
April 11, 2023 217
Rapid Recovery of Oracle DB Through Storage Systems Technologies
April 11, 2023 218
Oracle Enterprise Manager Grid Control
Monitor Trends and Threshold Alerts
Monitor Key Statistics
Monitor Utilization
•Ships with Oracle Enterprise Manager
•Developed, maintained and licensed separately by Oracle
Manage Storage System from Oracle Enterprise Manager 10g Grid Control
April 11, 2023 219
Automatic
Storage
Management
Oracle ASM
Disks
Logical Vol
File System
0010 0010 0010 0010 00100010 0010 0010 0010 0010Files
Tablespace
Tables
Disk Group
Logical Vol
File System
File Names
Tablespace
Tables
Before ASM ASM
Networked Storage (SAN, NAS, DAS)
April 11, 2023 220
Compatible Storage Adds Value to Oracle ASM
Oracle ASM Compatible Storage Oracle ASM + Compatible Storage
Data Resilience
Protect against Single Disk Failure Yes Yes Yes
Protect against Double Disk failure No Yes Yes
Passive Block corruption detection Yes Yes Yes
Active Block corruption detection Yes Yes Yes
Lost disk write detection No Yes Yes
Performance
Stripe data across ASM Disks Yes No Yes
Balance I/O across ASM Disks Yes No Yes
Stripe data across Physical Disks No Yes Yes
Balance I/O across Physical Disks No Yes Yes
I/O prioritization No Yes Yes
Storage Utilization
Free space management across physical disks
No Yes Yes
Thin provisioning of ASM Disks No Yes Yes
Space efficient Cloning No Yes Yes
Data Protection
Storage Snapshot based Backups No Yes Yes
Storage Snapshot based Restores No Yes Yes
April 11, 2023 221
Integrated Data Management Approach
Go from this…
Centralized Management
+ Administrator productivity+ Storage flexibility+ Efficiency+ Response time
…to THIS
Server-Based Management
Application-Based Management
Storage Management
Integration and
Automation
Data Setsand Policies
X High cost of managementX Long process lead timesX Rigid structuresX Low productivity
April 11, 2023 222
SnapDrive
SnapShot Management with Oracle Overview
Oracle 10g
Oracle 9i
Storage Systems
FCP, iSCSI and NFS*
• Provides easy-to-use GUI• Integrates with the host application• Automates complex manual effort
− Backup/Restores− Cloning
• Tight integration− RMAN− Automated Storage Manager (ASM)
SnapShot Managementwith
Oracle
April 11, 2023 223
SnapShot Management with Oracle
• Database cloning− Ability to clone consistent copies of online databases− GUI support for cloning − Added support for context sensitive cloning
• Increased footprint of platforms and protocols − Support for additional flavors of Unix
• SuSE 9, RHEL3/4 U3+, Solaris 9/10− 32-bit and 64-bit − NFS, iSCSI and FCP for various Unix platforms− HP-UX and AIX (NFS)− (Refer to compatibility matrix for specific details)
• Product hardening− Increased product stability and usability− Improved performance by utilizing snapshot vs. safecopy− Increase performance when dealing with high number of
archive logs
April 11, 2023 224
SnapShot Management with Oracle
• Database cloning to remote hosts− Ability to clone consistent copies of to remote
hosts− Previously clones were assigned to the host
(with SMO) that initiated the cloning request
• Increased footprint of platforms and protocols − HP-UX and AIX support across NFS, iSCSI and
FC
April 11, 2023 225
Database Backup and Recovery
Challenges• DBA’s time spent on non-
value-add backup/restore tasks
• Cold backups lead to lower SLAs
• Separate backups on each platform
• Time-to-recover from tape becomes prohibitive
April 11, 2023 226
Backup and Recovery with Snapshot and SnapShot Restore
• Significant time savings• Stay online• Reduce system and
storage overhead• Consolidated backups• Backup more often
Time in Hours
Time toBackup
Time toRecover
To Tape (60GB/Hr Best Case)
From TapeRedo Logs
300GB Database
0 1 2 3 4 5 6 7 8
Snapshot™
Redo Logs
SnapRestore®
April 11, 2023 227
SnapShot Management with OracleAutomates Backup and RecoveryPrimary Data Center
Benefits:
• Extremely fast and efficient
• No performance degradation
• Accurate data restore and recovery
• Reduce downtime from outages
• Automation reduces errors and saves time
• Backups in seconds
• Snapshot copies verified
• Near instantaneous restores
• Dramatically shortened recovery with automated log replays
• Automated recovery tasks
SnapShot Restore
Time to restore: minutes
DBServer
StorageSystem
Time to backup: seconds
Snapshot
April 11, 2023 228
Database Cloning and the Application Development Process
• Full or partial database copies required for:− App and DB Development− Maintenance (OS, DB upgrade)− Test and QA− Training and Demos− Reporting and DW ETL
• Ability to do this quickly, correctly, and efficiently directly impacts Application Development and Deployment
PROD SECONDARY (DR)
DEV MAINT TEST/QA RPT/ETL
April 11, 2023 229
Traditional Approaches to Cloning
• Copy− Offline− Online (using a mirror or
standby database, snapshots, and log-based consistent recovery)
• Redirected restore− From disk- or tape-− based backups
• Challenges− Limited storage resources− Long lead-time requirementsTest 1 Test 2 Test N
Production Mirrored Copy
Dev 1 Dev NDev 2
April 11, 2023 230
Database Maintenance with Flexible Volume Clones
Benefits
• Instantaneous copies
• Low resource overhead• Easily make copies of a
production database without impacting the database− Use clones to test migrations,
apply bug fixes, upgrades, and patchesTest 1 Test 2 Test N
Production Mirrored Copy
Dev 1 Dev NDev 2Production DB
Clones
April 11, 2023 231
New Database Development Methodology
• Mirror PROD for initial copy (DR)− Mirror from and to storage system
• Clone database replicas as needed
• Create Snapshot copies of replicas for instant SnapShot Restore of working databases
PROD Test/Dev/DR Clones
Develop ● Test ● Deploy
April 11, 2023 232
Traditional Approach: Application Development and Testing
Production database 100GB
Mirror copy 100GB
Development copies 300GB
Testing copies 300GB
Total: 800GB
• 8x actual storage requirement• Time consuming • Resource overhead
Test 1 Test 2 Test 3
Production Mirrored Copy
Dev 1 Dev 3Dev 2
April 11, 2023 233
SAN Approach: Application Development and Testing
Production database 100GB
Mirror copy 100GB
Development copies 30GB
Testing copies 30GB
Total: 260GB
• Over 67% reduction in storage required
• Near instantaneous copies • Negligible overhead• Ability to have many more test
and dev copies
Test 1 Test 2 Test 3
Production Mirrored Copy
Dev 1 Dev 3Dev 2
Assumption: up to 10% change in data in the test and dev environments
more clones = higher productivity
April 11, 2023 234
Need reliablebackup and
recovery solution
Oracle Applications Lifecycle
Install
Implement
Re-organize
Upgrade
PatchDeployDeploy
Pain Points
Plan
Tune &Maintain
Solutions
Configure systems,forecast storage
accurately
Provision andmaximize utilisation
withFlexVol
Testing requiresduplicate data,
lengthy and expensiveprocess
Flexible Clone: Fast &
space-efficient data
duplication
Backup andRecovery
solution with Snapshots,
SnapShot Restore
Mirror prod.data
to test and dev system,lengthy process
Mirror data with
Storage Mirroring, ReplicatorX
Create several clones,
lengthy process, expensive
Create clones with
FlexClone, automate with
SMO
Need reliable backup
and recovery solution
Use Snapshots, SnapShot Restore,
Need reliable backup restore, and DR
solution
Automate backups, restore
with SMO, SnapMirror,
ReplicatorX for DR
April 11, 2023 235
Server Virtualisation and Storage
April 11, 2023 236
Server Virtualisation Components
• Shared storage required for operation• HA (High Availability)• VMotion – move virtual servers seamlessly
between physical servers