Storage Trends:DoITT Enterprise Storage

Gregory Neuhaus – Assistant Commissioner: Enterprise Systems

Matthew Sims – Director of Critical Infrastructure

•Overview

•Traditional Storage Offering

•Current Storage Offering

•Emerging Technologies / Future Offerings

Overview: Environments

• Enterprise Systems – Mission Critical Storage Solutions provided as agency-wide service offering.

• Platforms include: Solaris, Linux, Windows, AIX, VMware, Mainframe

• Environments Included are:

• 311 – City Services Call Center

• NYC.gov – Public Web Portal

• NYCSERV – Department of Finance Revenue Application

• Exchange – Citywide Email Solution

• Cityshare – Employee Internal Web Portal

• GIS – City Geographical Information Services

• Interwoven – Content Deployment System

• Analytics – Enterprise Reporting System

Overview: Data Storage Highlights

• Disaster Recovery Solutions – Multi-site redundancy

• Data Replication – Block and File level for high availability

• DWDM Fiber Channel connection between three sites

• 311 Call Center Data

• Data Types: Flat Files, Block Devices, Exchange, Oracle, SQL

• GIS Citywide Geographical Imaging

• Silo approach

• Most systems – static-size internal disk

• Dedicated direct attached array for additional needs

• Limited “shared array” solutions

• Standalone arrays

Enterprise Storage: Traditional Offering (prior to 2004)

Traditional Storage Offering: Issues

• Rapidly growing data sets• Risk of data loss• Multiple array failures – risk of data loss• Extensive downtime due to disk failure• Extensive maintenance windows• “Ad-Hoc” replication, file copy strategies for DR• Inability to forecast and meet storage demand• High cost due to dedicated storage procurements

Solution: Enterprise Storage Area Network (2004 – 2006)

• Ability to quickly grow data sets• Implement Shared Fabric• Centralize critical storage• Enhanced capacity planning and provisioning• Lower overall costs• Increased security, manageability

• Storage:•Hitachi Storage Array•NetApp Filers•Shark Storage Array•SATA Storage Array

• Data Protection:•Veritas Netbackup•Traditional Tape Backup and Archiving•Offsite Vaulting•StorageTek Tape Silos

Overview: Technology Solutions

• Fabric Architecture – Brocade 12k, 48k, 4900• “Core-Edge” Topology. Bandwidth 1 to 4 Gbps• High performance, redundant• Robust, highly reliable / scalable• Established Fibre Channel Infrastructure• Based on best practices and industry experience• Increasing demand for more bandwidth• Expanding requirements for greater port density• Greater need for Fabric / Storage segregation

Current Offerings: SAN / Fabric

Enterprise Storage: Hitachi 9990 / 9980 Series

• Consolidated Approach for Data Center Storage

• Highly Integrated Service Offering

• Reduced Cost per unit of Storage

• Enhanced and streamlined supportability

• Efficient Utilization of Costly Disk Resources

• Simplified Provisioning, Virtualization, and Data Migration

• Enhanced Accessibility to Data Sets for SOA initiatives

• Transition to Tiered Storage Model

• Continue “Core” Enterprise-class Storage - Mission-critical data

• Implement SATA technologies for 2nd Tier Data

• SAN Segregation allows N-level Storage Tiers

• Facilitates data migration, Lifecycle Management

Enterprise Storage: Tiered Storage

Enterprise Systems: NetApp Filers

• Windows Systems – Due to unique replication and storage needs particular to the Exchange environment, NetApp Filer Technology will be used to house the Windows environments.

• This approach allows Exchange to run in a storage environment that has been optimized and supported by Microsoft.

• Environments Included are:

• Exchange – Citywide Email

• SQL Server – Microsoft Database Solutions

• Windows Hosting – Other Application Systems hosted on Windows

Security

• Fabric Security – Brocade Secure OS under implementation

• Access Control Lists

• Restricted Access

• Centralized and Consolidated Storage Administration

• Traditional tape backup & recovery• Highly robust, reliable• Mature technology• Based on best practices and industry experience• Increasing need for faster backup, recovery times• Need to minimize backup windows, increase data

availability

Current Offerings: Backup & Recovery

Current Offerings: Backup & Recovery

Citynet

B

SunFire V240

Sun

SunFire V240

Sun

PresentMetrotech 12/22/2005

DoITT NetBackup Architecture

SunSunFire V440

SunSunFire V440

SunSunFire V440

SunSunFire V440

Catalyst 6500 SERIES Catalyst 6500 SERIES

B

NetBackupMaster Servers

Media ServersNetBackup

Media Servers

SAN Storage Array

SAN Switches

StorageTekACSLS Servers

Powerhorn 9310 Silo

9940B Tape Drives(Total of 18)

San Media Servers(Unix/Windows/etc)

NetBackup Clients(Unix/Windows/etc)

LINK REFERENCE

========== Ethernet/WAN

========== Fibre Channel/SAN

========== Private Network

========== SCSI/Control

NetworkSwitches

Virtual Tape Library (VTL)

• Faster Backup and Restore Times• Reduced Application Impact• Smaller Backup Windows• Potential for Continuous Backup and Recovery

Windows• Allows for Disk-level Replication (DR strategy)

Planned Expansion: Backup & Recovery

Enhanced Offsite / DR Capability

• Cluster-enabled Backups• “Offsite capabilities to FISA”• Immediate availability of DR Backup Environment• Immediate availability of Offsite Tape / Disk• Integrate with VTL to enhance Recovery Time

Planned Expansion: Backup & Recovery

• Need for Enterprise-scale Solution•Must be heterogenous, vendor neutral•Allow end-to-end management•Allow end-to-end problem isolation

• AppIQ, EMC offer promising solutions (among others)

•Reduces “Shelf-ware” effect of disparate tools•Information Lifecycle Management

Emerging Technologies: Storage Resource Management

• Optimized Provisioning•Facilitates tiered storage model•Multi-protocol access points: NAS, FC, iSCSI•Multiple vendor offerings in this space

• Technology in infancy stage•Multiple approaches:

• Appliance approach• “Core” approach

Emerging Technologies: Virtualization

Gregory Neuhaus

Assistant Commissioner – Enterprise Systems

Office: 718-403-8633

Email: gneuhaus@doitt.nyc.gov

Matthew Sims

Director of Critical Infrastructure

Office: 718-403-8129

Email: msims@doitt.nyc.gov

Contact Information