datacenter health check terri-lynn thayer avp/cio, computing & information services educause...
TRANSCRIPT
Datacenter Health CheckTerri-Lynn ThayerAVP/CIO, Computing & Information ServicesEDUCAUSE Enterprise Technology Conference, May 2007
Copyright Terri-Lynn B. Thayer 2007
This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.
Presentation Overview Datacenter blood pressure is rising Pre-assessment work Reliability goals Assess existing space, power,
cooling, fire suppression, & security Assessment results & options Recommendations for a renovation Key considerations in site selection
Datacenter Blood Pressure is Rising Growing Demands
Increased number of servers Increase in breadth of customer base – taking in
previously distributed computing Technology Changes
Storage growth Power/cooling needs – new boxes are smaller but they
are energy hogs and they are hot, hot, hot (10X power for a fully populated rack and 3-4X power to cool it down)
Business Resumption Concerns 24 x 7 demands Well publicized disasters - 911, Katrina Many university datacenters today lack standby power
generation Research Support
National trend for R1s to take on more support centrally Our Datacenters are Old
Pre-Assessment Work Capacity Planning and Growth Analysis
Determine a planning horizon Identify services likely to be provided from
your datacenter during that time period Business Continuity and Disaster
Recovery Objectives How long can your University operate
without a functioning datacenter? Do you have a cold or hot site? Should you consider a multiple datacenter
approach?
More Homework Research computing support decision
is fundamental Review your insurance
How much do you have Other requirements of your insurer
Get professional help with the assessment Involve your university facilities engineers Seek advice from outside professionals
who are familiar with modern datacenter design and operation
What cost/risk profile is your institution comfortable with? Reliability goals
Numerical Rankings Terminology Summary Definition
(1) Unreliable Shared building power and cooling; no generator
(2) Partially Isolated, Unreliable
Dedicated power system; shared cooling system; unconditioned power; non-redundant air conditioning; no generator
(3) Isolated Unreliable Dedicated power and cooling systems; unconditioned power; non-redundant dedicated air conditioning units; no generator
(4) Isolated Conditioned Dedicated power and cooling systems; conditioned power; non-redundant dedicated A/C units; no generator
(5) Isolated Improved Dedicated power and cooling systems; uninterruptible power system; non-redundant dedicated A/C units; no generator
(6) Isolated, Mostly Reliable
Dedicated power and cooling systems; uninterruptible power system; redundant dedicated A/C units; no generator
(7) Reliable Dedicated power and cooling systems; uninterruptible power system; redundant dedicated A/C units; generator
(8) Reliable Redundant Dedicated power and cooling systems; redundant UPS systems; redundant dedicated A/C units; redundant generators
(9) Ultra-Reliable Redundant power train; redundant cooling system; redundant UPS systems; redundant dedicated A/C units; redundant generator systems; redundant fuel system
(10) State of the Art
Redundant power train; redundant cooling system, redundant UPS systems, redundant dedicated A/C units; redundant generator systems; redundant fuel system; site hardened for weather and geographic exposures; location minimizes exposure to jurisdictional closure from hazardous spill, terrorism, or similar risks.
BRUNS-PAK Data Center Reliability Ranking
BRUNS-PAK • 999 NEW DURHAM ROAD • EDISON, NJ 08817(732) 248-4455 • Fax: (732) 248-3644 • http://www.bruns-pak.com
Datacenter Evaluation
Space Electrical System Mechanical System Fire Protection System Security
Space Square footage of conditioned
space Raised floor
Access Elevators Door size
Machine room layout Furnishings, racks, command center Is the space expandable? Is this a multi purpose facility?
Electrical/Power Considerations Source and costs Patch Panel/Power Control Units/ Power
Distribution Units Standby power
Uninterruptible Power Supply (UPS) Redundant/non-redundant Battery type (wet vs dry), capacity, and monitoring
Generator Type Power and cooling
Are the systems expandable? Delicate balance
Other Surge protection, lightning protection, grounding
Mechanical Systems Evaluation Cooling & humidity control
Chilled water – do you have a dedicated chiller?
Computer Room Air Conditioner - CRAC units - # and location
Capacity and reliability Heat Detectors
Airflow distribution Water sensors
Fire Protection Systems Detection
Smoke alarms Heat detectors Air sampling
Abatement Halon system (production banned in
1994) Full flooding clean agent system
FM-200, NAF, Inergen etc
Sprinkler system Wet Pre-action
Security
Physical access to the facility elevators & doors Caged areas and visitors Multi-purpose facility Door access system
Windows Monitoring
Closed circuit TV
Ancillary Services and Support Ancillary Services
Tape storage Secure storage/staging Paper storage Test/setup lab Printers and print support General storage room Break room
Other staffing and services which are provided from your datacenter
Machine hosting and associated SLAs others University departments groups external to the University
Assessment Results & Options Most of us will find that our
datacenters are not adequate for the anticipated growth over the next five years
Majority will identify power and cooling as the most significant issue Cooling and the power to cool will be
the number one issue Space constraints will be the
runner up
Options
Renovate Build a new datacenter Both of the above Multi-datacenter campus Outsource or Hosting
Recommendations for Renovation Implement standby power generation
capable of supporting both power and cooling
Remove ancillary services from machine room and relocate to other spaces
Trade off between space and density is a complex issue High density racking results in significant
heat and power provision issues It is general cheaper to provide more space
than to keep a small space with high density equipment adequately powered and cooled
Improve Air Flow & Circulation Provide additional space between racks
to promote air circulation Open up plenum space by relocating
cabling to overhead trays Increase height of the raised floor if
possible Consider new cooling solutions and
rack technologies – everything old is new again Chilled water is far more efficient than cool
air for heat removal Reconfigure the layout to implement a
“double hot aisle/cold aisle” configuration
Distribute high density racks
Double Hot Aisle/Cold Aisle hot aisle/cold aisle layout is
where cold air is segregated in front of equipment cabinets and hot exhaust air is expelled behind equipment cabinets. This layout eliminates the direct transfer of hot exhaust air from one system into the intake air of another system
Double A CRAC unit is located between two
hot aisles
Site Selection for a New Facility An opportunity to consider cost
Corporate world has moved their datacenters in some cases quite remote from the rest of their operation which allows them to consider
Power costs Real estate cost Labor costs
Look for a Old Supermarket Single story Single use facility Slab Few windows Lots of open space around the
building Loading docks and delivery truck
access
Other Considerations Voice and data connectivity Flooding and other weather
related issues If moving to a multi-datacenter
approach as part of a business continuity plan then consideration should be given to put the two datacenters at a sufficient distance to reduce dependence on the same power grid and to minimize weather and other regional disasters
Staffing Implications Data Center Managers will need to be
more skilled in the area of environmental issues, engineering, and server technologies
Facilities organizations may need to devote more time and specialization to cooling and power technologies related to the datacenter
Managing data center renovation or build projects will be resource intensive and may result in downtime for key services
Go Green Reduce energy costs (datacenter build
may be more expensive) Legislation Environmental concerns and
institutional plans to reduce carbon emissions
Vendor products Rack and server cooling technologies CO2 for cooling, DC power systems
Design Considerations Solar panels and wind energy Heat recycling
Conclusion Our datacenters are under significant stress If we take a close look we will find that most of
us will experience power and cooling problems in the near future. Cooling and the power to cool will be the most substantial issue we face.
There are new technologies and best practices which will provide some relief
Many of us will build new datacenters over the next five years and we should consider remote locations, outsourcing, and green IT solutions
These project will require both significant financial resources as well as IT and Facilities staff time. We may need to employ new skill sets
It is highly recommended that you engage professional assistance to evaluate your facility and to assist in renovation and new build designs
Additional Resources
www.stonesoup.org Past Meetings Spring 2006 meeting Data Center Futures Workshop
Presentations