QUESTnet – Data Centre Workshop

Mike AndreaDirector &

National Data Centre Practice Manager

May 2007ICT master planners and strategists

• ICT management services• Data centre design and development• Telecommunications and networking• Project office services• Contact centre design and development

Topics

• Strategic Directions – Overview

• AFCOM – Five Bold Predictions

• Standards and Best Practice

• Design Considerations

• Build versus Lease

• Polaris Data Centre – Overview

• QLD Model… for discussion

The Strategic Directions Group

• Five Practice Areas:– ICT management services– Data centre design and development– Telecommunications and networking– Project office services– Contact centre design and development

• Vendor independent• Design Authority on the $200m Tier III+ Polaris Data Centre at

Springfield (14,000m2 N+2, high-availability, high-security co-location facility)

• External CIO for Springfield, including responsibility for the ICT Master Plan for the new city of Springfield under development to provide– 85,000 residents– Education City (USQ Campus)– Health City (public and private hospitals)– 1 million square metres of commercial space

Strategic Directions – Data Centre Practice

• Focus on business ICT service delivery through effective data centre facilities

• We act as the ‘glue’– Between IT and Facilities– Between specialist engineering disciplines– Between the business owner and the

delivery team (financials, design, efficiency and value for money)

Strategic Directions – Data Centre Practice

Recent international investigations include:• 2006

– China:• APC InRow Cooling Manufacturing Plant (Shanghai)

• 2007– UK:

• Reuters (media) and JP Morgan Chase Data Centres (London)– Spain:

• Santander Bank Data Centre (Madrid)– Germany:

• CommerzBank Data Centre (Frankfurt) • Fiducia Data Centre (Karlsruhe) – under construction• Osram Plant (Regensburg)• Piller Power Systems (Osterode)• MTU Diesel Engines (Friedrichshafen)

AFCOM - Predictions

• AFCOM is a leading association supporting the educational and business development needs of data center management, executives and vendors around the globe.

Report • “Five Bold Predictions for the Data Center that will

Change Your Future”, March 2006

Source: www.afcom.com

Five Bold Predictions for the Data Center that will Change Your FutureAFCOM – Data Center Institute, March 2006

AFCOM – Predictions Five Bold Predictions for the Data Center that will Change Your FutureAFCOM – Data Center Institute, March 2006

AFCOM – Predictions

• AFCOM discusses Mainframe skills…• In terms of large-scale design and construction (not retrofit) of data

centres in Queensland, the market is very small, and the pool of experienced / skilled design teams is even smaller

• Most businesses are structured with– IT groups targeted with business based ICT delivery… not design and build

of data centres

– Facilities teams focused on building maintenance, contracts and change management… not design and build of data centres

• Data centre design and build projects don’t occur that often, therefore in most cases, there’s been no need to invest in in-house skills

• Selective out-tasking of skills?

Five Bold Predictions for the Data Center that will Change Your FutureAFCOM – Data Center Institute, March 2006

AFCOM – Predictions

• If this is correct… it’s only 3 years away• Options include

– Upgrade or extend existing facilities– Retrofit an existing building with a new data centre facility– Relocate higher density servers / platforms to a 3rd party facility– Relocate the entire facility to a 3rd party facility– Others?

• What does the budget (CAPEX & OPEX) allow?• Has anyone forward planned budget increases?

– Power charges are likely to increase– Power and cooling density is increasing– “Power consumption has increased 7-fold in the last 7 years”

Five Bold Predictions for the Data Center that will Change Your FutureAFCOM – Data Center Institute, March 2006

AFCOM – Predictions

• The water crisis has recently seen power generation output of some Queensland power stations curtailed

• Queensland is growing… not just in the South East• The investment required in the energy network to continue

to meet residential growth and business demand is huge• Public holidays are some of the largest peak demand

periods for energy providers• Some commercial building owners are refusing, or are

unable, to upgrade mains power and water feeders in order to support tenants data centres

Five Bold Predictions for the Data Center that will Change Your FutureAFCOM – Data Center Institute, March 2006

AFCOM – Predictions

• Virtualisation… consolidation• Internet delivery of services and applications

– AFCOM notes IPv6

• Mobility… both devices and users– Increase in broadband and wireless internet use

• Virtual office working– Working from home, vehicle or remote office

• The data centre is at the heart…

Five Bold Predictions for the Data Center that will Change Your FutureAFCOM – Data Center Institute, March 2006

AFCOM – Predictions

• Business disruption could be caused through– Loss of power, cooling or communications– Fire or water damage– Natural disaster– Bomb threat / terrorism– Staff error / sabotage– Data loss– Security breach

• System availability of some server hardware can be less than the data centre itself

Five Bold Predictions for the Data Center that will Change Your FutureAFCOM – Data Center Institute, March 2006

Standards and Best Practice

There are plenty of Standards…

But what is ‘best practice’?

Standards and Best Practice

There are plenty of Standards…

But what is ‘best practice’?

Is it the best way to meet local standards?

Standards and Best Practice

• ANSI/TIA-942-2005 (approved 12 April 2005)– Telecommunications Industry Association – Telecommunications

Infrastructure Standard for Data Centers• Uptime Institute (Tier I to IV)

– Tier I (99.67%) to Tier IV (99.99%) availability– Requires consistent application of Tier concepts to 16 critical

subsystems • ASHRAE (American Society of Heating, Refrigerating and Air-

conditioning Engineers)• DSM (Defense Services Manual – Australian Federal Government)• EPA (Environmental Protection Agency) – Qld State Government• BCA (Building Codes Australia) and Local Council building

regulations and approvals

• Includes a mix of US, European and Australian ‘standards’– Different voltages, codes, and interpretations

Standards and Best Practice

• Formal due diligence reviews can get tied up comparing US standards against Australian building codes– EPO is a prime example (not desired– or required – on data

centre floors for fire management)– EPO in plant areas for electrical safety

• Fire separation of infrastructure and fire zones– Fire ratings of walls and doors (circular mantraps are a

problem)– Size of zones impacts fire suppression system selection

• Security has significant implications in the design of the physical building– Distance between redundant access points– Loading bay access control & management– Fire escape doors (e.g. quad locking, steel encased)

Standards and Best Practice

• Availability of a facility– How is it measured?

• Mixed levels of redundancy across a facility• What if the ICT infrastructure is single corded?

– Does it account for all components of the service delivery chain?

• Electrical, mechanical, security, fire suppression, hydraulics, telecommunications

• People, support and maintenance

– Where are the single points of failure?• Have you looked at the building riser lately?• Mains power and water service connections• Lifts and other access points

Design Considerations

Just a few items of note…

Design Considerations

• Most obvious is Power and Cooling• Mains supply to the building (and/or suburb) for both

power and water• Weight - floor slabs (if not on ground) typically

constructed in the range of 100kg/m2 to 400kg/m2

– Some 19” racks known to weigh over 1000kg (approx 1600kg/m2, with point load of 400kg)

– Batteries for UPS systems

• Floor to ceiling height, including height of raised floor (if you have one)

• Fire separation (diesel generator, UPS, batteries, transformers, switchboards) and associated fire suppression

Design Considerations

• Lead times, local support/maintenance and supply– Not just the obvious, big items… gas fire suppression, switch

boards, distribution boards

• Skills shortage… it’s real… there is a massive amount of infrastructure being built in Queensland– It is difficult to secure consultants and trades with experience

in data centres

• Ensure your design team understands the concept of a data centre, high-availability and high-density computing

• Services co-ordination, it can be a stretch to fit everything in the same riser or wall/floor penetration

• 415v … or 400v ?

Design Considerations

• Location and climate dictates some solutions• How are Acceptance Tests conducted?

– Initial build, and then during normal maintenance routines?• Pay as you grow

– UPS, diesel generators, chillers, cooling towers, transformers, switchboards, racks

– Cabling (power and comms), cable trays– Some keys to success

• Ensure the building itself doesn’t cause future fit-out problems – risers, diesel tank sizes and location

• Mains services (water and power) should be sized for ultimate load (cost of change later is too high and may not be possible)

• Design with concurrent maintainability and live upgrades in mind• Have the phase 1 project team document the process for future

fit-out

Build vs Lease

What’s your preference…

Build vs Lease …Business Metrics

• Does an investment deliver and/or reinforce core business – what is the return on the investment?

• Efficiency– Floor Space Ratio: ratio of plant room floor area vs

raised floor area… a direct result of density and appropriate infrastructure sizing

– Selection of redundancy level, UPS, cooling systems, fire suppression solution, and diesel generators all impact efficiency

– ‘Ride-through’ capacity to limit false diesel starts and loss of cooling capacity through brown-outs

Build vs Lease …Business Metrics

• Financials – CAPEX and OPEX– Lifecycle replacement costs and timeframes– Rent of the area (raised floor and plant) and/or

return (income) on the construction cost

• Sustainability & Availability– Ability to keep operating during mains power and/or

water loss– Total site availability across all components

• Risk– Who bears the risk of repairs, replacements,

capacity upgrades and/or non-performance

Build vs Lease …build (or retrofit)

• Considerations regarding the investment– Is it, or will it reinforce, core business?– Could capital (short term investment) provide a higher return spent

elsewhere?– Are the ongoing operational expenses budgeted and committed… for

the next 15 years?• Getting the right team together is key

– Electrical, mechanical, hydraulics, architectural, civil, construction, security, ICT/BMS, other…

• Complying with formal/traditional procurement processes can create issues– Cost of an experienced design team can be high– Long lead times (from order to delivery)– Fixed priced contracts may have exclusions relating to fluctuating

copper, stainless steel, concrete and water prices– Is the project big enough to ensure the right companies respond?

• Can the building cope?• Full operational costs are yours and can’t be shared

Build vs Lease …lease

• Considerations regarding the investment– All costs are OPEX, capital free for use elsewhere– Are the ongoing operational expenses budgeted and committed… for the next

15 years?– Possible additional carriage costs

• Reliant on a 3rd party to ensure operational performance, management and maintenance of the facility

– The 3rd party isn’t subject to formal procurement processes– Service levels available, allowing IT group to focus on IT service delivery

• In a co-location facility, the operational costs are spread across the total leased raised floor area

– Support staff, maintenance, security and operations– Consumables including diesel, water and gas suppression storage– Upgrades, repairs and replacements are the 3rd party’s responsibility

• Benefits may result through larger scale – Mains services redundancy– Carrier diversity– Higher redundancy and/or longer sustainability – Concurrent maintainability

• Pay for what you use

Polaris Data Centre

Overview…

Polaris Data Centre

A $200m private investment in Queensland’s ICT industry

Lead by Queensland companies

• Springfield Land Corporation

• Strategic Directions

• Suncorp

• Thiess

• Deicke Richards

• Multitech Solutions

• BRW Enterprises

Polaris Data Centre

7000m2 of raised floor in a 14000m2 purpose designed and built facility

• Tier III+ (N+2)

• 800W/m2 to 1500W/m2

• 3 data centre levels

• 2 plant levels

• Design nominal 99.999% availability

• N+N power distribution to the rack

• N+1 Inert gas fire suppression store

• 48 hours sustainability

Polaris Data Centre

• Concurrent maintainability• High security (24x7 security, DSM compliant intruder resistant

perimeter, full CCTV coverage, mantraps and biometric access)• Scalable, flexible, pay-as-you-grow modular system from

800W/m2 to 1500W/m2

• Slab loading to 1600kg/m2, with raised-floor to 1500kg/m2

• Piller Rotary UPS solution (UBDT) with coupled MTU diesel engine, and additional Rotary UPS powerbridge (UBT) for chillers

Polaris Data Centre

Polaris Data Centre – Key Learnings

• Everyone you talk to is a data centre design expert

• The IT and construction industries are poles apart

• A data centre is designed to fit within the physical boundaries presented – hence no data centre is the same

• Diesel engines require a lot of air – for cooling and combustion

• External smoke entry via fresh-air intake

Polaris Data Centre – Key Learnings

• Investigate the power grid with Energex– What appears simple can turn out very complex and

take a long time– Network upgrades and investment requires long term

planning (transformers, feeders etc)• The supply of water is critical (potable &

recycled)• Some equipment lead times can be up to 2 years• Meeting EPA requirements can be expensive

– Acoustics, emissions, diesel storage, waste disposal • Size does create efficiency… up to a point• 3000W/m2+… maybe in ‘Polaris 2’

Polaris Data Centre – Core Design Team

* Queensland Headquartered Companies

COMPANY ROLE FIRST INVOLVED

Springfield Land Corporation Project Owner 2003

The Strategic Directions Group Design Authority (& ICT Systems) 2004

Deicke Richards Architects 2004

Thiess Design & Construction Management 2006

Webb AustraliaElectrical, Communications and Security (& ICT Systems)

2006

MultiTech Solutions Mechanical, Dry Fire & BMS 2006

Robert Bird Group Civil 2005

BRW Enterprises Hydraulics & Wet Fire 2006

Clifton Coney Group Project Management 2005

APC Strategic Alliance Partner, Industry and R&D 2005

QLD model – for consideration

• Universities are– undertaking reviews – planning and/or performing upgrades – Considering the build of new facilities (retrofit)

• Why not pool the resources to build something specific to University needs?

• Dedicated, purpose-built data centre facility as DR, secondary or primary site

• Collaborative model

QLD Model – issues & discussion

• Who – Pays what?– Owns the facility?– Controls and manages the facility?

• How– Are telecommunications charges considered?– Are consumption charges considered? – Are different Uni requirements met?– Big should it be… what Tier?

QLD Model – issues & discussion

• Does the university funding / budget model support and/or allow for leasing data centre space?

• Private investment in developing such a facility is based on financial models outside the traditional ICT business case

• Springfield as a location– Energex Strategic Alliance with dual zonal substations (one

online, second planned)– Water connections – potable and recycled– Telecommunications – multi-carrier, with dark-fibre to Brisbane– Access to developed land with public transport, shopping centres

and easy access to Brisbane airport– Natural events are low risk– Close to experienced consulting, design and construction team

European Data Centres

Summary

• Strategic Directions – Overview

• AFCOM – Five Bold Predictions

• Standards and Best Practice

• Design Considerations

• Build versus Lease

• Polaris Data Centre – Overview

• QLD Model…

Data Centres… at the very heart of IT

Thank You

Mike AndreaEmail: mandrea@strategicdirections.com.au

Phone: 1300 786 566

Mobile: 0410 551 080

Web: www.strategicdirections.com.au

Polaris Data Centre: www.strategicdirections.com.au/polaris