Welcome

Product Training – CRAC/CRAH/High-Density

General topics - Data Center…

Availability

Safety

Energy Efficiency

Transparency

3http://www.stulz.de/en/newsroom/blog/the-data-center-standard-din-en-50600-in-brief-563/

4

Air-Conditioning requirements in a Data Center

“Correct“ temperature and “correct“ airvolume in front of the IT-equipment –24 hours a day and 365 days a year –all that as energy-efficient and reliableas possible and in dependence of the

actual heat load of the servers.

5

Air-Conditioning requirements in a Data Center

http://www.stulz.de/en/newsroom/blog/data-center-temperature-control-1467/

6

Low Density

< 7kW / Rack

Medium Density

7-15kW / Rack

High Density

> 15kW / Rack

Heat loads

ASHRAE TC9.9-2013

Recommended supply air temperature range:

18°C up to 27°C

Allowable supply air temperature range:

15°C up to 32°C

Requirements of the IT-equipment

7

Generally speaking, the required supply air temperature must be reached at a height of 1,5m in front of the rackThe delta T across the rack is typically between 10K and 15KThe air outlet temperature is normally not important for the IT equipment

∆T – Airside temperature difference

1,5m

18 - 27°C 28 - 42°C

Server Inlet Temperature

Server Outlet Temperature

8http://www.stulz.de/en/newsroom/blog/delta-t-91/

9

Server Inlet = 24°C

Return Air = 39°C

Supply Air = 24°C

Server Outlet = 39°C

𝛥𝛥T = 15K 𝛥𝛥T = 15K

∆T – Airside temperature difference (ideal)

SHR = Sensible Heat Ratio (best achievable: 1,0)

Target: Sensible Capacity = Total CapacityNo unwanted, efficiency decreasing dehumidification

SHR

SHR=Sensible Capacity

Total Capacity

10

EER = Energy Efficiency Ratio (the higher, the better)

Lowest total power input for a specific cooling capacity

EER

EER=Total Capacity

Total Power Input

11

AER = Airflow Efficiency Ratio (the lower, the better)

AER Unit: W / (m³/h)AER applies at specific airflow and external static pressure

AER

AER=Fan Power Input

Airflow

12http://www.stulz.de/en/newsroom/blog/aer-351/

PUE = Power Usage Effectiveness (the lower, the better)

KPI = Key Performance Indicator (EN 50600-4-2, “Green Grid”)

NO BENCHMARK!NO EFFICIENCY FACTOR!

PUE (pPUE, iPUE, dPUE)

PUE= Total facility power

IT power

13http://www.stulz.de/en/newsroom/blog/powerusageeffectivenesspueppue-1045/

The PUE calculation is a Data Center specific measure to understandand improve the energy consumption The PUE is based on MEASUREMENTS of a period of at least 12 months and therefore a yearly average value!

PUE (pPUE, iPUE, dPUE)

14

Source: EN 50600-4-2

PUE = (1.450 + 450 + 400) /(1.500) = 1,53

IT Energy

Cooling, Air-Conditioning, Lightning, Controls,…

Electricity

Natural gas

Block heatand power

plant

Elec. facilities

ElectricityOther EnergyMeasuring point

PUE “derivatives”:

“Interim” PUE = iPUE: period of measurements less than 12 months

“Partial” PUE = pPUE: consideration of a subset of the infrastructure

“Draft” PUE = dPUE: theoretical PUE based on a “real” DC planning

PUE (pPUE, iPUE, dPUE)

15

Products & Solutions

Mission Critical Cooling

17

STULZ is the world's leading solution provider of energy efficient temperature and humidity management technology, specifically for mission-critical applications.

Our vision

18

Products

19

20

IHS “Data Center Cooling Report“ 2016 – Cooling Market (Revenue)

21

STATISTICS

Chip/Server Row/rack Perimeter AHU Containment

2015

63%

9%

1%

13% 14%

Chip/Server Row/rack Perimeter AHU Containment

2020

57%

1%

14%15%

13%

EUROVENT Market Intelligence(Sold unitsEMEA)

22

STATISTICS

-4%

6,527

32,866

10,758

6,831

2013 2014

35,154

2015

DX

CW

15,581

34,837

17,215

TLC

2016

10,708

6,914

2012

35,655

16,307

11,481

7,867

2011

36,958

17,396 16,877

11,44611,143

8,419 29,746

9,684

13,592

6,470

EUROVENT Market Intelligence(Capacity EMEA)

23

STATISTICS

2013 2014 2015

DX

CW

201620122011

8,419

580k 550k 590k 550k 520k 500k

730k 650k790k660k

760k 730k

ROOM COOLING – Products

24

ROOM COOLING - CyberAir3(PRO)

CyberAir 3(PRO)

• Cooling Capacity ~ 20kW – 250kW• EC-fans• 7 different cooling systems• 7 unit sizes• 4 different airflow directions (Downflow, Upflow,

Raised Floor, High)• Different compressor types (Scroll, EC-Scroll,

tandem, one circuit, two circuits)• Three different refrigerants available

25

CyberAir 3(PRO) – COOLING SYSTEMS

Air-Cooled System (A, AS)

Water-Cooled System (G, GS)

Chilled Water System (CW)

Dual-fluid System (ACW, ASCW)

Dual-fluid Chilled WaterSystem (CW2)

Dual-fluid System (GCW, GSCW)

Indirect Free CoolingSystem

(GE, GES)

Cooling Systems – Dual-Fluid Example

Standard daytime operation:

Chiller on

Office A/C on

CW-circuit in ACW unit active

Compressor and condenser off

Night-time, weekends:

Chiller off

Office A/C off

DX-circuit in ACW unit active

Compressor and condenser on

http://www.stulz.de/en/newsroom/blog/dehumidification-in-data-centers-when-using-cw-units-at-high-temperature-levels-362/

Indirect Dynamic Free Cooling with GE and GES units.

Automatic choice of four different working modes• DX (compressor cooling)• Mix-Mode (Compressor and Free cooling)• EFC (Extended Free cooling)• FC (Free cooling)

28

Cooling Systems – GE/GES

DX

MIX

EF

C

FC

Regions for Free Cooling

29

Unit Sizes CyberAir 3(PRO)

Unit size 1 Unit size 3Unit size 2

950 x 890 x 1980mm

950 x 890 x 2495mm

Number of fans: 1

All unit types/cooling systems

1400 x 890 x 1980mm

1400 x 890 x 2495mm

Number of fans: 1

All unit types/cooling systems

1750 x 890 x 1980mm

1750 x 890 x 2495mm

Number of fans: 2

All unit types/cooling systems

30

Unit Sizes CyberAir 3(PRO)

Unit size 4 Unit size 5

2200 x 890 x 1980mm

2200 x 890 x 2495mm

Number of fans: 2

All unit types/cooling systems

2550 x 890 x 1980mm

2550 x 890 x 2495mm

Number of fans: 3

All unit types/cooling systems

31

Unit Sizes CyberAir 3(PRO)

Unit size 7 Unit size 8

3110 x 980 x 1980mm

3110 x 980 x 2495mm

Number of fans: 4

All CW-units, ASR/ALR…DX

3350 x 980 x 2495mm

Number of fans: 4

ASR/ASH…CW/CW2 only

ROOM COOLING – Airflow directions

32

DOWNFLOWASD

UPFLOWASU

ROOM COOLING – Airflow directions

33

RAISED FLOORASR

HIGHASH

Airflow directions / Available unit types

34

AAS

GGS

GEGES

CW CW2 ACWASCW

GCWGSCW

ASD X X X(Size 1)

X X X X

ALD - - X - - - -

ASU X X X(Size 1)

X X X X

ALU - - X - - - -

ASR X - X X X - -

ALR - - X - - - -

ASH - - - X - - -

EC-fan technology for energy efficient air flow in all operating points

EC-compressor technology for infinite compressorcontrol with maximum efficiency and precisetemperature control

Maximized heat exchanger and filter surface

Quiet running, long life, maintenance friendly

CyberAir 3(PRO) – Main Features

Compact dimensions

High-efficiency zig-zag filters plus filter controlmanagement

Electronically controlled expansion valves

Different coil lay-outs to meet all required operating points

Stulz C7000 microprocessor control of all activecomponents

CyberAir 3(PRO) – Main Features

Latest EC-motor, electronics and fan blade technologyFan blade made out of fibre-glass reinforcedmaterial• Aerodynamically optimized profiles• Noise optimized design• Economization of weightFans especially developed for Stulz

CyberAir 3(PRO) – Energy Saving Features

Lowest fan power consumption in fulland part loadVery low noise emissionMaintenance freeExcellent performance data (EC-Directive 2005/32/EC = Energyrelated products directive „ErP“)

CyberAir 3(PRO) – Energy Saving Features

Stulz

ACECErP 2013ErP 2015

EC-Compressor:Maximized efficiency especially in part load operation and Mix-modeConstant supply air temperature Integrated soft-startFast and precise reaction accordingto the actual heat loadLong compressor life time becauseof continous compressor operationwithout start- and stop-times

Page 39

CyberAir 3(PRO) – Energy Saving Features

Airflow optimisation:Aerodynamic design for lowest airside pressure dropPerfect air flow through unit, filtersand coils from air inlet to air outletSize optimized heat exchangers, filters and fansOptimization of fan positioning

CyberAir 3(PRO) – Energy Saving Features

High Performance Filtration:Use of „high-performance zig-zag-filters“ with large filter surface for reduced fan power consumption andhigher net sensible cooling capacityFilter control management

• Integrated intelligent airflow control• Adaption of fan speed in dependence of actual filterpressure drop

• Automatic indication of required filter change• Protection against too low airflow and „hot spots“• Constant airflow at any time and longer filter life-time

CyberAir 3(PRO) – Energy Saving Features

Reduction of internal air pressure lossesFan position ensures less turbulence and misdirection of the airBoth the above result in reduced fan consumption

The underfloor fan section (CW)

ASD 2080 CW (coil A):

Size 7, 35.000 m³/h, ESP: 20Pa• Fan power: 5,5 kW• AER: 0,16

ASR 2060 CW (coil A):

Size 7, 35.000 m³/h, ESP: 20Pa• Fan power: 4,0 kW• AER: 0,11

42http://www.stulz.de/en/newsroom/blog/cracs-with-underfloor-fan-section-569/

Reduction of internal air pressure lossesFan position ensures less turbulence and misdirection of the airBoth the above result in reduced fan consumption

The underfloor fan section (DX)

ASD 712 AS:

Size 4, 20.000 m³/h, ESP: 20Pa• Fan power: 4,1 kW• AER: 0,20

ASR 732 AS:

Size 4, 20.000 m³/h, ESP: 20Pa• Fan power: 3,6 kW• AER: 0,18

43http://www.stulz.de/en/newsroom/blog/cracs-with-underfloor-fan-section-569/

44

CW-Standby-Management

ON ON

ON ONON

ON

ONONON

ONOFF OFF

6x ASR 1710 CW (coil A) Passive redundancy (4 units ON, 2 units OFF)

Airflow: 4x 32.000m³/h = 128.000m³/h Fan power: 4x 4,5kW = 18,0kW Total sound pressure level: 63dB(A) Waterside pressure drop: 70kPa

6x ASR 1710 CW (coil A) Active redundancy (6 units ON)

Airflow: 6x 21.300m³/h = 128.000m³/h Fan power: 6x 1,6kW = 9,6kW Total sound pressure level: 55dB(A) Waterside pressure drop: 40kPa

http://www.stulz.de/en/newsroom/blog/standby-management-for-cw-units-77/

Differential Pressure Management… Permanent control of fan speed in

accordance of heat loadQuick and precise reaction on even smallest

changes Always provision of the correct airflow No interference between pressure and

temperature Always constant supply air temperature and

correct air volume

45

Option “Differential Pressure Management”

Pressure Independent Control Valves… combine the features of a dynamic pressure

control valve and a 2-way-control-valve ensure a steady and high valve authority ensure a precise and stable operation under

all load conditions ensure an easy installation and quick

hydraulic balancing reduce the commissionig and start-up times contribute to reduce energy costs of the

pumps

46

Option “Pressure Independent Control Valve”

http://www.stulz.de/en/newsroom/blog/use-of-pressure-independent-control-valves-in-cw-units-1409/

47

Option “Direct Free Cooling (FCP)”

Free Cooling Plenum (FCP)… Possibility of Direct Free Cooling in combination with

CRAC/CRAH No extra space needed (both dampers on top of the unit) Three different working modes:

• DX/CW mode• MIX-Mode• Free Cooling Frost protection included

CyberAir 3(PRO) – Other Features/Options

CyberAir 3(PRO)

• Raised floor stands• Different filter classes, different coils• Mechanical devices (ducts, dampers, …)• „Filter-Control-Management“• Differential Pressure Control• CW-Standby-Management• Direct Free Cooling (FCP)• Display of airflow, EER, cooling capacity• …

48

CyberAir 3(PRO)

• Dual power supply• UPS buffer controller• Pressure Independent Control Valves• Re-heat, electrical, proportional, LPHW,

refrigerant• Steam humidifier, control of external ENS• BMS-connections• Monitoring (WIB)

http://www.stulz.de/en/newsroom/blog/cw-units-heat-exchanger-62/

…and many more…

49

CyberAir 3(PRO) – Competition

http://www.stulz.de/en/newsroom/blog/cooling-capacity-how-to-compare-apples-with-apples-1400/

CyberAir 3(PRO) – Competition

50

At least 40 competitors producing CRAC/CRAH units worldwide that can begrouped into 3 different parts:• “Large Global Players“: Vertiv, Schneider, STULZ• “Mini Global Players“: Climaveneta, Airedale, GEA Denco, Silent-Aire, Huawei• “Local Players“: EMEA: WKT, HiRef, Tecnair, Aermec, RCGroup,…

APAC: Daikin, Citec, AIRSYS,…LAM: Data Aire,…

Three major tendencies are visible:• Main competition (Vertiv, Schneider, Airedale) is focussing on systems• Brochure data is published with higher return air and water temperatures• More and more competitors with only little information “Smaller“ Italian manufacturers are publishing technically questionable data and are

still focussing on the product Tendency to capacity optimization and higher capacity per footprint

ROOM COOLING – CyberAir 3(PRO) USPs

Flexibility: Airflow directions, cooling systems, refrigerants, coils, valves…

State-of-the-art operating temperatures (air-side and water-side)

Best “overall“ performance data available on the market (EER, AER)

Only manufacturer with a full DX-range with fan section underfloor

Filter-Control-Management, Differential Pressure Control

World-wide number one indirect free cooling system

51

ROOM COOLING – MiniSpace EC

MiniSpace EC

• Cooling Capacity 5kW – 25kW• For smaller Data Centers• 3 Cooling Systems (A, G, CW) • Upflow, Downflow, Displacement• EC-fans• Direct Free Cooling possibility with EcoCool

52

ROOM COOLING – MiniSpace EC

MiniSpace EC

Precise, reliable and cost-effective air-conditioning solutions Compact size – MiniSpace EC requires very little

floor space Easy integration in small to medium-sized server

and technology rooms Installation in existing server rooms with no

problem

53

CyberLab – ASU 211 AL

• Cooling Capacity up to 20 kW• “0-heat-load-unit“• According to German hygienic standard

VDI6022• Very precise controls• Easy access to all components• EC-fan, EC-compressor• Standardized solution

ROOM COOLING – CyberLab

54

Products

55

HIGH DENSITY – Products

56

HIGH DENSITY – CyberRow

• Targeted cooling of high(er) density racks• Independent of rack manufacturer• To be positioned directly between or next to the racks• Suitable for applications with or without raised floor• Suitable for use in open or contained hot-aisle/cold-

aisle applications• Infinitely variable cooling capacity• Cooling Capacity 20 kW – 58 kW• 3 unit sizes (300/400/600mm)• 4 Cooling Systems (A/AS, CW, GS, GES)

57

HIGH DENSITY – CyberRow

Special airflow deflectors between the fansplus airflow straighteners at the discharge:• To increase the maximum fan throwing

range• For an even and horizontal airflow• For a reduced fan power consumption Thanks to the horizontal (lateral) air

discharge, CyberRow creates uniform andclose-contact airflow directly in front of theracks.In most cases therefore a constructional

partition of hot aisle and cold aisle can beavoided.

58

Air velocity

HIGH DENSITY – CyberRow Cooling Systems

59

• Chilled Water (CW)• CRS/L 210/320/560 CW (300/400/600mm)• Proportional controlled 2-way-valve• 2 different coils per size

• Air-Cooled (A/AS)• CRS/L 211/251/361 A/AS (300/400/600mm)• Either stepless controlled EC-compressor or

on/off compressor with hot-gas-bypass• R410A or R134a• Oil separator for AS-units

HIGH DENSITY – CyberRow Cooling Systems

60

• Glycol-cooled (GS)• CRS/L 251/361 GS (400/600mm)• EC-compressor, R410A• Plate condenser and 2-way-valve

• Indirect Free Cooling (GES)• CRS/L 211/251/361 A/AS (400/600mm)• EC-compressor, R410A• The only (feasible) indirect free cooling

row-cooling unit available on the market

HIGH DENSITY – CyberRow Controls

61

6 temperature sensors for a close andenergy efficient cooling control in 3 independent horizontal zones (full load andpart load)• 3 supply air temperature sensors• 3 return ar temperature sensors 2 diverse set-points (parameter) for a fast

and precise reaction on all requirements• Fan speed control in dependence of temperature

difference between return and supply air• Compressor speed (valve opening, hot-gas-

bypass control) in accordance of supply airtemperature

100%

0%

Setpoint supply airMax (TIx1, TIx2, TIx3)

CW valve, compressor speed

16%

max

min

Setpoint delta TTRx - TIx

Fan x speed

HIGH DENSITY – CyberRow Airflow

62

Horizontal discharge forOpen Aisle Applications

Front discharge for ColdAisle Containment

Horizontal discharge forHot Aisle Containment

Increasing of efficiency/redundancy in most cases only possible with containment systems.

63

HIGH DENSITY – CyberRow Dimensions

Two different unit depths to meet all requirementsR

AC

K

Cyb

erR

ow

CR

Sa) 1.175/1.200mm (CRS = Standard) b) 1.375mm (CRL = Large)

RA

CK

Cyb

erR

ow

CR

S

RA

CK

Cyb

erR

ow

CR

L

Ap

pro

x. 1

.000

mm

Ap

pro

x. 1

.200

mm

Ap

pro

x. 1

.200

mm

64

HIGH DENSITY – CyberRow Options

Supply air discharge through the front (400/600mm)Side flow – left or right side onlyDual power supply2kg/h steam humidifier (400/600mm)Electrical re-heat (400/600mm)Humidity sensorCondensate pumpWater detection systemSupply air sensors to be positioned directly in front of the rackConnection to BMS systems

In general the market for “Row-Cooling“ can be devided into three different chapters:• “Stand-alone“ row-cooling CW• “Stand-alone“ row-cooling DX• “Close-Loop“ cooling solutions (e.g. Micro DC) with similar cooling equipment 3 main competitors: Rittal, Schneider (APC), Vertiv (Knürr)

• Rittal is focussing on “Close-Loop“ and “stand-alone“ CW• Schneider (APC) is focussing on “Hot-Aisle-Containment“ plus “stand-alone“ CW• Vertiv (Knürr) is focussing on “Close-Loop“ CW and “stand-alone“ CWOther competition either “Close-Loop“ (e.g. Minkels) or “stand-alone“ (e.g. Climaveneta)

In most cases the sales channel is different compared to CRAC/CRAH The majority of the projects is via “rack“ or “cabling“ – cooling not important…

65

HIGH DENSITY – CyberRow Competition

300mm CW-version – Vertiv, Rittal, Schneider

66

Airflow: 7.800m³/hCapacity: 59,5kWFan power: 2,5kWEER: 23,80

Airside delta T: 24K !!!

38°C 14°C

ENP

7°C 12°C

Airflow: 8.000m³/h Capacity: 55,0kWFan power: 2,8kWEER: 19,64

Airside delta T: 23K !!

44°C 21°C

Rittal

9°C 16,3°C

Airflow: 7.200m³/hCapacity: 39,4kWFan power: 1,8kWEER: 21,89

Airside delta T: 18K!

35°C 17°C

Schneider

10°C 15,5°C

http://www.stulz.de/en/newsroom/blog/delta-t-91/

HIGH DENSITY – CyberRow USPs

Control features (2 independent set-points, Delta T control, quantity of temperaturesensors) Flexibility (3 different sizes, different unit

depths, different airflow directions)Only manufacturer with a 400mm versionOnly manufacturer with a reliable and

good indirect free cooling solution “quasi-monopoly“ in terms of DX: number of

units, refrigerants, compressor types,… Energy efficiency (fan power consumption) Realistic, practice orientated air-side

temperature differences

67

HIGH DENSITY – CyberRack Active Rear Door

• Containment at rack level (instead ofroom level or for supplemental cooling)

• “Active Rack Cooling“• Cooling exactly where and when needed• For network and server cabinets with

passive or active ventilation• Standard racks can be converted into

high-density racks• Components: EC-fans, CW heat

exchanger, controller

68

HIGH DENSITY – CyberRack Active Rear Door

• Complete product consists of heat exchangermodule and connecting frame

• Independent from rack manufacturer (adapterframe needed for 600mm or 800mm rack, drawing required)

• 2 different capacity versions (19kW/32kW)• Dual power supply integrated• Plug connections

69

HIGH DENSITY – CyberRack Active Rear Door

Air discharged by the servers is immediately sucked in by the cooling unit and delivered back into the room after treatment

Therefore: no need of containment to separate hot and

cold air higher intake air temperature higher water temperatures high efficiency

70

45°C

Server

24°C

25°C

40°C

25°C

45°C

Server

24°C

40°C

45°C

Server

24°C

25°C

40°C

25°C

45°C

Server

24°C

40°C

WT,in:14°C

WT,out:19°C

WT,in:14°C

WT,out:19°C

HIGH DENSITY – CyberRack Active Rear Door

71

Application 1

Rooms w/o Raised Floor: CyberRack to be used as

a stand-alone cooling unitin small server rooms

Entire cooling capacityprovided

Application 2

Data Center: As a “cooling booster“ in

combination with CRAC/ CRAH to avoid hot-spots

In case the perimetercooling has reachedlimits

Application 3

High Density Data Centers: Individual areas (or

complete DC) with high-density racks only

No risk of hot-spots Almost no extra space

needed for cooling units

Airflow Management in Raised Floor Computer Room Environments

HIGH DENSITY – AirBooster/AirModulator

72

Airflow Management in Raised Floor Computer Room Environments• Elimation of hot-spots• increase of efficiency

HIGH DENSITY – AirBooster/AirModulator

73

1. AirModulator2. AirBooster3. Pressure Control Module4. Differential Pressure

Control

http://www.stulz.de/en/newsroom/blog/increased-efficiency-thanks-to-raised-floor-grilles-with-adjustable-opening-angle-818/

AirBooster2

• Fan assisted floor tiles to increase the local airflow at the intake area of the server rack

• Requirement: sufficient airflow provided by theCRAC/CRAH units

• Up to ~4.400m³/h (75W @ 2.900m³/h)• In two zones individual adjustable guide vanes• 3 air temperature sensors, option pressure control• Light and heavy duty grille• BMS connection Modbus RTU• Available with or without controller

HIGH DENSITY – AirBooster2

74

AirModulator2

• Variable air volume damper to enhance efficiency• Requirement: sufficient airflow and pressure provided

by the CRAC/CRAH units• Automatically adjustable damper, Servomotor with

spring return• Low pressure drop due to large damper surface• Low leakage rate• 3 air temperature sensors, option pressure control• Light and heavy duty grille• BMS connection Modbus RTU• Available with or without controller

HIGH DENSITY – AirModulator2

75

PCRAC

PPCMP

THANK YOU