cfd containment modeling
TRANSCRIPT
-
8/17/2019 CFD Containment Modeling
1/32
Computational Fluid Dynamic (C.F.D.)
Modeling of Containment Solutions
Design Model Deploy
-
8/17/2019 CFD Containment Modeling
2/32
What is C.F.D
CFD or computational fluid dynamic modeling is aprogram designed to simulate how fluids, like airact and react in an environment, from determiningflow patterns to heat absorption. Recently CFD
tools have been designed specifically for the datacenter or IT environment. Great Lakes has begunto offer CFD modeling to its customers in order tohelp address issues, plan capacity increases incurrent operating data centers, or to test a datacenter design or room design before ever startingthe build.
-
8/17/2019 CFD Containment Modeling
3/32
CFD Model of Cooling Solutions
• Physical Space – Dimensions: 40′ Wide by 60′ Long
– Raised floor height: 24″
– Ceiling height from raised floor: 14′ – 2403 sq. ft traditional raised floor data center
• IT Specifications – 7.2 kW per enclosure
– Total IT Load = 459kW
– 64 Racks at 44 RMU (1408 servers @ 2U) 2240 Ports (servers * 2 ports)
• Room Conditions – CRAC unit exhaust 75 degrees Fahrenheit
– False Ceiling used with chimneys
– Aisle contains 16, 40% perforated floor grilles
An evaluation of containment solutions for datacenters using CFD modeling. A base
line datacenter was used to provide a comparison of different containment
solutions. The following slides outline the datacenter facility and IT specifications.
-
8/17/2019 CFD Containment Modeling
4/32
3 Phase 480V 1000A service
500 kVA Generator x 1
500kVA UPS system x 2
In Room PDU for 480 to 220V step downPower run under floor (Rack PDU’s hardwired to
BusWay)
30AMP 3Phase Rack PDU with 208V output x 2
per enclosure
Power
-
8/17/2019 CFD Containment Modeling
5/32
Cooling
• 150 Ton Centrifugal Chiller Unit
• 40 Ton CRAC units with plenum x 4
Traditional Hot Aisle/Cold Aisle configuration500 floor tiles (388 solid, 112 perf when needed)
-
8/17/2019 CFD Containment Modeling
6/32
Enclosure Configurations
P S U F A N S U P F A B I O M
Cisco Nexus 7000 Series
1 2 3 4 5 6 7 8 9 1
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
12
1112
13 1
4
23 2
4
2526
3536
3738
47 4
8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M 1 4 8 G S - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
S T A T U S
I D
1
N 7 K - M1 4 8 GT - 1 1 L
2
3
4
5
6
7
8
1 2
1 1 1 2
1 3
1 4
2 3
2 4
2 5 2 6
3 5 3 6
3 7 3 8
4 7
4 8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
Server Enclosure (63 enclosures)GL840ES-304220 x Dell R710 2 RMU servers @ 400W
Network Enclosure (1 enclosure)GL840ES-30481 x Cisco Nexus 7010
Remaining RMU on each rack to be populated with FillerPanels. Sides of enclosures to have Brush Grommet kitsinstalled.
-
8/17/2019 CFD Containment Modeling
7/32
Configurations
Enclosure Level Containment
With Air Managers and Chimneys
Standard Cold Aisle Containment
Cold Aisle Containment With
Horizontal Polargy Panels
Standard Hot Aisle Cold Aisle
-
8/17/2019 CFD Containment Modeling
8/32
Enclosure Level Containment
With Air Manager and Chimneys(5 floor tiles per aisle)
Server Enclosure ConfigurationGL840ES-3042LC
Frame: 8401ES-3042 – 84”H x 30”W x 42”D
Top Panel: Solid Top - TPES-S
Front door: Mesh Contour – 8402E-MC29
Rear Door: Solid - 8403E-29
Air Manager: GL-AMES
Chimney: GL-EC-42-3246
BGRK30 – Side rail grommet kit
Network Enclosure Configuration
GL840ES-3048LC
Frame: 8401ES-3048 – 84”H x 30”W x 48”D
Top Panel: Solid Top - TPES-S
Front door: Mesh Contour – 8402E-MC29
Rear Door: Solid - 8403E-29
Air Manager: GL-AMES
Chimney: GL-EC-48-3246BGRK30 – Side rail grommet kit
Containment Components:
1.75-FPTL19 – Tool-less filler panels for
unused RMU
Additional Notes:
5 floor tiles per cold aisle
-
8/17/2019 CFD Containment Modeling
9/32
Enclosure Level Containment
With Air Manager and Chimneys(5 floor tiles per aisle)
PerformanceAir Handler Supply: 75
Air Handler Return: 98.2
Room Temp. Change: 6.7
-
8/17/2019 CFD Containment Modeling
10/32
Enclosure Level Containment
With Air Manager and Chimneys(5 floor tiles per aisle)
Temperature, cabinet bottom height
Cold aisle
temperatures are
very consistent with
no air recirculation.
-
8/17/2019 CFD Containment Modeling
11/32
Enclosure Level Containment
With Air Manager and Chimneys(5 floor tiles per aisle)
Temperature, halfway cabinet height
Use of chimneys
eliminates the hot
aisle maintaining
a consistenttemp. throughout
the data center.
-
8/17/2019 CFD Containment Modeling
12/32
Enclosure Level Containment
With Air Manager and Chimneys(5 floor tiles per aisle)
Temperature, cabinet top height
Temperatures in
the cold aisle at
the top of the
enclosures arewithin range of
the air handler
set-points due to
the elimination of
the hot aisle. The
overall temp inthe data center is
consistent.
-
8/17/2019 CFD Containment Modeling
13/32
Enclosure Level Containment
With Air Manager and Chimneys(5 floor tiles per aisle)
Temperature, 12″ above cabinet
Exhaust air from
the chimneys is
directed into the
drop ceilingwhere the air
handlers with
baffles installed
into the ceiling
draw the hot air
in to be
conditioned.
-
8/17/2019 CFD Containment Modeling
14/32
Enclosure Level Containment
With Air Manager and Chimneys(5 floor tiles per aisle)
cross section
Equipment
intake
temperatures
are consistentfrom top to
bottom. Hot air
is ducted into
the plenum
where hot air is
returned to the
CRAC unit.
Overall the system is highly efficient in segregating hot air
from the room providing a high return temperature to the
CRAC unit. However a plenum ceiling or plenum ducting is
needed to return to the CRAC unit.
-
8/17/2019 CFD Containment Modeling
15/32
Standard Cold Aisle ContainmentWith Horizontal Polargy Panels
Server Enclosure ConfigurationGL840ES-3042MS
Frame: 8401ES-3042 – 84”H x 30”W x 42”D
Top Panel: Solid Top - TPES-S
Front door: Mesh Contour – 8402E-MC29
Rear Door: Mesh Contour – 8402E-MC29
BGRK30 – Side rail grommet kit
Network Enclosure Configuration
GL840ES-3048MSFrame: 8401ES-2448 – 84”H x 30”W x 48”D
Top Panel: Solid Top - TPES-S
Front door: Mesh Contour – 8402E-MC29
Rear Door: Mesh Contour – 8402E-MC29
BGRK30 – Side rail grommet kit
Containment Components:
ACD840ES – Aisle Containment Doors
PPLEX-HPNL24 – Horizontal PolargyContainment Panels
1.75-FPTL19 – Tool-less fil ler panels for
unused RMU
-
8/17/2019 CFD Containment Modeling
16/32
Standard Cold Aisle Containment
With Horizontal Polargy Panels
Performance Air Handler Supply: 75°F
Air Handler Return: 98.2°F
Room Temp. Change: 19.2°F
-
8/17/2019 CFD Containment Modeling
17/32
Standard Cold Aisle Containment
With Horizontal Polargy PanelsTemperature, cabinet bottom height
End of row
containment
creates a
consistent cold
aisle temperature
-
8/17/2019 CFD Containment Modeling
18/32
Standard Cold Aisle Containment
With Horizontal Polargy PanelsTemperature, halfway cabinet height
-
8/17/2019 CFD Containment Modeling
19/32
Standard Cold Aisle Containment
With Horizontal Polargy PanelsTemperature, cabinet top height
Overhead
containment panels
reduce
recirculation in the
top of enclosures
creating very
consistent temps in
the aisles of the data
center.
-
8/17/2019 CFD Containment Modeling
20/32
Standard Cold Aisle Containment
With Horizontal Polargy Panels(cross section view)
Equipment
intake
temperatures
are consistentfrom top to
bottom
Overall the system is efficient in segregating hot air from the
room providing a high return temperature to the CRAC unit.
However end of row doors and overhead panels add
additional infrastructure and safety considerations.
-
8/17/2019 CFD Containment Modeling
21/32
Standard Cold Aisle Containment
Server Enclosure ConfigurationGL840ES-3042MS
Frame: 8401ES-3042 – 84”H x 30”W x 42”D
Top Panel: Solid Top - TPES-S
Front door: Mesh Contour – 8402E-MC29
Rear Door: Mesh Contour – 8402E-MC29
BGRK30 – Side rail grommet kit
Network Enclosure Configuration
GL840ES-3048MSFrame: 8401ES-3048 – 84”H x 30”W x 48”D
Top Panel: Solid Top - TPES-S
Front door: Mesh Contour – 8402E-MC29
Rear Door: Mesh Contour – 8402E-MC29
BGRK30 – Side rail grommet kit
Containment Components:
ACD840ES – Aisle Containment Doors
1.75-FPTL19 – Tool-less filler panels forunused RMU
-
8/17/2019 CFD Containment Modeling
22/32
Standard Cold Aisle Containment
Performance Air Handler Supply: 75°F
Air Handler Return: 98.1°F
Room Temp. Change: 22.1°F
-
8/17/2019 CFD Containment Modeling
23/32
Standard Cold Aisle ContainmentTemperature, cabinet bottom height
Cold Aisle
containment reduces
recirculation creating
more consistenttemperatures in the
hot and cold aisles
-
8/17/2019 CFD Containment Modeling
24/32
Standard Cold Aisle ContainmentTemperature, halfway cabinet height
-
8/17/2019 CFD Containment Modeling
25/32
Standard Cold Aisle ContainmentTemperature, cabinet top height
Recirculation still
occurs at the tops of
the enclosures creating
higher exhaust tempsand less even
temperatures in the
data center.
-
8/17/2019 CFD Containment Modeling
26/32
Standard Cold Aisle Containment(cross section view)
Equipment
intake
temperatures
fluctuate at theends of rows
and tops of
enclosures due
to re-circulation.
Overall the system is efficient in segregating hot air from the room
providing a high return temperature to the CRAC unit. However end ofrow doors add additional infrastructure and safety considerations and
cold air can by-pass the cold aisle resulting in losses in efficiency as well
as hot air re-circulation.
-
8/17/2019 CFD Containment Modeling
27/32
Standard (Hot Aisle/Cold Aisle)
Server Enclosure ConfigurationGL840ES-3042MS
Frame: 8401ES-3042 – 84”H x 30”W x 42”D
Top Panel: Solid Top - TPES-S
Front door: Mesh Contour – 8402E-MC29
Rear Door: Mesh Contour – 8402E-MC29
BGRK30 – Side rail grommet kit
Network Enclosure Configuration
GL840ES-3048MSFrame: 8401ES-3048 – 84”H x 30”W x 48”D
Top Panel: Solid Top - TPES-S
Front door: Mesh Contour – 8402E-MC29
Rear Door: Mesh Contour – 8402E-MC29
BGRK30 – Side rail grommet kit
Additional Accessories
1.75-FPTL19 – Tool-less fil ler panels for
unused RMU
-
8/17/2019 CFD Containment Modeling
28/32
Standard (Hot Aisle/Cold Aisle)
Performance Air Handler Supply: 75°F
Air Handler Return: 98.6°F
Room Temp. Change: 21.5°F
-
8/17/2019 CFD Containment Modeling
29/32
Standard (Hot Aisle/Cold Aisle)Temperature, cabinet bottom height
Recirculation in
the cold aisle
raises equipment
inlet temperature
-
8/17/2019 CFD Containment Modeling
30/32
Standard (Hot Aisle/Cold Aisle)Temperature, halfway cabinet height
-
8/17/2019 CFD Containment Modeling
31/32
Standard (Hot Aisle/Cold Aisle)Temperature, cabinet top height
Recirculation in the
cold aisle raises
equipment inlet
temperature which
creates a higher
exhaust temperature
-
8/17/2019 CFD Containment Modeling
32/32
Standard (Hot Aisle/Cold Aisle)(cross section view)
Recirculation from
around the aisle and
over the top of the
enclosures creates
high equipment
intakes in the cold
aisle.
Overall the system segregates hot air from the cold aisle.
However, hot air recirculation can occur in the top third RMU
of the equipment causing early equipment mortality and
thermal shutdowns.