sgi openfoam cloud benchmark part 3, openfoam study meeting for beginner@kansai, 11th aug. 2012
TRANSCRIPT
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Masashi Imano
SGI OpenFOAM CloudBenchmark
Part 3
1
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Test cluster conditions
• Hardware: SGI Altix ICE8200• CPU: Intel Xeon X5365 3.00GHz 4cores x 2• Node: 16 nodes• Memory: 16GB/node (No swap memory)• Interconnect: InfiniBand DDR• OS: SUSE LINUX ENTERPRISE SERVER 10.3• MPI:✓ SGI MPT (1.2.6)✓ OpenMPI(ThirdParty package distributed with OpenFOAM)
2
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
• OpenFOAM versions: 2.1.x• MPI (interconnect: InfiniBand):✓ OpenMPI (Ver. 1.5.3), mpirun options: --mca btl openib,self,sm✓ SGI MPT (Ver.1.2.6)• Number of InfiniBand rails: 1, 2 (export MPI_IB_RAILS=1 or 2)
• Benchmark problem✓ Flow: Lid-driven 2D cavity(tutorials), 3D cavity✓ Re: 100 (laminar)✓ solver: icoFoam (piso)✓ Linear solver: p(AMG), U(PBiCG)✓ Mesh:
• 2D (Δt=10-6 s, 100 steps): 1000 2, 2000 2, 4000 2
• 3D (Δt=10-5 s, 50 steps): 100 3, 200 3
x
Ux = 1 m/s
d = 0.1 m
y
Calculation conditions
3
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
75
150
225
300
1 2 4 8 16
113.3
155.8
204.4
102.3
54.5 52.371.2
135.9
101.9
54.8
16.329.1
61.2
89.4
53.5
Num
ber o
f Cas
es /
Day
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
2D cavity, mesh:10002 (1 Million)
4
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
0.2
0.4
0.6
0.8
1
1 2 4 8 16
Para
llel e
ffici
ency
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
2D cavity, mesh:10002 (1 Million)
5
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
18
35
53
70
1 2 4 8 16
63.0
43.4
35.3
17.7
8.7
33.029.0
33.6
17.7
8.8 10.812.4
21.116.5
8.5Num
ber o
f Cas
es /
Day
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
2D cavity, mesh:20002 (4 Millions)
6
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
0.2
0.4
0.6
0.8
1
1.2
1 2 4 8 16
Para
llel e
ffici
ency
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
2D cavity, mesh:20002 (4 Millions)
7
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
5
10
15
20
2 4 8 16
20.0
9.8
6.1
3.4
12.9
8.9
6.1
3.5
5.75.15.2
3.3Num
ber o
f Cas
es /
Day
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
2D cavity, mesh:40002 (16 Millions)
8
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
0.2
0.4
0.6
0.8
1
2 4 8 16Para
llel e
ffici
ency
(bas
e: 2
nod
es)
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
2D cavity, mesh:40002 (16 Millions)
9
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
225
450
675
900
1 2 4 8 16
531.1
835.8
675.2
456.2
203.4264.8
488.7421.3420.3
203.2 197.1
403.4434.4
389.5
194.1
Num
ber o
f Cas
es /
Day
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
3D cavity, mesh:1003 (1 Million)
10
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
0.2
0.4
0.6
0.8
1
1.2
1 2 4 8 16
Para
llel e
ffici
ency
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
3D cavity, mesh:1003 (1 Million)
11
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
38
75
113
150
1 2 4 8 16
142.9131.9
72.2
29.9
15.5
89.9
107.4
66.5
29.7
15.5
78.8
94.6
62.4
28.9
15.2Num
ber o
f Cas
es /
Day
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
3D cavity, mesh:2003 (8 Millions)
12
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
0.2
0.4
0.6
0.8
1
1.2
1 2 4 8 16
Para
llel e
ffici
ency
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
3D cavity, mesh:2003 (8 Millions)
13
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
15
30
45
60
4 8 16
58.5
38.1
11.0
42.4
34.8
10.6
38.1
32.7
10.3Num
ber o
f Cas
es /
Day
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
3D cavity, mesh:3003 (27 Millions)
14
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Benchmark results
0
0.5
1
1.5
2
4 8 16Para
llel e
ffici
ency
(bas
e: 4
nod
es)
Number of Nodes
OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails)
3D cavity, mesh:2003 (8 Millions)
15
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.)
Any Questions?
16