dr. rupak biswas – project manager 10 february 2011 · advertises rupak biswas, hecc project...
TRANSCRIPT
Dr. Rupak Biswas – Project Manager NASA Advanced Supercomputing (NAS) Division NASA Ames Research Center, Moffett Field, CA [email protected] (650) 604-4411
10 February 2011
• Using the Pleiades supercomputer, aerospace engineers at NASA Langley have produced advanced simulations of the Orion Crew Exploration Vehicle (CEV) during reentry into the atmosphere, in support of thermal protection system (TPS) design for NASA’s next-generation space vehicle.
• Over the past year, the resulting analyses have been used to: predict surface heating environments for TPS design; understand unsteady wake flow behind the vehicle; assess flow interactions and surface heating caused by the roll, yaw, and pitch jets used during descent; and reduce uncertainties associated with physical flow phenomenon coupled with chemistry when designing new spacecraft.
• This work has required extensive HECC resources to compute the complex flow structures in the CEV; each unsteady calculation solves for over 200 million unknown quantities, and requires about 31,000 CPU- hours on Pleiades for each trajectory point analyzed.
• HECC supercomputing resources have enabled unprecedented simulations that play a key role in understanding reentry environments to design a next-generation vehicle.
Mission Impact: By understanding the intense heating environment of a reentry spacecraft, the Agency can attain its spaceflight mission goals safely and efficiently."
POC: Stephen J. Alter, [email protected], (757) 864-7771, NASA Langley Research Center
Figure: Simulation of CEV wake flow structure showing asymmetry of the computed flowfield.
HECC Resources Support Analysis of CEV Reentry Flow Environments
10 February 2011 2 High End Computing Capabilities Project
• HECC’s Application Performance and Productivity (APP) experts have improved the performance of a Satellite Data Simulation Unit (SDSU) program developed at NASA Goddard.
• The code is used by NASA’s Global Modeling and Assimilation Office to analyze precipitation satellite data as part of a workflow to predict rainfall.
• APP staff first analyzed the time-consuming routines using HECC-developed profiling tools to identify performance hot spots; data in a “do-loop” had to be reordered before OpenMP was used to improve the parallel efficiency of poorly performing regions of the code.
• The parallel code scales almost linearly—achieving an 8-fold speedup using 8 OpenMP threads on a Nehalem node of Pleiades.
Mission Impact: By reducing the runtime of NASA Goddard’s Satellite Data Simulation code, users can now process multiple satellite datasets in a fixed window of time during a mission.
Performance Gains Attained on Satellite Data Simulation Unit (SDSU) Code
10 February 2011
POC: Samson Cheung, [email protected], (650) 604-0923, NASA Advanced Supercomputing Division, Computer Sciences Corp.
Figure: Precipitation observation with a high-density Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI) satellite dataset. The data captures the important feature in the center of the domain (where the tropical cyclone Erin was observed).
3 High End Computing Capabilities Project
10 February 2011
• In late December, eight new “Westmere” racks were added to the Pleiades cluster, increasing the system’s peak performance to 1.081 petaflops.
• The new racks add 6,144 Westmere cores and expands Pleiades to a total of 156 racks with 91,136 cores, which increases the computational capability of the system by 72 teraflops.
• HECC and SGI staff successfully added the new racks via a unique live integration technique, which enabled the system to remain available to users while the racks were added.
• This process saved over 3 million hours of computing time that would have been unavailable if the system had to be disabled for the integration work.
Mission Impact: The continued expansion of Pleiades provides increased computational capability to all NASA Mission Directorates.
POC: Bob Ciotti, [email protected], (650) 604-4408, NASA Advanced Supercomputing Division
Figure: Eight new SGI Westmere racks were installed on Pleiades, which added another 72 teraflops of computational capability to the system.
Installation of Additional Pleiades Westmere Racks Completed
4 High End Computing Capabilities Project
High-End Computing Capabilities Project 5
• The HECC Network team has developed Firewall Query Tool to help manage the highly complex access control environment on NASLAN, the NASA local area network. The benefits of the tool include: – Greatly reduces the amount of time needed while
engineers troubleshoot to identify where a firewall problem is in the network, by performing complex, automated queries end-to-end.
– Learns the dynamic routing table of each router on the network and updates firewall rules daily; this makes the software “network-aware” and able to calculate the path traffic takes through the network, as well as identify which firewalls are crossed.
– Allows non-technical users to easily and instantly determine if there is an access restriction on the network, without the need to contact an engineer.
– Identifies which specific firewall rules permit or block traffic, so engineers know exactly which rule to modify.
• A whitepaper describing the capabilities of Firewall Query Tool can be found at: http://www.nas.nasa.gov/News/Techreports/2011/PDF/nas-11-001.pdf
Mission Impact: The Firewall Query Tool enables HECC support staff to quickly identify service network accessibility across multiple security domains, and helps identify weaknesses in HECC’s security defense.
POC: Nichole Boscia, [email protected], (650) 604-0891, NASA Advanced Supercomputing Division, Computer Sciences Corp.
Figure: The above diagram shows how different systems can traverse the network across various routes. Some are permitted by the firewall, while others are blocked when trying to reach a service.
Network Team Develops Firewall Query Tool to Manage Access Control
10 February 2011 5 High End Computing Capabilities Project
• HECC users representing the NASA Advanced Supercomputing (NAS) Division at NASA Ames participated in the 49th AIAA Aerospace Sciences Meeting, held January 4–11, 2011, in Orlando.
• Users shared and disseminated their latest knowledge and research results, obtained using HECC resources, with aerospace scientists and engineers from around the world.
• Eight scientists presented papers and chaired/participated in panels on various applications of computational fluid dynamics to NASA aeronautics and aerospace projects; see slide #11.
• In addition, HECC staff supported the NASA exhibit (with participation from seven other technical organizations) to explain the Agency’s featured work to conference attendees and to hand out printed materials.
• The exhibit drew many visitors and was very well received; notable visitors to the booth were Administrator Charles Bolden, Chief Technologist Bobbie Braun, and industry partner executives from Boeing, General Electric, and Lockheed Martin.
Mission Impact: HECC support for NASA's AIAA exhibit enhanced the experience and interest of booth visitors, and helped increase the dissemination of NASA's state-of-the-art aerospace CFD research and engineering results.
POCs: Cetin Kiris, [email protected], (650) 604-4485, Gina Morello, [email protected], (650) 604-4462, NASA Advanced Supercomputing Division
Figure: Aerospace scientists and experts from around the world visited NASA’s AIAA booth in Orlando.
HECC Supports Technical Collaboration, Outreach at Annual Aerosciences Meeting
High-End Computing Capabilities Project 6 10 February 2011 6 High End Computing Capabilities Project
Presentations • Vision, Experience, and Investigation Projects at the NAS Supercomputing Center, W. Thigpen,
Universidad de Guadalajara, Mexico, January 26, 2011. • HECC Overview, W. Thigpen, Tecnológico de Monterrey in Guadalajara, Mexico, January 26, 2011. • Supercomputing at NASA: Current Impact, Future Challenges, W. Thigpen, Keynote address at the
Paraninfo of the Universidad de Guadalajara, Mexico, January 27, 2011. Papers • Testing an Eddy-Permitting Model of the Southern Ocean Carbon Cycle Against Observations, M.
Woloszyn, M. Mazloff, Dec. 2010, Elsevier, doi:10.1016/j.ocemod.2010.12.004.* • Crash: A Block-Adaptive-Mesh Code for Radiative Shock Hydrodynamics -Implementation and
Verification, B. van der Holst, et al, arXiv:1101.758v1 [astro-ph.SR] January 19, 2011.* • Learning from the Outer Heliosphere: Interplanetary Coronal Mass Ejection Sheath Flows and the
Ejecta Orientation in the Outer Atmosphere,” R.M. Evans, M. Opher, T.I. Gombosi, The Astrophysical Journal 728 (2011) 41, doi:10.1088/0004-637X/728/1/41.*
• Firewall Policy Query Tool Whitepaper, N. Boscia, January 2011.
Presentations and Papers
* HECC provided supercomputing resources and services in support of this work
10 February 2011 7 High End Computing Capabilities Project
Papers (continued) • AIAA Aerospace Science Meeting, January 11–14, 2011, Orlando *
OVERFLOW Validation for Predicting Plume Impingement of Underexpanded Axisymmetric Jets onto Angled Flat Plates, H. Lee, G. Klopfer.
Validation of OVERFLOW for Computing Plume Effects during the Ares 1 Stage Separation Process, J. Kless, H. Lee, G. Klopfer, J. Onufer, S. Pandya, W. Chan.
Numerical Investigation of the Flow Angularity Effects of the NASA Langley UPWT on the Ares I DAC1 0.01-Scale Model, H. Lee, G. Klopfer
Plume-Induced Flow Separation over a Cone-Cylinder Flare Body, J. Kless, G. Klopfer. Best Practices for Aero-Database CFD Simulations of Ares V Ascent, C. Kiris, J. Housman, M.
Gusman. Best Practices for CFD Simulations of Launch Vehicle Ascent with Plumes, M. Gusman, J.
Housman, C. Kiris. Dynamic Error Estimation and Mesh Refinement in Aerodynamic Shape Design, M. Nemec, M.
Aftosmis.
Presentations and Papers
* HECC provided supercomputing resources and services in support of this work
10 February 2011 8 High End Computing Capabilities Project
News and Events
10 February 2011
• Penn State CI Day to Showcase Cyberinfrastructure, Gant Daily, January 16, 2011 – advertises Rupak Biswas, HECC Project Manager, as presenter.
• ISUM Supercomputing Conference HECC Deputy Project Manager Bill Thigpen participated in the Instituto Universitario de Mercadotechnia (ISUM) Conference, hosted by the University of Guadalajara, Mexico; Thigpen’s visit promoted the use of supercomputing among research communities; numerous online pieces announced this visit.
• California Academy of Sciences: ‘Life: A Cosmic Story’ San Francisco Chronicle, January 21, 2011 – The HECC Data Analysis and Visualization group contributed significantly to this project and the article mentions NASA Ames
* HECC provided supercomputing resources and services in support of this work
9 High End Computing Capabilities Project
NAS Utilization
10 February 2011
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Pleiades Columbia Production
Share Limit
Job Drain
Dedtime Drain
Limits Exceeded
Specific CPUs
Insufficient CPUs
Held
Queue Not Schedulable
Not Schedulable
No Jobs
Dedicated
Down
Degraded
Boot
Used
January 2011
10 High End Computing Capabilities Project
NAS Utilization Normalized to 30-Day Month
10 February 2011
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
Stan
dard
Bill
ing
Uni
ts
NAS
NLCS
NESC
SOMD
SMD
ESMD
ARMD
Alloc. to Orgs
11 High End Computing Capabilities Project
NAS Utilization Normalized to 30-Day Month
10 February 2011
1 Allocation to orgs. increased to 80% 2 SMD augmentation 3 RTJones retired 4 32 Westmere racks added 5 Schirra retired, 4 Westmere racks added 6 RTJones compensation removed 7 8 Westmere racks added
12 High End Computing Capabilities Project
Tape Archive Status
10 February 2011
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
Unique File Data Total Tape Data Tape Capacity Tape Library Capacity
Peta
Byt
es
Capacity Used HECC Pre-mission NAS NLCS NESC SOMD SMD ESMD ARMD
January 2011
The data for Total Tape Data and Unique Tape Data include three tape libraries. We are in the process of migrating data and have run into hardware problems, so there is currently duplicate data. That is why the usage appears to exceed capacity.
13 High End Computing Capabilities Project
Tape Archive Status
10 February 2011
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11
Peta
Byt
es Tape Library Capacity
Tape Capacity Total Tape Data Unique Tape Data Unique File Data
The data for Total Tape Data and Unique Tape Data include three tape libraries. We are in the process of migrating data and have run into hardware problems, so there is currently duplicate data. That is why the usage appears to exceed capacity.
14 High End Computing Capabilities Project
Pleiades: SBUs Reported, Normalized to 30-Day Month
10 February 2011
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
Stan
dard
Bill
ing
Uni
ts (h
ours
* .9
[Har
pert
own/
Wes
tmer
e] o
r 1.1
[Neh
alem
])
NAS
NLCS
NESC
SOMD
SMD
ESMD
ARMD
Alloc. to Orgs
15 High End Computing Capabilities Project
Pleiades: Monthly SBUs by Run Time
10 February 2011
0
2,000,000
4,000,000
6,000,000
8,000,000
10,000,000
12,000,000
14,000,000
16,000,000
0 - 1 hours > 1 - 4 hours > 4 - 8 hours > 8 - 24 hours > 24 - 48 hours
> 48 - 72 hours
> 72 - 96 hours
> 96 - 120 hours
> 120 hours Stan
dard
Bill
ing
Uni
ts (h
ours
* .9
[Har
pert
own/
Wes
tmer
e] o
r 1.1
[Neh
alem
])
Job Run Time (hours) January 2011
16 High End Computing Capabilities Project
Pleiades: Monthly Utilization by Size and Mission
10 February 2011
0
2,000,000
4,000,000
6,000,000
8,000,000
10,000,000
12,000,000
14,000,000
16,000,000
18,000,000
20,000,000
1 - 32 33 - 64 65 - 128 129 - 256 257 - 512 513 - 1024 1025 - 2048
2049 - 4096
4097 - 8192
8193 - 16384
16385 - 32768 St
anda
rd B
illin
g U
nits
(hou
rs *
.9 [H
arpe
rtow
n/W
estm
ere]
or 1
.1 [N
ehal
em])
Job Size (cores)
NAS NLCS NESC SOMD SMD ESMD ARMD
January 2011
0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000
8193 - 16384
16385 - 32768
17 High End Computing Capabilities Project
Pleiades: Monthly Utilization by Size and Length
10 February 2011
0
2,000,000
4,000,000
6,000,000
8,000,000
10,000,000
12,000,000
14,000,000
16,000,000
18,000,000
20,000,000
1 - 32 33 - 64 65 - 128 129 - 256 257 - 512 513 - 1024
1025 - 2048
2049 - 4096
4097 - 8192
8193 - 16384
16385 - 32768 St
anda
rd B
illin
g U
nits
(hou
rs *
.9 [H
arpe
rtow
n/W
estm
ere]
or 1
.1 [N
ehal
em])
Job Size (cores)
> 120 hours > 96 - 120 hours > 72 - 96 hours > 48 - 72 hours > 24 - 48 hours > 8 - 24 hours > 4 - 8 hours > 1 - 4 hours 0 - 1 hours
January 2011
0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000
8193 - 16384
16385 - 32768
18 High End Computing Capabilities Project
Pleiades: Average Time to Clear All Jobs
10 February 2011
0
24
48
72
96
120
144
168
192
216
240
264
288
312
336
Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11
Hou
rs
ARMD ESMD SMD SOMD/NESC
19 High End Computing Capabilities Project
Pleiades: Average Expansion Factor
10 February 2011
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11
ARMD ESMD SMD SOMD NESC
4.90 7.89
20 High End Computing Capabilities Project
Columbia: SBUs Reported, Normalized to 30-Day Month
10 February 2011
0
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
Stan
dard
Bill
ing
Uni
ts (h
ours
* 1)
NAS
NLCS
NESC
SOMD
SMD
ESMD
ARMD
Alloc. to Orgs
21 High End Computing Capabilities Project
Columbia: Monthly SBUs by Run Time
10 February 2011
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
450,000
500,000
0 - 1 hours > 1 - 4 hours > 4 - 8 hours > 8 - 24 hours > 24 - 48 hours
> 48 - 72 hours
> 72 - 96 hours
> 96 - 120 hours
> 120 hours
Stan
dard
Bill
ing
Uni
ts (h
ours
* 1)
Job Run Time (hours) January 2011
22 High End Computing Capabilities Project
Columbia: Monthly Utilization by Size and Mission
10 February 2011
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
900,000
1 - 32 33 - 64 65 - 128 129 - 256 257 - 512 513 - 1024 1025 - 2048
Stan
dard
Bill
ing
Uni
ts (h
ours
* 1)
Job Size (cores)
NAS NLCS NESC SOMD SMD ESMD ARMD
January 2011
0
2,000
4,000
6,000
8,000
10,000
12,000
513 - 1024 1025 - 2048
23 High End Computing Capabilities Project
Columbia: Monthly Utilization by Size and Length
10 February 2011
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
900,000
1 - 32 33 - 64 65 - 128 129 - 256 257 - 512 513 - 1024 1025 - 2048
Stan
dard
Bill
ing
Uni
ts (h
ours
* 1)
Job Size (cores)
> 120 hours > 96 - 120 hours > 72 - 96 hours > 48 - 72 hours > 24 - 48 hours > 8 - 24 hours > 4 - 8 hours > 1 - 4 hours 0 - 1 hours
January 2011
0
2,000
4,000
6,000
8,000
10,000
12,000
513 - 1024 1025 - 2048
24 High End Computing Capabilities Project
Columbia: Average Time to Clear All Jobs
10 February 2011
0
24
48
72
96
120
144
168
192
Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11
Hou
rs
ARMD ESMD SMD SOMD/NESC
465
25 High End Computing Capabilities Project
Columbia: Average Expansion Factor
10 February 2011
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11
ARMD ESMD SMD SOMD NESC
24.80 8.84
26 High End Computing Capabilities Project