h p c – big data 1_8... · 2019-05-23 · high performance computing and big data for climate...
TRANSCRIPT
![Page 2: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/2.jpg)
ClimEx Symposium 2019 2
High Performance Computing and Big Datafor climate research
- Jens Weismüller, LRZ -
![Page 3: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/3.jpg)
ClimEx Symposium 2019
• 84,000,000 core-hours≈ 2,500 years on a modern Laptop
3
Image: redjar (Jared C. Benedict), Wikipedia
Image: pexels.com
Challenges and Benefits of
Supercomputing in ClimEx
![Page 4: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/4.jpg)
ClimEx Symposium 2019
• 84,000,000 core-hours≈ 2,500 years on a modern Laptop
• What do we needinstead?
4
Image: redjar (Jared C. Benedict), Wikipedia
Image: pexels.com
Challenges and Benefits of
Supercomputing in ClimEx
![Page 6: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/6.jpg)
ClimEx Symposium 2019 6
Run the wind turbinein Fröttmaning forabout nine months
Image: Hihiman / Wikipedia
Total energy consumption: ~1,25 GWh
Challenges and Benefits of
Supercomputing in ClimEx
Power ~350 householdsfor one year ~300 people flying
from Munich toMontreal and back
Image: Andreas Koll / Wikipedia Image: Dave Sizer / Wikipedia
![Page 7: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/7.jpg)
ClimEx Symposium 2019
Energy efficiency
Climate simulation means massive energy consumption!How do we use energy as efficiently as possible?
7
Innovative cooling technology
• Temperature recordhigh Munich: 37.1°C in August 2003
• SuperMUC: 40°C cooling water
• No active cooling
![Page 8: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/8.jpg)
ClimEx Symposium 2019
The ClimEx dataset consistsof ~400 TB. How much is that?
8
Image: Asim18 / Wikipedia
Challenges and Benefits of
Supercomputing in ClimEx
![Page 9: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/9.jpg)
ClimEx Symposium 2019
The ClimEx dataset consistsof ~400 TB. How much is that?
• 50 modern harddisks
9
Challenges and Benefits of
Supercomputing in ClimEx
![Page 10: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/10.jpg)
ClimEx Symposium 2019
The ClimEx dataset consistsof ~400 TB. How much is that?
• 50 modern harddisks
But we also need a backup…
• 100 modern harddisk?
10
Challenges and Benefits of
Supercomputing in ClimEx
![Page 11: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/11.jpg)
ClimEx Symposium 2019 11
SuperMUC
Other Computing Systems
Data exploration and accessibility
![Page 12: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/12.jpg)
ClimEx Symposium 2019 12
SuperMUC
Other Computing Systems
Sharing
Visualization
Data exploration and accessibility
![Page 13: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/13.jpg)
ClimEx Symposium 2019
www.climex-project.org/en/data-access
• Web-Interface for exploring the European data
13
Data exploration and accessibility
![Page 14: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/14.jpg)
ClimEx Symposium 2019
www.climex-project.org/en/data-access
• GLOBUS access for transferring large amounts of the European data already in place
14
Data exploration and accessibility
![Page 15: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/15.jpg)
ClimEx Symposium 2019
Visualisation
15
Match scientific data to the viewers everyday experiences
![Page 17: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/17.jpg)
ClimEx Symposium 2019
Key messages
• CRCM5 and WaSiM ported and adapted to HPC systems
• management of a huge job farming environment
• production of huge climate simulation data set
• data provisioning to the scientific community(e.g. Wasser Zukunft Bayern) and stakeholders in the agriculture, forestry, energy, health ... sectors
• pilot usage of new LRZ services and tools(e.g. Data Sharing, Data Science Storage)
• new data visualization approach to support the science-policy interface
17
![Page 18: H P C – Big Data 1_8... · 2019-05-23 · High Performance Computing and Big Data for climate research - Jens Weismüller, LRZ - ClimEx Symposium 2019 • 84,000,000 core-hours](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5b0f4b36f6663a206498b/html5/thumbnails/18.jpg)
ClimEx Symposium 2019
www.climex-project.org/en/data-access
18
Energy-aware climate modeling Visualization
Data explorationand sharing
Key messages