biryaltsev e.v., galimov m.r., demidov d.e., elizarov a.m. hpc cluster development and operation...
TRANSCRIPT
![Page 1: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/1.jpg)
Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M.
HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE
INVERSE PROBLEMS OF SEISMIC PROSPECTING
![Page 2: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/2.jpg)
HPC - INDUSTRIAL DEPLOYMENT2
MASS MARKET READY
![Page 3: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/3.jpg)
OIL & GAS INDUSTRY AND TECHNOLOGY REPLACEMENT3
NEW SOLUTIONS REQUIRED
PRICE DROP
PRODUCTION & EXPLORATION COMPLEXITY
![Page 4: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/4.jpg)
NUMERICAL SIMULATION4
GRID SIMULATION METHODS
![Page 5: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/5.jpg)
FULL WAVE INVERSION5
![Page 6: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/6.jpg)
LOW-FREQUENCY SEISMIC SOUNDING (LFS) 6
![Page 7: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/7.jpg)
APPLICATIONS 7
1. Oil & gas prospecting
2. Fracturing control
3. Borehole environment monitoring
![Page 8: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/8.jpg)
REQUIREMENTS8
ACCESSIBILITY
BIG DATA ARRAYS
PERFORMANCE
VISUALIZATION
30 000 models per cycle
109 – 1012 cells
~1 Tb per model
![Page 9: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/9.jpg)
RESEARCH & DEVELOPMENT9
1. The best performance for the massive parallelism tasks.2. High computation density per equipment units.3. Perspective technology.
GPU — EFFICIENT CHOICE
![Page 10: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/10.jpg)
GPU CLUSTER10
UPS Eaton 9155 Power Rating 8000 / 7200 (VA/Watts)0.7 Power factor
Server>80 Tb Data storage space + SSD cache
Virtualization serverCitirix XenServer platform2 x NVIDIA GK104GL with NVIDIA GRID K2 technology
Mellanox InfiniBand network switch 56 Gbit/s
6 GPU cluster nodes
4 x AMD FirePro W9100 (16Gb GDDR5, 5.24 Tflops peak)
2 x Intel Xeon E5-2643 v2 (25M cache, 3.50 GHz, 12 threads)
Performance35,6/64,9 Tflops
38th position in
Russian top 50 supercomputers rating
RAM –
320 GbOverall disk space
volumes
86 Тb
![Page 11: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/11.jpg)
HARDWARE & SOFTWARE ENVIRONMENT11
VDI
GPU
3D
1. GPU clusters ОC Gentoo & Open Grid Engine
2. VexCL (vector expression template library) OpenCL/CUDA
3. Simulation & analyze software systemС++, MPI, Java
4. VDI serverCitix XenDesktop & NVidia GRID
5. Data warehousePostgreSQL & NFS (RAID)
![Page 12: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/12.jpg)
WORKFLOW12
Sensors data
Simulation data
Processing
3D full wave simulation
GPU cluster
Interpretation
![Page 13: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/13.jpg)
SUCCESSFUL SOLUTIONS13
Cloud solutionAmazon
SOFTWARE – HPC ENVIRONMENTOpenCL, CUDA, OpenMP, MPI
HARDWARE – GPU CLUSTER
Local solutionT-platforms, Meijin, ARBYTE SC, STSS
![Page 14: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/14.jpg)
PROBLEMS14
DISTRIBUTED RENDERING
REMOTE & SHARED ACCESS
BIG DATA MANAGMENT
INTEGRATION
HPC 2D&3D
RD Store
![Page 15: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/15.jpg)
INTEGRATED FRAMEWORKS 15
SCHLUMBERGER OCEAN
HUE SPACE
NVIDIA INDEX
COMMERCIAL AND
PROPRIETARY
NVIDIA TECHNOLOGY
C++, .NET, Java
![Page 16: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/16.jpg)
SPECIALIZED FRAMEWORKS 16
Open Inventor, Nvidia Engines (SceniX, OptiX…), Equalizer, Chromium ...
HPC
Hadoop, HPCC, Array-based DBMS (Rasdaman, SciDB) ...
NVidia CUDA Toolkit, AMD OpenCL SDK ...
Big Data
Визуализация
![Page 17: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/17.jpg)
CONCLUSION17
HPC HARWARE & SOFTWARE EXISTS
INDUSTRIAL SPECIALIZATION
COMPLETE DATA & PROCESSES INTEGRATION METHODOLOGY
COMMON HPC SOFTWARE ECOSYSTEM
SOFTWARE FRAMEWORKS HARD CHOICE
![Page 18: Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC](https://reader034.vdocuments.us/reader034/viewer/2022051517/5697bf941a28abf838c907c7/html5/thumbnails/18.jpg)
18
THANK YOU FOR YOUR ATTENTION!