future of root runtime c++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 c++ modules in a nutshell - pre...
TRANSCRIPT
![Page 1: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/1.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Future of ROOT Runtime C++ modules
Yuka Takahashi - Pr inceton Univers i ty, CERNVasi l Geogiev Vas i lev - Pr inceton Univers i ty
![Page 2: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/2.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
3 Very ImportantSummary Slides
�2
![Page 3: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/3.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Experiments will be more Performant
�3
1. With Runtime C++ Modules,
![Page 4: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/4.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
End users don’t have to change anything in their code
�4
2. With Runtime C++ Modules,
![Page 5: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/5.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Set -Druntime_cxxmodules=ON(Available in future 6.16!)
�5
3. When you use Runtime C++ Modules,
![Page 6: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/6.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
1. C++ Modules in a nutshell2. Effects for experiments3. Effect for ROOT4. Implementation5. Status and roadmap
Agenda
�6
![Page 7: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/7.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
C++ Modules in a Nutshell
�7
![Page 8: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/8.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
C++ Modules in a Nutshell
�8
#include <vector>
![Page 9: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/9.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
C++ Modules in a Nutshell
�9
Textual Include PCH Modules
#include <vector>
ExpensiveFragile
Inseparable
![Page 10: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/10.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
C++ Modules in a Nutshell
#include "TVirtualPad.h"#include <vector>#include <set>
int main() {…
�10
Textual Include
Preprocess
…………
# 286 "/usr/include/c++/v1/vector" 2 3namespace std { inline namespace __1 {template <bool> class __vector_base_common { __attribute__ ((__visibility__("hidden"), __always_inline__)) __vector_base_common() {}
……# 394 "/usr/include/c++/v1/set" 3namespace std {inline namespace __1 {template <…> class set {public: typedef _Key key_type;
……
int main {……
Parse.o
vector
set
TVirtualPad.h
Compile
original code
one big file!
![Page 11: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/11.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
C++ Modules in a Nutshell
�11
#define PI 3.14…
Rcpp library
Users’ code#include <header.h>
…double PI = 3.14;// => double 3.14 = 3.14;
Textual Include
1. ExpensiveReparse the same header
2. FragileName collisions
.h
.c
.c
.c .o
.o
.o
![Page 12: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/12.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
1. Storing pre compiled header information (same as modules)
2. Stored in one big file
�12
PCH (Pre Compiled Header)
C++ Modules in a Nutshell
![Page 13: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/13.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter
b.pcma.pcm d.pcmc.pcm e.pcm
�13
C++ Modules in a Nutshell
- Pre compiled PCM files contain header information- PCMs are separated
Modules
![Page 14: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/14.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
C++ Modules in a Nutshell
�14
Compile-time scalabilityFragilitySeparable
Modules
- Pre compiled PCM files contain header information- PCMs are separated
![Page 15: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/15.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Effect for Experiments
�15
![Page 16: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/16.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Current problem of Experiment Software Stack
Effect for Experiments
�16
- Experiments are still using textual include- Not even PCH- PCH can’t be used because it’s too big
Experiments are parsing hundreds of headers at the startup time at the moment
![Page 17: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/17.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Current problem of Experiment Software Stack
Effect for Experiments
�17
- Experiments are still using textual include- Not even PCH- PCH can’t be used because it’s too big
Modules can do this as it’s separable
![Page 18: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/18.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop �18
Compile ROOT in CMS environment with modulesGenerate dictionary for CMS external libraries
Effect for Experiments
Current Status
WIP
![Page 19: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/19.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop �19
Effect for Experiments
Current Status
Working closely with CMSSWGetting a lot of feedback from CMS usage
- Leads to fix bugs in Clang & Cling
![Page 20: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/20.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop �20
Effect for Experiments
No actual benchmark yet
Simulated results in next slidesComparison of modules to textual includes
Performance Benefits
![Page 21: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/21.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop �21
Effect for Experiments
Memory
Modules
Textual Include
![Page 22: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/22.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop �22
Effect for Experiments
CPU Time
Modules
Textual Include
![Page 23: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/23.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop �23
Effect for Experiments
CPU Time }Experiments have hundreds of header files to parse
Thus the difference will be order of magnitude larger in experiments
![Page 24: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/24.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Effect for ROOT
�24
![Page 25: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/25.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
$ root -lroot [0] gMinuit // Cannot load variableIncrementalExecutor::executeFunction:symbol 'gMinuit' unresolved whilelinking [cling interface function]!
�25
Without Modules
Effect for ROOTCorrectness benefit
![Page 26: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/26.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
$ root -lroot [0] gMinuit // Could load libMinuit(TMinuit *) nullptr
�26
With Modules
Effect for ROOTCorrectness benefit
![Page 27: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/27.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
We can make ROOT modular for lazy installing packages- ROOT package manager- See Oksana’s talk for
more information!
�27
Effect for ROOTPackaging benefit
Slide from Oksana :)
![Page 28: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/28.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Implementation
�28
![Page 29: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/29.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Implementation
�29
![Page 30: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/30.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Implementation
�30
Clang
- External project under LLVM- Bi-weekly meeting with C++
Modules community- Reporting & fixing bugs- ROOT is the largest user of
Modules outside industry
![Page 31: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/31.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Status and roadmap
�31
![Page 32: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/32.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Memory - hSimple
Status and roadmap
�32
Preloading all modules
Autoloading libraries
Implement Bloom filter
Pin vtable
Check fCXXRecordDecl Cache
Enable Cling modulemap
…. And More!!
https://rootbnch-grafana-test.cern.ch/
Yellow line is PCH Green line is Modules
![Page 33: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/33.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Memory - hSimple
Status and roadmap
�33
Preloading all modules
Autoloading libraries
Implement Bloom filter
Pin vtable
Check fCXXRecordDecl Cache
Enable Cling modulemap
…. And More!!
https://rootbnch-grafana-test.cern.ch/
Yellow line is PCH Green line is Modules
![Page 34: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/34.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Status and roadmap
�34…. And More!!
Preloading Modules
All correctness benefit over PCH is due to this- Preloading of all modules- Replace old infrastructure
- rootmap
![Page 35: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/35.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Real time - hSimple
Status and roadmap
�35
https://rootbnch-grafana-test.cern.ch/
Yellow line is PCH Green line is Modules
Hunting down bottlenecks!
![Page 36: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/36.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Fundamental Construction in ROOT Core, which affects every code passed to ROOTWorking with industry and CMSSWGood progress in performance optimization
�36
Status and roadmap
Status
RoadmapReach complete production level before 6.16Continue working on optimizationModularise CMSSW!
![Page 37: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/37.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Thank you for your attention!
![Page 38: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/38.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
Backup Slides
![Page 39: Future of ROOT runtime C++ modules2a.pcm b.pcm c.pcm d.pcm e.pcm 13 C++ Modules in a Nutshell - Pre compiled PCM files contain header information - PCMs are separated Modules Yuka](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1f8/html5/thumbnails/39.jpg)
Yuka Takahashi 12.09.2018 Future of ROOT runtime C++ modules, ROOT Users workshop
C++ Modules is a mechanism to boost compilation time
Effect for ROOT
�39
For ROOT, it turns into runtime speed improvement as we have C++ interpreter