updates on root c++ modules...each pcm file (a.pcm) corresponds to a library (liba.so) interpreter...
TRANSCRIPT
![Page 1: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/1.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Updates on ROOT C++ modules
Yuka Takahashi - Pr inceton Univers i ty, CERNVasi l Geogiev Vas i lev - Pr inceton Univers i tyOksana Shadura - Univers i ty o f Nebraska L incoln
![Page 2: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/2.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
C++ Modules released as a technology preview in ROOT 6.16
- All related tests are passing- CI in ROOT Pull requests, hourly build in
incremental build
News!
�2
![Page 3: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/3.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
1. Our goal2. Status - Stability and tests3. Status - Correctness4. Status - Performance5. Steps to migrate to C++ Modules6. Future work
Agenda
�3
![Page 4: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/4.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Our goal
�4
1. Improve correctness of ROOT2. Avoid parsing header files at ROOT’s runtime3. Optimize performance of ROOT for third-party code
(most notably ALICE, ATLAS, CMS and LHCb)
![Page 5: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/5.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Our goal
�5
1. Improve correctness of ROOT2. Avoid parsing header files at ROOT’s runtime3. Optimize performance of ROOT for third-party code
(most notably ALICE, ATLAS, CMS and LHCb) WIP!
![Page 6: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/6.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
1. Stable tests- Vetoing 11 false positive tests out of 1865 tests
2. Continues integration in PR and incrementals
Status - Stability and tests
�6
![Page 7: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/7.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
$ root -lroot [0] gMinuit // Cannot load variableIncrementalExecutor::executeFunction:symbol 'gMinuit' unresolved whilelinking [cling interface function]!
�7
Without Modules
Status - correctnessCorrectness benefit
![Page 8: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/8.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
$ root -lroot [0] gMinuit // Could load libMinuit(TMinuit *) nullptr
�8
With ModulesCorrectness benefitStatus - correctness
![Page 9: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/9.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
[yuka@yuka-arch root-release]$ root -lroot [0] #include <m17n-core.h> // System headerroot [1] m17n_init_core() // Cannot autoload system libraries! IncrementalExecutor::executeFunction:symbol 'm17n_init_core' unresolved while linking [cling interface function]!
�9
Without ModulesCorrectness benefitStatus - correctness
![Page 10: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/10.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
[yuka@yuka-arch module-release]$ root -lroot [0] #include <m17n-core.h>root [1] m17n_init_core() // Could autoload libm17.so!root [2]
�10
With ModulesCorrectness benefitStatus - correctness
![Page 11: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/11.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �11
Status - performanceLong tests (30s~)
![Page 12: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/12.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �12
Status - performanceNon-PCH tests (~1s)
![Page 13: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/13.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �13
Status - performanceStartup & Hsimple (~0.4s)
![Page 14: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/14.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Memory - hSimple
�14
Preloading all modules
Autoloading libraries
Implement Bloom filter
Pin vtable
Check fCXXRecordDecl Cache
Enable Cling modulemap
…. And More!!
https://rootbnch-grafana-test.cern.ch/
Yellow line is PCH Green line is Modules
Status - performance
![Page 15: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/15.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Real time - hSimple
�15
https://rootbnch-grafana-test.cern.ch/
Yellow line is PCH Green line is Modules
Hunting down bottlenecks!
Status - performance
![Page 16: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/16.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �16
-Druntime_cxxmodules=ON to your CMake :)
Steps to migrate to C++ Modules
![Page 17: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/17.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �17
Steps to migrate to C++ Modules
Why to migrate?- For ROOT end users
- Correctness benefits
![Page 18: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/18.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �18
Steps to migrate to C++ Modules
Why to migrate?- For experiments
- Correctness benefits- Performance!
Experiments are parsing hundreds of headers at the startup time at the moment
- Experiments are still using textual include- PCH can’t be used because it’s too big
Modules can do this as it’s separable
![Page 19: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/19.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �19
Current StatusWorking closely with CMSSWGetting a lot of feedback from CMS usage
- Leads to fix bugs in Clang & Cling (fixed 3 bugs in Clang)
Steps to migrate to C++ Modules
Why to migrate?- For experiments
![Page 20: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/20.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Future work
�20
1. Test & Stability- Increase the coverage of all possible options (-D*)
2. Performance- Measure the performance numbers in CMSSW- Keep optimizing the performance!
3. Migration of codebase- Help with the migration process of the third-party
code, and in particular the major LHC experiments (ALICE, ATLAS, CMS, LHCb)
![Page 21: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/21.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Thank you for your attention!
![Page 22: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/22.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Backup Slides
![Page 23: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/23.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �23
![Page 24: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/24.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules �24…. And More!!
Preloading Modules
All correctness benefit over PCH is due to this- Preloading of all modules- Replace old infrastructure
- rootmap
Status - performance
![Page 25: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/25.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
C++ Modules in a Nutshell
�25
![Page 26: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/26.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
C++ Modules in a Nutshell
�26
#include <vector>
![Page 27: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/27.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
C++ Modules in a Nutshell
�27
Textual Include PCH Modules
#include <vector>
ExpensiveFragile
Inseparable
![Page 28: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/28.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
C++ Modules in a Nutshell
#include "TVirtualPad.h"#include <vector>#include <set>
int main() {…
�28
Textual Include
Preprocess
…………
# 286 "/usr/include/c++/v1/vector" 2 3namespace std { inline namespace __1 {template <bool> class __vector_base_common { __attribute__ ((__visibility__("hidden"), __always_inline__)) __vector_base_common() {}
……# 394 "/usr/include/c++/v1/set" 3namespace std {inline namespace __1 {template <…> class set {public: typedef _Key key_type;
……
int main {……
Parse.o
vector
set
TVirtualPad.h
Compile
original code
one big file!
![Page 29: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/29.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
C++ Modules in a Nutshell
�29
#define PI 3.14…
Rcpp library
Users’ code#include <header.h>
…double PI = 3.14;// => double 3.14 = 3.14;
Textual Include
1. ExpensiveReparse the same header
2. FragileName collisions
.h
.c
.c
.c .o
.o
.o
![Page 30: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/30.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
1. Storing pre compiled header information (same as modules)
2. Stored in one big file
�30
PCH (Pre Compiled Header)
C++ Modules in a Nutshell
![Page 31: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/31.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter
b.pcma.pcm d.pcmc.pcm e.pcm
�31
C++ Modules in a Nutshell
- Pre compiled PCM files contain header information- PCMs are separated
Modules
![Page 32: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/32.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
C++ Modules in a Nutshell
�32
Compile-time scalabilityFragilitySeparable
Modules
- Pre compiled PCM files contain header information- PCMs are separated
![Page 33: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/33.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Implementation
�33
![Page 34: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/34.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Implementation
�34
![Page 35: Updates on ROOT C++ modules...Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter a.pcm b.pcm c.pcm d.pcm e.pcm 31 C++ Modules in a Nutshell - Pre compiled PCM files](https://reader034.vdocuments.us/reader034/viewer/2022051903/5ff48f34a6bafe687720e1fa/html5/thumbnails/35.jpg)
Yuka Takahashi 19.11.2018 Updates on ROOT C++ modules
Implementation
�35
Clang
- External project under LLVM- Bi-weekly meeting with C++
Modules community- Reporting & fixing bugs- ROOT is the largest user of
Modules outside industry