![Page 1: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/1.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Presented by: Alparslan SARIDept of Computer & Information Sciences
University of Delaware
Collective Optimization
Grigori Fursin and Olivier Temam
{grigori.fursin,olivier.temam}@inria.fr
![Page 2: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/2.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Overview
• Introduction
• Experimental Setup
• Motivation
• Overview
• Collective Learning
• Collective Compiler
• Performance Evaluation
• Background and Related Work
• Conclusion and Future Work
![Page 3: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/3.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Introduction
• What is Iterative Compilation?
• Iterative Compilation vs Static Compiler Optimization
- Outperform?
- Quickly adapt to complex processor architecture?
- Machine Learning Algorithms?
![Page 4: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/4.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Introduction
• How are they overcome that practical obstacle using Collective Optimization?
- A Central Database
- Query for Optimization Suggestions
- Recompile
• Is it even make sense to use Collective Optimization?
![Page 5: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/5.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Introduction
• The most important hurdle is that iterative techniques almost rely on a large number of training runs.
• Why is that?
- Optimization Space?
![Page 6: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/6.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Introduction
• What is the key research issue?
• - Improve overall program performance
- Learn how it reacts to the various optimizations
![Page 7: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/7.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Experimental Setup
• GCC 4.2.0 compiler
• 88 program transformations has identified.
• Transformations are randomly selected
• AMD Athlon XP 2800 (AMD32) – 5 machines
• AMD Athlon 64 3700+ (AMD64) – 16 machines
• Intel Xeon 2.80GHz (IA32) – 2 machines
![Page 8: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/8.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Experimental Setup
Figure 1 from pg. 3, Collective Optimization, G. Fursin, O. Temam
![Page 9: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/9.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Experimental Setup
• Collecting information on a program run
- add to each program a routine(executed@end)
- it collects a program identifier
- architecture
- compiler identifiers
- applied optimizations
![Page 10: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/10.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Experimental Setup
• Last run
- performance measurements
- currently execution time
- profiling information
• After the information collection what next?
- Store them to database
![Page 11: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/11.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Motivation
Figure 2 from pg. 4, Collective Optimization, G. Fursin, O. Temam
![Page 12: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/12.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Overview
Figure 1 from pg. 3, Collective Optimization, G. Fursin, O. Temam
![Page 13: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/13.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Overview
• “Maturation” stages of a program
- Stage 1 : Program unknown, d1
- Stage 2 : Program known, a few runs only, d2
- Stage 3 : Program well known,heavily used, d3
![Page 14: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/14.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Collective Learning
Figure 2 from pg. 4, Collective Optimization, G. Fursin, O. Temam
![Page 15: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/15.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Collective Learning
• Building the program distribution d3 using statistical comparison of optimizations combinations
- Comparing two combinations C1, C2
- Execution times T1, T2
- T1 < T2 ?
• Cloned functions used
- if T1 < T2 then C1 > C2
• This approach requires no reference, test or training run.
![Page 16: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/16.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Collective Learning
• Building the aggregate distribution d1
- d1 is simply the average of all d3 distributions of each program.
- d1 reflects most common cases
![Page 17: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/17.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Collective Learning
• Building the matching distribution d2
- Characterize programs
- C1 > C2 is a reaction to program optimizations
• John Cavazos have shown that its possible to improve similar program characterizations by identifying and then restricting to optimizations which carry the most information using the mutual information criterion.
![Page 18: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/18.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Collective Compiler
• Program identification : uniquely identified using a 32byte MD5 checksum of all the files in its source directory.
• Termination routine : main() - exit()
• Cloning : Optimizations are evaluated through cloned routines. (Using gprof utility)
• Security : very little program info sent to DB(G/L)
![Page 19: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/19.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Performance Evaluation
Figure 3 from pg. 10, Collective Optimization, G. Fursin, O. Temam
![Page 20: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/20.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Performance Evaluation
![Page 21: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/21.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Background and .
• Several research works have shown how machine learning and statistical techniques can be used to select or tune program transformations based on program features.
• Java JIT compiler
- Tuning
- Predict optimization
• They have focus more on the impact of data sets from multiple users and the optimization selection robustness.
![Page 22: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/22.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Conclusions
• First contribution : identify the true limitations of the adoption of iterative optimization in production environment.
• The Second : showing that it is possible to simultaneously learn and improve performance across runs.
![Page 23: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/23.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Conclusions
• The Third : propose multi-level competition to understand the impact of optimization without even a reference run for computing speedups.
• The Fourth : highlight that knowledge accumulated across data sets for a single program is more useful, in the real and practical context of collective optimization, than the knowledge accumulated across programs.
![Page 24: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/24.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Questions
![Page 25: CISC 879 - Machine Learning for Solving Systems Problems Presented by: Alparslan SARI Dept of Computer & Information Sciences University of Delaware asari@udel.edu](https://reader030.vdocuments.us/reader030/viewer/2022033108/56649efd5503460f94c10bab/html5/thumbnails/25.jpg)
CISC 879 - Machine Learning for Solving Systems Problems
Questions