on utilizing experiment data repository for performance ...scalea (sc01, euro-par02, pvmmpi02)...
TRANSCRIPT
1
On Utilizing Experiment Data Repository for Performance Analysis
of Parallel Applications Hong-Linh Truong1, Thomas Fahringer2
1 Institute for Software Science,University of Vienna, Austria
2 Institute of Computer Science,University of Innsbruck, Austria
www.par.univie.ac.at/project/scalea
EURO-PAR 2003, Klagenfurt, August 26th, 2003
2
OutlineOutlineØØMotivationMotivationØØSCALEASCALEAØØExperiment Data RepositoryExperiment Data Repository
ExperimentExperiment--related Datarelated DataImplementation OverviewImplementation OverviewExperimentExperiment--related APIsrelated APIs
ØØAchievementsAchievementsSearch and Filter Performance DataSearch and Filter Performance DataMultiMulti--Experiment AnalysisExperiment AnalysisTools IntegrationTools Integration
ØØConclusions and Future WorkConclusions and Future Work
3
MotivationMotivationØØThe need to collect and archive performance data The need to collect and archive performance data forfor
MultiMulti--experiment analysisexperiment analysisPerformance comparisonPerformance comparison
ØØLimitation on supporting code complexity:Limitation on supporting code complexity:Handle largeHandle large--scale performance datascale performance dataBasic search on performance dataBasic search on performance data
ØØLack of capabilities to export, share, and exchange Lack of capabilities to export, share, and exchange performance data and tools interoperationperformance data and tools interoperation
4
SCALEA (SC01, SCALEA (SC01, EuroEuro--Par02Par02, PVMMPI02) , PVMMPI02) OpenMP,MPI,HPF, hybrid programs
Instrumented programs
Instrumentation description file
Executable Programs
System Sensors
Instrumentation Control
SIS Instrumentation Libraries:
SISPROFILING
Instrumentation
Compilation
Execution
EnvironmentPerformance
Analysis
Experiment Data
Repository
Target machines
SCALEA Sensor Manager
SCALEA Instrumentation System(SIS)
SCALEA Performance Analysis & Visualization
SCALEA Runtime System
5
ExperimentExperiment Data RepositoryData RepositoryØ For comparing performance across experiments with large amounts of data
Need to archive performance dataRequire a very strong and flexible database system
ØData repository is first step required by mining, knowledge discovery in performance dataØExperiment Data Repository:
Stores information about application, source code, machine information, and performance dataAssociates performance results with source code and machine informationSupports sophisticated analysis and easy integration with other tools
6
Data ModelData Model
1:n1:n 1:n
1:n
1:nn:1
1:n
m:n
1:n1:n
n:1 1:n
1:n
1:n
1:n1:n
7
Performance Metric CatalogPerformance Metric CatalogØWhy performance metric catalog:
Documentation about metrics providedHelp tools/users understanding and interpreting performance data
ØPerformance metric has 4 propertiesunique name is a value that distinguishes this metric from all othersdata type describes data type of measurement which describes the value of the metric.measurement unit is the unit of measurementsemantics (or well-defined meaning)
ØAll metrics are stored in a catalog in experiment data repository
8
Implementation Overview Implementation Overview
ØØPowering search and archive with great flexibility and robustnesPowering search and archive with great flexibility and robustnesssRelational database based on Relational database based on PostgreSQLPostgreSQL
ØØTaking advantages of portability and network capabilityTaking advantages of portability and network capabilityAll components written in JavaAll components written in JavaDatabase connection powered by JDBC Database connection powered by JDBC
9
ExperimentExperiment--related Data APIsrelated Data APIs
public class PerformanceMetric { private String metricName;private Object metricValue;public String getMetricName(){…};public void setMetricName(String metricName){…}; public Object getMetricValue(){…};public void setMetricValue(Object metricValue){…};... }
public class ProcessingUnit {private String computationalNode;private int processID ;private int threadID ;…public ProcessingUnit(String node,int process, int thread) {...}…}
ØØWellWell--defined APIs for accessing data in defined APIs for accessing data in Experiment Data Repository Experiment Data Repository ØØ Java implementationJava implementation
10
ExperimentExperiment--related Data APIs (const.)related Data APIs (const.)
public class ExperimentData {DatabaseConnection connection; ...public ProcessingUnit[] getProcessingUnits(Experiment e){...}public RegionSummary[] getRegionSummaries(CodeRegion cr, Experiment e){...}public RegionSummary getRegionSummary(CodeRegion cr, ProcessingUnitpu, Experiment e) {...}public RegionSummary getRegionSummary(CodeRegion cr, Experiment e,ProcessingUnit pu, RegionSummary parent) {...} public RegionSummary[] getChildOfRegionSummary(RegionSummary rs){...} public RegionSummary getParentOfRegionSummary(RegionSummary rs){...}...
}
public class RegionSummary {... public PerformanceMetric[] getMetrics(){...}public PerformanceMetric getMetric(String metricName){...}...}
11
CodeRegion cr = new CodeRegion(“IONIZE_MOVE”);Experiment e = new Experiment(“9Nx4P,P4”);ProcessingUnit
pu =new ProcessingUnit(``gsr402.vcpc.ac.at'',2,0);
ExperimentData ed = new ExperimentData(new DatabaseConnection(...));
RegionSummary rs = ed.getRegionSummary (cr,pu,e);
PerformanceMetric overhead =rs.getMetric(''oall_ident'');
PerformanceMetricwtime =rs.getMetric(``wtime'');
double overheadRatio=((Double)overhead.getMetricValue()).doubleValue()/
((Double)wtime.getMetricValue()).doubleValue();
Example of Using APIsExample of Using APIs
12
XML DataXML Data
ØFacilitate information exchange between toolsØAllow tools interoperation in a uniform way ØWith/without source code ØCall-graph, selective metrics
Performance Metric Catalog
13
Search and Filter CapabilitiesSearch and Filter Capabilities
ØExisting problems: - difficult to find interesting events via process time-lines, zooming, scrolling, call-graph-search and filter data based on files are not efficient, robust and fast
ØSearch and filter data based relational database and SQL: flexibility and robustnessØWe support generic search and filter
Picture taken from D. Picture taken from D. KranzlmuellerKranzlmueller ´s ´s talktalk
14
Search and Filter CapabilitiesSearch and Filter Capabilities
15
Search and Filter Capabilities (const.)Search and Filter Capabilities (const.)
16
Search and Filter Capabilities (const.)Search and Filter Capabilities (const.)ØDefining an expression of performance metrics ØDiscretizing the expression into quantitative characteristicsØSearch based on selected quantitative characteristics
17
MultiMulti--Experiment AnalysisExperiment AnalysisØMany ways to specify what you want to analysis
ExperimentsCode regionsPerformance metricsStatistic methods
ØVarious analysesPerformance comparison for different sets of experimentsOverhead analysis for multi-experimentPerformance speedup/improvement at both program and code region level
18
Select what to be analyzedSelect what to be analyzed
19
Different Sets of ExperimentsDifferent Sets of Experiments
21
Improvement/Speedup, EfficiencyImprovement/Speedup, Efficiency
22
System InformationSystem Information
Tools IntegrationTools Integration
ArchitecturesArchitecturesq NOWsq PC-Clustersq SMP Clustersq GRID Systemsq DM/SM Systems
q Parameter Studiesq Experiment
Management
ZenturioZenturio
Programming Programming ParadigmsParadigmsq MPI,GlobusMPIq OpenMP/MPIq HPF/OpenMPq JavaSymphony
ExperimentData
Repository
http://www.par.univie.ac.at/project/askalon/
q Instrumentationq Measuring q Performance Analysis
SCALEASCALEA
q Modelingq Simulationq Performance
Prediction
PerformancePerformanceProphetProphet
q AutomaticBottleneckAnalysis
AksumAksum
24
SCALEA
AKSUM
Experiment Data Repository
Applicationfiles
Instrument application
files
Files were instrumented
AKSUM employs SCALEA to:
• instrument files given an arbitrary code region
• transfer application files to the repository
• transfer the data generated by the monitoring to the repository
•use data provided by SCALEA for property analysis
Case 1: AKSUM ØØ URL: URL: http://www.par.http://www.par.univieunivie.ac.at/project/.ac.at/project/aksumaksum//
ØØFahringerFahringer and and SeragiottoSeragiotto, Jr., Jr.
Case 2: Performance ProphetCase 2: Performance ProphetØØ URL: URL: http://www.par.http://www.par.univieunivie.ac.at/project/pro.ac.at/project/prophet/phet/
ØØ FahringerFahringer, , PllanaPllana, and , and Testori Testori
ØØPP is a performance modeling and PP is a performance modeling and prediction systemprediction system
ØØ PP utilizes SCALEA to obtain timing PP utilizes SCALEA to obtain timing parameters for application.parameters for application.
ØØPP uses performance data in XML format PP uses performance data in XML format exported by SCALEA for automatic building exported by SCALEA for automatic building of cost functionsof cost functionsØØ Cost functions are used to develop a Cost functions are used to develop a hybrid analytical and simulation model of the hybrid analytical and simulation model of the applicationapplication
Instrument the application
Execute the application
Set the problem and/or machine size
Store the data in the performance database
[All experimentsdone]
[Else]
Build the cost functions
26
ConclusionsConclusions and Futureand Future WorkWorkØØConclusionsConclusions
Design of experiment data repository for performance Design of experiment data repository for performance analysis toolanalysis toolDemonstration of achievements gaining when employing Demonstration of achievements gaining when employing experiment data repositoryexperiment data repositoryààData repository has increasingly supported the automation Data repository has increasingly supported the automation
of performance analysis and optimization processof performance analysis and optimization process
ØØOngoing workOngoing workWorking on simple and efficient way to search on Working on simple and efficient way to search on performance dataperformance dataApplying automatic scalable analysis techniquesApplying automatic scalable analysis techniquesSemantic representation of performance dataSemantic representation of performance data
www.par.www.par.univieunivie.ac.at/project/.ac.at/project/scaleascalea