Download - An Automated Performance Evaluation Framework for the ...hanzt/talks/GAMM_GPE.pdf · Continuous Benchmarking Benefits • Archiving performance data along with execution parameters

KIT – The Research University in the Helmholtz Association

Hartwig Anzt, Terry Cojean, Goran Flegar, Thomas Grűtzmacher, Pratik Nayak, Tobias RibizelSteinbuch Centre for Computing (SCC)

www.kit.edu

AnAutomatedPerformanceEvaluationFrameworkfortheGinkgoSoftwareEcosystem

GAMM,February18th- 22nd,2019

MovingtowardsContinuousBenchmarking(CB)

GAMM20192 02/19/2019 Hartwig Anzt:AnautomatedperformanceevaluationframeworkfortheGinkgosoftwareecosystem

Developer

A Healthy Software Development Cycle


Developer


SourceCodeRepository

Push

• Versioncontrolsystemfortrackingchangesandcoordinatingcollaborativedevelopment.


ContinuousIntegration(CI)

CIBuild

Developer



Push


• CIBuildsystemcontinuously checksthecompilationsuccessondifferentarchitectures.



CIBuild

Developer



PushCITest



• Unittestsensurefunctionalityandvalidityofallbuildingblocks.



CIBuild

Developer



PushCITest

Schedule inBatchSystem

HPCSystem

CIBenchmarkTests




• Goal:Automatedbenchmarkrunseverytimenewcodeisadded.



CIBuild

Developer



PushCITest


HPCSystem

CIBenchmarkTests







CIBuild

Developer



PushCITest


HPCSystem

CIBenchmarkTests

SecurityThrust!• ExternalcodeisrunonHPCsystem• Possiblepointofattack







CIBuild

Developer



PushCITest


HPCSystem

CIBenchmarkTests

CodeReview

TrustedReviewer

MergeintoMasterBranch

• ManualcodereviewsbytrustedreviewerswhotakeresponsibilityforexecutingcodeontheHPCsystem.



CIBuild

Developer



PushCITest


HPCSystem

CIBenchmarkTests

CodeReview

TrustedReviewer


• ManualcodereviewsbytrustedreviewerswhotakeresponsibilityforexecutingcodeontheHPCsystem.

• Howtoarchivetheresultsandmakethemaccessible?



CIBuild

Developer



PushCITest


HPCSystem

CIBenchmarkTests

CodeReview

TrustedReviewer


PerformanceDataRepository

• Storedatainseparaterepository.• UseJSONdataexchangeformat.



CIBuild

Developer



PushCITest


HPCSystem

CIBenchmarkTests

CodeReview

TrustedReviewer



Users

• Storedatainseparaterepository.• UseJSONdataexchangeformat.• Cloningahugerepository is

expensive.• ”Interestinmyexperiments,only.”


CITest



DeveloperCodeReview

CIBuildSourceCodeRepository

Push


Web-Application

HPCSystem

TrustedReviewer

Users


CIBenchmarkTests


• Storedatainseparaterepository.• UseJSONdataexchangeformat.• Cloningahugerepository is

expensive.• ”Interestinmyexperiments,only.”• Interactiveweb-basedGinkgo

PerformanceExplorer(GPE).


CITest



DeveloperCodeReview

CIBuildSourceCodeRepository

Push


Web-Application

HPCSystem

TrustedReviewer

Users


CIBenchmarkTests


GPEGinkgoPerformanceExplorer


Ginkgo Performance Explorer (GPE)

DataSelectionTab

TransformationScriptEditor

DataandPlotViewer

1.SelectDatainGitrepository.

GPE



DataSelectionTab


DataandPlotViewer


2.WriteJSONatascripttovisualizedata(examplesareprovided).

GPE



DataSelectionTab


DataandPlotViewer


2.WriteJSONatascripttovisualizedata(examplesareprovided).

3.Analyzedatavisually.

GPE



GPE


Ginkgo’s Performance Evaluation Framework

GPE

ContinuousBenchmarkingBenefits

• Archivingperformancedataalongwithexecutionparametersensuresfullbenchmarkreproducibility.

• Comparing theperformanceresultsoverthecodelifetimeidentifiesperformancedegradations.

• Easeofuse:thesetupallowstolaunchbenchmarkwithfewclicks.


Ginkgo’s Performance Evaluation Framework

GPE

ContinuousBenchmarkingBenefits

• Archivingperformancedataalongwithexecutionparametersensuresfullbenchmarkreproducibility.

• Comparing theperformanceresultsoverthecodelifetimeidentifiesperformancedegradations.

• Easeofuse:thesetupallowstolaunchbenchmarkwithfewclicks.

GinkgoPerformanceExplorer(GPE)Benefits• ThedesignofGPEefficientlyrealizestheanalysisaswebservice,removingtheneedfordownloading

performancedatatolocaldiskorinstallingadditionalsoftware.

• ExternaldeveloperswithoutaccesstoHPCsystemscantestandengineertheircodesonHPCresources.

• Extensibility:Optiontocompareperformancewithothersoftware libraries.


• Open-sourceC++frameworkforsparselinearalgebra.

• Sparselinearsolvers,preconditioners,SpMV etc.

• Genericalgorithmimplementation:+referencekernelsforcheckingcorrectness;+architecture-specifichighlyoptimizedkernels.

• FocusedonGPUaccelerators(i.e.NVIDIAGPUs).

• SoftwarequalityandsustainabilityeffortsguidedbyxSDK communitypolicies:

•

•

•

•

https://xsdk.info/

PratikNayakTerryCojean GoranFlegar ThomasGrützmacher

TobiasRibizel

https://ginkgo-project.github.io/

Learn More about Ginkgo

Download - An Automated Performance Evaluation Framework for the ...hanzt/talks/GAMM_GPE.pdf · Continuous Benchmarking Benefits • Archiving performance data along with execution parameters

Top Related