performance analytics from visualization to insight · – squeezing the information in the...
Post on 03-Sep-2020
0 Views
Preview:
TRANSCRIPT
www.bsc.es
Jesús LabartaBSC
Performance Analytics From visualization to insight
VI-HPC 10th Anniversary WorkshopFrankfurt, June 23rd 2017
BSC Tools framework
CUBE, gnuplot, vi…
.prv+
.pcf
.dim
Time Analysis, filters
.cfg
Paramedir
.prv
Instruction level simulators
Tasksim, Sniper, gem5
XMLcontrol
ParaverLD_PREPLOAD,
Valgrind, Dyninst, PINPAPI
MRNETExtrae
DIMEMAS
VENUS (IBM-ZRL)
Machine description
Trace handling & displaySimulatorsAnalytics
Open Source http://www.bsc.es/paraver
The importance of detail and flexibilitySince ~ 1991
Performance analytics
prv2dim
.xls.txt
.cube
.plot
Instr. traces
3
Performance Analytics
Performance analysis: THE big data app– 10000 cores x 1 event/100us x 100 bytes/event x 1000s = 10 GB– Easily underestimated terms by orders of magnitude
Performance Analytics:– Squeezing the information in the captured data Insight
• About Machine Learning … and beyond, and before– Some BSC activities in the last 10 year (~)
Vision– Have to leverage data processing methods from ALL areas– Have to balance between first principles, pure statistical, black box
4
BSC Performance Analytics
Instantaneous metrics for ALL hardware counters at “no” costAdaptive burst mode tracing
Tracking performance evolution
26.7MB traceEff: 0.43; LB: 0.52; Comm:0.81
1600
cor
es
2.5 s
BS
C-E
S –
EC
-EA
RTH
BS
C-E
S –
EC
-EA
RTH
AM
G20
13
Flexible trace visualization and analysis
Advanced clustering algorithms
5
BSC Performance Analytics
eff.csv
Several core counts
eff_factors.py
extrapolation.py
Dimemas
No MPI noise + No OS noise
“Scalability prediction for fundamental performance factors ” J. Labarta et al. SuperFRI 2014
Models and Projection Data access patterns
Runtime Energy and performance model (EAR)
Inte
l –B
SC
Exa
scal
eLa
b
Leno
vo -
BS
C
G. Llort et al, “Scalable tracing with dynamic levels of detail” ICPADS 2011
Structure detection
About Analytics AND infrastructure
T0
ClusteringAnalysis
MRNetFront-end
T1 Tn
…
Back-end threads
Aggregatedata
Broadcastresults
Structure detection
Intel –BSC Exascale Lab
MRNET
PyCOMPSs
7
What we use: POP
8
Performance Analytics
Leverage methods from ALL areas
Balance between first principles, pure statistical, black box
– “Proper” choice of feature vector
Leverage big data infrastructure
Lots of opportunities for the next 10 years !!!
top related