apache systemml architecture by niketan panesar
TRANSCRIPT
SystemMLArchitectureNiketanPansare,BertholdReinwald
July25th,2016
Agenda
• High-levelDesign&APIs• ArchitectureOverview• Tooling• Importantlinks
2
Fromhttp://systemml.apache.org/
Agenda
• High-levelDesign&APIs• ArchitectureOverview
• Language• Compiler• Runtime
• Twoexamples:• SimpleDMLexpressionwithanexampledataset• LinearRegressionwithvaryingdatasizes
• Tooling• Importantlinks
3
Agenda
• High-levelDesign&APIs• ArchitectureOverview
• Language• Compiler• Runtime
• Twoexamples:• SimpleDMLexpressionwithanexampledataset• LinearRegressionwithvaryingdatasizes
• Tooling• Importantlinks
4
SystemML Design
5
DML (Declarative Machine Learning Language)
Hadoop or Spark Cluster (scale-out)
since 2010
In-Memory Single Node (scale-up)
since 2012 since 2015
DML Scripts
Data
CP+bsb_mVar1SPARKmapmmX_mvar1_mVar2RIGHTfalseNONECP*y_mVar2_mVar3
Hybridexecutionplans*
SystemML3.double [][]
1.Ondisk/HDFS
2.RDD/DataFrame
SystemML Design
6
Hadoop or Spark Cluster (scale-out)
since 2010
In-Memory Single Node (scale-up)
since 2012
DML Scripts
Data
SystemML
1.Ondisk/HDFS
2.RDD/DataFrame
3.double [][]
Command line API*
(also MLContext*)
-exechadoop
SystemML Design
7
Hadoop or Spark Cluster (scale-out)
In-Memory Single Node (scale-up)
since 2012
DML Scripts
Data
SystemML
1.Ondisk/HDFS
2.RDD/DataFrame
3.double [][]
Twooptions:1. –execsinglenode2. Usestandalonejar(preservesrewrites,but
mayspawnLocalMRjobs)
Command line API*
(also MLContext*)
SystemML Design
8
Spark Cluster (scale-out)
In-Memory Single Node (scale-up)
since 2012 since 2015
DML Scripts
Data
SystemML
1.Ondisk/HDFS
2.RDD/DataFrame
3.double [][]
Command line API*
(also MLContext*)
SystemML Design
9
Spark Cluster (scale-out)
In-Memory Single Node (scale-up)
since 2012 since 2015
DML Scripts
Data
SystemML
1.Ondisk/HDFS
2.RDD/DataFrame
3.double [][]
MLContext API- Java/Python/Scala
https://apache.github.io/incubator-systemml/spark-mlcontext-programming-guide.html
SystemML Design
10
In-Memory Single Node (scale-up)
since 2012
DML Scripts
Data
SystemML
1.Ondisk/HDFS
2.RDD/DataFrame
3.double [][]
JMLC API
https://apache.github.io/incubator-systemml/jmlc.html
Agenda
• High-levelDesign&APIs• ArchitectureOverview
• Language• Compiler• Runtime
• Twoexamples:• SimpleDMLexpressionwithanexampledataset• LinearRegressionwithvaryingdatasizes
• Tooling• Importantlinks
11
From DML to Execution Plan
12
Hadoop or Spark Cluster (scale-out)
In-Memory Single Node (scale-up)
DML Scripts DML (Declarative Machine Learning Language)
since 2010since 2012 since 2015
Data
CP+bsb_mVar1SPARKmapmmX_mvar1_mVar2RIGHTfalseNONECP*y_mVar2_mVar3
Hybridexecutionplans*
SystemML
From DML to Execution Plan
13
Hadoop or Spark Cluster (scale-out)
In-Memory Single Node (scale-up)
Runtime
Compiler
Language
DML Scripts DML (Declarative Machine Learning Language)
since 2010since 2012 since 2015
Data
CP+bsb_mVar1SPARKmapmmX_mvar1_mVar2RIGHTfalseNONECP*y_mVar2_mVar3
Hybridexecutionplans*
Assuming anexampledatasetX:100MX500,y:100MX1,b/sb:500X1
SystemML Compilation Chain
14
SystemML Compilation Chain
15
• Parsing• Parse input DML/PyDML using Antlr v4 (see Dml.g4 and Pydml.g4) • Perform syntactic validation• Construct DMLProgram (=> list of Statement and function blocks)
• Live Variable Analysis• Classic dataflow analysis• A variable is “live” if it holds value that may be needed in future• Dead code elimination
• Semantic Validation
SystemML Compilation Chain
16
• Dataflow in DAGs of operations on matrices, frames, and scalars• Choosing from alternative execution plans based on memory and cost estimates• Operator ordering & selection; hybrid plans
SystemML Compilation Chain
17
*DiscussedlaterinTooling
spark-submit--masteryarn-client --driver-memory20G--num-executors 4--executor-memory 40G--executor-cores 24SystemML.jar-ftest.dml-explain hops
SystemML Compilation Chain
18
• Low-level physical execution plan (LOPDags) • Over key-value pairs for MR• Over RDDs for Spark
• “Piggybacking” operations into minimal number Map-Reduce jobs
SystemML Compilation Chain
19
Spark
CP + b sb _mVar1SPARK mapmm X.MATRIX.DOUBLE _mvar1.MATRIX.DOUBLE
_mVar2.MATRIX.DOUBLE RIGHT false NONECP * y _mVar2 _mVar3
SystemML Runtime• Hybrid Runtime• CP: single machine operations & orchestrate jobs• MR: generic Map-Reduce jobs & operations• SP: Spark Jobs
• Numerically stable operators• Dense / sparse matrix representation• Multi-Level buffer pool (caching) to evict in-memory
objects• Dynamic Recompilation for initial unknowns
Control Program
RuntimeProgram
BufferPool
ParFor Optimizer/Runtime
MRInstSpark
InstCPInst
Recompiler
DFSIOMem/FSIO
GenericMRJobs
MatrixBlock Library(single/multi-threaded)
From DML to Execution Plan
21
Hadoop or Spark Cluster (scale-out)
In-Memory Single Node (scale-up)
Runtime
Compiler
Language
DML Scripts DML (Declarative Machine Learning Language)
since 2010since 2012 since 2015
Data
CP+bsb_mVar1SPARKmapmmX_mvar1_mVar2RIGHTfalseNONECP*y_mVar2_mVar3
Hybridexecutionplans*
Varyingdatasizes
LinearRegression.dml
ADataScientist– LinearRegression
22
X ≈
Explanatory/Independent Variables
Predicted/Dependant VariableModel
w
w = argminw ||Xw-y||2 + λ||w||2Optimization Problem:
next direction
Iterateuntilconvergence
initialize
stepsize
update w
initialdirection
accuracymeasures
Conjugate Gradient Method:• Start off with the (negative) gradient• For each step
1. Move to the optimal point along the chosen direction;2. Recompute the gradient;3. Project it onto the subspace conjugate* to all prior directions; 4. Use this as the next direction
(* conjugate = orthogonal given A as the metric)
A = XT X + λ
y
SystemML– RunLinRegCGonSpark
23
100M
10,000
100M
1yX
100M
1,000X
100M
100X
100M
10X
100M
1y
100M
1y
100M
1y
8 TB
800 GB
80 GB
8 GB …tMMp
…
Multithreaded Single Node
20 GB Driver on 16c6 x 55 GB Executors
Hybrid Planwith RDD caching
and fused operator
Hybrid Planwith RDD out-of-core and fused
operator
Hybrid Planwith RDD out-of-core and different
operators
…x.persist();
...
X.mapValues(tMMv)
.reduce ()…
Driver
Fused
Executors
…
RDDcache:X
tMMv tMMv
…x.persist();
...
X.mapValues(tMMv).reduce()
...Executors
…
RDDcache:X
tMMv tMMv
Driver
Spilling
…x.persist();
...// 2 MxV mult
// with broadcast,// mapToPair, and
// reduceByKey... Executors
…
RDDcache:X
Mv
tvM
Mv
tvM
Driver
Driver
Cache
Agenda
• ArchitectureOverview• Language&APIs• Compiler• Runtime
• Twoexamples:• SimpleDMLexpressionwithanexampledataset• LinearRegressionwithvaryingdatasizes
• Tooling• Importantlinks
24
SystemML’sCompilationChain/OverviewTools
25
EXPLAINhops
STATS
DEBUG
EXPLAINruntime
[MatthiasBoehmetal:SystemML'sOptimizer:PlanGenerationforLarge-ScaleMachineLearningPrograms.IEEEDataEng.Bull2014]
HOP (High-leveloperator)LOP(Low-leveloperator)
EXPLAIN*_recompile
Explain(UnderstandingExecutionPlans)• Overview
• Showsgeneratedexecutionplan(atdifferentcompilationsteps)• Introduced05/2014forinternalusage• Importanttoolforunderstanding/debugging optimizerchoices!
• Usage• hadoop jar SystemML.jar -f test.dml –explain
[hops | runtime | hops_recompile | runtime_recompile]• Hops
• Programw/hopdagsafteroptimization• Runtime(default)
• Programw/generatedruntime instructions• Hops_recompile:
• Seehops+hopdagaftereveryrecompile• Runtime_recompile:
• Seeruntime+generatedruntime instructionsaftereveryrecompile
26
Explain:UnderstandingHOPDAGs(simpleDML)
27
Spark• HOPID• HOPopcode• HOPinputdatadependencies (viaHOPIDs)• HOPoutputmatrixcharacteristics(rlen,clen,brlen,bclen,nnz)• Hopmemoryestimates(allinputs, intermediates,outputà
operationmem)• Hopexecutiontype(CP/SP/MR)• Optional:indicatorsofreblock/checkpointing (caching)ofhop
outputs
-explainhops-explainrecompile_hops
spark-submit--masteryarn-client --driver-memory20G--num-executors 4--executor-memory 40G--executor-cores 24SystemML.jar-ftest.dml-explain hops
Broadcastmembudget
Explain:UnderstandingHOPDAGs(entirescript)
• ExampleDMLScript(SimplifiedLinregDS)
28
X = read($1);y = read($2);intercept = $3; lambda = $4;
if( intercept == 1 ) {ones = matrix(1, nrow(X), 1); X = append(X, ones);
}
I = matrix(1, ncol(X), 1);A = t(X) %*% X + diag(I*lambda);b = t(X) %*% y;beta = solve(A, b);
write(beta, $5);
Invocation:hadoop jar SystemML.jar -f linregds.dml -args X y 0 0 beta
Scenario:X:100,000x1,000,1.0y:100,000x1,1.0(800MB,200+GFlop)
Explain:UnderstandingHOPDAGs(2)• ExplainHops
29
15/07/05 17:18:06 INFO api.DMLScript: EXPLAIN (HOPS):# Memory Budget local/remote = 57344MB/1434MB/1434MB# Degree of Parallelism (vcores) local/remote = 24/144/72PROGRAM--MAIN PROGRAM----GENERIC (lines 1-4) [recompile=false]------(10) PRead X [100000,1000,1000,1000,100000000] [0,0,763 -> 763MB], CP------(11) TWrite X (10) [100000,1000,1000,1000,100000000] [763,0,0 -> 763MB], CP------(21) PRead y [100000,1,1000,1000,100000] [0,0,1 -> 1MB], CP------(22) TWrite y (21) [100000,1,1000,1000,100000] [1,0,0 -> 1MB], CP------(24) TWrite intercept [0,0,-1,-1,-1] [0,0,0 -> 0MB], CP------(26) TWrite lambda [0,0,-1,-1,-1] [0,0,0 -> 0MB], CP----GENERIC (lines 11-16) [recompile=false]------(42) TRead X [100000,1000,1000,1000,100000000] [0,0,763 -> 763MB], CP------(52) r(t) (42) [1000,100000,1000,1000,100000000] [763,0,763 -> 1526MB]------(53) ba(+*) (52,42) [1000,1000,1000,1000,-1] [1526,8,8 -> 1541MB], CP------(43) TRead y [100000,1,1000,1000,100000] [0,0,1 -> 1MB], CP------(59) ba(+*) (52,43) [1000,1,1000,1000,-1] [764,0,0 -> 764MB], CP------(60) b(solve) (53,59) [1000,1,1000,1000,-1] [8,8,0 -> 15MB], CP------(66) PWrite beta (60) [1000,1,-1,-1,-1] [0,0,0 -> 0MB], CP
ClusterCharacteristics
ProgramStructure(inclrecompile)
UnrolledHOPDAG
Notes:ifbranch(6-9)andregularization removedbyrewrites
Explain:UnderstandingRuntimePlans(1)• ExplainRuntime(simplifiedfilenames,removedrmvar)
30 IBMResearch
15/07/05 17:18:53 INFO api.DMLScript: EXPLAIN (RUNTIME):# Memory Budget local/remote = 57344MB/1434MB/1434MB# Degree of Parallelism (vcores) local/remote = 24/144/72PROGRAM ( size CP/MR = 25/0 )--MAIN PROGRAM----GENERIC (lines 1-4) [recompile=false]------CP createvar pREADX X false binaryblock 100000 1000 1000 1000 100000000------CP createvar pREADy y false binaryblock 100000 1 1000 1000 100000------CP assignvar 0.SCALAR.INT.true intercept.SCALAR.INT------CP assignvar 0.0.SCALAR.DOUBLE.true lambda.SCALAR.DOUBLE------CP cpvar pREADX X------CP cpvar pREADy y----GENERIC (lines 11-16) [recompile=false]------CP createvar _mVar2 .../_t0/temp1 true binaryblock 1000 1000 1000 1000 -1------CP tsmm X.MATRIX.DOUBLE _mVar2.MATRIX.DOUBLE LEFT 24------CP createvar _mVar3 .../_t0/temp2 true binaryblock 1 100000 1000 1000 100000------CP r' y.MATRIX.DOUBLE _mVar3.MATRIX.DOUBLE------CP createvar _mVar4 .../_t0/temp3 true binaryblock 1 1000 1000 1000 -1------CP ba+* _mVar3.MATRIX.DOUBLE X.MATRIX.DOUBLE _mVar4.MATRIX.DOUBLE 24------CP createvar _mVar5 .../_t0/temp4 true binaryblock 1000 1 1000 1000 -1------CP r' _mVar4.MATRIX.DOUBLE _mVar5.MATRIX.DOUBLE------CP createvar _mVar6 .../_t0/temp5 true binaryblock 1000 1 1000 1000 -1------CP solve _mVar2.MATRIX.DOUBLE _mVar5.MATRIX.DOUBLE _mVar6.MATRIX.DOUBLE------CP write _mVar6.MATRIX.DOUBLE .../beta.SCALAR.STRING.true textcell.SCALAR.STRING.true
Literallyastringrepresentationofruntime instructions
Stats(ProfilingRuntimeStatistics)
• Overview• Profilesandshowsaggregatedruntimestatisticsofpotentialbottlenecks• Introduced01/2014forinternalusage,extensionofbufferpoolstats01/2013• Importanttoolforunderstandingruntimecharacteristicsandprofiling/tuningsysteminternalsbydevelopers
• Usage• hadoop jar SystemML.jar -f test.dml -stats
31 IBMResearch
SystemML Statistics
Totalexectime
Bufferpool stats
Dynamicrecompilationstats
JVMstats(JIT,GC)
Heavyhitterinstructions(incl.bufferpool times)
optional:parforstats(ifprogramcontainsparfors)
Debug(ScriptDebugging)• Overview
• Script-leveldebuggingbyend-users(anddevelopers)• Introduced09/2014asresultofinternproject• gdb-inspiredcommand-linedebuggerinterface
• Usage• hadoop jar SystemML.jar -f test.dml -debug
33
Agenda
• ArchitectureOverview• Language&APIs• Compiler• Runtime
• Twoexamples:• SimpleDMLexpressionwithanexampledataset• LinearRegressionwithvaryingdatasizes
• Tooling• Importantlinks
34
Important Links• Website:http://systemml.apache.org/
35
Important Links• Website:http://systemml.apache.org/
• InterestedinSystemML?• Gotohttps://github.com/apache/incubator-systemml and“Starit”
36
Important Links• Website: http://systemml.apache.org/• Interested in SystemML ?
• Go to https://github.com/apache/incubator-systemml and “Star it”• Want to contribute to SystemML ?
• See http://apache.github.io/incubator-systemml/contributing-to-systemml.html
• List of issues: https://issues.apache.org/jira/browse/SYSTEMML/• Ask any of our PMC members for suggestions
• Want to try out SystemML ?• Laptop: http://apache.github.io/incubator-systemml/quick-start-guide.html
(Does not require Hadoop/Spark installation)• Spark Cluster: http://apache.github.io/incubator-systemml/spark-
mlcontext-programming-guide.html (Includes Jupyter/Zeppelin demo)
37
ThankYou