scheduling jobs with dependenciessamir/dcscheduling18/slides...robert grandl, srikanth kandula,...

Scheduling Jobs With Dependencies: New Applications, Classic Problems

JanardhanKulkarni,MicrosoftResearch,Redmond.

31July2018,TTI,ChicagoTTICSUMMERWORKSHOP:DATACENTERSCHEDULINGFROMTHEORYTOPRACTICE

Roadmap

Ø  Whichtheorymodelsaremoreclosertodata-centersettings.

SrikanthKandulaRatulMahajanAmarPhanishayeeMoniaGhobadi

Ø Focusonalgorithms Evencomplexalgorithmscanhavealgorithmicintuitionswhichareusefulinpractice.

Ø OneexampleOnesystemheuristicandonecomplexprovablealgorithm(UsingLPHierarchies)thathasgoodheuristicvalue.

LuleåFBDataCenter,SouthofArticCircle


Itisbeautifullikethisfor3days…..


cold,cold,place…

5%“aslargeascities”

Efficiency Matters a Lot

Efficiency Matters a Lot:

“aslargeascities”Emphasis on Principled Algorithms

Cost

Time

Simpleheuristics

TheoreticallySoundAlgorithms

Simplicityisnoteverything!

How we Measure Efficiency

Ø Makespan

Minimizingthemaximumcompletiontimeamongasetofjobs.Lengthoftheschedule.

Ø  Average(ortotal)Flow-time(aka,JobCompletion-time)

•  sameasresponsetime•  measuresthetimeajobspendsinasystem

Fj = Cj � rj

How we Measure Efficiency

Ø Makespan

Minimizingthemaximumcompletiontimeamongasetofjobs.Lengthoftheschedule.

Ø  Average(ortotal)Flow-time(aka,JobCompletion-time)

•  sameasresponsetime•  measuresthetimeajobspendsinasystem

Fj = Cj � rj

Throughput,energy,fairness,utilization,etc..

Challenges of Data Center Scheduling

ResourcesHeterogeneous(FPGA+CPU,GPU+CPU)Multidimensional(CPU,memory,network)

JobsComplexdependencies:DAGs,Co-flows,etc.

Algorithms

Fast,simple,oftenonline.




Algorithms


Richtheorywithmanynicealgorithmswhenjobshavesimplestructures.

Scheduling on Heterogeneous Clusters

Ø SpecialpurposehardwareØ Datalocality

Ø Geographiclocation

Ø Privacyconcerns

Whyareclustersheterogeneous?

Scheduling on Heterogeneous Clusters

1000

100

300

jobsrunfasteronsomeclustersandsloweronothersModeling Heterogeneity

Jobsarriveovertime

jobs

machines

1151000…..10

661005…..98

11588…..13

1007889…..13

Modeling Heterogeneity jobsrunfasteronsomeclustersandsloweronothers

Jobsarriveovertime

jobs

machines

1151000…..10

661005…..98

11588…..13

1007889…..13

Heterogeneous == “Unrelated Machines Scheduling”

Assign(match)jobstoclusters+scheduletoMinimizeQoS.

Beautiful Algorithms For Unrelated Machines Scheduling Problems

MakespanFlow-timeEnergyLST’87 CGK’09 AGK’12 ST’89 AGK’12 KLS’10

Svensson’12 BK’15IKMP’14AAFPW’97 IKMP’14P’07

KD’18 A’06

Offline,Online,Multidimensional,Clairvoyant,Non-Clairvoyant,Stochastic,Truthfulness…`

Hasleadtodevelopmentofveryniceideas:Useofvertexsolutionsanddualityindesignofalgorithms,configurationLPs,potentialfunctions,connectionstogametheoreticideas…

Beautiful Algorithms For Unrelated Machines Scheduling Problems

MakespanFlow-timeEnergyLST’87 CGK’09 AGK’12 ST’89 AGK’12 KLS’10

Svensson’12 BK’15IKMP’14AAFPW’97 IKMP’14P’07

KD’18 A’06

Offline,Online,Multidimensional,Clairvoyant,Non-Clairvoyant,Stochastic,Truthfulness…`

Hasleadtodevelopmentofveryniceideas:Useofvertexsolutionsanddualityindesignofalgorithms,configurationLPs,potentialfunctions,connectionstogametheoreticideas…

RESEARCHDIRECTION:FewMachinetypes:Canwegetbetteralgorithmsforsomeclassicunrelated

machinesscheduling?




Algorithms


The plan

GRAPHENE:PackingandDependency-AwareSchedulingforData-ParallelClusters.OSDI2016.

OneHeuristic

RobertGrandl,SrikanthKandula,SriramRao,AdityaAkella,JanardhanKulkarni.

OneComplexTheoreticalFramework

Verygeneral,workswellinpractice,asbadasanyotheralgorithmonpaperJ

LeveyandRothvoss’16.Garg,Kulkarni,Li’18.Garg,Kukarni,Li’18.

Veryspecific,provable,andquitecomplex.

The plan


OneHeuristic




Oneofthebiggesthammersinapproximationalgorithms.“LiftandProject”



The plan


OneHeuristic






ADirectedAcyclicGraph(DAG)SchedulingProblem inLargeClusters



DAG Model Supported in Hadoop

Multidimensionality

Heterogeneityofclusters

Resourcesofacluster

(1,1,1)Dtypesofresources

Cluster Scheduling

AsinglejobrepresentedasaDAG(task)

Resourcesofacluster


Cluster Scheduling


DemandVector (1,0,…,1/2)

(1/2,1/2,…,1/2)

(1/4,1,…,1/10)

Resourcesofacluster


Cluster Scheduling


DemandVector (1,0,…,1/2)

(1/2,1/2,…,1/2)

(1/4,1,…,1/10)

Processinglength(duration)

Cluster Scheduling: Minimize Makespan

AsinglejobrepresentedasaDAG

(1,0),2

(0,1),1

(1,1),1

(1,1),1(0,1),1

1 234567 1 234567

(1,1)

Cluster

Is There a Good Algorithm?

Itisunlikely(UGC-hard)thatapolynomialtimealgorithmcanachievebetterthanDapproximationtotheDAGschedulingproblem.ThisholdsevenifalltasksoftheDAGhave1)samelength,2)requireexactlyoneresource.

Theorem:BansalandKhot‘09.

Ø  Anynon-idlingalgorithmisequallygoodorequallybad!

Notausefulintuitionforsystemdesigners.

Is There a Good Algorithm?

Itisunlikely(UGC-hard)thatapolynomialtimealgorithmcanachievebetterthanDapproximationtotheDAGschedulingproblem.ThisholdsevenifalltasksoftheDAGhave1)samelength,2)requireexactlyoneresource.

Theorem:BansalandKhot‘09.

1 234567

OptimalAlgorithm:Doagreedyschedulerespectingprecedenceconstraints

Atleastoneresourceisused.Congestionforthatresourcedecreases.

When did System Designers Care for Lowerbounds?



² CouldfindalmostoptimalsolutionsonMSdatasets.² Improvesmakespanby30%atleastcomparedtosimplegreedyheuristics.

Intuition of Graphene

“pathologicallybadschedulesintoday’sapproachesmostlyariseduetotworeasons:(a)long-runningtaskshavenootherworktooverlapwiththem,whichreducesparallelism,and(b)thetasksthatarerunnabledonotpackwellwitheachother,whichincreasesresourcefragmentation.”

Whatgreedyalgorithmsmiss?(List-Scheduling,CriticalPath,etc)


MainSteps

Ø  Ourapproachistoidentifythepotentiallytroublesometasks,suchasthosethatrunforaverylongtimeorarehardtopack.


MainSteps

Ø  Ourapproachistoidentifythepotentiallytroublesometasks,suchasthosethatrunforaverylongtimeorarehardtopack.Ø  Placethetroublesometasksfirstontoavirtualresource-timespace.Thisspacewouldhaved+1dimensionswhentasksrequiredresources;thelastdimensionbeingtime.


MainSteps

Ø  Ourapproachistoidentifythepotentiallytroublesometasks,suchasthosethatrunforaverylongtimeorarehardtopack.Ø  Placethetroublesometasksfirstontoavirtualresource-timespace.Thisspacewouldhaved+1dimensionswhentasksrequiredresources;thelastdimensionbeingtime.Ø  Ourintuitionisthatplacingthetroublesometasksfirstleadstoagoodschedulesincetheremainingtaskscanbeplacedintoresultantholesinthisspace.

Canweform

alizethisin

tuition?

A-ApproximationforMakespanSchedulingwithPrecedenceConstraintsusingLPHierarchies.(1 + ✏)

LeveyandRothvoss‘16

Identical Machines Scheduling

AsingleDAG.Eachtaskneedstobescheduledonexactlyonemachine.Eachtaskneeds1unitofCPU.

midenticalmachines(orCPUs)

MinimizeMakespan

(SpecialcaseofDAGscheduling)




Chainoflength4


GreedyorList-Schedulingis2approximationforminimizingmakespan.

Theorem.Graham1960.



Theorem.Graham1960.

BADSLOTS GOODSLOTS



Theorem.Graham1960.

BADSLOTS GOODSLOTS+Makespan



Theorem.Graham1960.

BADSLOTS GOODSLOTS

LengthofLongestchain n/m+Makespan



Theorem.Graham1960.

BADSLOTS GOODSLOTS

LengthofLongestchain n/m+Makespan OPT � OPT �



Theorem.Graham1960.

BADSLOTS GOODSLOTS


Ø Optimaltheoretically.Butconveysverylittleinformationinpractice.

Ø  Doesnotworkwellinpracticewhentherearemorethanoneresourcetype.


Thereisaquasi-polynomialtimeapproximationforminimizingmakespanwhenjobshaveunitlengths,whennumberofmachinesisaconstant.

Theorem.LevyandRothvoss’16.

(1 + ✏)

Garg’17madeitstrictlyquasi-polynomialtime.


Thereisaquasi-polynomialtimeapproximationforminimizingmakespanwhenjobshavearbitrarylengths,whennumberofmachinesisaconstant.Thealgorithmschedulesjobsonasinglemachineandmaypreemptjobswithinamachine.

Theorem.Kulkarni,Li’18.

(1 + ✏)


Thereisapolynomialtimeoptimalapproximationforminimizingweightedcompletiontimeofjobs,whennumberofmachinesandjobsizesareuniform.

Theorem.Garg,Kulkarni,Li’18.

(2 + ✏)



Theorem.Graham1960.

BADSLOTS GOODSLOTS


Crucial Observation

BADSLOTS GOODSLOTS

LengthofLongestchain n/m+Makespan OPT � ✏ ·OPT

(1 + ✏) ·OPT

Crucial Observation

BADSLOTS GOODSLOTS

LengthofLongestchain n/m+Makespan OPT � ✏ ·OPT

(1 + ✏) ·OPT

troublesometasks

Howtoscheduletroublesometasks?

Framework

TimeInterval

T0 T1 T2 T3

Partitionthetasksintoasetofbottomtasksandasinglesetoftoptasks.Foreachsetofbottomtaskswefindasub-intervalwheretheyshouldbescheduled.

Thendoarecursiveschedulingofbottomtasks.

Framework

TimeInterval

T0 T1 T2 T3

Toptasks

BottomtasksBottomtasksBottomtasks

Framework

TimeInterval

T0 T1 T2 T3


Precedenceconstraintsacrossbottomtasksareautomaticallysatisfied.

Framework

TimeInterval

T0 T1 T2 T3


Precedenceconstraintsgoingfrombottomtotoptasksareloose.

Framework

TimeInterval

T0 T1 T2 T3



[T2, T3]

Foreverytaskinthesetoftoptaskswehavebasedonthetentativeassignmentofbottomjobs.

T0 T1 T2 T3

[rj , dj ]


Thereisenoughspacetoscheduletoptasks

T0 T1 T2 T3


Thereisenoughspacetoscheduletoptasksiftherearenoprecedenceconstraintsbetweentoptasks.


[rj , dj ]

T0 T1 T2 T3



EDFwillschedulealltoptasksintheemptyspacebutmayviolatetheprecedenceconstraintsbetweentoptasks


[rj , dj ]


MainSteps


Framework

TimeInterval

T0 T1 T2 T3



[T2, T3]

Framework

TimeInterval

T0 T1 T2 T3



[T2, T3]

Thechainlengthamongtoptasksisverysmall.

Framework

TimeInterval

T0 T1 T2 T3



[T2, T3]


Thealgorithmhasrecognizedacrudeschedulefortroublesometasks.That’swhychainlengthamongtoptasksissmall.


MainSteps


LR’16 Framework

TimeInterval

T0 T1 T2 T3



[T2, T3]


Foreverytaskinthesetoftoptaskswehave

T0 T1 T2 T3

[rj , dj ]Precedenceconstraintsgoingfrombottomtotoptasksareloose.


EDFwillschedulealltoptasksintheemptyspacebutmayviolatetheprecedenceconstraintsbetweentoptasks

allexceptfew

TimeInterval

T0 T1 T2 T3

Toptasks


HowtopartitiontheDAG?

1.   Precedenceconstraintsbetweenbottomtasksshouldbeimplied.2.   Theprecedenceconstraintsbetweentopandbottomtasksareloose.3.   Thechainlengthamongtoptasksissmall.

Linear Programming Formulation

TX

t=1

xjt = 1

BinarysearchtheoptimalmakespanasT

Foreverytaskj

X

j

xjt m

isscheduled.

Fortimeslott hasatmostmjobs.

Forprecedencerelation issatisfiedateachtimestept.

xjt > 0Allvariables arenon-negative

i ! j,X

t0<t

xit0 �X

t0t

xjt0

LP Cheats…

2/3 1/3 2/3

Optimalmakespanis4butLPcancompletein3timeslots.

Time

DAG

LPcanscheduleajobfractionallyinatimeslot.

Interval of a task

Time

ConsidertheLPsolution.Intervalofataskissmallestintervalthatcontainsfractionalscheduleofthetask.

1/10 1/10 3/10 5/10

t1 t2

What LP gives? Anintervalforeachtask.

Time

What LP gives? Anintervalforeachtask.

Time

WeusetheseintervalstopartitiontheDAGintotopandbottomtasks.

Building Binary Tree

0 TLPSchedulesalltasksbetween [0, T ]

0 TT/2 T

2+ 1

0 T

4+ 1

T

4

T

2

Building Binary Tree LPSchedulesalltasksin[0, T ]

log T

[0, T ]

[0, T/2][T/2 + 1, T ]

Assigneachtasktothesmallestintervalnodeinthetreethatfullycontainsit.

Defining Top and Bottom Tasks [0, T ]

[0, T/2][T/2 + 1, T ]

(log log T )2


[0, T/2][T/2 + 1, T ]

(log log T )2

ThrowThemAway!!

log log T


[0, T/2][T/2 + 1, T ]

(log log T )2

TopTasks

BottomTasksSets

ThrowThemAway!!

log log T


[0, T/2][T/2 + 1, T ]

(log log T )2

TopTasks

BottomTasksSets

ThrowThemAway!!



[0, T/2][T/2 + 1, T ]

TopTasks

BottomTasksSets

ThrowThemAway!!

log log T

TimeInterval

0 T1 T2 T3

Precedenceconstraintsgoingfrombottomtotoptasksareloose. [T2, T3]

T4

Everytoptaskcanlooseoneintervaltotheleftandoneintervaltotherightintermsofspaceinwhichitshouldbescheduled.But,bottomintervalsaretinycomparedtotop,sothisisnotabigloss.

Toptasks

Bottomtasks

Bottomtasks

Lift and Project Method (LP Hierarchies) Dimensions

NumberofvariablesinLPthatyouwantintegral

OriginalLP

Allthevariablesareintegral.

Asystematicwayofplacingtroublesometasks!

Lift and Project Method (LP Hierarchies) Dimensions

NumberofvariablesinLPthatyouwantintegral

OriginalLP

Allthevariablesareintegral.

RunningtimeIncreasesbyafactorofn.

O(nS)

Asystematicwayofplacingtroublesometasks!

Lift and Project Method (LP Hierarchies)

Time

1/10 1/10 3/10 5/10

t1 t2

“Conditioning”

Touchavariable,anditbecomesintegral!


Time

10/10

“Conditioning”



Time

10/10

t1 t2“Conditioning”



Time

“Conditioning”


TheLPsolutionchangesinsuchawaythat,foreveryothertaskon,theintervalinwhichitisscheduledinthenewsolutiononlyshrinks.

Ihaveabetterunderstandingofwherethistaskgotscheduled.

Reducing Chain Length of Top Tasks [0, T ]

[0, T/2][T/2 + 1, T ]

TheintervalisoflengthT.Wewillmakesurethatthereisnochainoflengthassignedtothisinterval.

Reducing Chain Length of Top Tasks

0 T✏T



0 T✏Txjt > 0



0 T✏T

Reducing Chain Length of Top Tasks [0, T ]

[0, T/2][T/2 + 1, T ]



0 T✏T

Howmanyconditioningarerequired? m/✏Nowrecallthatnumberofintervalsintoptasksis 2(log logT )2 (log T )log log T



0 T✏T

Howmanyconditioningarerequired? m/✏Nowrecallthatnumberofintervalsintoptasksis 2(log logT )2 (log T )log log T

O(m/✏ · (log T )log log T )

Runningtime.

Thereisaquasi-polynomialtimeapproximationforminimizingmakespanwhenjobshavearbitrarylengths,whennumberofmachinesisaconstant.Thealgorithmschedulesjobsonasinglemachineandmaypreemptjobswithinamachine.

Theorem.Garg,Kulkarni,Li’18.

(1 + ✏)

Thereisapolynomialtimeoptimalapproximationforminimizingweightedcompletiontimeofjobs,whennumberofmachinesandjobsizesareuniform.

(2 + ✏)

Moresophisticateduseofconditioningandnewalgorithmsforschedulingtoptasks.


IntuitionofGraphene

Ø  UsingLiftandProjecttofigureoutplacinglongtasks.Isthereasimple,sayDPapproachtoit?Ø  CanweuseLPsupportforplacingtasks?

Ø  CanrecursionhelpinGraphenesetting?

LiftandProjectAlgorithms

Big Picture

IdenticalMachinesSchedulingandTrainingNeuralNetworks

PipeDream:FastandEfficientPipelineParallelDNNTrainingAaronHarlap,DeepakNarayanan,AmarPhanishayee,VivekSeshadri,NikhilDevanur,GregGanger,PhilGibbons

Training Deep Learning Models

Ø  Largefractionofthedatacenterworkloadsformanycompanies.

Ø  Improvingtrainingtimeisconsideredveryimportant.

Ø  DAGsaregoodabstractionsofDNNtrainingcomputations.

Ø  ConnectionstoDAGschedulingandcommunicationdelayproblems.

Two Paradigms

DataParallelism

ModelParallelism

Model Parallelism

Ø Schedulethelayersamongasetofmachines.TypicallyIdentical.

Ø Oratmost2types:CPU+FPGA,CPU+GPUsetc.

Model Parallelism

Ø Schedulethelayersamongasetofmachines.TypicallyIdentical.

Ø Oratmost2types:CPU+FPGA,CPU+GPUsetc.

Ø Thereiscommunicationbetweenlayers.Communicationcostiscrucial.

Model Parallelism

Theseproblemsarequitesimilartoschedulingwithcommunicationdelays,whenthereareprecedenceconstraints.(PY’90,VLL’90,MH’95,HLV’94)Verypoorlyunderstood.

Goodschedulinghassameeffectascaching!

ZhichengYin,JinSun,MingLi,JaliyaEkanayake,HaiboLin,MarcFriedman,JoséA.Blakeley,ClemensA.Szyperski,NikhilR.Devanur.BubbleExecution:Resource-awareReliableAnalyticsatCloudScale.PVLDB11(7).

PipeDream:FastandEfficientPipelineParallelDNNTrainingAaronHarlap,DeepakNarayanan,AmarPhanishayee,VivekSeshadri,NikhilDevanur,GregGanger,PhilGibbons

Summary: Data Center Scheduling



Algorithms


Summary: Data Center Scheduling



Algorithms


Ø  Oftenhardinworstcase.What’stherightmodel?Ø  UnderstandDAGsthatariseinpractice.SayDNNs.Ø  Whatarethehigh-levelalgorithmicintuitions?

scheduling jobs with dependenciessamir/dcscheduling18/slides...robert grandl, srikanth kandula,...

Documents