28 october 2010 challenge the future delft university of technology cost-driven scheduling of grid...
Post on 27-Mar-2015
216 Views
Preview:
TRANSCRIPT
28 October 2010
Challenge the future
DelftUniversity ofTechnology
Cost-driven Scheduling of Grid Workflows Using Partial Critical Paths
Dick EpemaDelft University of TechnologyDelft, the Netherlands
Saeid Abrishami and Mahmoud NaghibzadehFerdowsi University of MashhadMashhad, Iran
Grid 2010, Brussels, Belgium
2Partial Critical Paths
Introduction
• Utility Grids versus Community Grids
• main difference: QoS and SLAs
• Workflows: a common application type in distributed systems
• The Workflow Scheduling Problem
• community grids: many heuristics try to minimize the makespan of the workflow
• utility grids: other QoS attributes than execution time, e.g., economical cost, play a role, so it is a multi-objective problem
• We propose a new QoS-based workflow scheduling algorithm called Partial Critical Paths (PCP)
3Partial Critical Paths
The PCP Algorithm: main idea• The PCP Algorithm tries to create a schedule that
1. minimizes the total execution cost of a workflow
2. while satisfying a user-defined deadline
• The PCP Algorithm
• first schedules the (overall) critical path of the workflow such that
• its execution cost is minimized
• it completes before the user’s deadline
1. finds the partial critical path to each scheduled task on the critical path and executes the same procedure in a recursive manner
overall critical pathpartial critical path
4Partial Critical Paths
Scheduling System Model (1)
• Workflow Model• an application is modeled by a directed acyclic graph
G(T,E)• T is a set of n tasks {t1, t2, …, tn}
• E is a set of arcs between two tasks• each arc ei,j = (ti, tj) represents a precedence constraint
• dummy tasks: tentry and texit
tentry
t2
texit
t1
t3
t4
t5 t6
e1,4
5Partial Critical Paths
Scheduling System Model (2)• Utility Grid Model
• Grid Service Providers (GSPs)
• Each task can be processed by a number of services on different GSPs
• ET(ti,s) and EC(ti,s)
• estimated execution time and execution cost for processing task ti on service s
• TT(ei,j,r s) and TC(ei,j,r,s)
• estimated transfer time and transfer cost of sending the required data along ei,j from service s (processing task ti) to service
r (processing task tj)
• Grid Market Directory (GMD)
6Partial Critical Paths
Basic Definitions
• Minumum Exection Time:
• Minimum Transfer Time:
• Earliest Start Time:
• SS(ti): the selected service for processing the scheduled task ti
• AST(ti): the actual start time of ti on its selected service
),(min)( stETtMET iSs
ii
),,(min)( ,,
, rseTTeMTT jiSrSs
jiji
€
EST(tentry ) = 0
EST(ti) = maxt p ∈parents( ti )
EST(t p ) + MET(t p ) + MTT(ep,i)
used for finding thepartial critical paths
7Partial Critical Paths
The PCP Scheduling AlgorithmPROCEDURE ScheduleWorkflow(G(T,V), deadline)
• Request available services for each task in T from GMD• Query available time slots for each service from related GSPs• Add tentry and texit and their corresponding edges to G
• Compute MET(ti) for each task in G• Compute MTT(ei,j) for each edge in G• Compute EST(ti) for each task in G
1. Mark tentry and texit as scheduled 2. Set AST(tentry)=0 and AST(texit) = deadline3. Call ScheduleParents(texit)
4. If this procedure was successful make advance reservations for all tasks in G according to the schedule, otherwise return failure
8Partial Critical Paths
ScheduleParents (1)
• The Critical Parent of a node t is the unscheduled parent p of t for which EST(p)+MET(p)+MTT(ep,t) is maximal
• The Partial Critical Path of node t is:• empty if t does not have unscheduled parents • consists of the Critical Parent p of t and the Partial Critical
Path of p if t has unscheduled parents• Critical parent and partial critical path change over time
p t
126
69
281
9Partial Critical Paths
ScheduleParents (2)PROCEDURE ScheduleParents(t)
• If t has no unscheduled parents then return success
• Let CriticalPath be the partial critical path of t
• Call SchedulePath(CriticalPath)
• If this procedure is unsuccessful, return failure and a suggested start time for the failed node (try to repair)
• For all ti on CriticalPath /* from start to end */
Call ScheduleParents(ti)
1. Iterate over all non-scheduled parents of t
10Partial Critical Paths
SchedulingPath (1)
• SchedulePath tries to find the cheapest schedule for a Path without violating the actual start times of the scheduled children of the tasks on Path
• SchedulePath is based on a backtracking strategy
• A selected service for a task is admissible if the actual start times of the scheduled children of that task can be met
11Partial Critical Paths
SchedulingPath (2)
• Moves from the first task in Path to the last task
• For each task, it selects an untried available service
• If the selected service creates an admissible (partial) schedule, then it moves forward to the next task, otherwise it selects another untried service for that task
• If there is no available untried service for that task left, then it backtracks to the previous task on the path and selects another service for it
• This may lead to failure
12Partial Critical Paths
An Example (1)
S
1
2
3
E
4
5
6
7
8
9
Start: Call ScheduleParents(E)
13Partial Critical Paths
An Example (2)
S
1
2
3
E
4
5
6
7
8
9
find the Partial Critical Path for node E(this is the overall critical path of the workflow)
14Partial Critical Paths
An Example (3)
S
1
2
3
E
4
5
6
7
8
9
Call SchedulePath for path 2-6-9
Call ScheduleParents for nodes 2, 6, and 9, respectively
15Partial Critical Paths
An Example (4)
S
1
2
3
E
4
5
6
7
8
9
Node 2 has no unscheduled parents,so its partial critical path is empty
16Partial Critical Paths
An Example (5)
S
1
2
3
E
4
5
6
7
8
9
Node 6: find its partial critical path and then call SchedulePathScheduleParents is called for node 3 but it has no
unscheduled parentsIf SchedulePath cannot schedule this path, then it returns failure, which causes the path 2-6-9 to be rescheduled.
17Partial Critical Paths
An Example (6)
S
1
2
3
E
4
5
6
7
8
9
Node 9: find its partial critical path and then call SchedulePathScheduleParents is called for the nodes 5 and 8 but they have no
unscheduled parents
If SchedulePath cannot schedule this path, then it returns failure, which causes the path 2-6-9 to be rescheduled.
18Partial Critical Paths
An Example (7)
S
1
2
3
E
4
5
6
7
8
9
Now scheduling of the path 2-6-9 has been finished, and ScheduleParents is called again for node E to find its next partial critical path and to schedule that path
19Partial Critical Paths
Performance Evaluation
• Simulation Software: GridSim
• Grid Environment: DAS-3, a multicluster grid in the Netherlands
• 5 clusters (32-85 nodes)
• Average inter-cluster bandwidth: between 10 to 512 MB/s
• Processor speed: have been changed to make a 10 times difference between the fastest and the slowest cluster
• Processor price: fictitious prices have been assigned to each cluster (faster cluster has a higher price)
Experimental Setup (1): the system
20Partial Critical Paths
• Five synthetic workflow applications that are based on real scientific workflows (see next page)
• Montage
• CyberShake
• Epigenomics
• LIGO
• SIPHT
• Three sizes for each workflow: • small (about 30 tasks)
• medium (about 100 tasks)
• large (about 1000 tasks)
• Each task can be executed on every cluster
Performance EvaluationExperimental Setup (2): the workflows
21Partial Critical Paths
Montage
LIGO
SIPHT
CyberShakeEpigenomics
Performance EvaluationExperimental Setup (3): the workflows
22Partial Critical Paths
• Three scheduling algorithms to schedule each workflow:• HEFT: a well-known makespan minimization algorithm• Fastest: submits all tasks to the fastest cluster• Cheapest: submits all tasks to the cheapest (and
slowest) cluster
• The Normalized Cost and the Normalized Makespan of a workflow:
• CC : the cost of executing that workflow with Cheapest• MH : the makespan of executing that workflow with HEFT€
NC =total schedulecos t
CC HM
makespanscheduleNM
Performance EvaluationExperimental Setup (4): metrics
23Partial Critical Paths
Performance EvaluationExperimental Results (1)
Normalized Makespan (left) and Normalized Cost (right) of scheduling workflows with HEFT, Fastest and Cheapest
24Partial Critical Paths
Performance EvaluationExperimental Results (2)
Normalized Makespan (left) and Normalized Cost (right) of scheduling small workflows with the Partial Critical Paths algorithm
deadline=deadline-factor x MH
25Partial Critical Paths
Performance EvaluationExperimental Results (3)
Normalized Makespan (left) and Normalized Cost (right) of scheduling medium workflows with the Partial Critical Paths algorithm
26Partial Critical Paths
Performance EvaluationExperimental Results (4)
Normalized Makespan (left) and Normalized Cost (right) of scheduling large workflows with the Partial Critical Paths algorithm
27Partial Critical Paths
• One of the most cited algorithms in this area has been proposed by Yu et al.:
• divide the workflow into partitions
• assign each partition a sub-deadline according to the minimum execution time of each task and the overall deadline of the workflow
• try to minimize the cost of execution of each partition under the sub-deadline constraints
Performance EvaluationComparison to Other Algorithms (1)
28Partial Critical Paths
Performance EvaluationComparison to Other Algorithms (2): Cost
CyberShake Epigenomics
LIGO them
us
29Partial Critical Paths
Performance EvaluationComparison to Other Algorithms (3): Cost
Montage SIPHT
30Partial Critical Paths
Related Work• Sakellariou et al. proposed two scheduling algorithms
for minimizing the execution time under budget constraints:
• Initially schedule a workflow with minimum execution time, and then refine the schedule until its budget constraint is satisfied
• Initially assign each task to the cheapest resource, and then refine the schedule to shorten the execution time under budget constraints
31Partial Critical Paths
Conclusions
• PCP: a new algorithm for workflow scheduling in utility grids that minimizes the total execution cost while meeting a user-defined deadline
• Simulation results: • PCP has a promising performance in small and medium
workflows• PCP’s performance in large workflows is variable and depends
on the structure of the workflow
• Future work:• To extend our algorithm to support other economic grid models • Try to enhance it for the cloud computing model
32Partial Critical Paths
Information
• PDS group home page and publications database: www.pds.ewi.tudelft.nl
• KOALA web site: www.st.ewi.tudelft.nl/koala
• Grid Workloads Archive (GWA): gwa.ewi.tudelft.nl
• Failure Trace Archive (FTA): fta.inria.fr
top related