escalonamento e migração de recursos e balanceamento de carga 27-04-20151 carlos ferrão lopes nº...
TRANSCRIPT
Escalonamento e Migração de Recursos e Balanceamento de
carga
27-04-2015 1
Carlos Ferrão Lopes nº M6935
Bruno Simões nº M6082Celina Alexandre nº M6807
Paulo Cabral nº M6765 Quoc nº M6831
Tecnologias de Cloud e Datacenter
Professor Doutor Nuno M. Garcia
Overview
1. Stagger2. Scheduling Tasks
2.1. Algorithms Hetergeneous Earliest Finish Time (HEFT)
2.2. Algorithms Path Clustering Heuristic (PCH)
2.3 Scheduling algorithms for the comparison study
27-04-2015 3
What is Scheduling?
A system that uses multi-programming processes compete for processing, as they require dividing the execution time of each.
27-04-2015 5
Scheduling problem
Problem workflow scheduling a set of dependent services: Order of precedence of services; Costs of performance of the services; Communication costs between services; Resource processing capabilities; Data transmission capabilities of network connections,
which interconnect these resources;
27-04-2015 6
Heterogeneous Earliest Finish Time Algorithm (HEFT)
Algorithm Workflow escalation Using an acyclic digraph (DAG1) for a limited
number of heterogeneous computers. Task scheduling problem is NP-hard
27-04-2015 1DAG - Directed acyclic graph
9
Heterogeneous Earliest Finish Time Algorithm (HEFT) (Cont.)
The HEFT consists of two main phases: Prioritization phase Selection phase
27-04-2015 10
Prioritization phase
Each task should be prioritized considering the length of the critical path;
The list of tasks to be performed is then sorted in descending order of the critical path length;
A topological sorting is produced tasks, preserving the precedence constraints of the DAG;
27-04-2015 11
Selection Phase
To select which processor will perform a given task, calculating the earliest time of completion of a given task. With HEFT algorithm, the search for a vacant time slot on a processor P starts from the moment that the processor P becomes vacant.
27-04-2015 12
Path Heuristic Clustering Algorithm (PCH)
Clustering uses the technique to create groups (clusters) tasks;
Groups the ways of the DAG, thus reducing communication costs between tasks;
The tasks of the same cluster are scheduled on the same resource;
27-04-2015 13
Path Heuristic Clustering Algorithm (PCH) (Cont.)
Uses some attributes calculated for each task, estimating the start times (DS = Earliest Start Time) and end (EFT - Estimated Finish Time) of tasks and process;
EST and EFT are calculated using information that is provided by the middleware and the programming model;
27-04-2015 14
Scheduling algorithms for the comparison study
Applications used: Application Montage1
Application Epigenomics2
27-04-2015
1 is used to generate personalized sky mosaics using multiple points of input images.
2 is used to mapping of epigenetic state of human cells
on a large scale genomics.16
Analysis Applications
27-04-2015 17
Representation of epigenomics application used in the simulation.
Representation of montage application used in the simulation.
Architecture
Were also specified two sets: homogeneous and heterogeneous;
Have been implemented in addition to the scheduling algorithms HEFT, CPOP and PCH, a FIFO type of scheduling;
27-04-2015 18
Experimental Results
Applications to the Montage We observed that the FIFO strategy only performs
well in homogeneous architecture. The PCH algorithm does not perform well in both
architectures. The algorithms HEFT and CPOP perform well in both
architectures with an almost negligible difference.
27-04-2015 21
Analysis Applications
Workload simulation result Montage with a set Homogeneous
Workload simulation result Montage with a set Heterogeneous
27-04-2015
Makespan – The time difference between the beginning and end
of a sequence of jobs or tasks22
Experimental Results
Applications to Epigenomics The PCH algorithm improves performance in this type
of application. The CPOP algorithm preserves the good performance
in both architectures. The HEFT algorithm performs well in heterogeneous
architecture, but in the homogeneous architecture not.
27-04-2015 23
Analysis Applications
Workload simulation result Epigenomics
with a set Homogeneous
Workload simulation result Epigenomics
with a set Heterogeneous
27-04-2015
Makespan – The time difference between the beginning and end
of a sequence of jobs or tasks24
Conclusions
A FIFO strategy is appropriate only in homogeneous architectures.
In heterogeneous architectures it takes a study of both computational resources and tasks.
The HEFT and CPOP algorithms showed good performance and at the same time like in Montage application, while SHP not.
In applications with equipment that prevents the performance in a continuous process, such as montage application, it is not convenient grouping of tasks.
27-04-2015 25
Scheduling Procedure
A multi-programming computer has multiple processes running simultaneously, however, that all processes can not access the CPU at a time and so there is a fair way of implementation of these processes, an algorithm was created to organize tasks.
27-04-2015 26
Process Scheduling in Distributed System
Heterogeneity of the nodes prevents equal distribution;
In addition to the tasks to be performed it is necessary to: Processing power; Unemployment level;
If you are in possession of information: Weight of tasks; Estimated processing time;
27-04-2015 27
Information nodes
Physical characteristics: speed, memory, processor type
Degree of idleness of each node: the processing power is being used;
27-04-2015 28
Measure the level of unemployment
Average waiting time in the run queue Problem: Priority difference of tasks;
Generate CPU utilization rate
Problem: to test specific process from time to time;
27-04-2015 29
Measure the level of unemployment (Cont.)
Problem General: update frequency Excessive Update: scheduling algorithm becomes
main consumer; Insufficient Update: wrong choice of a node;
Algorithms: Static scheduling; Scheduling dynamic Adaptive scheduling;
27-04-2015 30
Scheduling static
Assessment of conditions of the nodes is done once;
Data are used during the entire process;
27-04-2015 31
Dynamic Scheduling
Constant analysis of the nodes characteristics: Waiting time in the CPU queue: little processing; CPU Utilization: Small background processes making
continuous testing;
27-04-2015 32
Adaptive Scheduling:
Dynamic Scheduling special case; You can adjust the intensity of CPU consumption:
May fail to perform some tests
27-04-2015 33
Overhead
Distributed memory: high cost in exchange of information between nodes;
Can occur over communications and performance problems: Nodes are communicating longer than processing;
27-04-2015 34
Algorithms of classification according to the policy
You can sort scheduling algorithms under policies: Transfer policy: determining a suitable node to receive
tasks; Selection Policy: determine the most appropriate task
to be sent to a node; Location Policy: determine the most appropriate node
to receive a given task; Information Policy: determine storage location of
information and frequency of update.
27-04-2015 35
Transfer Policy
Determine a receiver node: node is working below its processing capacity;
Is defined maximum load supported by each node;
27-04-2015 36
Transfer Policy (Cont.)
Problem: donor sends a task to a receiver that immediately becomes a donor: Task switching is done indefinitely.
Is defined maximum load supported by each node; Solution: Consider a donor node whose load is less
than the limit and will continue to be less than or equal after receiving the task;
27-04-2015 37
Selection Policy
Determine a task to be sent when a node is overloaded or eligible to receive;
In an overloaded node: Select the task that caused the overload;
27-04-2015 38
Selection Policy (Cont.)
Preemptive scheduling: A task already running can be transferred to another
node; High transfer costs of the task context (ex .: data in
memory); Usually picks up recently started tasks;
Non-Preemptive Scheduling: Only tasks that have not yet implemented can be
transferred;
27-04-2015 39
Location Policy
Select node to receive a task; Random method (most commonly used):
Node randomly chooses another node; Problem: chances of choosing a node in the same
situation; Immediate solution: test if selected node is receiver; Solution "better": vote between selected nodes.
Generates excessive communication.
27-04-2015 40
Information Policy
Corresponds to the exchange of information on their status across multiple nodes: Local storage: a specific node receives all the status
messages of others. Broadcast: each node sends to others; Sub-demand: each node has information about you.
Information about others only when it becomes donor or recipient.
27-04-2015 41
Scheduling algorithms
Started by the donor: The node that has a donor in accordance with the
transfer policy utilized, looking for a receiver; Algorithm Random (most common), making the choice
of a single receptor twice. Sub-demand: each node has information about you.
Information about others only when it becomes donor or recipient.
27-04-2015 42