semantic interoperability berlin, 25 march 2008 semantically enhanced resource allocator marc de...

Semantic Interoperability

Berlin, 25 March 2008

Semantically Enhanced Resource Allocator

Marc de Palol

Jorge Ejarque, Iñigo Goiri, Ferran Julià, Jordi Guitart

Jordi Torres and Rosa M. Badia.



Agenda

1. Introduction

1. BREIN

2. The BREIN Service Provider

2. The prototype

1. Architecture

2. Lifecycle

3. The idea, Semantic Scheduling

1. The Ontology

2. The Inference

3. The rules

4. Conclusions and future work



Introduction



Introduction: BREIN

BREIN (Business objective driven REliable and Intelligent grids for real

busiNess)

It has as objective bringing hte concepts developed in Grid research projects, for

example, Virtual Organizations towards a more business-centric model, by enhancing

the system with methods from artificial intelligence, intelligent systems and semantic

web.



Introduction: BREIN (II)

BREIN Service Provider



Introduction: The BREIN Service Provider

The BREIN Service Provider gets advantages from: - SLA - Semantics - Agents - Virtualization

and has as objectives:

- Maximizing profit, by making an optimal use of the resources and reducing the penalties incurred by not meeting the SLA terms.

- Given that the SP may have a level of preference of the customers, maximize the satisfaction of the costumers, and in case of conflict, penalize first those customers with less priority.



The prototype



The prototype: Architecture

We participated in WP4.2, rapid prototypes. Together with Atos Research, we

implemented a first implementation of a SP.

Client Manager: Client Manager: Manages the client’s Manages the client’s task execution by task execution by requesting the requesting the required resources required resources and running jobs.and running jobs.

Semantic Scheduler: Semantic Scheduler: Allocates resources to Allocates resources to each client’s task each client’s task taking into account:taking into account:- System status- System status- Task requirements- Task requirements- Client Priority- Client Priority

Resource Manager:Resource Manager:Creates virtual Creates virtual machines to execute machines to execute each client’s task each client’s task according to the according to the minimum resource minimum resource allocation. It is able to allocation. It is able to dynamically dynamically redistribute the redistribute the resources to the resources to the diferent virtual diferent virtual machines that are machines that are running.running.

Application Manager: Application Manager: Monitors the resource Monitors the resource usage and the SLA usage and the SLA parameters in order to parameters in order to evaluate if the SLA is evaluate if the SLA is being violated.being violated.



The prototype: Lifecycle

(1) Resource Filter.

The Client Manager queries the Semantic Metadata Repository, performing a semantic match making to get all resources that are capable of running the job.

(2) Task Scheduling.

The Client Manager updates the task description and contacts the Semantic Scheduler to get a time slot and resource to execute the job.

(3) Get Resource Metadata.

The Semantic Scheduler needs Semantic Metadata to take decisions, such as already scheduled and running jobs as well as client’s priority.

All this information is in the Semantic Metadata Repository.

(4) Scheduling Results.

The scheduler answers to the client with the decisions it has taken.

If the job can run into the system it gets queued in a Execution Queue controlled by the Scheduler.

(5) Create Virtual Machine.

When the time of execution comes, the Scheduler contacts the Resource Manager with the information of the Virtual Machine that it must create.

(6) Create Virtual Machine (II)

The Resource Manager creates the Virtual Machine.

(7) Virtual Machine Ready.

The Resource Manager notifies the Scheduler about the Virtual Machine creation.

(8) Job Execution.

The Client Manager executes the job via Globus Tookit 4 into the newly created Virtual Machine.

(9) Resource Monitoring.

The Application Manager monitors the job execution, taking into account the SLA restrictions.

(10) SLA Violation.

If the Application Manager detects a SLA violation it asks for more resources to the Resource Manager.

(11) Resource Provisioning.

If the Resource Manager has free resources, it assigns part of them to the Virtual Machine which is violating the SLA.

(12) SLA Failed.

If the Resource Manager is unable to grant more resources to the Virtual Machine and the SLA violation persists, the Client Manager is notified.



The Idea, Semantic Scheduling

11



The Idea: The Scheduler

11

The Semantic Scheduler is a proof of concept inmplementation of decision taking using reasoning, rules with complementary Jena 2 builtins and agents.

And of course, an ontology.

12



The Idea: The Ontology

We have worked in an ontology based in resource usage coordination ontologies.

There’s been some changes and extensions to make this coordination ontologies suitable for resource allocation.

At this point in time, our ontology does not aim to be complete. It’s just for a basic resource scheduling.

Major extensions:

Resource: –CPU

–Memory

–Disk

–Network

–Software.

Task:–CPU Requirement

–Memory Requirement

–Disk Requirement

–Network Requirement

–Software Requirement



The Idea: The Ontology (II)

14



The Idea: The Inference

14

Our knowledge base is composed of:

• Semantic Metadata describing each resource.

• Semantic Metadata describing already scheduled/running tasks.

• Semantic Metadata describing the clients that are executing the tasks in the system.

In the inference phase, the Scheduler gets all necessary information and infers the result.

We have created three Scheduling Rules, with the Jena 2 Rules and a set of auxiliary builtins (rule extensions written as Java application code) which are fired during the inference process.

15



The Idea: The Rules

Task 1Task 3

Task 2

Task 1Task 3

Task 2

15

We’ve implemented three basic rules so far for the prototype:

Rule1: First Come First Serve with EASY backfilling: This rule tries to find a time slot into the proposed hosts using the FCFS scheduling algorithm taking into account the hardware requirements of the task and the space left by the other running or scheduled tasks to allocate a new virtual machine.

Task 1

Task 3

Task 2

16



The Idea: The Rules (II)

Task 1Task 1

Task 2

Task 1

Task 2

16

Rule2: Task reallocation: If Rule 1 is not capable to find a time slot and a machine for executing the task, this one tries to move already scheduled tasks (not yet running) from one machine to another in order to find a time slot for the new task.

Task 1

Task 2

17



The Idea: The Rules (III)

Task 1Task 2

Task 1

Task 2

Task 2Task 1

Task 2

Task 2

17

Rule3: Less priority task(s) cancellation. If the second rule is not able to find a solution, then the rule 3 tries to cancel the scheduled tasks (not running) with less priority than the new one.

Task 1Task 2

18



18

Conclusions and future work

19



Conclusions

We have seen that adding semantics to the scheduling process adds:

• New possibilities of scheduling. Without semantics its hard to take into account the e-Business, such as Service Providers confidence on a Client. Semantics really add a vast range of possibilities.

• Interoperability. Adding semantics to the description of the elements that participate in the scheduling process gives the possibility to be used in other Job Schedulers.

20



Future Work

As stated before this was a prototype, now we’re working with the production implementation. The main changes are:

• Adoptation of the BREIN ontologies. This will allow a fully integration with the whole system. As well as compatibility with any other system or project that uses ontologies.

• Redesign of the scheduler: From the prototype implementation to the production implementation.

• Dynamic policies: Easy to add/remove and modify.

• More business factors: Add more business factors (related to the SLA’s)

• Extension of Rules

21



Thanks for your attention.

[email protected]

22



References

B.L. Smith, C. van Aart, M. Wooldridge, S. Paurobally, t.Moyaux, and V. Tamma. An Ontological Framework for Dynamic Coordination. In Proc. Of the Fourth International Semantic Web Conference, 2005

semantic interoperability berlin, 25 march 2008 semantically enhanced resource allocator marc de...

Documents