How to bring the power of Cloud and DistributedInfrastructures to Scilab?

Adela NEACSU Eddy CARON Benjamin DEPARDON

May 14, 2014

Adela NEACSU Scilab and DIET May 14, 2014 1 / 23

1 Motivations

2 The middlewareOverviewDIET architecture

3 Scilab: Integration architectureServer sideClient side: Scilab module

4 Example

5 Conclusions and Future work

Adela NEACSU Scilab and DIET May 14, 2014 2 / 23

Motivations

Motivations

MotivationsI Increase Scilab computational powerI Use Scilab in a distributed environmentI Brings the Scilab capabilities to a middlewareI Brings the middleware capabilities to Scilab

What we learn? What we show?I How to interface a middleware with Scilab?

Former approachesI Message-passing (PVM)I Parallel libraries (ScaLAPACK)

Adela NEACSU Scilab and DIET May 14, 2014 3 / 23

The middleware Overview

DIET: a Grid and Cloud middleware

I Distributed Interactive Engineering ToolboxI Context: Development of a toolbox for deploying application services

providers with a hierarchical architecture for scalabilityI A middleware designed for HPC in heterogeneous and distributed

environments (i.e. Grid and Clouds)I One simple approach: RPC programming model for large distributed

infrastructureI DIET is compliant with the Grid-RPC standard

I GRAAL research team and SysFera company starting 2000I Web: http://graal.ens-lyon.fr/DIET

Adela NEACSU Scilab and DIET May 14, 2014 4 / 23

The middleware DIET architecture

DIET components

ClientI Application which uses DIET to solve problemsI Diversity – Web page/PSE/compiled programI Transparency – only knows the Master Agent

Master Agent

Local Agent

Server Daemon

Adela NEACSU Scilab and DIET May 14, 2014 5 / 23

The middleware DIET architecture

DIET components

Client

Master AgentI Platform entry pointI Finds the best server instanceI Returns a reference to the client

Local Agent

Server Daemon

Adela NEACSU Scilab and DIET May 14, 2014 6 / 23

The middleware DIET architecture

DIET components

Client

Master Agent

Local AgentI Transmits information and data between MA and serversI Stores information for each subtreeI Used to create a scalable hierarchy

Server Daemon

Adela NEACSU Scilab and DIET May 14, 2014 7 / 23

The middleware DIET architecture

DIET components

Client

Master Agent

Local Agent

Server DaemonI Computational server – manages a processor or a clusterI Offers services and information about itself

Adela NEACSU Scilab and DIET May 14, 2014 8 / 23

The middleware DIET architecture

DIET architecture

Adela NEACSU Scilab and DIET May 14, 2014 9 / 23

Scilab: Integration architecture

Integration

IntegrationI Where does Scilab fit in a DIET architecture?

I ClientI Server

I What APIs to use?

Adela NEACSU Scilab and DIET May 14, 2014 10 / 23

Scilab: Integration architecture Server side

Server side: Multiple Scilab instances for multiple clients

One instance per request

! Own environment for request

! Serving multiple clients at once

% Requests overlap

% Variables name conflict

% Start and terminate – time consumingfunctions

Adela NEACSU Scilab and DIET May 14, 2014 11 / 23

Scilab: Integration architecture Server side

Server side: Single Scilab instance for multiple clients

Single instance with mutex

! Faster – no need to restart foreach client

! No overlapping

! No conflicts

% Blocking calls

% Requires clean-up

Adela NEACSU Scilab and DIET May 14, 2014 12 / 23

Scilab: Integration architecture Server side

Server side

I Choice – single instance with mutex lockI Use call_scilab interface

I Start/Stop a Scilab instanceI Send commands

I Access for non-Scilab users

Adela NEACSU Scilab and DIET May 14, 2014 13 / 23

Scilab: Integration architecture Client side: Scilab module

Client side: Scilab module

Scilab moduleI ATOMS (AuTomatic mOdules

Management for Scilab)I Fast and easy integrationI Well defined structureI Load and start in two simple steps

Adela NEACSU Scilab and DIET May 14, 2014 14 / 23

Scilab: Integration architecture Client side: Scilab module

ATOMS

What is it?The packaging system of Scilab external modules

What does it offer?I Multi-platformI GuidelinesI Skeleton to get startedI Calls to external C/C++/Fortran programs

Adela NEACSU Scilab and DIET May 14, 2014 15 / 23

Scilab: Integration architecture Client side: Scilab module

Data management

Data typesI DIET_FILE – easy transfer of scriptsI DIET_CONTAINER – ALL other types of data

I double, integer, string, booleanI scalars and matricesI sparse matricesI polynomialsI real and complex, signed or unsigned numbers

Adela NEACSU Scilab and DIET May 14, 2014 16 / 23

Example

Example

Four simple steps

1. Create a script / Declare variables

2. Load the sci2diet module

3. Call DIET from within Scilab

4. Get results !

Adela NEACSU Scilab and DIET May 14, 2014 17 / 23

Example

Example - client side view

Adela NEACSU Scilab and DIET May 14, 2014 18 / 23

Example

Example - Client side view

Adela NEACSU Scilab and DIET May 14, 2014 19 / 23

Example

Example - Server side view

Adela NEACSU Scilab and DIET May 14, 2014 20 / 23

Conclusions and Future work

Conclusions

I Successful integrationI Two-parts architecture

I Run Scilab tasks on serverI Call DIET from a Scilab instance

I Easy to useI Code available here: http://graal.ens-lyon.fr/DIET

Adela NEACSU Scilab and DIET May 14, 2014 21 / 23

Conclusions and Future work

Future work

Data persistencyI Store intermediary data on DIET platformI Save time and memoryI Requires pre-knowledge about data

Improve schedulingI Plug-in scheduler for DIETI Collect information from the Scilab engine directlyI Better and faster selection of servers

Adela NEACSU Scilab and DIET May 14, 2014 22 / 23

Thank youfor your attention !