from academic to business gridsstartup/prezentacja/piontek_1.pdf · qoscosgrid, challengers,...

From Academic To Business GridsGrid Enterprise Solutions from PSNC

Tomasz Piontek

piontek@man.poznan.pl

2

Outline

� PSNC

� Technology transfer

� What grids are?

�Gridge Enterprise Solutions� Summary

3

R&D CenterPSNC was established in 1993 (staff: 180)

and is an R&D Center in:

� New Generation Networks

� POZMAN and PIONIER networks

� 6-NET, SEQUIN, ATRIUM, MUPBED projects

� HPC and Grids

� 5FP: GRIDLAB, CROSSGRID, GRIDSTART

� 6FP (20/4): HPC-Europa, InteliGrid, GridCoord,

CoreGrid, EGEE, BREIN, BeInGrid, ACGT,

QoSCoSGrid, Challengers, OMII-Europe,

Phosphorus, BalticGrid, g-Eclipse, int.eu.grid,

GN2, EXPReS, Qualipso, MUPBED, MAGIX,

� 2 CELTIC Projects

� Sun Center of Excellence and Microsoft Innovation Center

� Portals and Content Management Tools

� GridSphere Portal Framework support and development� GridSphere team (6 persons) led by Michael Russell now at

PSNC

� Digital Libraries

4

Technology Transfer to industry and business

� PSNC plays many roles in various R&D Grid-related projects

� PSNC’s ambition is to transfer Grid and other related technologies to industry and commerce

� PSNC is now working on the incubator programme for SMEs to exploit and support Grid technologies

� ROI is important to us

5

What is the Grid ?

“A computational grid is a hardware and software infrastructure that provides dependable consistent, pervasive and inexpensive access to high-end computational capabilities”

Carl Kesselman & Ian Foster, 1998

6

What is the Grid? (2)

“... The real and specific problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations. ...”

The Anatomy of the Grid. Enabling Scalable Virtual Organizations

Ian Foster, Carl Kesselman, Steven Tuecke

2002

7

What is the Grid? (3)

A Three Point Checklist

� coordinates resources that are not subject to centralized control,

� ... using standard, open, general-purpose protocols and interfaces,

� ... to deliver nontrivial qualities of service

Ian Foster, 2002

8

Gridge Aims

� Development of efficient distributed environments for high performance computing and collaborative work

� Creation of virtual organisations for common problem solving

� Integration of applications in distributed environments

9

Gridge Components

� Portals

� Mobile User Services

� Gridge Resource Management System [GRMS]

� Gridge Authorization System [GAS]

� Data Management System [DMS]

� Monitoring Service

� Migrating Desktop

� Virtual User System

� System Checkpointer

11

The GridSphere Portal Framework

� Support for the easy development and integration of new portlet applications

� Portlet API implementation fully 100% JSR 168 compliant

� Built-in support for Role Based Access Control (RBAC)

� Sophisticated portlet service model that can encapsulate reusable portlet logic into services that may be shared between many portlet

12

The GridSphere Portal Framework

� Localization support in the Portlet API implementation and GridSphere core portlets support French, English, Spanish, German, Dutch, Czech, Polish, Hungarian, Italian and even Arabic, Japanese and Chinese!

� Open-source and 100% free! :-)

� used in many UE and commercial projects

13

GridSphere core portlets

� Login, Logout,

� Locale settings

� Profile personalization

� Layout customization

� Administration portlets for creation of users, groups

GridSphere’s Grid Portlets

� Original goal of the GridSphere project was to support Grid portal development.

� However, the GridSphere portlet container itself contains no support for Grid technologies.

� Instead, GridSphere Project has developed support for Grid related functionality in a web application we call Grid Portlets.

� Grid Portlets can be used to develop application-specific Grid portal applications.

GridSphere’s Grid Portlets

� Grid Portlets offers a collection of portlet services that provide a high-level API and model of the Grid.

� Grid Portlets contains a set of portlets that provide basic Grid functionality.

� Grid Portlets is distributed with support for GT2 services (Mds2, GRAM, GridFTP, etc).

� Grid Portlets supports the ability to plugin other Grid technologies in a variety of ways.

Credential Grid Portlet

File Browser Portlet

Job Submit Portlet

Grid Portals

Access for Mobile Users

�Grid with Mobile Devices - possibilities

� Mobile Device as a Grid node

� Mobile Device as a tool for Grid access

�Mobile devices weaknesses

� Limited memory

� Limited screen resolution and colors

� Limited processor power

� Limited network bandwidth

� Limited input possibilities

� but...they are MOBILE!

Mobile - Architecture

Mobile - FeaturesUser notifications as Email, SMS and various Internet Communicators

Storing notification messages in Gridge Message Box for further retrieval with mobile device

Browsing Message Box for messages

Visualization Service for Mobiles usage to show device capability-dependent images to the user

Gateway and services management via portlet interface

Off-line work possibility for Gridge Mobile Client

Grid Resource Management System job submission and management

Wide range of supported devices (Java - J2ME MIDP 1.0+)

Mobile Visualization Service

GRMS Client

� Advanced scheduling algorithms and plugins for:

� multicriteria optimization of resource allocations,

� task re-scheduling (using user-level checkpointing and task migration),

� time-constraint scheduling,

� Dynamic resource discovery,

� Remote job submission and control,

� Powerful job description providing wide variety of means to express application and user requirements,

� Workflow job support – precedence constraints based on task status changes,

GRMS - Features

GRMS - Features� Compatibility and integration with common

technologies, including Globus Pre-WS and new GT4/WSRF services, grid monitoring system, data management, and many others,

� Modular & plugable architecture

� User notifications support,

� Enhanced support for VO-level authorization scenarios with GAS,

� Portable (all modules implemented in Java)

� Web Service interface

GRMS - Architecture

simple workflow

<grmsJob appid="example15">

<task taskid="ps" persistent="true">

<resource>

<hostname>fury.man.poznan.pl</hostname>

</resource>

<executable type="single" count="1">

<execfile name="ps">

<url>file:////bin/ps</url>

</execfile>

<arguments>

<value>-ef</value>

</arguments>

<stdout>

<url>${TASK_DIR}/ps.out</url>

</stdout>

</executable>

</task>

GRMS Job DescriptionGRMS Job DescriptionGRMS Job DescriptionGRMS Job Description

<task taskid="tar" extension="ps">

<executable type="single" count="1">

<execfile name="tar">

<url>file:////bin/tar</url>

</execfile>

<arguments>

<value>cfz</value>

<value>ps.out.tgz</value>

<value>ps.out</value>

<file name="ps.out.tgz" type="out">

<url>gsiftp://fury.man.poznan.pl/~/examples/ps-${JOB_ID}.tgz</url>

</file>

</arguments>

</executable>

<workflow>

<parent triggerState="FINISHED">ps</parent>

</workflow>

</task>

</grmsJob>

GRMS Job DescriptionGRMS Job DescriptionGRMS Job DescriptionGRMS Job Description

simple workflow (2)

GRMS Workflow

GRMS Client

GRMS Client (2)

GRMS Client (3)

Gridge Authorization Service

� Decision point for all components of a system

� Storing security policies for all components

�The GAS complex data structure can be used to model many abstract and real world objects and security policies for such objects

�GAS is considered as independent of specific technologies used at lower layers, and it should be fully useable in any environment

� Implemented as GSI-enabled Web Service

GAS - Features� Support for two authorization security models (RAD,

RBAC);

� Possibility to generate security policies in a form dynamically specified by the end user;

� Possibility of getting an authorization decision based on information obtained from the operating system

� GAS is probably one of the fastest and most stable authorization system because it is written in ANSI C , which uses not much CPU power and a little amount of memory (almost all authorization systems are written in java).

GAS - Architecture

GAS Architecture - system

Admin. portlet GTK admin client

GRMS

Admin. client

OGSA-DAI service

Globus plugins OGSA-DAI resource

GUI layer

Service layer

Resource layerGAS server

DSP

GAS plugins for Globus

plug-in

Elements developed in the scope of the GridLab project

getauthorizationdecision

plug-in

Gatekeeper

GridFTP

Job Manager

GLOBUS

Gridmap

callout

GRAM

callout

GASAuthorization

Service

GRMSother services,

applications or users

GAS - Portlet admin client

GAS - GTK admin client

Data Management System

Distributed system of services capable of delivering mechanisms for seamless management of large amount of data

Virtual file system keeping the data organized in a tree structure

metadirectories -for building hierarchy

metafiles – logical representation of data

GDMS Architecture

DMS Components

� Data Broker – access interface

� Metadata Repository – information about data

� Data Container – data storing

� Proxy Broker – access to databases

User Interface: file/directory services

User Interface: security services

Gridge sample scenario

Project

GRMS

Cancel

Submit

MBS

Notifica.

Job1

Job2

GAS server Mobiles

Rage1

Use

r la

ye

r

Portal

GRMS

GAS portlet GRMS portlet

Po

rtal la

ye

rS

erv

ice

s la

ye

rR

esou

rce

s la

ye

r

GAS plugin GAS plugin

GLOBUS

MCC portlet DMS portlet

DMS

Host BHost A

GRMS

DMS Mercury Mobile

Master Slavejob submission GRMS_JOB_ID

start task

register notification

commit task

start task

register metrics

GRIDGE Scenario

Host BHost A

GRIDGE Scenario

GRMS

DMS Mercury Mobile

Master SlaveNotification: Slave is running

subscribe metrics

register checkpoint interface

migrate Slavecheckpoint

create logical files

query performance metric

Host BHost A

GRMS

DMS Mercury Mobile

Master Slave

get checkpointfiles locations

resume Slave

Slave

publish new progress metric

register checkpoint files

deliver metric

submit new slave

start task

Slave2

GRIDGE Scenario

Host BHost A

GRMS

DMS Mercury Mobile

Master

Slave

register Job notification

Slave2

register SMS notification

unregistermetrics

upload output files

GRIDGE Scenario

Host BHost A

GRMS

DMS Mercury Mobile

Mastercancel slave2

Slave2notification:

Slave finishedcancel

Nonification: job finished

send SMS

GRIDGE Scenario

� Gridge at WWW:http://www.gridge.org

53

Summary

� Go commercial if you have a bright idea and few persons willing to take a risk

� Start with your business idea!

� Carefully plan the steps and the technologies you want to use.

� Use industry approved standards where possible.

� Be careful, technology is not the only limitation!

54

Thank you!