the european grid initiative - a view from the austrian national grid initiative

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

www.eu-egee.org

The European Grid Initiative -A View from the Austrian National Grid Initiative

Dieter Kranzlmüller (EGI Coordinator)

GUP – Institute of Graphics and Parallel ProcessingJoh. Kepler Univ. Linz, Austria

CGW’06Cracow Grid Workshop

17 October 2006Cracow, Poland

The European Grid Initiative 2






Defining the Grid

• A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user.




Example: Biomedicine

• Parallel simulationof blood flowon the Grid

Cooperation with University Amsterdam




Example: Flooding Crisis

• Simulation of floodingon the Grid

Cooperation with Slowak Academy of Sciences




Example: Scientific Visualization

Visualization on Sony PSP




Example: WISDOM

• Grid-enabled drug discovery process for neglected diseases– In silico docking

compute probability that potential drugs dock with target protein

– To speed up and reduce cost to develop new drugs

• WISDOM (World-wide In Silico Docking On Malaria)– First biomedical data challenge – 46 million ligands docked in 6 weeks

Target proteins from malaria parasite Molecular docking applications:

Autodock and FlexX ~1 million virtual ligands selected

– 1TB of data produced – 1000 computers in 15 countries

Equivalent to 80 CPU years

• Significant results– Best hits to be re-ranked using Molecular Dynamics




Example: Avian flu

• Avian Flu H5N1– H5 and N1 = proteins on virus surface

• Biological goal of data challenge– Study in silico the impact of selected point

mutations on the efficiency of existing drugs – Find new potential drugs

• Data challenge parameters:– 5 Grid projects: Auvergrid, BioinfoGrid, EGEE,

Embrace, TWGrid– 1 docking software: autodock– 8 conformations of the target (N1)– 300 000 selected compounds >100 CPU years to dock all configurations

on all compounds

• Timescale: – First contacts established: 1 March 2006– Data Challenge kick-off: 1 April 2006– Duration: 4 weeks

N1H5

Credit: Y-T Wu

Credit: Y-T Wu




Example: Earthquakes

• Seismic software application determines epicentre, magnitude, mechanism

• Analysis of Indonesian earthquake (28 March 2005)– Seismic data within 12 hours after the earthquake– Solution found within 30 hours after earthquake occurred

10 times faster on the Grid than on local computers– Results

Not an aftershock of December 2004 earthquake Different location (different part of fault line further south) Different mechanism

Rapid analysis of earthquakes important for relief efforts

Peru, June 23, 2001Mw=8.4

Sumatra, March 28, 2005Mw=8.5




EGEE-II Overview




Defining the Grid





EGEE Infrastructure

Country participating

in EGEE

Scale (July 2006):~ 200 sites in 40 countries

> 26 000 CPUs

> 11 PB storage

> 35 000 concurrent jobs per day

> 60 Virtual Organizations




Defining the Grid





MiddlewareGlobus GT4 CondorAPST

PlatformInfrastructure

Unix Windows JVM TCP/IP MPI .Net Runtime

Environmental Sciences

Life & Pharmaceutical

Sciences

ApplicationsGeo Sciences

Building Software for the Grid

VPN SSH

Courtesy IBM

Slide Courtesy David Abramson




MiddlewareGlobus GT4 CondorAPST

PlatformInfrastructure

Unix Windows JVM TCP/IP MPI .Net Runtime

Environmental Sciences

Life & Pharmaceutical

Sciences

ApplicationsGeo Sciences

Building Software for the Grid

VPN SSH

Courtesy IBM,Lower Middleware

Upper Middleware & Tools

Bonds

Slide Courtesy David Abramson




Defining the Grid





EGEE Applications

• >20 applications– High Energy Physics– Biomedicine– Earth Sciences – Computational Chemistry– Astronomy– Geo-Physics– Financial Simulation– Fusion

• Further applications in evaluation

Applications now moving from testing to routine and daily usage




EGEE-II Overview

BUT …




EGEE and Sustainability

BUT …

• How does EGEE compare to other computing infrastructures?– Number of infrastructure users?– Number of application domains?– Number of computing nodes?– Number of years in service?

• What would happen, if EGEE is turned off?• What happens after April 2008

(End of EGEE-II)?




Sustainability: Beyond EGEE-II

• Need to prepare for permanent Grid infrastructure– Maintain Europe’s leading position in global science Grids– Ensure a reliable and adaptive support for all sciences– Independent of project funding cycles– Modelled on success of GÉANT

Infrastructure managed centrally in collaboration with national bodies (in EGEE-II: JRUs)




e-Infrastructure for Europe

• A European Vision for a Universal e-Infrastructure for Research(1)

– “An environment where research resources (H/W, S/W & content) can be readily shared and accessed wherever this is necessary to promote better and more effective research”

(1) Malcolm Read (Ed.) http://www.e-irg.org/meetings/2005-UK/A_European_vision_for_a_Universal_e-Infrastructure_for_Research.pdf

http://www.e-irg.org/meetings/2005-UK/A_European_vision_for_a_Universal_e-Infrastructure_for_Research.pdf

http://www.e-irg.org/meetings/2005-UK/A_European_vision_for_a_Universal_e-Infrastructure_for_Research.pdf




e-IRG Recommendations on Sustainable e-Infrastructures

I: Governments and the Commission should develop policies and mechanisms to encourage increased investment in a more coherent and interoperable way across Europe

II: The existing e-Infrastructure projects must be superseded by integrated sustainable services at national and European levels

III: e-Infrastructures must be application-neutral and open to all user communities and resource providers. National funding agencies should be encouraged to fund multi-disciplinary and inclusive infrastructures rather than disciplinary-specific alternatives

IV: e-Infrastructures must inter-operate and adopt international standard services and protocols in order to qualify for funding

V: The Commission should, within the seventh Framework Programme, develop a pan-European e-Infrastructure which explicitly encourages the further integration of national e-Infrastructure initiatives

e-IRG Task Force on Sustainable e-Infrastructures (SeI) http://www.e-irg.org/publ/2006-Report_e-IRG_TF-SEI.pdf




Grids in Europe

• Large European investment in developing Grid technology• Sample of National Grid projects:

– Austrian Grid Initiative– Belgium: BEgrid– DutchGrid – France: Grid’5000– Germany: D-Grid; Unicore– Greece: HellasGrid– Grid Ireland – Italy: INFNGrid; GRID.IT– NorduGrid– Portuguese Grid– Swiss Grid– UK e-Science: National Grid Service;

OMII; GridPP

• Multi-national, multi-science Grid infrastructures are a priority of the EC:– DEISA, EGEE plus several supporting projects




“…for Grids we would like to see the move towards long-term sustainable initiatives less dependent upon EU-

funded project cycles”

Viviane Reding,

Commissioner,

European Commission,

at the EGEE’06 Conference, September 25, 2006

26

Broad scale test-beds

Production quality

facilities

Sustainable

e-Infrastructures

(utility model)

Sustainable grid/data-based e-

Infrastructures (utility model)

Towards sustainable grid-empowered e-Infrastructures

Slide courtesy of Kyriakos Baxevanidis, EC

27

The first e-Infrastructure Call in FP7

Year 2009Year 2009Year 2008Year 2008Year 2007Year 2007

DRAFT

1. e-Science Grid Infrastructures

2. Scientific Digital Repositories

3. Deployment of e-Infrastructures for new Scientific Communities

4. New Research Infrastructures – Design studies

5. New Research Infrastructures – Preparatory phase

6. Support measures (studies, policy initiatives,international co-operation,…)

Publication: early 2007Closure: spring 2007

•Support conceptual design studies for new RI (or major upgrades of existing ones) of clear European dimension and interest; such studies will help to assess technical and financial feasibility of proposed new RI

•Action should also foster emergence of new organisational models to consolidate a sustainable approach to e-Infrastructures, in particular in the domain of grids and data repositories

•New service provisioning schemes to be more neutral and open to all user communities and resource providers

Slide courtesy of Kyriakos Baxevanidis, EC




European Grid Initiative: Vision

• … organizes the management and operation of a reliable production Grid infrastructure (heterogeneous resources, independently managed and configured) for the European Research Area (ERA)

• ... interoperates with different e-Infrastructures around the globe (incl. HPC, …), provides a common abstraction layer to grid resources, contributes to Grid standardisation and policy efforts

• … is composed as a federation of National Grid Initiatives (NGIs) – inclusive to those interested to join, open to those unable to join at the moment – and supports the formation of new NGIs and the improvement of current NGIs

• … supports applications from diverse communities, is driven by the user communities, establishes and maintains links to business and industry, and disseminates knowledge about the Grid through training and education.

Towards a European Infrastructure for e-Sciencehttps://edms.cern.ch/file/766455/1/EGIOverview.pdf




Structure

Federated model bringing together National Grid Initiatives (NGIs) to build a European organisation

Each NGI should be a national body• Recognised at the national level

– Important to be single point of contact– Organizations will differ; e.g. can be national coordination body

• Mobilises national funding and resources– Funding schemes will differ from country to country

• Operates the national e-Infrastructure• Supports user communities• Application independent, open to new user communities and resource

providers • Contributes and adheres to international standards and policies




EGI/NGI Responsibilities

• Responsibilities between NGI and EGI are split to be federated and complimentary

• Main role of EGI is to operate of the common multi-national, multi-disciplinary Grid infrastructure– Federation of the national infrastructures– Enables and supports international Grid-based collaboration– Provides support and adds value to NGIs– Does not repeat functions done at NGI level (no “super NGI”)– European co-ordination body– Liaises with corresponding infrastructures outside Europe




EGI Responsibilities in Detail

• The exact sharing of responsibilities between NGIs and EGI is an ongoing discussion – what follows reflects our current understanding– Further details can be found in the D-Grid use-cases document

• Operations management– Operational coordination on European level, provision of best practices, status

monitoring, training material, documentation repository etc.

• Policies– Definitions of policies between NGIs, SLAs, etc.

• Standards– Definition of standards or interoperable components that are needed to ensure

successful operation of the infrastructure

• Middleware testing and certification– Interoperability of different stacks will be required

– Provide build and test infrastructure to foster interoperability

– Provide source-forge like repository

– Initially provide a base distribution comprising certified components

• Dissemination and outreach




Evolution

European e-Infrastructure

Testbeds Utility ServiceRoutine Usage

National

Global

EGIEuropean

GridInitiative




Austrian Grid

EGIEuropean

GridInitiative




Austrian Grid Phase II (2007-2009)

Goals of Phase 2:

1. Continued expansion of expertise of Austrian Grid scientists, both middleware and applications

2. Transition from Prototype-Testbed into sustainable, service-oriented Grid architecture, with seamless integration into European Grid Initiative

3. Extension of existing application areas and exploration of new application areas

4. Increased information- and trainings efforts for research and other institutions, for which grids could be relevant

5. Implementation of focus points for information exchange and cooperation with business and industry




Austrian Grid Phase II - Structure

4 Components (distributed over Austria)• Research Centre

– Basic Grid Research: Middleware Extensions, Higher Levels of Grid Middleware

– Application Support: Grid Application Development and Enabling

• Development Centre– Contact point for Industry– User Information and Training

• Service Centre– Management of the Grid Infrastructure (Hardware) and

Middleware Distribution– Certification Authority

• E-Infrastructure Coordination Committee




Summary

• The need for a European e-Infrastructure has been widely agreed

• The current short-term project based models are reaching their limits

• A model committing the National Grid Initiatives and building a central coordination organisation is proposed

your input and feedback is actively sought

• Such a scheme will ensure a sustainable e-Infrastructure for research, help maintain Europe’s leading position, and prepare for a global interoperable Grid infrastructure

• An EGI Design Project will be proposed in FP7 to evolve the current thinking to arrive at a consensus of its structure, tasks and funding

the european grid initiative - a view from the austrian national grid initiative

Documents

grid standardisation

european infrastructure

european level

national levelimportant

national bodyrecognised

european organisationeach

european research area

formation of new ngis