egee production grid• large hadron collider (lhc): – four experiments / vos: alice atlas cms...

INFSO-RI-508833

Enabling Grids for E-sciencE

www.eu-egee.org

EGEE Production Grid

Dr. Ognjen PrnjatSouth East Europe Regional Operations Centre ManagerGreek Research and Technology Network, Athens

GGF 15 Production Grids Workshop, Boston, October 2nd 2005

2


INFSO-RI-508833

EGEE Grid and SEE Grid

• EGEE is the largest production Grid in Europe and perhaps the world

• South East Europe Grid is a subset of the EGEE Grid, being one of the 11 regional Grids in the project

• SEE and EGEE have full infrastructure and management compatibility, sharing the user base

• Focus of the talk: EGEE production Grid practices –operations/management

• Different/wider context from Campus Grids, but same principles of sharing and management apply!

3


INFSO-RI-508833

Enabling Grids for EsciencE

• Objectives– Provide consistent, robust and secure grid

infrastructure– Improve and maintain the middleware– Attract new resources and users from

industry as well as science– Cater for a wide range of disciplines

• Structure– 71 leading institutions in 27 countries,

federated in regional Grids– Leveraging national and regional grid

activities worldwide– Funded by the EU with ~32 M Euros for

first 2 years starting 1st April 2004

4


INFSO-RI-508833

eInfrastructure: Network and Grids

5


INFSO-RI-508833

EGEE Grid Infrastructure

Country providing resourcesCountry anticipating joining

In LCG-2:180 sites, 35 countries>14,000 cpu~5 PB storage

Includes non-EGEE sites:• 9 countries• 18 sites

6


INFSO-RI-508833

Grid m/w “Ecosystem”

. . .

LCG

2004

2001

EGEE

Used in

USA EU

NextGrid DEISAGridCC

Future grids

EDG

Globus MyProxyCondor ...

VDTDataTAG

CrossGrid ...

OSG, …

SRM

…

7


INFSO-RI-508833

EGEE Middleware

• To replace present LCG2 m/w with production quality services • Developed from existing components• Aims to address present shortcomings and needs from applications• Prototyping short development cycles for fast user feedback• Initial web-services based prototypes being tested• LCG: Redhat Enteprise 3 / binary compatible – SL, SLC, ports to

Debian

Globus 2 based Web services based

gLite-2gLite-1LCG-2LCG-1

8


INFSO-RI-508833

EGEE Middleware gLite

• First release of gLite end of March 2005 – Focus on providing users early access to prototype– Release 1.1 in May 05– Release 1.2 in July 05– Release 1.3 in August 05– see www.gLite.org

• Interoperability & co-existence with deployed infrastructure

• Robust: performance & fault tolerance

• Service oriented approach

• Open source license

9


INFSO-RI-508833

Architecture & Design

• Design team includes – Representatives from middleware providers (AliEn, Condor,

EDG, Globus…) – Colleagues from the Operations activity – Partners from related projects (e.g. OSG)

• gLite development takes into account input and experiences from applications, operations, related projects– Collaborative, open development frameworks

10


INFSO-RI-508833

Grid Operations• The grid is flat, but• Hierarchy of responsibility

– Essential to scale the operation• RC provides resources• ROCs responsible for organising

the operations in a region– Coordinate m/w deployment,

troubleshooting• CICs act as a single Operations

Centre– Operational oversight (grid

operator) responsibility– Rotates weekly between CICs– Report problems to ROC/RC

• OMC provides central coordination

CIC

CICCIC

CICCIC

CICCIC

CICCIC

CICCIC

RCRC

RCRC RCRC

RCRCRCRC

ROCROC

RCRC

RCRC

RCRCRCRC

RCRCRCRC

ROCROC

RCRC

RCRC RCRC

RCRC

RCRC

ROCROC

RCRC

RCRC

RCRC

RCRC

ROCROC

OMCOMC

RC = Resource CentreROC = Regional Operations CentreCIC = Core Infrastructure Centre OMC = Operations Management Centre

11


INFSO-RI-508833

Operations Structure• Operations Management Centre

(OMC):– At CERN – coordination etc

• Core Infrastructure Centres (CIC)– Manage daily grid operations -

monitoring, troubleshooting– Run essential infrastructure services– Provide 2nd level support to ROCs– UK/I, Fr, It, CERN, + Russia (M12)– Taipei also run a CIC

• Regional Operations Centres (ROC)– Act as front-line support for user and

operations issues– Coordinate m/w deployment– Provide local knowledge– One in each region – many distributed

• User Support Centre (GGUS)– In FZK – provide single point of

contact (service desk)

12


INFSO-RI-508833

Middleware Upgrades

• CERN integration and deployment team releases new version of LCG m/w approx every 3 months

• Deployment testing on the distributed 3-site testbed (Italy, UK, South-east Europe)

• ROCs responsible for coordination of m/w deployment

• In parallel, a pre-production services with next-generation m/w– Crucial for admins and users to get the hands-on with new m/w

13


INFSO-RI-508833

Grid Monitoring

• GIIS Monitor + Monitor Graphs• Sites Functional Tests• Live Job Monitor• GridIce – VO + Fabric View• Certificate Lifetime Monitor• work towards common

transport system and data schema

• GOC Data Base– Centralized site info: static

• CIC dashboard– Integrated info from MON tools

• Operation of Production Service: real-time display of grid operations

• Accounting Information• Selection of Monitoring tools

14


INFSO-RI-508833

Helpdesk and Troubleshooting: TPM

15


INFSO-RI-508833

Helpdesk and Troubleshooting: VO

16


INFSO-RI-508833

Tickets Processed

GGUS- Tickets

0

20

40

60

80

100

120

140

160

07/07

/2005

14/07

/2005

21/07

/2005

28/07

/2005

04/08

/2005

11/08

/2005

18/08

/2005

25/08

/2005

01/09

/2005

08/09

/2005

15/09

/2005

22/09

/2005

29/09

/2005

Week ending

Num

ber o

f tic

kets

17


INFSO-RI-508833

EGEE Authentication Scheme-EUGridPMA

• Policy Management Authority: “Club” of trusted Certification Authority managers www.eugridpma.org

• Green: CA Accredited• Yellow: being discussedOther Accredited CAs:• DoEGrids (US)• GridCanada• ASCCG (Taiwan)• CERN• Russia (HEP)• FNAL Service CA (US)• Israel• Pakistan

18


INFSO-RI-508833

Accounting

• Each VO should bring in the resources it is envisaging to use

• Each site supports VOs with certain % dictated by its funding sources / NGI

• Each site has appel accounting clients reporting usage per VO to the appel central accounting server, over R-GMA

• Appel: accounting processor for event logs

19


INFSO-RI-508833

EGEE Applications

Pilot New

20


INFSO-RI-508833

EGEE Pilot Applications

• High-Energy Physics (HEP)– Provides computing infrastructure (LCG)– Challenging:

thousands of processors world-widegenerating petabytes of data‘chaotic’ use of grid with individual user analysis (thousands of users interactively operating within experiment VOs)

• Biomedical Applications– Similar computing and

data storage requirements

21


INFSO-RI-508833

LHC Experiments

• Large Hadron Collider (LHC):– four experiments / VOs:

ALICE ATLAS CMS LHCb

– 27 km tunnel– Start-up in 2007

Mont Blanc(4810 m)

Downtown Geneva

22


INFSO-RI-508833

Concorde(15 Km)

Balloon(30 Km)

CD stack with1 year LHC data!(~ 20 Km)

Mt. Blanc(4.8 Km)

LHC Data

• 40 million collisions per second• After filtering, 100 collisions of interest

per second• A Megabyte of data for each collision

= recording rate of 0.1 Gigabytes/sec• 1010 collisions recorded each year • ~ 10 Petabytes/year of data • LHC data correspond to about

20 million CDs each year!

23


INFSO-RI-508833

BioMed Overview

• Infrastructure– ~2.000 CPUs– ~21 TB of disk– in 12 countries

• >50 users in 7 countries working with 12 applications

• 18 research labs

• ~80.000 jobs launched since 04/2004

• ~10 CPU years 0

5,000

10,000

15,000

20,000

25,000

Number of jobs

2004-09 2004-10 2004-11 2004-12 2005-01 2005-02 2005-03

Month

BIOMED

Month

Num

ber o

f job

s

PADOVA

BARI

15 resource centres

17 CEs16 SEs

24


INFSO-RI-508833

Bioinformatics• GPS@: Grid Protein Sequence Analysis

– Gridified version of NPSA web portalOffering proteins databases and sequence analysis algorithms to the bioinformaticians (3000 hits per day)Need for large databases and big number of short jobs

– Objective: increased computing power– Status: 9 bioinformatic softwares gridified– Grid added value: open to a wider community with larger

bioinformatic computations

• xmipp_MLrefine– 3D structure analysis of macromolecules

From (very noisy) electron microscopy imagesMaximum likelihood approach to find the optimal model

– Objective: study molecule interaction and chem. properties– Status: algorithm being optimised and ported to 3D– Grid added value: parallel computation on different resources of

independent jobs

25


INFSO-RI-508833

Drug Discovery

• Demonstrate the relevance and the impact of the grid approach toaddress Drug Discovery for neglected diseases

• Predict how small molecules, such as substrates or drug candidates, bind to a receptor of known 3D structure

Compounds database ~millions

Targets family ~10

Software methods ~10

Grid infrastructure

Parameter / scoring settings

UI

26


INFSO-RI-508833

Drug Discovery Data Challenge• 4 July – 26 August 2005, incl. testing

A. 2 weeks using commercial docking software B. 3 weeks using free (but slower) docking software

• Phase A: – 90 packets launched (~ 12900 jobs; 5 to >25 hours each)– ~ 20 CPU years (800 to >1000 CPUs concurrently used)– 5800 correct results collected– 500 GB of data produced

• Phase B:– 60 packets launched (~30000 jobs; 10 to >25 hours each)– ~ 40 CPU years– 1 TB will be produced

• Final data production: 1,5 TB

27


INFSO-RI-508833

Medical Imaging• GATE

– Radiotherapy planning– Grid Added Value: parallelisation reduces computing time

• CDSS– Clinical Decision Support System– Grid Added Value: ubiquitous, managed access to distributed

databases and engines• SiMRI3D

– 3D Magnetic Resonance Image Simulator– Grid Added Value: enables simulation of high-res images

• gPTM3D– Interactive tool to segment and analyse medical images

Need for very fast scheduling of interactive tasks– Grid Added Value: permanent availability of resources

28


INFSO-RI-508833

Generic Applications• EGEE Generic Applications Advisory Panel (EGAAP)• UNIQUE entry point for “external” applications

• Reviews proposals and make recommendations to EGEE management– Deals with “scientific” aspects, not with technical details– Generic Applications group in charge of introducing selected

applications to the EGEE infrastructure

• 6 applications selected so far:– Earth sciences (earth observation, geophysics, hydrology, seismology)– MAGIC (astrophysics)– Computational Chemistry– PLANCK (astrophysics and cosmology)– Drug Discovery– E-GRID (e-finance and e-business)– GRACE (grid search engine, ended Feb 2005)

29


INFSO-RI-508833

Earth Sciences Applications• Earth Observations by Satellite

– Ozone profiles

• Solid Earth Physics – Fast Determination of mechanisms

of important earthquakes

• Hydrology – Management of water resources

in Mediterranean area (SWIMED)

• Geology– Geocluster: R&D initiative of the

Compagnie Générale de Géophysique

A large variety of applications ported on EGEE which incites newusers

30


INFSO-RI-508833

MAGIC

• Ground based Air CerenkovTelescope 17 m diameter

• Physics Goals: – Origin of VHE Gamma rays– Active Galactic Nuclei– Supernova Remnants– Unidentified EGRET sources– Gamma Ray Burst

• MAGIC II will come 2007• Grid added value

– Enable “(e-)scientific” collaboration between partners – Enable the cooperation between different experiments– Enable the participation on Virtual Observatories

31


INFSO-RI-508833

Computational Chemistry

• The Grid Enabled Molecular Simulator (GEMS)– Motivation:

Modern computer simulations of biomolecular systems produce an abundance of data, which could be reused several times by different researchers.

data must be catalogued and searchable– GEMS database and toolkit:

autonomous storage resourcesmetadata specificationautomatic storage allocation and replication policiesinterface for distributed computation

32


INFSO-RI-508833

Planck

• On the Grid:> 12 time faster (but ~5% failures)

• Complex data structure

data handling important

• The Grid as– collaboration

tool– common

user-interface – flexible environment– new approach to data and S/W sharing

33


INFSO-RI-508833

EGEE Infrastructure Usage• VOs and users on the production service

– Active VOs:HEP: 4 LHC, D0, CDF, Zeus, BabarBiomedESR (Earth Sciences) Computational chemistryMagic (Astronomy) EGEODE (Geo-Physics)

– Registered users in these VO: 1000+ Many local VOs, supported by their ROCs

• Scale of work performed:– LHC Data challenges 2004:

>1 M SI2K years of CPU time (~1000 CPU years)400 TB of data generated, moved and stored1 VO achieved ~4000 simultaneous jobs (~4 times CERN grid capacity)

Number of jobs processed per month

(April 2004-April 2005)

34


INFSO-RI-508833

Current UsersName Discipline UsersAtlas Physics 400Alice Physics 50

LHCb Physics 58CMS Physics 350ESR Earth Sciences 33

Biomed Biomed 67Comp Chem Chemistry 5

Magic Astronomy 10dteam Infrastructure testing 424

EGEODE Geo-Physics 3Planck Satellites 0

Total 1400

35


INFSO-RI-508833

Job Duration• Average job duration January 2005 – June 2005

for the main VOs

36


INFSO-RI-508833

User Information & Support

• More than 140 training events across many countries– >2000 people trained

induction; application developer; advanced; retreats– Material archive online with >200 presentations

• Public and technical websites constantly evolving to expand information available and keep it up to date

• 3 conferences organized~ 300 @ Cork ~ 400 @ Den Haag ~ 450 @ Athens

• Pisa: 4th project conference 24-28 October ’05

37


INFSO-RI-508833

Green-field Regions and EGEE

• A number of regionalinfrastructure support projects

• SEE-GRID the flagship regional project, stimulating regional NGI establishment and bring the region up to speed with GRID developments

• “Start with a clean sheet”approach was important to objectively assess the approach to moving into production

38


INFSO-RI-508833

SEE-GRID vs EGEE

• Strategy: as sites mature EGEE production, adopt operational procedures

EGEE-SEE Grid SEE-GRID Grid

EGEE-wide VOsEGEE-SEE core services

SEE-GRID VOsSEE-GRID core services

39


INFSO-RI-508833

Conclusions

• Collaboration across national and international programmes is very important:– Grids are above all about collaboration at a large scale– Science is international and therefore requires an international

computing infrastructure

• EGEE I and II are always open to further collaboration

40


INFSO-RI-508833

Contacts

• EGEE Websitehttp://www.eu-egee.org

• How to joinhttp://public.eu-egee.org/join/

egee production grid• large hadron collider (lhc): – four experiments / vos: alice atlas cms...

Documents