pl-grid : national grid initiative in poland structure and status
DESCRIPTION
PL-Grid : National Grid Initiative in Poland Structure and Status. BigView. Jacek Kitowski Institute of Computer Science AGH-UST ACK CYFRONET AGH in collaboration with PL-Grid Representatives. HP-CAST Madrid, May 10-13, 2009. Outline. PL-Grid Consortium - PowerPoint PPT PresentationTRANSCRIPT
Polish InfrastructurePolish Infrastructurefor Supporting Computational Sciencefor Supporting Computational Science
in the European Research Spacein the European Research Space
EUROPEAN UNION
PL-Grid:PL-Grid:National Grid Initiative in PolandNational Grid Initiative in Poland
Structure and StatusStructure and Status
Jacek KitowskiJacek KitowskiInstitute of Computer Science AGH-USTInstitute of Computer Science AGH-UST
ACK CYFRONET AGHACK CYFRONET AGH
in collaboration within collaboration withPL-Grid RepresentativesPL-Grid Representatives
HP-CAST
Madrid, May 10-13, 2009
BigViewBigView
PL-Grid Consortium Systematic approach establishing the Consortium and NGI Motivation Rationales Foundations International Collaboration
PL-Grid Project in a nutshell Status
PL-Grid Project Infrastructure Workpackages
Summary
OutlineOutline
MotivationMotivation E-Science approach to research Integration activities ongoing in the world
3
Need for e-Science ApproachNeed for e-Science Approach E-Science: collaborative research supported by advanced distributed computations
Multi-disciplinary, Multi-Site and Multi-National Building with and demanding advances in Computing/Computer Sciences
Goal: to enable better research in all disciplines System-level Science: beyond individual phenomena, components interact
and interrelate to generate, interpret
and analyse rich data resources• From experiments, observations
and simulations• Quality management, preservation
and reliable evidence to develop and explore models and simulations
• Computation and data at all scales• Trustworthy, economic, timely and relevant results
to enable dynamic distributed collaboration• Facilitating collaboration with information
and resource sharing• Security, trust, reliability, accountability, manageability and agility
M. Atkinson, e-Science (...), Grid2006 & 2-nd Int.Conf.e-Social Science 2006, National e-Science Centre UKI. Foster, System Level Science and System Level Models, Snowmass, August 1-2, 2007
Thanks to CYFRONET activity
five High Performance Computing Polish Centres analysed according to: Participation in International and National Projects and Collaboration Need by Polish Scientific Communities Computational resources to date European/Worldwide integration Activities National Network Infrastructure ready (thanks to Pionier National Project)
Rationales behind PL-Grid ConsortiumRationales behind PL-Grid Consortium
GEANT2GEANT2
~35 international projects FP5, FP6, FP7 on Grids (50% common) Grid Projects: Crossgrid, Gridstart, Unicore, EGEE I,II,III, K-WfGrid, Coregrid,
Virolab, Gredia, int.eu.grid, Eurogrid, Grip, Unigrids, Balticgrid I,II, Gridlab, Intelligrid, Ringrid, Brein, Beingrid, Qoscosgrid, Chemomentum, EUChinagrid, PRACE,…
~10 Polish projects (50% common) Pionier, Progress, SGIgrid, Clusterix, NDS, Platon, IT-SOA, ….
Close collaboration with Institute of Computer Science AGH-UST
Participation in Projects and CollaborationParticipation in Projects and Collaboration
Need by Polish Scientific Communities Need by Polish Scientific Communities
Scientific Community /Representative % Polish publications
Warsaw / ICM 29,0 %Cracow / Cyfronet 16,4 %Wrocław / WCSS 11,1 %Poznań / PCSS 10,1 %Gdańsk / TASK 6,8 %
SUM 73,4 %
Poland 100 %
Distribution of Polish publications in period 01.2004 – 04.2008 according to
Science Citation Index Expanded + Social Science Citation Index + Arts & Humanities Citation Index
Partners’ Resources to DatePartners’ Resources to Date
TOP500 – Nov.2008 Polish SitesTOP500 – Nov.2008 Polish Sites
National and International Integration InitiativesNational and International Integration Initiatives
Ongoing European and Worldwide activities and consolidation EGEE EGI_DS, EGI DEISA PRACE ......
European e-
Infrastructure
2000 Testbeds 2010 Utility ServiceRoutine Usage
National
International
SGI Grid, Progress(Clusterix, National Data Store ...)
Chemomomentum, Virolab, CoreGrid, Gredia, int.eu.grid, Baltic Grid, GridLab,
Porta Optica, RINGRid, Phosphorus, QoSCoSGrid, Intelligrid, K-WfGrid, Unicore...
Other Activities – Collaboration and DisseminationOther Activities – Collaboration and Dissemination EGI_DS – CYFRONET activity
Policy Board (deputy: M. Turala), NGI Observers -- „Polish Experts” (WP3, WP5) participation in Workshops, resulted in EGI_DS
draft deliverables, use-cases ... EGEE
From Cyfronet: chairing Resource Allocation Group in EGEE-III – trying to influence EGI• Group has a mandate to manage current process of resource allocation in EGEE and
propose better model for future• Operate CE ROC • More: https://twiki.cern.ch/twiki/bin/view/EGEE/RAG
E-IRG – CYFRONET activity Workshops Discussions on EGEE – EGI – NGI future
EU Unit F3 „Research Infrastructure” Experts in Program Committee, nominated by the Polish Ministry of Science and High Education (since 2007) from CYFRONET
ENPG seminar in Cracow (2008) Others
CGW 2007 and 2008 Cyfronet Users’ Annual Conference (2008 and 2009) Cyfronet Open Day (2007, 2008) ICCS 2008 Conference
Other ActivitiesOther Activities
Computational Chemistry (VO) in EGEE Third CPU power consumer
• Nearly 3 million of jobs executed during 2007 Lead by Cyfronet since 2006
• Recently management of Computational Chemistry Cluster of Excellence In parallel current effort include:
• Grid ports of commercial and non commercial software packages with particular focus on their parallel version
• Development of “experiment centric” grid web portal for chemists
EGEE III / CE ROC
EGEE-IIIEGEE-III / CE ROC / CE ROC• CE ROC – coordinator: ACK CYFRONET-AGH
• Consortium consists of institutes from 7 countries:Austria (2), Croatia (3), Czech (1), Hungary (5), Poland (4), Slovakia (1), Slovenia (1)
• Operate, maintain and support EGEE Grid Infrastructure in CE region
• Virtual Organizations supported in Central European region:HEP, computational chemistry, biomedicine, pharmacology, astrophysics, earth scienceand regional users (VOCE VO)
Poznan-PSNC
Cracow-CYFRONET
Warsaw-ICM
cores storage
Published by sites
PL-Grid Foundations – SummaryPL-Grid Foundations – Summary
Motivation E-Science approach to research EGI initiative ongoing in collaboration with NGIs
Creation of Polish Grid (PL-Grid) Consortium: http://plgrid.pl Consortium Agreement signed in January 2007
PL-Grid Project (2009-2012) Application in Operational Programme Innovative Economy, Activity 2.3 (in Sept. 2008) Get funded March 2, 2009 (via European Structural Funds)
Consortium made up of five largest Polish supercomputing and networking centres (founders) ACK CYFRONET AGH (Cracow) – Coordinator
Polish Infrastructure for Supporting Computational Science in the European Research Space
Response to the needs of Polish scientists and ongoing Grid activities in Poland, other European countries and all over the world
PL-Grid Consortium Founders PL-Grid Consortium Founders
• Academic Computer Center Cyfronet AGH (ACK CYFRONET AGH) Coordinator • Poznań Supercomputing and Networking Center (PCSS)• Wrocław Centre for Networking and Supercomputing (WCSS)• Academic Computer Center in Gdańsk (TASK)• Interdisciplinary Center for Math. and Computat. Modelling, • Warsaw University (ICM)
GEANT2GEANT2
Grid infrastructure (Grid services)PL-Grid
App
licat
ion
App
licat
ion
App
licat
ion
App
licat
ion
Clusters High Performance Computers Data repositories
National Computer NetworkPIONIER
DomainGrid
Advanced Service Platforms
DomainGrid
DomainGrid
DomainGrid
Assumptions Polish Grid is going to have a common base infrastructure – similar to solutions adopted in
other countries. Specialized, domain Grid systems – including services and tools focused on specific types of
applications – will be built upon this infrastructure. These domain Grid systems can be further developed and maintained in the framework of
separate projects. Such an approach should enable efficient use of available financial resources. Creation of a Grid infrastructure fully compatible and interoperable with European and World
Grids thanks to cooperation with teams involved in the development of European Grid systems (EGEE, DEISA, OMII, C-OMEGA, ESFRI).
Plans for HPC and Scalability Computing enabled.
PL-Grid InfrastructurePL-Grid Infrastructure
Users
Nationalcomputernetwork
Grid Application
Programming Interface
Virtual organizations and
security systems
Basic Grid services
Gridservices
LCG/gLite(EGEE)
UNICORE(DEISA)
OtherGrids
systems
Gridresource
s
Distributedcomputationa
l resources
Grid portals, development tools
Distributed data
repositories
The PL-Grid Project is split into several workpackages
Planned Realization of AimsPlanned Realization of Aims
PLANNING AND DEVELOPMENT
OF INFRASTRUCTURE
P2
Coordination StructureOperation Rules Dissemination
PROJECT MANAGEMENTP1
SECURITY CENTER
P6
Training
SUPPORT FOR VARIOUSDOMAIN GRIDS
P5P4 GRID SOFTWAREAND USERS
TOOLS DEVELOPMENTEGEE DEISA … .
OPERATIONS CENTERP3
Main Project Indicators: • Peak Perf.: 215 Tflops • Disk Storage: 2500 TB
ChallengesChallenges Short term
To start – establishing PL-Grid VO using Partners’ local and EGEE resources To provide resources for covering operational costs To select computational/storage resources for PL-Grid infrastructure
Long term – continously To provide the necessary infrastructure (!!)
• Computer rooms, electrical power, many organizational issues Be prepared / work on approaching paradigms and integration development
• HPC – Scalable Computing• Clouds (internal-external, computing clouds, data clouds)• SOA paradigm, knowledge usage …
Thank you
Polish InfrastructurePolish Infrastructurefor Supporting Computational Sciencefor Supporting Computational Science
in the European Research Spacein the European Research Space
EUROPEAN UNION
PL-Grid:PL-Grid:National Grid Initiative in PolandNational Grid Initiative in Poland
Structure and StatusStructure and Status
Jacek KitowskiJacek KitowskiInstitute of Computer Science AGH-USTInstitute of Computer Science AGH-UST
ACK CYFRONET AGHACK CYFRONET AGH
in collaboration within collaboration withPL-Grid RepresentativesPL-Grid Representatives
HP-CAST
Madrid, May 10-13, 2009
Technical IssuesTechnical Issues
Grid infrastructure (Grid services) PL-Grid
App
licat
ion
App
licat
ion
App
licat
ion
App
licat
ion
Clusters High Performance Computers Data repositories
National Computer Network PIONIER
DomainGrid
Advanced Service Platforms
DomainGrid
DomainGrid
DomainGrid
Assumptions Polish Grid is going to have a common base infrastructure – similar to solutions adopted in
other countries. Specialized, domain Grid systems – including services and tools focused on specific types of
applications – will be built upon this infrastructure. These domain Grid systems can be further developed and maintained in the framework of
separate projects. Such an approach should enable efficient use of available financial resources. Creation of a Grid infrastructure fully compatible and interoperable with European and World
Grids thanks to cooperation with teams involved in the development of European Grid systems (EGEE, DEISA, OMII, C-OMEGA, ESFRI).
Plans for HPC and Scalability Computing enabled.
PL-Grid InfrastructurePL-Grid Infrastructure
Elements and FunctionalityElements and Functionality
PL-Grid software will comprise: user tools (portals, systems for
applications management and monitoring, result visualization and other purposes, compatible with the lower-layer software used in PL-Grid);
software libraries; virtual organization systems: certificates,
accounting, security, dynamic ; data management systems: metadata
catalogues, replica management, file transfer;
resource management systems: job management, applications, grid services and infrastructure monitoring, license management, local resource management, monitoring.
Users
Nationalcomputernetwork
Grid Application
Programming Interface
Virtual organizations andsecurity systems
Basic Grid services
Gridservices
LCG/gLite(EGEE)
UNICORE(DEISA)
OtherGrids
systems
Gridresources
Distributedcomputational
resources
Grid portals, development tools
Distributed data
repositories
Three Grid structures will be maintained: production, reseach, testing.
The PL-Grid Project is split into several workpackages
Planned Realization of AimsPlanned Realization of Aims
PLANNING AND DEVELOPMENT
OF INFRASTRUCTURE
P2
Coordination StructureOperation Rules Dissemination
PROJECT MANAGEMENTP1
SECURITY CENTER
P6
Training
SUPPORT FOR VARIOUSDOMAIN GRIDS
P5P4 GRID SOFTWAREAND USERS
TOOLS DEVELOPMENTEGEE DEISA … .
OPERATIONS CENTERP3
Main Project Indicators: • Peak Perf.: 215 Tflops • Disk Storage: 2500 TB
WP2: Planning of Infrastructure DevelopmentWP2: Planning of Infrastructure Development
Analysis of users’ requirement Analysis of worldwide activity in infrastructure development Expecting results:
• Current activity: organization of pilot PL-Grid VO using EGEE infrastructure• From analysis: clusters with x86 processors
WP3: Operations Center’s tasksWP3: Operations Center’s tasks
EGIGroup
EGI Testing
Middleware
EGI ProductionMiddleware
Coordination of Operation Management and accounting EGI and DEISA collaboration
(HPC and Scalability Computing) Users’ requirements More spectacular (using
researchand test infrastructures) Computational Cloud Data Cloud Internal and External
Clouds
WP4: Grid Software and Users Tools WP4: Grid Software and Users Tools
Workpackage on Development/Research• Analysis of users’ requirements• Software repository (different tools and components developed by Partners)• Reengineering of tools and applications (legacy codes, APIs, SOA applications)• Virtual Laboratory (component applications, workflows, applications monitoring) – from
EU FP6 Virolab Project• High-level virtual organizations using knowledge (semi-automatic creation using
contract, semantic description of resources, monitoring of SLA due to QoS), data access – FiVO from EU FP6 Gredia Project
• Tools for management, proactive monitoring and security (Bazaar, Permis)
WP5: Support and TrainingWP5: Support and Training
Users’ support Making commercial software available for the users (license activity) Training, education
WP6: SecurityWP6: Security
Users’ support • Authorization, authentication, ...• Certificates ...
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688 [email protected]
• Gaussian VO created supported by several Partners (EGEE activity)
• Accepted by Vendor
• Registration: https://voms.cyf-kr.edu.pl:8443/vo/gaussian/vomrs
GAUSSIAN in Grid
• VO Manager: Mariusz Sterzel (CYFRONET), EGEE II Comput. Chemistry coordinator [email protected]
ACK: Tomasz Szepieniec
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688 [email protected]
Never Born Protein Folding
ComputingElement
Euchina Virtual Organization (EGEE)
StorageElement
2. Transfer
application
User InterfacePortal
AABTDDSAD
1.Submit
sequence
3. Store protein
4. Visualize
PDB1.32 3.23 3.442.77 4.33 5.661.32 3.23 3.44
ACK: Irena Roterman, Jagiellonian University, Tomasz Szepieniec
ActivitiesActivities Short term
To start – establishing PL-Grid VO using Partners’ local and EGEE resources To provide resources for covering operational costs To select computational/storage resources for PL-Grid infrastructure
Long term – continously To provide the necessary infrastructure (!!)
• Computer rooms, electrical power, many organizational issues • Towards Cloud Computing (internal, external)• Towards HPCaaS and Scalable computing