ariel garcia datagrid wp6, heidelberg, 26 th september 2003 ariel garcía crossgrid testbed status...
DESCRIPTION
Ariel Garcia DataGrid WP6, Heidelberg, 26 th September 2003 PeopleTRANSCRIPT
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Ariel García
CrossGrid testbed statusCrossGrid testbed status
Forschungszentrum Karlsruhein der Helmholtz-Gemeinschaft
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
The EU CrossGrid ProjectThe EU CrossGrid Project
1 March 2002 – 30 April 2005
Main focus on interactive and parallel applications
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
PeoplePeople
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
CrossGrid WorkpackagesCrossGrid Workpackages
WP1 – CrossGrid Application Development • 1.1 - Interactive simulation and visualization of a biomedical system
• 1.2 - Flooding crisis team support
• 1.3 - Distributed data analysis in HEP
• 1.4 - Weather forecasting and air pollution modeling WP2 – Grid Application Programming Environment WP3 – New Grid Services and Tools WP4 – International Testbed Organization WP5 – Project Management
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Software Integration and Releases Software Integration and Releases
Integration of CrossGrid softwareDev TB
• Development tools•MPI verification•Benchmarking•Performance prediction•Monitoring
• Middleware:•RAS, Portal, Migrating Desktop•Parallel Interactive Scheduler•Monitoring•Data Access Optimization
•Application Prototypes
Test and validation
Validation TB
Prototypes - 1st release
Deployment
Production TB
Basic middleware EDG 1.4.3 Globus 2
CORE WP4 I Team
CrossGrid Integration Team
•WP2 + WP4 people
•WP3 + WP4 people
•WP1 + WP4 people
Basic middleware
Validation TB
First testbed Evolution: Prototype 0 Setup Support Prototype 1
Production prototypes
(repeat cycle)
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
WP2 toolsWP2 tools
• MARMOT – Tool for MPI app debugging– 1st dev release ready, C & Fortran, MPI 1.2– N+1 nodes (debugging server)
• Performance prediction / evaluation tools– Use application kernels
• GridBench – benchmarking
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
WP3 servicesWP3 services
• Application Portal• Migrating Desktop and RAS
• Resource management (MPI apps)
• Santa-G– Info from external sensors into R-GMA
(tcpdump, snort)• Optimization of data access
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Modified RB and MPI jobsModified RB and MPI jobs
• JDL:Executable=...JobType= mpi mpich-p4 mpich-g2NumCPU=16
• MPI jobs:– In cluster (mpich-p4)
• on EDG 1.4 middleware – testing stage– Across clusters (mpich-g2)
• Main CrossGrid goal – alpha stage• No private clusters
– Across clusters (PACX-MPI)• Private clusters
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Modified RB and MPI jobsModified RB and MPI jobs
• Modified RB:– matchmaking (CPU resources,MPICH env)– script generation– .rsl generation
• Mpich-p4 jobsubmission:– in test and validation stage
• Mpich-g2 jobsubmission:– in alpha stage– avoid deadlocks and failing clusters blocking the
whole job
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
X# Graphical InterfacesX# Graphical Interfaces
•Application Portal• Specific application oriented GUI
• Only basic GUI functionality (html,php)
• Usage: system without java eg. internet cafe, airport
• Thin client
•Migrating desktop• More sophisticated user friendly Java based GUI
• Whole environement for grid user
• Possibility of working with many grids at once
• Integration of many Crossgrid tools
• Local and grid application support
• Local and grid file management
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Application PortalApplication Portal
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Migrating DesktopMigrating Desktop
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Integration and ReleaseIntegration and Release
Clear integration procedure:
• Code into CVS• Tests in ad-hoc development TB• Code autobuilds• Software announced in Iteam list• Installation in DevTB• Request test to test & validation group (LIP)• Tests conducted in T&V TB• Package gets approved, announced to Iteam• Release tagged, installation in ProdTB
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Integration and releaseIntegration and release
• Integration procedure took place for– Modified RB– Santa-G
• WP4 technical work to do:– Nodes – services relationship– Security updates for EDG 1.4 (RH 6.2)
• Coding guides• Quality control group
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Integration and ReleaseIntegration and Release
Comments:
• Site profiles in CVS (*)• Common LCFG profiles (*)
– #include "../../common/ng_source/***.h"
• Developer Workstation– DW = UI + Building Environment
• LCFG server amalgamation– LCFG + LCFGng– UML LCFGng servers
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Partners in the CrossGrid TestbedPartners in the CrossGrid Testbed
17 partners in9 countries
Geant network
Based on EDG/Globus middleware
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
TestbedsTestbeds
• Production TB – EDG 1.4.12– RB, II, MyProxy, RC, VO server at LIP– Monitoring (Mapcenter and hostchecking)
• Test & Validation TB – EDG 1.4.12– LIP & Demokritos– Modified RB at LIP
• Development TB – EDG 1.4.12– Modified RB at Valencia, II at Crakow
• Development TB – EDG 2.0– LIP + FZK– RB, RGMA at LIP
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Support infrastructureSupport infrastructure
Helpdesk
http://cg1.ific.uv.es/hlpdesk/
• In collaboration with DataGrid• Question-Answer Mechanism:
"tickets"• Interaction levels:
– User– Supporter– Administrator
• Based on OneOrZero v1.4– a GPL web based helpdesk system
based on PHP, MySQL, Javascript
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Support infrastructureSupport infrastructure
Software repository
http://gridportal.fzk.de
• Customized GNU Savannah– Projects with
• Mailinglists/forums• Bugtrackers• CVS access• Webpages
• CVS repository (viewcvs)• RPM repository• EDG repository mirror
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Support infrastructureSupport infrastructure
Software repository
http://gridportal.fzk.de
• Current usage:• ca. 70 web-visits/day• 1.100.000 code-lines 51.000 doc-lines• 18 pakages autobuilding
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Support infrastructureSupport infrastructure
Autobuild
– EDG autobuild (Yannick Patois)– Mild patching to make it more customizable– RH 6.2 & 7.3
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Support infrastructureSupport infrastructure
Mapcenter
http://mapcenter.lip.pt
Site checking
http://www.lip.pt/ computing/ cg-services/site-check
• Daily checks• On demand checks
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Support infrastructureSupport infrastructure
Site checking
• Using globus-job-run• Port checking• MDS info checking• PBS configuration• Gsiftp tests• CA certs and CRLs
checks• VO config checks• Etc
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
DisseminationDissemination
• Participation in the GriDis testbed– GGF and CERN summer schools– Several demos
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
PlansPlans
• Move to EDG 2.x– Applications already under Autobuild RH 7.3– Evaluation sites running– Several months timeframe (stability ;-)
• Network QoS• Extension to new sites (cgext VO)• Security!• Next middleware steps?
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
ConclusionsConclusions
• CrossGrid testbed is operative– Regular tests and use, including direct MPI execution
• Evolution– Testing and validation for:
• Applications (WP1)• Programming environment (WP2)• New services & tools (WP3)
– Emphasis on interoperability with DataGrid• Extension of GRID across Europe • Advice to follow GT3 while need to follow
LCG…
Ariel Garcia DataGrid WP6, Heidelberg, 26th September 2003
Questions ?
Thanks for your attention!