nw-grid campus grids workshop liverpool31 oct 2007 nw-grid campus grids workshop liverpool31 oct...
TRANSCRIPT
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Moving Beyond Campus Grids
Steven YoungOxford NGS Manager
Oxford e-Research Centre
University of Oxford
Partner Joining Consultant
National Grid Service
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Beyond Campus Grids: My brief
• Connecting Campus Grids to broader grids
• How Campus Grids can be used to bridge to broader area grids like the NGS or NW-GRID
• What are the impediments?
• What can Campus Grids do to make things easier?
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Straw Poll
• Who’s running a Campus Grid?– Condor-based, Other batch system (SGE,
PBS, LSF), Globus-based
• Level of service?– Grown organically, Well developed service
• User engagement?– Handful of heavy users, embedded into
institution’s teaching and research
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Talk Summary
• Joining the NGS– The NGS Site Joining Process
– Benefits & difficulties
• Providing your users with a uniform interface to resources– Options for local deployment
– Useful NGS services and resources
• Workshop discussion
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
NGS expansion
Two levels of membership1. Affiliates run compatible stack, integrated
support2. Partners also contribute “significant
resources”.• Run compatible middleware
– as defined by NGS Minimum Software Stack– and verified by compliance test suite
• Partners define their level of service commitment through a Service Level Definition
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Joining Quickstart
• http://www.ngs.ac.uk/guide/
• Partner’s Joining Guide
• Minimum Software Stack
• Conformance Testing
• Quickstart guide
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Joining Procedure
• Initial contact
• Registration of Partner
• Contacts: Security, Operations, Helpdesk
• Testing of Site
• Validation of Site
• Continuous Monitoring
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
NGS Minimum Software Stack
• Access, Job submission, File transfer• Globus GRAM, GridFTP, IGTF CAs, GSIssh, SRB
• Information Publishing• GLUE schema, MDS
• People Management• edg-mkgridmap, VOMS
• Compiling• C/C++/Fortran/Java
• Resource Control• Accounting, Job Scheduling
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Levels of Conformance
• Level 0: Globus, CAs, GLUE, MDS, VO Management, Accounting, Resource Scheduler
• Level 1: Level 0 + gsissh, compilers, SRB clients
• Level 2: Level 1 + Resource Broker, SRB servers/vaults/MCAT
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Conformance Testing
• Inca framework: http://inca2.ngs.ac.uk• “When the partner and the buddy are satisfied
that the partner site can pass all the tests the partner site is progressed to the validation process.”
• Site needs to pass tests for a 7 day period.• Final report, SLD are written• Approval by NGS Technical Board and GOSC
Board• Continuous monitoring: “failures must be
corrected within 2 working days.”
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Benefits
• Access to wider grid expertise
• Work put in to formalise processes for being part of NGS is not wasted
• NGS brand has value
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Difficulties
• Justifying the effort of joining– Full Economic Costing
• Split focus– NGS software stack and interfaces may not
be what your users care about
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Uniform Interface for Users
Things that could be deployed locally
• Condor-G
• Nimrod
• Gridway
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Condor-G
• http://www.cs.wisc.edu/condor/condorg/
• Condor-G
• OxGrid is based on Condor-G technology– Some additional “magic” to generate Grid
resource ClassAds.
• Glide-in mechanisms
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Nimrod-G
• http://www.csse.monash.edu.au/~davida/nimrod
• Tool for Distributed Parametric Modelling
• Actuators which can pass jobs to Globus resources as well as other local batch systems (Condor, PBS, SGE, LSF).
• Nimrod Portal
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Gridway Metascheduler
• http://www.gridway.org/
• Works on top of Globus services– Which you will have if you become an NGS
affiliate.
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Useful NGS services (1)
• NGS Resource Broker– A gLite Workload Management System and
User Interface node is available for use.
• NGS VOMS service– We can host VOMS instances for institutions,
groups, virtual organisations.
• NGS GridSAM EPR– Belfast provide GridSAM end point references
for NGS sites.
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Useful NGS services (2)
• NGS Application Repository portal– JSDL repository for user applications. Could
allow generalised job submission to more than NGS resources.\
• NGS RUS and UAS– NGS UAS User Accounting System aims to
conform to OGF UR (Usage Record) and OGF RUS (Resource Usage Service) specifications.
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Workshop session
• Discussion around how practical issues surrounding:– Joining of campus grids to NGS
• Eg. Globus interface node in front of a Condor pool
– Implementation of middleware solutions
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Questions?
NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007
Shibboleth
• JISC funded projects to work on Shibboleth Grid integration
• Shibboleth SP used as a portal using “some X509 certificate mechanism” in the background to provide access to Grid resources.