federating the grid
DESCRIPTION
Federating the Grid. David Kelsey TNC2010, Vilnius 2 Jun 2010. Introduction. “Real-life use cases in a cross-federated environment” What is happening in the production Grids in this area? Outline of talk The European Grid Infrastructure (EGI) The Grid Use Case(s) - PowerPoint PPT PresentationTRANSCRIPT
Federating the Grid
David KelseyTNC2010, Vilnius
2 Jun 2010
2 Jun 10 Kelsey, TNC2010 2
Introduction“Real-life use cases in a cross-federated environment”• What is happening in the production Grids in this area?Outline of talk• The European Grid Infrastructure (EGI) • The Grid Use Case(s)• Federated Identity Management for the Grid (IGTF)• Federated Security Policies (JSPG)• Future directions• not addressed here: operations, security incident response,
support, …Disclaimers and thanks: • My personal views
– not the official views of any Grid project, IGTF etc.• Thanks to (for slides): Steven Newhouse, Bob Jones, Sergio Bertolucci and David
Groep– With modifications by me
• Thanks to all my numerous colleagues in the Grids and IGTF – credit all due to them!
The European Grid Infrastructure
2 Jun 10 3Kelsey, TNC2010
European e-Infrastructure
• European Data Grid (EDG)– Explore concepts in a testbed
• Enabling Grid for E-sciencE (EGEE)– Moving from prototype to production– Federation started in 2004 (with development since 2001)
• European Grid Infrastructure (EGI)– Routine usage of a sustainable e-infrastructure
4EGI-InSPIRE - EGEE UF5 4
EGI.eu
• A legal entity created in Feb 2010. Offices in Amsterdam.• Operate a secure integrated production grid infrastructure
that seamlessly federates resources from providers around Europe
• Coordinate the support of the research communities using the European infrastructure coordinated by EGI.eu
5Bob Jones - April 2010 5
The EGI-InSPIRE Project Integrated Sustainable Pan-European Infrastructure for Researchers in Europe
• A 4 year project with €25M EC contribution– Project cost €69M– Total Effort ~€330M– Staff ~ 170FTE
Project Partners (48) EGI.eu, 37 NGIs, 2 EIROs, 8 AP
Funded
Un-Funded
EGI-InSPIRE - EGEE UF5 66
The Grid Use Case
2 Jun 10 Kelsey, TNC2010 7
Security model• Many 100s Resource Providers (Sites)• Many 10s countries (National Grids)• Many 10,000s of Users (Global Grids)
– In 100s of VOs (each using many Grids)
• Keep AuthN and AuthZ separate• User gets an electronic ID (X.509 cert)• User registers once with the VO
– And does not register with Sites
2 Jun 10 Kelsey, TNC2010 9
Security model (2)• Single Sign-on per user session• Common AuthN and AuthZ middleware
– Mutual authentication – client and server
• Authorisation attributes per session from the VO (e.g. VOMS)– Groups, Roles and/or other attributes
• Delegation is essential• Common security policies: AUP, Site & VO
2 Jun 10 Kelsey, TNC2010 10
CERN Large Hadron Collider: An example of a Global Scientific Community
Sergio BertolucciCERN
5th EGEE User ForumUppsala, 14th April 2010
11
Sergio Bertolucci, CERN 1214th April 2010
12
Sergio Bertolucci, CERN 13
The LHC Computing Challenge
14th April 2010
Signal/Noise: 10-13 (10-9 offline) Data volume
High rate * large number of channels * 4 experiments
15 PetaBytes of new data each year
Compute power Event complexity * Nb. events *
thousands users 200 k of (today's) fastest CPUs 45 PB of disk storage
Worldwide analysis & funding Computing funding locally in major
regions & countries Efficient analysis everywhere GRID technology
13
De-FZK
US-FNAL
Ca-TRIUMF
NDGF
CERN
Barcelona/PIC Lyon/CCIN2P3
US-BNL
UK-RAL
Taipei/ASGC
14th April 2010 Sergio Bertolucci, CERN 14
Today we have 49 MoU signatories, representing 34 countries:
Australia, Austria, Belgium, Brazil, Canada, China, Czech Rep, Denmark, Estonia, Finland, France, Germany, Hungary, Italy, India, Israel, Japan, Rep. Korea, Netherlands, Norway, Pakistan, Poland, Portugal, Romania, Russia, Slovenia, Spain, Sweden, Switzerland, Taipei, Turkey, UK, Ukraine, USA.
WLCG TodayTier 0; 11 Tier 1s; 61 Tier 2 federations (121 Tier 2 sites)
Amsterdam/NIKHEF-SARA
Bologna/CNAF
14
• Running increasingly high workloads:– Jobs in excess of 650k / day;
Anticipate millions / day soon
– CPU equiv. ~100k cores• Workloads are:
– Real data processing– Simulations– Analysis – more and more
(new) users
• Data transfers at unprecedented rates
Sergio Bertolucci, CERN 15
Today WLCG is:
e.g. CMS: no. users doing analysise.g. CMS: no. users doing analysis
15
Federated Identity Management for Grids:The International Grid Trust Federation (IGTF)
2 Jun 10 16Kelsey, TNC2010
Grid Identity Management• International Grid Trust Federation (IGTF)
– Formed in Oct 2005• after 5 years of development in EU DataGrid,
CrossGrid & EUGridPMA
– 3 geographical Policy Management Authorities• EU (plus Middle East/Africa), The Americas, Asia Pacific
• Coordinates a Global PKI (X.509)– Used by many different Grids
• X.509 chosen because it was the best (only?) solution (in 2000) – we need delegation
2 Jun 10 Kelsey, TNC2010 17
Identity Management (2)• Keep Authentication and Authorisation
separate– Authentication best done by employing
institute– Authorisation attributes assigned by the
Virtual Organisation (VO)
• IGTF defines minimum requirements and best practices– Accredits CAs against– 3 different authentication profiles
2 Jun 10 18Kelsey, TNC2010
OGF28 CAOPS/IGTF – Mar 2010 - 19David Groep – [email protected]
Geographical coverage of the EUGridPMA
25 of 27 EU member states (all except LU, MT) + AM, CH, HR, IL, IR, IS, MA, ME, MK, NO, PK, RO, RS, RU,
TR,UA, SEE-GRID + CERN (int), DoEGrids(US)*
Pending or in progress SY, ZA, SN
20
TAGPMA Membership
ANSP - BrazilNRC – Canada ESnet (DOEGrids) – USA EELA – InternationalFermi National Accelerator Laboratory - USAHEBCA/USHER/Dartmouth College – USAIBDS (ANSP) - BrazilWLCG – InternationalNCSA – USANCSA CILogonNERSC – USANICS UT/ORNL– USANIH Dorian - USAOpen Science Grid – InternationalPurdue University – USA REUNA – ChileSan Diego Supercomputer Center – USA SENAMHI – PeruTACC – USATeraGrid (PSC) – USA Texas High Energy Grid – USAUniversity of Virginia – USA UFF – BrazilULA – Venezuela UNAM – MexicoUNIANDES - Colombia UNLP – Argentina
IGTF Accredited CA OperatorsCA Accreditation in progressInterested in accreditationRelying Party
APGridPMA Members (15 + 1)15 Accredited CAs15 Accredited CAs
AIST (JP)
APAC (AU)
ASGC (TW)
CNIC (CN), SDG
IGCA (IN)
IHEP (CN)
KEK (JP)
KISTI (KR)
NAREGI (JP)
NCHC (TW)
NECTEC (TH)
NGO/Netrust (SG)
PRAGMA-UCSD (US)
HKU (HK)
Mongolia - under accreditation
Coverage by RAsCoverage by RAsPhilippine, Vietnam, Malaysia, Indonesia, New Zealand & Sri Lanka (soon)
CA: 9 CountriesRA: + 6 CountriesNew: +1 Country
Relying Parties and IGTF
• Relying Party: a consumer of the certificates
• Important aspect of IGTF success• The PMAs allow for membership by
Relying Parties– Important for input of end user
requirements, e.g. naming, LoA, etc.
2 Jun 10 22Kelsey, TNC2010
APGridPMA Plenary Meeting, March 2010 - 23David Groep – [email protected]
Growth issues
A few statistics: 86 trust anchors 3 operational authentication profiles 71 distinct authorities Mid-size CA: 500 active users Large CA: 5000- 20000 users Small CA: 1-10 users Research and educational community
in a small country: ~ 1 000 000 people Number of end-users that understand PKI: << 1 %
How can we maintain both trust and scalability? But not disenfranchise small communities And with a focus on end-to-end security risks
Federated CAs- To make use of other IdM
systems
2 Jun 10 24Kelsey, TNC2010
Grid Certificates from other IdPs
• Two IGTF profiles– Short Lived Credential Service (SLCS)
• Certificate lifetime <1M seconds• Certificates linked to another
authentication system – large site or federation
– Member Integrated Credential Service (MICS)• Longer-lived certificates (<13 months)
2 Jun 10 25Kelsey, TNC2010
Grid & IGTF requirements on federations
• LoA requirements on identity proofing• Persistent and unique naming
• Used for Authorisation and traceability
• Reasonable representation of names– Given name and surname– privacy issues
• Revocation needs to be handled
2 Jun 10 26Kelsey, TNC2010
APGridPMA Plenary Meeting, March 2010 - 27David Groep – [email protected]
Federation-based SLCS-only countries
TERENA Certificate Service• A very important recent development• https://www.terena.org/activities/tcs/• Use national AAI federations
– And the already existing IdPs
• Issue certificates quickly and easily to end users – eScience Personal TCS
• Certs issued by a commercial CA• TCS also issues eScience Server certs2 Jun 10 Kelsey, TNC2010 28
APGridPMA Plenary Meeting, March 2010 - 29David Groep – [email protected]
TERENA eScience Personal eligible
Federated IGTF CAs elsewhere
• USA - CIlogon– Leverage InCommon Silver for a SLCS
certificate– http://www.cilogon.org/
• Australia - ARCS SLCS CA– National federation backed (AAF)– Shibboleth based– http://wiki.arcs.org.au/bin/view/Main/SLCS
2 Jun 10 30Kelsey, TNC2010
Federated Security Policies
2 Jun 10 31Kelsey, TNC2010
32
Policy Interoperability• The Joint (EGEE/WLCG) Security Policy Group
aimed to– prepare simple and general policies– applicable to the primary stakeholders, but– also of use to other Grid infrastructures (NGI's etc)
• common policies eases the problems of interoperability (and scaling)
• Users, VOs and Sites all accept the same policies during their (single) registration (with Grid or VO)
• Other participants then know that their actions are already bound by the policies– No need for additional negotiation, registration or
agreement
2 Jun 10 Kelsey, TNC2010
33
Security Policy
Site & VOPolicies
Certification Authorities
Traceability and Logging
SecurityIncident Response
Accounting DataPrivacy
Pilot Jobs and VO Portals
Grid & VOAUPs
JSPG Security Policies
2 Jun 10 Kelsey, TNC2010
2 Jun 10 Kelsey, TNC2010
Security Policies: from EGEE to EGI
34
2 Jun 10 Kelsey, TNC2010
EGI Security Policy Group
• Primary stakeholders: NGIs, Sites, Application communities
• Starting with the current set of JSPG policies• SPG will build on this to develop a policy
framework– And produce template policies
• And to address issues not yet fully covered– More formal responsibilities, privacy
35
NRENs and Grids
Advertise the upcoming “NRENs and Grids” workshop at EGI Technical Forum– Jointly organised by TERENA and EGI
• 15 Sep 2010 - Amsterdam• http://www.terena.org/activities/nrens-n-
grids/• Indeed the whole Tech Forum (14-17 Sep)
2 Jun 10 Kelsey, TNC2010 36
Future Directions
• Production Grids already “federated”• AuthN scalability being actively addressed
– Will be more use of AAI federations– Number of Grid-specific CAs will decrease– Privacy will become more of an issue
• Will Grids start to use other AuthN middleware?• Control of Authorisation will grow in importance
– Need to define best practice for VO attribute services– work has started in IGTF
• Policy development will continue– e.g. Liabilities, responsibilities and data privacy
2 Jun 10 Kelsey, TNC2010 38
Links• EGI http://www.egi.eu/• IGTF http://www.igtf.net/• EUGridPMA http://www.eugridpma.org/• JSPG: http://www.jspg.org• EGEE http://www.eu-egee.org/• WLCG http://lcg.web.cern.ch/LCG/
2 Jun 10 Kelsey, TNC2010 39
Questions?
2 Jun 10 Kelsey, TNC2010 40