an ogsa-based accounting system for allocation enforcement across hpc centers
DESCRIPTION
An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers. TS10 – Service Applications Thomas Sandholm [email protected] Olle Mulmo Peter Gardfjäll Erik Elmroth Lennart Johnsson. ?. Key Question. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/1.jpg)
1
CENTER FORPARALLEL
COMPUTERS
DEPARTMENT OF COMPUTER SCIENCE
DEPARTMENT OFCOMPUTING SCIENCE
2nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers
TS10 – Service Applications
Thomas Sandholm [email protected] MulmoPeter GardfjällErik ElmrothLennart Johnsson
![Page 2: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/2.jpg)
22nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Key Question
?How do we share national Grid compute resources in a fair, secure, open, and
scalable way
![Page 3: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/3.jpg)
32nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Outline
• Requirements on Software Qualities• Open Grid Services Architecture• SweGrid National Grid Testbed• SweGrid Accounting System• Results• Lessons Learned• Q & A
![Page 4: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/4.jpg)
42nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Fairness vs. Utilization
• Trade-off: Fair resource distribution and optimal resource utilization
• Soft real-time quota enforcement• User preferences• Resource policies• Allocation authority policies
Fair Distribution
Maximum Utilization
SecureOperation
Scalable Efficiency
![Page 5: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/5.jpg)
52nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Security vs. Scalability
• Integrity & Privacy• Single Sign-on/Impersonation• DoS/Replay Attack prevention• Privilege Delegation• Message Level vs. Transport Level• Policy Driven Authorization: PDP, PAP, PIP, PEPScale:• National Grid • No single point of failure but
coordinated allocationenforcement
Fair Distribution
Maximum Utilization
SecureOperation
Scalable Efficiency
![Page 6: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/6.jpg)
62nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Openness & Interoperability
• Systems Integration Platform• Scheduler/Workload Manager Agnostic• Programming Language/Model Agnostic• Portable (100% pure Java)• XML Based Standards: XPath, XQuery, XSLT, GGF-
UR, XML-Signature, XML-Encryption, XACML• Web/Grid Services Standards: SOAP, WSDL, WS-
Security, OGSA, GGF-UR, GSI, GSSAPI, OGSI/WSRF
![Page 7: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/7.jpg)
72nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Outline
• Requirements on Software Qualities• Open Grid Services Architecture• SweGrid National Grid Testbed• SweGrid Accounting System• Results• Lessons Learned• Q & A
![Page 8: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/8.jpg)
82nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Open Grid Services Architecture• Global Grid Forum Standardization Initiative• Architecture extending SOA and WSA to dynamically
share stateful resources across organizational boundaries (=realizing the Grid vision)
• “… defining, within a service-oriented architecture, a set of core capabilities and behaviors that address key concerns in Grid systems.” OGSAv1
• Assumes state modeling according to OGSI/WSRF Core Infrastructure offering Inspection,
Discovery, Lifetime Management, Notifications, Fault Handling
• WS-Resource = stateful resource and associated Web service. Provide context for message exchange
• Addresses Grid security requirements such as Delegation and Single Sign-On
![Page 9: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/9.jpg)
92nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Role of Accounting in OGSA
Accounting foundational service to: • Job Execution
Make sure that only jobs with sufficient quota can be executed on the compute resource
Decide queue priority based on available funds and usage history
• QoS/SLA Management Negotiate pricing based on resource usage Optimizing Utilization SLA Attainment/Policing
• Security Auditing Access Control PEP/PIP
![Page 10: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/10.jpg)
102nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Outline
• Requirements on Software Qualities • Open Grid Services Architecture • SweGrid National Grid Testbed• SweGrid Accounting System• Results• Lessons Learned• Q & A
![Page 11: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/11.jpg)
112nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
SweGrid
• SweGrid connects 600 compute nodes (Intel P4) across 6 Swedish HPC centers interconnected by 10Gbs GigaSunet network
• 400 HPC users at all centers (some overlapping)
• Inaugurated March 2004• ~50 currently active researchers • Up to 10k jobs per month per site
![Page 12: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/12.jpg)
122nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
SweGrid Continued
• Resource quotas allocated by Swedish National Allocations Committee (SNAC) after peer-review of promising research projects with high computational demands (c.f. NRAC)
• Initially homogeneous hardware but heterogeneous scheduling, security, and accounting environment (policies, tools, data, processes, etc)
• Wanted: Uniform resource quota use & allocation
![Page 13: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/13.jpg)
132nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Outline
• Requirements on Software Qualities• Open Grid Services Architecture• SweGrid National Grid Testbed• SweGrid Accounting System• Results• Lessons Learned• Q & A
![Page 14: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/14.jpg)
142nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
SweGrid Accounting System (SGAS) Key Design Points
1. Decentralized accounting solution based on standard, open protocols in compliance with the proposed OGSA
2. 3-party (user, resource, allocation authority) policy customization
3. Non-intrusive to local site accounting systems4. All components governed by a scalable cross-
organizational authorization framework
![Page 15: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/15.jpg)
152nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
SGAS Component Overview
PAT
Bank LUTS
Resource
Scheduler
Resource Manager
BrokerUser
WSDL WSDL
WSDL WSDL
JARM
Policy Administration ToolLogging and Usage Tracking ServiceJob Account Reservation Manager
SubmitJob
Reserve/Release PublishUR
QueryAddUser
![Page 16: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/16.jpg)
162nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
SGAS Security Design
PAT
Bank
Resource
Scheduler
Resource Manager
BrokerUser
JARMPEP
PDP
PAP
Site Policy Manager
PIP
PDP
LUTSPIP
PAP
External Authorization Service
PDP
Membership/CommunityService
PIP
Policy Administration PointPolicy Decision PointPolicy Information PointPolicy Enforcement Point
Credential Delegation
WS-SecureConversation
XML-SignatureXML-Encryption
PKI
Kerberos
![Page 17: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/17.jpg)
172nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Outline
• Requirements on Software Qualities• Open Grid Services Architecture• SweGrid National Grid Testbed• SweGrid Accounting System• Results• Lessons Learned• Q & A
![Page 18: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/18.jpg)
182nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Overdraft XACML Policy
1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97
175%
150%
125%0
20000
40000
60000
80000
100000
120000
140000
Execution Time (ms)
Jobs
Overdraft Limit
<Condition FunctionId= "urn:oasis:names:tc:xacml:1.0:function:integer-less-than-or-equal"> <Apply FunctionId= "urn:oasis:names:tc:xacml:1.0:function:integer-one-and-only"> <EnvironmentAttributeDesignator AttributeId= "sgas:overdraw:percent:requested" DataType= "http://www.w3.org/2001/XMLSchema#integer"/> </Apply> <AttributeValue DataType= "http://www.w3.org/2001/XMLSchema#integer"> 175 </AttributeValue></Condition>
![Page 19: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/19.jpg)
192nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Overdraft Fuzzy Logic Policy
1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97
Fair
Fuzzy
Unfair0
20000
40000
60000
80000
100000
120000
140000
Execution Time (ms)
Jobs
Submission Flow
R1: overdraft is low allocation left is much allow reservationR2: overdraft is high allocation left is little disallow reservationR3: allocation proximity is soon overdraft is high allocation left is much allow reservationR4: allocation proximity is soon overdraft is low allocation left is little allow reservation
![Page 20: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/20.jpg)
202nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Super Computing 2004 Demonstration
![Page 21: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/21.jpg)
212nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Outline
• Requirements on Software Qualities• Open Grid Services Architecture• SweGrid National Grid Testbed• SweGrid Accounting System• Results• Lessons Learned• Q & A
![Page 22: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/22.jpg)
222nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Conclusions
• Document centric communication in conjunction with semi-structured native XML databases is a very flexible combination
• Batch charging and eager prepare reservation needed for scalability
• Timestamp based allocations distributed in a staggered monthly flow result in the best trade-off between fairness and utilization
• Generic PEP/PDP/PIP/PAP model useful for encapsulating and evolving authorization code
• OGSI/WSRF state management ideal for controlling fine grained service state such as account quotas, reservations and policies in a standard way
![Page 23: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/23.jpg)
232nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Future Work
• With large-scale flexibility and configurability comes complexity and it becomes hard to optimize high-level goals and to realize detailed user QoS requirements – development of an SLA Management framework and user/resource goal driven optimizing agents (WS-Agreement, ContractNet)
• Initial focus has been on scientific community resource sharing - support economic brokering and for-profit banks
• Multi jobs may overload the bank - SAML assertions (c.f. cheques) as a multi-allocation payment and reservation method
![Page 24: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/24.jpg)
242nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Learn more…
http://www.sgas.se
http://www.swegrid.se
![Page 25: An OGSA-Based Accounting System for Allocation Enforcement across HPC Centers](https://reader036.vdocuments.us/reader036/viewer/2022062518/568148a1550346895db5b5f7/html5/thumbnails/25.jpg)
252nd International Conference on Service Oriented Computing, New York, NY, 15-18 November, 2004
Outline
• Requirements on Software Qualities• Open Grid Services Architecture• SweGrid National Grid Testbed• SweGrid Accounting System• Results• Lessons Learned• Q & A