wp 10 information services giovanni aloisio, massimo cafaro, italo epicoco [email protected]...
TRANSCRIPT
WP 10
Information Services Giovanni Aloisio, Massimo Cafaro, Italo
Epicoco
[email protected]@unile.it
University of Lecce, Italy
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Outline
Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring & adding information providersGridLab WP 10 activitiesWork in progress
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Grid Information Services
System information is critical to operation of the grid and construction of applications
What resources are available?Resource discovery
What is the “state” of the grid?Resource selection
How to optimize resource use Application configuration and adaptation?
We need a general information infrastructure to answer these questions
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Examples of useful Information
Characteristics of a compute resourceIP address, software available, system administrator, networks connected to, OS version, load
Characteristics of a networkBandwidth and latency, protocols, logical topology
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Grid Information: facts of Life
Information is always oldchanging system stateneed to provide quality metrics
Distributed state hard to obtaincomplexity of global snapshot
Component will failScalability and overheadMany different usage scenarios
heterogeneous policy, different information organizations, different queries, etc.
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Grid Information Service
Provide access to static and dynamic information regarding system componentsA basis for configuration and adaptation in heterogeneous, dynamic environmentsRequirements and characteristics
Uniform, flexible access to informationScalable, efficient access to dynamic dataAccess to multiple information sourcesDecentralized maintenance
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
The GIS Problem: Many Information Sources, Many Views
?RR
R
RR
?
R
R
RR
R?
R
R
R
RR
?
RR
VO A
VO B
VO C
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
What is a Virtual Organization?
• Facilitates the workflow of a group of users across multiple domains who share (some of) their resources to solve particular classes of problems
• Collates and presents information about these resources in a uniform view
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Outline
Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring & adding information providersGridLab WP 10 activitiesWork in progress
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Two Classes of MDS Servers
Grid Resource Information Service (GRIS)Supplies information about a specific resourceConfigurable to support multiple information providersLDAP as inquiry protocol
Grid Index Information Service (GIIS)Supplies collection of information which was gathered from multiple GRIS serversSupports efficient queries against information which is spread across multiple GRIS serverLDAP as inquiry protocol
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS 2.2 Architecture
A A
Customized Aggregate Directories
R RR R
Standard Resource Description Services
Registration
Protocol
Users
Enquiry
Protocol
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Information Protocols
Grid Resource Registration ProtocolSupport information/resource discoveryDesigned to support machine/network failure
Grid Resource Inquiry ProtocolQuery resource description server for informationQuery aggregate server for informationLDAP V3.0 in Globus
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS 2.2 Service Architecture
Dynamic Registration via Reg. Protocol (GRRP)Resource Inquiry via Info. Protocol (GRIP)
Co-located with resource on network
Resource Discovery (via GRIP or other)Using GRIP allows resource/directory hierarchy
discovery (GRIP?)
lookup (GRIP)registration (GRRP)
VO-specific Aggregate Directories
standard Resource Description services
R R R R
A A?
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Distributed Services
Service scales with Grid growthLoose consistency model tolerates failuresInteroperability by GRIP/GRRP protocols
R
R R
R R
R
R R
fault-partitionregistrationmessages
replicated directories divergent directories
R
R
RR
RR R
RR
VO-A VO-B
RR
R
R
RR R
RR
R
R R
R R
R
RR D
D
D
D
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Metacomputing Directory Service
Uses LDAP as Inquiry Access information in a distributed directory
Directory represented by collection of LDAP serversEach server optimized for particular function
Directory can be updated by: Information providers and toolsApplications (i.e., users)Backend tools which generate info on demand
Information dynamically available to tools and applications
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
LDAP Details
Lightweight Directory Access ProtocolIETF StandardStripped down version of X.500 DAP protocolSupports distributed storage/access (referrals)Supports authentication and access control
Defines:Network protocol for accessing directory contentsInformation model defining form of information Namespace defining how information is referenced and organized
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS Components
LDAP 3.0 Protocol EngineBased on OpenLDAP with custom backendIntegrated caching
Information providersDelivers resource information to backend
APIs for accessing & updating MDS contentsC, Java, PERL (LDAP API, JNDI)
Various tools for manipulating MDS contentsCommand line tools, Shell scripts & GUIs
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Grid Resource Information Service
Server which runs on each resourceGiven the resource DNS name, you can find the GRIS server
well known port = 2135
Provides resource specific informationMuch of this information may be dynamic
Load, process information, storage information, etc.GRIS gathers this information on demand
“White pages” lookup of resource informationEx: How much memory does machine have?
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Grid Index Information Service
GIIS describes a class of serversGathers information from multiple GRIS serversEach GIIS is optimized for particular queries
Ex1: Which Alliance machines are >16 process SGIs?Ex2: Which Alliance storage servers have >100Mbps bandwidth to host X?
Akin to web search engines
Organization GIISThe Globus Toolkit ships with one GIISCaches GRIS info with long update frequency
Useful for queries across an organization that rely on relatively static information (Ex1 above)
“Yellow pages” lookupEx: Which machines have large memory?
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Server Registration
A GRIS or GIIS server can be configured to (de-) register itself during startup/shutdown
Targets specified in configuration file
Soft-state registration protocolGood behavior in case of failure
Allows for federations of information serversE.g. Argonne GRIS can register with both Alliance and DOE GIIS servers
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Soft-state Registration
Periodic notification“Service/resource is available”Expected-frequency metadata
Automatic directory constructionAdd new resources to directoryInvite resources to join new directory
Self-cleaningReduce occurrence of “dead” references
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Hierarchical MDS Deployment
GRISes
GIIS
GIIS
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS Commands
LDAP defines a set of standard commandsldapsearch, etc.
Globus Toolkit defines MDS-specific commands
grid-info-search, grid-info-host-search
APIs are defined for C, Java, etc.C: OpenLDAP client API
ldap_search_s(), …
Java: JNDI
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Information Services API
RFC 1823 defines an IETF draft standard client API for accessing LDAP databases
Connect to serverPose query which returns data structures contains sets of object classes and attributesFunctions to walk these data structures
Globus does not provide an LDAP API and recommends the use of OpenLDAP, an open source implementation of RFC 1823.LDAP APIs available in other languages
E.g. Java JDNI, Perl, Python, etc.
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS 2.2 Features
Security MechanismsGSI mutual-authenticationFine-grained access control by GSI name
Performance EnhancementsBetter query speedsLess stale information
New Information Model (schema)Better representation of computersCleaner namespace management
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS 2.2 Software Stack
OpenLDAP 2.0.xImplements LDAPv3 protocolClient and server components
Cyrus-SASLGeneric securityWe provide loadable SASL/GSS plugin
Globus GSIProvides GSS-API interface to PKIShared library used by our SASL plugin
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS 2.2 Information Model
Structural informationResource hierarchy maps to objectsNamed positions in LDAP DIT
Merged informationSome parents “join” child dataSimplifies common query patterns
Auxiliary informationUniform representation of leaf/parent dataUses LDAP auxiliary objectclasses
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GRIS Host Objects
/scratch1dev=
diskdev group=
DISK
DISK netdev group=
eth0dev=NET
NET
hn=hostname
cpu 0dev=CPU
cpu 1dev=CPU
CPUsdev group=
CPU
CPU
dev=RAM VMdev=RAM VM
RAM
VM
dev group=memory
software=OS
CPU
CPU
RAM
VM
DISK
NET
OS
OS
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GRIS Object Hierarchy
Mds-Host-name=hostnameMds-Software-Deployment=operating systemMds-Device-Group-name=processors
Mds-Device-name=cpu 0
Mds-Device-Group-name=memoryMds-Device-name=physical memoryMds-Device-name=virtual memory
Mds-Device-Group-name=filesystemsMds-Device-name=/scratch1Mds-Device-name=/scratch2
Mds-Device-Group-name=networksMds-Device-name=eth0
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GRIS Structural Class Hierarchy
MdsAttr: Mds-validfrom (like createtime)Attr: Mds-validto (accuracy metadata)Attr: Mds-keepto (discard metadata)
MdsHostMdsDeviceMdsDeviceGroupMdsSoftwareDeployment
Every MDS object: name, time metadata
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GRIS Auxiliary Class Examples
MdsCpuAttr: Mds-Cpu-vendorAttr: Mds-Cpu-modelAttr: Mds-Cpu-speedMHz
MdsCpuCacheAttr: Mds-Cpu-Cache-L1kB
MdsCpuSmpAttr: Mds-Cpu-Smp-size
MdsCpuTotalAttr: Mds-Cpu-Total-count
Once per CPU
Once per CPU
Once per SMP
Once per MPP
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GRIS Auxiliary Class Examples
MdsCpuFree (once per SMP)Attr: Mds-Cpu-Free-1minX100Attr: Mds-Cpu-Free-5minX100Attr: Mds-Cpu-Free-15minX100
MdsCpuTotalFree (once per MPP)Attr: Mds-Cpu-Total-Free-1minX100Attr: Mds-Cpu -Total-Free-5minX100Attr: Mds-Cpu -Total-Free-15minX100
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Client Tools
Globus Toolkit includes 2 command line client tools for querying MDS services
grid-info-search: General purpose clientgrid-info-search –h <host> -p <port> -b <base> \ -T <timeout> [<filter>] [<attributes>]-x: Anonymous access
grid-info-host-search: Same as grid-info-search, but defaults to GRIS standard port
E.g. grid-info-host-search –h localhost
Both clients can search for specific system information and filter results.
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Filtering
Filters allow selection of object based on relational operators
=, ~=,<=, >=grid-info-search “cputype=*”
Compound filters can be construct with Boolean operations
&, |, !grid-info-search “(&(cputype=*)(cpuload1<=1.0))”grid-info-search “(&(hn~=sdsc.edu)(latency<=10))”
Hints:white space is significant
use -L for LDIF format
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Outline
Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring & adding information providersGridLab WP 10 activitiesWork in progress
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
dc-n1.isi.edu
grid-info-site-policy.conf
grid-info-resource-register.confgrid-info-resource-ldif.conf
dc-n2.isi.edu
grid-info-resource-register.confgrid-info-resource-ldif.conf
GRIS
GIIS
GRIS
Configuration files for Registration
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Configuration files for Registration
A resource that is hosting a GIISgrid-info-site-policy.conf
Determines whether to accept incoming registrationsAccept everything, or only registrations from the resources explicitly defined in this conf file (can use wildcards)
Default policy is to only accept registrations from self, and from port 2135
In a hierarchical GIIS environment, this file must be modified from the default
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Default Policy Data:objectclass: MdsRegistrationPolicypolicydata: (&(Mds-Service-hn=mako.isi.edu)(Mds-Service-port=2135))
Change ‘policydata’ to this for completely open policy:(Mds-Service-hn=*)
Change ‘policydata’ to this to restrict to 2 specific hosts:(&(|(Mds-Service-hn=dc-n2.isi.edu)(Mds-Service-hn=dc-n3.isi.edu))(Mds-Service-port=2135))
grid-info-site-policy.conf
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Configuration files for Registration
A resource registering GRIS information with a GIIS
grid-info-resource-register.confWhich GIIS’s this GRIS should register to, and howGIIS could be on the same machine, but may not be
grid-info-resource-ldif.confDetermines which GRIS providers are active and available to send data to the GIIS’s that this GRIS is registering with
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Registering a GRIS with a GIIS
dn: <LDAP add object DN>regtype: mdsreg2reghn: <host to send reg to>regport: <port to send reg to>regperiod: <how often to send reg (seconds)>[service attribute/value]...
where service attribute entries depend on the type of LDAP object being published
$GLOBUS_LOCATION/etc/grid-info-resource-register.conf
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
ExampleGRIS on dc-n2 registers with GIIS on dc-n1
dn: Mds-Vo-Op-name=register, Mds-Vo-name=site, o=gridregtype: mdsreg2reghn: dc-n1.isi.eduregport: 2135regperiod: 600type: ldaphn: dc-n2.isi.eduport: 2135rootdn: Mds-Vo-name=local, o=gridttl: 1200timeout: 20mode: cachedumpcachettl: 30
$GLOBUS_LOCATION/etc/grid-info-resource-register.conf
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Registration Control Parameters
regperiodHow often this GRIS will send a message to the GIIS noting its existence
ttlHow long the recieving GIIS should keep the registration information before assuming that the GRIS is no longer availableIn general ttl should be: ttl = 2 x (regperiod)
cachettlRecommendation to the GIIS about how long to maintain in cache, the GRIS information provided by this resource
$GLOBUS_LOCATION/etc/grid-info-resource-register.conf
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS 2.2 GRIS Providers
grid-info-cpu reports CPU/load infogrid-info-fs reports filesystem infogrid-info-mem reports RAM/VM infogrid-info-net reports NIC/net info grid-info-os reports OS info grid-info-platform reports arch. info grid-info-merged merges all host info*Extensible for other sources, e.g. GRAM
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GRIS Dispatch Logic
For each provider:1. Could search intersect provider? No, then skip.2. Is provider cache stale? Yes, then refill.3. Apply search filter to cache data.
Combine all intersecting providers’ results
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GRIS Response Issues
MDS 2.2 is lazyProbes are not issued unless queried
Some system probes are slow“Best” probe may take several seconds
How to avoid stale data?Clients set time-out per queryGRIS/GIIS define time-out per sourceFresh data found before timeout is returnedCache fill continues after client time-out
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
New Information Providers
Decide what information to publish into MDS
Create a program that implements the I/O interface requirements of a GRIS Information Provider
Enable the information provider for an MDS installation
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
New Information Providers
Decide what information to publish into MDSOID: conflict avoidance
OID Registered with IANA Private Enterprise Numbers http://www.iana.org/cgi-bin/enterprise.pl 1.3.6.1.4.1.3536.* Globus OID subspace 1.3.6.1.4.1.3536.2.* Globus Info Services OID subspace 1.3.6.1.4.1.3536.2.6.* MDS OID subspace
GridLab project OID: 1.3.6.1.4.1.3536.2.6.3536.9Object Naming: conflict avoidance
ISI will coordinate prefix naming. Request a prefix from: mailto:[email protected]
GridLab prefix: “GridLab”Schema
Data Modeling problemLDAP schema syntax problem$GLOBUS_LOCATION/etc/grid-info-resource.schema
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
New Information Providers
Create your programAny language. The only requirements are the I/O interface of your program:
Must be callable by fork and exec from the slapd process You can pass in arguments to your program
Must return data in LDIF format Data returned must match the LDAP schema RFC 2849
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
New Information Providers
Enable your new information provider by adding a configuration block$GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf
# generate memory info every minutedn: Mds-Device-Group-name=memory, Mds-Host-hn=mako.isi.edu, Mds-Vo-name=local, o=gridobjectclass: GlobusTopobjectclass: GlobusActiveObjectobjectclass: GlobusActiveSearchtype: execpath: /globus/libexecbase: grid-info-mem-linuxargs: -devclassobj -devobjs -dn Mds-Host-hn=mako.isi.edu,Mds-Vo-name=local,o=grid -validto-secs 60 -keepto-secs 60cachetime: 60timelimit: 10sizelimit: 3
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
New Information Providers
dnWhere the object lives in the DIT
These lines must be included:objectclass: GlobusTopobjectclass: GlobusActiveObjectobjectclass: GlobusActiveSearchtype: exec
pathPath to the information provider program
baseName of the information provider program
argsArguments to be passed to the information provider program
cachetimeIn seconds, how long GRIS will consider the data to not be stale
timelimitIn seconds, how long the GRIS should wait for the information provider to return data before giving up on it
sizelimitMax number of LDIF objects to be read from the output of the information provider
$GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Outline
Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring servers & adding information providersGridLab WP 10 activitiesWork in progress
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GridLab WP 10 Tasks
T10.1 Analysis of current state of the art GIS. CompletedT10.2 Comparison of GIS capabilities and GridLab dynamic grid computing requirements. CompletedT10.3 Specification of information model. CompletedT10.4 Security mechanisms & policies.T10.5 Development of extended GIST10.6 Development of high-level APIs
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GridLab WP 10 Deliverables
D10.1 Analysis report. DeliveredD10.2 Requirements & specification. DeliveredD10.3 Prototype release. DeliveredD10.4 Extended GIS releaseD10.5 First release reportD10.6 Second release report
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GridLab WP 10 Milestones
M10.1 Month 3 Requirements analysis report. On timeM10.2 Month 6 Technical specification. On timeM10.3 Month 12 Prototype GIS server. On timeM10.4 Month 24 First release of GISM10.5 Month 36 Documented second release of GIS
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Requirements of GridLab MDS
PerformanceScalabilitySecurityUniformityExpressivenessExtensibilityMultiple sourcesDynamic dataFlexible accessDeployabilityDecentralized control
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
WP 10: Current release
MDS schema extension to provide information about:ServicesSoftwareUsersFirewallsVirtual OrganizationsCertification Authorities
Access to MDSWe developed a GSI enabled version of gSOAP supporting mutual authentication and delegation of credentialsDeveloped GSI enabled Web Services in order to:
Register informationUnregister informationLookup informationGeneric MDS search
Developed GSI enabled clients for the above functionalities
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Implementation
GT 2.x basedC based
Uses LDAP API
WEB SERVICEGSI enabled; uses gSOAP ToolkitWSDL description for clients
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
The gSOAP Tooolkit
by Robert Van Engelen, Florida State University, USAdevelopment environment for C/C++ Web serviceseasy-to-use RPC compiler: produces the stub and skeleton routines to integrate (existing) C or C++ applications into SOAP/XML Web servicesautomatically maps native C/C++ application data types to semantically equivalent XML types and vice versasimple API relieving the user from the burden of SOAP detailsSOAP 1.1/1.2 and WSDL 1.1Zlib compression, SSL encryption, and streaming direct internet message encapsulation (DIME) attachments
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
The GSI plugin for gSOAP
By Massimo Cafaro, Daniele Lezzi (University of Lecce) and Robert Van Engelen (FSU)Overrides gSOAP callbacks to provide transport-level GSI securityProvides mutual authentication and delegation of credentialsVery simple API relieving the user from the burden of GSI details
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
A GSI enabled, threaded Web Service
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Information about Services
GridLab-Mds-Service-name: service nameGridLab-Mds-Service-port: service portGridLab-Mds-Service-type: service protocolGridLab-Mds-Service-description: service descriptionGridLab-Mds-Service-publisher: service publisher
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Information about Software
GridLab-Mds-Software-name: nameGridLab-Mds-Software-version: versionGridLab-Mds-Software-path: pathnameGridLab-Mds-Software-totalLicences: number of total software licencesGridLab-Mds-Software-freeLicence: number of available software licencesGridLab-Mds-Software-licenceInfo: information about software licenceGridLab-Mds-Software-startupEnvironment: multi-valued software startup environmental variablesGridLab-Mds-Software-executable: software executableGridLab-Mds-Software-arguments: multi-valued software argumentsGridLab-Mds-Software-description: software descriptionGridLab-Mds-Software-helpURL: software help URLGridLab-Mds-Software-usage: software usage
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Information about Users
GridLab-Mds-User-ID: user’s login name on local resourceGridLab-Mds-User-Mapped-DN: multi-valued attribute representing the Distinguished Name mapped on the userGridLab-Mds-User-homedir: user’s home directoryGridLab-Mds-User-shell: user’s shellGridLab-Mds-User-UID: user’s UIDGridLab-Mds-User-GID: user’s GIDGridLab-Mds-User-comment: a short comment about the user
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Information about Firewall
GridLab-Mds-Firewall-hostname: firewall hostnameGridLab-Mds-Firewall-ports: multi-valued attribute representing open ports (range)GridLab-Mds-Firewall-validityTime: time frame during which open ports ranges are validGridLab-Mds-Firewall-adminDN: Distinguished Name of firewall administrator
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Information about VOs
GridLab-Mds-Vo-name: Virtual Organization to which a specified computational resource belongs toGridLab-Mds-Vo-helpDeskPhoneNumber: multi-valued help desk phone numberGridLab-Mds-Vo-helpDeskURL: URL pointing to a Virtual Organization’s web pageGridLab-Mds-Vo-adminName: administrator name of the VO
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Information about CAs
GridLab-Mds-Certificate-Subj: Distinguished Name of the recognized certification authorityGridLab-Mds-Certificate-version: CA’s certificate versionGridLab-Mds-Certificate-serialNumber: CA’s certificate serial numberGridLab-Mds-Certificate-signatureAlgorithm: CA’s certificate signature algorithmGridLab-Mds-Certificate-issuer: CA’s certificate issuerGridLab-Mds-Certificate-validity-from: beginning date of the CA’s certificate validityGridLab-Mds-Certificate-validity-to: end date of the CA’s certificate validityGridLab-Mds-Certificate-publicKeyAlgorithm: CA’s certificate public key algorithmGridLab-Mds-Certificate-RSAPublicKey: CA’s certificate RSA public key
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS Web Service
gridlab_searchthis method can be used to query an arbitrary MDS server to retrieve specific information
gridlab_getServiceDescriptionthis method can be used to get the description of the service
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS Web Service
gridlab_register_servicethis method allows the user to register a new service.
gridlab_unregister_servicethis method removes an existing service from the GIS
gridlab_lookup_servicethis method lookups an existing service and returns the information about the services that have been found. It contacts the supplied MDS server on the default port
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS Web Service
gridlab_register_softwarethis method allows the user to register a new software package
gridlab_unregister softwarethis method removes an existing software package from the GIS
gridlab_lookup softwarethis method lookups an existing software package and returns all the information about the software. It contacts the supplied MDS server on the default port
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS Web Service
gridlab_register_firewallthis method allows the registration of information about a firewall installed on the grid resource.
gridlab_unregister_firewallthis method removes information about a registered firewall from the GIS
gridlab_lookup_firewallthis method lookups firewall information. It contacts the suppliedMDS server on the default port
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS Web Service
gridlab_register_vothis method allows the user to register the virtual organization to which the grid resource belongs.
gridlab_unregister_vothis method removes information about a Virtual organization from the GIS
gridlab_lookup_vothis method lookups information about a specified virtual organization. It contacts the supplied MDS server on the default port
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
MDS Web Service
gridlab_lookup_userthis method lookups the information about a specified user. It contacts the supplied MDS server on the default port
gridlab_lookup_certthis method lookups the information about a specified certification authority. It contacts the supplied MDS server on the default port
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
Outline
Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring & adding information providersGridLab WP 10 activitiesWork in progress
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
WP10: Current Activities
Configuration of the GridLab MDS to allow GSI connections through SASL bindingDevelopment of C APIs (library) to access MDS
Anonymous Binding and GSI-SASL Binding
Analisys of the GT3 Index ServiceTo migrate GridLab MDS when GT3 is ready for production
Development of information providers for:Information about jobs submitted to PBS schedulerInformation about PBS queues
Development of GSI-plugin for gSOAP
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GridLab MDS – SASL Configuration
MDS supports the following security featuresAuthentication through Cyrus SASLPrivacy and integrity protections through the use of TLSAccess Control to database entries based on LDAP authorization information
IP addressDomain nameOthers
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GridLab MDS – SASL configuration
grid-info-slapd.confit sets basic access control rules and sets anonymous binding. Additionally, this file designates the GIIS and GRIS provider components to OpenLDAP, establishes LDAP and MDS information schema, and defines back ends supported by the slapd server. The anonymousbind and access to parameters in this file set anonymous binding and access control, respectively.
grid-info-site-policy.confit controls the acceptance of registration messages by a GIIS. This file can be used to create an open policy where all registrants are welcome, or a closed system whereby only specified resources can register with a GIIS.
grid-info-resource-register.confit lists the GIIS servers to which a GRIS or “child” GIIS will register directly. The default is to register to the local GIIS on the host. This file identifies host names, ports, and time values that control registration messages from a GRIS or GIIS to a GIIS server.
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GridLab MDS – SASL configuration
Mutual authentication between GRIS-GIIS and GIIS-GIIS in a hierarchy
In file grid-info-resource-register.conf set:bindmethod: AUTHC-ONLY
SASL authorization is performed through:Screening against the grid-mapfileStatic ACLs stored in file grid-info-slapd.conf To activate SASL authorization set, in file grid-info-slapd.conf:
Anonymousbind noAccess to <what> [by <who> <access>]+
What: select the entries and/or attributes to which the access apply Who: which entities are granted access (DN) Access: specifies the type of access granted (search, read, write,
compare, …)
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GT3 Index Service
MDS functionality appears throughout GT3GT3 GRAM service data (GRAM reporter)GT3 Index Service (GIIS)OGSI Service Data interface
Information services capabilities are now more tightly integrated in (the right places in) GT3 than they were in GT2.
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GT3 Index Service
Dynamic service data creation and management via information provider programs Aggregation of service data from multiple instances Registration of Grid service instances
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GT3 ServiceDataProviders
SimpleSystemInformation Java native system probe. This provider enumerates the following data; CPU count, Memory statistics, OS type, and Logical Disk Volumes.
HostScriptProviderLinux-specific set of shell scripts that monitor system-specific host data.
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GT3 Index Service
Three main mechanismsPoll Automated cache refillingSubscribe to / receive notifications from arbitrary services
The Index Service will aggregate Service Data it receivesNo fixed schema for resource dataInformation Providers now output XML fragments
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GT2 & 3 Index Service differences
Subscription & notificationsThis subsumes the role of the GRIS backend server module in GT2
GRAM, GridFTP, … Service Data providers These Service Data sources subsume the role of the sample GRAM and host-information provider scripts in GT2
Service Data Descriptions in Service Type WSDL
These subsume the role of MDS schema written in the RFC2252 LDAP schema format.
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
C APIs for accessing MDS
struct mdsInfo *gridlab_search (char *giis, int port, char *base_dn, int scope, char **attributes, char *filter, int verbose)
Binds anonimously to an MDS server and queries it
struct mdsInfo *gridlab_search_auth()Binds through GSI-SASL to an MDS server and queries it
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
New Information Providers
To publish queues informationCurrently working on the OpenPBS scheduler
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
GSI plugin for gSOAP
Continuous development to ensure compatibility with the latest gSOAP versionAddition of new features, as requested by the users
2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro
References
GridLab WP 10 Information Serviceshttp://www.gridlab.org/WorkPackages/wp-10/index.html
Globus Toolkit MDShttp://www.globus.org
gSOAP Toolkithttp://www.cs.fsu.edu/~engelen/soap.html
GSI plugin for gSOAPhttp://sara.unile.it/~cafaro/gsi-plugin.html