wp 10 information services giovanni aloisio, massimo cafaro, italo epicoco [email protected]...

87
WP 10 Information Services Giovanni Aloisio, Massimo Cafaro, Italo Epicoco [email protected] [email protected] [email protected] University of Lecce, Italy

Upload: gwendolyn-shields

Post on 26-Dec-2015

215 views

Category:

Documents


0 download

TRANSCRIPT

WP 10

Information Services Giovanni Aloisio, Massimo Cafaro, Italo

Epicoco

[email protected]@unile.it

[email protected]

University of Lecce, Italy

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Outline

Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring & adding information providersGridLab WP 10 activitiesWork in progress

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Grid Information Services

System information is critical to operation of the grid and construction of applications

What resources are available?Resource discovery

What is the “state” of the grid?Resource selection

How to optimize resource use Application configuration and adaptation?

We need a general information infrastructure to answer these questions

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Examples of useful Information

Characteristics of a compute resourceIP address, software available, system administrator, networks connected to, OS version, load

Characteristics of a networkBandwidth and latency, protocols, logical topology

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Grid Information: facts of Life

Information is always oldchanging system stateneed to provide quality metrics

Distributed state hard to obtaincomplexity of global snapshot

Component will failScalability and overheadMany different usage scenarios

heterogeneous policy, different information organizations, different queries, etc.

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Grid Information Service

Provide access to static and dynamic information regarding system componentsA basis for configuration and adaptation in heterogeneous, dynamic environmentsRequirements and characteristics

Uniform, flexible access to informationScalable, efficient access to dynamic dataAccess to multiple information sourcesDecentralized maintenance

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

The GIS Problem: Many Information Sources, Many Views

?RR

R

RR

?

R

R

RR

R?

R

R

R

RR

?

RR

VO A

VO B

VO C

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

What is a Virtual Organization?

• Facilitates the workflow of a group of users across multiple domains who share (some of) their resources to solve particular classes of problems

• Collates and presents information about these resources in a uniform view

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Outline

Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring & adding information providersGridLab WP 10 activitiesWork in progress

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Two Classes of MDS Servers

Grid Resource Information Service (GRIS)Supplies information about a specific resourceConfigurable to support multiple information providersLDAP as inquiry protocol

Grid Index Information Service (GIIS)Supplies collection of information which was gathered from multiple GRIS serversSupports efficient queries against information which is spread across multiple GRIS serverLDAP as inquiry protocol

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS 2.2 Architecture

A A

Customized Aggregate Directories

R RR R

Standard Resource Description Services

Registration

Protocol

Users

Enquiry

Protocol

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Information Protocols

Grid Resource Registration ProtocolSupport information/resource discoveryDesigned to support machine/network failure

Grid Resource Inquiry ProtocolQuery resource description server for informationQuery aggregate server for informationLDAP V3.0 in Globus

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS 2.2 Service Architecture

Dynamic Registration via Reg. Protocol (GRRP)Resource Inquiry via Info. Protocol (GRIP)

Co-located with resource on network

Resource Discovery (via GRIP or other)Using GRIP allows resource/directory hierarchy

discovery (GRIP?)

lookup (GRIP)registration (GRRP)

VO-specific Aggregate Directories

standard Resource Description services

R R R R

A A?

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Distributed Services

Service scales with Grid growthLoose consistency model tolerates failuresInteroperability by GRIP/GRRP protocols

R

R R

R R

R

R R

fault-partitionregistrationmessages

replicated directories divergent directories

R

R

RR

RR R

RR

VO-A VO-B

RR

R

R

RR R

RR

R

R R

R R

R

RR D

D

D

D

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Metacomputing Directory Service

Uses LDAP as Inquiry Access information in a distributed directory

Directory represented by collection of LDAP serversEach server optimized for particular function

Directory can be updated by: Information providers and toolsApplications (i.e., users)Backend tools which generate info on demand

Information dynamically available to tools and applications

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

LDAP Details

Lightweight Directory Access ProtocolIETF StandardStripped down version of X.500 DAP protocolSupports distributed storage/access (referrals)Supports authentication and access control

Defines:Network protocol for accessing directory contentsInformation model defining form of information Namespace defining how information is referenced and organized

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS Components

LDAP 3.0 Protocol EngineBased on OpenLDAP with custom backendIntegrated caching

Information providersDelivers resource information to backend

APIs for accessing & updating MDS contentsC, Java, PERL (LDAP API, JNDI)

Various tools for manipulating MDS contentsCommand line tools, Shell scripts & GUIs

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Grid Resource Information Service

Server which runs on each resourceGiven the resource DNS name, you can find the GRIS server

well known port = 2135

Provides resource specific informationMuch of this information may be dynamic

Load, process information, storage information, etc.GRIS gathers this information on demand

“White pages” lookup of resource informationEx: How much memory does machine have?

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Grid Index Information Service

GIIS describes a class of serversGathers information from multiple GRIS serversEach GIIS is optimized for particular queries

Ex1: Which Alliance machines are >16 process SGIs?Ex2: Which Alliance storage servers have >100Mbps bandwidth to host X?

Akin to web search engines

Organization GIISThe Globus Toolkit ships with one GIISCaches GRIS info with long update frequency

Useful for queries across an organization that rely on relatively static information (Ex1 above)

“Yellow pages” lookupEx: Which machines have large memory?

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Server Registration

A GRIS or GIIS server can be configured to (de-) register itself during startup/shutdown

Targets specified in configuration file

Soft-state registration protocolGood behavior in case of failure

Allows for federations of information serversE.g. Argonne GRIS can register with both Alliance and DOE GIIS servers

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Soft-state Registration

Periodic notification“Service/resource is available”Expected-frequency metadata

Automatic directory constructionAdd new resources to directoryInvite resources to join new directory

Self-cleaningReduce occurrence of “dead” references

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Hierarchical MDS Deployment

GRISes

GIIS

GIIS

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS Commands

LDAP defines a set of standard commandsldapsearch, etc.

Globus Toolkit defines MDS-specific commands

grid-info-search, grid-info-host-search

APIs are defined for C, Java, etc.C: OpenLDAP client API

ldap_search_s(), …

Java: JNDI

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Information Services API

RFC 1823 defines an IETF draft standard client API for accessing LDAP databases

Connect to serverPose query which returns data structures contains sets of object classes and attributesFunctions to walk these data structures

Globus does not provide an LDAP API and recommends the use of OpenLDAP, an open source implementation of RFC 1823.LDAP APIs available in other languages

E.g. Java JDNI, Perl, Python, etc.

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS 2.2 Features

Security MechanismsGSI mutual-authenticationFine-grained access control by GSI name

Performance EnhancementsBetter query speedsLess stale information

New Information Model (schema)Better representation of computersCleaner namespace management

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS 2.2 Software Stack

OpenLDAP 2.0.xImplements LDAPv3 protocolClient and server components

Cyrus-SASLGeneric securityWe provide loadable SASL/GSS plugin

Globus GSIProvides GSS-API interface to PKIShared library used by our SASL plugin

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS 2.2 Information Model

Structural informationResource hierarchy maps to objectsNamed positions in LDAP DIT

Merged informationSome parents “join” child dataSimplifies common query patterns

Auxiliary informationUniform representation of leaf/parent dataUses LDAP auxiliary objectclasses

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GRIS Host Objects

/scratch1dev=

diskdev group=

DISK

DISK netdev group=

eth0dev=NET

NET

hn=hostname

cpu 0dev=CPU

cpu 1dev=CPU

CPUsdev group=

CPU

CPU

dev=RAM VMdev=RAM VM

RAM

VM

dev group=memory

software=OS

CPU

CPU

RAM

VM

DISK

NET

OS

OS

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GRIS Object Hierarchy

Mds-Host-name=hostnameMds-Software-Deployment=operating systemMds-Device-Group-name=processors

Mds-Device-name=cpu 0

Mds-Device-Group-name=memoryMds-Device-name=physical memoryMds-Device-name=virtual memory

Mds-Device-Group-name=filesystemsMds-Device-name=/scratch1Mds-Device-name=/scratch2

Mds-Device-Group-name=networksMds-Device-name=eth0

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GRIS Structural Class Hierarchy

MdsAttr: Mds-validfrom (like createtime)Attr: Mds-validto (accuracy metadata)Attr: Mds-keepto (discard metadata)

MdsHostMdsDeviceMdsDeviceGroupMdsSoftwareDeployment

Every MDS object: name, time metadata

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GRIS Auxiliary Class Examples

MdsCpuAttr: Mds-Cpu-vendorAttr: Mds-Cpu-modelAttr: Mds-Cpu-speedMHz

MdsCpuCacheAttr: Mds-Cpu-Cache-L1kB

MdsCpuSmpAttr: Mds-Cpu-Smp-size

MdsCpuTotalAttr: Mds-Cpu-Total-count

Once per CPU

Once per CPU

Once per SMP

Once per MPP

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GRIS Auxiliary Class Examples

MdsCpuFree (once per SMP)Attr: Mds-Cpu-Free-1minX100Attr: Mds-Cpu-Free-5minX100Attr: Mds-Cpu-Free-15minX100

MdsCpuTotalFree (once per MPP)Attr: Mds-Cpu-Total-Free-1minX100Attr: Mds-Cpu -Total-Free-5minX100Attr: Mds-Cpu -Total-Free-15minX100

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Client Tools

Globus Toolkit includes 2 command line client tools for querying MDS services

grid-info-search: General purpose clientgrid-info-search –h <host> -p <port> -b <base> \ -T <timeout> [<filter>] [<attributes>]-x: Anonymous access

grid-info-host-search: Same as grid-info-search, but defaults to GRIS standard port

E.g. grid-info-host-search –h localhost

Both clients can search for specific system information and filter results.

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Filtering

Filters allow selection of object based on relational operators

=, ~=,<=, >=grid-info-search “cputype=*”

Compound filters can be construct with Boolean operations

&, |, !grid-info-search “(&(cputype=*)(cpuload1<=1.0))”grid-info-search “(&(hn~=sdsc.edu)(latency<=10))”

Hints:white space is significant

use -L for LDIF format

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Outline

Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring & adding information providersGridLab WP 10 activitiesWork in progress

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

dc-n1.isi.edu

grid-info-site-policy.conf

grid-info-resource-register.confgrid-info-resource-ldif.conf

dc-n2.isi.edu

grid-info-resource-register.confgrid-info-resource-ldif.conf

GRIS

GIIS

GRIS

Configuration files for Registration

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Configuration files for Registration

A resource that is hosting a GIISgrid-info-site-policy.conf

Determines whether to accept incoming registrationsAccept everything, or only registrations from the resources explicitly defined in this conf file (can use wildcards)

Default policy is to only accept registrations from self, and from port 2135

In a hierarchical GIIS environment, this file must be modified from the default

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Default Policy Data:objectclass: MdsRegistrationPolicypolicydata: (&(Mds-Service-hn=mako.isi.edu)(Mds-Service-port=2135))

Change ‘policydata’ to this for completely open policy:(Mds-Service-hn=*)

Change ‘policydata’ to this to restrict to 2 specific hosts:(&(|(Mds-Service-hn=dc-n2.isi.edu)(Mds-Service-hn=dc-n3.isi.edu))(Mds-Service-port=2135))

grid-info-site-policy.conf

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Configuration files for Registration

A resource registering GRIS information with a GIIS

grid-info-resource-register.confWhich GIIS’s this GRIS should register to, and howGIIS could be on the same machine, but may not be

grid-info-resource-ldif.confDetermines which GRIS providers are active and available to send data to the GIIS’s that this GRIS is registering with

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Registering a GRIS with a GIIS

dn: <LDAP add object DN>regtype: mdsreg2reghn: <host to send reg to>regport: <port to send reg to>regperiod: <how often to send reg (seconds)>[service attribute/value]...

where service attribute entries depend on the type of LDAP object being published

$GLOBUS_LOCATION/etc/grid-info-resource-register.conf

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

ExampleGRIS on dc-n2 registers with GIIS on dc-n1

dn: Mds-Vo-Op-name=register, Mds-Vo-name=site, o=gridregtype: mdsreg2reghn: dc-n1.isi.eduregport: 2135regperiod: 600type: ldaphn: dc-n2.isi.eduport: 2135rootdn: Mds-Vo-name=local, o=gridttl: 1200timeout: 20mode: cachedumpcachettl: 30

$GLOBUS_LOCATION/etc/grid-info-resource-register.conf

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Registration Control Parameters

regperiodHow often this GRIS will send a message to the GIIS noting its existence

ttlHow long the recieving GIIS should keep the registration information before assuming that the GRIS is no longer availableIn general ttl should be: ttl = 2 x (regperiod)

cachettlRecommendation to the GIIS about how long to maintain in cache, the GRIS information provided by this resource

$GLOBUS_LOCATION/etc/grid-info-resource-register.conf

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS 2.2 GRIS Providers

grid-info-cpu reports CPU/load infogrid-info-fs reports filesystem infogrid-info-mem reports RAM/VM infogrid-info-net reports NIC/net info grid-info-os reports OS info grid-info-platform reports arch. info grid-info-merged merges all host info*Extensible for other sources, e.g. GRAM

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GRIS Dispatch Logic

For each provider:1. Could search intersect provider? No, then skip.2. Is provider cache stale? Yes, then refill.3. Apply search filter to cache data.

Combine all intersecting providers’ results

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GRIS Response Issues

MDS 2.2 is lazyProbes are not issued unless queried

Some system probes are slow“Best” probe may take several seconds

How to avoid stale data?Clients set time-out per queryGRIS/GIIS define time-out per sourceFresh data found before timeout is returnedCache fill continues after client time-out

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

New Information Providers

Decide what information to publish into MDS

Create a program that implements the I/O interface requirements of a GRIS Information Provider

Enable the information provider for an MDS installation

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

New Information Providers

Decide what information to publish into MDSOID: conflict avoidance

OID Registered with IANA Private Enterprise Numbers http://www.iana.org/cgi-bin/enterprise.pl 1.3.6.1.4.1.3536.* Globus OID subspace 1.3.6.1.4.1.3536.2.* Globus Info Services OID subspace 1.3.6.1.4.1.3536.2.6.* MDS OID subspace

GridLab project OID: 1.3.6.1.4.1.3536.2.6.3536.9Object Naming: conflict avoidance

ISI will coordinate prefix naming. Request a prefix from: mailto:[email protected]

GridLab prefix: “GridLab”Schema

Data Modeling problemLDAP schema syntax problem$GLOBUS_LOCATION/etc/grid-info-resource.schema

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

New Information Providers

Create your programAny language. The only requirements are the I/O interface of your program:

Must be callable by fork and exec from the slapd process You can pass in arguments to your program

Must return data in LDIF format Data returned must match the LDAP schema RFC 2849

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

New Information Providers

Enable your new information provider by adding a configuration block$GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf

# generate memory info every minutedn: Mds-Device-Group-name=memory, Mds-Host-hn=mako.isi.edu, Mds-Vo-name=local, o=gridobjectclass: GlobusTopobjectclass: GlobusActiveObjectobjectclass: GlobusActiveSearchtype: execpath: /globus/libexecbase: grid-info-mem-linuxargs: -devclassobj -devobjs -dn Mds-Host-hn=mako.isi.edu,Mds-Vo-name=local,o=grid -validto-secs 60 -keepto-secs 60cachetime: 60timelimit: 10sizelimit: 3

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

New Information Providers

dnWhere the object lives in the DIT

These lines must be included:objectclass: GlobusTopobjectclass: GlobusActiveObjectobjectclass: GlobusActiveSearchtype: exec

pathPath to the information provider program

baseName of the information provider program

argsArguments to be passed to the information provider program

cachetimeIn seconds, how long GRIS will consider the data to not be stale

timelimitIn seconds, how long the GRIS should wait for the information provider to return data before giving up on it

sizelimitMax number of LDIF objects to be read from the output of the information provider

$GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Outline

Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring servers & adding information providersGridLab WP 10 activitiesWork in progress

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GridLab WP 10 Tasks

T10.1 Analysis of current state of the art GIS. CompletedT10.2 Comparison of GIS capabilities and GridLab dynamic grid computing requirements. CompletedT10.3 Specification of information model. CompletedT10.4 Security mechanisms & policies.T10.5 Development of extended GIST10.6 Development of high-level APIs

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GridLab WP 10 Deliverables

D10.1 Analysis report. DeliveredD10.2 Requirements & specification. DeliveredD10.3 Prototype release. DeliveredD10.4 Extended GIS releaseD10.5 First release reportD10.6 Second release report

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GridLab WP 10 Milestones

M10.1 Month 3 Requirements analysis report. On timeM10.2 Month 6 Technical specification. On timeM10.3 Month 12 Prototype GIS server. On timeM10.4 Month 24 First release of GISM10.5 Month 36 Documented second release of GIS

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Requirements of GridLab MDS

PerformanceScalabilitySecurityUniformityExpressivenessExtensibilityMultiple sourcesDynamic dataFlexible accessDeployabilityDecentralized control

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

WP 10: Current release

MDS schema extension to provide information about:ServicesSoftwareUsersFirewallsVirtual OrganizationsCertification Authorities

Access to MDSWe developed a GSI enabled version of gSOAP supporting mutual authentication and delegation of credentialsDeveloped GSI enabled Web Services in order to:

Register informationUnregister informationLookup informationGeneric MDS search

Developed GSI enabled clients for the above functionalities

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Implementation

GT 2.x basedC based

Uses LDAP API

WEB SERVICEGSI enabled; uses gSOAP ToolkitWSDL description for clients

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

The gSOAP Tooolkit

by Robert Van Engelen, Florida State University, USAdevelopment environment for C/C++ Web serviceseasy-to-use RPC compiler: produces the stub and skeleton routines to integrate (existing) C or C++ applications into SOAP/XML Web servicesautomatically maps native C/C++ application data types to semantically equivalent XML types and vice versasimple API relieving the user from the burden of SOAP detailsSOAP 1.1/1.2 and WSDL 1.1Zlib compression, SSL encryption, and streaming direct internet message encapsulation (DIME) attachments

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

The GSI plugin for gSOAP

By Massimo Cafaro, Daniele Lezzi (University of Lecce) and Robert Van Engelen (FSU)Overrides gSOAP callbacks to provide transport-level GSI securityProvides mutual authentication and delegation of credentialsVery simple API relieving the user from the burden of GSI details

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

A GSI enabled client

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

A GSI enabled, threaded Web Service

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Information about Services

GridLab-Mds-Service-name: service nameGridLab-Mds-Service-port: service portGridLab-Mds-Service-type: service protocolGridLab-Mds-Service-description: service descriptionGridLab-Mds-Service-publisher: service publisher

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Information about Software

GridLab-Mds-Software-name: nameGridLab-Mds-Software-version: versionGridLab-Mds-Software-path: pathnameGridLab-Mds-Software-totalLicences: number of total software licencesGridLab-Mds-Software-freeLicence: number of available software licencesGridLab-Mds-Software-licenceInfo: information about software licenceGridLab-Mds-Software-startupEnvironment: multi-valued software startup environmental variablesGridLab-Mds-Software-executable: software executableGridLab-Mds-Software-arguments: multi-valued software argumentsGridLab-Mds-Software-description: software descriptionGridLab-Mds-Software-helpURL: software help URLGridLab-Mds-Software-usage: software usage

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Information about Users

GridLab-Mds-User-ID: user’s login name on local resourceGridLab-Mds-User-Mapped-DN: multi-valued attribute representing the Distinguished Name mapped on the userGridLab-Mds-User-homedir: user’s home directoryGridLab-Mds-User-shell: user’s shellGridLab-Mds-User-UID: user’s UIDGridLab-Mds-User-GID: user’s GIDGridLab-Mds-User-comment: a short comment about the user

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Information about Firewall

GridLab-Mds-Firewall-hostname: firewall hostnameGridLab-Mds-Firewall-ports: multi-valued attribute representing open ports (range)GridLab-Mds-Firewall-validityTime: time frame during which open ports ranges are validGridLab-Mds-Firewall-adminDN: Distinguished Name of firewall administrator

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Information about VOs

GridLab-Mds-Vo-name: Virtual Organization to which a specified computational resource belongs toGridLab-Mds-Vo-helpDeskPhoneNumber: multi-valued help desk phone numberGridLab-Mds-Vo-helpDeskURL: URL pointing to a Virtual Organization’s web pageGridLab-Mds-Vo-adminName: administrator name of the VO

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Information about CAs

GridLab-Mds-Certificate-Subj: Distinguished Name of the recognized certification authorityGridLab-Mds-Certificate-version: CA’s certificate versionGridLab-Mds-Certificate-serialNumber: CA’s certificate serial numberGridLab-Mds-Certificate-signatureAlgorithm: CA’s certificate signature algorithmGridLab-Mds-Certificate-issuer: CA’s certificate issuerGridLab-Mds-Certificate-validity-from: beginning date of the CA’s certificate validityGridLab-Mds-Certificate-validity-to: end date of the CA’s certificate validityGridLab-Mds-Certificate-publicKeyAlgorithm: CA’s certificate public key algorithmGridLab-Mds-Certificate-RSAPublicKey: CA’s certificate RSA public key

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS Web Service

gridlab_searchthis method can be used to query an arbitrary MDS server to retrieve specific information

gridlab_getServiceDescriptionthis method can be used to get the description of the service

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS Web Service

gridlab_register_servicethis method allows the user to register a new service.

gridlab_unregister_servicethis method removes an existing service from the GIS

gridlab_lookup_servicethis method lookups an existing service and returns the information about the services that have been found. It contacts the supplied MDS server on the default port

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS Web Service

gridlab_register_softwarethis method allows the user to register a new software package

gridlab_unregister softwarethis method removes an existing software package from the GIS

gridlab_lookup softwarethis method lookups an existing software package and returns all the information about the software. It contacts the supplied MDS server on the default port

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS Web Service

gridlab_register_firewallthis method allows the registration of information about a firewall installed on the grid resource.

gridlab_unregister_firewallthis method removes information about a registered firewall from the GIS

gridlab_lookup_firewallthis method lookups firewall information. It contacts the suppliedMDS server on the default port

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS Web Service

gridlab_register_vothis method allows the user to register the virtual organization to which the grid resource belongs.

gridlab_unregister_vothis method removes information about a Virtual organization from the GIS

gridlab_lookup_vothis method lookups information about a specified virtual organization. It contacts the supplied MDS server on the default port

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

MDS Web Service

gridlab_lookup_userthis method lookups the information about a specified user. It contacts the supplied MDS server on the default port

gridlab_lookup_certthis method lookups the information about a specified certification authority. It contacts the supplied MDS server on the default port

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

Outline

Grid Information ServicesGlobus Toolkit 2.x MDSConfiguring & adding information providersGridLab WP 10 activitiesWork in progress

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

WP10: Current Activities

Configuration of the GridLab MDS to allow GSI connections through SASL bindingDevelopment of C APIs (library) to access MDS

Anonymous Binding and GSI-SASL Binding

Analisys of the GT3 Index ServiceTo migrate GridLab MDS when GT3 is ready for production

Development of information providers for:Information about jobs submitted to PBS schedulerInformation about PBS queues

Development of GSI-plugin for gSOAP

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GridLab MDS – SASL Configuration

MDS supports the following security featuresAuthentication through Cyrus SASLPrivacy and integrity protections through the use of TLSAccess Control to database entries based on LDAP authorization information

IP addressDomain nameOthers

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GridLab MDS – SASL configuration

grid-info-slapd.confit sets basic access control rules and sets anonymous binding. Additionally, this file designates the GIIS and GRIS provider components to OpenLDAP, establishes LDAP and MDS information schema, and defines back ends supported by the slapd server. The anonymousbind and access to parameters in this file set anonymous binding and access control, respectively.

grid-info-site-policy.confit controls the acceptance of registration messages by a GIIS. This file can be used to create an open policy where all registrants are welcome, or a closed system whereby only specified resources can register with a GIIS.

grid-info-resource-register.confit lists the GIIS servers to which a GRIS or “child” GIIS will register directly. The default is to register to the local GIIS on the host. This file identifies host names, ports, and time values that control registration messages from a GRIS or GIIS to a GIIS server.

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GridLab MDS – SASL configuration

Mutual authentication between GRIS-GIIS and GIIS-GIIS in a hierarchy

In file grid-info-resource-register.conf set:bindmethod: AUTHC-ONLY

SASL authorization is performed through:Screening against the grid-mapfileStatic ACLs stored in file grid-info-slapd.conf To activate SASL authorization set, in file grid-info-slapd.conf:

Anonymousbind noAccess to <what> [by <who> <access>]+

What: select the entries and/or attributes to which the access apply Who: which entities are granted access (DN) Access: specifies the type of access granted (search, read, write,

compare, …)

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GT3 Index Service

MDS functionality appears throughout GT3GT3 GRAM service data (GRAM reporter)GT3 Index Service (GIIS)OGSI Service Data interface

Information services capabilities are now more tightly integrated in (the right places in) GT3 than they were in GT2.

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GT3 Index Service

Dynamic service data creation and management via information provider programs Aggregation of service data from multiple instances Registration of Grid service instances

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GT3 ServiceDataProviders

SimpleSystemInformation Java native system probe. This provider enumerates the following data; CPU count, Memory statistics, OS type, and Logical Disk Volumes.

HostScriptProviderLinux-specific set of shell scripts that monitor system-specific host data.

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GT3 Index Service

Three main mechanismsPoll Automated cache refillingSubscribe to / receive notifications from arbitrary services

The Index Service will aggregate Service Data it receivesNo fixed schema for resource dataInformation Providers now output XML fragments

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GT2 & 3 Index Service differences

Subscription & notificationsThis subsumes the role of the GRIS backend server module in GT2

GRAM, GridFTP, … Service Data providers These Service Data sources subsume the role of the sample GRAM and host-information provider scripts in GT2

Service Data Descriptions in Service Type WSDL

These subsume the role of MDS schema written in the RFC2252 LDAP schema format.

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

C APIs for accessing MDS

struct mdsInfo *gridlab_search (char *giis, int port, char *base_dn, int scope, char **attributes, char *filter, int verbose)

Binds anonimously to an MDS server and queries it

struct mdsInfo *gridlab_search_auth()Binds through GSI-SASL to an MDS server and queries it

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

New Information Providers

To publish queues informationCurrently working on the OpenPBS scheduler

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

GSI plugin for gSOAP

Continuous development to ensure compatibility with the latest gSOAP versionAddition of new features, as requested by the users

2nd GridLab ConferenceEger - Hungary, March 31 - April 1 2003Massimo Cafaro

References

GridLab WP 10 Information Serviceshttp://www.gridlab.org/WorkPackages/wp-10/index.html

Globus Toolkit MDShttp://www.globus.org

gSOAP Toolkithttp://www.cs.fsu.edu/~engelen/soap.html

GSI plugin for gSOAPhttp://sara.unile.it/~cafaro/gsi-plugin.html