3 September 2004

NVO Coordination Meeting 1

Grid-TechnologiesGrid-Technologies

NVO and the Grid

Reagan W. Moore

George Kremenek

Leesa Brieger

Ewa Deelman

Roy Williams

John Good

Bruce Berriman

3 September 2004

NVO Coordination Meeting 2

Why Grids?

• Support for large-scale data processing

• Collection-based distributed data management– Replication of large collections onto the Teragrid

• Access to distributed resources– Distributed data caching– Workflow processing systems– Distributed job execution– Common authentication environment– Service life cycle management

3 September 2004

NVO Coordination Meeting 3

Grid Technologies

• Grid Portals - GridPort• Workflow control pipelines - Chimera/Pegasus• Job scheduling management - CondorG• Job execution system - GRAM • Data caching and replication - RLS• Authentication system - GSI• Large file data transport – GridFTP, RFT• Metadata catalog - MCS, MCAT• Collection management – SRB• Database Access on the Grid: OGSA-DAI

3 September 2004

NVO Coordination Meeting 4

NVO Grid ComponentsView 1 - resources

• Portals - web service interfaces to analysis procedures (cone search, VOTable catalog query, simple image access)

• Process management systems - data processing pipelines to create derived data products

• Data access layer - management of methods on data encoding formats

• Data grid - management of distributed collections. Provide logical name space for global persistent identifiers.

• Persistent data caches - interactive access to sky survey image collections (Grid Bricks)

• High performance disk caches - high speed access for bulk data analysis.

• Computational grid - access to distributed compute resources.

• Persistent archives - management of technology evolution.

3 September 2004

NVO Coordination Meeting 5

NVO Grid ComponentsView 2 - data virtualization

• Data grids provide a standard access mechanism to image archives– Install data grid interface to an existing archive– Install NVO service on top of data grid

• NVO services ported to Data Grids – Image cutout– Mosaicing

• Integrate processing pipeline on top of Grid– Workflow composed from NVO services– Chimera and Pegasus

3 September 2004

NVO Coordination Meeting 6

NVO Grid ComponentsView 3 - services

• NVO Services– Montage, Image cutout, VOTable

• Image processing pipelines– Rome

• Image archives– 2MASS, DPOSS, SDSS, USNO-B, …

• Grid infrastructure– GSI– Chimera / Pegasus– CondorG– GRAM– RLS

3 September 2004

NVO Coordination Meeting 7

NVO Grid ComponentsView 4 - services

• Globus2: Access to distributed compute resources

• SRB: Collection-based distributed data management

• NVO registry: Distributed semantic grid with OAI transport. It provides the kind of publication, searching, and capability information to enable a community to build a virtual organization of services.

• OpenSkyQuery and SIAP: These services are providing bulk data via SRB to Grid projects on Teragrid (Atlasmaker). They will become the basis of astronomical data delivery via OGSA services.

• OGSA: A specification only, but Globus3 and OGSAdotNet are implementations. Based on SOAP and web services. We can convert OpenSkyQuery to OGSA services.

• Grist: This new project at Caltech and JPL (if formally funded by NSF) will build a library of OGSA services for data mining in astronomy, with a dataflow mechanism to control them.

3 September 2004

NVO Coordination Meeting 8

Challenges

• Grid technology is rapidly evolving; activities in progress– GT3 - OGSA based services

• Compatibility with WSDL from the Semantic Web

– GridFTP rewrite, protocol redesign– Chimera redesign (support collection access)– OGSA-DAI data access interface – Data Format Description Language– Grid File System design

3 September 2004

NVO Coordination Meeting 9

Next Steps

• Grid environment – OGSA based service support

• Grid enabled NVO services– Support access to data through data grid

• Demonstration of an analysis of an entire sky survey image archive– 2MASS mosaic

• Integration of additional image archives into grid

• Grid enabled processing pipelines