grid-enabling ogc web services
DESCRIPTION
Grid-enabling OGC Web Services. Andrew Woolf, Arif Shaon STFC e-Science Centre Rutherford Appleton Lab. Outline. OGC background Profiling approach Refactoring approach Grid-enabling WPS. OGC background. Key architectural frameworks - PowerPoint PPT PresentationTRANSCRIPT
Grid-enabling OGC Web Services
Andrew Woolf, Arif ShaonSTFC e-Science CentreRutherford Appleton Lab
Outline
• OGC background• Profiling approach• Refactoring approach• Grid-enabling WPS
OGC background
• Key architectural frameworks– Reference Model for Open Distributed
Processing (RM-ODP, ISO 10746-{1,2,3,4})
– ISO TC211 Domain Reference Model (ISO 19101)
• Enterprise viewpoint– roles, activities, policies (incl. VO)
• Information viewpoint– semantics of information and information
processing (static, invariant, dynamic schema)
• Computational viewpoint– interfaces and computational objects (cf. CORBA
IDL, WSDL portTypes)
• Engineering viewpoint– distribution infrastructure (e.g. web services,
WSRF vs OGSI)
• Technology viewpoint– choices of technology (e.g. app servers, DBMS)
RM-ODP
ISO 19101 ‘Domain Reference Model’
A geospatial dataset…
…consists of features and related objects…
…in a defined logical
structure…
…delivered through
services…
…and described by metadata.
servicesservices
SOA
WSRF
OGCWSDL
metadatametadata
resource discovery
catalogue
datadataaccess
integration
information
XML
registries
‘Spatial Data Infrastructure’
Summarymetadata data services
Enterprise •discovery •download, view •process
Information •ISO 19115•ISO 19139
•OGC Geography Markup Language (ISO 19136)
•OWS Common
Computational
•OGC Catalogue Service for the Web (CSW)
•OGC Web Feature Service (ISO 19142)•OGC Web Coverage Service•OGC Web Map Service (ISO 19128)
•OGC Web Processing Service (WPS)
Engineering
Technology
Grid-enabling OGC: profiling
• Use existing specifications• Incorporate ‘Grid’ functionality
through convention, e.g.– specific parameters for security
tokens
• Implementation uses Grid middleware at backend
Grid-enabling OGC: refactoring
• Problems:– tight coupling: service ↔ data
• data providers have to be service providers
– latency• general approach for asynchronous
– service coherence• data → (view, download, process)
– workflow abstraction• abstraction: data, operations
– service consistency• data resource: WPS vs. W{M,F,C}S
Grid-enabling OGC: refactoring
• WSRF:– separates ‘service’ and ‘stateful
resource’ upon which service acts– WS-Resource (‘implied resource pattern’):
• identity (WS-Addressing)• lifetime (WS-ResourceLifetime)• state/properties (WS-ResourceProperties)
– publish-subscribe mechanism for state changes (WS-Notification)
Grid-enabling OGC: refactoring
Existing OGC WSRF-refactored
Grid-enabling WPS• WPS provides standardised means of
publishing, discovery and use of geospatial processes on the web
• Fundamentally, both WPS and Grid provide a remote interface for invoking computational processes
Grid-enabling WPS• JSDL provides a standardised description of a
computational job and its required resources• Similarities with WPS:
– process description information (Identifier, JobIdentification/JobName)
– process outputs and inputs (DataInputs, POSIXApplication/Input,Argument,Output, DataStaging/Source)
– sufficient to produce a valid JSDL document
• Differences with WPS:– resource requirements (filesystem parameters, disk space,
operating system, CPU, etc.)– data staging instructions for input and output
Grid-enabling WPS• WPS+JSDL as interface to encapsulated Grid
resource
• JSDL consumed by compliant implementation, e.g. GridSAM
• Two broad options:– conceptually distinct data input parameters (through a
specific ‘JSDL’ parameter)– “ordinary” data input parameters
Grid-enabling WPS• Three approaches to distinct JSDL data input parameters:
– URL to a complete pre-created JSDL document, or valid snippethttp://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGenerator&DataInput=other_inputs=xxx;JSDL=http://www.foo.com/myfoo.jsdl@Format=text/xml&storeExecuteResponse=true&status=true
– URL-encoded JSDL document or snippethttp://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGenerator&DataInput=other_inputs=xxx;JSDL=%3CJobDefinition%3E%3CJobDescription%3E%3CResources%3E%3CTotalDiskSpace%3E%3CExact%3E5%3C%2FExact%3E%3C%2FTotalDiskSpace%3E%3CTotalCPUCount%3E%3CExact%3E1%3C%2FExact%3E%3C%2FTotalCPUCount%3E%3C%2FResources%3E%3C%2FJobDescription%3E%3C%2FJobDefinition%3E@Format=text/xml@Schema=http://server/jsdl.xsd&storeExecuteResponse=true&status=true
– Microformathttp://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGenerator&DataInput=other_inputs=xxx;JSDL=[TotalDiskSpace=5;TotalCPUCount=1]@Format=text/kvp&storeExecuteResponse=true&status=true
Grid-enabling WPS
• JSDL parameters encoded as individual WPS DataInput parameters
http://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGenerator&DataInput=other_inputs=xxx;TotalDiskSpace=5;TotalCPUCount=1&storeExecuteResponse=true&status=true
• BUT....– JSDL parameters are not inputs to the WPS process,
but rather specify required resources
Grid-enabling WPS
• Security– OGC security is subject of
considerable ongoing work– for now, used lightweight (and
dodgey) solution:• MyProxy parameters embedded in
request• secure transport layer
Grid-enabling WPS
• Implementation:– UK National Grid Service– BADC Trajectory Service– part of OWS-6 testbed– WPS 1.0.0-compliant
Grid-enabling WPS
Grid-enabled
WPS
Pylons Framework
GridSAM Client
UK NGS
GridSAM Service
Grid
WPS SOAP/Proxy Layer
MyProxy Server
Execute Process (SOAP)
Verify Cert.
Job Id
Query/Response
Response
WPS Response
Grid-enabling WPS
Conclusions• OGC adopts a sophisticated architectural
approach to distributed geospatial data/processing– NB: OGC-OGF MoU, OGF-28 (Munich Mar.
2010)
• Grid-enabling OGC:– ‘Profiling’ approach– ‘Refactoring’ approach
• Example: grid-enabled WPS profile
More info...
• OWS-6 WPS Grid Processing Profile Engineering Report– http://portal.opengeospatial.org/files/?artifact_id=3497
7
• OWS-6 interactive demonstration– http://www.opengeospatial.org/pub/www/ows6/index.h
tml
• WPS Change Request for controlling & checking asynch processes– https://portal.opengeospatial.org/files/?artifact_id=367
32