comp3019 coursework: introduction to gridsam steve crouch [email protected], stc@ecs school of...
TRANSCRIPT
![Page 1: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/1.jpg)
COMP3019 Coursework:Introduction to GridSAM
Steve Crouch
[email protected], stc@ecs
School of Electronics and Computer Science
![Page 2: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/2.jpg)
A Reminder…
For materials (inc. slides, software, materials, handout):– http://www.ecs.soton.ac.uk/~stc/
COMP3019/
Coursework deadline: March 27, 2014 at 4pm
![Page 3: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/3.jpg)
GridSAM
Job Submission and Monitoring
![Page 4: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/4.jpg)
GridSAM Overview
What is GridSAM?– A Job Submission and Monitoring Web
Service– Allows compute resources to be
accessible securely over the internet– Uses Job Submission Description
Language (JSDL) to describe requirements for jobs
![Page 5: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/5.jpg)
GridSAM Overview
What is GridSAM to the resource owners?– A Web Service to expose heterogeneous
execution resources uniformly Single machine through Forking or SSH Condor Portable Batch Scheduler (PBS) Load Sharing Facility (LSF) Sun Grid Engine Globus
– Acts as a client to these resources
![Page 6: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/6.jpg)
GridSAM Overview
What is GridSAM to end-users?– A set of end-user tools and client-side APIs
to interact with GridSAM Web Services Submit and start jobs Monitor jobs Terminate jobs File transfer Client-side submission scripting Client-side Java API
![Page 7: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/7.jpg)
GridSAM Server and Client Server comprised of:
– Web Services Container Tomcat/Axis - Web Services provider/container Optional WS-Security - handles authentication
(contains X509 certificate for security) Can be run over HTTP or HTTPS
– GridSAM Java web service
Client comprised of:– Client security ‘container’.
Contains Configuration for security (including X509 certificate security credentials)
Used by client applications to interact with services on OMII server
– GridSAM Java client
![Page 8: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/8.jpg)
Web Service Definition
“A service is the logical manifestation of some physical or logical resources (databases, programs, devices, humans, etc) and/or some application logic that is exposed to the network”
Service interaction is facilitated by message exchanges
![Page 9: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/9.jpg)
Web Services (WS) I
XML: Platform neutral mechanism to describe data
SOAP: Mechanism to describe message exchange
– Simple Object Access Protocol Not simple and nothing to do with Objects!
– Service Oriented Access Protocol Re-engineering of acronym to fit current use!
WSDL: Defines the service interface– Web Services Definition Language
![Page 10: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/10.jpg)
Web Services (WS) II
Services have to reside in a supporting environment:– Called: hosting environment or container
e.g. Axis on Tomcat
– Marshals requests into and response out of the service– Service can discover local configuration parameters– Provides a standard infrastructure for service developers
Processing incoming requests & outgoing responses– Called: Message handlers– Manipulates elements of the message header
Primarily the SOAP header
– Handlers can be applied to message traffic into or out of the whole container or a specific service
![Page 11: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/11.jpg)
Architecture
WS-Security
AXIS
TOMCAT
GridS
AM
…
Client
…
…
![Page 12: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/12.jpg)
GridSAM – Publications & Enabled Activities
![Page 13: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/13.jpg)
GridSAM Architecture A staged event-driven architecture
– Submission pipeline is constructed as a network of stages connected by event queues
– Each stage performs a specific action upon incoming events
Example Pipeline: Condor
![Page 14: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/14.jpg)
Scenario: Condor Pool
GSIFTPGSIFTPFTPFTP WEBDAVWEBDAV HTTPHTTP…
![Page 15: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/15.jpg)
Example using File Staging Objectives: submit simple job with
data input and output requirements and monitor progress
OMII GridSAM
Client
OMII GridSAM
Server
submit JSDL
monitor
OMII GridSAM
FTP Server1 output file
2 input files
![Page 16: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/16.jpg)
JSDL Example <omii_client_home>/gridsam/data/examples/remotecat-staging.jsdl:
<JobDescription>
<JobIdentification> … </JobIdentification>
<Application>
<POSIXApplication xmlns="http://schemas.ggf.org/jsdl/2005/06/jsdl-posix">
<Executable>bin/concat</Executable>
<Argument>dir2/subdir1/file2.txt</Argument>
<Output>stdout.txt</Output>
<Error>stderr.txt</Error>
<Environment name="FIRST_INPUT">dir1/file1.txt</Environment>
</POSIXApplication>
</Application>
…
![Page 17: COMP3019 Coursework: Introduction to GridSAM Steve Crouch s.crouch@software.ac.uk, stc@ecs School of Electronics and Computer Science](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649eb15503460f94bb6a34/html5/thumbnails/17.jpg)
JSDL Example<DataStaging>
<FileName>bin/concat</FileName>
<CreationFlag>overwrite</CreationFlag>
<Source>
<URI>ftp://ftp.do:55521/concat.sh</URI>
</Source>
</DataStaging>
<DataStaging>
<FileName>dir1/file1.txt</FileName>
<CreationFlag>overwrite</CreationFlag>
<Source>
<URI>ftp://ftp.do:55521/input1.txt</URI>
</Source>
</DataStaging>
<DataStaging>
<FileName>dir2/subdir1/file2.txt</FileName>
<CreationFlag>overwrite</CreationFlag> <Source> <URI>ftp://ftp.do:55521/input2.txt</URI> </Source></DataStaging><DataStaging> <FileName>stdout.txt</FileName> <CreationFlag>overwrite</CreationFlag> <DeleteOnTermination>true</
DeleteOnTermination> <Target> <URI>ftp://ftp.do:55521/stdout.txt</URI> </Target></DataStaging></JobDescription></JobDefinition>