![Page 1: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/1.jpg)
www.neresc.ac.ukwww.neresc.ac.uk
Dynasoar Dynamic Deployment of Web Services
on a Grid or the Internetor
Why it’s good to be Jobless
Dynasoar Dynamic Deployment of Web Services
on a Grid or the Internetor
Why it’s good to be Jobless
Paul Watson & Chris FowlerSchool of Computing Science
University of Newcastle
Paul Watson & Chris FowlerSchool of Computing Science
University of Newcastle
The Dynasoar team: Chris Fowler, Paul Watson, Charles Kubicek, Arijit Mukherjee, John Colquhoun, Savas Parastatidis, Mark HewittThe GridShed team: Isi Mitrani, Jennie Palmer, Rob Smith, Paul McKee (BT) & Mike Fisher (BT)
![Page 2: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/2.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 2
Why Jobs & Services?
• Grid applications are being built from Web Services• If the computational requirements can’t be met by the
service hosting environment then a job must be created
• Do we need both jobs and services?
• Dynasoar• a service-only approach to building grid applications• an infrastructure for the dynamic deployment of web services
![Page 3: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/3.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 3
Web Services
C WS
req
res
Web ServiceConsumer
![Page 4: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/4.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 4
Dynasoar Components
• Web Service Provider (WSP)• exposes service endpoints• accepts the incoming SOAP message sent to the endpoint• chooses a Host Provider and passes the message to it• holds a copy of service code
• Host Provider (HP)• manages computational resources (e.g. a cluster or a grid)• accepts the message from the WSP• dynamically deploys the service if necessary• processes the message and returns any response
Consumer
C WSP HP
req
res
req
res
![Page 5: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/5.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 5
Routing to an Existing Service Deployment
C WSP
req
res
Host Provider
node 1s2, s5
…
node 2
node ns2
Web Service Provider
Consumer
A request for s2 is routed to an existing
deployment of the service
![Page 6: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/6.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 6
Dynamic service deployment
C WSP
req
res
1
Host Provider
node 1s2, s5
…
node 2
node ns2
Web Service Provider
3
2: service fetch &deploy
Consumer
R
The deployed service remains in place andcan be re-used - unlike job scheduling
A request to s4 cannot be met by an existing deployment of the service
![Page 7: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/7.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 7
Dynasoar Advantages
• Simplicity: just services• Efficiency: a deployed service can process many
messages• important if cost of deployment is high… e.g. VMs
• Support a range of new e-science/ e-business models:• defining the interactions between the major components
allows them to be distributed in a variety of ways
![Page 8: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/8.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 8
Dynamic Outsourcing
• Biocorp are experts in writing bioinformatics services• They don’t want to manage their own compute resources• Therefore, they use Hosting Inc to process messages sent to
their services• In e-science, BioCorp could be a research group writing specialist
e-science services, and Hosting Inc the NGS
HPWSP
C
C BioCorp Hosting Inc
…..
BioCorpCustomers
![Page 9: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/9.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 9
The National Grid Service as a Host Provider
• A researcher writes their own services but does not have sufficient local compute resources
• They deploy a local WSP, and configure it so that it sends messages to the National Grid Service• their services are then transparently deployed on the NGS as
required
C WSP
National Grid
node …... node
Researcher’sLocal Resources
..node node
Quarantine Nodes
![Page 10: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/10.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 10
A Marketplace for e-Science
HP1WSP1
Marketplace
HPnWSPn
......
......
Web Service Providers
Host Providers
LocalCampusGrid
NationalGridService
![Page 11: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/11.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 11
Moving Computation to Data
• In many e-science applications analysis services operate on data extracted from a data store (e.g. OGSA-DAI, SRB…)• often large amounts of data are transferred• this may severely limit the performance
C
req
resAnalysis Service
DatabaseService
req
res
![Page 12: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/12.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 12
Moving Computation to Data
• The data owner provides compute resources close to a database• Researchers can write services and deploy them on their own
WSP• The service is dynamically deployed close to the database when
requests are sent to the WSP
C WSP
req
res
1
Host Provider
node 1
…
node 2
node n
Web Service Provider
offering the Analysis Service
3
2: service fetch &deploy
Consumer
Database Service
High PerformanceNetwork
![Page 13: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/13.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 13
Results for Deploying a Service Close to a Database
![Page 14: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/14.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 14
Tripartite Security Model
C HP
WSP
TRIPARTITETRUSTDOMAIN
The 3 actors can define policies (XACML) that Dynasoar enforces at run-time, e.g….
HP: accept only messagesfrom WSPs trusted to notsend malicious code
WSP: only use Host Providerstrusted to not re-use thedeployed service without payment
C: only send the message to a HP trustednot to look at the contents
![Page 15: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/15.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 15
Service Provider
QoS Policies
Deployed Service Registry
Web ServiceCode Store
Host Provider
Code Store SOAP interface
Security Policies
SOAP Endpoints
Message Handler
Computational resourcesHosting deployed services
Host Provider Registry
DynasoarHost Provider
Interface
Host Provider Endpoint
Service ProviderEndpoint
Current Implementation
GridShed ClusterManagement
![Page 16: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/16.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 16
New Host Provider Architecture
• Layer as high-level infrastructure over lower level grid fabric• Use OMII Job Submission and Monitoring Service to provide
stable interface to different underlying fabrics• Newcastle Grid (Condor), National Grid Service, local clusters,….
Host Provider
Grid Fabric
DynasoarHost Provider
Interface
OMII Job Submission and Monitoring Service
(Grid-SAM)
![Page 17: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/17.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 17
Current Work
• Deploying Services in Virtual Machines• can encapsulate a complex service implementation
environment
• Use of QoS to enhance decisions on where to deploy a service
• Dynamic database deployment• ogsa-dai, ogsa-dqp
![Page 18: Paul Watson & Chris Fowler School of Computing Science University of Newcastle](https://reader035.vdocuments.us/reader035/viewer/2022062407/56812ca7550346895d914e5f/html5/thumbnails/18.jpg)
www.neresc.ac.ukwww.neresc.ac.uk 18
Conclusions
• Grid applications can be built entirely from services• jobless grid computing• simpler conceptual model• performance improvements due to sharing the cost of service
deployment over multiple requests
• Dynasoar is built as a high-level infrastructure on top of existing grid fabrics
• Separating the Web Service Provider from the Host Provider – with a well-defined interface – opens up a range of e-science/ e-business models
• Demo available at the North East Regional e-Science Centre stand