containers and web services grid middleware 7 david groep, lecture series 2005-2006
Post on 19-Dec-2015
217 views
TRANSCRIPT
Grid Middleware VII 2
Outline
Why web services? Web services revisited
SOAP, WS-I basic profile, encoding styles Envelope versus body
Containers why containers pluggable frameworks (e.g. the GT4 security stuff) some leaky abstractions: file ACLs in the (Globus) RLS
Implementing grid middleware where to put the abstractions (crunching paper stuff) is stuff in the envelope really standard? syntax vs. semantics of the interface
Performance: Java, C, Python, .NET
Grid Middleware VII 3
Why web services came about in grid
The definition of WSRF means that the Grid and Web services communities can move forward on a common base.
Grid Middleware VII 4
Web Services
WS are just one implementation of a SOA we could have done ASN.1 over BEEP over avian carriers
(which might even have been faster ) but we don’t, to help interoperation
Web services are basically RPC using XML largely self-describing
Grid Middleware VII 5
Web Services hints
Web Services are just a syntax
Define proper sementics and document that to make your service re-useable
Match up the semantics with related services
Make sure your services are compliant with relevant standards (like WS-I Basic Profile) with the chosen hosting system (GT4)
Grid Middleware VII 7
Types of WSDL SOAP binding
RPC/encoded
RPC/literal
Document/encoded
Document/literal
Grid Middleware VII 9
RPC/Encoded WSDL
<message name=“myMethodRequest”><part name=“x” type=“xsd:int”/>
</message>
<message name=“empty”/>
<portType name=“PT”><operation name=“myMethod”>
<input message=“myMethodRequest”/><output message=“empty”/>
</operation></portType>
Binding is RPC/encoded
Grid Middleware VII 10
SOAP message
<soap:envelope>
<soap:body>
<myMethod>
<x xsi:type=“xsd:int”>value</x>
</myMethod>
</soap:body>
</soap:envelope>
Grid Middleware VII 11
Advantages/disadvantages
Advantages Simple WSDL Operation name appears in the message
Disadvantages Type encoding information overhead SOAP message cannot be validated except against WSDL
Grid Middleware VII 12
RPC/Literal WSDL
<message name=“myMethodRequest”>
<part name=“x” type=“xsd:int”/>
</message>
<message name=“empty”/>
<portType name=“PT”>
<operation name=“myMethod”>
<input message=“myMethodRequest”/>
<output message=“empty”/>
</operation>
</portType>Binding is RPC/literal
Grid Middleware VII 13
SOAP message
<soap:envelope>
<soap:body>
<myMethod>
<x>value</x>
</myMethod>
</soap:body>
</soap:envelope>
Grid Middleware VII 14
Advantages/Disadvantages
Advantages WSDL is simple Operation name appears in the message Type encoding information is minimal
Disadvantages Nearly all the definitions in WSDL so not independently
validatable
Grid Middleware VII 16
Document/literal WSDL
<types><schema>
<element name=“xElement” type=“xsd:int”/></schema>
</types>
<message name=“myMethodRequest”><part name=“x” element=“xElement”/>
</message><message name=“empty”/>
<portType name=“PT”><operation name=“myMethod”>
<input message=“myMethodRequest”/><output message=“empty”/>
</operation></portType>
Grid Middleware VII 17
SOAP message
<soap:envelope><soap:body>
<xElement>value</xElement></soap:body>
</soap:envelope>
Grid Middleware VII 18
Advantages/Disadvantages
Advantages No type encoding information The body of the soap message is all defined in a schema and
so can be validated independently
Disadvantages WSDL is more complicated Operation name is lost
Grid Middleware VII 19
Document/wrapped WSDL
<types><schema>
<element name=“myMethod”/><complexType>
<sequence><element name=“x” type=“xsd:int”/>
</sequence></complexType>
</element></schema>
<types><message name=“myMethodRequest”>
<part name=“parameters” element=“myMethod”/></message><message name=“empty”/>
<portType name=“PT”><operation name=“myMethod”>
<input message=“myMethodRequest”/><output message=“empty”/>
</operation></portType>
WSDL schema has a wrapper around the parameters
Grid Middleware VII 20
SOAP message
<soap:envelope>
<soap:body>
<myMethod>
<x>value<x>
</myMethod>
</soap:body>
</soap:envelope>
Grid Middleware VII 21
Characteristics
Input message has a single part Part is an element Element has the same name as the operation Element’s complex type has no attributes
Grid Middleware VII 22
Advantages/disadvantages
Advantages No type encoding information Soap body is defined in a schema – validation Method name in the soap message
Disadvantages WSDL is complicated
Generally this is the best style to use.
Grid Middleware VII 23
When not to use document/wrapped
Document literal wrapped style does not allow for overloading
Cannot have two elements with the same name in XML (element has to have same name as operation)
In this case you may wish to use RPC/literal so that the operation name is available.
Grid Middleware VII 24
SOA Editor
Cape Clear SOA Editor, Cape Clear 2003WSDL © EGEE Consortium and partners, see www.gLite.org
Grid Middleware VII 26
Container
A framework to accept, interpret, (de-)marshall the I/O for your application or service
Grid Middleware VII 27
Container
Various vendors Oracle Application Server SUN Application Server IBM WebSphere Apache Catalina/Tomcat Apache Axis .NET
although the WS on the outside may be the same, tools and interface on the business end vary
Grid Middleware VII 28
Containers
Many containers, but not all the same
Vary in what they support, e.g., Platform: JVM, .NET, Unix Language (Java, C, Python) WSDL bindings (rpc or document/literal) Security mechanisms (none or GSI)
VL-e supplied container GT4 (Axis 2.0-RC2++) Modified to do WS-RF, -Addressing, -Notification With GSI Security framework Almost like standard Axis 2
Grid Middleware VII 29
Data structures
All data structures in the language (Java, C, &c) need to be serialized
Easy when you stick to:Global types (int, double, String, array etc.) Java beans (set and get methods)
Otherwise need write custom serializers commands like WSDL2Java generate (amonst other things)
stubs to fill with the serialization code
slide source: Machiel Jansen, Vrije Universiteit Amsterdam: VL-e web services induction 2006
Grid Middleware VII 30
Wrapping existing application
Define input and output of the application write the appropriate WSDL
Generate the Java stubs Execute application from Java Redirect the output Deploy as webservice
slide source: Machiel Jansen, Vrije Universiteit Amsterdam: VL-e web services induction 2006
Grid Middleware VII 31
Wrapping existing application
public int run(String cmd, Writer out) throws IOException {RunTime r = Runtime.getRuntime(); Process p = r.exec(cmd); FileIO.copyFile(new InputStreamReader(p.getInputStream( )),
out, true);try { p.waitFor( ); // wait for process to complete } catch (InterruptedException e) { return -1; } return p.exitValue( ); }
slide source: Machiel Jansen, Vrije Universiteit Amsterdam: VL-e web services induction 2006
Grid Middleware VII 32
WS Clients – 3 types
Stub based (easy: use WSDL2Java) Service endpoint is static
Dynamic proxy Service endpoint dynamically at runtimeDynamic Invocation Interface (DII) Discover services at runtime then invoke methods
slide source: Machiel Jansen, Vrije Universiteit Amsterdam: VL-e web services induction 2006
Grid Middleware VII 33
Clients - DII
Axis user guide uses Call object (DII) in examples.WSRF and GT4 examples uses stubs.
slide source: Machiel Jansen, Vrije Universiteit Amsterdam: VL-e web services induction 2006
Grid Middleware VII 34
WSRF and GT4
GT4 uses the WSCore library.http://www-128.ibm.com/developerworks/edu/gr-dw-gr-eclipseide-i.html
Notes: GT4 WSRF is a little different from Apache WSRFGT4 uses GAR files and JNDI deployment.Eclipse has a GT4 plugin…
slide source: Machiel Jansen, Vrije Universiteit Amsterdam: VL-e web services induction 2006
Grid Middleware VII 35
Moving into the container
GT4 puts most of the grid elements in the container support for resource properties support for the AuthZ framework information system/MDS4 …
makes application service development easier if the service is independent of those features otherwise, lots of callouts are needed
as in the RLS security interface from lecture #6
Grid Middleware VII 36
Container capabilities
Axis 1.x, x<2 support RPC/encoded deployment of services requires restart
Axis 1.3 support Document/literal still no hot deployment
GT4 Like Axis 1.3, but with grid services built-in, WS-RF support
Tomcat5, Axis2 hot deployment of services WS-RF, WS-Notification, WS-Addressing in the pipeline somewhere
Sun & Oracle Application Server Hot deployment supported
Grid Middleware VII 37
Implementation
Remember that web services are stateless
Model state via Resources Don’t keep running after the service invocation is complete Use database/memory based storage, not process-based state
retention Much more friendly on the CPU Resilient to machine/VM restarts
The ultimate horror scenario: R-GMA producer & consumer threads, see Concurrency P&E paper on the web
Grid Middleware VII 39
Container deployment issues
Container needs inbound connectivity to get to the services most sites & compute clusters provide only outbound
connectivity how to get to the service?
proxy box inside the network perimeter needs specific site support opens a security hole in the perimeter
a better protocol unlikely that beep will ever replace http …
tunnel layer on top of protocols like, e.g., jabber, and push messages through
that with a proxy box
overlay network build a ‘quasi-VPN’ for your grid, e.g. using the GCB
Grid Middleware VII 41
Service Deployment Example
Farm of GT4 containers provided on the edge of the site Based on the concept of “VO Boxes” For now: login to these boxes via gsissh Not for compute-intensive work
You cannot run a container continuously on worker nodes No inbound IP connectivity Resource management and utilization
And you should not want to, because All services in a container content for CPU and (disk) bandwidth JVM does not insulate services from each other
Grid Middleware VII 42
Other constraints
Worker nodes are allocated in a transient way
Jobs run in a one-off scratch directory No shared file system for $HOME
(and if it happens to be there don’t use it!) Jobs will be queued
Short jobs get to run faster Small jobs get to run faster Priority depends on your VO affiliation
You can connect out, but cannot listen Your job is limited in wall time
Grid Middleware VII 43
Central Hosting
A hosting environment can offered on a set of edge boxes Visible from the outside world Highly monitored systems for security reasons Can talk to the ‘inside’
Requirements for deployment on the CF Appropriate AuthN/AuthZ must be used for all services
(this is available by default in GT4) Allow for request tracability (e.g. via log files) then, these services are allowed to talk to the ‘inside’
Grid Middleware VII 45
Some general caveats
Re-use services Use generic services where possible
‘job submission service’, ‘data location srv’ Don’t over-do it
WS are not suitable for low-latency HPC or bulk file transfer We’re not running a course on WS-RF …
Not all services (nor architectures) are mature jet Be prepared to make compromises … … as long as it’s in-line with long-term trend
Grid Middleware VII 46
Brokered Job Submission Service Submit brokered jobs in JDL
through WS-interface to the Matrix Resource Broker WSDL specification
http://mu7.matrix.sara.nl/SaraGridService/SaraGridServiceLCG.wsdl
For detailed info: ask Bart Heupers, SARA Future version of the basic Grid middleware will have a
native WMS WS interface
Grid Middleware VII 47
Defining and building Services in WS-*
[See Introduction to Web Services Tutorial]
1. Start with the WSDL2. Then, generate your interfaces
• E.g. with GT4 tools or JWSDP• See appropriate tutorial(s): “Technologies for Building Grids” or
“Sotomajor”
3. Fill the stubs with an implementation4. Deploy your services in a container
EGEE NA3 Training Home has plenty of material:http://www.egee.nesc.ac.uk/trgmat/