nec, tokyo, japan, may 2003 use of workflow techniques for grid management junwei cao ( 曹军威 )...
TRANSCRIPT
NEC, Tokyo, Japan, May 2003
Use of Workflow Techniques for Grid Management
Junwei Cao (曹军威 )C&C Research Laboratories
NEC Europe Ltd.Germany
NEC, Tokyo, Japan, May 2003
CCRLE
Bonn, Germany ~20 research staffs 3 working teams:
Numerical simulation MPI development Grid Computing
GEMSS
NEC, Tokyo, Japan, May 2003
My Previous Experience PACE: performance prediction of
parallel and distributed systems Titan: prediction based job
scheduling on clusters and multiprocessors
ARMS: agent-based resource management for grid computing
GridFlow: workflow management for grid computing (CCGrid 2003)
NEC, Tokyo, Japan, May 2003
Outline
Background GEMSS objectives Medical simulation applications Service performance prediction Workflow simulation and
scheduling Dynamic service federation Summary
NEC, Tokyo, Japan, May 2003
Grid Workflow Management
Workflow DefinitionWPDL, BPEL4WS, GSFL, ASCI Grid Workflow SystemsWebFlow, Symphony, GridAnt, BPWS4J,
TENT Component-based SystemsCCA/XCAT, SCIRun, CXML Other SystemsCondor DAGMan, UNICORE, USC Grid
failure handling
NEC, Tokyo, Japan, May 2003
GEMSS Objectives Demonstrate that the grid can improve pre-operative planning &
near- real-time surgical support by providing access to advanced simulation and image-processing services.
Build middleware on existing or developing grid technology standards to provide support for authorization, workflow, security and Quality of Service aspects.
Develop, evaluate and validate a test-bed for the GEMSS system, including its deployment in the end-user’s working environment.
Anticipate privacy, security and other legal concerns by examining and incorporating into its grid services the latest laws and EU regulations related to providing medical services over the Internet.
NEC, Tokyo, Japan, May 2003
Medical Applications
Pre- surgery Maxillo-facial surgery Post- surgery
Medical simulation supports the optimization of operation procedures and the planning of therapeutic strategies.
NEC, Tokyo, Japan, May 2003
Image Pre-processing
An intensity based algorithm (adaptive fuzzy C-means algorithm) is used to provide good quality segmentations for structures of the human head.
CT Images Identification of substructures
NEC, Tokyo, Japan, May 2003
Numerical Modeling
Next to the image processing step follows the geometric modeling of the structures suitable for Finite Element simulations.
Mesh manipulation using the halo positioning tool
Mesh generation withOr without smoothing
NEC, Tokyo, Japan, May 2003
HPC Simulation & Visualization
The geometric face change Calculated deformation of the skull
NEC, Tokyo, Japan, May 2003
The Problem …
The application includes the use of a complete chain of tools necessary for the entire process from geometric model generation from scan data (segmentation, mesh generation and mesh manipulation) to computer simulation and visualisation.
NEC, Tokyo, Japan, May 2003
Application Workflow
Image Segmentation
Mesh Generation
Mesh Manipulation
Finite Element Simulation
Visualization
Computation intensive / Semi interactive
Computation intensive / parallel
Full interactive
Computation intensive / parallel
Data intensive / Full interactive
NEC, Tokyo, Japan, May 2003
Grid Enabling …
Interface definition of each module using the WSDL
Implementing each module as a web service using Apache Axis or a grid service using the GT3
Definition of the whole process using the BPEL4WS
Using a BPEL4WS engine for service invocation
Applied only to Computation intensive parts
NEC, Tokyo, Japan, May 2003
Challenges
Could you finish the process in 1 hours? – QoS and adaptation support is becoming the most active research topic in grid computing community.
Service performance prediction Workflow simulation and scheduling Dynamic service federation
NEC, Tokyo, Japan, May 2003
Grid Performance Services
Building performance services as high-level grid services based on OGSA core and base services:
Managing performance-related data Defining performance metrics Developing performance analysis
algorithms Developing new APIs for grid service
performance prediction
NEC, Tokyo, Japan, May 2003
Performance-related Data
Application parameters that have an impact on application performance
System status e.g. CPU load, job queue and network bandwidth
Managing performance-related data using OGSA service data support
NEC, Tokyo, Japan, May 2003
Performance Metrics & Analysis Algorithms
Performance metrics Execution time Memory usage Price, and more …Performance analysis – historic
information based Statistical analysis algorithms Self organizing mapping More …
NEC, Tokyo, Japan, May 2003
Workflow Simulation, Scheduling & Rescheduling
S2
startT = 0exeT = 3endT = 3
S1
startT = 0exeT = 0endT = 0
S3
startT = 0exeT = 5endT = 5
S4
startT = 5exeT = 7
endT = 12
S5
startT = 5exeT = 4endT = 9
S6
startT = 12exeT = 0
endT = 12
/ 7/ 12
S2
S1
S3
S4
S5
S6
/ 5/ 5
NEC, Tokyo, Japan, May 2003
Dynamic Service Federation
Using the BPEL4WS service references to select and assign actual partner service dynamically.
Extending BPEL4WS <partner> with some kind of <candidate> element to indicate candidates of a grid service partner.
NEC, Tokyo, Japan, May 2003
The Solution … Grid Service (WSDL)
Performance Services
Service data
Application portTypes
WSDL
WorkflowSimulation Engine
WorkflowExecution Engine
Grid Service (WSDL)
Performance Services
Service data
Application portTypes
UDDI
User
NEC, Tokyo, Japan, May 2003
Other Aspects Accuracy of performance prediction Workflow execution monitoring Security and legal issues Grid workflow GUIs Grid data management Enabling interactive applications Using workflow techniques for business
process management in GEMSS
NEC, Tokyo, Japan, May 2003
Summary Programming models for the grid –
workflow specification – a candidate? QoS support and application adaptability –
performance prediction + workflow simulation –> dynamic service federation – a solution?
Medical simulation applications – a right target application of the grid?
AgileGrid: agile computing on business grids – a next generation computing paradigm?