Grid Workflow Research update in SIG

Dr. ZhenChun Huang Tsinghua Univ.

NRSCC/RSGS/SIG TeamSep. 8th, 2004

Grid, Application Grid and SIG

• Grid– connects distributed resources by network– co-operates resources by standard protocols– creates virtual organizations dynamically and resolves application

problems by the support of grid software.

• Application Grid– grid for a special application area– based on the general grid technology– provides many features to fit the application area additionally

• Spatial Information Grid– Application Grid in Spatial Information Application area– provides features for the Spatial Information Applications

Grid Architecture

Grid Infrastructure

Resources as Services

Applications

ToolsGrid Middleware

Changes in infrastructure

Application for the area

special resources (services)

Tools for the areaThe

application grid

Workflow

Object of grid workflow

• Regard a grid application as a workflow based on resources in grid

• Provide a easy-to-use environment to develop, deploy and execute complex grid applications

• 3R –––– Object of workflow• Choose the Right resource(person or machine)• Do the Right thing• At the Right time

• Grid WorkFlow Language, shortly GWFL• Based on the open-source workflow engine osworkflow 2.6• Its language is a bit different from that designed by WfMC.• Uses finite automaton model

Roadmap

• First half year of 2002– No workflow support in our grid system– Develop application by Java/JSP directly

• Last half year of 2002—first half year of 2003– Introduce BPEL to our grid system and develop demo applicatio

ns based on it– (thank for He Chuan and Gu Lei, they led us to use workflow in g

rid)

• 2003 last half year — now – Give up BPEL and adopt GWFL based on OSWorkflow 2.6 inste

ad– And develop some down-to-earth applications

Why GWFL instead

• BPEL • GWFLCan Invoke Web Service only

Invoke almost anything by “plug-in”

No user/permission Provides interfaces for user/permission

Very difficult to interact with user in workflow

Interact with user by “view” in workflow

No source code and copyright reserved

Open-source and can be modified freely (OSWorkflow)

Workflow Definition of GWFL

A workflow

Startup step

Finish step (optionally)

Step 2Step …

Step …

Step 1

• A workflow has a “startup” step, a “finish” step (optionally) and several “internal” steps• A step has several “status”, a step and its current status make up of a status of finite

automaton

“Step” in GWFL• Each step has several actions, and a action can perform a

state-transition between steps.

A Step

ActionAction

ActionAction

Other Step

Other Step

Other Step

Other Step

Other Step

Other Step

“Action” in GWFL• An action includes:

– 1 view(usually Java Server Page) to show for the user– 1 restricted condition to restrict availability of this action, and it can

be a complex condition linked by AND, OR or NOT operator– 0..n validators to validate the input data– 0..n pre-function & post-function to perform before or after the acti

on– 0..n results to transit the status of finite automaton. One of the resu

lts should be “unconditional result”, and the others are “conditional results” with conditions

– Or Split into several status, or join several status into one.– Results can have pre-function or post-function, too– All the components except view should be “plug-ins”, they may be:

• Java class, BeanShell, and BSF(support perlscript、 vbscript、 javascript)

An Action• An action includes:

An Action

View

Restricted Condition

Validators

Post-function

Conditional Results &

Un-Conditional Result

Split

Join

Pre-function

Condition Plug-in

Validator Plug-in

Function Plug-in

Function Plug-in

Condition Plug-ins

Function Plug-ins

A Sample workflow

• A sample workflow with 4 internal steps, a finish step, and a hidden startup step

The hidden startup step

Internal steps

The finish step

Actions in a step

• A Step can have several actions (in the sample, it has 8 actions)

An Action

• An action with restrict-to conditions, pre-functions and 2 results.

View of action

Restrict-to condition

• A complex restrict-to condition to restrict when the action is available

• Here, the restrict-to condition is: the status is “Underway” and the user is owner of this workflow

• The “condition” tag indicates a condition plug-in, it is a Java class here

Pre-functions and post-functions

• pre-functions or post-functions plug-in can perform some operation before or after the state-transition

The function plug-in is Java class: org.caf.sig.workflow.function.Registry.SearchAnd it needs 2 arguments named “serviceURL” and “type”

Results and state-transition

• This action has 2 results: a conditional result and a un-conditional result

Result tag indicates a conditional result, its condition is a simple condition which plug-in is written in JavaAn unconditional-result tag can be regarded as a “default” result The next step

State-transition

Develop function plug-in by Java

• A function plug-in class in java should implement the interface org.thgrid.gwfl.workflow.FunctionProvider and do any thing you want to do in function execute()

• The following is a example

public class NewFunction implements FunctionProvider { public void execute(Map transientVars, Map args, PropertySet ps) throws WorkflowException { WorkflowContext context = (WorkflowContext) transientVars.get("context"); String argName = ParamUtil.getValue(transientVars, args, "name");

ps.setString("error", "ok"); }}

Develop condition plug-in by Java

• A condition plug-in class in java should implement the interface org.thgrid.gwfl.workflow.Condition and return whether condition is true in function execute()

• The following is a example

public class CustomSelect implements Condition { public boolean passesCondition(Map transientVars, Map args, PropertySet ps) throws WorkflowException { WorkflowContext context = (WorkflowContext) transientVars.get("context"); String argError = ParamUtil.getValue(transientVars, args, “error");

return “ok".equalsIgnoreCase(argError ); }}

View of action

• A view of action is a special JSP fragment

<%@ page contentType="text/html; charset=gb2312" %><%@ taglib prefix="c" uri="http://java.sun.com/jstl/core" %><%@ taglib uri="/WEB-INF/tld/gwfl-propertySet.tld" prefix="gwfl" %>

<form action="workflow.do"> <input type="hidden" name="action" value="formsubmit"> <input type="hidden" name="id" value="<c:out value='${param.id}'/>"> <input type="hidden" name="do" value="<c:out value='${param.do}'/>">

<gwfl:getString entryId="${param.id}" property="statusString" /> <gwfl:getObject entryId="${param.id}" var="retList" property="retList“/>

<input type=“submit” value=“Next Step”></form>

These codes must be included for the execution of workflowBy gwfl custom tags, properties in PropertySet can be accessed

Update in SIG

• Workflow in SIG is the same as it in general grid essentially

• Because the SIG workflow is used for EO information application, there should be some additional features and functions in SIG workflow. And these should be our work– Develop plug-in library for Spatial Information applications in SIG– Make up a “workflow library” framework so that typical workflows

can be stored for SIG– More powerful toolkits for developing, deploying and executing

workflows in SIG– And so on

Plug-in library

• Plug-ins for the functions(pre- & post-), conditions and validators often be used,– Typical functions such as RS image processing, GIS da

ta processing, RS & GIS data discovery, etc.– Typical conditions such as RS meta-data conditions, R

S data & GIS data feature conditions, etc.– Typical validators such as RS meta-data & image valida

tors, GIS data validators (for format, area, precision, etc)

Workflow library & toolkits

• Workflow library framework– Workflow design & test– Workflow register & discovery– User management & authorization– Co-operate with other parts of SIG, such as user

interface, resource registry, etc.• Some typical workflow to put in the library• Workflow toolkits should include

– Workflow editor with a friendly UI (beta version available )

– Workflow debugger to debug the workflow (on the road)– And so on

Applications

• Used in the 863 project: “Application Grid on Forest Resources and Forestry Ecology” (or often shortly, National Digital Forestry Grid, DFG)– We develop almost all the applications in this project

by GWFL– The example we showed is one of these

applications, too

• And it will be open-source (under GPL or LGPL), too– We are arranging source codes for release

Thank you