workﬂow automation: applications, technology and research

Workflow Automation:Applications, Technology and Research

Tutorial notes, SIGMOD Conference, May 1995, California.

c 1995, Amit P. Sheth.

Prof. Amit ShethLarge Scale Distributed Information Systems Lab

Dept. of Comp. Science, The Univ. of Georgia415 Graduate Studies Research Center

Athens GA 30602-7404 USA

Tel. +1 706-542-2310, Fax: -2966Email: [email protected]

http://www.cs.uga.edu/LSDIS/

LSDIS

Collaborations/Acknowledgments: Bellcore (N. Krishnakumar,.. ),U. of Houston (M. Rusinkiewicz, ..), GTE (D. Georgakopoulos)

MCC (M. Singh,..), ETH-Zurich, LSDIS/UGA (J. Miller, K. Kochut), Clients: Ameritech, CHREF

Future revisions of this presentation can be found at: http://www.cs.uga.edu/LSDIS/

SIGMOD95Multidatabase Interoperation

C 1995, Amit P. Sheth

LSDIS

1

PreliminariesThis talk will emphasize Workflow Management for Mission Criticaland Enterprise-wide Applications involving heterogeneous informa-tion systems....

We will only look at some of the issues under the “WorkflowUmbrella” [Georgakopoulos, Hornick, Sheth 95].

Workflow

Business Processspecification/mapBusiness Processre-engineering

Workflow automationWorkflow implementationWorkflow specification

Business Processesautomation

Business Process

Workflow managementWorkflow management system

Note: Trademarks are those of respective owners. ActionWorkflow: Action Tecnologies; Encina: Transarc Corp., FloMark: IBM; InCon-cert: X Soft; Lotes Notes: Lotus Corp.; ProcessIT: Digital Equipment Corp., SAP Business SQL Server: Microsoft; Workflow: SAP AG;WorkFlo: Filenet. Any missing reference to trademark is unintentional.



LSDIS

2

Overview

• What’s workflow?Origin, Advantages, and Example Applications

• Review of Commercial State-of-the-Art,Markets and Related Technologies

• Basic concepts and specification of workflows• Components of a Workflow Management System• Research: past and in progress• Trends



LSDIS

3

Origins of WorkflowTen years ago, a team of engineers conceived the idea that computersoftware could be used to automate paper-driven business processes.They called it “workflow software.” [Smith 93]

imagingdocument flow

enhanced emailsworkgroup support

multi-system apps.



LSDIS

4

Definition of Workflow (Management)(some samples)

“Workflow refers to a new set of software and tools for automatingand improving business processes.” [Dyson 1992]“Workflow is a process by which individual tasks come together tocomplete a “transaction” -- a clearly defined business process --within an enterprise.” [Silver in [2]]“Workflow is the sequence of actions or steps used in businessprocess. Automated workflow applies technology to process, thoughnot necessary to every action.” [Marshak in [2]]“Workflows are computerized models of business processes...”[Hollingsworth 94]“Workflow management is an important tool for structuring andoptimizing business processing... and for supporting the practicalimplementation of business process re-engineering.” [Fritz 95]



LSDIS

5

The lure of workflow:it fits the trend

Workflow fits nicely with other trends such asre-engineering, downsizing,

network computing, groupware, andclient-server computing.



LSDIS

6

The lure of workflow: a large potential marketMarket/Revenue Forecast for

Workflow Software($-mil)

Sources: ID = IDC & Advante; DL = Delphi Consulting; IT = International Data Corp.

Year Workflow-All

ProductionWorkflow

“TransactionalWorkflow”

1992 226 (ID)186 (DL)

115 (DL)

1993 628 (ID) 250 441994 1200 (ID) 540 1061995 1800 (ID) 810 1841996 2500 (IDC)

2500 (DL)1120 (DL) 293

Forecast data used above is old (1993), but it shows an importantreason for vendors’ interest in this market.



LSDIS

7

Trade Press Characterization of Workflows

Administrative workflows involve repetitive, predictable processes with simple task coordination rules,such as routing an expense report or travel request through an authorization process.Ad-hoc workflows involve human coordination, collaboration, co-decision, and often appear in office pro-cesses such as product documentation or sales proposal.Production workflows involve repetitive and predictable business processes, such as loan applicationsor insurance claims. Unlike administrative workflow, production workflow encompasses an information processinvolving access to one or more distributed/heterogeneous/autonomous information systems.

Complex

SimpleSimple Complex

Task Structure

Task

Com

plex

ity

Insurance ClaimsLoan Applications

Product DocumentationSales Proposals

Press ReleasesExpenses

Travel Requests, Purchase RequestsMessages

Production

Ad Hoc

Administrative

Source: IDC/Avante Technologies, Inc.

Multi-systemApplications &TransactionalWorkflows

[McCready 92, Georgakopoulos, Hornick and Sheth 94]



LSDIS

8

Trade Press Characterization of Workflows (contd..)

Yet another classification:Mail-centric, document-centric and process-centric (see [Fry94]).

Application Integration Complexity

Org

aniz

atio

nal

Source: BSG Corp. (see [Als94])

Ad-hocWorkgroupTools

TaskAutomation

DocumentFlow

ProcessAutomation

Prod

uctiv

ityBe

nefit

s



LSDIS

9

Workflow (WF) Automation Software:Example Products

WorkFlo (Filenet)InConcert (XSoft)WorkMan (Reach)FloWare (Plexis)AWS (Action Tech)WorkManager (HP)LinkWorks, ProcessIT, ObjectFlow (DEC)SAP Business Workflow (SAP AG)FloMark (IBM Vienna)

Trademarks are those of respective owners.

About 200 products claim to support workflow featuresand/or workflow management!

See [2] and [Georgakopoulos, Hornick, Sheth 95] for a list of vendors and products.



LSDIS

10

Overview of Current Commercial State-of-the-Art• Emphasis on office processes:

– imaging– document flow– enhanced mail

• Reasonable support for administrative and ad-hocworkflows

• Many products are little more than fancy diagramming tools(Dataflow, Digraph, Flowchart, Network, Orgchart,Pertchart,...), with layout support, capture/import/export ofdata from/to databases, spreadsheets, simulation tools

• Some are specialized electronic data managementsystems: e-mail, imaging, databases, electronic forms, text,engineering drawing,..

• Alliances between image/document management, GUIbuilder and tools companies (e.g., simulation) are common



LSDIS

11

Maturing Infrastructure: A Driving Force

Communication Infrastructure usedby Commercial WorkflowManagement Systems

e-mailWork-group (Notes)Distributed Object ManagementCurrent Generation TransactionProcessing MonitorsAgents

Early 90s - already mature1993 - almost mature1995 - very active1996?

1997?



LSDIS

12

Workflow Management System forOffice Automation vs Enterprise Automation

(Typical Case)Current workflow/workgroup software supports officeautomation functions (involving user tasks) (most products arein PC and mainframe env.). There is little support for applicationautomation (involving both user and application tasks withvaried level of transactional properties).

User Task User TaskUser Task User Task Appl. TaskAppl. Task

workflows supported by workflows supported bymost current WMSs some new/emerging WMSs



LSDIS

13

More Demanding Workflowssome examples--

• Business loan processing in Banks requires coordinatinguser tasks such as loan application entry and risk exception,and application tasks such as risk evaluation, risk update, andloan decision recording.

• Patient health care support in a health care grouppractice requires coordinating user tasks such as patientregistration, doctor’s record review/update, lab work, andapplication tasks such as automated billing and statisticscompilation.

• Service provisioning in Telcos require the coordinatedexecution of heterogeneous tasks on heterogeneous systems:

• Product life-cycle management.

Support for most mission-critical multi-system enterprise applica-tions is lacking (Finance, Healthcare, Manufacturing,...)



LSDIS

14

Enter loanrequest

Client creditworthiness

Riskevaluation

Riskupdate

Risk updatecompensation

Riskexception

Recorddecision

[Breitbart, et al. 93: ETH]

Workflow may involve both user and application tasks, aswell as different types of application tasks. Some tasks canbe compensated.

An Example Application in Business DataProcessing (Loan Processing)



LSDIS

15

Patientregistration Provider

encounter

Insurancecompany

LabSpecialist

Billing

LabworkRequest

review

Workflow may involve multiple organizations, data of differenttypes/media, different notions of transactions (Domain specific[e.g., HL7 in Healthcare], EDI, DB), may take several days...

An Application (segment)in a Healthcare Environment

Reference Info.

internet



LSDIS

16

Customer

Central Office Central Office

Loop

Inter-Office/Trunk

L-D Carrier L-D Carrier

Inter-Office/Trunk

An Application in Telecommunication:Provisioning a Telephone Service



LSDIS

17

Use of legacy application/information systems is unavoidable.Migration to new systems and modern distributed computingenvironment continue.

ServiceOrder

Processor

Billing

LoopAssignment

Intra-Off.Assignment

TrunkAssignment

OperationsSupport

Work-ForceAdm.

Planning Design

Controller

Customer

CustomerRepresentative

OutsidePlantEngineer

1

2

3a 3b

4a 4b

5a

6

7

8

5b

Manual error resolutions

[Ansari et al 92,Aslo see: Georgakopoulos et al 93]

Provisioning a Telephone Service (contd..)



LSDIS

18

Business Challenges

• Improve flexibility for re-engineering• Increase automation to reduce cost and improve response time• Support evolution and migration (accommodate both existing

systems and new systems; increase use of the latter)

Example (Case Study of a Service Automation):• 40 persons -> 12 persons (work center

reorganization, re-engineering) -> 2-3 person(automation)

• Six Weeks -> Next Day -> Few Minutes



LSDIS

19

Some of the key new requirements• Automation involving heterogeneous software tasks/processes

and user tasks.• Integration with legacy systems and heterogeneous information

systems; support with-in enterprise and across enterprises(local, wide area, and wireless). Performance and scalability.

• Use of transaction concepts and technology, reliability, failurehandling/recovery.

• Support for domain-specific tasks (e.g., HL7 in Healthcare),objects (e.g., EDI) and repositories.

Task and info. sys. heterogeneity

Object/ Data complexity

Coordination/Correctness



LSDIS

20

Key Promising Technologies(and Challenges)

Distributed Object ManagementLegacy System InteroperabilityCustomized Transaction Management



LSDIS

21

Another Characterization

(that is sensitive to new requirements)

Human-oriented System-oriented

CSCW

Commercial WFM Systems

Transactional workflows

Commercial TP Systems

[Georgakopoulos, Hornick and Sheth 94]



LSDIS

22

On Transactional Workflows

Transactional workflows are activities that involve coordinatedexecution of multiple related tasks on distributed/heteroge-neous/autonomous information systems and support(provide) selective use of transaction properties at individualtask and (intra- and inter-) workflow levels.

In particular, they use transaction management concepts andtechnology for specifying and ensuring workflow correctnessand reliability in distributed/heterogeneous/autonomous infor-mation system environments.

More on this later...

[Georgakopoulos, Hornick and Sheth 94; Early use: Sheth and Rusinkiewicz 93. Also see Mohan et al 95]



LSDIS

23

(Distributed) Transaction Processing (DTP) vs.Advanced WFS

(Typical Case)DTP/D-OLTP is focused on efficient execution of relativelysimple tasks with no coordination across heterogeneous tasks(in different task groups). Advanced (transactional) workflowsrequire coordinated execution of heterogeneous tasks, withvaried levels of transactional properties, on a variety of systems.

Task Groups

Task

Task1Task2

Task3Task4

Workflow Group

In TWFIn DTP• Earlier queued message systems and “chaining of transactions”.

Problems: insufficient control over transaction properties, one type of task,interactions among concurrent activities difficult.

[STDL, Encina,...]



LSDIS

24[Rusinkiewicz and Sheth 94, Mohan tutorial; ETM vs Workflow Model-- see Breitbart et al 94]

Related Work:Database Literature

• ACID transactions and their nested derivativesProblems: inflexible, difficult to implement in multi-systems.

• Extended/Relaxed Transaction Models:Sagas and Nested Sagas [Garcia-Molina et al. 88, 90],ConTracts [Reuter89]. Flexible Transactions [Elmagarmid et al 90, Rusinkiewicz et al 90],Multi-transaction Activities [Garcia-Molina et al. 90], OpenNested Transactions [Weikum & Schek 92] and Others (e.g., in [Elmagarmid92]), ACTA framework [Chrysanthis & Ramamritham 91/92]

• “Workflow” and hybrid models:Long-Running Activities [Dayal et al. 91], DOM model/TSME[Buchmann et al 92, Georgokopoulos et al. 93], Third Generation TP Monitors [Dayalet al. 93], ASSET [Biliris et al. 94]



LSDIS

25

Workflow Management

WFM involves:• defining workflows, i.e., describing those aspects of

processes that are relevant to controlling andcoordinating the execution of its tasks (and possibly theskills of individuals or information systems required toperform each task), and

• providing for automation and re-engineering (fast(re)design and (re)implementation) of the processes asbusiness needs and information systems change.




LSDIS

26

Workflow Management Issues


ProcessWorkflow

SpecificationWorkflowImplementation

workflow

application=

• Business Process Modeling/Reengineering(BPM/R)

• Workflow model & specification language• Executable application code• Run time support



LSDIS

27

WMS Conceptual Architecture(system components)

BPM toolkit– process view

– organization view

– data view

– re-engineering analyzer– TQM advisor

– ...

WorkflowDevelopment toolkit

– graphical design tool

– developer’s workbench

– testing tool

– simulation tool

– ...

WMS run-timesystem and tools

– scheduler

– task managers/interfaces

– processing entities

– monitoring tool

– tracking tool

– reporting tool

– ...



LSDIS

28

Process Modeling Methodologies

There are many methodologies ...Communication-based methodologies stem from Winograd/Flores“Conversation for Action Model” [WF87]

1.preparation - a customer requests an action to be performed or a per-former offers to do some action

2.negotiation - both customer and performer agree on the action to be per-formed and define the terms of satisfaction

3.performance - the action is performed according to the terms established4.acceptance - the customer reports satisfaction (or dissatisfaction) with

the actionFigure 1.

preparation negotiation

performanceacceptance

Customer PerformerWorkflow Loop

[2], [Georgakopoulos, Hornick and Sheth 94]



LSDIS

29

Process Modeling Methodologies.. continued

Example of Communication-based Model:Modeling Materials Procurement Process

Procure Materials

Verify Status

Get Bids

Place Order

accountsoffice

vendors

vendor

investigator procurementoffice

[2], [Georgakopoulos, Hornick and Sheth 94]



LSDIS

30

Process Modeling Methodologies.. continued

Activity-based methodologies focus on modeling the work insteadof modelling the commitments among humans.

Example of Activity based process modeling:

Procure Materials

Verify Status Get Bids Place Ordertask nesting


Process modeling does not capturesignificant computational aspects



LSDIS

31

Process modeling to Workflow Implementation--an example (Action WorkFlow)

Process Modeling

Implementationon Lotus Notes

Implementationon SQL Server

Analyst

Application Builder

Manager



LSDIS

32

Workflow Modeling:The example of METEOR model

task

proc.

task

proc.

task

proc.

taskstart end

entity

entity

entity

interface

interfaceinterface

filter

Aux. Sys.

[Krishnakumar and Sheth 94]

METEOR: Managing End-To-End OpeRations



LSDIS

33

About the environment:Types of Tasks

• user tasks involving humans in processing task• application tasks:

• scripts involving terminal emulations to remote systems• predefined interfaces to legacy application systems (e.g.,

Bellcore “contracts”)• stored procedure calls• client programs or servers invoking other servers• database transactions

[Krishnakumar and Sheth 94]



LSDIS

34

About the environmentProcessing Entities

• humans (may appear as a GUI; may use document/imageprocessing systems and applications)

• script interpreters and compilers (for processing scripts andapplication programs)

• (legacy) application systems• servers in client-server and transaction processing systems• DBMSs

[Rusinkiewicz and Sheth 94, Krishnakumar and Sheth 94]



LSDIS

35

About the environmentTypes of Interfaces

• RPC and t-RPC mechanisms using transaction processingsystems

• queue managers• proprietary workstation to mainframe interfaces for

– “contracts”– terminal emulation

• (distributed computing/communication infrastructure: CORBA,DCE, Notes-like)

[Sheth and Krishnakumar 94]

Workflow management can be seen as a newdistributed computing paradigm...



LSDIS

36

Workflow Model and WMS wish list(Requirements and Features)

• Modeling heterogeneous tasks– task behavior/structure: externally visible states of the task,

initial state, termination states, significant events and theirattributes

– task inputs and outputs– task (operation) semantics, e.g., compatibility, relaxed

isolation• Modeling Interfaces and Processing Entities:

– type of interface/processing entity:communication infrastructur(s) and associated APIs

– interface/processing entity (system) properties/semantics--e.g., isolation granularity, order preservation, idempotency,monotonicity

[Krishnakumar and Sheth 94/95, Sheth and Rusinkiewicz 93/94]]



LSDIS

37


• Coordination and Inter-relationships: Routing, Rules, Policies,Practices, ..• Inter-task dependencies

– state-based– value-based: I/O objects and external variables

• Roles• Work-lists: Work-prioritizing, Dynamic Work Distribution• Data Management

– different task formats: message, contract, form, transaction;EDI;use of auxiliary systems for complex data manipulation

– different types of data: structured, text, image, voice,video,...

[Krishnakumar and Sheth 94/95, Sheth and Rusinkiewicz 93/94, Dayal and Shan 93]



LSDIS

38


• Dynamic Aspects– processing entity not known at design time– new tasks can be added dynamically– multiple concurrent invocation of the same task types– . . .

• Intra- and Inter-workflow Execution requirements:– failure atomicity (A)– execution atomicity (I)– workflow recovery– inter-workflow concurrency

[Krishnakumar and Sheth 94/95, Sheth and Rusinkiewicz 93/94, Dayal and Shan 93]



LSDIS

39


• support for long running workflows and tasks• Error Handling

– Systems Errors– Logical Errors

• Forward Recovery• Monitoring• Status tracking• Reporting



LSDIS

40

Some Technical Challenges

• Different types of tasks => homogeneous modeling; modelingexecution behavior

• Correct and Executable (hence well defined) specifications andcorrect and safe execution (including recovery)

• Heterogeneous processing environments and systems =>semantics of applications and processing entities, their impacton correct execution and performance

• Performance- many more messages/tasks in “application/operation automation” as compared to “office/user taskautomation”



LSDIS

41

Modeling Heterogeneous Tasks(Task Structures)

Different state transition diagrams for different types of tasksrepresenting what is observable and what is controllableby the WMS (i.e., can WMS enable that transition)

[Attie et al 93, Rusinkiewicz & Sheth 93, Sheth and Krishnakumar 94]]

Initial

Executing

DoneFailed

start

Initial

Executing

CommittedAborted

start

donefail abort commit

Initial

Executing

CommittedAborted

start

abortcommit

abort Done

Prepared

A non-transactional task A transactional taskAn open 2PC transactional task

done

prepared



LSDIS

42

Intertask DependenciesPreconditions for initiating each scheduler-controllable transitionin a task.Klein’s primitives [KL91]:• Order Dependency: e1 < e2.

If both e1 and e2 occur, then e1 precedes e2.• Existence Dependency: e1 -> e2.

If event e1 occurs sometimes, then event e2 also occurssometimes.

– Conditional Existence Dependency [KL91]: e1 -> (e2 -> e3)Examples from multidatabase transaction models:

• Commit Dependency [CR92]: cmB < cmA• Abort Dependency [CR92]: abB -> abA

[Attie et al 93, also see ACTA [Chrysanthis & Ramamritham 92], Georgakopoulos et al 94, Bilris et al 94,...]



LSDIS

43

Intertask DependenciesThere are many ways to model/specify dependencies, routing, pol-icies, etc.

Examples of the types of dependencies from database (extendedtransaction model) literature include, execution dependencies,data/value dependencies, temporal dependencies. For example,Flexible Transactions [Elmagarmid et al 90], ConTracts [Reuter 89],ACTA [Chrysanthis and Ramamritham 92], Multitransactions [Gar-cia-Molina et al 90], Multidatabase Transactions [Rusinkiewicz et al92, Mauro thesis 1993], DOM [Georgakopoulos et al 94]....

Rule-based (ECA) specification [Dayal et al., others] is popular,especially in research.



LSDIS

44

Simple Workflow Example

Initial

Executing

CompletedFailed

task AInitial

Executing

CompletedFailed

task B

Initial

Executing

CompletedFailed

task C(SD1,DD1)

(SD2,DD2)

(SD3,DD3)

Initial

Executing

CompletedFailed

task A

Compound Task



LSDIS

45

Another ExampleTask Graph

DELETEBOOKING

deletebooking

(dB)decrementsummary

(ds)updatealarm(u?a)

incrementsummary

(is)updatealarm(u?a)

&

Intertask Dependencies-><

->->

->s(dB) -> s(dS)c(ds) -> s(u?a)c(iS) -> s(u?a)(a(dB) & c(dS)) -> s(iS)(a(dB) < d(dS)) -> a(dS)

[Woelk et al 93]



LSDIS

46

CKT-Type_1

start

executing

doneabort

SO

AGG_task_1

SEGMENT-1

SEGMENT-2

SEGMENT-3

ADD_COMPONENTS NEW_ADD

ROUTE=YES

ROUTE=NONEW_SWITCH

“CHG_GR”

A somewhat complex example



LSDIS

47

Desirable Specification Language Features• definition of individual tasks (basic operation definition)• support for application as well as use tasks (incl. transactions

and nonelectronic operations)• state-based and value based intertask dependencies and data

management (control and data flow definitions)• failure and exception handling• business rules and constraints• security and role resolution

[Dayal and Shan 93 (terms in parentheses), Krishnakumar and Sheth 94,]



LSDIS

48

Several Approaches to Language Specification• Based on Extended Transaction Models (e.g., [ASSET [Biliris et al 94]),

based on rule-based specification,based on logic-based specification

• Extending Multidatabase Query Language (e.g., MSQL extensions [Rusinkiewicz

et al])

• Extending a script or “agent-based” language (e.g., ConTract/APRICOT

[Watcher & Reuter 92, Schwenkreis 93], work at MCC-Carnot/InfoSleuth)

• Extending general programming languages (e.g., as in Tranactional-C [see

Encina manuals], or IPL [Chen et al 93, Bukhres et al 95])

• Develop a Enterprise-wide Multi-system ApplicationDevelopment Environment (tasks in different languages,executed through/by different interfaces/processing entities) (e.g.,

Graphical Intefrace -> METEOR “intermediate-level” sub-languages)

• Add graphical interface and visual programming paradigm tothe above



LSDIS

49

An Approach to SpecificationMETEOR (Sub-)Languages

proc.

proc.proc.

end

entity

entityentity

interfaceinterface

Aux. Sys.

WFSL

tasktask

taskstartfilter

interface

task

TSL

[Sheth and Krishnakumar 94]

WFSL: WorkFlow Specification LanguageTSL: Task Specification Language



LSDIS

50

Components of WFSL (partial)• Task types: task structures, data input/output• Task classes, Task instances• Inter-task dependencies (logical error handling)• Data exchange statements• Filter (interface def.)

Components of TSL (partial)• processing entity specific statements• statements for revealing task structures• statements for identifying interfaces and dealing with systems

errors

[Krishnakumar and Sheth 94/95]

Languages can be targeted for end user programming ordevelopers. For example, WFSL and TSL are “intermediate”languages targeted to developers/administrators.



LSDIS

51

QMS Term.DBMS

Log

Workflow

Contract Script.UserTask

Open_2PCTran.

TP Sys.

Workflow Group

Log LogLog

Controller

Task Logs

Interfaces

Task Programs

LogTP Sys.

TP Sys. TP Sys. TP Sys.

Emulator

OSS OSS GUIDBMS

Resource Mgr.

Proc. Entities

Run-time Architecture(METEOR Approach)

[Krishnakumar and Sheth 94/95]

Workflow initiation,Workflow monitoring

USER

Work flowSpecificationandSimulation

ApplicationDeveloperandSystemAnalyst



LSDIS

52

Enforceable Dependencies• Dependencies may not be enforceable.

For example, ab(A) -> cm(B)

• Event attributes determine whether a dependency is enforceable.For example,– e1 -> e2 is run-time enforceable if

rejectable(e1) [delay e1 until e2 is submitted, reject e1 iftask 2 terminated without submitting e2],or forcible(e2) [force execution of e2 when e1 is acceptedfor execution].

– e1 < e2 is run-time enforceable ifrejectable(e1) [let e2 be executed when it is submitted,thereafter reject e1 if submitted],or delayable(e2) [delay e2 until either e1 has been acceptedfor execution, or task 1 has terminated without issuinge1].

[Attie et al 93]



LSDIS

53

Scheduling Approaches• Based on Petri-net Models [Elmagarmid et al 90]

• Executor for Flex. Trans. in a logically parallel language L.0[Ansari et 92]

• Interpreter of MDB transaction specification Language (VLP)[Kuehn et al 91]

• Interpreter of ECA rules [Dayal et al 92]

• Games vs. Nature [Rusinkiewicz et al 92]

• Fine-state Automata [Jin et al 93]

• Scheduling and enforcing intertask dependencies usingtemporal propositional logic and finite automata [Attie et al 93]

• Scheduling through Distributed Events [Singh and Tomlinson 94]

[Rusinkeiwicz and Sheth 94]



LSDIS

54

Scheduler/Controller Implementations

• Centralized (e.g., [Attie et al 93])

• Distributed (e.g., [Singh and Tomlinson 94])

• Scheduler per Workflow (e.g., [Jin et al 94])

• Scheduling by Workflow Objects (objects/tasks that carry stateinfo.)-- “ambulatory” (i.e., INCA-style [Barbara et al 94])



LSDIS

55

A Prototype Scheduler(a centralized approach)

Task Scheduler Dispatcher

DependencyAutomata

PendingSet

Accepted RequestsSubmitted Requests

Rejected Requests

Queries Replies

DelayedRequestsRe-attempt

Execution

[Attie et al 93]

Agents & event notifications



LSDIS

56

Correctness & Execution Requirements• Executable/ run-time enforceable specifications• Scheduling

– correctness (no violation of intertask dependencies)– safety (progress towards acceptable states)– termination

• Recovery– forward recovery

• Concurrency Control– serializability ??

[Attie et al 93, Georgakopoulos et al 93, Krishnakumar and Sheth 94]



LSDIS

57

Transactional Workflow--What it is and is not

Transactional workflows (try to) address application specificand user-defined correctness, reliability, and functionalityrequirements.Transactional workflows share the objectives of someextended/relaxed transaction models about selectiverelaxation of transactional properties based on applicationsemantics.

... the term is likely to evolve, as it hassignificant appeal. For example, see DOM and Exotica projects ...

[Early use: Sheth & Rusinkiewicz 93, Rusinkiewicz and Sheth 94, Georgakopoulos et al 94][Additional discussions: Georgokopoulos, Hornick & Sheth 94/95, Krishnakumar & Sheth 94/95, Mohan et al 95]



LSDIS

58


Transactional workflows does not imply workflows are similar/equivalent to DB transactions or support all ACID transactionproperties. Usually WMSs do not support consistency of dataacross multiple databases, especially when there are failures.Typically WMSs do not support some of the important featuressupported by TP Monitors (e.g., concurrency control,backward recovery, consistency of data). WMSs today do notsupport concurrency control similar to those involved between“transaction groups” in TP monitors. WMS applicationsoftenrely on local concurrency control.



LSDIS

59


A WMS may provide transactional properties to support forwardrecovery, and/or use system and application semantics to supportsemantic-based “correct” multi-system application execution.Example levels of transactional support a WMS may provide are:

– use of TM concepts/techniques (log input/output, beforeimage, compensation...) to enable forward recovery andfailure atomicity

– part of a workflow has transactional properties (extendedtransactional model with component transactions)

– support a “general” two phase commit (WMS schedule mayprovide commit coordination) or interface with an externalcommit coordinator.



LSDIS

60


A WMS may use transaction management technology, such as– transactional-RPC between two components of a WMS

(e.g., scheduler and task manager),– an external commit coordinator– XA-like protocol between task manager and resource

manager (Interface/Proc. Ent.).

TaskManager

Interface/Proc. Ent.

Scheduler

Application



LSDIS

61

Application/Task and System Semanticsto simplify CC and Recovery Management

Semantics (Application/Task, System) Impact (CC: Con. Control, R: Recovery)limited commutativity (apply) fewer exclusive locks (CC)relaxed isolation (appl) no global commitment (CC)order preservation + rigorousness (sys) early release of locks (CC)idempotency (sys) resubmit transactions (R)+ monotonicity roll-forward recovery (R)

[Jin et al 93, Jin et al 94]



LSDIS

62

Enabling technologies and standards

DCECORBA, OLE/COMNotesX/OPEN TxRPC, ...STDLEDI....SIGs in some of the standardization bodies may addressworkflow issues. For example, Common Facilities in CORBAmay address some workflow issues.



LSDIS

63

Transaction Monitor as a building blockAdvantages second generation TP-monitors (e.g., Encina):

– Ease of implementing fault tolerance: Transactional-RPCs,utilities such as Queue Managers or Structured FileSystems

– Limited support for persistence of server stateDisadvantages:

– Lack of interoperability between monitors– Products still lack good performance, stability, lack of ease

of use/administration, ORProblems with support for state persistence andmulti-threading



LSDIS

64

CORBA as a Building BlockAdvantages• Distributed Computing Infrastructure with several features.

• Support for distributed client-server system on differenthardware and operating system platforms:ORBeline-- SunOS, Solaris, HP/UX, IBM/AIX, OSF,Unixware, MS-Windows, Windos-NT, etc.;Orbix-- Windows NT,Unix, etc.;ObjectBroker-- MS-windows, Unix, Mac, OpenVMS, OS/2.

• Unified support for all data types defined in CORBA.• Support for low-level data format transformations between

different systems.• Multiple concurrent invocation of tasks.

Most ORBs support multi-threading.• Dynamic Interface Invocation.



LSDIS

65

CORBA as a Building Block ... continued

Advantages (continued)• Error Handling.

Support for System Exceptions and User Exceptions(variable level of support by different products).

• Security Service.Permission for every object; additional services (accesspermissions as user-programmed filters or at methodinvocation)

• Reliability/Fault tolerance.Varied support, outside CORBA: ORBeline’s smart agenttracks all object & clients, notifies failures, reroutes lost data



LSDIS

66

CORBA as a Building Block ... continued

Disadvantages/Problems• Lack of interoperability between different commercial ORBs

and/or other distributed infrastructure.Some exceptions: DOE--Orbix; Orbix--Tuxedo/MS OLE/ISIS-RDO; ObjectBroker--OLE-COM/DDE; ORBeline--??

• Lack of access to Legacy Applications.Some exceptions: ObjectBroker-- script servers; DOE --wrapper?

• Limited Mapping of IDL to different languages.Orbix-- C++; ORBeline-- C++; ObjectBroker-- C++;HyperDesk-- C++, DOE-- C++ and C?

• Lack of transaction management support.Not part of core specification. Common FacilitiesArchitecture tries to build transaction support on ORB.



LSDIS

67

Lotus Notes as Building BlockAdvantages• supports client-server application development• supports and incorporates across several hardware platforms:

Windows, NT, OS/2, Solaris, AIX, SCO/ HP Unix, Macintosh• supports multiple network protocols (TCP/IP, Ethernet, IPX,

NetBIOS, AppleTalk, Token Ring)• comprehensive email application included• support for enhanced email and document routing• provides a consistent interface for all applications written for

Notes• ease of use for end users• large user base



LSDIS

68

Lotus Notes as Building Block ... continued• supports multimedia data: formatted text, graphics, audio and

video objects may be embedded in Notes documents• remote access of data

– users may dial up or log in across a network• provides secure access to data

– kerberos based data encryption; password protection• limited replication of databases allowed• can work with other applications (through OLE in Windows)• Notes API provides a good basis for creating workflow

applications

Notes provides a good infrastructure to build a WMS forAdministrative and Ad-hoc workflows.



LSDIS

69

Lotus Notes as Building Block ... continued

Disadvantages/deficiencies• no support for transaction processing

– no support for ACID properties– lack of locking mechanism– latency in replication

• no real time collaboration support– no conferencing/ shared screen support

• no native support for integrating legacy applications• supports only the Notes database format

Currently Notes lacks good support for implementingProduction workflows requiring integration withheterogenous information systems and transactionsupport.



LSDIS

70

Research Projects and Prototypes onWorkflow and Related Issues

(a partial list)APRICOT (Germany),

METEOR (Bellcore, Georgia, UofH),Interbase (Purdue),

ASSET (Bell Labs),TSME/DOM (GTE Lab),Pegasus (HP Lab),TriGSflow (Linz, Austria), Exotica (IBM Almaden),Aachen, INRIA, ETH ...Earlier projects: ETM (DEC), Carnot (MCC), INCA (MITL),



LSDIS

71

Multi-paradigm WorkflowsWorkflow applications in large/complex enterprises and thosespanning multiple enterprises require support for multipleparadigms in terms of:• Types of workflows: production, ad-hoc,...• Communication infrastructures:

– Async (e-mail, document flow/work-group based, messagebased)

– Sync (PRC, t-RPC, ...)– local-area -- internet - wireless

• Computing Structures/Semantics: e.g., transactions inElectronic Commerce, Transaction Processing Systems, andDBMSs



LSDIS

72

Going Forward (commercial technology:Development vs. Market Models

A: Market/Application Domain (Healthcare, Financial, Mfg.,...)

WMS

W: Workflow Management SystemI: Infrastructure

WMSInfrastructureHorizontalVertical Multi-paradigm

A1 An

WMS: different features,same infrastructure

W1 WnInfrastructure

A1 An

I1 In

A1 An

(infrastructure)



LSDIS

73

Trends/Standardization Efforts

Horizontal “Interoperability” Focus– Workflow Coalition-- lack of “information system”

perspective and “transaction” support so far

Vertical “Market” Focus– not yet started, but likely to start soon-- Healthcare,

Finance, Manufacturing– SIGs are forming

Infrastructure components forMulti-paradigm WMS

Distributed Proc.

- CORBA, DCE,

DTP

Groupware

Trans.Workflow

- Lotus Notes

- EncinaWorkflowSoftware

Infrastructure

X.400, X.500

CTM/ETM

Component infrastructures forfuture “multi-paradigm” workflowmanagement systems:

– e-mail– internet– Lotus Notes– CORBA– Transaction MonitorsSupport

e-mailinternet



LSDIS

75

A Small List of Research Challenges

• Model: multiple views of what is modeled, when to usetransaction properties/features, correctness issues

• Language: ease of specification vs. features; logical errorhandling; use of visual programming

• Development: next generation enterpirse applicationdevelopment scenario (multi-system applications running withinand across business enterprises), testing, simulation

• Run-time System: error handling, failure handling/recovery,correctness, heterogeneous objects, scalability (e.g.,scheduling for environment with many concurrent workflows),performance

• Standards: plenty of relevant efforts, what is useful now?

Conclusions• New paradigm for distributed computing? Perhaps it is the

way to provide glue to handle legacy systems, and toto support migration/evolution.

• Technology related to business processes andapplications-- better relevance and visibility thanheterogeneous DDBMS and extended transactions;still database and transaction management haveimportant roles to play.



LSDIS

77

Brief biography of the Tutorial Speaker:

Dr. Amit Sheth directs the Large Scale Distributed Information Systems (LSDIS) Lab and is an Associ-ate Professor of Computer Science at the University of Georgia. Earlier he worked for nine years in theR&D labs at Bellcore, Unisys, and Honeywell. He has lead projects on heterogeneous DDBMS, factoryinformation system, integration of AI-database systems (BrAID), transactional workflows (PROMT andMETEOR), federated database tools (BERDI and TAILOR), multidatabase consistency, and data qual-ity (Q-Data). Dr. Sheth has published over 75 papers, given over 45 invited talks and 14 tutorials, andlead two international conferences and a workshop as a General/Program (Co-)Chair. He has alsoserved twice as an ACM Lecturer, has been on over twenty five program and organization committees,and is on the editorial board of four journals.

The LSDIS lab maintains very active collaboration with industry, and has won significant projects in theareas of interoperable information system and workflow management (under the Healthcare Informa-tion Infrastructure Program awarded by NIST) and global information system and management of het-erogeneous digital data (awarded in the Massive Digital Data Systems initiative). Industrial partners onthese projects are Bellcore, MCC and CHREF. The lab acknowledges sponsorship/industrial affiliationof the HP Labs and the Persistence Software Inc.

http://www.cs.uga.edu/~amit



LSDIS

78

Partial BibliographySpecial Issues, Proceedings and Edited Collections:

[1] M. Hsu, Ed., Special Issue on Workflow and Extended Transaction Systems, Bulletin of the Technical Committee on Data Engi-neering (IEEE Computer Society), 16 (2), June 1993.

[2] T. While and L. Fischer, New Tools for New Times: The Workflow Paradigm-- The Impact of Information Technology on Busi-ness Process Reengineering, Future Strategies Inc., Book Division, Alameda, CA, 1994.

[3] O. Bukhres and e. Kueshn, Eds., Special Issue on Software Support for Work Flow Management, Distributed and Parallel Data-bases -- An International Journal, 3 (2), April 1995.

Papers on Workflow and Multisystem Applications: (* Paper accessible at: http://www.cs.uga.edu/LSDIS/)

[4] * M. Ansari, L. Ness, M. Rusinkiewicz, and A. Sheth, "Using Flexible Transactions to Support Multi-system TelecommunicationApplications," in Proc of the 18th Int’l Conf on Very Large Data Bases, August 1992.

[5] * P. Attie, M. Singh, A. Sheth, and M. Rusinkiewicz, "Specifying and Enforcing Intertask Dependencies," in Proc. of the 19thIntl Conf. on Very Large Data Bases, August 1993.

[6] * Y. Breitbart, A. Deacon, H.-J. Schek, A. Sheth, and G. Weikum, “Merging Application-centric and Data-centric Approachesto Support Transaction-oriented Multi-system Workflows,” in SIGMOD Record 22 (3), September 1993.

[7] * D. Georgakopoulos, M. Hornick and A. Sheth, “An Overview of Workflow Management: From Process Modeling to Infra-structure for Automation, Technical Report TR-CS-94-003, LSDIS Lab, Dept. of CS, Univ. of GA, November 1994. Also, in [3].

[8] R. Gunthor, "Extended Transaction Processing Based on Dependency Rules," in Proc. of the RIDE-IMS '93: Intl. Workshop onMultidatabase Systems, April 1993.

[9] F-J. Fritz, Workflow implementation based on the R/3 reference model, SAP AG (Report/Manuscript), 1995?

[10] D. Hollingsworth, “The Workflow Reference Model,” Workflow Management Coalition, TC00-1003, December 1994.

[11] M. Hsu and C. Kleissner, ObjectFlow: Towards a Process Management Infrastructure,” submitted for publication.

[12] P. Korzeniowski, “Workflow Software Automates Processes,” Software Magazine, February, 1993.

[13] * N. Krishnakumar and A. Sheth, “Managing Heterogeneous Multi-system Tasks to Support Enterprise-wide Operations,” Tech-nical Report TR-CS-94-002, LSDIS Lab, Dept. of CS, Univ. of GA, September 1994.A (shorter) version appears in [3].



LSDIS

79

[14] F. Leymann and W. Altenhuber, “Managing Business Processes as an Information Resource,” IBM Systems Journal, 33 (2),1994.

[15] F. Leymann and D. Roller, “Business Process Management with FlowMark,” Proceedings of IEEE Compcon, March 1994.

[16] R. Marshak, “Software to Support BPR - The value of Capturing Process Definitions”, Workgroup Computing Report, PatriciaSeybold Group, Vol. 17, No. 7, July, 1994.

[17] S. McCready, “There is more than one kind of Work-flow Software,” Computerworld, November 2, 1992.

[18] C. Mohan, G. Alonso, R. Gunthor, and M. Kamath, “Exotica: A Research Perspective on workflow Management Systems,” Bul-letin of the Technical Committee on Data Engineering (IEEE Computer Society), 18 (1), March 1995.

[19] * Rusinkiewicz and A. Sheth, "Specification and Execution of Transactional Workflows," in the Modern Database Sys-tems: The Object Model, Interoperability, and Beyond, W. Kim, Ed., Addison-Wesley, 1994.

[20] T. Smith, “The Future of Work flow Software,” INFORM, April 1993.

[21] * D. Woelk, P. Attie, P. Cannata, G. Meridith, A. Sheth, M. Singh, and C. Tomlinson, "Task Scheduling using Intertask Depen-dencies in Carnot," in Proc. of ACM SIGMOD Conf. on Management of Data, May 1993.

For additional references and pointers to products, see for example papers in [1] and [2].



LSDIS

80

A few papers on relaxed transactions (model and language issues only):

A. Biliris, S. Dar, N. Gehani, H.V. Jagadish and K. Ramamritham,"ASSET: A system for supporting extended transactions," in Proc. ofthe 1994 ACM SIGMOD Conference on Management of Data, 1994.J. Chen, O. Bukhres, and A. Elmagarmid, “IPL: A Multidatabase Transaction Specification Language,” In Proc. of the 13th Intl. Conf.on Distributed Computing Systems, Pittsburgh, PA, May 1993.P. Chrysanthis and K. Ramamritham, "A Formalism for Extended Transaction Models," in Proc. of the 17th VLDB Conference, 1991.U. Dayal, M. Hsu, and R. Ladin, "A Transactional Model for Long-Running Activities," in Proc. of the 17th Int'l Conference on VeryLarge Data Bases, September 1991.A. Elmagarmid, Y. Leu, W. Litwin, and M. Rusinkiewicz, “A Multidatabase Transaction Model for InterBase,” Proceedings of the 16thInternational Conference on VLDB, 1990.H. Garcia-Molina, D. Gawlick, J. Klein, K. Kleissner, and K. Salem, "Coordinating Multi-transaction Activities," Technical Report CS-TR-247-90, Princeton University, February 1990.D.Georgakopoulos, M. Hornick, P. Krychniak, and F. Manola, "Specification and Management of Extended Transactions in a Program-mable Transaction Environment, in Proc. of the Intl. Conf. on Data Engineering, February 1994.W. Jin, N. Krishnakumar, L. Ness, M. Rusinkiewicz, and A. Sheth, “Multidatabase transactions in the telecommunications environment:Modeling, Concurrency Control and Recovery Issues,” Bellcore Technical Memorandum, September 1993.E. Kuehn, F. Puntigam, and A. Elmagarmid, “Transaction Specification in Multidatabase Systems Based on Parallel Logic Program-ming,” in Proc. of the RIDE-IMS '91: Intl. Workshop on Interoperability in Multidatabase Systems, April, 1991.J. Klein, "Advanced Rule Driven Transaction Management," in Proc. of the IEEE COMPCON, 1991.F. Schwenkreis. APRICOTS - A prototype implementation of a ConTract system: Management of the Control Flow and the Communi-cations System," in Proc. 12th Symposium on Reliable Distributed Systems, 1993.



LSDIS

81

Additional items that may be covered in this tutorial if time permits ...

screens from some workflow design toolsdetailed example of process modeling and workflow modelingcomparison of some products wrt to their models and featuresdescription and comparisons of some research prototypeslanguage issues in more detialsrun-time issues in more detail

workﬂow automation: applications, technology and research

Documents