making workflows work prof. yike guo dept. of computing imperial college london inforsense...

Making Workflows Work

Prof. Yike Guo

Dept. of Computing Imperial College London

InforSense Limited

Proprietary and Confidential

DiscoveryNet Project Funding

One of the Eight UK National e-Science Projects (£2.2 M) Sept 2001 – March 2005

Partners

Achievements Constructing the World’s First Infrastructure for Building Analytical Services by

Scientists For the First time Discovery Net Realises the Dynamic Construction of Compositional

Services on GRID for Real Time Knowledge Discovery and Decision Making

Outputs Software Research: DNet platform commercialized by InforSense Ltd (>100

customers) Total user numbers > 2000 Applications Research: Application out puts in sensor technology commercialized by deltaDot Ltd Number papers published: 10 Journal Papers, 30 Conference Papers 8 PhD completed and 50 Master students Ranked OUTSTANDING at the project final review

http://www3.imperial.ac.uk/portal/page/portallive/0FDF9D27B17B43D0E0440003BACD13A5

http://www.deltadot.com/

http://www.inforsense.com/


InforSense Introduction100+ customers (70% Fortune 200 companies)

2006 3rd fastest growing company in UK (Sunday Times Tech Track)

2007 8th fastest growing venture-based company in UK (Financial Times)

Global footprint with offices in London (HQ and R/D), Boston (USA HQ) and Shanghai (Asia HQ and Development base)

Global sales with 70% outside of Europe

7 years of delivering products and services to pharmaceutical and Financial industries

Spin out from Imperial College London

Invented “Distributed Data Mining ”

‘98 First Enterprise Deployment

Embedding Analytics Technology3rd fast growing company in UK

‘01

‘05

2004

,03

IEEE Super-computing Award –Grid based analytics

InforSense FormedIntroducedKDE Analytics Platform

Discovery Net Project

Embedding Analytics in Major Enterprise Systems

‘06

‘00

Innovation in Embedding Analytics


CAMBRIABIOSCIENCES

Those who are using our workflow

http://www.huawei.com/

http://www.erasmusmc.nl/index.htm

http://www.labvantage.com/index.html

http://images.google.com/imgres?imgurl=http://www.oeb.harvard.edu/Library/harvard_logo.gif&imgrefurl=http://www.oeb.harvard.edu/logos.htm&h=187&w=189&sz=23&tbnid=R5SoTX1GT-LefM:&tbnh=102&tbnw=103&prev=/images%3Fq%3Dharvard%2Blogo%26um%3D1&start=2&sa=X&oi=images&ct=image&cd=2

http://www.servier.com/index.asp

http://www.astellas.com/global/index.html

Proprietary and Confidential Excel

EMR Database

s

OraclePre-processing

OraclePre-processing

3rd partyAnalytics

3rd partyAnalytics Web servicesWeb services Biomedical

Informatics tools

BiomedicalInformatics tools

Multiple data sources

Multiple data sources

Interactive Knowledge Discovery

Interactive Solution Building

Rapid Application Deployment

Portal / Dashboard

Application

InforSense Workflow Methodology

Files

Automation & Scheduling

Data

Applications

Components

InforSenseAnalytics

InforSenseAnalytics

Integrative Analytics Workflow Environment

Delivery to End User

Dynamic Data & App Integration

Business Process

Administrator Clinician Disease Biologist


What is InforSense WF System Designed for ? InforSense workflow system is not an application but a

framework to build and deliver applications directly to scientist/business user:

Chem-Studio

ADMET Browser


Pipelining

Web ServiceOrchestration

ETL

EnterpriseService Bus

Data/Text Mining

Business ProcessManagenment

Simulation& Modelling

InforSense Generic Workflow Engine


Experience of 7 years in WF business

Building workflow is easy ! However,

Building a USABLE workflow is not easy Building a REUSABLE workflow is hard Building a REUSABLE workflow applications is

very hard Building a REUSABLE workflow application for

EVERYONE is very very hard Building a function is easy, building an application

is hard, it is even harder if we enable a non-IT person to build a good reliable application for other people to use everyday!


InforSense Workflow System Development

Workflow ExecutionReliable Enterprise Wide

Execution

Workflow ManagementCollaborative Knowledge Management

Workflow Deployment:Building Reusable WF Applications

WorkflowWarehousing

Resource Mapping

Service Abstraction

Workflow AuthoringComposing services

Condor-GCondor-G

Native MPINative MPI OGSA-serviceOGSA-service

Web ServiceWeb Service

UnicoreUnicoreOralce 10g

Web WrapperWeb WrapperSun Grid Engine

Workflow EmbeddingPervasive WF applications


Three Tiers of Workflow Framework

Building Layer

Application Layer

Embedding Layer

Analytical Workflow Development

Rapid Application Development

Service Orchestration

Business Rules

Embed in Other Applications

Analytic Service Encapsulation

Publish Services for Display

BPEL

InforSense Workflow Building:

Not about another graph notation but about how to build a meaningful graph


Current model of workflow authoring/execution

No help provided to user (authoring/execution) Model is based on expert user who know about services Model requires user to be trained in a workflow language/system Interoperability between workflow systems is only at run-time


The key the success : End User Oriented Workflow Construction

Build semi-automatic tools that advise/assist user in wf authoring

Make use of previous knowledge about developing workflows

Explicit/Expert knowledge Implicit knowledge in previous

workflows

The aim is to help user, not replace him


Guided Workflow Construction

User is presented by high-level descriptions of predefined task steps

User is guided iteratively in instantiating the task descriptions using workflow templates

User can retrieve workflows and workflow templates from repository

Approach supports using workflows from multiple systems using existing run-time interoperability mechanisms


Workflow Advisor: InforSense Customer Hubs

http://www.inforsense.com/


Extended infrastructure:Workflow warehousing and mining

Workflow Advisor Initial implementations of prototype for bio

applications

Workflow Assistant Abstract component initial prototypes

Workflow Mining Repository of workflows from Southampton

Workflow Annotations independent from

workflow language

Warehouse Search and execute web services/Grid

services and workflows Syntactic and semantic search


Extended infrastructure: Workflow warehouse/registry

InforSense Embedding and Deployment

Workflow output is not a data, but an application/service


InforSense KDE Deployment Strategies

Deploy workflows to InforSense portal

Deployment features: multi-page, service chain, layout editor

Multi-stage applications: group workflows into stages

Component based deployment

Portlet based deployment

Portlet component: JSR 168 compatible portlet components

Business process workflow

Based on control flow orchestrated workflows and role based deployment


Web-based Deployment

Portal Container allows users to

build dashboards

Each Workflow generate data for a

dashboard component

Workflow results viewed in simple charts - can be linked to other

pages


Deployment Features (2)

Define multiple pages

Move to next page


Chip QC Normalise Analyse Interpret

Design ExperimentDesign Study groups for transcriptomics portal

Gene Expression ProfilingPre-process and Analyse the results of a gene expression analysis to compare control vs. test populations

Splice Variance AnalysisPre-process and Analyse the results of an Exon Chip to find differences in splice variance between control vs. test populations

Results

Slide 1

Slide 2

Slide 3

Slide 4

Slide 5

Slide 6

QC Step

Recommended rerun

Chip must be rerun Normalisation services

•RMA (recommended)•LiWong•ETC

Next Steps

Submit to Report>

Example ApplicationAnalytical stage

Workflow configured to group according to stage Portal look and feel can

be customized by style sheet






Results

Analysis services

•Volcano Plot (recommended)•PCA•Dendrogram

Next Steps

Submit to Report>

Example Application






Results

Next Steps

Submit to Report>

Save Result to Report

Analysis services

•Select Transcripts

•Filter Data

Example Application






Results

Next Steps

Submit to Report>

Save Selected Items to Report

Interpretation services

•Send Data to Ingenuity•Send Data to Gene Go •Send Data to•Text Analysis

Example Application






Results

Next Steps

Submit to Report>


•Send Data to Gene Go•Text Analysis

Related Pathway

Save to Report

Example Application






Results

Next Steps

Submit to Report>


•Send Data to Gene Go•Text Analysis

Related Pathway

Select Subset for Text Analysis

Example Application


Business Process Management Development

A Business Process Management (BPM) describes the orchestration of different tasks to complete a specific business objective

Business Processes need to orchestrate

Automated Tasks

User Tasks

Exception Handling

Running Tasks in parallel

Synchronisation of parallel tasks

Business Process Workflow (1)


InforSense Control Flow

InforSense Control Flow for Orchestrating Workflows for Business Process

Run

Task

Handle Exceptions

Initiate Parallel Tasks

Synchronize Parallel Tasks

Apply Rules



Orchestra business analytics by control flow

Workflow A Workflow B Workflow C

Sub-process 1 Sub-process 2

Control Flow Represents a Business Process

Deploy to Portal

ApplicationBuilding Blocks

services

Process Building Blocks

definition of linkage/control

and user interactions



Workflow interoperabilityWorkflows and business processes (BPEL)


Embedding Workflow Analytics into Applications

Process View

Lifetime Value Service

Risk Service

Churn Service

Embeddable Analytic Applications

Analytical Workflows

Model Repository

Business Rules and Model

Deploy New Actions

customer data

Predictive scores

Risk data

Risk Evaluation

Acceptable Risk?

Yes

No

Get Value Score

Normal Service

Get Churn Score

Risk Assessment

Upgrade offer

KVM


Integrating Analytics with Business Rules: Adaptive Business Process

En

terp

ris

e S

erv

ice

s B

us

Business Process

Business Portal

Business

operational

data

Analytics to drive adaptive processes

Rule engine Rule Engine


Embedding with Applications

InforSense Tools as one item in Windows based application system


“One of the biggest barriers to achieving productivity and responsiveness is IT – it has become a bottleneck. Another barrier to achieving the goal is the lack of intelligence that drives most IT applications. They are just operating as a rapid functional replacement, and failing to exploit the data which is being generated within other elements of the IT infrastructure.

A product that could meet that challenge and enable business to generate and deploy intelligence with speed, accuracy and without the need for specialized skills would be remarkable.

I believe that InforSense is that remarkable tool.”

-- David Norris, Senior Analyst, Bloor Research

Making Workflow Work


Thank You !

making workflows work prof. yike guo dept. of computing imperial college london inforsense...

Documents

reusable workflow applications

wf businessbuilding

inforsense wf system

application outputs

good reliable application

london hq

applications research

growing venturebased