prime prime : provenance incorporating methodology steve munroe (sjm@ecs.soton.ac.uk) the eu grid...

PrIMe

PrIMe: Provenance Incorporating MethodologySteve Munroe (sjm@ecs.soton.ac.uk)

The EU Grid Provenance ProjectUniversity of Southampton UK

www.gridprovenance.org

2

PrIMe

EU Grid Provenance Consortium

• University of Southampton– Luc Moreau, Steve Munroe, Sheng Jiang, Paul Groth, Simon Miles

• IBM UK (Project Coordinator) – John Ibbotson, Neil Hardman, Alexis Biller

• University of Wales, Cardiff – Omer Rana, Arnaud Contes, Vikas Deora

• Universitad Politecnica de Catalunya (UPC)– Steven Willmott, Javier Vazquez

• SZTAKI– Laszlo Varga, Arpad Andics

• German Aerospace– Andreas Schreiber, Guy Kloss, Frank Danneman

3

PrIMe

Overview of Talk• Introducing Provenance• Introducing PrIMe• Stepping through PrIMe

– Step 1. Provenance use cases– Step 2. Information items– Step 3. Identifying actors– Step 4. Actor interactions– Step 5. Knowledgeable actors– Step 6. Adaptations

• Evaluation• Summary• Conclusions

4

PrIMe

Provenance: dictionary definition

• Oxford English Dictionary: – the fact of coming from some particular source

or quarter; origin, derivation– the history or pedigree of a work of art,

manuscript, rare book, etc.; concretely, a record of the ultimate derivation and passage of an item through its various owners.

5

PrIMe

Provenance Definition

• Our definition of provenance in the context of applications for which process matters to end users:

The provenance of a piece of data is the process that led to that piece of data

• Our aim is to conceive a computer-based representation of provenance that allows us to perform useful analysis and reasoning to support our use cases

• We use the notion of Process Documentation, which is composed of p-assertions

6

PrIMe

Provenance Applications

Aerospace engineering: maintain a historical record of design processes, up to 99 years.

Organ transplant management: tracking of previous decisions, crucial to maximise the efficiency in matching and recovery rate of patients

8

PrIMe

Types of p-assertions (1)

– Interaction p-assertion: is an assertion of the contents of a message by an actor that has sent or received that message

I received M1, M4I sent M2, M3

9

PrIMe

Types of p-assertions (2)

– Relationship p-assertion: is an assertion, made by an actor, that describes how the actor obtained output data or the whole message sent in an interaction by applying some function to input data or messages from other interactions.

M2 is in reply to M1M3 is caused by M1M2 is caused by M4

M3 = f1(M1)M2 = f2(M1,M4)

10

PrIMe

Types of p-assertions (3)

– Actor state p-assertion: assertion made by an actor about its internal state in the context of a specific interaction

I used sparc processor

I used algorithm xversion x.y.z

11

PrIMe

The Provenance Middleware Architecture

12

PrIMe

Introducing PrIMe

• A software engineering methodology for making applications provenance-aware

13

PrIMe

Introducing PrIMe:Key aimsProvide software engineering guidelines for:• identifying and expressing provenance use cases• identifying the kinds of information items that are

required to satisfy use cases• identifying actors and the interactions between them in

order to effect the recording of process documentation• identifying the set of adaptations that integrate the

provenance architecture with the application to expose the right kinds of information.

14

PrIMe

Introducing PrIMe:Overview

Step 6. Adaptations

Step 5. Knowledgeable Actors

Application Structure

Step 3. Actors Step 4. Interactions

Step 1. Use Cases

Step 2. Information Items

15

PrIMe

Step 1: Provenance Use Cases

Application StructureStep 1. Use Cases

17

PrIMe

Step 1: Provenance Use Cases Gathering Use Cases• It is not always obvious to users what use cases

they could expect the provenance middleware architecture to support.

• We provide a simple requirements elicitation process to help designers collect the core provenance use cases– Give definitions of provenance– Give examples of the general questions that can be

answered using the architecture

18

PrIMe

Step 1: Provenance Use Cases Definition of Provenance

• The provenance of a result is the process that produced that result.

22

PrIMe

Step 1: Provenance Use Cases Elicitation Steps

• 3 Important steps: – Step (1) Describe something that already

happens in the application.

– Step (2) Describe a specific provenance-related use case question that cannot be answered (easily), but our functionality could help to achieve.

– Step (3) Identify the relevant services required for answering the use case.

23

PrIMe

Step 1: Provenance Use Cases Example use case

• Donor A’s organs are screened for potential donation.

• What is the provenance of the donor organ’s diagnosis?

25

PrIMe

Step 2: Information items

Application StructureStep 1. Use Cases

Step 2. Information Items

26

PrIMe

Step 2: Information ItemsOverview• The kinds of information that will answer

your use case

• May be one piece or many pieces of information– e.g. a given result, or a sequence of decisions

• For each use case, identify the information items required to satisfy the use case.

27

PrIMe

Step 2: Information ItemsExamples• Information items may be :

– Data items, i.e. the result of some calculation, decision (found in interactions or actor state).

– Whole or part processes, e.g. the sequence of decisions that led to a donor’s organ being rejected for donation.

– Relationships, e.g. what were the causal determinants of a given decision.

28

PrIMe

Step 2: Information ItemsCapture• Information items are to be captured by

process documentation, i.e. p-assertions

– Data items: Interaction or actor state p-assertions

– Processes: Interaction and relationship p-assertions

29

PrIMe

Step 3: Actors

Step 3. Actors

Step 1. Use Cases

Step 2. Information Items

Application Structure

30

PrIMe

Step 3: ActorsDescription

• An actor is an entity within the application that performs actions, e.g. Web Services, components, machines, people etc. and interacts with other actors.– One actor may be seen as being composed of

other actors.

32

PrIMe

Step 3: Actors Identification Heuristics

• Identify the components that receive information. E.g. a component/service in a workflow, a script command, the GUI/desktop application into which a user enters information.

• Identify the components that provide the information in each interaction. These could be, for example, a workflow engine, a script executor, a user.

34

PrIMe

Step 4: Interactions

Application Structure

Step 3. Actors Step 4. Interactions

Step 1. Use Cases

Step 2. Information Items

35

PrIMe

Step 4: InteractionsInformation exchange

36

PrIMe

Step 3: ActorsInformation

Message ID Data item Receiver ID

M2

M4

M6

Q1

Pid

r1

EHCR

Testing Lab

UI

Donor data collector

Sending

Receiving

Message ID Data item Sender ID

M1

M3

M5

q1

pid

R1

UI

EHCR

Testing lab

Actor : Donor data collector

Actor: Donor data collector

42

PrIMe

Step 5: Knowledgeable Actors

Step 5. Knowledgeable Actors

Application Structure

Step 3. Actors Step 4. Interactions

Step 1. Use Cases

Step 2. Information Items

43

PrIMe

Step 5: Knowledgeable Actors

• Knowledgeable actors have access to Information items

• Sometimes, for a given information item, a knowledgeable actor cannot be found

• Further decomposition might be necessary or,

• New actors may need to be introduced (Step 6)

44

PrIMe

Step 5: Knowledgeable ActorsWho knows what?

Hospital

EHCRS Testing lab

45

PrIMe

Step 5: Knowledgeable Actors OTM Example

User interface

Donor data collector

Electronic health care records

Testing laboratory

Hospital

47

PrIMe

Step 3,4,5: Knowledgeable ActorsRepeat as necessary

– Step 3: Identify actors– Step 4: Identify interactions– Step 5: Identify knowledgeable

actors

48

PrIMe

Step 6: Adaptations

Step 6. Adaptations

Step 5. Knowledgeable Actors

Application Structure

Step 3. Actors Step 4. Interactions

Step 1. Use Cases

Step 2. Information Items

49

PrIMe

Step 6: AdaptationsModifying actors

• A non-knowledgeable actor may be modified so that it gains access to information items not currently available to itself or other actors in the system.

50

PrIMe

Step 6: AdaptationsActor Introduction

•A new actor can be introduced to the application to help in the answering of use cases

51

PrIMe

Step 6: AdaptationsInteraction Extension

• An interaction in the application can be extended to exchange more information between a knowledgeable actor and a non-knowledgeable actor, making the latter knowledgeable.

Actor Actor

Before

Actor Actor

After

a

a,b

52

PrIMe

Step 6: AdaptationsInteraction Introduction

• A new interaction between actors can be introduced into the application in which a knowledgeable actor sends the information item to another actor, which then becomes knowledgeable.

Actor Actor

Before

Actor Actor

Afterb

53

PrIMe

Step 6: AdaptationsProvenance Functionality

• The provenance wrapper exposes an actor’s input and output data, relationships and aspects of the actor’s state.

54

PrIMe

Step 6: AdaptationsThe Client Side Library• A collection of functions

– To allow provenance-aware applications to communicate with provenance store services

– An implementation of the actor side library should contain at least one of the query library, the record library and the management library

– Helps application developers follow architecture rules

56

PrIMe

Recording Provenance

57

PrIMe

Evaluation• Protein compressibility experiment

• 10% overhead for asynchronous recording

58

PrIMe

Summarising PrIMe

• Step 1: Identify the provenance use cases

• Step 2: Identify relevant information items

– Step 3: Identify actors– Step 4: Identify interactions– Step 5: Identify knowledgeable actors

• Step 6: Make necessary adaptations

Granularity

59

PrIMe

Conclusions

• PrIMe provides a clear and easy guide to make applications provenance-aware

• Crucial in the adoption of the Provenance Middleware Architecture

60

PrIMe

Questions?

Steve Munroe

sjm@ecs.soton.ac.uk