transparency in the data supply chain

Paul Groth (@pgroth)Web & Media GroupDepartment of Computer ScienceVU University Amsterdamhttp://www.few.vu.nl/~pgroth

Transparency in the Data Supply Chain

Outline

• Data integration for analysis– i.e. remixing data

• The need for transparency• Two solutions• The future

http://openphacts.orgpmu@openphacts.org

@Open_PHACTS

Why?

Public Domain Drug Discovery Data:Pharma are accessing, processing, storing & re-processing

LiteraturePubChem

GenbankPatents

DatabasesDownloads

Data Integration Data AnalysisFirewalled Databases

Repeat @ each

companyx

Prioritised Research QuestionsNumber sum Nr of 1 Question

15 12 9 All oxido,reductase inhibitors active <100nM in both human and mouse

18 14 8Given compound X, what is its predicted secondary pharmacology? What are the on and off,target safety concerns for a compound? What is the evidence and how reliable is that evidence (journal impact factor, KOL) for findings associated with a compound?

24 13 8 Given a target find me all actives against that target. Find/predict polypharmacology of actives. Determine ADMET profile of actives.

32 13 8 For a given interaction profile, give me compounds similar to it.

37 13 8 The current Factor Xa lead series is characterised by substructure X. Retrieve all bioactivity data in serine protease assays for molecules that contain substructure X.

38 13 8 Retrieve all experimental and clinical data for a given list of compounds defined by their chemical structure (with options to match stereochemistry or not).

41 13 8

A project is considering Protein Kinase C Alpha (PRKCA) as a target. What are all the compounds known to modulate the target directly? What are the compounds that may modulate the target directly? i.e. return all cmpds active in assays where the resolution is at least at the level of the target family (i.e. PKC) both from structured assay databases and the literature.

44 13 8 Give me all active compounds on a given target with the relevant assay data46 13 8 Give me the compound(s) which hit most specifically the multiple targets in a given pathway (disease)59 14 8 Identify all known protein-protein interaction inhibitors

www.openphacts.org

Research question 15: All oxido reductase inhibitors active < 100nM in both human and mouse

From Mabel Loza - USC team

From Mabel Loza - USC team

From Mabel Loza - USC team

From Mabel Loza - USC team

Research question 15: All oxido reductase inhibitors active < 100nM in both human and mouse

ChEMBL:

Search target Oxidoreductase: 481 targets from different species

Selection of all the oxidoreductases and filtering bioactivities with the criteria IC50 < 100 (no units could be selected): 11497 data obtained

Table exported to a excel spreadsheet and manually filtered

From Mabel Loza - USC team

5 people

Working 6 hours

Problem: Data Integration

DataSource

DataSource

Data Warehouse

Queries

ExtractTransformLoad

DataSource

DataSource

Mediator

Queries

QueryReformulation

Using the Power of Open PHACTS, London, 22-23 April 2013

RDFNanopub

Db

VoID

Data Cache (Virtuoso Triple Store)

Semantic Workflow Engine

Linked Data API (RDF/XML, TTL, JSON)DomainSpecificServices

Identity Resolution

Service

Chemistry RegistrationNormalisation & Q/C

IdentifierManagement

Service

index

Co

re P

latf

orm

P12374EC2.43.4

CS4532

“Adenosine receptor 2a”

RDF

VoID

Db

RDFNanopub

Db

VoID

RDF

Db

VoID

RDFNanopub

VoID

Public Content Commercial

Public Ontologies

User Annotations

Applications

15

Open PHACTS Explorer

16

Open PHACTS Explorer

?

Credits: Curt Tilmes, Peter Fox

Tilmes, C.; Fox, P.; Ma, X.; McGuinness, D.L.; Privette, A.P.; Smith, A.; Waple, A.; Zednik, S.; Zheng, J.G., "Provenance Representation for the National Climate Assessment in the Global Change Information System," Geoscience and Remote Sensing, IEEE Transactions on , vol.51, no.11, pp.5160,5168, Nov. 2013

Problem: I don’t trust your assessment what is it based on?

Tension:

Integrated & SummarizedData

Transparency& Trust

Solution

Integrating and exposing provenance provided by multiple sources

provbook.org

National Climate Change Assessment Provenance

PROV the database as a black box

Q

Goal

• the capability to trace back, for each query result, the complete list of sources and how they were combined to deliver a result.

Implement In a Graph Database at Scale

Marcin WylotPhilippe Cudré-MaurouxExascale LabUniversity of Fribourg

http://diuf.unifr.ch/main/xi/diplodocus

TriplePROV [WWW2014]

Provenance Polynomials

Test on large messy data

• Billion Triple Challenge– Crawled from the linked open data cloud

• Web Data Commons– RDFa, Microdata extracted from common crawl

• 115 million triples (25 GB)• 8 Queries defined for BTC

– T. Neumann and G. Weikum. Scalable join processing on very large rdf graphs. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, pages 627–640. ACM, 2009.

External + Internal Provenance

• Unified queries over external and database provenance

• Adapting query results based on provenance

• Performance improvements

FUTURE

60 % of time is spent on data preparation

Big Data is often lots of small data

http://www.data2semantics.org/prov-reconstruction-challenge/

Questions?

• More info:– openphacts.org– data2semantics.org– provbook.org– Paul Groth, "Transparency and Reliability in the Data Supply

Chain," IEEE Internet Computing, vol. 17, no. 2, pp. 69-71, March-April, 2013

– Paul Groth, "The Knowledge-Remixing Bottleneck," Intelligent Systems, IEEE , vol.28, no.5, pp.44,48, Sept.-Oct. 2013

– Marcin Wylot, Philippe Cudré-Mauroux and Paul Groth. TripleProv: Efficient Processing of Lineage Queries over a Native RDF Store. WWW 2014

Backup

Hack Sparql

What’s the overhead? Setup

Source and complete trace (i.e. triple level)

Annotations:

Propagate annotations through the query processing pipeline

What’s the overhead?