impact observatory of clinical research regarding data sharing karmela krleža-jerić on behalf of...

IMPACT Observatory of Clinical Research Regarding Data Sharing

Karmela Krleža-JerićOn behalf of IMPACT Observatory

Ottawa Group -IMPACT- Dpt of Research in Biomedicine and Health, University of Split School of Medicine

krlezajk@hotmail.com

Karmela Krleža-Jerić

No conflict of interestbut

interest in research &public disclosure of research data

KKJ

Open trial data

• Public health aspect; ultimate user and beneficiary aspects

• International aspect: dialogue, initiatives, actions, standards, IMPACT

• Clinical trial aspect – transition regarding changes in data sharing ;shift of paradigm

KKJ

outline

• Clinical trials• Data sharing• IMPACT • IMPACT Observatory • IMPACT observatory issues

KKJ

IMPACT

IMProving Access to Clinical Trial data

http://ottawagroup.ohri.ca/disclosure.htmlKKJ

CLINICAL TRIAL and RCT

• “clinical trial” is a prospective (research) study evaluating the effects of one or more health-related interventions assigned to human participants.

• RCT (randomized controlled/clinical trial) is using random allocation, determining whether - an intervention is effective or harmful.or- an innovative intervention leads to improved health.

KKJ

5 Open Data

KKJ

Tim Berners-Lee, the inventor of the Web and Linked Data initiator, suggested a 5 star deployment scheme for Open Data.

Available on web under open licence

Structured data (excel instead of image scan of table)

Non-proprietary formats (CSV instead of excel)

Use URI to denote things so peoplecan point on your stuff

Link your datato other data to provide context

to increase reliability of evidence for evidence informed decision

making

Why do we want to open CT data?

KKJ

to speed up knowledge creation and development of better diagnostics

& therapies

Why do we want to open CT data?

KKJ

KKJ

Pooled analysis

data- knowledge-innovation pyramid

New

therapies

Wisdomknow how & what to do

Knowledge - evidence

Information - results

Data – findings

KKJ

And if we do not share data?

KKJ

biased clinician

impact on patients

Lack of information

biased researcher & sponsor

inefficient and irresponsible research

KKJ

IMPACT Observatory story

KKJ

2004

2006

2008

2012

2013

2014

Ottawa Statement

WHO Trial registration standards

PROCTOR Ottawa (CIHR-IRSC)

Environmental Scan of Repositories of Clinical Research Data

Planning Meeting IMPACT Ottawa (British High Commission, University of Ottawa, participants)

IMPACT Observatory NewFelPro Fellowship

kkj-CIHR May 2008

IDEAPROTOCOL

ID#

TRIALID#

ID#

kkj-CIHR May 2008

OTTAWA STATEMENT 1&2

AmendmentsRecruitment statusAdverse eventsInterim analysesEarly stopping (if applicable)

OTTAWA STATEMENT 2

Register 20 WHO items plus

• consent form• details of design• ethics approval• countries & sites• key dates• full protocol• contract/ financial agreements• systematic review

OTTAWA STATEMENT 1(2004)

• ID• Protocol• Results

TRIALID#

DATADATA

DATADATAOTTAWA STATEMENT 3Results Disclosure 1

Content: CONSORT + OS2 itemsSites / CountriesAny independent validation of resultsList of any publications

DATADATA

DATADATA

OTTAWA STATEMENT 3Results Disclosure 2

Individual participant level datasetSystematic review after the trial

DATADATA

Ottawa Statement

Results reporting PROCTOR 2008

• Build research on all knowledge, not just partial

• Current system is suboptimal and inefficient in the Internet era

• Without results reporting trial registration is an unfulfilled promise

• Perform secondary analysis

CMJ 200810.3325/cmj.2008.2.267KKJ

PROCTOR 2008: Public Reporting Of Clinical Trials Outcomes and Results

Design Conduct Analysis

Trial Registry

Results Database

Pro

toco

l Ele

men

ts

Updates

Aggregate Data

Final Update

Results and Findings

Raw

Data- IP

DTrial Data Repository

cross-referenceA

ll Trial D

ocume

ntatio

n

KKJ

Excitement

Hesitation

KKJ

Intellectual property, TRIPS

Resources

CitabilityAvailability

Privacyanonymization

data format standards accuracy quality

Repositories and data exchange standards

Barriers

Academic credit

Prepublication

KKJ

Public Disclosure of Results

Standards (to be developed)

Protocol standards (SPIRIT)

Trial registration (WHO/internatio

nal) standards

Publication reporting standards

(CONSORT)

Better & more efficient trials

KKJ

KKJ

KKJ

IMPACT (2013)(IMProve Access to Clinical Trial data)

• Explore definitions of raw data, meta-data data formats, privacy repositories, access to data for re-analysis and pooled

analysis• Develop

methodologies, standards, guidelines• Inform

operationalisation, implementationhttp://ottawagroup.ohri.ca/disclosure.html

KKJ

Environmental scan of repositories

Objective • Identify and explore essential features and practices of

repositories that • accept clinical trial data including IPD• facilitate their public disclosure

Use/ KT • Inform IMPACT (IMProve Access to Clinical Trial data)

• Raise awareness• Help plan the future

Funded by the COPE grantKKJ

Methodology

KKJ

Selection/inclusion criteria Search Data collection

Repositories that harvest clinical trial raw data with a goal of enabling sharing and public disclosure

InternetRegistries of data repositoriesLiteratureOther sources

Define features-develop headingsExtract data from websites; Interview managersLiterature search (a)repositories(b) methodologies

Data repositories

KKJ

necessary to achieve the full transparency of research open data

store and maintain

data

provide access to

dataarchive

share, reuse

KKJ

Databib n+598re3data n=608

Manual search n=3

Repositories that accept human

datan=93

Repositories excludedDuplicatesInclusion criteria not met

Repositories that accept clinical

researchn=11

Repositories that accept clinical

trialsn=5

Repositories that accept raw data from

clinical trialsn=4+5 (?)

Repositories derived from search

Selected repositories

• Dryad • Figshare• MMMP • Databox• ICPSR • YODA• Res Data Australia • Zenodo• Edinburgh DataShare

KKJ

URL

REPOSITORY N

AME

TYPE OF R

EPOSITORY

GEO-COVERAGE

NETWORKIN

G

ACCEPTABLE

SIZ

E

DEFINIT

ION O

F RAW

DATA

REQUIREM

ENTS TO F

ACILIT

ATE MULT

IPLE

SECONDARY U

SES

TYPE OF R

ESEARCH DATA

CONTROL OF Q

UALITY

PARTNERSHIP

CURATED

UPDATES (ARCHIV

E VS. O

VERWRIT

E)

POLICIE

S

CITABIL

ITY O

F DATA

ETHICS G

UIDELI

NES

DE-IDENTIF

ICATIO

N PRACTIC

ES

PRIVACY

GOVERNANCE MODELS

TYPE OF O

WNERSHIP

COST AND F

INANCIA

L STRUCTURE

ARCHITECTURE

SERVICE L

EVEL AGREEM

ENT (SLA

)

GRAPHICAL

USER INTERFA

CE (GUI)

SOCIAL

NOTIFIC

ATION S

YSTEM

API - A

PPLICATIO

N PROGRAM

MIN

G INTERFA

CE

ONLINE H

ELP

DATA F

ORMATS A

ND STA

NDARDS

WHO C

AN UPLO

AD

SECONDARY USE A

CCESS

SCIENTIF

IC A

REA

SCIENTIFIC AREA

• clinical trials• any research

CITABILITY OF DATA

• DOI• URN• ARK• Handle• PURL• Persistent identifier system HDL• ORCID• cite the repository• none

DATA FORMATS AND STANDARDS

No universal standardsLeave to submitter and journal

Range of standards: • Defined • No specific • No specific fields• Any• As is

Formats: • as is• multiple formats to enable accessibility

WHO CAN UPLOAD

• Anyone• Limited (by application)• Closed (no outside institutions)• Certain scientific area (malignant melanoma)• Only if published

SECONDARY USE ACCESS

Openfree to download & no legal barriers to reuse

Limited /Restrictedupon requestqualified researcher

 Conditionsdata use agreement no sharing

Closed responsible organisation only

Combination

PARTNERSHIP

Journals /publishersDryad- BMJFigshare- PLoS, Faculty 1000

ORCID- 37 partners

Funding agencies

Governments

Consortia

PharmaYoda-Medtronic

TYPE OF RESEARCH DATA

• publications • any standard file• images, graphs, video• multimedia & audiovisual• datasets• outcome data• summary data • raw data• individual data elements for each

participant

CONTROL OF QUALITY

Extensively curated

To certain extend

As is

ARCHITECTURE

GUI - rare

Online analysis

Report templates

Social notification

Social commentary

Repositories

• Directories/Registries of research data repositories re3data , Databib

• DataCite • Repositories that take clinical trial data are

General Country/region Institutional Topic

• No universally accepted standardsKKJ

Barriers, opportunities, trends 2013

• Improvement and development needed

KKJ

Directories/Registries of repositories Repositories

Description / categorisation of repositories Features

Search engine Policies

Duplication: same repositories in R3data and Databib

Standards

Coordination btw re3data and Databib No domain repository for CT data

Barriers, opportunities, trends 2015• Number of data repositories increase from 600 in May 2013

to 1275 in June 2015. but

• No new repository hosts clinical trial raw data• No domain repository for clinical trial data

• re3data, DataCite and Databib partner in 2014• re3data & Databib merged in March 2015

• DataCite assigns persistent identifiers -DOI• DataCite manages re3data from May 2015• RDA –Research Data Alliance

KKJ

Barriers, opportunities, trends 2015

• Improvement and development needed

KKJ

Directories/ registries of repositories Repositories

Description / categorisation of repositories

Features

Search engine Policies

Standards

No domain repository for CT data

IMPACT Observatory

IMPACT-IMProving Access to Clinical Trials data; Observatory of ongoing transition in clinical

research regarding data sharing, opportunities and barriers

KKJ

Observatory or natural experiment

• Experiment is not controlled by researcher• No intervention by researcher • Identifies changes/ transition • Points to barriers and facilitators of …• Measures the impact of changes on…• Informs and thus indicates trends

KKJ

IMPACT Observatory

KKJ

explore development in the past

examine ongoing changes of transparency of CT data

measure the impact of changes on clinical research

establish potential future development

Methodology

KKJ

barriersopportunities

gaps

Scoping review

Websites & literature

Surveys & Interviews

Initiatives

Repositories

Indicate trends

Impact transition

Public release of findings

Issues

• no specific MESH terms• no domain repository • no standards for public disclosure and reuse of

data. • Observatory: complex; numerous players

KKJ

Res

earc

her

s

Regulators

Fu

nd

ers

JournalsPublishers

Pharmaceutical IndustryAcademia

Co

nsu

mer

Ad

voca

tes

Med

ia

Data Repositories

PolicyCulture

Infrastructure

Acc

essPreservation

Kn

ow

ho

w

Complexity of observatory

Initiative related issues and how we deal with them

• Initiatives Not much published

KKJ

Internet, contacts

Selected Initiatives

• IOM Report• RECODE Conference- Athens• BioCADDIE- BD2K• RDA• BioCaDDIE Group at RDA• Repository group at RDA

KKJ

Survey

• Monkey survey • Sample: corresponding authors of CTs

published in 2013• Responded: 2250• Limitation:

-low response rate although high number• Peek into some preliminary Results

KKJ

Peek into some preliminary results

• Trial registration: 60 +8% registered/ 22% did not/ 7% did not know

Published studies• Summary results: 21% public….but• Data sets from published studies

last : 58% my computer/43% org web/ 4% sent to journal

earlier: 11% yes/ 35% upon request /53%no

KKJ

Expected results

KKJ

impact the transition

indicate potential trends

identify barriers, gaps and opportunities of opening clinical trial data

identify tendencies of transitions, dynamics and roles of different players

Potential long term outcomes

IMPACT Observatory is established as a longitudinal study/ tool to measure the • Impact of interventions by many players on clinical

trial research How it is done How it is reported How data are shared How secondary analysis is done

• Indicate trends• Establish future developments

KKJ

Who is IMPACT Observatory

• We are - anyone interested• IMPACT Observatory team

• Everything we do depends on collaboration

• Multi nodal network• Network of people/groups of people with good

ideas: OG, PROCTOR, IMPACT, Cochrane, RDA, ECRIN

KKJ

IMPACT Observatory team 2015

KKJ

Karmela Krleža-Jerić, NEWFELPRO, Dpt on Research in Biomedicine and Health, Univ of Split School of Medicine, Split, Croatia

IMPACT Observatory

Thank you

http://ottawagroup.ohri.ca

KKJ

IMPACT Observatory

KKJ

explore development in the past

examine ongoing changes of transparency of PR data

measure the impact of changes on quality/ knowledge management

establish potential future development

KKJ

KKJ

What did you do with datasets of last published trial

KKJ

Saved on my computer Saved in my organisation’s database or website

Sent to a Journal along with the manuscript

Data was saved by sponsor/funding agency

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

Last published trial - what did you do with the datasets/raw data/individual patient data?

PROCTOR

• PROCTOR: Public Reporting Of Clinical Trials Outcomes and Results

• An international meeting held by CIHR in Ottawa in March 2008

• Wide range of constituencies• No consensus building but identified issues and challenges• Identified a need to develop international standards for public

disclosure of trial results

kkj-CIHR May 2008

Participating constituencies

• Clinicians• Researchers• Systematic reviewers• Consumers• Ottawa Group • Policy makers• Trial registries

• Industry• Ethicists• Medical journal

editors• Public Funders• Methodologists• CONSORT• WHO – PAHO

kkj- Oct- 2009

PROCTOR discussion topics

• why• for whom• when • which trials• how much detail• what format• responsibility

• monitoring quality• accountability• website features• funding• who should run websites• barriers

kkj-CIHR May 2008

kkj-CIHR May 2008

Regain public trust

• Provide global open access to information• Reduce unnecessary duplication• Prevent harmful trials to be repeated• Enable monitoring of adherence to ethical

principles• Provide practical benchmark for

accountability

kkj-CIHR May 2008

WHO International Standards

1. Primary Register and Trial ID #

2. Date of Registration in Primary Register

3. Secondary ID#s

4. Source(s) of Monetary or Material Support

5. Primary Sponsor

6. Secondary Sponsor(s)

7. Contact for Public Queries

8. Contact for Scientific Queries

9. Public Title

10. Scientific Title

11. Countries of Recruitment

12. Health Condition(s) or Problem(s)

13. Intervention(s)

14. Key Inclusion & Exclusion Criteria

15. Study Type

16. Date of First Enrollment

17. Target Sample Size

18. Recruitment Status

19. Primary Outcome(s)

20. Key Secondary Outcomes

PROCTOR discussion topics

• why• for whom• when • which trials• how much detail• what format• responsibility

• monitoring quality• accountability• website features• who should/not fund• who should/not run

websites• barriers

kkj- March 2009

What we need to do to achieve the universal results reporting?

Changes culture of researchers regarding

• data sharing• research of existing data

policies of • regulators • funders• journals • pharmaceutical industry (pharma)• academia

data repositories • features• policies• standards for data accepting, curatorship and sharing

KKJ

Speed up knowledge

creation

BENEFITS OF DATA SHARING

32

45

Enable re-analysis, meta and

pooled analysis

Minimize unnecessary duplication of

research

Overcome publication,

and outcome reporting

bias, selective reporting

Use opportunities

offered by the Internet

1

Build on all knowledge,

not just partial 6

EFFICIENT AND RESPONSIBLE RESEARCH

Data repositories

• store and maintain data• provide access to data• archive• share, reuse• necessary to achieve the

full transparency of research; open data;

KKJ

From: from: http://ottawagroup.ohri.ca:

Protocol

Ethics approval

Recruitment

Results

Publication

Unique IDProtocol registration

Public release of protocol

Recording of amendments

Registration of results

Public release of results

Ottawa statement: general time-line for trial registration process (2004)

• (i) to analyse peer review in different scientific areas by integrating quantitative and qualitative research and incorporating recent experimental and computational findings;

• (ii) to evaluate implications of different models of peer review and to explore new incentive structures, rules and measures to improve collaboration in all stages of the peer review process;

• (iii) to involve science stakeholders in data sharing and testing initiatives,

• (iv) to define collaboratively a joint research agenda that points to an evidence-based peer review reform.

KKJ

How

KKJbaseline

ongoing tendencies

Public sharing of findings

potential development in the future (trends)