evolving risk indicators

Evolving Risk Indicators

Duncan Hall, CEO Triumph Research

Intelligence

The Risk-basedmonitoring company

Founded in 2013 Sister company to Triumph Consultancy Entirely Quality Oversight and RBM focused Creators of OPRA RBM platform One complete solution

Implementation services Study specific services Technology Hosting

Presentation Synopsis

EVOLVING RISK INDICATORSIdentifying Relevant Risk Indicators for Each Stage of the Trial to Optimize Quality Oversight by Mitigating Risk Early and Cost Effectively

Study Design Feasibility Country selection

Site Selection

Site Initiation

Patient Recruitment

Treatment / Monitoring

Why Should RBM Evolve During A Trial?

Internal process External process

Patients not involved Patients involved

Historic data Performance data

Clinical data

What Are The Key Variables?

Data SourcesKRI / KPIsData subsetFrequencyApproach

Study Design

Study Design

Feasibility

Country selection

Site Selection

Site Initiation

Patient Recruitment


Evolution of Study Design

Robust Synopsis Final Protocol

Simplify where possible to reduce risk For the risks we have to accept:

What KRIs will help detect that risk? What KRIs are needed for endpoints / safety? What KRIs / KPIs will drive operational oversight?

Audience Poll #1

Question: What best describes your approach to Protocol risk assessment?

Answer:A. We perform risk assessment early

and revise the protocol accordinglyB. We perform risk assessment but

don’t revise protocolC. We don’t perform risk assessment

Study Design

Study Design

Feasibility

Country selection

Site Selection

Site Initiation

Patient Recruitment


Are all indicators equal? Do you plan to roll up indicators

to give a site score? Are safety indicators more

important than performance?

Indicator ClassificationCommon across all protocols•Safety•Compliance•Data quality

Common across a therapy area/indication•Endoscopy•Mayo scoring system

Specific to the study•Subject response to treatment

Audience Poll #2

Question: What is your approach to different indicator types?

Answer:A. We use different indicators for each

studyB. We re-use some indicators but also

use study specific indicatorsC. We use the same indicators for all

studies

Feasibility

Study Design

Feasibility

Country selection

Site Selection

Site Initiation

Patient Recruitment


Focus is on historic data What indicators have we used previously? How effective were they? What did we learn? Did we see any regional trends or bias? Do we need to adjust monitoring plans accordingly?

Use of Heat Maps Study profiling Overall historic risk bias Specific indicators

Country SelectionStudy

Design

Feasibility

Country selection

Site

Selection

Site

Initiation

Patient

Recruitment

Treatment

/ MonitoringRecommend Country Risk Assessments

Country specific standard of care practice Focused on risk factors, not recruitment /

performance factors May want to consider sub-setting the

country data

Countries in Region 1

Adverse Events Region 1 Adverse Events Region 2

Countries in Region 2

Data Sub-setting

Site SelectionStudy

Design

Feasibility

Country selection

Site

Selection

Site

Initiation

Patient

Recruitment

Treatment

/ Monitoring

Site Risk Assessments Focused on risk factors, not recruitment / performance

factors Have we used the site before? What were the findings and how do we learn and manage

this time? May want to adjust monitoring plans prior to recruitment

Adverse Events Region 2

Studies for site X

Absolute risk score

Relative ranking to other study

sites

How do we quickly assess historic / current use of a site?

Cross Study Site Assessment

Site InitiationStudy Design

Feasibility

Country selectio

n

Site

Selection

Site Initiatio

n

Patient Recruit

ment

Treatment / Monitoring First look at current data

KPIs - doc turnaround times Quality assessments

Multiple data sources Looking for correlation between initiation metrics

and subsequent quality issues May want to adjust monitoring plans prior to

recruitment

Patient RecruitmentStudy Design

Feasibility

Country selectio

n

Site

Selection

Site Initiatio

n

Patient Recruit

ment


Focus on the process of recruitment and screening

Performance and quality considerations Ratios can be good indicators:

Enrolment / screen failure Screen failure / withdrawal

Patient RecruitmentStudy Design

Feasibility

Country selectio

n

Site

Selection

Site Initiatio

n

Patient Recruit

ment


KRI • Does the KRI show risk?Companion

•Review companion metrics

Data

Deeper dives should be

possible at to refine the risk

signal and determine

actions

Adverse Events Region 2

What are the main reasons for

patient discontinuation?

What is the distribution of reasons across all

sites?

What is the distribution for

this site?

Companion Metrics

Drill down to the KRI data

set

What is the distribution for

this site?

Sort, group and export multiple

data sets for comparison

Treatment / MonitoringStudy Design

Feasibility

Country selectio

n

Site

Selection

Site Initiatio

n

Patient Recruit

ment


Focus of most RBM and oversight activity KRI and data sources all now fully utilised Let’s consider the concept of data sub

setting from a time based perspective…


Feasibility

Country selectio

n

Site

Selection

Site Initiatio

n

Patient Recruit

ment


This study made use of endoscopy images taken by sites

Looked at % images either missing or reported more than 10 days after procedure


Feasibility

Country selectio

n

Site

Selection

Site Initiatio

n

Patient Recruit

ment


Sites with poor late endoscopy scores tended to score poorly in other areas

Monitoring corroborated that there were the sites with more issues


Feasibility

Country selectio

n

Site

Selection

Site Initiatio

n

Patient Recruit

ment


Sites highlighted for more monitoring support improved, continued to be flagged as outliers

Looking at data in rolling six month subsets allowed a more current view on what is happening ‘now’

The results were significant:


Feasibility

Country selectio

n

Site

Selection

Site Initiatio

n

Patient Recruit

ment


Dec 2014

Apr 2015

This showed both the efficacy of the monitoring interventions as well as the sub setting of the data

Summary

Clinical trials are by nature dynamic, as should our approach to quality oversight and RBM

RBM starts with Protocol design Different indicators, data sources, data subsets, frequencies and weightings can all

be used to evolve your approach throughout the trial Historic information can help inform selection and oversight of countries and sites Sub-setting of data can help deal with regional skew Companion metrics can help with root cause analysis and fine tuning of KRIs Should be continually evaluating the efficacy of KRIs and adjusting approach as we

learn All decisions and changes to approach need to be documented and justified

Questions / Contact Details

Duncan HallCEO and Founder

By website:www.tritrials.com

By email:[email protected]

evolving risk indicators

Healthcare