planning and experiences with risk based monitoring
TRANSCRIPT
Planning and experiences with
Risk Based Monitoring
M ONDAY 25 JAN 2016 16 .10 -16 .50
THOM AS COQUELLE , M SC PHARM
About me
MSc Pharm 2000
15 years working in Clinical Research
6-7 years working for Lundbeck in Global Study
Management latest as Principal Clinical Study
Manager
Recently started as International Project Leader in
TFS – Trial Form Support.
WHAT IS RISK BASED MONITORING?
No visibility
No
assurance
about the
quality
before the
harvest
Using a lot
of resources
Conventional
WHAT IS RISK BASED MONITORING
Look for
trends and
signals
Target our
actions and
use our
resources
wisely
Detailled
knowledge
about quality
before data
harvest
Risk Based Monitoring
Background
Mitchel et al. Aug 01, 2013 Applied Clinical Trials
(http://www.appliedclinicaltrialsonline.com/risk-based-approaches)
Defensive
practices (SDV,
many site visits)
for monitoring of
clinical trials have
been adopted.
fear based on
perception and interpretation of regulatory
requirements
Current processes play a
large role in high costs without
delivering reasonable
value
Propagation of practices based on paper CRFs without considering optimization of the processes
to take advantage of the technology provided by EDC
1996
institutionalization /prisoner of high cost, unproven
value, manpower-intensive practices that do little to
serve the interests of subject safety or quality data
TIME TO RETHINK
Mitchel et al. Aug 01, 2013 Applied Clinical Trials (http://www.appliedclinicaltrialsonline.com/risk-based-approaches)
Authorities reacted
the MHRA (Oct 2011), FDA (Aug 2013), EMA (Aug
2011) and NICE published guidance/reflection paper.
Trancelerate
TransCelerate BioPharma Inc. was launched in 2012 as a non-profit organization to improve the health of people around the world by simplifying and enhancing the research and development of innovative new therapies.
Members: Abbvie, Allergan, Amgen, Astellas, AstraZeneca, Boehringer-Ingelheim, BMS, EMD-Serono, GSK, J&J, Lilly, medgenics, Merck, Novo-Nordisk, Pfizer, Roche, Sanofi, Shionogi, UCB
Partners with Societies/Associations representing CROs, Clinical sites/Clinical Trial Standards and collaborating with FDA, EMA and PDMA.
Five strategic priorities:
Improve Site Investigator Experiences
Facilitating sharing of information
Enable harmonization of Clinical Trial Processes
Enhance sponsor efficiencies
Improve patient experience
http://www.transceleratebiopharmainc.com/
Trancelerate and Risk
Based Monitoring
The Risk Based Monitoring Initiative was
established in 2012 as one of the five initial goals
created by TransCelerate to drive efficient and
effective solutions into the R&D industry.
May 2013 Position paper: Risk-Based Monitoring
Methodology.
http://www.transceleratebiopharmainc.com/
Definitions
Critical data and processes: Data or processes should be prospectively be identified, that if performed inaccurately, not performed, or performed incorrectly, would threaten the protection of human subjects or the integrity of the study results.
Critical data and processes may include:
Data that support primary and key secondary objectives
Data critical to subject safety (e.g. serious adverse events, other events leading to discontinuation of treatment)
Processes that underpin subject safety and ethical treatment (e.g. seeking appropriate medical consultation or scheduling extra visits/procedures in the event of significant clinical or laboratory findings)
Processes that underpin data quality (e.g. blinding, referring events for adjudication, controlling inter-rater variability)
Food and Drug Administration Aug 2013: Oversight of Clinical Investigations - A Risk-Based Approach to Monitoring
Trancelerate Position Paper: Risk-Based Monitoring Methodology 4, FINAL 30May2013
Definitions
Source Data Verification (SDV) is the process by which
data within the CRF or other data collection systems
are compared to the original source of information
(and vice versa) to confirm that the data were
transcribed ccurately (i.e. data from source matches
data in the CRF or other system and vice versa)
Position Paper: Risk-Based Monitoring Methodology 4, FINAL 30May2013,
Definitions
Source Data Review (SDR) involves review of source
documentation to check quality of source, review
protocol compliance, ensure the Critical Processes and
source documentation (e.g. accurate, legible,
complete, timely, dated) are adequate, to ascertain
Investigator involvement and appropriate delegation,
and assess compliance to other areas (e.g. SOPs, ICH
GCPs). SDR is not a comparison of source data against
CRF data. SDR is necessary to evaluate areas that do
not have an associated data field in the CRF or system
available for more timely remote review.
Position Paper: Risk-Based Monitoring Methodology 4, FINAL 30May2013,
Definitions
Mandatory data (Informed consent, SAEs, safety scales)
ACTIVITIES RELATED TO RBM
Risk assessments and categorization
Study Risk Categorization
Request for proposal and work order
Critical Data and Processes
Targeted SDV module in electronic CRF
Risk Based Monitoring plan
Monitoring Plan
Risk Based Monitoring review meetings
CRO
Selection
FPFV
Risk Assessment and level
source data monitoring (example)
Study/program category
based on decision tree
SDV
mandatory
data
SDV
Critical
data
Source
Data
Review
A
Randomized Controlled
Trials (RCTs) with regulatory
purpose and/or studies with
high risk/impact
100% 30% 30%
B RCTs other than category A 100% 20% 20%
C Pragmatic or open-label
studies 100% 10% 10%
D
pharmacoepidemiological
NIS but with non-routine
diagnostic or monitoring
procedures
100% 0% 0%
(%of patients
pr. site)
Example category C study
in primary care
Alcohol Dependence
5 major countries in Europe (DE, UK, FR, IT, and ES)
Approx 70 primary care sites (GPs)
Target 475 patients (There should be 95 in each
country)
Medidata Rave as eCRF
SQM and TSDV modules
CATEGORY C
ON-SITE MONITORING EXPECTATIONS
Study/program category
based on decision tree
SDV
mandatory
data
SDV
Critical
data
Source
Data
Review
C Pragmatic or open-label
studies 100% 10% 10%
(%of patients
pr. site)
100% of patients/site:
Mandatory Data
(Informed consent, SAEs)
10% of patients/site: Critical data
(Primary endpoint, Key secondary
endpoints, Selection Criteria,
AE, IMP Compliance)
10% of patients/site reviewing compliance
to the protocol
Study team decided no overlap with SDV
patients Sudy team decided to have SDR for the first
patient
SDV algoritm programmed into
targeted SDV-module in eCRF
CRITICAL DATA AND PROCESSES
Type Data Process TSDV in eCRF
Primary /
secondary
Alcohol
consumption data TLFB interview TLFB calendar
Secondary Clinical Global
Impression score
Clinical Global
Impression rating
Clinical Global
Impression form
IMP compliance IMP data TLFB interview / IMP
accountability
TLFB calendar
Safety endpoint adverse events safety reporting AE forms
Selection criteria
Vital signs, medical
history, recent
medication, illegal
drugs
Vital signs and
collection of
demographics
Relevant forms
Secondary Liver function tests Blood samples to
central lab
Not applicable –
followed by
Medical monitor
Note: Medical monitor is reviewing all critical data (Centralized monitoring)
Monitoring activities
Monitoring activities planned pr. WORK ORDER
3 onsite visits pr. site
2 remote visits pr. site
Assumes 20% the sites would need for-cause visit
Monitoring activities planned in SMT
Medical monitoring / Safety Review
SQM (System tool)
RBM review
Monitoring visit schedule
in monitoring plan
Montoring visit type When Monitoring includes
Pre-check: remote Data Review
1st patient enrolled (V2 done)
Remote Data Review
On site MV#1
Within 2 weeks of 2nd
patient enrolled (V2 done)*
SDR 1st patient + part of tSDV 2nd patient. Focus to
review processes (Inclusion/Exclusion criteria, ICF,
AEs/SAEs, staff resources, IMP inventory and
accountability, lab kits, ISF, findings from data review, TLFB, CGI, PSS, PROs)
Remote MV#1
3rd patient completed
(Week 16 – visit 7 safety follow up)
Process review (see also 21.2/ 3.b from remote monitoring section)
On site MV#2 6th patient enrolled (V2
done)
remaining SDR first patient + remaining tSDV of
patient#2 in Critical Data + SDV on all mandatory
data for all patients. Review processes with site.
Reassess recruitment plan + confirm new IMP shipment)
Remote MV#2 9th patient enrolled (V2
done) Process review (see also 21.2/ 3.b from remote
monitoring section)
On site MV#3
After last patient
completed (including
follow-up period)
SDR on 11th patient + tSDV on critical data for new
randomly selected patient (if applicable) + process
review
*If the 2nd patient is not screened before the 1st patient is completed, then move on with the 1st MV after the 1st subject has been completed.
This at least will enable a full SDR of the first patient
Risk Based Monitoring plan
Definition of study specific risks (Risk indicators)
Critical Data Variables and Processes
Communication at on-site visits and remote
monitoring visit (CRO)
Remote Monitoring (Centralized Sponsor Review)
Regular RBM Review Meetings
Risk indicators
Risk indicators can be derived by operational or clinical data from EDC.
Risk indicators can also be obtained by selected reviewers (Medical monitor, data manager, CQA, safety reviewes, onsite monitors)
Risk Indicators should always be measured in red, yellow green either by thresholds calculations by the system or the the responsible reviewers make a professional evaluation of coloring red/green.
Primary endpoint
Data entry time
Health Economic data
Health Econo-
mic data
Risk indicators
# Risk Indicator Category Risk Level Risk Indicator (How to
measure): Threshold (Green,
Yellow, Red) Responsible
1 Visit Data to CRF Completion Date
High SQM tool: Visit to Data Entry Time
Automated System tool
Data Manager
2 Data Quality-Excessive Queries, low queries outliers
High
SQM tool Query Rate Automated System tool
Data Manager
3 Excessive Query Response Time
High SQM Tool Query Response Time
Automated System tool
Data Manager
4 Screen failure rate higher or lower than anticipated
High SQM Screen Failure Rate metric
Automated System tool
Data Manager
5 Outliers and trends in subject discontinuation
High
SQM: Early Termination Rate
Automated System tool
Data Manager
Qlikview MMSR/eCRF: Reason for Withdrawal
feedback from Medical Expert
Medical Expert
6 Outlier or trend in number of Adverse Events per subject per site
High Safety Advisor will provide reports from MMSR of AEs and SAEs.
feedback from Safety Advisor
Safety Advisor
7 Protocol Deviations High Monitor IPD frequency, Other PD trends (IPD, PD Log)
Feedback from CRO CRO
Additional risk indicators
added after 1st RBM
review # Risk Indicator
Category Risk Level
Risk Indicator
(How to
measure):
Threshold (Green,
Yellow, Red) Responsible
8 Data Manager High
Feedback on
data
entry/queries
Subjective
evaluation Data Manager
9 Medical Monitoring
feedback
High
Feedback from
Medical
Monitoring
Subjective
evaluation Medical Expert
10 MMSR feedback High Feedback from
Safety Review
Subjective
evaluation Safety Advisor
11 RQM feedback High
Feedback from
spotchecks/site
visits
Subjective
evaluation Quality Manager
12 CRO feedback High
General
feedback from
remote and
onsite monitoring
Subjective
evaluation CRO (Lead CRAs)
RBM plan:
Communication with
investigators
SDV/SDR Process reviews
on-site or remote
RBM plan:
Communication with
investigators
Process reviews
on-site or remote
Process review: Ask if the study supplies are adequate (IMP and other supplies such as lab kits) Ask for follow up on eTMF/iTMF
documents (for issues identified via site visits and spot check assessments) Discuss any findings from data review to assess issues/risks and evaluate action needed Obtain feedback from PI/Sis/raters,
whether there are questions/doubts about endpoint assessments other key assessments
Confirm Informed Consent Process
review: All subjects have signed and dated current IRB/IEC approved version of informed consent This has been obtained by all subjects prior to the start of data collection
Consent discussion and signature has been performed by delegated staff member listed on the site signature/responsibility log Subject received a full copy of the signed ICF
Discuss Inclusion/Exclusion criteria have been met by all subjects in the study Discuss procedire for AEs/SAEs reporting Discuss about IMP procedures for
IXRS, dispensing and accountability are being followed. Discuss the Source Data requirements are being followed. Confirm that site staff has adequate time and resources for the study. If new staff hired, discuss training
needs.
After SDV/SDR/Remote monitoring of systems and processes, the CRA should
follow up with the investigator(s)/ site staff to
review any questions and issues. Process reviews should
also be performed by the CRA in a proactive fashion on
topics/processes where there are no immediate issues
identified during the monitoring
to confirm that the investigator are following the correct
procedures.
RBM tracker – visual map
Data
Manager
Quality
Manager Safety
Advisor Medical
Expert CRO
Risk indicators
Int.Study
Manager
Action / Outcome
RBM tracker – example green
site #1
Data Manager Medical Expert Safety Advisor CRO
RBM tracker – example green site #2
Medical Expert Safety Advisor CRO CRO
Quality Manager
Trial Manager
RBM tracker – example red/yellow site #1
Data Manager Medical Expert Safety Advisor
CRO Medical Expert
RBM tracker – example red/yellow site #2
Safety
Advisor CRO CRO Quality
Manager Trial Manager
RBM tracker – risk evaluation site XX002
16-Oct: Site enrolled 5 patients in one day
RBM review
date
Risk indicators/feedback Recommended actions Outcome
12-nov-2014 9 IPDs identified, No alcohol data entered. Lack of
understanding procedure for conducting patient
interview (no telephone allowed), eligibility criteria
and screening procedures not followed, IMP
assignment before inclusion visit.
Retraining needed and PI need
to prioritize study.
For-cause visits and
accompanied visit with LM.
Email from ISM – IVRS turned off
i.e new necruitment not
allowed
Site difficult to reach
OSV planned 24/nov
Additional OSV 9/dec
performed
16-dec-2014 System triggers now red, Site visits 24/nov and 9/dec
No source documentation in Scales binder, Medical
History , Physical Exam. New Important PDs identified.
reconsent of patients not followed up. Lack of Source
documentation continues and there is huge data
entry backlog.
Request for Quality Manager
visit.
14-jan-2015 System triggers for data entry time red. Open queries
up to 11w old, still a lot data missing in eCRF – difficult
for medical monitor to monitor compliance. No
progress is seen,
CRO is trying to setup weekly
calls – but is difficult.
Two calls with PI in
Janury but there was
not enough time to
resolve outstanding
issues
11-feb-2015 System triggers data entry time and query response
time are still red. Missing data entry from many follow
up visits. Treatment compliance is good. Quality
manager feedback from Site Visit 22/jan is that there
is poor source data to non-existing - many data
entered directly into the eCRF, much later than the
visits toke place. PI instructed the patients to take one
table every day, which is not according to protocol.
IMP return was checked by CRA and not PI. PI expect
the CRA to be on site and resolve all issues. Difficulities
to use eCRF. No progress notes for CGI.
After this review SMT decided to
inform the site that no further
recruitment will be allowed.
ISM wrote email to the site.
Still a lot of outstanding issues
with this site.
Site quality was not
increased, however
the contibution of
poor data were
mitigated rapidly
WHAT ABOUT THE MONEY?
Cost savings RBM
Total direct fees Total direct fees
Total pass-throughs Total pass-throughs
Risk Based Monitoring Full Monitoring
= 291kEUR
= 62 kEUR
Total costs Total costs
= 722 kEUR
= 187 kEUR
= 353kEUR = 909 kEUR
Total cost saving*: 500 to 600 kEUR * Potential effect of changed monitoring strategy on site management costs and project management costs is not included
in calculation
Other CRO costs
Risk Based Monitoring costs
Additional cost (Full monitoring)
Current CRO costs (w/RBM): 1.499 kEUR
Additional monitoring: 500-600 kEUR
Total CRO costs (w. full m): 1.999-2099 kEUR
Estimated savings from
RBM:
25-29 %
Risk Based Monitoring – is it worth it
Thank you - questions