svmpharma real world evidence – afraid of non-randomised data?
Post on 14-Apr-2017
52 Views
Preview:
TRANSCRIPT
SVMPharma Ltd, Landmark House
Station Road,Hook,Hampshire, UK,
RG27 9HA
CONTACTUSenquiry@svmpharma.com
+44(0)1256962220www.svmpharma.com
Afraid Of Non-Randomised
Data?
Afraid of non-
randomised data?Does the thought of dealing with non-
randomised data fill you with dread?
Do you feel non-randomised data
collection brings more than its fair
share of problems and challenges?
Wouldn’t you rather invest your time
and resource in the gold standard - a
randomised controlled trial?
If you answered yes to these questions, let’s
look at something that could change your
mind:
Non-randomised data collection has been
revitalised with a landmark publication from
the National Institute of Clinical Excellence
Decision Support Unit (NICE DSU).1
The significance of this report is that it allays
a number of fears consistently raised by
stakeholders2, the report: a) demonstrates
that non-randomised data can be used to
make robust estimates of treatment effect b)
provides the guidance that was lacking and
sought after c) shows that NICE are actively
encouraging and accepting non-randomised
data within HTA submissions.
Non-randomised data falls within real-world
evidence (RWE) – which can be defined as
any clinical data collected outside of a
conventional randomised controlled trial
(RCT).
In RCTs, random allocation of patientsbetween intervention and comparatorensures that factors which could influencethe outcome of interest are evenly balancedbetween treatment groups. The change inoutcome attributable to the treatment (thetreatment effect) is simply the difference inoutcome between the treated and thecontrol groups.
In non-randomised studies, estimates oftreatment effect face an increased risk ofbias. There are a number of methods whichseek to minimise the risk of selection biaswhen estimating treatment effect. Most ofthese attempt to make treatment and controlgroups comparable (via matching or inverseprobability weighting), or control howprognostic factors affect the outcome (e.g.regression adjustment, via propensity scoresor with instrumental variables). Othermethods mimic randomisation with naturalexperiments (e.g. difference-in-difference andregression discontinuity). Finally, there aremore advance methods which modelselection and the effect of treatment on theoutcome jointly (e.g. with structural modelsor the correction approach).1
As you can imagine, with such a vast array of
methodologies, deciding how best to
estimate treatment effect isn’t easy, and it is
especially difficult to judge the quality of the
trials and the data in order to make a
decision. The NICE DSU document details
each of the methodologies above, and
provides an algorithm to inform the best
choice.
2 ©2016 SVMPharma Ltd. All rights reserved
www.svmpharma.com
©2016 SVMPharma Ltd. All rights reserved
NICE recognise the need for harmonisationand standardisation of non-randomisedstudies. The Consolidated Standards ofReporting Trials (CONSORT) statement wasfirst published in 1996, and provides a widelyadopted checklist and guidelines for theconduct of RCTs which have been updatedregularly over the last 20 years.3
But where is the equivalent for non-randomised data?
There have been a number of attempts tostandardise and advise the reporting of non-randomised data. Examples of this include theGood Research for Comparative Effectiveness(GRACE) checklist4 which provides astreamlined and easy-to-use set of questionsthat should be asked of the data. A similarapproach is taken by the Strengthening TheReporting of Observational Studies inEpidemiology (STROBE) statement.5 An in-depth report has also been produced by theInternational Society For Pharmacoeconomicsand Outcomes Research (ISPOR) Task Force.6
NICE DSU have compared and contrastedthese existing publications, identifying thecommon themes (how selection andconfounding biases were minimised,specification of the outcomes equation,heterogeneity in treatment effect,comparability of treated and control groupsand the assessment of uncertainty). NICE DSUhave proposed their own version of thechecklist titled Quality of EffectivenessEstimates from Non-randomised Studies(QuEENS). This is a set of 25 questions whichcover the essential aspects of conducting anon-randomised study- and offers astructured way of assessing non-randomiseddata. 1
1. Data suitability
•You need to consider whether the data available is appropriate to answer the decision problem and to inform the estimation of the parameter of interest for the cost-effectiveness model.
•You need to think about the patient population, the interventions, the comparators, the setting, and any data available on potential confounders.
2. Choice of methodology and implementation
•You need to explain why you have decided to use your selected method and describe how you have gone about using it.
•The NICE DSU algorithm gives you a tool to make the most appropriate choice from the multitude of options available, and further questions are specific to the modelling approach selected.
3. Interpretation in light of existing knowledge
•You should interpret your data against a background of existing data, knowledge, and understanding, being aware of counter-intuitive associations between covariates and the outcome of interest (this may suggest unobserved confounding).
•You would expect differences in the results but consistency between them (or inconsistencies that are easily explained) will give credibility to the results.
4. Analysis using different methods
•You should conduct pre-planned sensitivity analysis with methods that use contrasting approaches (e.g. matching vs multivariate regression) and different implementations of the same method (e.g. for matching, using a different set of variables/ inclusion of interactions/ polynomial terms).
5. Assumptions and transparency
•You should state upfront the assumed mechanism of causality and explain the assumptions of the methods and implications of the results.
•You should ensure transparent and comprehensive reporting of the all analyses conducted in such a way that clinical experts can understand the analysis and validate the plausibility of the results.
The 5 key areas that you need to cover
3www.svmpharma.com
©2016 SVMPharma Ltd. All rights reserved
1. Faria et al. NICE DSU Technical Support Document 17: The use of observational data to inform estimates of treatment effectiveness for Technology Appraisal: Methods for comparative individual patient data. NICE Decision Support Unit 2015. Available from www.nicedsu.org.uk/TSD17%20-%20DSU%20Observational%20data%20FINAL.pdf Accessed Jan 2016.
2. GetREAL. Review of Policies and Perspectives on Real-World Data (RWD). 2015. Available from www.imi-getreal.eu/Publications Accessed Jan 2016
3. Schulz et al. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010. Available from www.consort-statement.org/Media/Default/Downloads/CONSORT%202010%20Statement/CONSORT%202010%20Statement%20-%20BMJ.pdf Accessed Jan 2016
4. Dreyer N. A validated checklist for evaluating the quality of observational cohort studies for decision-making support. graceprinciples.org 2014. Available from www.graceprinciples.org/doc/GRACE-Checklist-031114-v5.pdf Accessed Jan 2016
5. von Elm et al. The STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med 2007. Available from www.ncbi.nlm.nih.gov/pmc/articles/PMC2034723/Accessed Jan 2016
6. Berger et al. A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NCP good practice task force report. Value in Health 2014. Available from www.ispor.org/observational-health-study-use-guideline.pdf Accessed Jan 2016
There has long been need for standardisation of the methodology and assessment of non-
randomised data as there is with the CONSORT Guidelines for RCTs.
NICE DSU have provided comprehensive guidance that incorporates existing efforts whilst
putting forward their own recommendations. By using the checklist and recommendations in
this document you are working to a defined methodology for non-randomised data.
Do you need to be afraid of using non-randomised data within your HTA?
4www.svmpharma.com
SVMPharma is an innovative strategic consultancy, specialising in Real World Evidence (RWE) forthe pharmaceutical industry. SVMPharma generates RWE within UK and Europe through bespokeonline Real World Treatment Evaluators, leading to successful health technology appraisal (HTA)submissions. Clinical trial programmes do not reflect real-world clinical practice and outcomes,RWE supplements and enhances clinical datasets. SVMPharma’s specialist teams focus ondelivering the outcomes that matter to your brand. SVMPharma also provides Global RWE, PatientReal World Outcomes, and Big Data Analytics.
To find out more call +44 (0) 1256 962 220
CONTACT SVMPHARMA
Vaneet NayarDirector SVMPharma
enquiry@svmpharma.com
Tel: +44 (0) 1256 962 220
SVMPharma LtdLandmark HouseStation RoadHook, HampshireRG27 9HA
Visit svmpharma.com
Follow @svmpharma
Follow on Google+
Connect via LinkedIn
Get In Touch
top related