evaluation of the effectiveness of risk minimization measures
TRANSCRIPT
COMMENTARY
Evaluation of the effectiveness of risk minimization measures
Luis Prieto1*, Almath Spooner2, Ana Hidalgo-Simon1, Annalisa Rubino1, Xavier Kurz1 and Peter Arlett1
1European Medicines Agency, London, UK2Irish Medicines Board, Dublin, Ireland
key words—risk; risk management; legislation; pharmacovigilance; pharmacoepidemiology; public health
Received 8 March 2012; Revised 11 May 2012; Accepted 16 May 2012
INTRODUCTION
Before the introduction of risk management planning(RMP), pharmacovigilance was usually considered aseries of process steps starting with data collectionthrough signal detection, risk evaluation, action to pro-tect public health, communication, and then evaluationof the effectiveness of the actions taken (a processstep that was only rarely taken in practice). The intro-duction of RMP has ensured greater proactivity topharmacovigilance and postauthorization benefit riskmanagement. The final and perhaps most difficult stepis the introduction of systematic evaluation of theeffectiveness of risk minimization measures (RMM),thereby demonstrating public health protection.1
Risk management planning
The marketing authorization of a new medicinal prod-uct for human use is granted on the basis of a favorablebenefit–risk balance for its target population and indi-cation. However, not all risks will have been identifiedat the time when an initial authorization is sought andmany of the risks associated with the use of a medicinewill only be discovered or fully characterized afterauthorization.RMP includes a set of pharmacovigilance activities
and RMM. These are based on the safety specification,which describes what is known and not known aboutthe safety profile of a medicinal product.RMP has been implemented in the European Union
(EU) since 20052, and the new EU pharmacovigilance
legislation further embeds RMPs as a key tool inproactive pharmacovigilance.3,4
Risk minimization measures
In practice, RMM contained in RMP are public healthinterventions intended to prevent the occurrence ofadverse drug reactions (ADRs) associated with theexposure to a drug (or to reduce its severity should itoccur).RMM aims to optimize the safe and effective use of a
medicinal product throughout its life cycle. The benefit–risk balance of a medicinal product can be improvedby reducing the burden of adverse reactions or byoptimizing benefit, through targeted patient selectionand/or exclusion and on treatment management. RMMshould therefore guide optimal use of a medicinalproduct in normal medical practice with the goal ofsupporting the provision of the right drug, at the rightdose, at the right time, to the right patient, and with theright information and monitoring.
Evaluating the effectiveness of RMM: a dual-evidence-based approach
Public health interventions aimed at minimizing the riskof a product should be shown to achieve the desiredeffect to reduce the burden of adverse reactions andoptimize health outcomes. In addition, implementationof particular measures may involve a substantial invest-ment of resources, and their performance in healthcaresystems should be assessed. In the event that a particularmeasure proves ineffective, it is imperative that alterna-tive interventions should be identified and implementedbased on best available evidence. In this public healthcontext, the new EU legislation on pharmacovigilance
*Correspondence to: L. Prieto, European Medicines Agency, 7 Westferry Circus,Canary Wharf, London E14 4HB, UK. E-mail: [email protected]
Copyright © 2012 John Wiley & Sons, Ltd.
pharmacoepidemiology and drug safety (2012)Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.3305
explicitly requires the active monitoring of the outcomeof RMM contained in RMP, placing obligations onregulators and industry for this, and thus completes thepharmacovigilance process steps by ensuring that theeffectiveness of regulation is demonstrated.3,4
In this article, we describe an approach to evaluatingthe effectiveness of RMM that builds on the assessmentof two distinct levels of evidence. On the basis of thisapproach, the evaluation of the effectiveness of RMMshould differentiate between the actual implementationof the RMM and the attainment of its final objective(s)(Figure 1). If the RMM is unsuccessful, this strategywillhelp to ascertain whether the interventionwas inherentlyineffective (failure of the intervention concept) or badlydelivered (failure of implementation). The assessmentrequires research encompassing analysis of implementa-tion (process indicators) and traditional epidemiologicresearch addressing the attainment (final outcomeindicators) of RMM.5
Assessment of the implementation of RMM: processindicators
Implementation refers to the course of action taken to putin practice an intervention. Therefore, the assessment ofthe implementation of RMM should start with a detaileddefinition or plan of the intervention(s), including targetpopulation, timeline of activities, and instruments used,and should be followed by an examination of the deliv-ery of the intervention(s) against the definitions/plan inpractice. Quantitative measures of the extent of imple-mentation of the original plan and/or variations in itsdelivery should be provided for this purpose.
To eliminate or avoid sources of bias and errorthat could threaten the reliability and validity of theconclusions, scientific rigor and recognized principlesof research6 should guide the assessment of processindicators. These include, but are not limited to, anappropriate consideration of research objectives, studydesign, sample size and representativeness, operationaldefinition of dependent/independent variables, choiceof measurement instruments, and statistical analysis.7,8
If surveys and/or measurement scales are involved inthe assessment of RMM, appropriate survey methodsand psychometric properties of the instruments involvedshould be considered.9,10 When RMM involve theprovision of information to healthcare professionals,resulting clinical actions should be measured, not justclinical knowledge.11 The structure of healthcaresystems12 should also be taken into account in theassessment of RMM, as differences in the organizationof health systems across countries in Europe caninfluence the practical implementation of RMM andpreclude the comparability of results in multinationalstudies. Last but not least, it is important to highlightthat process indicators should be defined in view of allstakeholders concerned by the implementation of theRMM, paying special attention to the implications forhealthcare professionals and patients.
Assessment of the attainment of RMM objective(s):final outcome indicators
Process indicators, as described earlier, should notreplace but should rather complement the assessmentof the attainment of the objective of the RMM, that
Du
al e
vid
ence
of
RM
M
effe
ctiv
enes
s
Risk
Risk Minimisation Measures (RMM)
Risk Management Plan
Attainment of the RMM objectives
(effects of the RMM)
Final Outcomes Indicators:
ADR occurrence or severity
Post Authorisation Safety Studies (PASS)
Post intervention ADR compared to reference value
Implementation of RMM
Process Indicators: successful
implementation of RMM according to plan psychometrics, etc)
Clinical Knowledge
Clinical Actions
Pre-Post comparison of ADRs
Research protocol (survey methods,
Tim
e
Figure 1. Evaluating the effectiveness of RMM by means of a dual-evidence approach
l. prieto et al.
Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, (2012)DOI: 10.1002/pds
is, to minimize risk. In this respect, process indicatorsshould be considered in conjunction with final outcomeindicators. Information regarding the successfulimplementation of an intervention is useful, but it isnot a guarantee of the accomplishment of the intendedobjective for RMM.Final outcome indicators (i.e., ADR occurrence or
severity) should be the key endpoint when assessingthe attainment of RMM objectives. This will involvethe comparison of epidemiologic measures of outcomefrequency that quantify the happening of interest beforeand after the implementation of the RMM (pre–postdesign). For this purpose, cumulative incidence orincidence density measures13 should be obtained in thecontext of post-authorization safety studies (PASS)aimed at describing patterns of outcome occurrence.Indeed, the newly revised EU definition of PASSincludes “. . .measuring the effectiveness of riskmanagement measures” within it.3
The comparison in a pre–post design of spontaneoussuspected ADR reporting rates may constitute acomponent of the assessment of the attainment ofRMMobjectives. In contrast, it must be emphasized thatfor certain risks, this approach will not be reliable, forexample, where the event has a high background ratein the exposed population. In addition, many factorscan influence adverse reaction reporting rates, andfactors such as stimulated reporting following theimplementation of a RMM will complicate suchanalysis.14
When a pre–post design is not feasible (e.g., in thecase that RMM are put in place before the medicineis marketed), the comparison of an outcome frequencymeasure obtained postintervention against a predefinedreference value (i.e., literature review, historical data,expected frequency in general population, outcomefrequency in the prelicensing clinical trials) may beacceptable. The selection of a particular reference valueshould be appropriately justified. For most RMM,reference values have yet to be validated and, in theshort term, determined by expert consensus and laterby evidence-based approaches.Whatever the study design, scientific rigor and recog-
nized principles of epidemiologic research8,15 shouldalways guide the assessment of the final outcomeindicators of interest. When feasible, the publicationof the assessment results is encouraged to ensure thetransparency of the evaluation and appropriate informa-tion to healthcare professionals and all stakeholders onthe impact of the risk minimization measures. Whenassessment comes from a PASS, results should alwaysbe made public. In the medium term, decisions on theattainment of RMM objectives are likely to require
synthesis of evidence from multiple sources, includingstudies sponsored by the marketing authorizationholder and, for the most important public health issues,independent public-funded research. Methodologicalaspects on the synthesis of diverse sources of evidenceshould be considered,16 and guidelines on systematicreview and, where appropriate, meta-analysis shouldbe followed.The dual-evidence approach introduced here will
constitute the core of the module on good pharmacov-igilance practices (GVP) dedicated to the practicalimplementation of the new EU pharmacovigilancelegislation3,4 on the evaluation of the effectiveness ofRMM (GVP Module: Risk-minimisation measures:selection of tools and effectiveness indicators).17 GVPis a set of detailed guidelines drawn up to facilitate theperformance of pharmacovigilance in the EU.
Concluding remarks
The proposed dual-evidence-based approach is consis-tent with ICH E2E principles, whereby risk minimiza-tion is informed by a safety specification that may evolvethroughout the product life cycle. According to theseprinciples, RMM are introduced based on the availableknowledge at a point in time, and as further informationon the effectiveness of RMM and pharmacovigilancedata emerges, RMM may be adapted based on en-hanced characterization of risk and effectiveness of theRMM.18
The development of an RMP leads to the designand implementation of RMM aimed at preventing orreducing the severity of ADRs associated with exposureto a medicinal product. The evaluation of the effective-ness of these interventions is a crucial aspect ofcontinuous pharmacovigilance and life cycle benefit–risk management, which requires an analysis ofinterventions implemented and whether they have beensuccessful in reducing risk.A dual-evidence-based approach is described here;
however, the guiding principle of the evaluation of theeffectiveness of RMM is to retain a focus on the finaloutcome of the intervention. The assessment of twodistinct levels of evidence provides an advantage overtheir isolated consideration. If the evaluation of theeffectiveness of RMM indicates that the interventionhas not been successful in reducing risk, a dual-evidenceapproach will help to determine whether the lack ofsuccess was caused by a failure in the implementationof the intervention or by a conceptual error in theRMM design. In this sense, the dual approach is asuitable tool to identify whether any change is neededto improve the success of the RMM.
effectiveness of risk minimization
Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2012DOI: 10.1002/pds
CONFLICT OF INTEREST
The authors declare no conflict of interest.
KEY POINTS• Many of the risks associated with the use of amedicine will only be fully characterized afterauthorization.
• The newEU pharmacovigilance legislation embedsRMP as a key tool in proactive pharmacovigilance.
• The development of a RMP leads to the design andimplementation of RMM aimed at preventing/reducing the severity of ADRs associated withexposure to a medicinal product.
• The evaluation of the effectiveness of RMM is acrucial aspect of continuous pharmacovigilance.
• The evaluation of the effectiveness of RMMshould differentiate between the actual implemen-tation of the RMM and the attainment of its finalobjective/s.
AKNOWLEDGEMENTS
We appreciate the comments made on an earlier versionof this manuscript by Jim Slattery and Antonio Addis.We also appreciate the input provided by Hans-GeorgEichler, Michael Berntgen, and Alessandro Spina.Finally, we are grateful to the anonymous peerreviewers for their useful comments to the first versionof the manuscript submitted to the journal.
DISCLAIMER
The views expressed in this article are the personalviews of the authors and may not be understood orquoted as being made on behalf of or reflecting theposition of the European Medicines Agency or oneof its committees or working parties.
REFERENCES
1. Waller PC, Evans SJ. A model for the future conduct of pharmacovigilance.Pharmacoepidemiol Drug Saf 2003; 12:17–29.
2. European Union Legislation - Eudralex. Volume 9A of The Rules GoverningMedicinal Products in the European Union. Guidelines on Pharmacovigilance forMedicinal Products for Human Use. September 2008. http://ec.europa.eu/health/files/eudralex/vol-9/pdf/vol9a_09-2008_en.pdf (accessed 8 February 2012)
3. Directive 2010/84/EU of the European Parliament and of the Council of 15December 2010 amending, as regards pharmacovigilance. Directive 2001/83/EC on the Community code relating to medicinal products for human use.Official Journal L 348, 31/12/2010, p. 74–99. http://ec.europa.eu/health/files/eudralex/vol-1/dir_2010_84/dir_2010_84_en.pdf (accessed 8 February 2012)
4. Commission Regulation (EU) No 1235/2010 of the European Parliament and ofthe Council of 15 December 2010 amending, as regards pharmacovigilanceof medicinal products for human use, Regulation (EC) No 726/2004 layingdown Community procedures for the authorisation and supervision ofmedicinal products for human and veterinary use and establishing a EuropeanMedicines Agency, and Regulation (EC) No 1394/2007 on advanced therapymedicinal products. Official Journal L 348, 31/12/2010 p. 1–16. http://ec.europa.eu/health/files/eudralex/vol-1/reg_2010_1235/reg_2010_1235_en.pdf(accessed 8 February 2012)
5. Rychetnik L, Frommer M, Hawe P, et al. Criteria for evaluating evidence onpublic health intervention. J Epidemiol Community Health 2002; 56: 119–127.
6. Bowling A. Research methods in health, 2nd Edition. Open University Press:Buckingham, 2002.
7. ENCePP. Checklist for Study Protocols. European Network of Centres forPharmacoepidemiology and Pharmacovigilance, London, July 2011. http://www.encepp.eu/standards_and_guidances/documents/ENCePPChecklistfor-Study Protocols.doc (accessed 8 February 2012)
8. ENCePP. Guide on Methodological Standards in Pharmacoepidemiology.European Network of Centres for Pharmacoepidemiology and Pharmacovigilance,London, July 2011. (accessed 8 February 2012)
9. Fowler FJ. Survey Research Methods, 4th Edition. Sage publications: ThousandOaks, California, 2009.
10. Streiner DL, Norman GR. Health measurement scales: a practical guide to theirdevelopment and use, 4th Edition. Oxford University Press: Oxford, 2008.
11. Miller GE. The assessment of clinical skills/competence/performance. Acad Med1990; 65: S63–S67.
12. Hoven JL, Haaijer-Ruskamp FM, Vander-Stichele RH. Indicators of prescribingquality in drug utilisation research: report of a European meeting (DURQUIM,13–15 May 2004). Eur J Clin Ppharmacol 2005; 60: 831–834.
13. Szklo M, Nieto FJ. Epidemiology: beyond the basics. Aspen Publishers:Gaithersburg, Maryland, 2000.
14. Motola D, Vargiu A, Leone R, et al. Influence of regulatory measures on therate of spontaneous adverse drug reaction reporting in Italy. Drug Saf 2008;31: 609–616.
15. RothmanKJ, Greenland S, Lash TL.Modern Epidemiology, 3rd Edition. LippincottWilliams & Wilkins: Philadelphia, 2008.
16. Popay J. Moving beyond effectiveness in evidence synthesis. Methodologicalissues in the synthesis of diverse sources of evidence. National Institute ofClinical Excellence, 2006. http://www.nice.org.uk/niceMedia/docs/Moving_beyond_effectiveness_in_evidence_synthesis2.pdf (accessed 30 April 2012)
17. European Medicines Agency (EMA). Good Pharmacovigilance Practices. EMA, 2012.http://www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2012/02/news_detail_001451.jsp&mid=WC0b01ac058004d5c1&jsenable-d=true (accessed 1 May 2012)
18. ICH. Harmonised Tripartite Guideline Pharmacovigilance Planning E2E. Interna-tional Conference on Harmonisation of Technical Requirements for Registrationof Pharmaceuticals For Human Use, 2004. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E2E/Step4/E2E_Guideline.pdf(accessed 8 February 2012)
l. prieto et al.
Copyright © 2012 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, (2012)DOI: 10.1002/pds