biomedcom evidem a practical framework to support healthcare decisionmaking, assess quality of...

BioMedComBioMedCom

EVIDEM

A practical framework toA practical framework tosupport healthcare decisionmaking, support healthcare decisionmaking, assess quality of evidence, and assess quality of evidence, and define explicit data needsdefine explicit data needs

Mireille M Goetghebeur PhDMireille M Goetghebeur PhD, Monika Wagner PhD, Hanane , Monika Wagner PhD, Hanane Khoury PhD, Randy Levitt PhD, Lonny J Erickson PhD, Donna Khoury PhD, Randy Levitt PhD, Lonny J Erickson PhD, Donna

Rindress PhDRindress PhD

BioMedCom Consultants incBioMedCom Consultants incMontreal, CanadaMontreal, Canada

Evidence and Value: Impact on DEcisionMaking -A practical framework applying Multi-Criteria

Decision Analysis to support healthcare decisionmaking

BioMedComBioMedCom

Healthcare decisionmaking is a complex process that requires Healthcare decisionmaking is a complex process that requires simultaneous integration of numerous disparate types of evidence simultaneous integration of numerous disparate types of evidence and value judgmentsand value judgments

There is a need, nationally and internationally for a transparent There is a need, nationally and internationally for a transparent access to evidence and values on which decisions are based for access to evidence and values on which decisions are based for coverage/use of healthcare interventions. coverage/use of healthcare interventions.

Ad-hoc assessment of a healthcare intervention without breakdown Ad-hoc assessment of a healthcare intervention without breakdown into components may result in loss of information and possibly into components may result in loss of information and possibly biased valuationbiased valuation

Healthcare decisionmaking could be facilitated by structuring Healthcare decisionmaking could be facilitated by structuring evidence and value judgments on which it is based into a practical evidence and value judgments on which it is based into a practical and transparent architectureand transparent architecture

Transparency will enhance understanding of healthcare decisionsTransparency will enhance understanding of healthcare decisions

Background & HypothesesBackground & Hypotheses

BioMedComBioMedCom

Breakdown components of healthcare decisionmaking into Breakdown components of healthcare decisionmaking into practical instruments structuring and quantifying practical instruments structuring and quantifying assessment of healthcare interventions to facilitate assessment of healthcare interventions to facilitate decisionmakingdecisionmaking

Build an iceberg architecture to provide multiple layers of Build an iceberg architecture to provide multiple layers of transparent access to components of decisionmakingtransparent access to components of decisionmaking

ObjectivesObjectives

Ad hoc assessment

Burden of disease Budget impact

Type of medical service Healthcare savings

Improvement of efficacy Equity

Cost-effectiveness Etc…

Ad hoc assessment

Burden of disease Budget impact

Type of medical service Healthcare savings

Improvement of efficacy Equity

Cost-effectiveness Etc…

Ultimately, optimize health by best use of healthcare Ultimately, optimize health by best use of healthcare interventionsinterventions

Intrinsic value of treatment

Value MatrixMulti Criteria Decision Analysis

(MCDA)

Decision

Quality of evidence

Quality Matrix International standards

Decisionmaking body requirementsCriteria of quality

Synthesized evidence

For each component of Value MatrixFully referenced

Extrinsic value

Evidence

Full text source documents (published, registries, proprietary)



(MCDA)

Decision

Quality of evidence





Extrinsic value



(MCDA)

Decision

Quality of evidence





Extrinsic value

Evidence


BioMedComBioMedCom

Conceptual frameworkConceptual frameworkComponents of decisionmakingComponents of decisionmaking

1Tunis SR. Reflections on science, judgment and value in evidence-based decision making: a conversation with David Eddy. Health Affairs. 2007 (26): w500-w515; 2Eddy DM. Clinical decision-making: from theory to practice-Anatomy of a decision. JAMA 1990 (263):441-443.

BioMedComBioMedCom

Quality of of

evidence

Scientific judgment Value judgment

Decisionmaking process

Intrinsicvalue

of product

Extrinsicvalue

of product

1Tunis SR. Reflections on science, judgment and value in evidence-based decision making: a conversation with David Eddy. Health Affairs. 2007 (26): w500-w515; 2Eddy DM. Clinical decision-making: from theory to practice-Anatomy of a decision. JAMA 1990 (263):441-443.

Based on international standards Comprehensive analyses can be

systematized Not highly dependent on evaluator

perspective

Similar value system Requires value judgment Dependent on evaluator

perspective

No shared value systemDependent on

region/institution priorities

Improvement over existing treatment

None Lower

Major Higher

Efficacy

Validity of efficacy trial

Low Small

High Large

Number of patients Etc…

Type of population treated (equity)

Low priority? High priority?

Vulnerable vs productive patients

Etc…

EVIDEM – What?EVIDEM – What?Practical iceberg architecture to support Practical iceberg architecture to support decisionmakingdecisionmaking

BioMedComBioMedCom

Decision

Quality of evidence

Quality Matrix Criteria of quality

International standards Decisionmaking body requirements


Key info for each component of Value Matrix

Fully referenced


Value Matrix Multi Criteria Decision Analysis

(MCDA)

Extrinsic value

Evidence


BioMedComBioMedCom



(MCDA)

Decision

Quality of evidence





Fully referenced

Extrinsic value

Evidence




(MCDA)

Decision

Quality of evidence





Fully referenced

Extrinsic value

Evidence




(MCDA)

Decision

Quality of evidence





Fully referenced

Extrinsic value

Evidence


BioMedComBioMedCom

EVIDEM – Who?EVIDEM – Who?Communication tool to connect stakeholdersCommunication tool to connect stakeholders

Decisionmakers

Data producers

EVIDEM investigators & experts

Transparent

communication

tool



(MCDA)

Decision

Quality of evidence





Fully referenced

Extrinsic value

Evidence




(MCDA)

Decision

Quality of evidence





Fully referenced

Extrinsic value

Evidence




(MCDA)

Decision

Quality of evidence





Fully referenced

Extrinsic value

Evidence


BioMedComBioMedCom

EVIDEM - Why?EVIDEM - Why?Sharing of value judgments and structured multilevel Sharing of value judgments and structured multilevel access to dataaccess to data

Focus on valueSharing of weights & scores

Transparent access

to quality of data and methods

Transparent multilayeraccess to evidence

Limit obstruction of thinking process

by large amount of data

Can be explored more easily

More transparent and understandable

EVIDEM – How?EVIDEM – How?Integrated process – example “Improvement of Integrated process – example “Improvement of efficacy”efficacy”

Value Matrix component T3: Improvement of medical service - efficacy

Extrinsic value

Decision

Value Matrix (MCDA)

Evidence - Full text sources/links

RegistriesPublished

trialsProprietary

data

Meta-Analyses

(Cochrane)

Synthesized clinical evidence

References

Quality Matrix: quality of clinical evidence

Component Questions Comments Score

Clinical data How comprehensive is the clinical program? Are results across trials consistent?Are standard elements relevant and valid?(COMPLETE INSTRUMENT BELOW -1 per comparator)

BioMedComBioMedCom

2

BioMedComBioMedCom

EVIDEM tools – MCDA Value Matrix EVIDEM tools – MCDA Value Matrix 15 components defining intrinsic value of treatment15 components defining intrinsic value of treatment

Cluster Components of value assessment (alphabetical by cluster) Weight

1-5 Synthesized information*

Score 0-3

Value

Quality of dossier and evidence

Q1 Adherence to requirements of decisionmaking body (dossier)

Q2 Comprehensiveness and accuracy of evidence presented (dossier)

Q3 Relevance and validity of evidence (literature and dossier)

Disease impact

D1 Disease severity (risk of death & disability, acuteness)

D2 Size of population affected by disease

Treatment

T1 Current clinical guidelines on treatment or products of same class

T2 Current treatment limitations

T3 Improvement of medical service – efficacy

Independent of product

Structured summary of evidence for

treatment

Scale anchors and guidelines

Value score (linear model)

T4 Improvement of medical service – safety & tolerability

T5 Improvement of medical service – patient reported outcomes, convenience & adherence

T6 Public health interest (prevention & risk reduction)

T7 Type of medical service (symptom relief, prolonging life, cure etc.)

Economics

E1 Budget impact of reimbursing treatment on drug plans

E2 Cost-effectiveness of treatment

E3 Savings with treatment (impact on healthcare spending excluding treatment cost)

Total Aggregated value score (% of maximum score)

*Available information from public domain and manufacturer dossier

Quality Matrix

Components identified from literature review and current decisionmaking processes, and selected to fulfill MCDA guidelines on clustering, completeness, redundancy, operationality & mutual independence (Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14.; National Economic Research Associated. Multi-criteria analysis manual 2005. http://www.communities.gov.uk/pub/252/MulticriteriaanalysismanualPDF1380Kb_id1142252.pdf Accessed Jan2007)

http://www.communities.gov.uk/pub/252/MulticriteriaanalysismanualPDF1380Kb_id1142252.pdf

BioMedComBioMedCom

EVIDEM tools - Quality MatrixEVIDEM tools - Quality Matrix12 types of evidence, 3 criteria of quality12 types of evidence, 3 criteria of quality

Structured based on criteria defining quality and on requirements from more than 20 decisionmaking bodies worldwide ( INTERFACE database. 2000-2008. www.biomedcom.org); Assessment for each component based on instruments derived from quality standards (e.g., CONSORT, CHEC, STROBE, Siegel et al, Mauskopf et al) as well as guidelines and processes from decisionmaking bodies

http://www.biomedcom.org/

BioMedComBioMedCom

Pilot study in Canadian contextPilot study in Canadian contextProof of concept using historical casesProof of concept using historical cases

Historical cases: 10 medicines (cardiovascular diseases, endocrinology, infectious diseases neurology, oncology, ophthalmology,) using data from literature review and manufacturer dossiers submitted to the Canadian Common Drug Review (CDR) and Québec Conseil du Médicament (CM)

QM scores: EVIDEM investigators; VM weights and scores, and feedback on process: Canadian Value Panel

composed of representative stakeholders across Canada (decisionmakers, specialists, generalists, nurses, pharmacists, health economists/epidemiologists)

Feasibility: Algorithm developed to operationalize each cell of the matrices; applicable to any therapeutic areas and jurisdiction

Practicality: 30 min on average to apply VM by stakeholders; about 250 hrs to build the structured package of fully traceable information (quality, synthesized format and full text access) and value scores

Feedback from panelist: Value of EVIDEM in structuring evidence, assessing strengths and weaknesses systematically, and sharing values and value scores

Limitations of pilot: limited access for panelists to underlying source data (electronic interface to be developed); extrinsic components not covered (to be explored)

BioMedComBioMedCom

Pilot study - Value MatrixPilot study - Value Matrix Value assessment example for clusters 1 & 2Value assessment example for clusters 1 & 2

ClusterComponents of value assessment

(alphabetical by cluster)Scale anchors

% Max Value score – Mean (SD)

Quality of dossier and evidence

Q1 Adherence to requirements of decisionmaking body

0=Low adherence3=High adherence

56 (17)

Q2 Comprehensiveness and accuracy of evidence presented

0=Many gaps / inaccurate3=Comprehensive and accurate

72 (14)

Q3 Relevance and validity of evidence 0=Low relevance/validity3=High relevance/validity

78 (17)

Disease impact

D1 Disease severity (risk of death & disability, acuteness)

0=Not severe (minor inconvenience)3=Very severe

100 (0)

D2 Size of population affected by disease 0=Extremely rare disease3=Common disease

22 (27)

Anchors defined from a societal perspective, i.e., optimizing health at the societal levelAnchors defined from a societal perspective, i.e., optimizing health at the societal level Score options translate to 0% (score of 0), 33% (score of 1), 66% (score of 2) and 100% (score of 3) Score options translate to 0% (score of 0), 33% (score of 1), 66% (score of 2) and 100% (score of 3)

BioMedComBioMedCom




Treatment

T1 Current clinical guidelines on treatment or products of same class*

0=Not recommended3= Strong first-line recommendationNA=not available

67 (21)

T2 Current treatment limitations 0=No or very minor limitations3=Major limitations

89 (17)

T3 Improvement of medical service – efficacy 0=Lower efficacy than comparator treatments presented3=Major improvement in efficacy vs comparator

treatments presented

72 (25)

T4 Improvement of medical service – safety & tolerability

0=Lower safety / tolerability than comparator treatments presented

3=Major improvement in safety / tolerability vs comparator treatments presented

61 (14)

T5 Improvement of medical service – patient reported outcomes (PRO), convenience & adherence

0=Worse PRO / lower convenience / lower adherence vs comparator treatments presented

3=Major improvement

50 (18)

T6 Public health interest (prevention & risk reduction)

0=No risk reduction compared to no intervention3=Major risk reductionNA=not applicable

NA†

T7 Type of medical service (symptom relief, prolonging life, cure etc.)

0=Minor service3=Major service

72 (14)

Pilot study - Value MatrixPilot study - Value Matrix Value assessment example for cluster 3Value assessment example for cluster 3

BioMedComBioMedCom




Economics

E1 Budget impact of reimbursing treatment on drug plans

0=Strong impact on drug plan budget (significant additional expenditures)3=Substantial savings for drug plans

61 (25)

E2 Cost-effectiveness of treatment 0=Not cost-effective3=Highly cost-effective

89 (17)

E3 Savings with treatment (impact on healthcare spending excluding treatment cost)

0=Substantial additional healthcare spending, excluding product cost3=Substantial savings in healthcare spending, excluding product cost

67 (21)

Pilot study - Value MatrixPilot study - Value Matrix Value assessment example for cluster 4Value assessment example for cluster 4

BioMedComBioMedCom

Potential applicationsPotential applications Transparent access to evidence and valuesTransparent access to evidence and values

EVIDEM framework

BioMedComBioMedCom

Adapt framework to existing

decisionmaking processes

Basis to develop extrinsic value components

Application axisProspective

Application axisRetrospectiveValidate process

in various jurisdictions

Explore context of past decisions

Generate data on quality of evidence

Planning tool for researchers and intervention developers to meet

explicit needsDevelop methodology to generate data tailored to critical data needs

Collaboration axisResearch planning

Define explicit needs of decisionmakers

Establish reasonable requirements

Collaboration axisExplicit data needs

BioMedComBioMedCom

Future developmentsFuture developments

Collaborative studies and iterative processes to explore the value of EVIDEM in context

Interactive web based architecture integrating evidence and value for various healthcare interventions

Expected outcome of a systematized and shareable approach for data access and value assessment is to optimize resources, decisions and health

BioMedComBioMedCom

AcknowledgmentsAcknowledgments

EVIDEM Canadian Value panelEVIDEM Canadian Value panel: : Jean-Francois Bussières BPharm MSc MBA FCSHP, Director, Pharmaceutical Practice Research Unit, Dept of Pharmacy, CHU Sainte-Justine Research Center; Clinical Associate Professor, Faculty of Pharmacy, University of Montréal.

Benoit Cossette BPharm MSc, Pharmacist, Coordinator, Drug Disease Management Program, Fleurimont Hospital, Sherbrooke University Hospital Centre.

Doug Coyle PhD, Associate Professor, Faculty of Medicine, Epidemiology and Community Medicine, University of Ottawa.

Cheri Deal MD PhD FRCPC, Associate Professor and Program Director, Endocrine Service, CHU Sainte-Justine Research Center, University of Montréal.

Roland Grad MD CM MSc CCFP FCFP, Associate Professor, Department of Family Medicine, McGill University.

Christine Lee MD, FRCPC Assistant Professor, Director of the Microbiology Residency Program, Department of Pathology and Molecular Medicine, McMaster University.

Mitchell Levine MD FRCPC FISPE, Professor, Director of Centre for Evaluation of Medicines Department of Clinical Epidemiology & Biostatistics, Department of Medicine, McMaster University

Diane Lowden RN MSc, Clinical Nurse Specialist, Montreal Neurological Institute & Hospital, McGill University.

John Mancini MD FRCP, Professor/Program Director Continuing Medical Education, Department of Cardiology, University of British Columbia.

Paul Oh MD FRCPC, Medical Director of the Cardiac Rehab and Secondary Prevention Program, Department of Medicine, Clinical Pharmacology and Toxicology, University of Toronto.

Genevieve Tousignant N MScN, Clinical Nurse Specialist, Montreal Neurological Institute & Hospital, McGill University.

Wendy Ungar PhD, Senior Scientist, Child Health Evaluative Sciences, The Hospital for Sick Children; Associate Professor, Health Policy, Management, and Evaluation, University of Toronto.

Marie-Claude Vanier BPharm MSc, Clinical Associate Professor, Faculty of Pharmacy, University of Montréal; Pharmacist UMF-GMF Cité de la Santé de Laval.

Funded by an unrestricted research grant from Pfizer CanadaFunded by an unrestricted research grant from Pfizer Canada

BioMedComBioMedCom

Thank youThank you

EVIDEMEVIDEM tools are open accessEVIDEM tools are open access

We welcome input and collaborationsWe welcome input and collaborationsPlease contact the EVIDEM Group @Please contact the EVIDEM Group @

www.evidem.org

[email protected][email protected]

biomedcom evidem a practical framework to support healthcare decisionmaking, assess quality of...

Documents