What Randomized Clinical Trials Are Possible / Necessary

In Phlebology

Mark H. Meissner, MDMark H. Meissner, MDProfessor of SurgeryProfessor of Surgery

University of Washington School of MedicineUniversity of Washington School of Medicine

Levels of Evidence for Therapeutic StudiesLevels of Evidence for Therapeutic StudiesStraus SE, Evidence-Based Medicine 3rd EdStraus SE, Evidence-Based Medicine 3rd Ed

Level of Level of EvidenceEvidence

Studies of therapy, prevention, etiology, harmStudies of therapy, prevention, etiology, harm

1a1a Systematic review with homogeneity of RCTsSystematic review with homogeneity of RCTs

1b1b Individual RCT with narrow confidence intervalsIndividual RCT with narrow confidence intervals

1c1c All or noneAll or none

2a2a Systematic review with homogeneity of cohort studiesSystematic review with homogeneity of cohort studies

2b2b Individual cohort study or low quality RCTIndividual cohort study or low quality RCT

3a3a Systematic review with homogeneity of case-control studiesSystematic review with homogeneity of case-control studies

3b3b Individual case-control studyIndividual case-control study

44 Case seriesCase series

55 Expert opinion without explicit critical appraisalExpert opinion without explicit critical appraisal

What Do We Really Care About?What Do We Really Care About?

Incorporation of societal valuesIncorporation of societal values Societal costsSocietal costs Comparative effectiveness of different technologiesComparative effectiveness of different technologies

The best available estimate of benefits and harms The best available estimate of benefits and harms (estimate of treatment effect)(estimate of treatment effect)

Application of the evidence to the individual patient Application of the evidence to the individual patient (generalizeabilty)(generalizeabilty)

Where Does Clinical Evidence Come From?Where Does Clinical Evidence Come From?How Do We Measure the Magnitude of Effect?How Do We Measure the Magnitude of Effect?

Semi – experimentalSemi – experimental

Comparison with historical controlsComparison with historical controls

Fatally biasedFatally biased

Observational studies (all with concurrent controls)Observational studies (all with concurrent controls)

Cross sectional - Compares proportion with disorder based Cross sectional - Compares proportion with disorder based on exposure at one point in timeon exposure at one point in time

Cohort studies – Prospective evaluation of outcome based on Cohort studies – Prospective evaluation of outcome based on exposureexposure

Case - control studies – Retrospective evaluation of Case - control studies – Retrospective evaluation of exposure based on outcomeexposure based on outcome

Randomized, controlled clinical trialsRandomized, controlled clinical trials

Determinants of Evidence QualityDeterminants of Evidence Quality

DeterminantDeterminant DefinitionDefinition QualityQuality BiasBiasTreatment Treatment

EffectEffect

Systematic Review of RCTsSystematic Review of RCTs HighHigh LowLow PrecisePrecise

Randomized Clinical TrialsRandomized Clinical Trials

Observational StudiesObservational Studies

MethodologyMethodology Cohort StudiesCohort Studies

Case-Control StudiesCase-Control Studies

Case SeriesCase Series UnknownUnknown

Expert OpinionExpert Opinion LowLow HighHigh UnknownUnknown

ConsistencyConsistency

Directness

Similarity of treatment effect across studiesSimilarity of treatment effect across studies

Appropriateness of groups and outcomes

RCTs – The Holy GrailRCTs – The Holy Grail

Require true clinical equipoise (RR 0.4 – 0.9)Require true clinical equipoise (RR 0.4 – 0.9) Difficult to justify if observational studies showDifficult to justify if observational studies show

Large harmful effectsLarge harmful effects Large (risk ratio < 0.4) beneficial effectsLarge (risk ratio < 0.4) beneficial effects Small beneficial effects (risk ratio 0.9 - 1.0)Small beneficial effects (risk ratio 0.9 - 1.0)

Are expensiveAre expensive May be difficult to generalize (Restrictive inclusion criteria)May be difficult to generalize (Restrictive inclusion criteria) Usually not powered to detect harms of treatmentUsually not powered to detect harms of treatment May be better, worse, or complimentary to observational May be better, worse, or complimentary to observational

studiesstudies

Why are RCTs the holy grail?Why are RCTs the holy grail? Comparison to standard of careComparison to standard of care Minimizes bias & confoundersMinimizes bias & confounders Provides a precise estimate of effectProvides a precise estimate of effect

But …But …

Not All Questions Require RCTsNot All Questions Require RCTs

““We think that everyone We think that everyone might benefit if the most might benefit if the most radical protagonists of radical protagonists of

evidence-based medicine evidence-based medicine organised and participated in organised and participated in

a double blind, placebo a double blind, placebo controlled, crossover trial of controlled, crossover trial of

the parachute”the parachute”

This is NonsenseThis is Nonsense

Magnitude of effect is Magnitude of effect is importantimportant

All or None PhenomenonAll or None Phenomenon

Nor Is There An RCT For Every QuestionNor Is There An RCT For Every QuestionIoannidis et al: JAMA 2001Ioannidis et al: JAMA 2001

48 interventions with randomized and observational trials48 interventions with randomized and observational trials Results highly correlated (correlation coefficient - 0.83)Results highly correlated (correlation coefficient - 0.83) Larger treatment effect in nonrandomized trialsLarger treatment effect in nonrandomized trials

Trial DesignTrial DesignA Continuum Rather Than A HierarchyA Continuum Rather Than A Hierarchy

Treatment Effect

Example

Huge (All or None)

ParachutesEpinephrine/AnaphylaxisUFH/DVT

Large Bypass for CLI

ModerateStatinsHCSE

Case SeriesCase Series

Case SeriesCase Series

Observational Observational StudiesStudies

Case SeriesCase Series

Observational Observational StudiesStudies

RCTRCTss

Standard of Care Established

Standard of Care Established

Standard of Care Established

What Are The Important Questions?What Are The Important Questions?Chronic Venous DiseaseChronic Venous Disease

Is the use of compression prior to intervention cost effective ?Is the use of compression prior to intervention cost effective ? What is the best treatment for C2 & C3 disease?What is the best treatment for C2 & C3 disease?

InterventionsInterventionsCompressionCompression

Pharmacotherapy (HCSE, MPFF)Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)Ablation (RF, laser, foam) OutcomesOutcomes

Patient important benefits – Pain, quality of life, recurrencePatient important benefits – Pain, quality of life, recurrence

Costs to health care systemCosts to health care system Perforating veinsPerforating veins

The The pathologicalpathological perforator – Which are clinically important? perforator – Which are clinically important? C5, 6 disease – Healing and recurrenceC5, 6 disease – Healing and recurrence

Is 1Is 1stst rib resection after a first effort thrombosis warranted? rib resection after a first effort thrombosis warranted? What is the accuracy of CTV / MRV for iliac obstructionWhat is the accuracy of CTV / MRV for iliac obstruction

… … And Many OthersAnd Many Others

What Are The Important Questions?What Are The Important Questions?Chronic Venous DiseaseChronic Venous Disease

Is there a role for extended prophylaxis other than THR Is there a role for extended prophylaxis other than THR and malignancy?and malignancy?

Are there ANY prophylactic indications for IVC Filters?Are there ANY prophylactic indications for IVC Filters?

The treatment of acute DVTThe treatment of acute DVT

Pharmacomechanical thrombolysisPharmacomechanical thrombolysis

Iliofemoral DVTIliofemoral DVT

Femoropopliteal DVTFemoropopliteal DVT

Isolated calf vein thrombosisIsolated calf vein thrombosis

Is there any role for U/S (using US protocols) in Is there any role for U/S (using US protocols) in determining the duration of anticoagulation?determining the duration of anticoagulation?

… … And Many OthersAnd Many Others

How Do We Answer the Questions?How Do We Answer the Questions?

Clinical Question RCT Observational Outcomes

Value of Compression (C2) √ √ QoL, Cost (ICER)

Comparative effectiveness of different technologies

√ QoL, Cost (ICER, cost-consequence)

Definition of the pathologic perforator

√ Ulcer healing / recurrence

Extended prophylaxis √ Recurrent DVT, Bleeding

Pharmacomechanical lysis √ QoL, Bleeding, Cost

Calf vein thrombosis √ Recurrent thrombosis,

Bleeding, cost

U/S & anticoagulation √ Recurrent thrombosis

Role of 1st rib resection in effort thrombosis

√ Recurrent thrombosis

The CLASS TrialThe CLASS Trial

HTA (UK) funded randomized clinical trialHTA (UK) funded randomized clinical trial 1000 C 1000 C 2-6 2-6 patients (6 centers)patients (6 centers)

Saphenous surgerySaphenous surgery Foam sclerotherapyFoam sclerotherapy Laser ablation with adjuvant foam sclerotherapyLaser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)1º outcomes (6 months, possible 5 yr) Disease specific – Aberdeen VV QuestionnaireDisease specific – Aberdeen VV Questionnaire Generic – EuroQol, SF-36Generic – EuroQol, SF-36

2º outcomes2º outcomes Validated return to function instrumentValidated return to function instrument Incremental cost effectivenessIncremental cost effectiveness

ATTRACT TRIALATTRACT TRIAL 692 patients692 patients

28 North American centers28 North American centers

Randomized toRandomized to

Best medical therapyBest medical therapy

Pharmacomechanical lysisPharmacomechanical lysis

Trellis 8Trellis 8

Angiojet powerpulseAngiojet powerpulse

Iliofemoral & femoropopliteal armsIliofemoral & femoropopliteal arms

Clinically relevant endpointsClinically relevant endpoints

Objective PTS (Villalta)Objective PTS (Villalta)

Quality of lifeQuality of life

The DiVeTAS Trial – Specific AimsThe DiVeTAS Trial – Specific AimsDIDIstal stal VEVEnous nous TThrombosis: hrombosis: AAnticoagulation vs nticoagulation vs SSurveillanceurveillance

To compare the short-term efficacy and safety of standard anticoagulation To compare the short-term efficacy and safety of standard anticoagulation versus duplex ultrasound surveillance for a first episode of acute symptomatic versus duplex ultrasound surveillance for a first episode of acute symptomatic DVT confined to the calf veins. The primary endpoint will be a composite of DVT confined to the calf veins. The primary endpoint will be a composite of proximal propagation, symptomatic pulmonary embolism (PE), major bleeding, proximal propagation, symptomatic pulmonary embolism (PE), major bleeding, and all-cause mortality occurring during the first 3 months of treatment.  and all-cause mortality occurring during the first 3 months of treatment.  

To evaluate the relationship between baseline characteristics, including D-Dimer To evaluate the relationship between baseline characteristics, including D-Dimer and other biomarkers, and the risk of proximal propagation and other and other biomarkers, and the risk of proximal propagation and other endpoints, with the goal of identifying high risk and low risk sub-groups which endpoints, with the goal of identifying high risk and low risk sub-groups which may differ in treatment efficacy. may differ in treatment efficacy.

  To compare long-term outcomes of calf DVT after treatment with standard To compare long-term outcomes of calf DVT after treatment with standard anticoagulation versus duplex ultrasound surveillance with respect to the anticoagulation versus duplex ultrasound surveillance with respect to the development of objectively defined PTS and quality of life. development of objectively defined PTS and quality of life.

To compare the cost and cost-effectiveness of standard anticoagulation versus To compare the cost and cost-effectiveness of standard anticoagulation versus duplex ultrasound surveillance for the management of isolated calf vein duplex ultrasound surveillance for the management of isolated calf vein thrombosis. thrombosis.

Comparative Effectiveness ResearchComparative Effectiveness ResearchThe “New” Holy GrailThe “New” Holy Grail

BackgroundBackground Interventional technology – 50% of healthcare resources Interventional technology – 50% of healthcare resources

(50 million procedures / yr)(50 million procedures / yr) Clinical data in < 15% of 510k approvalsClinical data in < 15% of 510k approvals Adoption after only 10-20% perceived implementationAdoption after only 10-20% perceived implementation Practice integration before value, risks, and costs establishedPractice integration before value, risks, and costs established

Comparative effectivenessComparative effectiveness ““a rigorous evaluation of different treatment options”a rigorous evaluation of different treatment options”

(Congressional Budget Office)(Congressional Budget Office) May focus on benefits/risks or cost/benefitMay focus on benefits/risks or cost/benefit > $1 billion dollars appropriated by Congress> $1 billion dollars appropriated by Congress

CDRH Device ClassificationCDRH Device Classification Class IClass I

Low risk devices (tongue depressors, scalpels)Low risk devices (tongue depressors, scalpels) General controlsGeneral controls

Good manufacturing practicesGood manufacturing practicesQuality systems regulationQuality systems regulation

Class IIClass II Venous lasers, RF devicesVenous lasers, RF devices Special controls - Performance standards, registries, Special controls - Performance standards, registries,

postmarket surveillancepostmarket surveillance Most approved through Premarket Notification (510k)Most approved through Premarket Notification (510k) Safety / effectiveness equivalent to predicate deviceSafety / effectiveness equivalent to predicate device

Class IIIClass III Insufficient information to ensure safety & effectivenessInsufficient information to ensure safety & effectiveness Most approved through Premarket Application (PMA)Most approved through Premarket Application (PMA)

Growth in Varicose Vein TreatmentGrowth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*Economic Analysis*

ModelModel DescriptionDescription Pros / ConsPros / Cons

EconomicEconomicQuantitative, statistical Quantitative, statistical analysis of economics analysis of economics

onlyonly

Simple, but neglects Simple, but neglects clinical outcomesclinical outcomes

Cost-Cost-consequenceconsequence

Economic & clinical Economic & clinical outcomes evaluated in outcomes evaluated in

commoncommon

Allows evaluation of Allows evaluation of “trade offs”“trade offs”

Model-basedModel-basedPreviously reported Previously reported data used as model data used as model

inputinput

Flexible, but relies Flexible, but relies on high quality dataon high quality data

* All require data from comparative trials* All require data from comparative trials

The REACTIV TrialThe REACTIV TrialRatcliffe , Br J Surg 2006Ratcliffe , Br J Surg 2006

246 patients extensive vv and saphenous reflux randomized to246 patients extensive vv and saphenous reflux randomized to Conservative measures (n = 122)Conservative measures (n = 122) Saphenous stripping / phlebectomy (n = 124)Saphenous stripping / phlebectomy (n = 124)

24 mo cost effectiveness of £4682 per QALY gained24 mo cost effectiveness of £4682 per QALY gained Below NHS threshold of £20,000 per QALYBelow NHS threshold of £20,000 per QALY

ConservativeConservative SurgerySurgery Mean DifferenceMean Difference

Mean NHS CostMean NHS Cost £344.53£344.53 £733.10£733.10 £388.57£388.57

AUC SF-6DAUC SF-6D 1.421.42 1.501.50 0.0830.083

ICER *ICER * £4682£4682

* Incremental cost effectiveness ratio

ConclusionsConclusions

The questions are important and need prioritizationThe questions are important and need prioritization

The goals, not the methods, are most importantThe goals, not the methods, are most important Precise estimates of harms, risks, and benefitsPrecise estimates of harms, risks, and benefits Minimizing bias and unknown confoundersMinimizing bias and unknown confounders

Every question requires a comparison groupEvery question requires a comparison group An RCT is not necessary, feasible , or even desirable An RCT is not necessary, feasible , or even desirable

for every questionfor every question

But …But …

Developing Phlebology as a Clinical ScienceDeveloping Phlebology as a Clinical Science

Demands for industryDemands for industry

Clinical evidence prior to marketingClinical evidence prior to marketing

Research with patient important endpointsResearch with patient important endpoints

Demands for ourselvesDemands for ourselves

Avoid herd mentality in the absence of dataAvoid herd mentality in the absence of data

Pay attention to costs to the health care systemPay attention to costs to the health care system

Consider comparative effectiveness of technologyConsider comparative effectiveness of technology

Demands for phlebologyDemands for phlebology

Raise the bar for presentation / publicationRaise the bar for presentation / publication

Fellowships in epidemiology & health systems researchFellowships in epidemiology & health systems research