Clinical Practice Guidelines Development: Practical

Applications for CliniciansApplications for Clinicians

Cecilia A. Jimeno, M.D.

Objectives

• To define clinical practice guidelines• To discuss the process of CPG

development• To describe the structure of CPG’s• To list some potential problems with

CPG’s• To give some pointers regarding

the use of CPG’s

Trivia regarding CPG’s

• 1295 guidelines at www.guideline.gov

• 103 guidelines are on endocrinology

• Guidelines have been developed about various health concerns

• Many guidelines abroad were developed because of pressure from HMO’s or government insurance (processing of reimbursement claims)

Guideline Development Model

CLINICAL CLINICAL RESEARCHESRESEARCHES CLINICAL CLINICAL

RESEARCHESRESEARCHESJOURNALS & JOURNALS & CONFERENCESCONFERENCESJOURNALS & JOURNALS & CONFERENCESCONFERENCES

PRACTITIONERPRACTITIONERPRACTITIONERPRACTITIONERPATIENTPATIENTPATIENTPATIENT

GUIDELINE GUIDELINE DEVELOPMENDEVELOPMEN

TT

GUIDELINE GUIDELINE DEVELOPMENDEVELOPMEN

TT

DISSEMINATIONDISSEMINATIONDISSEMINATIONDISSEMINATION

Rationale for CPG’s

• Improvement of pt care delivered by both the individual physician & groups of doctors i.e. medical organizations or subspecialty societies

• Strategies for improving quality of health care:1. Continuing medical education (CME) activities like

conventions or post-graduate courses;

2. Generic strategies which are hospital-based or institutional programs like quality of care audits;

3. Dev’t of clinical practice guidelines (CPGs).

What are Clinical Practice Guidelines?

• User-friendly statements that bring together the best external evidence & other knowledge or experience necessary for decision making about a specific health problem.

• CPG’s = Research evidence + experience (expert

opinion)

What are CPG’s?

CURRENT PRACTICE & associated outcomes

ALTERNATIVE PRACTICES & associated ideal clinical outcomes

Tools for clinicians

END goal of CPG’s: to improve health care delivery to individual patients.

How are CPG’s developed?

1. Organization

2. Search & retrieval of literature

3. Appraisal of validity & applicability of literature

4. Evidence grading

5. Synthesis of evidence

6. Consensus building

7. Public forum & dissemination

8. Implementation

DEVELOPMENT OF CPG’s1. Organization-

1a. Who will be involved in the project?• Administrative group• Technical group (TRC)• Expert panel

1b. What is the issue or topic? • Usually broad & disease-based.

1c. What has previously been done? • Appraise available guidelines for validity and applicability

to the local setting. • Then, decide if new guidelines need to be made.

1d. What should be studied? Identify research questions

2. Search and retrieve - systematic review of the literature to make reasonable estimates of the effect of different interventions. 2a. Appraise validity and applicability- critical appraisal

of the literature

2b. For each study, analyze the study design and decide if the results are credible

2c. Analyze if the results are applicable to the different types of pts you see, the types of treatment you use, and the specific outcomes you expect

DEVELOPMENT OF CPG’s

3. Evidence grading- • For each outcome, summarize the results of

different studies• Then grade the evidence according to a

standardized scale

4. Synthesis of evidence• For each intervention, summarize the evidence (net

benefit or net harm) across outcomes since the literature may be voluminous.

DEVELOPMENT OF CPG’s

5. Consensus Building• Develop judgments by consensus• For each intervention, decide if you will recommend

(or not recommend) their use.• Panelists cast their votes re. various options

6. Public Forum• opportunity for stakeholders to react to the

guidelines; • reactions may be the basis for modifications &

further votes by panel members

DEVELOPMENT OF CPG’s

7. Plan the implementation- • Discuss the best way to make sure that

the guidelines are implemented.

8. Monitor implementation and impact of the CPG on the practice of groups of physicians

• Identify & monitor measures of success of the guideline.

DEVELOPMENT OF CPG’s

• Question or Issue

• Recommendation

• Summary of Evidence

• Evidence Grade

• Strength of Recommendation

• Comparison with other guidelines

Anatomy of Guidelines

Anatomy of Guidelines

Recommendation: 1 or 2 statements which represent the recommendations of the consensus panel regarding the issue after they have examined the evidence presented by the TRC

Question or Issue: May be a phrase or question; may be about definitions of disease; burden of illness; causation or risk factors for a disease; what tests to use or management approach

Summary of Evidence: Brief review & synthesis of the available literature

Key points:• Characteristics of available local & foreign

studies (RCT’s, observational data, case reports),

• Validity of the studies• Actual results & the estimates of effectiveness or

relevant statistics, • Applicability of results to our local patient profile

Anatomy of Guidelines

Evidence Grade: • Usually a numerical rank which represents

how good the evidence is; • Others just give labels: good, moderate,

poor, very poor evidence, scarce data• Grade is determined by the study design,

study quality (validity), consistency and applicability of the study to the specified health problem (directness)

Anatomy of Guidelines

Evidence Grading

5

Expert opinion

Non-exp’tal designs-Cohort studies-Case control

4

Case Series

3

Non-randomised controlled studies

2

Randomised controlled trials

1

Predictable improvements in Health care outcomes

Source: New Zealand Guidelines Group, Oxford Center for EBM

Strength of Recommendation:• Represents the judgment of the consensus

panel regarding how forcefully they would make the recommendation based on the evidence they have reviewed plus their own experience in their practice

• Usually a represented by letters of the alphabet (A to D) or specific actions (do it, don’t do it, probably do it, probably don’t do it)

Anatomy of Guidelines

Grades of Recommendation

A Consistent level 1 studies

B Consistent level 2 or 3 studies OR extrapolations from level 1 studies

C Level 4 studies OR extrapolations fromlevel 2 or 3 studies

D Level 5 evidence or troublingly inconsistent or inconclusive studies of any level

Source: New Zealand Guidelines Group

Difficulties with Guidelines

• Organizations use various systems to grade the quality of evidence ---- there are as many grading schemes of evidence, as there are bodies which make CPG’s

• Differences & shortcomings in these grading systems: may be CONFUSING and impede effective communication

• Some groups combine strength of recommendation & level of evidence into one label (usually letters of the alphabet)

Levels of Evidence from the Canadian Diabetes Association• For studies of Treatment & PreventionLevel 1A Systematic overview or meta-analysis of high-

quality RCT’s

Appropriately designed RCT’s with adequate power to answer the question posed by the investigators

Level 1B Non-randomized clinical trial or cohort study with indisputable results

Level 2 RCT or systematic overview that does not meet level 1 criteria

Level 3 Non-randomized clinical trial or cohort study

Level 4 Other

ADA Evidence Grading System for Clin. Practice Recommendations

A

B

• Clear evidence from well-conducted, generalizable, randomized controlled trials that are adequately powered including– Evidence from well-conducted multicenter trial– Evidence from a meta-analysis incorporated quality ratings

in the analysis

• Supportive evidence from well-conducted cohort studies including– Evidence from a well conducted prospective cohort or

registry– Evidence from a well conducted meta-analysis of cohort

studies• Supportive evidence from well-conducted case-

control studies

ADA Evidence Grading System for Clin. Practice Recommendations

C

E

• Supportive evidence from poorly controlled or uncontrolled studies including– Evidence from RCT’s with one or more major or three or

more minor methodologic flaws that could invalidate the results

– Evidence from observational studies with high potential for bias (such as case series with comparison with historical cohort)

– Evidence from case series or case report• Conflicting evidence with the weight of the evidence

supporting the recommendation

• Expert consensus or clinical experience

Grades of Recommendation Based on Sources of Evidence

Evidence Category

Sources of evidence

Definition

A RCT’s.

Rich body of data

• Evidence is from endpoints of well-designed RCTs that provide a consistent pattern of findings in the pop’n for which the recommendation is made. •Requires substantial numbers if studies involving substantial numbers of participants.

Source: GOLD Guidelines

Evidence Category

Sources of evidence

Definition

B RCT’s. Limited body of data.

• Evidence is from endpoints of intervention studies that include only a limited number of pts, posthoc or subgroup analysis of RCTs, or metanalysis of RCTs. •Few RCTs, small in size, or undertaken in a pop’n that differs from the target pop’n of the recommendation

Grades of Recommendation Based on Sources of Evidence

Evidence Category

Sources of evidence

Definition

C Nonrandomized trials. Observational studies.

• Evidence is from outcomes of uncontrolled or nonrandomized trials or from observational studies.

Grades of Recommendation Based on Sources of Evidence

Evidence Category

Sources of evidence

Definition

D Panel Consensus Judgment.

• Used only in cases where the provision of some guidance was deemed valuable but the clinical literature addressing the subjects was deemed insufficient to justify placement in one of the other categories.• Panel consensus is based on clinical experience or knowledge.

Grades of Recommendation Based on Levels of Evidence

EXAMPLES

Clinical Question: What interventions are safe and effective

for the primary & secondary prevention of cardiovascular events

among diabetics?

•Several interventions

•Several outcomes

Anatomy of Guidelines

SHOULD WE ROUTINELY RECOMMEND ASPIRIN FOR TYPE 2 DIABETICS?

Recommendation: Use Aspirin (ASA)/Antiplatelet Medication Unless Contraindicated

Summary of Evidence: Patients with type 2 diabetes are at a significantly high risk for development of heart disease. For patients with type 2 DM, initiate low-dose aspirin therapy (81 to 325 mg daily) in patients 40 and older unless there is a contraindication to aspirin therapy. [Conclusion Grade I: See Conclusion Grading Worksheet --Appendix D -- Annotation #9D (Aspirin Use) in the original guideline document].

Evidence Grade & Strength of Recommendation: Evidence supporting this recommendation is of classes: A, B

If aspirin is contraindicated, consider use of clopidogrel (Plavix®) or ticlopidine (Ticlid®). For more information, please refer to the NGC summary of the ICSI guideline Stable Coronary Artery Disease.

Canadian Diabetes Association Guidelines for the Management of Diabetes Mellitus

Anatomy of Guidelines

Example

• Clinical question: What antibiotics are effective in the treatment of acute uncomplicated cystitis (P) among women?– P: Acute uncomplicated cystitis– I: Anti-microbial tx (specifically 3-day course)– O: Treatment (e.g. resolution of sx, rendering

the urine sterile)

Example

• Guideline Recommendation: Any of the antimicrobials listed below can be used (Grade A recommendation).– 3-day regimen for acute uncomplicated cystitis

• TMP/SMX 160/800 mg BID• Nitrofurantoin 100 mg QID• Ciprofloxacin 250 mg BID• Pefloxacin 400 mg BID• Ofloxacin 200 mg BID• Co-amoxiclav 375 mg TID

• Summary of Evidence:– The antimicrobials used in the RCTs that

were included in the 3 meta-analyses can be used (Leibovici 1991, Alejandria 1998, de Guzman 1998). Because of consistently high (40 to 64%) rates of resistance of the most common uropathogen (E. Coli) to ampicillin & amoxicillin, these drugs should be avoided (Dytan 1998, Raco 1998).

Example

Other Difficulties with CPG’s

• Conflicts of interest –TRC or panelists may have some interests in the pharmaceutical industry– Questions of credibility of proponents & of the CPG

• Appplicability to local setting– Foreign CPGs are oftentimes adopted in toto by local

doctors without thoroughly analyzing the validity, applicability, and appropriateness of these guidelines to the socio-economic, demographic and health conditions in the Philippines.

• Hence, many organizations usually adapt foreign CPG’s rather than adopting them wholly.

A final word…

• Guidelines are not meant to replace the judgment of physicians

• Clinicians must critically appraise these clinical practice guidelines before adopting and utilizing them.– Readers’ Guide for Appraising Clinical

Practice Guidelines– INCLEN Validity, Applicability & Equity Lenses

PLEASE READ THE NEXT FEW SLIDES …. WILL NOT YET MAKE SENSE NOW BUT MAYBE AFTER NEXT WEEK

Critical appraisal of CPG’sVALIDITY Criterion1. Did recommendations consider all relevant pt groups?2. Did the recommendations consider all possible

management options?3. Did the recommendations consider all possible

outcomes (benefits and harms)?4. Is there a systematic review of evidence linking options

to outcomes for each relevant question?5. Was there an explicit process to consider the relative

preferences or values of pts for different outcomes?6. Did the authors indicate the strength of their

recommendations?

APPLICABILITY Criterion

1. Has the evidence on which CPG is based been evaluated for local applicability?

• Epidemiology and Burden of illness• Socioeconomic factors

2. Have the recommendations been subjected for scrutiny by a local panel for feasibility and acceptability?

Critical appraisal of CPG’s

EQUITY Criterion

1. Is the health problem a priority for all stakeholders, including potentially disadvantaged populations?

2. Did the guidelines look into the possibility of differential effects of treatment (benefits and harms) in potentially disadvantaged populations?

3. Is the voice/interest of potentially disadvantaged populations represented in the expert panel?

Critical appraisal of CPG’s

EQUITY Criterion4. Is the voice/interest of potentially

disadvantaged populations represented in the feedback process?

5. Were feasible knowledge transfer strategies laid out to address barriers to the implementation of the guidelines in potentially disadvantaged populations?

6. Does the impact assessment include evaluation of health gains across potentially disadvantaged populations?

Critical appraisal of CPG’s

Applicability of ClinicalTrials Results

• Will they help me caring for my patients?

• How will apply these data to my patient care?

Outline

• Applicability

• Clinician’s Role in applying effectiveness data to patient care

• Guides to Applicability

• Killer B’s

Applicability

• generalizability

• issues related to the overall impact of treatment in patients– individual patients– Treatment policies

• Aim: to confer to our patients/population the benefit shown by the clinical trial

• We cannot afford to repeat every trial simply because of doubts regarding applicability

Clinician’s concern

• decide whether the biology of the treatment effect will be similar in patients they are facing

• determine their patients' risk of the target event that treatment is designed to prevent

Clinician’s concern

• Know the side effects that may accompany treatment

• Be able to deliver the intervention in a safe, effective, and efficient manner

5B’s of generalizability

• Biology

• Barriers

• Beliefs

• Burden

• Bargains

Guides to applicability• Biologic Issues

– Are there pathophysiologic differences in the illness under study that may lead to a diminished treatment response?

– Are there patient differences that may diminish the treatment response?

• Social and Economic Issues – Are there important differences in patient compliance that may diminish

the treatment response? – Are there important differences in provider compliance that may diminish

the treatment response?

• Epidemiologic Issues – Do my patients have co-morbid conditions that significantly alter the

potential benefits and risks of the treatment? – Are there important differences in untreated patients' risk of adverse

outcomes that might alter the efficiency of treatment?

I. Biologic Issues

1. Are there pathophysiologic differences in the illness under study that may lead to a diminished treatment response?

• These differences can sometimes lead to diminished treatment responses due to – divergence in pathogenetic mechanisms or – biologic differences in the causative agent

• Sources of information – basic and laboratory studies, animal studies,

genetic studies– observational studies documenting

• pathologic changes in affected individuals• Variation in treatment response• evaluating the biology of causative agents (eg -

surveys on drug resistance)

• 2. Are there patient differences that may diminish the treatment response?– differences in drug metabolism– differences in immune response– differences in environmental factors that affect

drug toxicity

• Sources of Information– Pharmacokinetic– bioavailability studies – dose-ranging – descriptive studies of patients receiving

treatment

– Post-marketing surveillance studies – Clinical trials

Scenario• Use of streptokinase in a Filipino patient with acute

myocardial infarction• Biology:

– autopsies done on Filipino MI patients -pathologic changes in the coronary arteries and myocardium were similar to those noted among North Americans ; non-atherosclerotic causes of coronary disease were rare.

– Clinical surveys- Filipinos share the same risk factors for coronary disease as North Americans

– post-marketing data/clinical data - Filipinos experience the same reperfusion arrhythmias and the bleeding complications when given streptokinase at the same dose as North Americans

II. Social and Economic Issues

1. Are there important differences in patient compliance that may diminish the treatment response?

• different compliance with treatment, affect treatment effectiveness

• Variability in compliance between populations may stem from – resource limitations in a particular setting– attitudinal or behavioral idiosyncrasies.

• 2. Are there important differences in provider compliance that may diminish the safety and efficacy of the treatment?

• Availability of diagnostic tests, monitoring equipment, interventional requirements and other technical specifications

• availability and skill of health personnel may influence treatment effectiveness

• Sources of Information– practitioners themselves – formal quality of care assessments– post-marketing surveillance of adverse effects

Scenario

• Use of streptokinase in a Filipino patient with acute myocardial infarction

• Socio economic issue:– Long term compliance not a problem– cost– Administration of streptokinase carries potential

hazard of bleeding• Facilities for emergency administration of cryoprecipitate,

fresh frozen plasma or whole blood must be available

• may tip the balance between benefit and harm.

III. Epidemiologic Issues

• When biologic, social or economic differences do not compromise applicability, examine patient characteristics that can influence either the magnitude of the benefit or the risks of treatment

• 1. Do my patients have co-morbid conditions that significantly alter the potential benefits and risks of treatment?– Co-morbid conditions may affect treatment efficiency

in two possible ways• competing diagnostic possibilities• competing etiologies of outcome

• Source of information– Case control studies– Cohort studies

Scenario

• Use of streptokinase in a Filipino patient with acute myocardial infarction

• Epidemiology– RCT data on the use of streptokinase in AMI

• 25% reduction in mortality

– Local data- • 20% in-hospital deaths in a cohort of 149 patients

admitted to a gov’t. hospital ;(30/149)» Infarct Survival In the Phil-(ISIP)

Use of streptokinase in a Filipino patient with acute myocardial infarction

• we expect streptokinase to reduce this 20% death rate to 15% (ARR=5%)– a closer look at the local data shows that not

all deaths were a direct result of cardiac ischemia

• In the Philippine study, non-cardiac causes—mostly pneumonia with sepsis—were responsible for 11 of the 30 deaths.

• Streptokinase will not confer similar reduction in mortality.

• 6. Are there important differences in untreated patients' risk of adverse outcomes that might alter the efficiency of treatment?– Baseline risk– Magnitude of risk reduction

Baseline Risk and NNT for Streptokinase in AMI based on wall involvement

RISK% NNT

MASSIVE ANTERIOR

23 18

ANTERIOR/

LATERAL

21.4 20

INFERIOR/ POSTERIOR

12.8 32

Recommendation for Treatment of Hypertension

Not at Goal Blood Pressure (<140/90 mmHg) (<130/80 mmHg for those with diabetes or chronic kidney disease)

Initial Drug Choices

Drug(s) for the compelling indications

Other antihypertensive drugs (diuretics, ACEI, ARB, BB, CCB)

as needed.

With Compelling Indications

Lifestyle Modifications

Stage 2 Hypertension (SBP >160 or DBP >100 mmHg)

2-drug combination for most (usually thiazide-type diuretic and

ACEI, or ARB, or BB, or CCB)

Stage 1 Hypertension(SBP 140–159 or DBP 90–99

mmHg) Thiazide-type diuretics for most.

May consider ACEI, ARB, BB, CCB,

or combination.

Without Compelling Indications

Not at Goal Blood Pressure

Optimize dosages or add additional drugs until goal blood pressure is achieved.

Consider consultation with hypertension specialist. JNC7

Hypertension• Biology

– Mostly essential– Similar risk factors– Similar end organ damage

73% 68%

0

20

40

60

80

100

Detected Treated

Hypertension Among Patients with ACSPhilippines

ASPAC’98

Risk of AMI Associated With Risk Factors in the Overall Population

Risk FactorControls (%)

Cases (%)

OR (99% CI)Adjusted forAll OtherRisk Factors

ApoB/apoA1 (5 vs 1)* 20.0 33.5 3.25(2.82-3.76)

Current smoking 26.8 45.2 2.87(2.58-3.19)

Diabetes 7.5 18.5 2.37(2.07-2.71)

Hypertension 21.9 39.0 1.91(1.74-2.10)

Abdominal obesity (3 vs 1)† 33.3 46.3 1.62(1.45-1.80)

Psychosocial - - 2.67(2.21-3.22)

Daily consumption of vegetables and fruit

42.2 35.8 0.70(0.62-0.79)

Exercise 19.3 14.3 0.86(0.76-0.97)

Alcohol intake 24.5 24.0 0.91(0.82-1.02)

INTERHEART Lancet. 2004;364:937-952.

Prevalence of Risk Factors Among PAD Patients

Risk Factors No. (%) % of PAD

Hypertension 192(64) 32.81

Hyperchole 50(16.67) 20

Smoking 169(56.33) 30.81

PhilPad, PJIM 03

01020304050

60708090

100

Risk Factors

Pe

rce

nta

ge

HPN

Smokers

DM

TIA

MI

RHD

AF

RIFASAF’99

Prevalence of Risk Factors Among Stroke Patients

Hypertension

• Biology– Post marketing surveillance/dose ranging

studies• efficacy • safety

Hypertension• Socio economic

– Barriers and Beliefs• Patient compliance on treatment

Aware hypertensives 59%

On medication 37%

Good Compliance 44%

Controlled 43%

PRESYON’98

Hypertension

• Socio economic– Provider compliance

• Barriers• Providing, follow-up, lab monitoring

Hypertension

• Epidemiology– Burden and Bargain

• Patient’s risk of event- outcome and side effects• Co-morbid conditions• Benefit and harm of long term treatment

– NNT, side effect data– RCT, Post marketing surveillance– Clinical experience

Risk of Events and BP level

Treatment Events Duration Risk of no treatment

NNT

Diastolic BP 115-129mmHg

Death, stroke, MI

1.5yrs 0.13 9

Diastolic BP 90-109 mmHg

Death, stroke, MI

5.5 0.05 128

5B’s of generalizability

• Biology

• Barriers

• Beliefs

• Burden

• Bargains

Guides to applicability• Biologic Issues

1.Are there pathophysiologic differences in the illness under study that may lead to a diminished treatment response?

2. Are there patient differences that may diminish the treatment response?

• Social and Economic Issues 1.Are there important differences in patient compliance that may diminish

the treatment response? 2.Are there important differences in provider compliance that may

diminish the treatment response?

• Epidemiologic Issues 1.Do my patients have co-morbid conditions that significantly alter the

potential benefits and risks of the treatment? 2.Are there important differences in untreated patients' risk of adverse

outcomes that might alter the efficiency of treatment?

• Burden and Bargains– Side effect data

• Impotence• Coughing• Electrolyte abnormalities• Edema• Cramps• Etc.

• In making decisions regarding applicability, – strike a balance between making “unjustifiably

broad generalizations and being too conservative in one’s conclusions.”

“…clinicians must decide whether the biology of the treatment effect will be similar in patients they are facing, their patients' risk of the target event that treatment is designed to prevent and of the side effects that may accompany treatment, and their own ability to deliver the intervention in a safe and effective manner”

» Friedman, et al. Fundamentals of Clinical Trial

THANK YOU FOR YOUR TIME AND

ATTENTION!