Critically appraising research in pain management

Making sense of the evidence

M.Sc. The nature of pain and its’ management, 2006

• Why do we need to appraise the literature on pain ?

• What is the current state of the art ?

• How do we go about it ?

• What are good sources of information ?

Critical Appraisal

The process of “weighing

up” the evidence to assess

how useful it is in making

decisions about clinical

care

Some “drivers” promoting critical review of research literature

• Evidence based practice

Clinical effectiveness > guidelines, standards and pathways

• Continuing Professional Development (CPD)

• Modernisation Agenda eg. patient choice

• Research – asking the right questions

Types of Evidence

• Primary

Generated from first hand experience

eg. diaries, letters, research reports

• Secondary (synthesis of others work)

eg. narrative and systematic literature reviews, scholarly papers

Information explosion

c. 20.000 journals, 2 million papers p.a.

Pain Management – The Evidence

• Meta-analyses, RCT’s• Systematic reviews• Cohort studies• Case studies• Narrative reviews• Updates• Reports, editorials, working papers, product

updates• Conference proceedings, abstracts• Dissertations etc.

Traditional Hierarchy of EvidenceQuantitative Research

Which research has most “weight” ?

• Meta-analysis, systematic review

• High quality RCT

• Controlled study without randomisation

• Observational study eg. case control study

• Descriptive study

• Expert consensus

• Clinicians and clients/patients views

Systematic review

- The “sharp end” of critical appraisal- Used for development of clinical guidelines

Many stages including:

• Development of a protocol• Systematic selection of studies• Data Extraction• Quality assessment• Statistical or other synthesis of findings• Rating of the overall body of evidence

NICE • Cox II selective inhibitors are not recommended

for routine (regular) use in patients with rheumatoid arthritis or osteoarthritis (2001, Guidance 27)

• Current evidence on the safety and efficacy of percutaneous disc decompression using coblation for lower back pain does not appear adequate to support the use of this procedure without special arrangements for consent and for audit or research (Consultation Document, 25 May 2004)

• Adalimumab, etanercept and infliximab for the treatment of ankylosing spondylitis (in progress, due Feb. 2007)

Assessing the quality “critical appraisal”

a) Quality of individual publications

• Checklists• Guidelines/reading guides• Questionnaires• Frameworks,• Criterion based scoring tools

b) Quality of bodies of evidence

• Grading schemes• hierarchies

Quality Evaluation Tools

• Many “off-the-shelf” tools available for different types of studieseg. Jadad, Delphi, Maastricht, SIGN

• Debate on scales v scores v weighting

• Few have been properly validated and the criteria for validation are unknown

• Move towards customised checklists

• Critical Appraisal Skills Programme

http://www.phru.org.uk/-casp/

• Scottish Intercollegiate Guidelines Network

http://www.sign.ac.uk/guidelines

• Jadad et. al. (1996) Control Clin Trials 17:1–12

• Newcastle Ottawa Observational Study checklist http://www.ohri.ca/programs/clinical _epidemiology/oxford.htm

Quality in quantitative research eg. RCT’s

Emphasis on issues such as: Random allocation of subjects, “blinding”Allocation concealmentSimilarity at baselineStandard, reliable and valid outcome assessmentDrop-outs Intention to treat analysis (ITT)

ValidityReliability

Quality in qualitative research • Dependability (reliability) eg. - justification of methods - audit trail - providing information about the researcher

- member checking, inter-researcher comparisons, triangulation • Credibility eg. - providing raw quotes - reference to accepted procedures for analysis - discussion of how researcher may have effected outcomes

- clear distinction between data and interpretation

• Transferability eg. - detail about participants and context

- identifying differences between individual participants and participants and researcher

CASP – Qualitative Studies

Rigour: has a thorough and appropriate approach been applied to key research methods in the study?

Credibility: are the findings well presented and meaningful?

Relevance: how useful are the findings to you and your organisation?

http://www.phru.nhs.uk/casp/casp_qualitative_tool.pdf

Structure of a research paper

• Title, author• Abstract: summary of what the paper is about• Introduction: Background including previous

research, aims, research question/hypothesis, ethics

• Methods: patients, methods, equipment, data analysis

• Results• Discussion (may include results in qualitative

research)• Conclusion, recommendations for clinical

practice and further research

General questions for any study• Are the aims clearly stated ?• Was it ethical ?• Does the design match the aims ?• Was the sample size justified ?• Sufficient methodological detail eg. drop outs ?

Missing data ?• Are the measurements/outcomes valid and

reliable ? (NB. study population)• Are statistical methods/data analysis described ?• Are the main findings explicit ?• Are important effects considered ?• Are results/findings compared with previous

work?• Are implications for practice discussed ?

Papers for review:

Effects of pool-based and land-basedaerobic exercise on women with fibromyalgia/

chronic widespread muscle painJentoft et alArthritis Care and Research, 2001, 45:42-47

In the system: the lived experience of chronic back pain from the perspectives of those

seeking help from pain clinicsWalker et alPain, 1999, 621-628