Psychosocial Interventions in Cancer Care: The Evidence is

Worse Than it First Looks

James C. Coyne, Ph.D.University of Pennsylvania

With Special Thanks to Drs. Steve Palmer and Joan Cook—and to NCI, SBM, the Cancer SIG, and my fellow panelists.

We must not allow a shared commitment to improving the wellbeing of cancer patients to be

exploited with exaggerated claims and poorly conceived, poorly conducted, and poorly

reported clinical trials.

How the Literature Fails Us

• Does not provide a rationale basis for favorably recommending interventions to cancer patients.

• Does not provide a credible basis for advocating for reimbursement by third party payers.

Most Robust Finding: No Benefit

• Average cancer patient receiving interventions does not achieve a reduction in distress.

• Average cancer patient does not benefit from participating in:

Group Cognitive Behavior Stress Management

Supportive Expressive Group Psychotherapy

Writing exercise (0-3).

The Literature is Worse Than it First Looks

The Literature is Worse Than It First Looks

• Cannot accept positive appraisals of a particular study or the literature at face value.

• Endemic confirmatory bias.

• Myth that combinations of similarly flawed studies can yield an informative contribution to the literature: blend them together, you get tainted scrapple, not pate.

The Literature is Worse Than It First Looks

• Late adoption of CONSORT leaves an accumulated literature with serious methodological shortcomings.

• Adoption of CONSORT tends to be associated with 30% reduction in confirmatory bias.

• “Best Foot Forward” takes precedence over best available evidence.

CONSORT

A list of requirements for uniform reporting of clinical trials with the overall aim of improving the reporting of Randomized Controlled Trials, to facilitate their critical appraisal, and to facilitate their inclusion in systematic reviews.

Published in 1996, revised 2001

CONSORT Checklist

Not a Cure or even a Treatment for an Ailing Literature.

Not a Diagnostic Instrument.

More of a Screening, a First Assessment of Transparency of Reporting.

Most Effective When Used Preventively by Authors, Rather than Later By Readers.

What are the Endemic Problems in the Design,

Conduct, and Reporting of Trials?

Poor Rationale for Trials

• Claims of high prevalence of distress are not reflected in samples.

• Distress and depression do not predict survival.

• Salivary cortisol not related to progression.• Measures of immune function not related

to progression. • Psychological interventions do not effect

immune function in cancer patients.

The Norm: Lack of Intent to Treat Analyses

• Data from patients who do not complete trial or all measurements are discarded.

• “As treated” analyses ignore informative missing data.

• Intervention and control patients have different reasons for not providing data and this introduces bias in the available data.

• “As treated” data do not generalize back to patients entering a trial.

Are We Done Yet? Check the Data Again and See if We Have Got a

Finding to Report

• A priori power analysis the occasional exception rather than the rule.

• Operative Rule: Peek and stop when results are looking good.

• Must beware of modest sized trials claiming strong effects--likely to be false positives.

• Must beware of studies with odd numbers of patients accumulated without a power analysis.

Telling It Like It Ain’t: All the Results That Fit

• Primary endpoint typically needs to be inferred, not stated.

• Ignore negative results for presumed endpoints: Emphasize any positive effect, ignore larger number of null findings.

• Favor secondary and subgroup analyses and endpoints developed post hoc over negative findings for presumed analyses.

• Discuss negative findings as if positive in subsequent publications.

• Accommodate existing literature “as is” rather than qualifying interpretation with reference to methodological shortcomings.

Just What Is Wrong With Post Hoc

Subgroup Analyses? • High profile papers in the behavioral medicine

literature routinely emphasize subgroup analyses when they are positive in the face of negative primary analyses (Classen et al, 2001; Schneiderman et al., 2004).

• In the broader clinical trials literature, this practice is uniformly criticized as inappropriate (Yusuf et al., 1991).

• Unplanned subgroup analyses frequently yield spurious results (Assman et al., 2000; Senn & Harrel, 1979), and “only in exceptional circumstances should they affect the conclusions drawn from the trial” (Brooks et al., 2004, p 229).

Reporting a Major Intervention Trial Focusing on Distress (Part 1)*

• First report published in Archives of General Psychiatry.

• CES-D data collected but not reported.• Request for information: ‘Null, so not reported’.• POMS data: Post hoc analyses suggested

strategy of dropping last observation for some patients, allowing marginally significant result.

Classen, Butler, Koopman, Miller, DiMiceli, Giese-Davis, Fobair, Carlson, Kraemer & Spiegel 2001

Reporting a Major Intervention Trial Focusing on Distress (Part 2)*

• Second report of same trial in Journal of Consulting and Clinical Psychology.

• First paper cited in passing, but null results not reported.• No a priori endpoint.• Outcomes selected on basis of analysis of control group

data.• These outcomes justified in terms of a literature in which

they have served as process variables for reduction of distress that was not obtained in this trial.

*Giese-Davis et al., J Consult Clin Psych 2002

Misusing and Misunderstanding the Profile of Mood States (POMS)

• Provides general mood score and several subscales.• Allows multiple correlated tests of whether intervention

affects mood.• Any positive findings emphasized and negative findings

and capitalization on chance ignored.• Emphasized over CES-D when CES-D yields null

findings.• CES-D preferable.• Investigators tend to ignore normative data

suggesting that cancer patients entering trials have mood as good or better than college students.

Rescuing a Null Trial With A Newly Invented Outcome: Benefit Finding*

• A priori primary endpoint was distress.• Primary analysis yielded null results, but subsequent

reports have reported a secondary analysis in which there was an effect for a subgroup of patients.

• Subsequent reports give main emphasis to benefit finding as an endpoint.

• Intervention not designed to affect benefit finding, no theoretical reason for assuming an effect.

• Benefit finding has unknown clinical significance. *Antoni et al, Health Psychology 2001

How Not to Solve the Problem of Weak Effects

Dominant strategy involves bigger, more intense and complex interventions which:

• Aggravate problems of selection bias.

• Create problems of selective exposure to interventions and breakdown in adherence.

• Preclude elucidation of mechanism.

With possible exception of dying patients, no obvious theoretical or technical reason for assuming that psychological interventions cannot reduce distress---not even an interesting question---if cancer patients are selected for clinical need.

CONSORT

• Will help, but not a panacea.

• Provides an evaluation of transparency, but not directly the quality with which a trial has been designed and executed.

• Biased analysis and interpretation more obvious, but not eliminated.

What is Needed• Registry of Trial Designs at NCI so that reader

can compare what is reported with what was proposed.

• Reviewers start incorporating CONSORT NOW!• Abstracts faithfully represent what was done in

trials NOW.• Readers cultivate own critical appraisal skills.• Graduate training provide more emphasis on

design, conduct, analysis, interpretation, and reporting of clinical trials.

What is Needed

• Trials targeting patients with significant distress.• Recruitment strategies based on accurate

estimates of prevalence of distress in population.• Comparisons of psychotherapeutic intervention

to access to information and improved management of symptoms and side effects.

• Patient preference and randomized encouragement designs.

What is Fair

• Given quality of evidence for efficacy of intervention, don’t oversell.

• Not every cancer patient needs therapeutic intervention, wrong to suggest that they do.

• Lower barrier to patients accessing intervention if they want it, beware of aggressive outreach.

• Informed consent procedures should not mislead patients concerning evidence for benefit of treatments to which they are being recruited.

Simpler Interventions with Some Hope of Dissemination

“Dissemination of efficacious interventions into real-world settings may be hampered by inclusion of many activities, techniques, and strategies that go far beyond their underlying theories” (Rotheram-Borus & Duan, 2003).

Beware of Unnecessarily Complex Interventions

Must Come to Terms with Poor Quality of Existing

Literature

Take Down the Slums and Build Anew With Studies With Adequate Designs, Suitable

Patient Populations, and Fair Reporting.