challenges in classifying adverse events in cancer clinical trials
DESCRIPTION
Challenges in Classifying Adverse Events in Cancer Clinical Trials. Steven Joffe, MD, MPH Dave Harrington, PhD David Studdert, JD, PhD Saul Weingart, MD, PhD Damiana Maloof, RN . Disclosure. Member of clinical trial adverse event review board for Genzyme Corp (not oncology-related). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/1.jpg)
Challenges in Classifying Adverse Events in Cancer
Clinical Trials
Steven Joffe, MD, MPHDave Harrington, PhD
David Studdert, JD, PhDSaul Weingart, MD, PhD
Damiana Maloof, RN
![Page 2: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/2.jpg)
Disclosure
• Member of clinical trial adverse event review board for Genzyme Corp (not oncology-related)
![Page 3: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/3.jpg)
Adverse Events in Clinical Trials
• Adverse events (AEs) are critically important outcomes of clinical trials– Human subjects protection– Endpoints for judgments about benefits &
risks of study interventions
• Captured on Case Report Forms• Reported to oversight agencies
![Page 4: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/4.jpg)
Components of AE Assessment
• Type• Severity• Relatedness to study agent(s)• Expectedness
![Page 5: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/5.jpg)
Components of AE Assessment
• Type• Severity• Relatedness to study agent(s)• Expectedness
Global judgment about reportability to IRB
![Page 6: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/6.jpg)
Reporting Criteria(to Dana-Farber IRB)
• Grade 5 (fatal)• Grade 4, unless specifically exempted• Grade 2/3, if unexpected AND possibly,
probably or definitely related
• Virtually identical to NCI’s Adverse Event Expedited Reporting System (AdEERS) criteria
![Page 7: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/7.jpg)
AE Grading in Oncology• NCI’s Common Terminology Criteria for
Adverse Events (CTCAE) typically used– Effort to standardize nomenclature– developed by consensus methods; no formal
process to establish reliability of grading
http://ctep.cancer.gov/protocolDevelopment/electronic_applications/ctc.htm#ctc_v30
![Page 8: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/8.jpg)
Aims1. To assess the validity of physician reviewers’
determinations about whether AEs in cancer trials meet IRB reporting criteria
2. To assess the interrater reliability of reviewers’ determinations about whether AEs that occur in cancer trials meet IRB reporting criteria
3. To assess the validity and reliability of revie-wers’ judgments about the components of AEs
![Page 9: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/9.jpg)
Study Methods
![Page 10: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/10.jpg)
Panelists’ Roles
• Review primary data from criterion sets of AEs
• Rate each AE:– Classification– Grade– Relatedness– Expectedness– Reportable to IRB
} from CTCAE
![Page 11: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/11.jpg)
Panelist DemographicsExpert Panel
(n=3)Second Panel
(n=10)
Years since fellowship trainingMean 20 yrs 6.3 yrs
Range 10 – 32 yrs 2 – 17 yrs
Academic rankInstructor / Asst Prof 1 10
Assoc Prof / Prof 2 0
![Page 12: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/12.jpg)
Panelists’ Experience
Expert Panel(n=3)
Second Panel(n=10)
Clinical trials served as overall Principal Investigator0 – 5 0 7
≥ 6 3 3
Clinical trials served as PI, site PI, or Co-Investigator 0 – 5 0 2
6 – 20 0 3
>20 3 5
![Page 13: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/13.jpg)
Panelists’ ExperienceExpert Panel
(n=3)Second Panel
(n=10)Patients personally enrolled in a clinical trial during past 3 years
0 – 10 0 1
11 – 30 1 4
>30 2 5
Adverse event reports personally filed with the IRB during past 3 years0 – 10 1 6
11 – 30 0 1
>30 2 3
![Page 14: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/14.jpg)
Statistical Analysis
• Validity of judgments regarding reportability to IRB– % agreement with gold standard
• Interrater reliability of raters’ judgments– Kappa coefficients
![Page 15: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/15.jpg)
Results
![Page 16: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/16.jpg)
Criterion Set of AEsType of AE Grad
eRelated Expected Reportable
High triglycerides 4 Definite Y YOsteonecrosis 3 Definite Y NSensory neuropathy 1 Probable Y NCardiac ischemia 4 Possible Y YRash 2 Probable Y NThrombosis 4 Unlikely N YHigh uric acid 4 Probable N YCardiac dysfunction 2 Definite Y NThrombotic thrombo- cytopenic purpura 4 Possibly N Y
Renal failure 4 Definite Y Y
![Page 17: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/17.jpg)
Validity of Judgments Regarding Reportability to IRB
Adverse Event Not Reportable Reportable % Agree1. High triglycerides 0 10 1002. Osteonecrosis 6 4 60
3. Sensory neuropathy 10 0 1004. Cardiac ischemia 0 10 1005. Rash 9 1 90
6. Thrombosis 0 10 1007. High uric acid 0 10 1008. Cardiac dysfunction 8 2 809. TTP 0 10 10010. Renal failure 0 10 100
TOTAL 93%
![Page 18: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/18.jpg)
Interrater Reliability of Panelists’ Judgments
Judgment Kappa P value
Reportability 0.75 <0.0001
Grade 0.52 <0.0001
Relatedness 0.22 <0.0001
Expectedness 0.88 <0.0001
![Page 19: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/19.jpg)
Role of Experience: Rank
Kappa
![Page 20: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/20.jpg)
Role of Experience: Service as PI
Kappa
![Page 21: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/21.jpg)
Role of Experience: Number of AE Reports Filed
Kappa
![Page 22: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/22.jpg)
Conclusions• Oncologists’ judgments about whether or not
AEs require reporting to the IRB show high agreement with gold standard
• Interrater reliability of oncologists’ judgments about components of AEs varies– High: expectedness of AE; need for reporting– Moderate: grade of AE– Low: relationship of AE to study agents
![Page 23: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/23.jpg)
Limitations• Small sample sizes
– Criterion set of AEs– Panel of physician reviewers
• Generalizability of set of AEs
• Reviewers may not reflect population of investigators who file AE reports
• Judgments based on document review rather than on firsthand knowledge
![Page 24: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/24.jpg)
Thoughts About Direction of Bias in Agreement Statistics
• Factors biasing towards less agreement– Reviewer experience
• Factors biasing towards greater agreement– Standardized set of documents for review– Criterion set selected based on maximum
agreement among expert panel reviewers
![Page 25: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/25.jpg)
Implications• Judgments about AEs are complex
– Human subjects: efforts to enhance reliability, or to minimize reliance on judgments about causation, are needed
– Science: toxicity data from uncontrolled trials may be misleading
– RCR: education about need for reporting is important but insufficient
![Page 26: Challenges in Classifying Adverse Events in Cancer Clinical Trials](https://reader035.vdocuments.us/reader035/viewer/2022062816/5681685f550346895ddea807/html5/thumbnails/26.jpg)
Acknowledgments• Debra Morley• Anna Mattson-DiCecca• Physician panelists
• ORI• NCI• Milton Fund