challenges in classifying adverse events in cancer clinical trials steven joffe, md, mph dave...
Post on 28-Dec-2015
215 Views
Preview:
TRANSCRIPT
Challenges in Classifying Adverse Events in Cancer
Clinical Trials
Steven Joffe, MD, MPH
Dave Harrington, PhD
David Studdert, JD, PhD
Saul Weingart, MD, PhD
Damiana Maloof, RN
Disclosure
• Member of clinical trial adverse event review board for Genzyme Corp (not oncology-related)
Adverse Events in Clinical Trials
• Adverse events (AEs) are critically important outcomes of clinical trials– Human subjects protection– Endpoints for judgments about benefits &
risks of study interventions
• Captured on Case Report Forms
• Reported to oversight agencies
Components of AE Assessment
• Type
• Severity
• Relatedness to study agent(s)
• Expectedness
Global judgment about reportability to IRB
Reporting Criteria(to Dana-Farber IRB)
• Grade 5 (fatal)
• Grade 4, unless specifically exempted
• Grade 2/3, if unexpected AND possibly, probably or definitely related
• Virtually identical to NCI’s Adverse Event Expedited Reporting System (AdEERS) criteria
AE Grading in Oncology
• NCI’s Common Terminology Criteria for Adverse Events (CTCAE) typically used– Effort to standardize nomenclature– developed by consensus methods; no formal
process to establish reliability of grading
http://ctep.cancer.gov/protocolDevelopment/electronic_applications/ctc.htm#ctc_v30
Aims
1. To assess the validity of physician reviewers’ determinations about whether AEs in cancer trials meet IRB reporting criteria
2. To assess the interrater reliability of reviewers’ determinations about whether AEs that occur in cancer trials meet IRB reporting criteria
3. To assess the validity and reliability of revie-wers’ judgments about the components of AEs
Panelists’ Roles
• Review primary data from criterion sets of AEs
• Rate each AE:– Classification– Grade– Relatedness– Expectedness– Reportable to IRB
} from CTCAE
Panelist Demographics
Expert Panel
(n=3)
Second Panel
(n=10)
Years since fellowship training
Mean 20 yrs 6.3 yrs
Range 10 – 32 yrs 2 – 17 yrs
Academic rank
Instructor / Asst Prof 1 10
Assoc Prof / Prof 2 0
Panelists’ Experience
Expert Panel
(n=3)
Second Panel
(n=10)
Clinical trials served as overall Principal Investigator
0 – 5 0 7
≥ 6 3 3
Clinical trials served as PI, site PI, or Co-Investigator
0 – 5 0 2
6 – 20 0 3
>20 3 5
Panelists’ Experience
Expert Panel
(n=3)
Second Panel
(n=10)
Patients personally enrolled in a clinical trial during past 3 years
0 – 10 0 1
11 – 30 1 4
>30 2 5
Adverse event reports personally filed with the IRB during past 3 years
0 – 10 1 6
11 – 30 0 1
>30 2 3
Statistical Analysis
• Validity of judgments regarding reportability to IRB– % agreement with gold standard
• Interrater reliability of raters’ judgments– Kappa coefficients
Criterion Set of AEsType of AE Grad
eRelated Expected Reportable
High triglycerides 4 Definite Y Y
Osteonecrosis 3 Definite Y N
Sensory neuropathy 1 Probable Y N
Cardiac ischemia 4 Possible Y Y
Rash 2 Probable Y N
Thrombosis 4 Unlikely N Y
High uric acid 4 Probable N Y
Cardiac dysfunction 2 Definite Y N
Thrombotic thrombo- cytopenic purpura
4 Possibly N Y
Renal failure 4 Definite Y Y
Validity of Judgments Regarding Reportability to IRB
Adverse Event Not Reportable Reportable % Agree
1. High triglycerides 0 10 100
2. Osteonecrosis 6 4 60
3. Sensory neuropathy 10 0 100
4. Cardiac ischemia 0 10 100
5. Rash 9 1 90
6. Thrombosis 0 10 100
7. High uric acid 0 10 100
8. Cardiac dysfunction 8 2 80
9. TTP 0 10 100
10. Renal failure 0 10 100
TOTAL 93%
Interrater Reliability of Panelists’ Judgments
Judgment Kappa P value
Reportability 0.75 <0.0001
Grade 0.52 <0.0001
Relatedness 0.22 <0.0001
Expectedness 0.88 <0.0001
Conclusions
• Oncologists’ judgments about whether or not AEs require reporting to the IRB show high agreement with gold standard
• Interrater reliability of oncologists’ judgments about components of AEs varies– High: expectedness of AE; need for reporting– Moderate: grade of AE– Low: relationship of AE to study agents
Limitations
• Small sample sizes– Criterion set of AEs– Panel of physician reviewers
• Generalizability of set of AEs
• Reviewers may not reflect population of investigators who file AE reports
• Judgments based on document review rather than on firsthand knowledge
Thoughts About Direction of Bias in Agreement Statistics
• Factors biasing towards less agreement– Reviewer experience
• Factors biasing towards greater agreement– Standardized set of documents for review– Criterion set selected based on maximum
agreement among expert panel reviewers
Implications
• Judgments about AEs are complex
– Human subjects: efforts to enhance reliability, or to minimize reliance on judgments about causation, are needed
– Science: toxicity data from uncontrolled trials may be misleading
– RCR: education about need for reporting is important but insufficient
top related