Financial Disclosure

Robert Nishikawa:– paid consultant for Carestream and Siemens

– shareholder in and receives royalties & research funding from Hologic, Inc.

• But I am here representing myself

Guidance Document on CADe

• Where the research is clear, then the document is reasonable

• However, whenever there is uncertainty, the approach is to error on the side of requiring more rather than less

• This is extremely burdensome

• This document if it goes into affect, it could kill new and innovative products in CADe

FDA is in a No-Win Situation

• If requirements are: too lenient

» non-effective products may be marketed

– Money wasted

– Patients may be harmed

too restrictive

» effective products will be delayed

– Patients’ healthcare is compromised

Guidance Document Statement

• “Test data, used once before, does not constitute independent data for testing a CADe device”

• Datasets are extremely costly and time consuming to collect

• This extremely overly burdensome

• This will stifle innovation

Training to the Test Dataset

• Difficult unless the performance on individual cases is known– Only release results of the performance on the

whole set, not individual cases

– Mitigates the example Dr. Gwise gave on overfitting

Proposed Observer Study Protocol

1. a conventional reading without the CADe device (i.e., reader alone)

2. CADe output is displayed immediately after conducting a conventional interpretation [sequential read]

3. a concurrent or simultaneous read

Proposed Observer Study Protocol

• Three reads, when only one is needed, is overly burdensome:– Costs are higher

– Much, much harder to recruit radiologists

– Time to complete study is much longer

• The increase is more than a factor of 3!

Proposed Observer Study Protocol

1. a conventional reading without the CADe device (i.e., reader alone)

2. CADe output is displayed immediately after conducting a conventional interpretation

3. a concurrent or simultaneous read

“Reading scenarios should be consistent with the intended use of the device.”

Proposed Observer Study Protocol

1. a conventional reading without the CADe device (i.e., reader alone)

2. CADe output is displayed immediately after conducting a conventional interpretation

3. a concurrent or simultaneous read

“Reading scenarios should be consistent with the intended use of the device.”

Concurrent Reading

• CADe is labeled as a second reader

• There is a possibility that radiologists will use it as a concurrent reading

• There is no clinical data to show whether this is good or bad

• It is overly burdensome to have manufactures answer a research question in their submission that goes beyond their labeling claims

Proposed Observer Study Protocol

1. a conventional reading without the CADe device (i.e., reader alone)

2. CADe output is displayed immediately after conducting a conventional interpretation

Two studies have shown that a conventional read and the unaided reading in a sequential read are comparable

Proposed Observer Study Protocol

1. CADe output is displayed immediately after conducting a conventional interpretation

Radiology: June 2009

Post-approval study

• Post-approval studies are not needed

• Clinical studies show that CADe can increase sensitivity with a comparable increase in recall rate

• CADe can increase radiologists’ sensitivity (~9%) while keeping PPV1

approximately constant (4.8% vs 4.7%)

Clinical Evaluation Methodology

• Using cancer detection rate in clinical studies that employ historical controls is not viable for measuring the effectiveness of CADe

Radiology: June 2009

Power Calculation

• Dr. Gwise power calculation for se at a fixed sp is not the same as the power required for (se,sp) true operating point