status and potential for further collaboration with rsna qiba qin meeting, march 28, 2014

Status and potential for further Status and potential for further collaboration with RSNA QIBAcollaboration with RSNA QIBA

QIN Meeting, March 28, 2014

D. Sullivan, MDDuke University;

RSNA

Premise

• Variation in clinical practice results in poorer outcomes and higher costs.

RSNA’s Perspective:

• Extracting objective, quantitative results from imaging studies will improve the value of imaging in

clinical practice.

Quantitative Imaging Biomarkers Alliance (QIBA): Background

• Started in 2007• Mission: Improve value and practicality of

quantitative imaging biomarkers by reducing variability across devices, patients, and time.

– “Industrialize imaging biomarkers”

QIBA Criteria for Biomarker Selection

• Transformational– addresses a significant medical need 

• Translational– will likely result in significant improvement in the development,

approval, or delivery of care to patients.   • Feasible

– end goals can likely be achieved in a specific timeframe  

• Practical– leverages preexisting resources (e.g., intellectual capital,

personnel, facilities, specimens, reagents, data) wherever possible; warrants access to RSNA resources and support.

• Collaborative– the biomarker has the support of the stakeholder community and

the organizational impetus to sustain continued efforts.

QIBA Committees

Quantitative Magnetic Resonance Imaging [Q-MR] Perfusion, Diffusion, and Flow-MRI (PDF-MRI) Functional MRI (fMRI)

Quantitative Computed Tomography [Q-CT] CT Volumetry in Solid Tumors and Lung Nodules CT Densitometry in COPD Airway Morphology in Asthma

Quantitative Nuclear Medicine [Q-NM] FDG-PET SUV Amyloid-PET

Quantitative Ultrasound [Q-US] Shear Wave Speed for liver fibrosis

Assays are characterized by their:

• Technical Performance

• Clinical Performance Clinical validation Clinical utility

Imaging Assays

QIN

1. Image acquisition variability

2. Radiologist/Reader variability

3. Measurement method variability

Variability in imaging measurements is related to:

8

QIBA Profiles

A QIBA Profile describes a specific performance Claim and how it can be achieved.

QIBA Claim Template• List Biomarkers/Measurand(s)• Specify: Cross-sectional vs. Longitudinal

measurement• List Indices:

– Bias Profile (Disaggregate indices) – Precision Profile

• Test-retest Repeatability (Repeatability coefficient)• Reproducibility (Reproducibility coefficient; Intra-class

Correlation Coefficient [ICC]; Concordant Correlation Coefficient [CCC]).– Specify conditions, e.g.,

» Measuring System variability (hardware & software)

» Site variability» Operator variability (Intra- or Inter-reader)

• Clinical Context

True Biologic Change …

• … is approximately twice the variability

• Clinical Significance of that change needs to be determined by clinical studies.

Topics for Collaboration Discussion:

1. Image acquisition variabilitya) Test objects – physical and virtual

2. Radiologist/Reader variability

3. Measurement method variabilitya) Algorithm comparisons

Reducing variability in imaging measurements is important to both

QIN and QIBA:

13

How do we deal with the fact that different algorithms that purport to measure the same thing give different answers?Methodology for comparing algorithmsMetrics of performance on same taskCriteria for acceptability (compliance).

Measurement method variability

14

Thank you.