extraction and analysis of biomarkers from medical images bernard gibaud medicis, ltsi, u1099 inserm...

Extraction and analysis of biomarkers from medical

images

Bernard Gibaud

MediCIS, LTSI, U1099 InsermFaculté de médecine, Rennes

[email protected]

1

Imaging Biobanks: from genomic to radiomic in the era of personalised medicine, March 5, ECR 2015, Vienna

Overview

• Introduction (definition of imaging biomarkers)

• Part 1. Change of paradigm (led by imaging biomarkers) and need to share semantics about imaging biomarkers

• Part 2. Toward using ontologies and other semantic technologies in biomarkers sharing and use

• Conclusion 2


Part 1. Change of paradigm and need to share the semantics of

imaging biomarkers

8


Patient management (today)

Reality

Human sujectAnimal subject

Specimenetc.

Acquisition

Images

MR imageCT image

PET imageetc.

Imaging biomarkers

Processing

Volume of anatomical structure

Fractal dimensionMean reg. blood

volumeLesion load (MS)

etc.

FactsPlans, etc.

Decision

Diagnosis of ADDiagnosis of MS

Resp to treatmentetc.

9


Knowledge-based patient management (tomorrow)

10

Clinical case

Image acquisition

Clinical decision models

determines

Imaging biomarkers

determines

Detailed imaging protocol

determines

Imaging biomarkers’calculation

Application of clinical decision models

Final clinical decision

Knowledge-based scheduling stack Classical workflow


Modeling of knowledge

11

Clinical case

Clinical decision models

determines

Imaging biomarkers

determines

Detailed imaging protocol

determines

Need to model role(s) of biomarkers in clinical decision

Need to model how to obtain imaging biomarkers

Need to model essential clinical features driving clinical decision


Towards deep knowledge supporting clinical decision

• Today: classical framework– Decision models are empirical

• evidence of relevance of a decision model for a particular use

– no explanatory theory-based model

• Ideally: Deep knowledge framework– Decision models based on personalized models and

simulation (cf. Virtual Physiological Human vision)– theoretical models of underlying phenomena

• A promising way to progress: radiogenomics

12


Towards deep knowledge supporting clinical

decision: Radiogenomics

• Goal: to relate imaging traits to molecular markers (e.g. gene expression signatures) using association maps – To improve our knowledge of the biological

characteristics of the tissues that are highlighted/measured with those imaging traits

• Added value– Helps relating particular imaging traits to genes and so,

through information ressources like GO and GenBank, to potential biological explanations of their relevance

13


Towards deep knowledge supporting clinical

decision: Radiogenomics: example of GBM

• Relate 10 imaging traits derived from MR images to various tumor gene expression signatures (gene clusters), for example:– Contrast enhancement correlated with Hypoxia and ECM– Contrast / Necrosis ratio correlated with EGFR overexpression – Mass effect correlated with Proliferation– Infiltrative correlated with Immune cells– …

14


Consequence (1/2)

• It is fundamental to share imaging biomarkers at a broad scale so that to more easily – compare their performance– understand of their deep meaning (i.e.

what biological features they characterize)

– potentially invent new ones for other purposes

15


Consequence (2/2)

• Which assumes– A common model for describing

biomarkers and actual measurements – Suitable computer infrastructures

• to store, share and reuse research imaging data, i.e. imaging biobanks

• to relate it to biomolecular markers• as well as pathology data, and of course

clinical data

16


Part 2. Toward using ontologies and other semantic technologies

in biomarkers sharing and use

17


Basic questions

• How to model imaging biomarkers ?• How to better share them within and

between imaging biobanks

Use of ontologies and other semantic web technologies– Languages– Software tools

• Reasoners, query engines, « triple stores »


18

19

Ontologies

• Definition (informatics and AI)– « a formal, explicit

specification of a shared conceptualization »

(Gruber 1993)

• Two basic aspects– A shared vocabulary– Formal semantics : axioms

expressed in a logical language


20

Formal semantics

• Definitions of classes of objects– Taxonomy of classes: subsumption (i.e. ’subClassOf’ relation)

• glioblastoma ‘subClassOf’ ‘brain tumor’– Instanciation (‘is a’ relation between an individual and a

class)• tumorPatientX ‘is a’ ‘glioblastoma’

• Definitions of properties– Taxonomy of properties– Domain and range, inverse properties, etc.

Processing by a reasoning engine– Check consistency– Classification of instances– Semantic query


Toward modeling imaging biomarkers and sharing them within and between imaging

biobanks

• A three-step approach – 1st step: Conceptual modeling step– 2nd step: Ontology design step– 3rd step: Implementation and deployment

step


21

Towards an ontological model of imaging biomarkers:

Conceptual modeling step

• Need to distinguish between several interrelated aspects – Biomarkers as measureable quantities

(that characterize a biological object)

– Biomarkers as measurement instruments

– Biomarkers as decision support instruments

22




• Biomarkers as measureable quantities– can be measured through some measurement process– using different sorts of quantity value (num., ordinal,

nominal, etc.)– addressing a specific measurand (quality being measured)– characterizing a real-world object or process

• Examples– Sum of the longest-diameter for all target lesions of Patient

X– Brain gyrification index of frontal lobe f of Patient Y

23




• Biomarkers as measurement instruments– used as instruments in some measurement process– in practice, these are complex instruments,

composed of several parts, i.e.• Acquisition instruments: imaging system + suitable

imaging protocol + patient preparation protocol• Possibly complex image processing (preprocessing +

segmentation + measurement of a specific parameter over a ROI)

• Examples– Several examples available in QIBA* web pages

24

* Quantitative Imaging Biomarkers Alliance (RSNA)




• Biomarkers as decision support instruments– specifies what kind of clinical decision, e.g. for

staging, prognosis, response assessment, etc.– in which clinical context (disease, pathway,

detailed stratification criteria)– with what validation status

25



Ontology design step

• Need to relate our domain’s entities and relationships to existing upper level ontologies– To ensure consistency with other views from

other viewpoints: physics, biology, computer science, etc.

– Especiallly, absolutely needed to connect descriptions of similar phenomena at various scales or using different imaging modalities

26


Relevant ontologies and models

• Upper level ontologies: BFO or DOLCE– Provide a common modeling framework & top-level entities

• Measurement and information artifacts: OBI / IAO• Qualities: PATO• Provenance: PROV• Imaging: RadLex, AIM, OME• Imaging datasets: OntoNeuroLog• Medicine in general: SNOMEDCT, ICD, NCIT, etc.• Imag. biomarkers: QIBO (Buckler et al.), BiomRKRS (Ofoghi

et al.)

27But of inequal quality and completeness


QIBO (Buckler et al. 2013)(Quantitative Imaging Biomarkers Ontology)

• Proposed by Buckler and coll. in 2013

28


Main entities introduced in QIBO

29

Quantitative Imaging

Biomarker

Biomarker use

Imaging agent

enhances (is enhanced by)

Post-proc algorithm

estimates (is estimated by)

Acquisition device

generates(is generated by)has (is

present in)

Imagingsubject

images (is imaged with)

Indicated biology

involves (participates

in)Biological

target

measures (is measured by)

is measu-rement of (is quantified by)

is used for

(uses)

is applicable to (pertains to)

is benefit from

(is used in subject)

is of modality (uses imaging

agent)

Biological interventionhas undergone

(is performed on)

influences(is influenced by)


QIBO (Buckler et al. 2013) (Quantitative Imaging Biomarkers Ontology)

• Many interesting aspects– Taxonomy of imaging subject– Taxonomy of biological targets (of imaging agents)

• Distinguishing between cellular, molecular, organismal and systemic targets

– Available in OWL

• But unfortunately– Not based on an upper level ontology, nor on existing ontologies– No taxonomy of usal biomarkers– Objects properties are defined but never used– Poor documentation (OWL file not consistent with Buckler’s

paper)

30


BiomRKRS (Ofoghi et al. 2014)

31



32



• Several interesting aspects– Common model suitable to any kind of biomarkers– Reuse of many relevant ontologies

• But unfortunately– Not based on an upper level ontology– Does not take into account imaging biomarkers’

complexity– Version in OWL not available

33


• Both require the collaboration of specialists with complementary background– Radiology and other imaging specialities– Image processing– Physics of imaging modalities– Ontology engineering– Standards, especially DICOM

34


Ontology design step


Proposed ontology structure

35

e.g. DOLCE or BFOUpper level ontology

Set of generic ontologies

PATO UO OBI/IAO PROV FMA RadLex etc.

Core ontology of imaging biomarkers IBO

Specific ontologies of speciality domains’ imaging biomarkers

NeuroIB

CardiovascIB

OncoIB

qualities units of meas. info artef. provenance anatomy radiology


…..

Implementation and deployment step

• Deploy experimental systems in research infrastructures and imaging biobanks.

• How ?– First, by complementing existing infrastructures– then, encourage evolution/migration to native

ontology-based data schemas

36


37

Imaging Biobank #1

Images and imaging biomarkers

SPARQL queries

images


Sharing information about imaging biomarkers - what they are measuring - how they are calculated - what is their clinical role

Sharing / retrieving the measurement data they allowed to produce

Ontology-based descriptions of

imaging biomarkers

Imaging Biobank #n


…..

Federation of Imaging Biobanks


imaging biomarkers

Query tool


imaging biomarkers

38

Imaging Biobanks: from genomic to radiomic in the era of personalised medicine, March 5, ECR 2015, ViennaImaging Biobank #1


Ontology-based descriptions of imaging

biomarkersFederation of

Imaging BiobanksOntology-based descriptions of

imaging biomarkers

…..Imaging Biobank #n


Ontology-based descriptions of imaging

biomarkers

This approach can provide a natural set of standards to support the interoperability with other information such as specimen and derived biomolecular data

Biobank #1

Specimen data

…..

Biomolecular data

Federation of Biobanks

Specimen data Biomolecular

data

Ontology-based desc. of specimen and derived

biomolecular data

Biobank #n

Specimen data Biomolecular

data

Ontology-based desc. of specimen and derived

biomolecular data

images

specimen

biomolecular data

39

Conclusion / Summary

• We underlined the importance of imaging biomarkers in research (both clinical and translational research) and future decision support systems for care delivery

• We underlined the critical importance of designing suitable ontological models of imaging biomarkers, and made suggestion regarding their development

• And finally we suggested how such models could help implementing interoperable federated repositories (especially imaging biobanks)


Thank you for your attention

40


Acknowledgements

• Former partners of the NeuroLOG project (supported by ANR)

• CrEDIBLE project (CNRS initiative for Big Data in science), and my colleagues from this project

Gilles Kassel Michel Dojat Bénédicte Batrancourt Lynda Temal Johan Montagnat Alban Gaignard (Amiens) (Grenoble) (Paris) (Paris)

(Sophia-Antipolis)

41


42

Some references• Horrocks I. Ontologies & the semantic web. Comm. of the ACM,

2008;51(12)58-67.

• QIBO: A.J. Buckler et al. Quantitative Imaging Biomarker Ontology (QIBO) for knowledge representation of biomedical imaging biomarkers. J Digit Imaging (2013) 26:630-641.

• BiomRKRS: B. Ofoghi et al. A biomarker retrieval and knowledge reasoning system. Proc. of the 7th Workshop on Health Informatics and Knowledge Management (HIKM 2014), Auckland (NZ).

• DICOM standard: http://dicom.nema.org/


extraction and analysis of biomarkers from medical images bernard gibaud medicis, ltsi, u1099 inserm...

Documents

era of personalised

viennaimaging biobanks

imaging biomarkers chairperson

biomarkers definitions

analysis of biomarkers

nowfocused clinical

controlled clinical

treatment key aspect