image ontology

Image OntologyBarry Smith

http://ontology.buffalo.edu

LogisticsWebcasting: Karel Skinner (NIH/NIDA); Jody Sachs (NIH/NHLBI)http:// irt-video-02.stanford.edu/ramgen/broadcast/view.rm Signatures needed


AcknowledgementsTo the NIH: Roadmap for Medical Research Grant 1 U 54 HG004028 http://nihroadmap.nih.gov/bioinformatics.


Acknowledgements


Image OntologyBarry Smith


What this meeting is aboutto promote interoperability of image and imaging ontologies in the biomedical domainthrough the application of principles of sound ontology construction through the coordination of current ontology development efforts to promote compatibility of image ontologies with ontologies of the biomedical entities which images represent


Special topics

explaining the role of a reference ontology such as the FMAdefining relations among images, features, interpretations, and the underlying reality building an ontology of imaging tools and datapresenting the services of the National Center for Biomedical Ontology


The Reality

Biomedicine is marked by gimmicky, low quality, half-finished ontologies by incompatible, special-purpose, terminologies la UMLSby the assumption that data integration can be brought about by somehow mapping ontologies built for different purposes


Most ontologies (and terminologies) are execrable; but some exemplars of good practice do existas far as possible dont reinventuse the power of combination and collaborationontologies are like telephones: they are valuable only to the degree that they are used and networked with other ontologiesbut choose working telephonesmost UMLS telephones were broken from the start


Why do we need rules/standards for good ontology?Ontologies must be intelligible both to humans (for annotation) and to machines (for reasoning and error-checking): unintuitive rules for classification lead to errors Simple, intuitive rules facilitate training of curators and annotatorsCommon rules allow alignment with other ontologies (and thus cross-domain exploitation of data)Logically coherent rules enhance harvesting of content through automatic reasoning systems


Ontologies built according to common logically coherent ruleswill make entry easier and yield a safer growth path

You can start small, annotating your data/images with initial fragments of a well-founded ontology, confident that the results will still be usable when the ontology grows larger and richer


AssumptionsThere are best practices in ontology development which should be followed to create stable high-quality ontologies Shared high quality ontologies foster cross-disciplinary and cross-domain re-use of data, and create larger communities


A methodology for building and testing ontologiesapplied thus far in the biomedical domain on:

FMAGO + other OBO OntologiesFuGOSNOMEDUMLS Semantic NetworkNCI ThesaurusICF (International Classification of Functioning, Disability and Health)ISO Terminology StandardsHL7-RIM


Biomedical science needs to find uniform computable ways of representing the reality captured in (image) data


Ad hoc creation of new database schemas for each research group / research hypothesisvs.Pre-established interoperable stable reference ontologies in terms of which all database schemas need to be definedTwo Strategies


How to create the conditions for a step-by-step evolution towards gold standard reference ontologies in the biomedical research domain?


The solutionThe OBO Foundry


Goal of the OBO Foundry projectTo introduce some of the features of scientific peer review into biomedical ontology development


Some OBO ontologies are of high qualitySome not


OBO FoundryA subset of OBO ontologies whose developers agree in advance to accept a common set of principles designed to assure intelligibility to biologist curators, annotators, usersformal robustness stabilitycompatibilityinteroperability support for logic-based reasoningThe OBO Foundry


OBO FoundryOBO-UBO / Ontology of Biomedical Realityunifying framework for clinical trial database schemata Anatomy PathoanatomyPhysiologyPathophysiologyMk. II NCI ThesaurusThe OBO Foundry


will provide a small reward for those doing good work in science-based ontology (analogue of peer review)It will provide a step towards the day when interoperability through controlled vocabularies can be enforced through agreements with biological research groups, clinical guidelines bodies, and scientific journals

The OBO Foundry


OBO FoundryOBO-UBO / Ontology of Biomedical Realityunifying framework for clinical trial database schemata Anatomy [FMA?]PathoanatomyPhysiologyPathophysiologyMk. II NCI ThesaurusThe OBO Foundry


OrthogonalityOrthogonality: ontology groups who choose to be part of the OBO Core thereby commit themselves to collaborating to resolve disagreements which arise where their respective domains overlap(They commit themselves to conceiving ontology as a science, not as a hobby)


Reference Ontology vs. Application OntologyA reference ontology is analogous to a scientific theory; it seeks to optimize representational adequacy to its subject matter


Reference Ontology vs. Application OntologyAn application ontology is comparable to an engineering artifact such as a software tool. It is constructed for specific practical purposes.


Reference Ontology vs. Application OntologyApplication ontologies are often built afresh for each new task; commonly introducing not only idiosyncrasies of format or logic, but also simplifications or distortions of their subject-matters. To solve this problem application ontology development shoud take place always against the background of a formally robust reference ontology framework


OBO FOUNDRY EVALUATION CRITERIAFurther criteria will be added over time in order to bring about a gradual improvement in the quality of ontologies included in the OBO core.

http://ontology.buffalo.edu/obofoundry


The ontology is open and available to be used by all without any constraint other than (1) its origin must be acknowledged and (2) it is not to be altered and subsequently redistributed under the original name or with the same identifiers. The ontology is in, or can be instantiated in, a common formal language.The ontology possesses a unique identifier space within OBO. The ontology provider has procedures for identifying distinct successive versions.


The ontology has a clearly specified and clearly delineated content.The ontology includes textual definitions for all terms. The ontology is well-documented.The ontology has a plurality of independent users.The ontology uses relations which are unambiguously defined following the pattern of definitions laid down in the OBO Relation Ontology.


Towards an Ontology of the (Radiological) Image


AcknowledgementsWerner CeustersMatthew FieldingLouis GoldbergDirk MarwedeJose L. Mejino, Jr.Cornelius Rosse


Entity =defanything which exists, including things and processes, functions and qualities, beliefs and actions, software and images


Representation =defan image, idea, map, picture, name, description ... which refers to, or is intended to refer to, some entity or entities in reality

in what follows this or is intended to refer to should always be assumed


Ontologies are representational artifactsWe are interested in ontologies to support high-level scientific research


Ontologies which support high-level scientific research are windows on reality; they are relational entities, which link cognitive agents (and computers) to entities in reality


Images are representational artifacts


Images, too, are windows on reality;they are relational entities, which link viewers to reality


... and they can do this even in the absence of the object


What is an ontology?


A representation of entities


Catalog vs. inventory


Catalog of Types


instancestypes


Type =Def. that which (1) a collection of similar instances share in common and which (b) is a potential object of investigation by science

Types existed many trillions of years before there were words, concepts, or scientific theories


TypesTypes exist, through their instances, in objective reality including types of image, of imaging process, of brain region, of clinical procedure, of protocol, of assay, etc.


Two kinds of representational artifactDatabases, inventories, images: represent what is particular in reality = instances Ontologies, terminologies, catalogs: represent what is general in reality (exists in multiple instances) = types (universals, kinds)


Images represent instances in reality


Ontologies represent types in reality


Ontologies do not represent concepts in peoples heads


lung is not the name of a conceptconcepts do not stand in part_ofconnectednesscausestreats ...relations to each other


The clinician has a cognitive representation of what is general, based on knowledge derived from textbooks


typesmammalinstancesfrog


An ontology is like a scientific text; it is a representation of types in reality


Complex representations=def. representations built out of parts (sub-representations) which are also representations


Representational units =defterms, icons, bar codes, alphanumeric identifiers ... which (1) refer, or are intended to refer, to entities in reality and (2) which are not built out of further sub-representations

Representational units are the a t o m s in the domain of representations


Modular representation =def.a representation which is(1) built out of representational units(2) each of which refers to an entity in reality(3) these representational units form a structure S(4) the corresponding entities in reality form a structure S(5) S mirrors (or is intended to mirror) S


Periodic TableThe Periodic Table


Ontology =def. A modular representational artifact whose representational units are intended to represent1. types in reality2. the relations between these types which obtain universally (= for all instances)lung is_a anatomical structurelobe of lung part_of lung


What is special about those representations we call images?


Images are complex but non-modular representations they are not built out of representational units


Pixels, silver halide crystals, do not represent


The sub-representations of an image are regions


Images are continuous (analogue) representations = they can be subjected to regional segmentation in a variety of cross-cutting ways


Continuants vs. OccurrentsAnatomy vs. PhysiologySnapshot vs. VideoStocks vs. FlowsCommodities vs. ServicesProducts vs. Processes


Continuantshave continuous existence in timepreserve their identity through changeexist in toto whenever they exist at allOccurrents have temporal partsunfold themselves in successive phasesexist only in their phases


You are a continuantYour life is an occurrent

You are 3-dimensionalYour life is 4-dimensional


For continuants we take snapshots at successive points in timet1t3t2


For occurrents we take videosGastrulation


All occurrents are dependent entitiesThey are dependent on continuants as their bearers (participants, agents ...)


Top-Level OntologyContinuantOccurrent(always dependent on one or more independent continuants)IndependentContinuantDependentContinuant


Top-Level OntologyContinuantOccurrentIndependentContinuantDependentContinuantimage bearer (film, medium)act of creating an image, viewing, interpreting, processingimage feature, attribute


Top-Level OntologyContinuantOccurrentIndependentContinuantDependentContinuantFunctioning Side-Effect, Stochastic Process, ...Function


Subtypes of ContinuantsIndependent continuants (organisms, organs, cells)Dependent continuants (features, attributes, qualities, roles, functions)Spatial regions, environments


Top-Level OntologyContinuantOccurrentIndependentContinuantDependentContinuantFunctioning Side-Effect, Stochastic Process, ...Function


Top-Level Ontologyinstances (in space and time)


Some dependent entities are monadic they have one single independent bearer or carriera feeling of self-hatreda temperature qualitya shape quality a disease


Some dependent entities are relational (have more than one bearer)an act of kissing (occurrent)an act of photographing (occurrent), relates a camera (film) and a photographed object a relation of reference (aboutness) between an image and an imaged object (dependent) continuant)


Some dependent continuants are realizableprojection of a filmutterance of a sentenceapplication of a therapycourse of a diseaseexecution of an algorithmrealization of a planexpression of a gene


Functions vs Functioningsthe function of your heart = to pump blood in your bodythis function is realized in processes of pumping blood


Video images are continuants which are created through and concretized in occurrentsShooting the videoProjecting Viewing (acts)Interpreting (acts)Annotating (acts)Processes in reality which the image


(Developmental) Anatomy is about Continuants adultembryofetus


Realist PerspectivalismThere is a multiplicity of ontological perspectives on reality, all equally veridical i.e. transparent to reality

Two photographs taken from different angles can both be veridical representations of the same reality


Cardinal perspectivesA (snapshot) view of the continuants in a given domain of realityA (videoscopic) view of the occurrents in a given domain of reality


Cardinal PerspectivesGranularity

(Molecular, cellular, organ, organism ...) (Pixel vs. Region)


Modularity


NEPHRON


FUNCTIONAL SEGMENTS


Entities in granular layers connected by instance-level part-of relationsJohns life


Relations crossing the border between continuants and occurrents are never part-relationsJohns life


Granularityspatial regionindependent continuantparts of independent continuants are always independent continuants


Granularityspatial regionsubstanceparts of spatial regions are always spatial regions


Granularity

processparts of processes are always processes


FMA: a cross-granular representation of the canonical adult human organism embracing both the regional and the structural view


Cardinal perspectivesRegional vs. structural


Regions (RCC-8)


Fiat vs. Bona Fide Boundarieswww.enel.ucalgary.ca/ People/Mintchev/stomach.htm


Connectedness and ContinuityTwo continuants are continuous on the instance level if and only if they share a fiat boundary. Regional parts related by continuous_with Structural parts by attached_to (muscle to bone) synapsed_with (nerve to nerve and nerve to muscle)


2-D Regions in the Image Representing 3-D Structures in Realitywww.enel.ucalgary.ca/ People/Mintchev/stomach.htm


Raw imagesvs.processed imagesannotated imagesimages in the presence of the imaged objectbar coded images (linked to the imaged object by proxy)


Dependent Continuants (Attributes of the Image)Monadic:image qualitiesimage shapesRelationalofness or aboutness (the image is of some entity outside the image)


The Methodology of AnnotationsScientific curators use experimental observations reported in the biomedical literature to link gene products with GO terms in annotations.

The gene annotations taken together yield a slowly growing computer-interpretable map of biological reality, The process of annotating literature also leads to improvements and extensions of the ontology itself, which institutes a virtuous cycle of improvement in the quality and reach of future annotations and of future versions of the ontology.


The Methodology of AnnotationsRadiologists use images to link imaged entities (anatomical structures ) with anatomy terms in annotations.The more coherent the ontology from which the annotations are drawn the more powerful the reasoning tools which we can apply to mine the annotated data.


When we annotate the record of an experimentwe use terms representing types to capture what we learn about the instancesthis experiment as a whole (a process)these instances experimented upon the instances are typical they are representatives of a type


When we annotate an imagewe use terms representing types to capture what we know about certain instances:this image as a wholethese regions in the imagethese qualities of these regionsthat part of the world the image representsas representative of a typeas this specific instance (the fracture in your thumb)


=def. a mapping of regions in an image segmentation to -pairs (= to instances in reality identified as instances of certain types)Image Interpretation


California Land CoverxA map is a mapping between points in reality and the types represented in a map legend


What is a radiological image?a pattern of radio-opacities and radio-translucencies (regions)everything else is interpretation (= the mapping of regions to types)


ArtifactsNot every region in a diagnostic segmentation need be mapped to an pair in reality:an image can include regions which do not designate and are not interpreted as designating


radiodensityCT-Scan of Thorax Showing the Pulmonary Arteries with Embolism


Region of radiodensityinterpretation maps to an (instance of) abnormal accumulation of fluid in the right pleural cavitycould equally be blood (hemothorax), lymph (chylothorax) or an excessive amount of pleural fluid (pleural effusion)ideally map to types as catalogued in a reference ontology such as the FMA


Interpretations of images1. based purely on information provided by single regions2. refined by ancillary information fromother regions within the imagepatient identifiers clinical history of the patient...


radiodensityCT-Scan of Thorax Showing the Pulmonary Arteries with Embolism relative radiolucency


Ancillary information from within the imageinterpretation of relative radiolucency: shape and location suggest mapping to thrombus (similar thrombi lodged in the pulmonary arterial bed further downstream would lead indirectly to an increase in pleural fluid, making the diagnosis of pleural effusion most likely among the options)


radiodensitymaps to: CT-Scan of Thorax Showing the Pulmonary Arteries with Embolism relative radiolucencymaps to


Fundamental distinctionactual entities present in the image (2D regions) vs. most likely interpretation assigned to them by observerA reference ontology for radiology should be about the former, and should be joined to reference ontologies for anatomy and pathoanatomy


Regional segmentationsactual entities present in the image are regional segmentations created by our fiat demarcationsthere are diagnostically relevant segmentations (image patterns)but also other kinds of segmentations (e.g. segmentations according to quantitative and qualitative spatial coordinates) Theory of Granular Partitions


Canonical (annotated) image ontologyAnnotated Image = Def. A partial function mapping regions in a regional segmentation of a 2D surface onto pairs in the imaged reality whereby:Departures from the canonical case = interpretational errors


Importance of common relationsThe success of ontology alignment demands that ontological relations (is_a, part_of, ...) have the same meanings in the different ontologies to be aligned. See Relations in Biomedical Ontologies, Genome Biology May 2005.


Pleural CavityInterlobar recessMesothelium of PleuraPleura(Wall of Sac)VisceralPleuraPleural SacParietal PleuraAnatomical SpaceOrganCavitySerous SacCavityAnatomicalStructureOrganSerous SacMediastinalPleuraTissueOrgan ComponentOrgan CavitySubdivisionSerous SacCavitySubdivisionpart_of is_a


OBO Foundry CriteriaThe ontology uses relations which are unambiguously defined following the pattern of definitions laid down in the OBO Relation Ontology.Assumption: if we are to create ontologies which support logical reasoning then we need to take time and instances into account


is_a (sensu UMLS)A is_a B =def A is narrower in meaning than B

disease prevention is_a disease


Three kinds of relations Marys heart part_of Mary Marys heart instance_of heart human heart part_of human


is_aA is_a B =def

For all x, if x instance_of A then x instance_of B

cell division is_a biological process


is_aA is_a B =def



adult is_a child ???


is_a (for occurrents)A is_a B =def




is_a (for continuants)A is_a B =def

For all x, t if x instance_of A at t then x instance_of B at t

abnormal cell is_a celladult human is_a humanbut not: adult is_a child


two kinds of parthoodbetween instances:Marys heart part_of Marythis nucleus part_of this cellbetween typeshuman heart part_of humancell nucleus part_of cell


Part_of as a relation between typesheart part_of human being ?human heart part_of human being human being has_part human testis ?testis part_of human being ?


Definition of part_of as a relation between typesA part_of B =Def all instances of A are instance-level parts of some instance of B

human testis part_of adult human being


part_of (for occurrents)A part_of B =def.

For all x, if x instance_of A then there is some y, y instance_of B and x part_of ywhere part_of is the instance-level part relation

EVERY A IS PART OF SOME B


part_of (for continuants)A part_of B =def.

For all x, t if x instance_of A at t then there is some y, y instance_of B at t and x part_of y at twhere part_of is the instance-level part relation

ALL-SOME STRUCTURE


GALEN: Vomitus contains carrotAll portions of vomit contain all portions of carrotAll portions of vomit contain some portion of carrotSome portions of vomit contain some portion of carrotSome portions of vomit contain all portions of carrot


BrainCerebrumTemporal Mesial temporalHippocampusCerebral cortexCVLTFrontal CognitiveimpairmentCognitionAssessmentNeuropsychologyAmnesiaMemoryLearning


Often, universal relational assertions can be gained by focusing on the order of termschild transforms_into adultadult transformation_of childpneumococcal virus causes pneumococcal pneumoniapneumococcal pneumonia caused_by pneumococcal virushippocampus has_condition amnesia amnesia condition_of hippocampus


A part_of B, B part_of C ...The all-some structure of the definitions in the OBO-RO allows cascading of inferences (i) within ontologies(ii) between ontologies(iii) between ontologies and repositories of instance-data (e.g. EHRs)


Cascading inferencesWhichever A you choose, the instance of B of which it is a part will be included in some C, which will include as part also the A with which you beganThe same principle applies to the other relations in the OBO-RO:

located_at, transformation_of, derived_from, adjacent_to, etc.


continuous_with on the instance levelis always symmetric

On the type level we have:

lymph node continuous_with lymphatic vessel

but there are lymphatic vessels (e.g. lymphs and lymphatic trunks) not continuous with any lymph nodeContinuity on the type level is not symmetric.


Instance levelthis nucleus is adjacent to this cytoplasmimplies:this cytoplasm is adjacent to this nucleus

Type levelnucleus adjacent_to cytoplasmNot: cytoplasm adjacent_to nucleus


Rules on typesDont confuse types with instancesDont confuse instances with leaf nodesDont confuse types with ideasDont confuse types with ways of getting to know typesDont confuse types with ways of talking about typesDont confuse types with data about types


Rules on termsTerms should be in the singularAvoid abbreviations even when it is clear in context what they mean (breast for breast tumor)Think of each term A in an ontology is shorthand for a term of the form:the type A


Rules on DefinitionsThe terms used in a definition should be simpler (more intelligible) than the term to be defined; otherwise the definition provides no assistance to human understandingto machine processing


Definitions should be intelligible to both machines and humansMachines can cope with the full formal representationHumans need clarity and modularity


The Use-Mention ConfusionSwimming is healthy and has 8 letters

Poland =def. The name of Poland


BIRNLexmore properly, and more simply:stratum radiatum of hippocampus =def. the layer of the hippocampus which ...


When defining terms use Aristotelian definitionsThe definition of A takes the form:

an A =def. a B which ...

where B is As parent in the hierarchy

A human being =def. an animal which is rational


FMA ExamplesCell =def. an anatomical structure which consists of cytoplasm surrounded by a plasma membrane

Plasma membrane =def. a cell part that surrounds the cytoplasm


Use of Aristotelian definitionsMakes defining terms easierEach definition encapsulates in modular form the entire parentage of the defined termThe entire information content of the FMAs term hierarchy and definitions can be translated very cleanly into a computer representationNow accepted by GO


RadLexGoal: to provide a uniform structure for capturing, indexing, and retrieving a variety of radiology information sources, such as teaching files, research data, and radiology reports. For what purpose? just for information retrieval for hospital management? translational medicine?for reasoning, and integration with other terminologies/ontologies?


RadLex Term Categories= groupings of terms appearing in a typical radiology report 2.3.1. Examination Type This category describes what form of imaging was performed. For example MRI of the spine or CT of the abdomen. Examination type is a term category?


RadLex Term Categories2.3.2. Technique Imaging equipment: manufacturer, model 2.3.4. Observer Context Who viewed the images An ontology is a representation of types; it should link to a database containing names of particular instances (e.g. manufacturers)(a controlled vocabulary for astronomy would not include the name 'Buzz Aldrin')


RadLex Term Categories2.5. Image Quality This category defines the overall quality of the examination of the examination? or of the image?


RadLex Term Categories2.7.1. Visual Features These terms describe features on the image that can be described without reference to specific physical, anatomic or pathological processes or structures. Examples include, opacity, high signal, low attenuation, and companion shadow. runs together dependent continuants with independent continuants


RadLex Term Categories2.7. Findings This category embodies the salient observations about the images. These terms can be considered as a continuum from observations and features to syndromes, diagnoses, or etiologies (e.g., adenocarcinoma, pneumothorax, pneumonia, fracture). Runs together everything and its uncle


RadLex Term Categories2.8.1. Causal Relationships Causal relationships typically occur between findings (e.g., the pleural effusion causes atelectasis, the increased signal is due to a brain tumor, the pneumonia is manifested by consolidation).

Mixes cognitive acts, products of cognitive acts, phenomena in reality to which they refer, and names -for all of these. How could a causal relationship be a visual feature?


RadLex is a subsumption hierarchy (child is_a parent)upper lobe of the right lung is_a right lung OR:

term for upper lobe of the right lung is_a term for right lung

lungs is_a thoraxneck muscles is_a thorax


pneumoniapneumonia is_a morphologic and physiologic processes pneumonia is_a diagnoses and etiologies


The methodology of reference ontologiesis future safecan allow application ontologies developed for special purposes to take advantage of logical tools and methods for reasoning across large bodies of data


e.g. menopause part-of aging, aging part-of death

Database searching is one of the important kinds of reasoning we want to enablewith thanks to Andrew LonieChristian Freksa, Neighborhood Preservation Table, Conceptual Neighborhood and its role in temporal and spatial reasoning

http://www.tahoecons.ca.gov/library/rip_data/rd_grnd_samp.html

image ontology

Documents

good ontology

wellfounded ontology

reference ontology

ontology of imaging

sound ontology construction

imaging ontologies

halffinished ontologies

edumost ontologies