1

Ontology-based annotation of atomic and abstract petrographic image features

Luís A. Lima Silvab, Mara Abela,

John A. Campbellb, Laura S. Mastellaa, Luiz F. De Rosc

aInstituto de Informática, Universidade Federal do Rio Grande do Sul; Av. Bento Gonçalves, 9500; Campus

do Vale; Caixa Postal: 15064; CEP: 91501-970; Porto Alegre, RS, Brasil. E-mail: {mastella; marabel}@inf.ufrgs.br

bDepartment of Computer Science, University College London; Malet Place, London; WC1E 6BT, UK. E-mail: {l.silva; j.campbell}@cs.ucl.ac.uk

cInstituto de Geociências, Universidade Federal do Rio Grande do Sul; Av. Bento Gonçalves, 9500; Campus do Vale; Caixa Postal: 15001; CEP: 91501-970; Porto Alegre, RS, Brasil. E-mail: lfderos@inf.ufrgs.br

Abstract Ontology-based annotation of images is intended to allow standard and explicit meanings to

be assigned to objects and features that an observer perceives there. In practice, interpretation of images in sedimentary petrology dealing with atomic items by themselves in an ontology (such as concepts, attributes and values) is insufficient to capture higher-level semantics (e.g. evidence, findings, justifications). As a response to that problem, the paper proposes two conceptual annotation levels: atomic, and abstracted. We describe here a knowledge-based model that helps users to make their image-interpreting knowledge explicit and apply in interpretation of oil-reservoir rock images. Two concepts are specific to this work: visual chunks, which represent visual patterns of domain terms, and K-graphs, which are used to state knowledge-intensive connections between items of observed evidence and interpretations for this evidence. The essential details of the framework are not at all constrained by the geological application: they can be applied without change in any other subject-areas where reasoning and annotation of images are the primary requirement.

Keywords: Ontology-based annotation of images, image interpretation, visual chunks, sedimentary petrography

1 Introduction A certain item of information can be described

loosely as data or knowledge by different people, or even by the same person at different times. In the perspective of artificial intelligence (AI), however, there is a clear distinction. The item is data if it is passive and uninterpreted. It is knowledge when it is interpreted, or involved actively in some process of interpretation. The interpretation mechanism, whether it is an AI program or something in a human brain, is itself knowledge-intensive, and derives or extracts new knowledge from the given information.

If the information is symbolic, AI has no shortage of symbolic means of interpretation, e.g. rules in expert systems. When it is primarily sense-data, such as

sensor inputs to a computer, varieties of neural-net or genetic mechanisms permit or help interpretation to occur. When the information consists of images more complicated than those for which traditional methods of AI computer vision work, the situation is much more challenging, and interpretation - extracting knowledge from the images - has still not found its exact analogue of the standard expert systems, neural nets etc. that support the derivation of knowledge from non-image information.

In the last few years, the most common response to the challenge has been the development of ontologies and ontology-based annotation of images. Unlike the methods above, the annotation schemes are not automated; it is expected that human users will provide the detailed annotations of particular images, subject to the contents and constraints of the ontology. The point of the completed annotations is to offer a knowledge

2

base (knowledge = image data + annotations) B which can assist interpretation of new images of the same kind. Typically a user will input some incomplete characterization C of a new image, which will be matched against corresponding entries in B to achieve retrieval of the most relevant images and their complete annotations. The user can then study the retrieved knowledge and adapt it, or learn from it, with respect to the new image. (We remark in passing that we appreciate the strong similarity of this process to analogical and case-based reasoning (CBR) [1]; we are already using CBR in our subject-area, but this is outside the scope of the present paper).

This paper is a contribution to the activity of designing ontological frameworks and preparing ontologies for image knowledge bases. It is in the same line of research as others with the same broad intention, e.g. [2-7].

An expert's view of knowledge includes components at various conceptual levels, starting with a basic level at which the irreducible or atomic terms/labels of any eventual ontology are stated. Typically, existing ontological approaches to the expression of knowledge are "all-in" approaches, mixing together items that belong at different conceptual levels. Also typically, people who exploit such ontologies operate largely, even exclusively, by using atomic terms as cues for retrieval of annotated image knowledge.

We propose here a clarification and simplification of such approaches, by distinguishing just two levels (one of which is atomic) - which, broadly speaking, mirrors the kinds of material that novices and experts respectively use as cues when they wish to interrogate a knowledge base of annotated images. Our scheme has emerged from extensive experience with manipulation and annotation of stocks of images in sedimentary petrology, in the PetroGrapher project.

While desirable, such a simplification cannot be achieved without suitable means of support - as the multi-level nature of typical previous ontological schemes demonstrates. Our "means of support" are two knowledge structures: visual chunks, and knowledge graphs (K-graphs).

We give further information about these structures in sections 2 and 4 below. Section 2 also indicates technical issues that make the subject of our paper relevant to a wider audience inside computer science. We summarize the PetroGrapher effort in section 3, to give some background to the origins of this work. Section 5 deals with ontology as such, from our perspective. This is followed by a section concerning work in progress, and some concluding and summarizing remarks.

2 Relevant knowledge-based research issues for computer science The area of application of our work is the

description and interpretation of oil-reservoir rock images in petroleum exploration. Successful interpretation involves the capture of expert-level knowledge about the application, and making user-friendly novice-level access possible to image databases and their annotations, e.g. for training. But such objectives are not specific to geology. Other applications with the same general character, and where an ontological approach is relevant, include description and interpretation of X-ray images in industrial and medical domains, and diagnosis of human diseases in biochemical assays. Any improvements in AI technique for any one application have immediate significance for a wide variety of applications involving images. And the state of the art in image interpretation leaves many openings for further research. One aim of the present section is to indicate some of these openings.

One question that we have treated only informally so far is the question of what features in images users actually wish to annotate, and how the physical extent of what is annotated can be made clear to others who access the image databases subsequently. Often an annotation is attached to some representative point in an image to refer to a region or property for which that point is presumably typical. It is then hoped that a subsequent user's general intelligence and perception will allow that user to identify the region or property that the annotator intended. If a region's boundary is clear-cut and its appearance contrasts well with its surroundings, as in some parts of Fig. 4, this is a simple and reliable process. If it is not, which is more common in many image-based subjects besides geology, then how can we improve the reliability of annotation so that users are helped to see precisely what the annotator wants them to see? This will involve research that exploits connections between AI and cognitive science, and improves them.

We have observed that novices identify and reason about the low-level (atomic) features of images, while experts prefer more abstracted and higher-level terms which also figure in (and were probably initially derived from) their activities of image interpretation and associated problem-solving. This obviously leads to a two-level ontology. Because of the expected uses of PetroGrapher, we had to find a means of supporting both classes of user. In particular, it was desirable to give novices access to expert-level knowledge in ways that were consistent with their limited understanding of sedimentary geology. It has turned out that we did not need any ad hoc scaffolding or patching of the two-level ontology to achieve this; with two conceptual tools which have principled justifications, we have

3

been able to bridge the levels effectively. These tools are visual chunks and K-graphs.

Visual chunks were introduced in [8], in a geological application, for acquisition and representation of visual knowledge. The concept arises from ideas proposed earlier [9, 10] for chess and other fields. A visual chunk in geology is an aggregation of rock properties to represent geometric features drawn from a predefined ontology and significant to an expert when/if seen in an image. We assert that visual chunks can also be used as a way of organizing annotation schemes in any image-based area where interpretation requires expert knowledge.

The visual knowledge acquired and annotated by visual chunks is most conveniently associated with high-level interpretations using knowledge graphs [11]. A knowledge graph (K-graph) can be understood as a knowledge acquisition and representation model, having greater expressivity and more expressivity concerning granularity when compared to other formalisms that associate items of evidence with hypotheses, such as production rules or Bayesian nets. Both visual chunks and K-graphs are, in effect, knowledge representations. As such, they are potentially relevant anywhere in AI if one is trying to find the best representation's for a given job. We are interested to recommend them for that purpose, especially in applications concerned knowledge related to images.

We conclude this section by mentioning one further opening for research which has appeared during our work at which we have not yet followed up in detail. Constructing ontologies is not an exact subset of the activity of knowledge acquisition and knowledge engineering, which we have found to be well supported by reliable methodologies, particularly [12]. It is therefore desirable to consider what changes are needed in such methodologies so that building ontologies can be made as systematic as possible. Moreover, although knowledge engineering traditionally focuses on the expert, our experience in the PetroGrapher project is that "novice expertise" should also be elucidated, in the same way, if we are to have a good basis for building the atomic level of an ontology. (One of the four levels that CommonKADS recognizes, the domain level, looks particularly appropriate for this).

We now give a little background to the PetroGrapher project, before turning to the ontological technical issues that it has raised.

3 The Petrographer project The PetroGrapher project was firstly proposed to

systematize the task of rock-reservoir description and interpretation, providing the user – a geologist trained

in sedimentary petrography – with a set of programs that support each stage of the petrographic description and analysis. The system allows the user to describe geological features in a novice level of expertise and connects these features with a high level evidence level that supports inference. The recognized evidences allow to suggest interpretations to (1) compositional name of the rock, (2) provenience area of the sediments that form the rock; (3) diagenetic environment where the rock has been consolidated; (4) diagenetic sequence of events that has caused to transform the sediments in a rock.

The whole knowledge content applied by the system, including the new information provided by the user, are stored in a relational database system that acts as a repository for the knowledge and data. These repository is further searched by the reasoning methods that infer new information, which is also stored in the database. The repository is investigated by geologists applying a multidimensional analytic interface of query.

The system has now been in use in a petroleum company by the geologists involved in qualitative reservoir analysis in order to analyze the potential of new deposits and also to support the definition of production methods in oil fields under productions. The system was defined to attend two levels of users: the novice level, typically a technician trained in petrographic description with few experience in interpretation, and the expert level, a geologist responsible for the reservoir evaluation and production techniques, whose decision is based on the information provided by the technician and firstly interpreted by the system.

It is reasonable to foresee that the same software structure and repository can serve as a foundation for integration of other related kinds of geological expert knowledge such as paleontological recognition or core description, e.g. concerning annotation of features and regions in the images themselves. In any case, our experience with PetroGrapher has been partly responsible for leading us to work on annotation and on related ontological issues, as described elsewhere in this paper.

4 Visual chunks The notion of “chunk” has a long history in AI and

cognitive science. It is not the history of a single concept, though: several different interpretations for the same word have existed, and often coexisted. They all have in common the viewpoint that information needed for the use of memory (recall, reasoning, learning) is routinely held in assemblies of simple(s) facts and percepts, and that there are some constraints on what kinds and numbers of assemblies occur in

4

human cognition.

The earliest instance of this last point in the literature is Miller’s “7 +/- 2” paper [13] on the observation that the number of such assemblies manageable in human memory varies between 5 and 9, but no more widely, over a quite diverse set of topic-areas. In many subsequent works in cognitive sciences, authors have referred to these assemblies – whatever their internal constitution may be - as chunks.

Where internal constitution is considered explicitly, there have been three approaches: suggestion of particular structures in connection with particular models of memory and reasoning, e.g. as in SOAR [14] and ACT-R [15]; identification of contents of chunks for particular topics, such as chess [9, 16, 17] and diagrams of electronic circuits [18]; appreciation of the fact that intelligent behavior, at a very basic level, includes the association of heterogeneous items because of their actual or potential (e.g. for forming hypotheses, or learning) usefulness in understanding the world [19]. The second of these three is not confined to cognitive science: there are many anecdotal but reliable first-hand accounts of the same phenomenon from specialist observers in their own subjects, e.g. Go [20] (pp. 9-15) and the operation of steam engines [21] (pp. 55-58 and 141-142). And the third approach underlies all the AI literature on frames. This literature treats frames as representations for knowledge that comes, so to speak, ready-made: if a frame with 13 slots of heterogeneous types is set up, it is assumed that the knowledge source already has these 13 items ready for storage. In that respect, it undervalues the importance of the frame concept as something elastic, capable of growth by accretion when a reasoner sees something new that seems to belong with the material already in some particular frame, and not even needing a name until and unless the reasoner reaches a stage of understanding that can be partly captured, and encoded, through the assignment of a name.

In the memory-based approaches, e.g. ACT-R, the emphasis is on “chunking”: replacing a set or sequence of actions, e.g. production rules, by a single action with the same effect or the part of their effect that is observed to be useful. In that case, the components of the chunk are actions rather than items of data, but the general principle is the same.

Our use of the idea of a visual chunk in organizing geological image data and knowledge connects somewhat with all of the lines of history above. It originated from careful consideration of what experts and semi-experts in geological image interpretation actually do. In some sense, the perception of what information they assemble, and at what level, came first, and the recognition of a match with the history was secondary. But it was also an inescapable match: hence the name of “visual chunk” for our concept.

5 Related work on ontologies and image annotation The most common definition of ontologies are

asserts that “an ontology is a formal, explicit specification of a shared conceptualization” [22]. A more precise description [23] is that an ontology includes a vocabulary of terms and how they are interrelated, which imposes a structure on the domain and constrains the possible interpretations of terms.

Ontology representation have been based on different paradigms and languages. Languages that implement Description Logics (DL) (e.g., LOOM [24]) basically allow the representation of ontologies with three kinds of constructs: concepts, roles (which describe binary relations between concepts and also concept properties) and individuals. A set of logical constructs (conjunction, disjunction, etc.) is also available for stating sentences about the domain.

RDF [25] presents a structure for representing ontologies that is equivalent to semantic networks: directed labeled graphs in which nodes represent concepts, instances and values. Edges represent properties of concepts or relationships between concepts. RDF Schema [26] was built as an extension of RDF so as to define the relationships between properties and resources with frame-based primitives. The combination of both RDF and RDF Schema proposals is known as RDF(S), which combines semantic nets and frame paradigms. RDF(S) provides the most basic primitives for ontology modeling, such as concepts, relations between concepts and instances. Functions and axioms are not constructs of the RDF(S) language. Concepts are described as classes, which can present several properties.

RDF(S) have established the foundations for other languages for building ontologies, such as OIL [27], whose formal semantic is based on description logics, and OWL [28]. OWL is built on an RDF(S) foundation and reuses some of its primitives representing concepts as classes. Concept attributes are also defined as class properties, but they differ from properties that represent binary relations. Instances are described as data types. Neither functions nor axioms can be represented in OWL – which has not been a limitation for us so far, though it may be an undesirable shortcoming for other users.

5.1 Indexing frameworks for image annotation

We need at least two levels in our ontology, for atomic and for abstracted or higher-level terms. It is therefore helpful to consider indexed schemes, especially where they have been influenced by possible application to the description of images.

5

Jörgensen et al. [29] and Jaimes et al. [30] present a study about the levels of an indexing framework for description of images. The framework is mainly organized as syntactic (e.g., colors in the image) and semantic (e.g., objects in the image) characteristics, but also handles the definition of percepts (e.g., patterns of light) and visual concepts (e.g., interpretation of objects).

Starting from perceptual characteristics of image descriptions, the following syntactic levels are presented: 1) type / technique: the most basic distinctions about the image, like if the image is color or grayscale; 2) global distribution: the image content is considered in terms of more organized spectral signals, like distributions of color and texture signals; 3) local structure: the image content is now considered as sets of local parts or components, like local colors and local geometries and 4) global composition: the specific arrangements of images are considered as a whole, like balance and center of interest.

The next levels of this indexing structure deal with knowledge and interpretation of visual objects, unlike the data-oriented lower perceptual levels. Distinctions of objects and scenes descriptions are considered in their generic, specific and abstract aspects [31], which are the parts of conceptual descriptions. Lower-level descriptions of generic objects and scenes are followed by specific-level descriptions of particular objects and scenes, being now more precisely identified and named. Abstract characteristic of objects and scenes are the highest conceptual levels in which more interpretative knowledge is required to formulate an image description.

The relationships among percept / syntactic levels one each other, as well as among visual concept / semantic are described according to spatial, temporal and visual relations. The relationship among levels are also considered in the generic and specific dimensions, and only semantic relations are considered in an abstract dimension. The indexing structure is related to more semantic levels where conceptual image descriptions follow the annotation questions of who, what action, what object, where and when [31].

Proposing an integration among many image description and search aspects, Hollink et al. [32] presents an indexing framework that is mainly divided in terms of non-visual, perceptual and conceptual characteristics, but also making basic assumptions about different image domains, type of search tasks that are used and levels of expertise of users.

Non-visual refers to contextual information of image descriptions, usually called metadata information. This type of information is usually not directly derived from the image content. It is objective information not affected by interpretation, e.g. descriptions of material and date aspects.

The perceptual description must be derived directly from images and their low-level attributes. However, as proposed by Jörgensen et al. [29] and Jaimes et al. [30], no knowledge of the world should be required in descriptions that use perceptual aspects. These are detailed as image elements and visual characteristics such as color, shape and texture, for example. In addition, visual characteristics can also be formed by the integration of image elements according to both position and relative spatial relationships.

The conceptual description is about general, specific and abstract subjects of image content. Conceptual objects and scenes are the main entities of conceptual descriptions, describing characteristics such as event (i.e. what question), place (i.e. where question), time (i.e. when question) and relations among conceptual objects. A conceptual object has a direct relationship with one perceptual image element, and this perceptual image element may be related to conceptual objects.

These schemes appear to be richer than ours, but our present approach has so far been fully adequate to represent the ontological concepts that occur in sedimentary petrology, even though the images themselves and their interpretation are quite complex. As Albert Einsten is reputed to have said: “make is as simple as possible, but not simpler”.

5.2 Applications of ontology-based annotation of images

Using the PetroGrapher system, users essentially need to retrieve geological images of kinds specified through ontological information (rather than simple keyword retrieval). The most relevant works in image annotation describe application in different domains and not conform so closely to the "Einstein" quotation above. Some of those domains are: photographs of apes [2], art images [3], pollen grains [4] and brain images [6].

The annotation ontology for the ape database is represented in RDF(S), and is applied via a template of annotation organized around the concepts of agent, action, object and setting [33]. Modifiers, such as individual labels, time stamps and relative-location coordinates, can be added to the basic concepts.

The art-image scheme applies a domain-specific thesaurus, besides the basic ontology, involving information about artists (and, by implication, art history, where terms and even their meanings can change over time) to retrieve information about the images. The components of the ontology can be interconnected by equivalence links, subclass links and domain-specific links. This allows users to retrieve images annotated using concepts from one part of the ontology while searching with terms taken from other parts. This complexity is a consequence of the subject

6

complexity and potential ambiguities in its terms - which is not the main problem for us because sedimentary geology is not art, but if in the future we wish to make connections between the ontology created by experts and the terms that students use while learning about the field, the experience of [3] will certainly be helpful.

Both the ape and art projects are template-oriented, while our geological subject is more open-ended. The same appears true of the work of Maillot et al. [4] on pollen grains. It is perhaps paradoxical that the topics whose images contain more (and more heterogeneous) information are the ones where templates are needed - but it may be that their complexity itself is the reason why it is desirable to give the user such prosthetic assistance.

In the study of pollen grains, the ontology of visual concepts is set up in such a way that there is a close connection between its terms and numerical information taken directly from image-processing input. During the classification of an object, numerical descriptors for texture and color are computed and then linked to symbolic attributes. This would be desirable in principle for our work, but the pollen images are processed in a strictly controlled environment, and routine geological issues such as the choice of sample orientation and illumination with varying polarizations of light (which is something of an art, rather than something done according to a set of rules or a manual), which complicate our situation, do not seem to arise in the pollen application.

The brain-image work [6] is closest to ours with respect to the structure of typical images and the annotations of features and regions that they require. In particular, interpretation of visual features during annotation demands of the users that they call on information/knowledge that is not explicit in the images. It differs from ours in the relative complexity of the material that is permitted in annotations: while doing much the same job as our own, it does not exhibit the same clear two-level (atomic and abstracted) structuring that we use. We probably owe the clarity to the notions of visual chunk and K-graph: in some sense, their existence has dictated the relatively simple overall conceptual structure of our scheme.

6 Conceptual levels for ontology-based annotation of images We propose the specification and refinement of a

set of knowledge models, such as knowledge engineering models, where visual knowledge acquisition and representation approaches are exploited to help users in the semantic description of more abstract visual information rather than only simplified visual objects. Instead of only having one annotation

level, users are faced with two conceptual annotation levels: an atomic level of annotation where users explore the annotations that are required to precisely find an image for a particular purpose, and an abstract level of annotation where they are also interested in the description of visual aggregations (e.g. visual chunks) that are the required knowledge to support the development of reasoning tasks. These levels involve different annotation approaches, which need to be connected appropriately to each other. This then represents part of a framework where different users can gradually specify and refine the annotations, allowing them to adapt the expert-level knowledge in such a way that it can support novice-level knowledge while respecting the ways in which novices perceive the images.

6.1 Description of petrographic images The main focus of this work is the description and

interpretation of microscopic images of rock samples, which are analyzed using crossed and/or parallel polarizer light sources, producing different digital images accordingly. The description of rock images really starts when users are analyzing the specimens on microscope. Using one sample only, a number of photos can be produced and afterwards described. What photos should be taken or not is a matter of user’s goals, and it is not clear in many cases.

Employing polarized approaches of microscopic image sampling, which are not defined as a standardize set of steps and sampling conditions, imposes important limitations for any automatic approach of image annotation. It is hard to say that having no control on the rotations of the polarizers there are discriminating low-level features to support more effective image processing methods, such as it is show by [34-36]. Moreover, experts are not aware of what a good training set of image samples is mainly due to the large visual variability in the domain, and even simplified atlases about visual features of rock [37] are formed by very different visual appearances.

In this perceptual context of annotation, we acknowledge the importance of perceptual ontologies (like the visual concept ontology [4] and the imaging ontology [5], for example), and how they may decrease the gap between images and symbols. However, our current approach remains on the advantages of annotating image samples using concepts that are manually selected from the domain ontology.

The main role of the oil-reservoir description ontology is to propose a standard vocabulary for guiding description and interpretation of images of oil-reservoir rocks. The description and interpretation of oil-reservoir rocks is mainly guided by different semantic pieces of information, following the research in which conceptual levels of image description can be identified [29, 32]. Most of these pieces of information

7

used to describe oil-reservoir rocks are detailed as domain-specific concepts representing visual features symbolically, but it is also about more semantic and interpreted contextual information.

The oil-reservoir description ontology is full of visual, spatial and temporal meanings (in terms of possible interpretations for visual appearances), which are frequently understood by geologists only. The concepts of the domain ontology are symbolic representations of these different levels of abstraction, which might frequently be understood on the light of geological theories such as [38-40]. Unfortunately, our work is still trying to make explicit great part of these domain-specific meanings using visual [41, 42], spatial [43, 44] and temporal [45-48] formalisms. However, even not being represented in terms of formal models, the rock images are indeed precisely described by users using concepts from the domain ontology, which is the formal model for them. Exploring concepts for ontologies is a first step of formalizing the large amount of knowledge that is presented on one single image indeed.

What the domain ontology allows to describe is an important matter for ontology-based image description approaches. In fact, description of images using a closed set of symbols is about to have a good set of symbols to express the image features. An analysis of the oil-reservoir description ontology allows to say that it is formed by a set of concepts that allows to describe:

• objects, which are the constituents of the rock, including mineral and pores (the spaces among mineral grains);

• how the visual appearances of these objects are presented and/or modified by space-time conditions. For example: X is a mineral with an appearance of lamellas, whose habit is called “booklets” into the ontology;

• relative spatial information among objects, which is characterized by the description of location aspects. Using a simple example: mineral X is on the border of the mineral Y;

• additional semantic domain characteristics, which might be interpreted as space-temporal meanings arising from geological theories. For example: mineral X is replacing a grain of mineral Y indicates that X comes first than Y in the process of rock formation;

• general visual aspects about rock features, like size, shape, orientation, sorting and structure of visual features;

• contextual information, which is used to describe the identification of specimen, the field, basin and sedimentary unit where the rock sample where collected the petrographer responsible by the description and others.

We believe the identification of these types of concepts in the domain ontology is the first step of developing different reasoning strategies using the information that is described from images. For example, visual reasoning strategies can be formalized by using concepts related to visual appearances, spatial reasoning inferences by using spatial concepts and temporal reasoning methods by using domain concepts with temporal representations. In addition, more abstract inferences can be based on patterns, which are combinations of visual, spatial and temporal semantics. Either to make these semantics explicit by using different ontology models, or to infer them using the domain concepts that are used to describe images, seems to be a good hypothesis of exploring knowledge from images.

Fig. 1 presents part of the oil-reservoir description ontology, which was originally represented using CommonKADS [12] and is exhibited here in PROTÉGÉ [49, 50]. The main hierarchy of the oil-reservoir description ontology is defined in terms of sedimentary rock features (the most common features of rock in oil reservoirs). The other types of rock can be detailed in the same way using the oil-reservoir description structure, and further users can add other basic rock features as they wish.

The oil-reservoir description ontology was specially prepared to accept concepts at different granularity levels, ranging from concepts for detailed visual information, such as exhaustive technical terms for mineral names (“volcanic_rock_fragment_with_ felsitic_texture”, for example), to those covering coarse visual aspects, and describing only the class of some mineral (e.g. “volcanic_rock_fragment”).

Domain concepts represented in hierarchies have an important role for this kind of conceptual description, where different levels of the hierarchy usually hold concepts at different abstraction levels. However, abstraction hierarchies are only one part of the whole scheme of conceptualization of visual features. The description of visual patterns as aggregations, which is being explored in this work, also has a role in description and interpretation of rock images.

Using concepts from the oil-reservoir ontology, it is possible to describe more conceptual information about visual features, e.g. mineral X is replacing another mineral Y, even though the geologist has not seen Y in the specific instance of the image being analyzed; simpler visual aspects, such as so that the orientation of grains is “chaotic”, are, of course, also covered. It is because this domain is strongly image-oriented that geologists have to use domain-specific terms for description of visual features at different granularities. In addition, concepts from the ontology allow them to describe both details of rock features and global characteristics of images, something similar to what scene and object descriptions are characterized by [29,

8

30].

Figure 1: Part of the oil-reservoir description ontology, embedded in PROTÉGÉ

In summary, it is possible to make two statements about the oil-reservoir description ontology in relation to the visual aspects that this ontology is intended to describe. First, the domain ontology allows the use of formal concepts to describe a very large set of visual features, when all of these features may in principle figure in reasoning about the image(s) under examination. Second, new visual appearances will always be found in rock images, and it should be easy for new concepts for describing them to be formalized and the ontology expanded accordingly.

6.2 Description using concepts

Having a picture of a rock sample, users browse the concepts from the ontology and choose the most appropriate ones to describe what is being seen. As a result, multiple instances of concept-attribute-value triples are produced, which are typical of image descriptions over a large variety of task requirements.

For illustration, we can identify two tasks that focus on these image descriptions:

• the systematic description of the main visible visual features in the image using concepts from the domain ontology. It is offer to the user 132 attributes of

a rock sample to be described, according to different levels of detail in petrographic analysis;

• the description of particular visual features, which can be broad visual patterns in the image as a whole as well as visual features in special regions of interest. This kind of description concerns features that are clearly visible for geologists, like a mineral that is rare and/or not commonly found along with those that are present. Additionally, we need the ability in geology to describe “space” between explicitly visualized objects. Recognizing these visual aspects in images and name them properly using concepts from the domain ontology is part of the task of image annotation (it can be described for geologists as the analysis of rock porosity and the factors that have some impact on the porosity). The image in Fig 2 is an example of this last type of description.

Fig 2 presents some of the conceptual objects that can be described using a rock image. Among other visual features that users could annotate using this image, only two sets of concept-attribute-values are involved here, partly because those are most relevant to the analysis of porosity that is central to what a user here typically needs.

The concepts, attributes and values, which are

9

annotated on the image of Fig. 2, are atomically selected from the oil-reservoir description ontology. They are part of a diagenetic description, which is the analysis of the origins of the rock in terms of the conceptual features that are visible [51]. Helped by the ontology, users are able to see what terms are available and choose the best oil reservoir concept to describe visual features of interest semantically.

Figure 2: This picture presents only the rock features in an atomic annotation approach that are extract directly from the domain ontology. Here, we are

describing the rock features that are related to Illite and Iron Oxides/Hydroxides visual chunks

6.3 Description of petrographic patterns The higher levels of image description are needed

because ontology-based annotations of perceptual and low-level conceptual visual features, considering individually, are only peripherally useful in leading to inferences. The point of the higher levels is to express the connections between atomic level features that allow an expert to make such inferences. For example, high-level classification inferences [52] can be based on simple lists of object attributes, which determine an object’s class when their members are taken together. Assessment inferences [12] are based on high-level case descriptions, which can also be expressed as sets of integrated parts. Diagnosis inferences [53] are based on complaints and finding, which are usually not

merely individual pieces of information. When describing an image, to link concepts with visual features is an important task of any process of reasoning based on images, but for most of the time this is secondary for high-level reasoning tasks where the relevant items of evidence are combinations of annotated visual features.

In our studies in the domain of petrography [8], the interpretation of rock images is supported by visual knowledge, which can be acquired and stated neatly in terms of visual chunks. Reasoning procedures that have only one step, connecting evidence immediately to interpretations, are particularly transparent and concise when expressed in this way.

We represent visual chunks as simple logical aggregations of domain concepts. It is also important to say that domain concepts for describing these aggregations of atomic information may not even be in the domain ontology initially. Often we see that they don’t have a formal name on the domain ontology at that stage, being understood as pieces of tacit knowledge, a fact that has been identified and studied by several authors [54] [55].

In some cases these aggregations are described as unique concepts in the domain ontology, and/or described through higher-level expert terminology. We have observed, however, that can seldom be identified by novices during image recognition and annotation. That is, novices may not be able to recognize these logical combinations in the image sample, but more trained people are. Nevertheless, non-experts are able to inspect these simple annotations, and even to start making descriptions of their own abstract aggregations. What kinds of training environments for image description and interpretation [56, 57] that can be supported by these ontology-based annotation structures and give effective service to novices deserve future investigation. This is an issue in applied psychology and cognitive science.

The AND/OR tree structure of a visual chunk is built from concepts from the ontology. Concepts grouped with an AND operation must all be present in an image for the corresponding high-level interpretation to apply. OR means that an interpretation applies when at least one of the atomic terms that it associates is present.

Fig. 3 and 4 show annotated visual chunks (the visual chunk of Fig. 3 being based on the image presented in Fig. 2), where only the given combination of rock features, when seen by a geologist, can lead to the indicated rock interpretation. The visual features that are actually seen on the images of Fig. 2 and Fig. 4 are described as hatched ellipses on the trees that are drawn in Fig. 3 and Fig. 4. However, experts are also able to describe additional concepts as parts of these trees.

Oil-Reservoir Description Ontology

Diagenetic Composition • Constituent Name: Iron Oxide /

Hydroxide and Hematite • Habit: Coating • Location = Intergranular

Continuous Pore-Lining

Diagenetic Composition • Constituent Name: Illite • Habit: Fibrous and Rim • Location = Intergranular

Discontinuous Pore-Lining

10

Figure 3: The Illite visual chunk and its set of logical relationships with geometric rock features. Ontology-based rock concepts label the intermediate nodes above the leaf nodes.

It has been shown many times that experts can see more than novices when describing images. They can also describe abstract and interpreted features using a more precise vocabulary than novices know. Our experience with users has been that the medium of visual chunks is one effective way of making more explicit these kinds of expert-novice differences when annotating information related to images: not only for answering queries over image repositories, but also in support of reasoning tasks, either for practical geological work or for teaching.

6.4 Reasoning for rock interpretation As modeled by Abel [8] and Silva [58], the process

of rock interpretation is expressed as a symbolic pattern-matching process on visual chunks.

A K-graph can be described as a tree where a) the root node states the interpretation hypothesis, which in our study is a broad natural environmental characterization that is inferred from the analysis of the visual features of the image, and b) the leaf nodes contain visual chunks, identified by the experts, which are pieces of evidence necessary to support the interpretation. In terms of reasoning, each interpretation is associated with a threshold value, which represents the minimum amount of evidence needed to indicate that the interpretation holds. For reaching this threshold, the visual chunks in the K-graph nodes can be combined to increase the influence and the certainty of the interpretation stated.

Up to the current time, 6 K-graphs have been constructed, expressing 6 possible interpretations of the “diagenetic environment” in oil-reservoir rocks. For example, the leaf nodes in the “Continental Meteoric Eodiagenesis Under Dry Climate Conditions” K-graph have visual chunks/weights such as: “Silcrete” (weight 6), “Sulphate” (weight 6), “Calcrete” (weight 5), etc.

These weights represent significance indexes for the interpretation, which can only be valid when the combination of significance indexes of observed visual chunks has reached the threshold of weight associated with it.

The reasoning can be driven either by hypothesis (starting with a selected K-graph) or by data (starting from some observed evidence). In a data-driven inference, the main inference steps are the following:

• a set of visual chunks (those with the higher weights) is logically matched to the concept-attribute-values mentioned in the atomic level image description. They are only matched when a minimal logical set of concept-attribute-values is found in the annotated image;

• the matched visual chunks are used to select other K-graphs that include the same visual chunk. In turn, these K-graphs are used to select new visual chunks with the aim of confirming a hypothesis of interpretation.

This process is repeated for as long as K-graphs and visual chunks remain to be analyzed over the entire knowledge base. A solution is indicated if any K-graph has sufficient confirmation indicated by a set of activated chunks, considering the weights and threshold values. The inferred solutions comprise the K-graphs and the whole set of visual chunks matched.

Using a “Continental Meteoric Eodiagenesis Under Wet Climate Conditions” K-graph as examples, the visual chunks “Kaolinization Displacive” and “Kaolinization Replacive” (Fig. 4) can be selected initially (the visual chunks with the largest weights). The set of logical aggregations of concept-attribute-values of these visual chunks is then matched to the image description. If one of these visual chunks is matched, the other chunks “Siderite”, “Kaolinite Cement”, “Matrix Kaolinization”, etc, are also selected.

Oil-

Res

ervo

ir D

escr

iptio

n O

ntol

ogy

Diagenetic Composition • Constituent Name: Illite • Habit: Fibrous and Rim • Location = Intergranular Discontinuous Pore-Lining

Visual Chunk

or or

Constituent Name Habit Location

I llite Intergranular discontinuous pore-lining

Intergranular pore-filling

R adiated Fibrous R im

and

Diagenetic Composition

Illite

B ridge Intragranular Pore-filling

11

This process can increase the certainty about the K-graph currently taken as hypothesis. The new set of selected visual chunks is analyzed, and the certainty about this K-graph is thereby strengthened. Finally, the “Continental Meteoric Eodiagenesis Under Wet Climate Conditions” solution is indicated by the confirmed K-graph and by the matched visual chunks. In another example, we can show that the image in Fig. 2 could be inferred through two different

interpretations: “Deep Burial Diagenesis (Deep Mesodiagenesis) Conditions” and “Continental Meteoric Eodiagenesis Under Dry Climate Conditions”, which can be indicated respectively by “Illite” (Fig. 3) and by “Iron Oxides/Hydroxides” visual chunks. Along with other geological criteria, this kind of interpretation is an important part of a qualitative analysis of oil reservoirs.

Figure 4: The Kaolinization Replacive visual chunk and its set of logical relationships with geometric rock features described as concepts in the intermediate nodes. The white ellipses are Diagenetic Compositions having Constituent

Name = Kaolin AND Location = Intragranular Replacive. The black arrows indicate Habit = Vermicule AND the white and back ellipses indicate Paragenetic Relation = Replacing Grains of Detrital Feldspar.

7 A Tool for defining using K-graphs and visual chunks We present an ontology-based knowledge

acquisition and annotation tool, which was built to

describe the more abstract and interpreted visual information in the context of PetroGrapher project. The tool was conceived to give support for expert-level users of the system when studying and documenting rock samples coming from new areas of study.

The tool (Fig. 5) is a graphical interface to acquire, inspect and annotate the visual knowledge, recognized

Location

Replacing Grain of Metamorphic Rock Fragment

Replacing Grain of Detrital Feldspar

Booklet Replacing Grain of Mud Intraclast

Replacing Grain of Mica/ Chlorite

Replacing Grain of Volcanic Rock Fragment

Replacing Grain of Sedimentary Rock Fragment

Vermicule Lamella Kaolin

and

or or

Constituent Name

Habit

Diagenetic Composition

Kaolinization Replacive

Intragranular Replacive

Paragenetic Relation

Replacing Grain of Plutonic Rock Fragment

12

during the description and interpretation of rock sample images. Geologists, for example, can be guided to recognize both simplified and complex visual characteristics, where they can use the K-graph and visual chunk structures to imprint more semantic

annotation to the image. This tool allows identifying interpretations, and then connecting the visual chunks that support an interpretation, decreasing the granularity down to the simplified concepts of the domain ontology.

Figure 5: K-graphs and visual chunks filling the gap between the image-annotation levels

Using this tool, users can describe the visual chunks and connecting them to basic features described in the novice level. However, most of the time, they do not have a name in the domain ontology. This tool allows treating the semantic of the cognitive objects (e.g. the visual chunks), even thought they do not have a propositional identification. As the first information that users see in the image when trying to interpret it is the aggregates of visual features, visual chunks is the first thing that is described using this evidence-hypothesis model. Users name and describe the visual chunks that support some interpretation, assigning them weights, which means how significant each chunk is to confirm some interpretation. The tool automatically generates the causal relation between the nodes that represent the visual chunks and the interpretation they support.

The tool makes available, for each one of the visual chunks defined by the user, the domain ontology model. The domain concepts, attributes and values can

be selected from lists on the system interface, and forward added as the image evidences which are formed by aggregations of more simplified visual concepts. When users select one concept and one attribute from this concept, the system retrieves the domain terminology from the oil-reservoir description ontology, displaying the list of possible values that can be selected to fill the attribute, allowing the inspection of the ontology in order to select the term that better represents the visual characteristic they want to describe. Finally, users can save all the information in a relational database system.

8 Analysis and recommendations The images that we are dealing contain a large

number of visual features, and there is no control of the methods that are used to produce these digital images. The visual features in images, simplified and complex

d) List of possible attributes from the ontology, which are related to the selected concept

a) Nodes representing visual objects that are described during low-level annotation steps

b) Nodes representing abstract features that are described during high-level annotation

c) List of concepts from the ontology, displayed to the user during the graph configuration

13

ones, can contain multiple colors, textures and shapes, depending on the way they were digitalized. The features can show multiple relationships between them, and many of these visual relationships can be only characterized using special concepts of the domain that describe geological associations among features. The bitmap file of the image (e.g. set of pixels) is not the input to the inference; differently, our modeling view makes use of a symbolic approach to image analysis opposed to numeric approaches oriented towards low-level image attributes.

We are working on high-level annotations due to the fact that even simplified rock features are harder to identify automatically. Moreover, visual chunks are not recognized by geometric or physical processing, such as the image processing algorithms, then many of them can only be identified individually by experts who investigate and interpret an image in a specific context. Therefore, our knowledge-based model can be understood as a high-level approach to complement automatic and semi-automatic pattern recognition and feature extraction methods.

According to [2, 3], an ontology captures the information that is necessary to index collections of images. The indexes are built as a template model, whose primitives are agents (passive and active agents), actions, objects and settings. This work applies a similar notion using concepts, attributes and values as the basic input for the content annotation. Actions are not well appropriated when analyzing rock images, but we believe that some kind of event annotation according to temporal aspects should be described.

The annotation structure of visual chunk is being introduced as a means of organizing and helping to develop higher ontology-based levels of semantic annotation. Our visual chunk annotation model can be used to explicitly describe the information required to develop distinct knowledge-based reasoning tasks of images, such as rock images and medical images. In addition, the annotation structure of visual chunks can be instantiated to logically connect the visual information that is described through different levels of granularity, which can still be supported by the notion of agents-action-object-setting.

Gertz and colleagues, in [6], asserts that an image annotation framework should define semantic ontology-based schemes where scientists would use to describe instances of visual features at different levels of detail, using these metadata to query image repositories. The image annotation is also based on the annotation of regions of interest, which allows fine grained data annotations instead of just whole image annotations. We believe that both ontology-based annotation and regions of interest can apply our visual chunk modeling strategy to inspect more abstract structures of visual knowledge. Specifically related to these approaches, we believe that semantic visual

features can be only characterized using high-level concepts from the ontology, and these concepts can be correctly identified and pointed by trained people. For these reason, we propose visual chunks as a structured approach to represent symbolic patterns, allowing the description of findings and diagnostic features as a combination of simplified image features that are annotated using simplified concepts from the ontology. Along with concepts extracted from an ontology and associated to regions of interest in an image, the visual chunks and K-graphs are true indexing structures to investigate the description of more semantic visual features.

9 Concluding remarks We justify the assumption that abstract types of

visual knowledge are crucial to produce solutions to problems of image interpretation as well as to develop more effective ontology-based annotation tools, which allows making explicit the knowledge that an image can retain. Two conceptual annotation levels are proposed; one exploring the ontology model to produce atomic level semantic image annotations, and the other using the visual chunk model to capture more abstract patterns of visual features. Finally, the K-graph model allows connecting these visual patterns to interpretations, showing how to use effectively this information to solve tasks of knowledge-based interpretation of images.

References 1. Kolodner, J.L., Case-based reasoning. 1993, San

Mateo, CA: Morgan Kaufmann Publishers, Inc. 2. Schreiber, A.T.G., et al., Ontology-based photo

annotation. IEEE Intelligent Systems and Their Applications, 2001. 16: p. 66-74.

3. Hollink, L., et al. Semantic annotation of image collections. in In Proceedings of the KCAP'03 Workshop on Knowledge Markup and Semantic Annotation. 2003. Florida, USA.

4. Maillot, N., M. Thonnat, and A. Boucher, Towards ontology-based cognitive vision. Machine Vision and Applications, 2004. 16(1): p. 33-40.

5. Hu, B., et al. Ontology-based medical image annotation with description logics. in 15th IEEE International Conference on Tools with artificial Intelligence, Nov 3-5 2003. 2003. Sacramento, CA, United States: Institute of Electrical and Electronics Engineers Inc.

6. Gertz, M., et al. Annotating scientific images: a concept-based approach. in Proc. 14th International Conference on Scientific and

14

Statistical Database Management. 2002. Edinburgh, Scotland: IEEE Computer Society.

7. Hyvönen, E., A. A. Styrman, and S. Saarela. Ontology-based image retrieval. in Towards the semantic web and web services, Proceedings of XML Finland 2002 Conference. 2002. Helsinki, Finland.

8. Abel, M., The study of expertise in Sedimentary Petrography and its significance for knowledge engineering (in Portuguese: "Estudo da perícia em petrografia sedimentar e sua importância para a engenharia de conhecimento"), in Programa de PG em Ciência da Computação. 2001, UFRGS: Porto Alegre. p. 239.

9. de Groot, A.D., Thought and choice in chess. 1965, Mouton: The Hague.

10. VanLehn, K., Problem-solving and cognitive skill acquisition, in Foundations of Cognitive Science, M.I. Posner, Editor. 1989, The MIT Press: Cambridge. p. 526-579.

11. Leão, B.F. and A.F. Rocha, Proposed methodology for knowledge acquisition: a study on congenital heart disease diagnosis. Methods of Information in Medicine, 1990(29): p. 30-40.

12. Schreiber, A.T.G., et al., Knowledge engineering and management - The CommonKADS Methodology. 2000, Cambridge: The MIT Press. 465.

13. Miller, G.A., The magical number seven, plus or minus two: some limits on our capacity for processing information. The Psychological Review, 1956. 63: p. 81-97.

14. Laird, J.E., A. Newell, and P.S. Rosenbloom, SOAR: An architecture for general intelligence. Artificial Intelligence, 1987. 33(1): p. 1-64.

15. Anderson, J.R. and C. Lebiere, The atomic components of thought. 1998, Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc. 536.

16. de Groot, A.D., Perception and memory versus thought: Some old ideas and new findings, in Problem solving, B. Kleinmuntz, Editor. 1966, Wiley: New York. p. 19-50.

17. Chase, W.G. and H.A. Simon, Perception in chess. Cognitive Psychology, 1973. 4: p. 55-81.

18. Egan, D.E. and B.J. Schwartz, Chunking in recall of symbolic drawings. Memory and Cognition, 1979. 7(2): p. 149-158.

19. Atran, S., Cognitive foundations of natural history: towards an anthropology of science. 1990, Cambridge, UK: Cambridge University Press.

20. Nakayama, N., The treasure chest enigma. 1988, Tokyo: Ishi Press.

21. McKillop, N., Enginemen elite. 1958, London: Ian Allan. 154.

22. Studer, R., V.R. Benjamins, and D. Fensel, Knowledge engineering: principles and methods. Data & Knowledge Engineering, 1998. 25: p. 161-197.

23. Uschold, M. and R. Jasper. A Framework for understanding and classifying ontology applications. in IJCAI99 - Workshop on Ontology and Problem-Solving Methods: Lessons Learned and Future Trends. 1999. Stockholm, Sweden: CEUR Workshop Proceedings.

24. MacGregor, R., Inside the LOOM classifier. SIGART Bulletin, 1991. 2(3): p. 70-76.

25. Lassila, O. and R.R. Swick, Resource Description Framework (RDF) Model and Syntax Specification. 1999.

26. Brickley, D. and R.V. Guha, Resource description framework (rdf) schema specification 1.0. 2000, W3C Proposed Recomendation.

27. Horrocks, I., F.v. Harmelen, and P.F. Patel-Schneider, Reference description of the DAML+OIL (March 2001) ontology markup language. 2001.

28. Dean, M. and G. Schreiber, OWL Web Ontology Language Reference: W3C working draft. 2003.

29. Jörgensen, C., et al., A conceptual framework and empirical research for classifying visual descriptors. Journal of the American Society for Information Science and Technology (JASIST), 2001. 52(11): p. 938-947.

30. Jaimes, A. and S.-F. Chang. A conceptual framework for indexing visual information at multiple levels. in IS&T/SPIE Internet Imaging. 2000. San Jose, CA.

31. Shatford, S., Analyzing the subject of a picture: A theoretical approach. Cataloging & Classification Quarterly, 1986. 6(3).

32. Hollink, L., et al., Classification of user image descriptions. International Journal of Human Computer Studies, 2004. 61(5): p. 601-626.

33. Tam, A.M. and C.H.C. Leung, Structured natural-language descriptions for semantic content retrieval of visual materials. Journal of the American Society for Information Science and Technology (JASIST), 2001. 52(11): p. 930 - 937.

34. Ross, B.J., F. Fueten, and D. Yashkir, Automatic mineral identification using genetic programming. Journal of Machine Vision and Applications, 2001. 13(2): p. 61-69.

35. Thompson, S., F. Fueten, and D. Bockus, Mineral identification using artifical neural networks and the rotating polarizer stage. Computers & Geosciences, 2001. 27(9): p. 1081-1089.

36. Goodchild, J.S. and F. Fueten, Edge detection in petrographic images using the rotating polarizer stage. Computers and Geosciences, 1998. 24(8): p. 745-751.

37. Choh, S.-J., K.L. Milliken, and E.F. McBride, A tutorial for sandstone petrology: architecture and development of an interactive program for teaching highly visual material. Computers & Geosciences, 2003. 29(9): p. 1127-1135.

15

38. Folk, R.L., Petrology of sedimentary rocks. 1974, Austin: Hemphill Pub. Co. 170.

39. Zuffa, G.G., Provenance of Arenites. NATO-ASI Series C: Mathematical and Physical Sciences. Vol. 148. 1985, Dordrecht: Reidel Publishing Company. 408.

40. Dickinson, W.R., Interpreting detrital modes of graywacke and arkose. Journal of Sedimentry Petrology, 1970. 40: p. 695-707.

41. Bober, M., MPEG-7 visual shape descriptors. IEEE Transactions on Circuits and Systems for Video Technology, 2001. 11(6): p. 716-719.

42. Manjunath, B.S., et al., Color and texture descriptors. IEEE Transactions on Circuits and Systems for Video Technology, 2001. 11(6): p. 703-715.

43. Bailey-Kellogg, C. and F. Zhao, Qualitative spatial reasoning: extracting and reasoning with spatial aggregates. AI Magazine, 2004. 24(4): p. 47-60.

44. Haarslev, V., A logic-based formalism for reasoning about visual representations. Journal of Visual Languages and Computing, 1999. 10(4).

45. Allen, J.F., Time and time again: the many ways to represent time. International Journal of Intelligent Systems, 1991. 6: p. 341-355.

46. Jensen, C.S., et al., The consensus glossary of temporal database concepts. 1998, Christian S. Jensen.

47. Montanari, A. and B. Pernici, Temporal reasoning, in Temporal databases: theory, design, and implementation, A.U. Tansel, et al., Editors. 1993, Benjamin/Cummings Pub.: Redwood City.

48. Mastella, L.S., Um modelo de conhecimento baseado em eventos para aquisição e representação de seqüências temporais, in Programa de PG em Ciência da Computação. 2005, UFRGS: Porto Alegre. p. 163.

49. Noy, N.F., R.W. Fergerson, and M.A. Musen. The knowledge model of Protégé-2000: combining interoperability and flexibility. in 12th International Conference in Knowledge Engineering and Knowledge Management (EKAW'00). 2000. Juan-Les-Pins, France: Springer-Verlag, Berlin, Germany.

50. Grosso, W.E., et al., Knowledge modeling at the millennium (The design and evolution of Protege-2000). 1999.

51. Morad, S., J.M. Ketzer, and L.F. De Ros, Spatial and temporal distribution of diagenetic alterations in siliciclastic rocks: implications for mass transfer in sedimentary basins. Sedimentology, 2000. 47: p. 95-120.

52. Clancey, W.J., Heuristic classification, in Artificial Intelligence. 1985: Amsterdam. p. 289-350.

53. Benjamins, R. and W. Jansweijer, Toward a competence theory of diagnosis. IEEE Expert, 1994. 9(5 Oct): p. 43-52.

54. Nonaka, I., H. Takeuchi, and H. Takeuchi, The knowledge-creating company: how Japanese companies create the dynamics of innovation. 1995, New York: Oxford University Press. xi, 284.

55. Polanyi, M., The tacit dimension, in Knowledge in Organizations, L. Prusak, Editor. 1966, Butterworth-Heinemann: Newton, MA. p. 135-146.

56. Garlatti, S. and M. Sharples, The use of a computerized brain atlas to support knowledge-based training in radiology. Artificial Intelligence in Medicine, 1998. 13(3): p. 181-205.

57. Sharples, M., et al., Structured computer-based training in the interpretation of neuroradiological images. International Journal of Medical Informatics, 2000. 60(3): p. 263-280.

58. Silva, L.A.L., The use of problem-solving methods in rock interpretation tasks (In Portuguese: "Aplicando métodos de solução de problemas em tarefas de interpretação de rochas"), in Programa de PG em Ciência da Computação. 2001, UFRGS: Porto Alegre. p. 160.