designing intelligent systems for teaching visual …iaied.org/pub/1052/file/1052_paper.pdfdesigning...

International Journal of Artificial Intelligence in Education (1997), 8,44-70

44

Designing Intelligent Systems forTeaching Visual Concepts

ALEXANDRE I. DIRENE School of Cognitive and Computing Sciences

University of Sussex, Brighton, BN1 9QH, United Kingdom______________________________

Currently at:Departamento de Informática, Centro Politécnico-UFPR

Curitiba - PR, 81.531-990, Brazil

The paper describes how high-level knowledge about visual images couldbe represented and further interpreted through system-active and system-passive tutorial interactions. The ideas lend themselves to the design andimplementation of a category of Intelligent Tutoring Systems aimed at theteaching of abnormalities in highly visual domains, like medical diagnosticimaging. The involved problems are treated through (1) a domain-independent, object-oriented method for managing the complexity ofdesign representations and (2) a model of dialogue interpretation forimplementing tutorial interactions. The method and the model are bothsupported here by implemented, computer-based tools that integrate themulti-layer environment RUI. To evaluate the power of RUI, empiricalobservations have been carried out, focussing on the generality of thedesign methodology as well as on the usability of the interface. We drawconclusions about the suitability of object-oriented principles in thecontext of ITSs evolution.

INTRODUCTION

This paper describes how high-level knowledge about visual imagescan be represented and further interpreted through system-active andsystem-passive tutorial interactions. The ideas lend themselves to thedesign and implementation of a category of Intelligent Tutoring Systems(ITSs) aimed at the teaching of abnormalities in highly visual domains, likemedical diagnostic imaging for chest X-rays (Figure 1) and for MR-scansof the head (Figure 2). The perspective adopted for knowledgerepresentation is object-oriented in that pedagogic behaviour is“encapsulated” with domain expertise “inside” anatomical components toprovide for the consistency of tutorial dialogues.

Designing Intelligent Systems for Teaching Visual Concepts

45

Figure 1. Dialogue about cardiac X-rays

We shall argue that, in highly visual domains, a learner's knowledgeand expertise can be best developed through a computer tutor if he or shehas access to the same surface knowledge representations created byexperts, also using computer-based tools. However, new computationalconcepts are needed to cope with the complex phenomenon of knowledgecommunication (from expert to machine and from machine to student).This paper is primarily an exercise in knowledge engineering that carries astrong cognitive flavour. Related work in the field of ITS gives acomprehensive view of domain-independent design methods and tools forbuilding computer tutors (Major, 1995; Murray, & Woolf, 1992; Woolf,1991; Spensley, & Elsom-Cook, 1988; Scott, 1987; Sleeman, 1987;Bonar, Cunningham, & Schultz, 1986; Nicolson, & Scott, 1986; O'Shea,Bornat, du Boulay, Eisenstadt, & Page, 1984; Bunderson, 1974), but noaccount is taken for domains of visual concepts.

Direne

46

Figure 2. Dialogue about scans of the head

First, I shall discuss a domain-specific approach developed for theRadiology Tutor (Sharples, & du Boulay, 1988; Sharples, 1989) because itaddresses, in computational and pre-computational forms, some aspects ofthe approach presented here.

Sharples and du Boulay view the learning of medical visual concepts(such as pathologies), through a computer-based tutor, as the acquisitionof competence in a space of feature dimensions where boundaries ofpathologies are defined, and that such competence can be developed byinteracting with ordered sets of images via the ITS. They point out that, innormal practice, much of a student's knowledge about pathologies tends tobe acquired inductively, giving rise to over-generalisation. The immediateconsequence is that these over-general beliefs cover not only exampleimages but also non-examples (see Figure 3), requiring the ITS to takefurther action in order to bring the student's beliefs to a consistent state.Explicit “emendation” and “discrimination” teaching actions, they argue,


47

are needed to diminish the fragility of knowledge which is acquiredinductively.

Figure 3. 2-D view of a feature space

VISUAL CONCEPT TUTORING AND ITS DESIGN

Visual concept tutoring is not a new idea but past work has tended toconcentrate on the theoretical principles of how humans acquire expertisein visual recognition (Lesgold et al, 1989; Myles-Worsley, & Johnston,1988; Howard, 1987; Lesgold, 1984; Mervis, & Rosch, 1981; Rosch,1978). The few implementations there have been are domain-specific(Sharples et al., 1995; Jeffery et al., 1993; Parkes, 1989; Sharples, & duBoulay, 1988; Rivers, 1988; Swett, & Miller, 1987). However, theproblem of providing a domain-independent framework for describingknowledge of ITSs that teach visual concepts has been neglected. Alsoneglected is the question of how meta-level knowledge can be encoded inorder to regulate tutorial dialogues about abnormal image features whileenforcing consistency of such dialogues.

Direne

48

From the architectural viewpoint, the design of ITSs has been greatlyinfluenced by the four-box approach which suggests dividing therepresentations of such systems into domain, student, pedagogic andinterface “modules.” Orthogonal to variations of architectural divisions,design methods must also take into consideration that ITSs tend to becostly and complex machines. The wide variety of subject domains has ledto a considerable effort in exploring the psychological, educational andcomputational questions underlying learning and teaching viewpoints,making it difficult for inventors to map conceptual aspects intoimplemented structures. One such method, Bites (Bonar, Cunningham, &Schultz, 1986), explicitly tries to deal with complexity by means of anobject-oriented architecture to provide the curriculum-independent part ofan ITS, organised around “abstraction hierarchies.”

We describe RUI (Representations for Understanding Images), (1)an object-oriented method and software tools for managing the complexityof ITS design, and (2) a domain-independent model of dialogueinterpretation, integrated with the method, for implementing tutorialinteractions. Besides the fact that visual interpretation tasks in differentdomains have much in common (Rosch, 1978), computer tutors for visualconcepts differ from more traditional tutoring systems in that the skills tobe communicated to students are closely linked to the interpretation ofimage patterns as a primary task. Therefore, these systems must includefacilities for students to manipulate and display large stocks of visualimages. Likewise, design methods and tools for producing these tutorsmust provide experts with mechanisms for creating and assigning high-level, symbolic descriptions to such images.

RUI (Direne, 1993; Direne, 1994) meets the above requirements. Itis fully implemented as three domain-independent tools, each for adifferent level of abstraction (see Figure 4). The design steps involve twolevels: conceptual and production. The third level, instructional, aims atthe communication of expertise. The next sections will describe theselevels.


49

Figure 4. Communication layers in RUI

THE INSTRUCTIONAL LEVEL

In trying to understand what constitutes radiological expertise, Lesgoldet al. (1989) suggest that it involves substantial amounts of bothprincipled knowledge and experiential knowledge. Principled knowledgerefers to the distinct bodies of medical knowledge which are alreadyformalised (e.g. anatomy, theories of medical disease and the projectivegeometry of radiography) whereas experiential knowledge involves theintegration of these bodies of knowledge in clinical practice to produceaccurate diagnoses. Compared to the basic learning mechanism ofAnderson’s ACT* theory (Anderson, 1984), the knowledge compilationprocess, and more recently to the ACT-R theory of visual attention(Anderson, 1993), it is reasonable to assume that principled knowledgecorresponds to the idea that knowledge is first acquired declarativelythrough instruction. Later, it is transformed into procedures throughpractice (experiential knowledge). This section describes how these two

Direne

50

types of knowledge can be communicated to students by interacting withRUI's Learning Tool.

At the instructional level, the Learning Tool allows a student toacquire expertise by means of two distinct interfaces: the Image DatabaseBrowser (IDB) and the Guided Tutorial Mode (GTM). The IDB aims atthe communication of principled knowledge while the GTM aims at thecommunication of experiential knowledge. In using the IDB, the system-passive interface, students, possibly assisted by human experts, search andselect example images based on anatomical information, observeindividual feature values as well as feature locations, make interactivemeasurements of ratios and compare complete diagnoses to acquire theunderlying principled knowledge necessary for classifying abnormalities.In using the GTM, the system-active interface, students stretch theirexperiential knowledge by engaging in Socratic-like dialogues (see Figures1 and 2) where, constantly challenged by the system, they make use oftheir conceptual principles to form more accurate and complete diagnoses.

It is important to remark that, up to the present moment, RUI'sLearning Tool should be understood as an experimental ITS forsupplementary teaching, for example, to help expert radiologists in theirteaching tasks. Thus, experts can choose an image themselves from theIDB and advise the trainee to explore it according to all the facilitiesavailable (see next subsection) or even switch to the GTM and interactwith it under the chosen image. This means that the main long-termpedagogic decisions are left to human experts, and that there is no strictorder of which interface should be used first by the trainee, the IDB or theGTM.

The IDB and Free Exploration

The IDB interface is conceived as a learning-by-discoveryenvironment. Free exploration encourages students to become autonomouslearners by allowing them to compose their own questions, hypothesiseabout the concepts of a domain and draw conclusions from hypotheses. Invisual domains, this includes understanding the nature of visual conceptstranslated into features such as shape, size and location of anatomicalcomponents. Principled knowledge expresses the ability to recogniseabnormalities based on deformations and variations of such anatomicalstructures, projected onto 2-D image regions (Lesgold, Rubinson, Glasser,Klopfer, & Wang, 1989; Lesgold, 1984).

In order to display an overview of what a subgroup of images look likein a given class of abnormality, the IDB allows the student to search forimages, according to set criteria. This interactive facility is mainly guided


51

by pointing an anatomic component with the mouse and by selectingpossible feature values from a pop-down menu. It permits one to composesearch conditions according to conjunctions of the referred features andtheir values, or even store the currently selected subgroup of images forperforming further searching over the same subgroup.

Figure 5. Snapshot of the Learning Tool

After the desired features are selected, if the “SEARCH” button ispressed (see Figure 5), the IDB will look for all the example images in thecurrent class that match the conjunction of such features. The IDB thenmakes these images (if any) available under the inspection modes (seebelow). For instance, Figure 5 shows the third exemplar image, in a totalof 6 images, obtained with the search command, applied according to thefollowing search condition: <COMPONENT = “lesion”, FEATURE” =“size”, VALUE = “ large”> AND <COMPONENT = “cortical grey”,FEATURE = “state”, VALUE = “ affected by the lesion”>.

To inspect the example images of a class of abnormality in more detail,or even a subgroup of these images, the IDB offers several options:

Direne

52

O1: Button “FF>>> ” is used to move forwards in the subgroup ofimages;

O2: Button “<<<BB ” is used to move backwards in the subgroup ofimages;

O3: Button “MEASURE RATIO” allows the interactive measurement of aratio by requesting two consecutive pairs of mouse-click on the image,which are followed by visual feedback of each measure as well as theratio value;

O4: Button “SHOW PROTOCOL” is used to display clinical data such asage, sex and relevant facts of the medical history for the patient. Italso includes a complete diagnosis, written by domain experts, basedon the image features;

O5: Button “SHOW FEAT. LOC. ” is used to draw graphic elements(also called graphic locators), which are overlayed on the 2-D imagesurface, aimed at the identification of a feature location, or even fordisplaying some illustrative information related to a selected feature.

O6: Button “TRANSFER TO TUTORIAL MODE” turns the current imageavailable under the GTM for a guided session with it.

Because of the exploratory nature of the IDB, it only permits studentsto learn most diagnostic relationships inductively, rather than explicitlydiscussing these relationships during the interaction. However, knowledgeacquired by induction is fragile (Sharples, & du Boulay, 1988) and, as aresult, students could be expected to show misconceptions even afterusing the IDB. One source of diagnostic error is based upon the possibilityof students inferring the existence of images which “fit in” the featurespace, but in practice do not occur. That is, as instruction proceeds,students' beliefs are expected to become “over-general,” needing explicit(guided) interventions to avoid the problem.

The GTM and the Model of Dialogue Interpretation

To provide explicit advice and thus help the development ofexperiential knowledge, the GTM engages students in Socratic-likedialogues similar to those of the earlier Radiology Tutor, a domain-specific ITS for developing the skills of interpreting cardiac X-rays. Forexample, Figure 1 shows a dialogue with the GTM while teaching aboutknowledge designed (with RUI) for the domain of cardiac radiology. Infact, a similar dialogue to that in Figure 1 can also be carried out by theRadiology Tutor.

The GTM comments on students' partial diagnoses of images, critiquesinconsistencies in such diagnoses, indicates important areas of the image


53

with graphic elements, and gives continuity to the dialogue by askingabout other abnormal image features. The GTM guides the interaction bymonitoring students' actions as they progress with their hypotheses and byoffering intelligent feedback whenever they deviate from accuratediagnoses. To achieve these goals, the GTM is supported by a domain-independent model of dialogue interpretation based on four mechanisms:(1) feature visibility, (2) anatomy granularity, (3) the propagation ofdomain-specific teaching/protection rules, and (4) default teaching actions.

The feature visibility mechanism makes use of visibility information,available from image descriptions (see Section 4), matching it against thestudent's diagnosis and, if necessary, providing feedback. Basically, ithelps the student in understanding what knowledge is not directly visiblein an image but can still be inferred by a relationship among differentfeatures. For an example of this mechanism's effect on the tutorial dialoguesee lines 20-24 of Figure 1.

Sometimes a student's diagnosis, although not incorrect, may requiremore detailed information to be considered accurate. The anatomygranularity mechanism aims at tuning the student's perception on whereexactly, in the anatomical structure, the abnormal feature is situated. Lines5-7 of Figure 1 show a fragment of dialogue where this mechanism isactivated, challenging the student to provide a more accurate interpretationabout the feature calcification which is not found to be generalised in theaorta. In fact, the aorta is described in the knowledge base by a finer levelof granularity consisting of three subcomponents (see Figure 6), in one ofwhich (the aortic arch) the calcification is located.

The third mechanism refers to the propagation of domain-specificteaching and protection rules. RUI's dialogue interpreter traverses theanatomical structure defined in a class of abnormality (see Section 5), in adepth-first fashion, searching for such domain-specific rules and givesthem control precedence over the other three domain-independentmechanisms (1, 2 and 4). Depending on the conditional clause of a rule,the dialogue interpreter may fire the body of actions of this rule,propagating the effect on the student model which may result in theselection of another rule by this mechanism thus repeating the cycle.Through this mechanism, the GTM avoids the problem of over generalstudent beliefs by eventually firing actions of domain-specific rules whichare specially designed by the domain expert to enhance the power ofexplicit advice. See lines 10, 11 and 13-18 of Figure 1 and lines 4, 5, 10,11, 16 and 17 of Figure 2 as examples.

Direne

54

Figure 6. Anatomical structure of the heart-enlargement class

Finally, like the above mechanism, the default teaching actionsmechanism can also be activated by the model of dialogue interpretation todiscuss the value V of a feature F in an anatomical component C.However, default teaching actions are generated automatically by meta-level procedures that simply combine V, F and C to construct discoursestructures in a slot filling way. It is only executed whenever the dialogueinterpreter does not find a domain-specific rule referring to F and C infocus.

In other words, even if the expert does not define any domain specificrules at the conceptual level of design, the dialogue interpreter, combiningthis mechanism with the first two mechanisms (feature visibility andanatomy granularity), is still capable of guiding tutorial interactions withstudents. Examples of this mechanism's effect are found in lines 1, 4-7 and9 of Figure 1 and lines 1, 9, 13-16 and 20-22 of Figure 2.


55

Figure 7. Anatomical structure of the meningioma class

THE PRODUCTION LEVEL

At the production level, domain experts map existing classes ofabnormality (see Section 5) into specific representations, viewed as imagedescriptions. The image descriptions discussed here are those referred toin the literature as “high level scene analysis.” The main challenge forresearch at this level can be addressed by its strong links with semanticknowledge and the final purpose of information contained in a scene. Forexample, such information may be required to complement what isimmediately visible with what can be inferred from the scene.

At present, very little has been done in the field of automatic mappingbetween information from lower level processing and the input needs ofhigh-level systems for image understanding (Fischler, & Firschein, 1987).Compared to lower level representations, the high-level model ofknowledge about images and their descriptions is much more complex,often involving large symbolic databases which are manipulated byinference-like procedures (Ballard, & Brown, 1982). Previous approaches

Direne

56

for dealing with high-level, pictorial information in computer tutors(Sharples, 1989; Parkes, 1989; Rivers, 1988) also seem to be inaccordance with the above issues. RUI has no power for performing low-level, automatic image analysis. Therefore, only high-level descriptionscan be encoded through the Image Description tool, which allows anexpert to create image description entries for a database of pixel files.Note that the inference-like procedures for discussing image descriptionsare incorporated by the model of dialogue interpretation (see Section 3).

Figure 8. Snapshot of the Image Description Tool

In RUI, an image description entry consists of two parts: pixelinformation and symbolic information. Pixel information is held in theform of grey-level arrays which can be displayed on the screen as a visualimage. Associated with the pixel information, symbolic information deals


57

with the high-level description of visual concepts. This includes (1)anatomical information about the object portrayed in an image, (2) featuresand values that these features can take in different parts of the objectanatomy, (3) visibility information (some features may be hidden in anexample image, some others are not) and (4) graphic locators (geometricand text elements used by the domain expert to annotate images).

Figure 8 shows a snapshot of the Image Description Tool after beingused to set a value for the feature cadiothoracic-ratio of an imagedescription entry (associated with the heart enlargement class ofabnormality). The image has also been annotated with (graphic)knowledge about the projective geometry of radiography for this feature.

THE CONCEPTUAL LEVEL

The construction of the conceptual level description, the first designstep, is based on the idea of classes of abnormality. A class ofabnormality consists of an object-oriented description guided byinformation about the anatomical structure of the object (e.g. componentsand subcomponents of the human body) portrayed in example images ofthe class. In RUI, to allow information-hiding (an important object-oriented principle), each component of the anatomy encapsulates twoelements: domain knowledge and teaching operations.

Domain knowledge is defined in terms of feature dimensions andfeature restrictions while teaching operations are viewed as domain-specific teaching rules and protection rules. A feature dimension (orsimply feature) is a set of feature values (or simply values) given by theexpert to define a unit of visual concept. In Figure 9, for example, size is afeature of the heart component that can take one of a set of values in aspecific example image.

Feature restrictions are approached as physical relationships over thefeature space to provide more accurate means for experts to defineconsistent “boundaries” for a class of abnormality. In Figure 10, themeaning of feature restriction lv_h_abnormal is: “whenever the size of theleft ventricle is abnormal, so is the size of heart.” First-order predicatecalculus is used as the interface language of the Specification Tool fordefining feature restrictions, following the notation suggested by Hayes(1985). Therefore, a feature restriction contains instruction about the truth-value of a relationship among features (and values) and thus can bereferred to by its name in the conditional clause of domain-specificteaching and protection rules. Feature restriction lv_h_abnormal is used inthe conditional clause of protection rule tell_heart_size1 (also in Figure10).

Direne

58

Figure 9. Partial description of the heart component

Figure 10. Partial description of the left ventricle component

Domain-specific teaching rules are defined using a rule-basedprogramming language in the conventional sense. Likewise, protectionclauses are defined as rules, except for the peculiarity that designers are


59

allowed to “call” previously defined teaching rules. For example, in Figure10, the actions of protection rule tell_heart_size1 are fired by the model ofdialogue interpretation whenever feature restriction lv_h_abnormal isviolated by a student's answer, i.e. when NOT(lv_h_abnormal) is true. Theaim of protection rule tell_heart_size1 is to keep tutorial dialoguesconsistent by adding more information to the content of teaching ruletell_heart_size defined in Figure 9). This (additional) information,contained in the body of actions of a teaching or protection rule, isdescribed by commands constructed with language primitives like ASK(ask student about the condition of a feature), TELL (explain about thecondition of a feature) and GRAP_LOC (draw graphic locator to helpfocus on the current image feature).

The definition of teaching and protection rules constitutes an importantdesign task since, at the instructional level, they are regulators of theunidirectional process of knowledge communication from the tutor to thelearner in the sense that any feature (and value) of a class of abnormalitycan only be discussed through the active teaching operations of the class.An active teaching operation is either a default teaching action or aprotection rule or a teaching rule that has not been protected. Thisdiscipline helps to avoid the problem of encapsulation violation due toinheritance effect (Snyder, 1987) which is, in turn, just the computationalversion of the over-general student beliefs: the same problem if analysedfrom a more educational viewpoint.

The Subpart Incrementer

The description of a class of abnormality is constructed interactively,through the Specification Tool, using a graphic editor to create anatomicalcomponents. A component C may be added without any link to apreviously defined one. In this case, C is called root (see thorax in Figure6 and head in Figure 7). But after the root, a new component must bedefined with the help of those already existing, using two classincrementers: subpart and connection.

A component C1 is said to be created as a subpart of C0 (using thesubpart incrementer) when C1 contains new feature dimensions andfeature restrictions as well as all the features and feature restrictionsinherited from C0. Likewise, C1 inherits all rules from C0 and contains thedefinition of new teaching rules and protection rules, which may berequired to maintain consistency of tutorial dialogues.

The aim of the subpart incrementer is to enhance the power of dataabstraction (another important object oriented principle) by allowingdesigners to break an anatomical component into a finer level of

Direne

60

granularity, which means, into a set of more specific subcomponents.Furthermore, this partitioning procedure can be recursively extended toother subcomponents. Figure 6 shows an example of this, where the leftventricle is part of a more detailed description of the heart componentwhich, in turn, belongs to a finer level of granularity of the thorax. Asimilar situation is found in Figure 7. In both figures, components createdwith the subpart incrementer are represented by rectangles whereas thosecreated with the connection incrementer are represented by circles.

The Connection Incrementer

Similarly, a component Cn+1 is created to connect componentsC1,...,Cn (using the connection incrementer) when Cn+1 contains newfeature dimensions, feature restrictions and rules, as well as all theelements inherited from C1,...,Cn. The connection incrementer isnecessary because sometimes, when building new relationships amongcomponents of a class, the task demands the definition of new features andfeature restrictions that would not be allowed through the hierarchicalinheritance paths (created by means of the subpart incrementer). As anexample, component svc_ra, shown in Figure 6, contains a featurerestriction that relates features of the right atrium and the superior venacava.

It is important to note that the class incrementers, subpart andconnection, should not be confused with formalisms used by existing ITSarchitectures for curriculum definitions such as taxonomies andpartonomies (Howard, 1987). RUI's incrementers are even more generalformalisms. In fact, taxonomies and partonomies are just a subset amongthe universe of abstraction mechanisms that could be obtained byappropriately designing feature restrictions and domain-specific rules ofanatomical components.

As a final word on the RUI environment, we stress the maincontributions of this research as being the object-oriented design methodand tools for managing the complexity of knowledge representation forITSs that teach about visual images. The method of design is integratedwith the model of teaching in the sense that courseware authors canexternalise their expertise by using high-level formalisms (such as visualimages, names of visual features and graphical elements) which aresimilar to those presented to students for acquiring expertise. Thisintegration is achieved by fully implemented software tools that supportthe method and inform the tutoring shell about domain-specificknowledge. Existing diagnostic expert systems and computer tutors forvisual concept recognition, such as the Radiology Tutor (Sharples, & du


61

Boulay, 1988), KBET (Rivers, 1988), CLORIS (Parkes, 1989) and ICON(Swett, 1987), are domain-specific teaching tools and do not provideauthoring tools for describing images and their classes according to themedical practice. In addition, these expert systems and computer tutorsoverlook the issue of formalising surface knowledge representation underthe same language used both for experts to author courseware and forstudents to acquire expertise in highly visual domains, i.e. a rich, unified,expert-student framework of knowledge communication.

THE EVALUATION OF RUI

To evaluate the effectiveness of RUI, four experiments have beencarried out in different domains of expertise. Both the generality of RUI'sdesign method, and the usability of RUI's design tools were empiricallystudied by observing four subjects with varying backgrounds. The positiveresults obtained from this initial evaluation tend to indicate, among otherfindings, that RUI is an appropriate environment for encoding complexrepresentations of ITSs that teach about abnormal features in visualdomains of expertise.

However, it is worth observing that a fuller evaluation is needed with agreater number of experts in an attempt to cover a wide range ofapplication domains, not only those of medical Radiology. Anotherimportant aspect is that the Learning Tool has not been evaluated with realtrainee radiologists and, although the dialogues produced with it werefound reasonable by domain experts, nothing more precise is known aboutlearner’s performance variations after using the tool.

The Scope of the Experiments

As mentioned above, the evaluation attempted to test two hypotheses:H1: RUI's object-oriented design method is general enough to produce a

wide range of tutors for visual concepts while keeping high-leveldescriptions to a manageable size.

H2: The interfaces of the three tools are appropriate for usabilitypurposes in the sense that they not only accelerate the design processaccording to the proposed method but also provide an unified expert-student framework for communicating knowledge.

Hypothesis H1 suggests that one can deal with complex descriptions indifferent domains of expertise by repeatedly decomposing their bodies intosmaller, abstract modules of equivalent joint meaning. Hypothesis H2addresses the issues of productivity in time associated with the two design

Direne

62

tools (the Specification and the Image Description tools) and issues ofinterpretability of design representations through the Learning Tool(Direne, 1994). The following paragraphs briefly describe the threeprinciples focused through the hypotheses: abstraction, productivity andinterpretability.

The extent to which a software development methodology permits anexpert/designer to encode complex, high-level descriptions can beassessed by a measure of its language abstraction power. Managing alarge body of knowledge descriptions often imposes an artificialsegmentation of design requirements, at particular points in time. In usingabstraction principles, one admits that a problem is complex and, ratherthan try to describe it as an entire body, one concentrates only on part ofit. Besides that, the presence of domain-independent knowledge (e.g.hard-coded in a teaching shell) can dispense with a fair amount of domain-specific code that should be designed by the expert. Thus, a languageabstraction power can be intuitively explained as the capacity of allowingcomplex representations to become a growing number of smallerdescription components.

Productivity can be improved by means of software tools that providepractical mechanisms to reuse as much code as possible from existing“classes” of domain objects. This increases the amount of descriptions thatcan be created in a given time interval. Productivity is thus closely relatedto the idea of effort multiplication by means of automatic tools. Besidesthat, since design requirements always evolve in time, changes to thecurrent state of a description are virtually unavoidable and should result inthe need for faster maintenance interventions. Nevertheless, it is worthnoting that the emphasis on productivity tends to be at odds with thecoexistence of quality (Nicolson, & Scott, 1986).

Last but not least, a key, subjective parameter is interpretability. RUIis an integrated environment for design and teaching and as such, it mustprovide for the fact that medical radiological concepts tend to be taught ina rather unstructured way. Expert radiologists do not seem to have aconsistent method of teaching and, still in present days, the bulk oftraining in radiology is mostly on an apprenticeship basis. As a result,RUI’s systematic teaching procedure, guided by the structure of domain-specific knowledge, certainly contributed to formalise the interactionmethod of dialogue interpretation. The GTM’s formal method of teachingshows advantages such as being stable across different knowledgedomains and producing satisfactory results in terms of dialogue standards.


63

Managing Complex Descriptions in a Short Time

The implicit meaning of complexity management, in one way, refers tothe reduction of computer code required to build an ITS. A briefcomparison between the Radiology Tutor and the XRAY-TUTOR (thefirst knowledge base designed with RUI, aimed at the reconstruction ofthe Radiology Tutor) indicates that an ITS usually requires many morerules and modules of procedural knowledge when it is coded in aconventional language, even in those for AI programming, such as Prolog,Lisp or Pop-11 (Barrett et al., 1985). This code compaction is due to twomain reasons. Firstly, the high-level design language and toolssubstantially reduce the amount of code the programmer must specify todefine the parts of an ITS. Such high-level tools act as code generatorsthat designers can parameterise to meet the needs of their specific ITSapplications. Secondly, the template-based interface simplifies theintegration of this generated code into the application.

Table 1Development Time and Number of Rules for Knowledge Bases

KnowledgeBase

Number ofDomain-SpecificRules

DevelopmentTime (in Hours)

XRAY-TUTOR 10 15MRI-TUTOR1 23 45MRI-TUTOR2 7 15CT-TUTOR 12 15

From the numbers in Table 1, for example, only 10 domain-specificrules were necessary for the XRAY-TUTOR to reproduce very similartutorial dialogues to those of the Radiology Tutor. In fact, just thepedagogic model of the early Radiology Tutor is implemented with 86rules. One of the main reasons for such a drastic reduction in the numberof tutorial rules from 86 to 10 is explained by the activity of defaultteaching actions through RUI's model of dialogue interpretation. Withoutthem, it would be necessary to create at least one domain-specific rule fortelling and one for asking about each feature dimension defined in a classof abnormality.

In addition to the effect of default teaching actions, it is relevant toconsider here the other two types of domain-independent knowledge thatconstitute part of the mechanisms of RUI's model of dialogueinterpretation: feature visibility and anatomy granularity. The relevance isdue to the fact that they contribute even more to reduce the amount ofdomain-specific teaching operations which would have to be designed in

Direne

64

order to produce similar effects in the tutorial dialogue. All this domain-independent knowledge is hard-coded in RUI's Learning Tool along withthe rest of the code for the tool’s functionality procedures and interfaceobjects.

Another way of measuring the productivity improvement achieved by adesign system is to quantify the time required to create an application. Inusing RUI, the XRAY-TUTOR was defined in 2-3 (two to three) days ofwork (see Table 1). It would take a very experienced programmer a lotmore time to produce the same system, even using an appropriate AIprogramming language, a frame system, a production system and aninterface package.

Limitations of RUI

Since the focus of this evaluation has been both on the generality of thedesign method and on the usability of the tools, it is worth addressing thelimitations of RUI also based on these issues. However, before discussingon such limitations, it is necessary to note that RUI is an experimentalenvironment, conceived to examine possible ways in which one can adaptan automatic tool for teaching about abnormal image features. All fourtutors produced during the evaluation stage are not yet intended to be usedin real teaching environments since their knowledge bases are incompleteand their stock of images reduced to a convenient number. Moreover, thedesign language and tools include limitations that, although possible to bebypassed, still constitute a major barrier for allowing the specification ofmore elaborate tutorial dialogues.

So, the two main limitations found during the evaluation are:

(1) Hierarchical structure to model object anatomy:During the very first attempts to design a knowledge base with RUI,

experts seemed to find it difficult to describe anatomical components interms of a hierarchical representation. According to them, it seemedunnatural at times to model complex parts of the body, such as the brain,using purely simple hierarchical structures since each node of a hierarchycan only have one parent node. For example, in the design of the MRI-TUTOR2, this meant that the fourth ventricle was modeled as a subpart ofthe “brainstem” to be kept close to the “pons” and to the “medullaoblongata” from the teaching order viewpoint. However, for differentreasons (say physiological reasons), the same “fourth ventricle” could beseen as a direct subpart of the brain, like the “third ventricle.”


65

(2) Single pathology description:In using the Image Description Tool, a designer can create a symbolic

representation for an image as an instance of only one class of abnormality(pathology). That is not the case in some real situations when a patient canshow multiple diseases, which are easily identified by human experts.Although all the features of the secondary disease can be describedtogether, the symbolic description only gets assigned to one primary classof abnormality. In fact, even if the Image Description Tool was not limitedin this way, the Learning Tool would not be capable of discussing multiplepathologies present in an example image. As a result, if this limitation is tobe overcome, the problem of multiple pathologies must be tackled in bothtools.

Is RUI an authoring system or a design environment ?

A genuine authoring system would be ideally one that can allow anexpert to create his or her own course material by “programming” themachine without any help of a knowledge engineer. We could even gofurther and regard this type of system as an Intelligent Authoring System,meaning that it should be able to recognise the level of design expertise ofthe user and adapt itself accordingly. We should not think of authoringsystems as computer software capable of producing miracles, but whetherthey represent a substantially different approach to the development ofcourseware or not.

RUI's two design tools do not account for any type of intelligence inthe interaction with an expert/designer. Their interfaces rely on intuitivecomputer metaphors which could be ideally understood by expert authors.However, the real situation observed from this evaluation shows adifferent picture. Two of the four subjects, for example, were not able toprogram feature restrictions and certain types of domain-specific teachingrules. Nor were they capable of realising when meta-level teachingknowledge (such as that of default teaching actions) would be activated bythe Learning tool. A quick glance at the subjects’s background reveals thatthe Specification tool's interface language is not appropriate to be used bya medical expert. The tool seems to be more characteristic of a designenvironment than an authoring system.

On the other hand, none of the domain experts seemed to have anyproblems in using the Image Description tool's interface. Because it doesnot yet provide any on-line help, initial instructions in spoken languagewere necessary and after a few trials, all the subjects were completelyused to the simple description tasks associated with the tool.

Direne

66

Whatever the best classification for RUI is, from our experience, thereseems to be little doubt about the need for authoring systems and designenvironments that help experts in developing courseware. Although it hasnot been our aim to study the appropriate and precise level of abstractionof RUI's design language we have left the chance for domain experts andcomputer experts to work together in a cyclic manner, filling the gapscreated by the requirements when constructing an ITS.

CONCLUSION

The object-oriented method outlined provides a formal approach forstructuring domain knowledge and mechanisms for reasoning aboutsymbolic image descriptions of abnormal features guided by a model ofdialogue interpretation. Philosophical aspects of the method andmechanisms have been discussed, highlighting solutions for specificproblems like dealing with over-general student beliefs and enforcingconsistency of tutorial dialogues. The design and tutoring tools are fullyimplemented and allow designers to go through a complete journey of theautomatic life cycle for developing an ITS.

To substantiate generality claims about RUI, we have combined theevaluation of the software tools with the definition of knowledge bases forfour different domains of radiological expertise. The evaluation proceduresconcentrated on the power of the modeling formalisms as well as on theusability of the tools. The positive results obtained so far suggest that RUIis a new and suitable framework for authoring and interpreting imagedescriptions in the context of ITS evolution.

The project keeps progressing in two main research streams. The firstone concentrates on interface issues. Recent accounts of ITS architecturesattributed a key role to interface design (Bell, & Kedar, 1995; Szekely,Luo, & Neches, 1993; Wenger, 1987), claiming that knowledge should becommunicated in different “forms” (e.g. when applying a variety oftutorial styles), which can be achieved by inserting cognitive principles asdomain-independent knowledge into interface objects. Yet, there has beenlittle effort in detailing the nature and content of these objects. On goingwork in RUI is devoted to explore and implement a number of classes ofinterface objects, called Itelligent Tutoring Widgets (ITWs) so that domainexperts can create system-active and system-passive tutorial interactionsby directly manipulating such objects. This is achieved with a visualprogramming authoring tool which offers access to the more internalknowledge structures of an ITS (domain knowledge, student model andpedagogical directives) through the definition of the interface module in


67

an object-oriented, event-driven style, similarly to the more traditionalCBT authoring tools.

The second research stream refers to student modeling issues. In thecurrent implementation of RUI, it is not possible to design and maintaininformation about students. However, the variety of trainees involved withthe learning of visual concepts can be quite wide. For example, in ahospital, there are various classes of professionals who might be involvedwith the analysis of X-rays for different objectives (e.g. doctors, nurses,ambulance drivers, and others). To cover these needs we are alsoextending the authoring tools and the Learning Tool to allow the definitionof classes of students for an ITS. Such classes include ways of (1)grouping learners according to a given scale of stereotypes and attributes(Rich, 1983), and (2) showing how the studied skills progress along thetime.

References

Anderson, J.R. (1984). Cognitive psychology and intelligent tutoring. InProceedings of the Cognitive Society Conference. (pp.37-43).

Anderson, J.R. (1993). Rules of the Mind. Hillsdale, NJ: Erlbaum.Ballard, D.H., & Brown, C.M. (Eds.). (1982). Computer Vision. PrenticeHall.Barrett, R, Ramsay, A., & Sloman, A. (Eds). (1985). Pop-11: A Practical

Language for Artificial Intelligence. Ellis Horwood.Bell, B., & Kedar, S. (1995). When less is more: Supporting authoring and

interface building via special-purposetask models. In Proceedings of theWorld Conference on Artificial Intelligence in Education. (pp.533-540).

Bonar, J., Cunningham, R., & Schultz, J. (1986). An object orientedarchitecture for intelligent tutoring. In Proceedings of the ACMConference on Object-Oriented Programming Systems, Language andApplications. (pp. 269-276).

Bunderson, C.V. (1974). The design and production of learner-controlledcourseware for the TICCIT system: A progress report. InternationalJournal of Man-Machine Studies, 6:479-492.

Direne, A.I. (1993). Methodology and tools for designing concept tutoringsystems. In Proceedings of the World Conference on ArtificialIntelligence in Education. (pp. 58-65).

Direne, A.I. (1994). Methodology and tools for designing concept tutoringsystems. Technical Report-CSRP 294 (Doctoral Thesis), School ofCognitive and Computing Sciences - The University of Sussex - UK.

Direne

68

Fischler, M.A., & Firschein, O. (Eds.). (1987). Intelligence: The Eye, theBrain, and the Computer. Addison-Wesley.

Hayes, P.J. (1985). The logic of frames. In R.J. Brachman & H.J.Levesque (Eds.), Readings in Knowledge Representation. MorganKaufmann.

Howard, R.W. (Ed.). (1987). Concepts and Schemata. CassellEducational.Jeffery, N., Teather, B.A., Teather, D., Sharples, M., du Boulay, B.,

Direne, A.I., du Boulay, G.H. (1993). Statistical modeling andintelligent tutoring of visual concepts for MR imaging of the head. InProceedings of the World Conference on Artificial Intelligence inEducation. (pp. 562).

Lesgold, A.M. (1984). Acquiring expertise. In J.R. Anderson & S.M.Kosslyn (Eds.), Tutorials in Learning and Memory: Essays in Honor ofGordon Bower. W.H. Freeman.

Lesgold, A.M., Rubinson, H., Glasser, P.F.R., Klopfer, D., & Wang, Y.(1989). Expertise in a complex skill: Diagnosing x-ray pictures. In M.Chi, R. Glasser & M. Farr (Eds.), The Nature of Expertise. LawrenceErlbaum.

Major, N. (1995). Modelling Teaching Strategies. Journal of ArtificialIntelligrnce in Education, 6 (2/3):117-152.

Mervis, C. B., & Rosch, E. (1981). Categorisation and natural objects.Annual Review of Psychology, 32:89-115.

Murray, T., & Woolf, B. (1992). Results of encoding knowledge withtutor construction tools. In Proceedings of the Tenth NationalConference on Artificial Intelligence (AAAI-92). (pp. 17-23).

Myles-Worsley, M., & Johnston, W.A. (1988). The influence of expertiseon x-ray image processing. Journal of Experimental Psychology:Learning Memory and Cognition, 14(3):553-557.

Nicolson, R. I., & Scott, P. J. (1986). Computars and education: Thesoftware production problem. British Journal of Education Technology,17:26-35.

O'Shea, T., Bornat, R., du Boulay, B., Eisenstadt, M., & Page, I. (1984).Tools for creating intelligent computer tutors. In A. Elithorn & R.Barneji (Eds.), Human and Artificial Intelligence. North-Holland.

Parkes, A.P. (1989). The prototype CLORIS system: Describing,retrieving and discussing videodisk stills and sequences. InformationProcessing and Management, 25(2):171-186.

Rich, E. (1983). Users are individuals: Individualizing user models.International Journal of Man-Machine Studies, 18, 199-214

Rivers, R. (1988). KBET: Final report. Technical report, LogicaCambridge.


69

Rosch, E., & Lloyd, B.B. (Eds.). (1978). Cognition and Categorization.Lawrence Erlbaum.Sharples, M., & du Boulay, B. (1988). Knowledge representation,

teaching strategy and simplifying assumptions for a concept tutoringsystem. In Proceedings of European Conference on ArtificialIntelligence. (pp. 268-270).

Sharples, M. (1989). The Radiology Tutor: Computer-based teaching ofvisual categorisation. In D. Bierman, J. Breuker & J. Sandberg (Eds.),Artificial Intelligence and Education: Synthesis and reflection(Proceedings of the 4th International Conference on AI & Education).Amsterdam, Netherlands: IOS.

Sharples, M., du Boulay, B., Teather, B.A., Teather, D., Jeffery, N., duBoulay, G.H. (1995). The MR tutor: Computer-based training andprofessional practice. In Proceedings of the World Conference onArtificial Intelligence in Education. (pp. 429-436).

Sleeman, D.H. (1987). PIXIE: a shell for developing intelligent tutoringsystems. In R. Lawler & M. Yazdani (Eds.), AI and Education:Learning Environments and Intelligent Tutoring Systems. AblexPublishing.

Snyder, A. (1987). Inheritance and the development of encapsulatedsoftware systems. In B. Shriver & P. Wegner (Eds.), ResearchDirections in Object-Oriented Programming. MIT Press.

Spensley, F., & Elsom-Cook, M. (1988). DOMINIE: Teaching andassessment strategies. Technical Report-37, Institute of EducationalTechnology - The Open University.

Swett, H.A., & Miller, P.L. (1987). ICON: A computer-based approach todifferential diagnosis in radiology. Radiology, 163 (2):555-558.

Szekely, P., Luo, P., & Neches, R. (1995). Beyond interface builders:Model-based interface tools. In Proceedings of INTERCHI. (pp. 383-390).

Wenger, E. (1987). Artificial Intelligence and Tutoring Systems:Computational and Cognitive Approaches to the Communication ofKnowledge. Morgan Kaufmann.

Woolf, B. (1991). Representing, acquiring and reasoning about tutoringknowledge. In H. Burns, J.W. Parlett & C.L. Redfield (Eds.), IntelligentTutoring Systems: Evolutions in Design. Lawrence Erlbaum.

Acknowledgements

The author is grateful to Mike Sharples and Ben du Boulay for theproductive discussions and supervision and to the Poplog development

Direne

70

team at the University of Sussex - UK. The expertise in MR imaging hasbeen provided by the Medical Systems Research Group (De MontfordUniversity - Leicester), by George du Boulay (Institute of Neurology -London) and by T. Arvanitis (University of Sussex), with the MR dialoguebeing implemented by Nathan Jeffery. As to the CT imaging expertise, thecredits go to Penelope Gordon Gordon (Royal Sussex County Hospital).The research was supported by grant CNPq-Brazil-204154/88-0.

Author Note

This article is based on a doctoral dissertation at the University ofSussex-UK (Direne, 1994). Parts of this dissertation have been publishedearlier in Direne (1993).

designing intelligent systems for teaching visual …iaied.org/pub/1052/file/1052_paper.pdfdesigning...

Documents