Compurers Eu’uc Vol. 15. No. l-3. pp. 69-81. 1990 Pnnxd ,n Great l3r1wn All rights reserved

0360-1315 90 53 00 -0.00 CopyrIght c 1990 Pergamon Prrjs plc

ECAL: BRIDGING THE GAP BETWEEN CAL AND INTELLIGENT TUTORING SYSTEMS

MARK T. ELSOM-COOK and CLAIRE E. O’MALLEY Institute of Educational Technology, The Open University, Walton Hall,

Milton Keynes MK7 6AA, England

Abstract-In the traditional CAL system the author generates material to be presented by computer, and the computer simply follows the explicit instructions of the author in interacting with a student. Intelligent tutoring systems rely on abstracting the implicit knowledge of a CAL system into explicit separable representations. There is a large gap between these two approaches which ECAL attempts to bridge by incorporating simple versions of ideas from artificial intelligence as extensions to a traditional CAL tool. It can be regarded as an experiment in minimalism in intelligent tutoring but it is also a practical educational tool. This paper introduces some of the authoring issues raised by ECAL, followed by a general view of the architecture of the system, and a discussion of the educational model which is implicit in ECAL. The final sections discuss the ECAL presentation system and the authoring environment.

INTRODUCTION

In CAL the author generates materials to be presented by a computer, and the computer simply follows the explicit instructions of the author in interacting with a student. This requires the author to develop new ways of thinking about presenting material, many of which are to do with the strengths and limitations of the system itself, rather than with educational aspects. This is a difficult task which involves the author in learning new skills outside her domain of expertise, and which may be of limited usefulness in the future. Often, creators of CA1 courseware simply map practice from some other medium onto the computer, and the result is frequently educationally unexciting at best, and can be positively harmful. If we examine the practical use of computers in education, however, almost all the examples which we see have their origins in the CAL methodology.

Research on the application of artificial intelligence (AI) techniques to educational computing has been in progress for about 20 years. AI systems [we will refer to them as ITS (intelligent tutoring systems) from now on, but there are other ways to use AI with which we are not concerned in this paper] rely on abstracting out the knowledge which is implicit in a CAL system, into exlicit separable representations. This promises benefits, such as being able to change the subject domain of a system without having to change the way it teaches, improving adaptiveness to the student, etc. These AI systems are complex and difficult to understand, and are generally constructed as research tools, rather than as practical educational applications.

There is a large gap between these approaches, and between the respective communities. The system described in this paper is an attempt to bridge that gap by incorporating simple versions of AI ideas as extensions to a traditional CAL tool. The resulting system (ECAL) is robust and easily understood.

ECAL stands for “extended computer assisted learning”. This system is similar to traditional CAL authoring systems in that it allows the author to enter and modify teaching materials. It differs from such systems in that the author is relieved of the burden of specifying the flow of control of the teaching materials. The author concentrates on the specification of content, and of an abstract structure for the domain to be taught. The system “extends” traditional CAL by incorporating a range of simple AI features. This places the system at an intermediate level between traditional CAL and Intelligent tutoring systems. An overview of the system can be found in [l-3].

ECAL can be regarded from three different perspectives:

(1) ECAL is an experiment in ITS minimalism. It allows us to investigate what the fundamental role of the components of an ITS are. Additionally, we can explore how simple it is to make each component, such that the component still retains a meaningful role within the system. Following from these questions, we can ask what ECAL tells us about the necessary complexity of ITS systems.

69

70 ‘MARK T. ELSOM-COOK and CLAIRE E. O’MALLEY

(2) ECAL is a practical educational tool. While remaining within a frame-based approach, ECAL facilitates the development of higher quality frame-based courses in shorter time periods by providing the author with more educationally meaningful tools. Additionally. it frees the author to some extent from having to interleave the differing forms of knowledge implicit in a CAL course. For the student, ECAL can provide a higher level of adaptivity than is possible (for similar expenditures of effort) in a CAL system.

(3) ECAL is a bridge between CAL and ITS. By acting as an extended CAL system or a cut-down ITS, ECAL provides a discussion point between the two approaches. It aims to combine the practicality of CAL systems with some of the innovations of ITS.

In the remainder of this paper we will introduce some of the authoring issues which are raised by ECAL. This will be followed by a general overview of the architecture of the system, and by a discussion of the educational model which is implicit in ECAL. The following two sections introduce the presentation system and the authoring tools.

THE AUTHORING TASK IN CA1 AND ICAI

In a traditional CAL system the computer acts as a presentation device for material prepared by an author. The author is given a set of tools to enable her to generate textual and graphical ‘frames’ for presentation to the student. The author must also link these frames into a branching structure where choices on what to do next are explicitly wired into the system based upon the result of anticipated user behaviours. Such systems force the author to consider the content of the course, the structure of the material and the sequence of presentation concurrently. The lack of modularity of these components mean that any subsequent changes in content or structure cause interactions whose effects are difficult to predict. It is often easier to completely rewrite a course than to modify it. The author must also manage these activities simultaneously at a number of levels: e.g. the level of the content of a particular frame, how it is presented, whether it is understandable to the student; the level of the module of which the frame forms a part, whether the content of the frame is coherent when viewed at this level; whether there is any impact on the content of frames in other modules. The problems here are similar to those involved in any task which requires the simultaneous management of multiple constraints (see e.g. [4]).

In an ITS, the problems are somewhat different. The computer tutor generates teaching materials and teaching sequences from a set of sources of knowledge about teaching, interaction, learning and the nature of the particular domain. The claim is that these sources are, in the ultimate tutoring system, sufficiently abstracted to be independent. It is intended that an ‘author’ should be able to change the domain being taught by changing the representation of domain knowledge, without changing other parts of the system. With one exception [5], tutoring systems which have been used in more than one domain rely on the presence of an AI programmer to change the domain representation at the level of program code. If tutoring systems are to achieve this goal of domain independence (even to a limited degree), then it is necessary to consider what we now mean by the author’s task, and what tools we should provide to support it.

Some of the general issues involved in providing authoring tools for ICAI (intelligent computer assisted instruction) authors are discussed in Spensley [6]. Here we focus on two main issues: externalizing what is in an author’s head, and helping the author to predict the behaviour of the system.

Externalization of domain knowledge

The key to generative tutoring is the creation of abstracted descriptions of the domain to be taught. This knowledge is often implicit in teachers, and in a form which is difficult to access. The internal form which a teacher uses may not be sufficiently abstracted for the purposes of a tutoring system. Elsewhere [7] we have discussed a model of teaching as negotiated dialogue. A key part of this model is that the teacher must convert her internal representation into one suitable for communication by separating the material to be taught from other internal structures and ‘tying the loose ends’ in such a way that they can be integrated with the students existing knowledge once

Bridging the gap between CAL and ITS 71

the structures have been transferred. It is exactly this process of externalizing material in an accessible form that must be explicit in authoring.

A key aim of supporting tools for ICAI authors must be to facilitate the externalization of the author’s knowledge at a suitable level of abstraction. How this is achieved depends on the audience which is being targetted. The process has some similarity to knowledge elicitation for expert systems, but the rather different requirements of knowledge representation in ICAI impose different constraints. In particular, a shallow, simple representation such as a set of rules is inadequate for capturing the knowledge in a way which is meaningful to humans. It is necessary to elicit representations of a form which are educationally useful, and which are, therefore, considerably deeper than the sorts of representations on which most elicitation work is done.

Predicting the behaciour of an ICAI system

In a traditional CAL system the author builds in the presentation sequence. She knows what the system will do in each situation and can follow through the complete set of sequences (given enough time). Because the form and content of an ICAI interaction is generated in response to the current situation, it is not possible to do a similar thing for such a system. The author does not necessarily know how the system will choose to present things, and it is not possible to run through the complete set of generated interactions (since this set is, in general, infinite). It is therefore important that a set of tools is provided to allow the author to get a feel for the interactions and to make appropriate modifications.

GENERAL ARCHITECTURE OF THE SYSTEM

ECAL can be divided into three major components (although in the implementation they are integrated). The first component is the set of authoring tools. These tools allow the author to generate the actual material which the student will see. The material is constructed as a set of frames, either presentation of diagnostic in nature. At present these frames are purely textual, although the mechanisms of ECAL are equally effective with graphics, animation, sound, interactive video or CD-ROM. In addition to creating material, the author specifies an abstact structure which relates the material. This can either be achieved explicitly or implicitly. The authoring tools produce a static knowledge representation describing the course which is passed to the presentation system. The presentation system controls the interaction with the student. It makes use of the static information, together with a model of interaction and a model of the student, to generate a course sequence. The structure of this sequence is adapted to individuals. From the interaction, the presentation system develops a ‘dialogue history’ which can be used in debugging the course.

As has been mentioned earlier, it is important that the decisions taken by an AI teaching system should be comprehensible and explicit. To this end, the third component of ECAL is a collection of debugging tools. These tools allow the author to test the system on sample students, to examine its behaviour and the decisions that were taken, and to modify its decision making processes.

Authoring Debugging tools tools

Presentation system

Fig. I. General architecture of the ECAL system.

72 MARK T. ELSOM-COOK and CLAIRE E. O’IMALLEY

EDUCATIONAL BASIS OF ECAL

A key idea for ITS is that the educational model of the system should be made explicit in the system itself. If this is not possible, then the model should at least be expressed as clearly as possible of paper. In the case of ECAL, in which authoring plays such a central role, this is even more important. If an author is to make effective use of the capabilities of the system she must understand the educational model with which the system is working. To this end, ECAL is based around a theory of curriculum design which, while not being widely used in detail, is probably familiar in general form to most designers of non computer based courses. It is not being claimed that this theory is the ‘right’ way to design courses, nor that it is inherently more appropriate than others to a computer based approach. Any one of a number of course design models could have been used.

The model is a simplified version of that proposed by Posner and Rudnitsky [S]. The overall mechanism is to construct a curriculum by identifying intended learning outcomes (ILOs) and clustering them into groups to form a course. The set of types of intended learning outcomes form a hierarchy which is shown in Fig. 2. For the purposes of ECAL we will concern ourselves only with cognitive skills and non-skills. The other types of IL0 require further work (in some cases, considerable work) before they can be used effectively by a computer.

Posner and Rudnitsky offer a variety of mechanisms for clustering these ILOs into units which form parts of a course at a variety of levels of granularity. In ECAL we assume jusr one of these clustering approaches: a subject-related concept clustering. In effect, this means that we assume that the structure of the course to be determined only by the structure of the subject-matter. rather than by other social or individualistic factors, and that within a course we take the major concepts to be the key points around which our clusters are built.

THE ECAL PRESENTATION SYSTEM

The starting point for the system is a traditional CAL frame-based authoring system. In such a system the coursewriter prepares a set of presentation frames and diagnostic frames nhich contain material about the domain. She then links them together into a standard branching structure, with branching decisions based on the outcom e of the diagnostic frames. Such systems are pure electronic presentation devices which do not ‘know’ anything about what they are doing. They simply follow a branching structure without any model of the process in which they are engaged.

ECAL can make a number of major improvements to such an authoring system by using simple AI techniques such a pupil model, a dialogue model, a teaching algorithm and an explicit knowledge representation. Each of these components will now be discussed individually.

Explicit representation of knowledge

One of the key distinguishing factors of an ICAI system is the abstraction of information about teaching and about the domain. The explicit representation of such information allows us to

,.,k;:;

_ -

P

Cognitiue I\

Cognitiue Affectiue

Psychomotor/perceptual A Conceptual

Propositional

Fig. 2. Hierarchy of IL0 types (redrawn from Posner and Rudnitsky [81).

Bridging the gap between CAL and ITS 73

provide the computer with rules which allow it to reason about the activities in which it is engaged. This reasoning obviates the necessity for the coursewriter to explicitly envisage all possible situations which might arise during the course of the educational interaction.

Most AI systems use complex representation mechanisms in order to allow the full power of AI-based reasoning techniques to be applied to them. Such systems require a deep understanding of the AI methodology involved if they are to be used successfully. The ECAL system uses a similar, but simpler technique which can be easily understood by a person with no programming experience. The author provides a sequence of keywords associated with each presentation frame, and with each diagnostic frame and response. (These keywords constitute the ILOs around which the system is structured.) The computer can then take decisions about sequencing of material based upon these keywords. Such an approach is far less effective than knowledge representation techniques in AI-based systems, but nevertheless eases the load on the courseware author. This is because the author no longer has to explicitly encode all possible sequence information about the teaching material.

The keywords are identified as being ‘connected’ or not, depending on whether two keywords are used together in describing a particular frame. If the keywords are used together in more than one frame and the strength of their connectedness is correspondingly greater. ECAL produces a ‘correctedness matrix’ from this information.

By using a standard clustering algorithm, ECAL generates a set of relationships between the ILOs which reflects the strengths of their relationships to all other ILOs in the course. Any pair of ILOs has such a relatedness, and this is used heavily in the decision-making process of the presentation system. In addition to this information, the knowledge representation captures the notions of ‘importance’ and ‘generality’ for an individual ILO. Importance is simply a measure of how much an IL0 is used. Generality relates to the breadth with which that use occurs within the course. Two ILOs may be of equal importance, but the one which is more widely distributed through the course is considered to be more general. These relationships are obtained using straight-forward algorithms. Figure 3 shows an example of a concept map generated in this way for a simple course.

Models of the pupil

A student model is necessary if the system is to go beyond being simply reactive to input by the user. Longer term understanding of the state of the pupil allows decisions about the most appropriate action for the system to take to be specified at a more abstract level. Research on ICAL has shown that student modelling is one of the most difficult areas of the field. The most promising approach is to try and create a model of the student by modelling the learning process. This learning model can then be used to generate a model of the student which has a structure independent of the expertise within the computer.

Rather than trying to develop a sophisticated student model, the ECAL system uses a simplification of a well understood modelling technique, namely ‘subset modelling’ (or overlay modelling). This technique assumes that the student’s knowledge of the domain is regarded as a subset of the expertise within the system, which is structured in an identical manner. The modeiling

Fig. 3. A simple concept map.

73 MARK T. ELSOWCCMX and CLAIRE E. OXALLEY

task is then to decide which pieces of a system’s knowledge are understood by the student. The teaching process involves identifying those pieces in the user model which are not understood and presenting information about them to the pupil.

Subset modelling usually requires the division of the domain into units so small that we can be certain whether the student understands them or does not. Because the ECAL system is not designed to work at such a fine level of granularity, the user model cannot make this assumption. Instead, we have generalized the idea to that of a numeric overlay model’ which assigns confidence factors to each piece of the expert knowledge. This is simply a list of the ILOs in the system with a numeric value associated with each one. The numeric value (range from 0 to 1) reflects the confidence of the system that the user understands the concept associated with this ILO. A value of 0 indicates that the pupil does not know it. A value of 1 indicates that the pupil does. These values are continuously updated by the system as frames are presented to the student.

When the student is presented with a frame, the ILOs associated with that frame are updated accordingly. In the case of a diagnostic frame, the author additionally specifies a number of possible answers with associated ILOs for each. If the student gives that answer, it is taken to indicate that the student has misunderstood the concepts denoted by those ILOs. This is a simple technique. The most interesting part about it is the attempt to solve the credit assignment problem.

When a student answers a question involving several ILOs incorrectly, how can the system assign the ‘blame’ for the error to a particular ILO? This is achieved by updating the student model according to a mechanism which takes into account both the behaviour of the student and the previous state of the student model. Changes in the value for a particular IL0 are expressed as percentages of the previous value for that ILO. These percentages are themselves calculated using the previous value.

The key feature of the functions used for their purpose is that they impose a damped envelope around the changes to the student model. In essence this means that, as the system gains more confidence that the student understands a particular concept. it becomes harder for the system to change its confidence about the pupil’s understanding. If the pupil answers incorrectly to a question which involves one IL0 in which the system has high confidence, and one item in which the system has low confidence, then the system will lay more blame upon the IL0 in which the confidence was lower initially.

Dialogue model

If the system is to maintain a smooth educational interaction, it requires knowledge about the history of the interaction to date. Recent work in AI allows us to think about dialogue as a form of planning. This implies (among other things) a model of dialogue focus which corresponds to the set of goals which our utterances are currently attempting to satisfy. The past history of the interaction corresponds to sets of goals which have been satisfied, and the future, unplanned, part of the interaction corresponds to goals which have yet to be satisfied.

There are three major components to the dialogue model: a dialogue history, a current focus and a set of goals remaining to be satisfied. In the ECAL system dialogue goals are approximated by the ILOs. The set of goals remaining to be satisfied by the system is a list of ILOs. The focus model is also a list of ILOs. This is a much shorter list describing those goals which the system is currently trying to satisfy. It contains a number of elements from the previous few diagnos- tic/presentation frames. One of these elements has the special status of “explicit focus”. This indicates the currently active teaching goal. This is illustrated in Fig. 4. Unused and completed topics are shown in the two buckets at the top of the diagram. Unused frames are shown in the bucket at the bottom of the diagram. The centre of the diagram shows the way in which the system constructs sequences of moves in ‘focus’ which are in turn used to select frames. This is discussed in more detail in the following section.

Teaching algorithm

The teaching algorithm is really a simple model of the presentation decisions which the course writer would take at the time she prepares the material. By taking the decision interactively during the presentation process she need for such decisions by the designer can be obviated. In more sophisticated AI systems, this decision involves complex reasoning about the teaching process and

Bridging the gap between CAL and ITS

completed topics new topics

unusedframes

Fig. 4. The ECAL dialogue model.

the goals to be satisfied. For ECAL it is simply a matter of choosing the next frame based on information about the state of the dialogue, the state of the student, and the available teaching material.

The presentation of frames is controlled by directly accessing the student model, dialogue model, knowledge representation, and presentation frames. The system selects a frame to use next. presents that frame, obtains any response and updates the student model and dialogue model accordingly.

The choice of frame is determined from information about the focus, dialogue history, unsatisfied dialogue goals and student model. The system first attempts to choose a frame which maintains the current focus of interaction. If this is not possible, it selects a new focus and chooses an appropriate frame.

The focus consists of one item which is explicitly in focus and a number of items which are implicitly in focus. The implicitly focused items are those ILOs which are mentioned in frames which have been used to present the item which is the main focus.

Given a particular focus, the system collects all frames which explicitly mention that focus in the IL0 list and which have not yet been presented. There may be a number of presentation and diagnostic frames in this category. The frames are given an order of priority such that those which have maximum priority are those in which all other ILOs (other than the focus) are known, according to the student model. Secondary priority is given to frames where all ILOs (other than the focus) are implicitly in focus. Other frames must contain one or more ILOs which are not in focus. These are ordered such that the one with the fewest ‘new’ ILOs has the highest priority.

If both presentation and diagnostic frames are present, then the system chooses which to present according to certain heuristics. If this process fails to find any suitable frames, then the system must change focus and repeat the process. Changing focus involves promoting one of the implicitly focused items to explicit focus.

76 MARK T. ELSOM-COOK and CLAIRE E. O’MALLEY

If none of the implicitly focused items can be promoted, then a major change of focus occurs. The system achieves this by searching the set of untaught ILOs and examining the presentation frames associated with each. It chooses the frame containing the minimum number of untaught ILOs. and promotes the untaught IL0 which is mentioned in the maximum number of frames to be the current focus.

When a presentation or diagnostic frame has been given to the pupil, the system must update its knowledge. In particular the dialogue history, focus and student model must be updated. If the explicit focus has not been changed, and a presentation frame has been used. then the implicit focus and dialogue history must be updated. The focus is updated by adding all ILOs of the current presentation frame to the implicit focus list, deleting duplicates. The dialogue history is updated by marketing the frame which has been presented as used, and by increasing the ‘taught’ marker of each IL0 in that frame, according to the numeric algorithm.

If the focus has not been changed and a diagnostic frame has been used. then the user model and dialogue history must be updated. If the pupil answered the question correctly, then the confidence in each of the ILOs associated with that answer, as returned by the frame, is increased. The dialogue history is updated by marking the frame as used, and no change is made to the dialogue history with respect to ILOs. If the pupil got the answer to the diagnostic frame incorrect, then the student model is updated to decrease the confidence in each of the associated ILOs. The dialogue history is updated as above.

The situation is somewhat different if the focus has been explicitly changed. This implies that the frame used must have been a presentation one. The dialogue history and focus must be updated. The dialogue history is updated as in the case where there is no explicit change of focus. The focus is updated differently, because this is a major topic change. This implies that all the items which were explicitly focussed are no longer in focus. The updating is therefore achieved by replacing the entire implicit focus with the list of ILOs from this frame.

An example of one set of rules for these decisions would be as follows:

(1) Choosing to present/diagnose Diagnose if there is some confidence in every item in the whole focus

(2) Frame selection (there are actually 4 sets of rules like this) (a) Choose frame most related to previous frame (b) Choose most focused frame (c) Choose frame with highest confidence (d) Choose frame with most importance (e) Choose frame with lower generality than previous frame

(3) Choosing a new focus (there are 3 sets of rules like this) (a) Choose most general IL0 (b) Choose most important IL0 (c) Choose IL0 with highest confidence (d) Choose IL0 most related to current explicit focus.

Where there are multiple rules they are in order of priority, but the decision is a combination of the recommendations of all these rules.

In this section we have outlined the major components of a system which presents material to the student in an adaptive manner. The following section will briefly explore the authoring and debugging tools provided with the system.

AUTHORING ENVIRONMENT

Since the environment for generating materials and the tools for debugging a course are both intended for use by the author, we will discuss them together under the general term ‘authoring environment’.

Material generation encironment

The decision of the authoring environment will be divided into two sections. In the first we will present a minimal environment that can be used for the simplest form of authoring. This serves

Bridging the gap between CAL and ITS 77

to demonstrate the basic ideas. In the second section we will show some of the tools which exist in the extended environment, and discuss some of the extensions which are currently being considered.

The key component of ECAL course design is establishing the relationship between frames of material to be presented to the learner and ILOs which describe those frames. This is, a many to many mapping between frames and ILOs. The minimal version of the ECAL system provides a tool which allows the author to create textural presentation and diagnostic frames, and to associate a number of ILOs with each frame by a method of incremental indexing. The author can move between the frames sequentially or by searching for specific words. It is possible to have a number of frames visible at the same time, but only one is active (i.e. susceptible to changes).

Clearly, the minimal environment is of limited usefulness in manipulating the structure of a course. It supports the creation of frames, but modification and manipulation of the course structure (or even seeing it clearly) are not well supported.

In an extended version of the authoring environment it is necessary to provide a number of tools for enabling the author to see and manipulate the structure which is being imposed on the course. This is achieved by providing a range of views on the material. Currently three tools are supported, but later versions of the system will extend this to a fuller toolkit.

Frame editor. The frame editor is essentially like that shown above, except that it allows multiple frames and IL0 windows to be visible simultaneously.

Concept map browser. A key idea in ECAL is the ‘concept map’. This is a network which descirbes the interrelationships between the ILOs. This is an important part of the operation of the system. By allowing the author direct access to this map, the structure of the course is made clearer. The map can be generated automatically from the ILOs given in frames but, if the author wishes, ILOs and can be added and deleted by hand.

The concept map is represented graphically as a network in which nodes are ILOs and links represent frames shared by ILOs. Any two ILOs which appear in the same frame have a link between them. Shorter links indicate a larger number of common frames (the more frames a pair of ILOs, share, the more closely they are assumed to be related).

The visual appearance of the concept map is similar to that shown in Fig. 3. Frame concept mapping. When a link appears between two ILOs in the concept map, that implies

that there is one or more frames linking the ILOs. To enable the author to explore the link in more detail, and to modify it without actually removing it in its entirety, a frame concept mapping is offered. Using this tool, the author can ‘zoom in’ on a particular link, and see the internal structure. For example, Fig. 6 shows a connection between the ILOs ‘model’ and ‘principle’. The link has grown to show the individual frames involved in the link. These frames can be inspected or deleted, and the consequences of deleting them can be investigated.

Debugging environment

Development of a course of material is an iterative process in which decisions may be revised as the course is being produced. Supporting incremental modification of the course can increase

About Edit Tutorial Frames ConceDts

Ecal ideas 1 I I

Ecal is a tool for generating courses using fil techniques. The courses adapt to the students

Ecal Al adaption

I IU

Fig. 5. A simple ECAL authoring environment.

MARK T. ELSOM-COOK and CLAIRE E. O’MALLEY

Liip Model Principle

Lisp

model

Prolog model principle

Fig. 6. Frame-concept mapping

processes implies that special consideration needs to be given to ‘debugging tools’ to assist in modifying a (partially) existing course.

The need to change the material content of a single frame during the course material preparation phase is a straightforward activity with no other repercussions. The need to change the overall structure is well supported and does not imply difficult or lengthy operations, given the tools described above. What requires additional support is the process of testing the course and identifying the changes in structure or content which need to be made.

It was mentioned earlier that it is not possible to exhaustively check every teaching sequence in an intelligent tutoring system. Testing the ECAL courseware does not involve checking or correcting program syntax, predominant in traditional CAL courseware testing. Testing of the teaching strategy mechanisms is also unnecessary since these are embedded in the system. Although, as was mentioned earlier, some support for the tutor in selecting strategies and designing the interface to the presentation environment may be offered. What needs to be tested centres mainly on confirming that the observed sequence of material presented conforms with the objectives of the course. In other words, ECAL course testing is a domain-oriented activity as opposed to the program oriented activity of traditional CAL.

For this reason, the author is provided with a set of debugging tools to allow her to examine the interaction at a high level of abstraction as well as being given a view of the teaching sequence seen by the learner. The main question which the tools are intended to answer is “Why did this frame come now?“. It will be noted that the tools often provide duplicate information. This is in line with the philosophy of ECAL as providing an open system in which authors can adopt their own style, rather than dictating a style to the author. It is anticipated that different authors will find different debugging tools to be appropriate to their needs.

Dialogue fool. One of the key criteria used by ECAL in deciding the presentation sequencing is the dialogue state. Decisions about presentation are largely guided by the current focus of interaction and those foci which the system regards as currently relevant. For this reason it is proposed that the author should be able to view the dialogue history and see the currently relevant ILOs. In Fig. 7, the dialogue history progresses from left to right. The lowest item in each case

(-frame (Previous frame ) CExamine decision) (Quit

model

lisp

[ prolog

basic model cmd

Syntax I, ; prolog basic

Fig. 7. Dialogue debugging tool.

Bridging the gap between CAL and ITS 79

lisp

Principles

Prolog

Model

unused fmmes

Syntax

Fig. 8. ECAL student model.

is the explicit focus, while the other items are in implicit focus. Focus changes are marked by vertical lines.

In making this tool more powerful for the author, we anticipated allowing the author to ask for additional information about why a particular focus was selected. The author can examine the relatedness etc. in the light of the teaching strategy being used in order to revise her structure for the course.

Student model. The student model is not modifiable by the author, but it is relevant information which should be viewable. In the case of ECAL, the student model is simply depicted as a bar graph. The bars have values from 0 to 1 reflecting the confidence that the student understands a particular ILO. These bars can be seen to vary in length as the interaction progresses. If the author clicks on a bar, a graph is shown of the previous values of that bar against the frames which have influenced that bar. This effectively provides a historical explanation of the current value of the student model. An example is shown in Fig. 8.

Linking frames to ILOs. The interaction is seen by the student as a sequence of frames. but is regarded by the system (and hopefully the author) as a path among ILOs. In order for the author to see the way in which this mapping occurs during presentation, the debugging environment displays the actual frames and ILOs in a piano roll notation (Fig. 9).

As the interaction progresses, each frame is marked along the top of the piano roll. and the ILOs which it uses are blocked in the body of the diagram. Since the ILOs are ordered in a manner linked to their clustering, the net effect is to generate a trace which indicates how the ILOs have been used in this presentation sequence. The clustering means that discussions on a particular topic should be centred in a particular (vertical) segment of the piano roll. This trace is not passive, but allows the author to query it. Selecting a block in the body of the trace provides information about the role which a given IL0 played in selection of that frame (i.e. the values of the static and dynamic parameters at that point). Selecting an IL0 name gives information about how that IL0 has been taught (i.e. summary of materials used, percentage of presentation and diagnosis. current confidence in student knowledge). Selecting a frame provides an explanation of the way in which that frame was chosen, and details of the ‘next best’ alternatives,

Hot concept map. The way in which the concept map can be viewed and edited by the author has been discussed earlier. Since this structure is much used by the author, it is highly meaningful to her. It is sensible, therefore, to try and use the concept map to present information to the author in such a way that she can use it directly for editing. This inforrnation will be at a level of

Fig. 9. Piano roll model.

80 MARK T. ELSOM-COOK and CLAIRE E. O’MALLEY

abstraction from the actual presentation sequence, but can be made to present a number of relevant forms of information. Unlike the tools discussed earlier, the hot concept map requires a colour machine in order to operate in a clear manner, although an approximation to the behaviour can be achieved using grey scale.

As with the normal concept map, the hot concept map can be represented in any of the three views. As the interaction progresses, the IL0 which is the current explicit focus of the dialogue becomes the largest object on the screen. ILOs which are in implicit focus are intermediate in size between this and the unfocused items (see Fig. 10). Each node in the network is a circle divided into three slices (like a pie chart) with the name of the IL0 superimposed. The three slices represent the relevance of that IL0 to the current focus, the confidence that the student understands the ILO, and the fraction of available material for that IL0 which has been used. These segments vary in colour as the values of these parameters change.

In Fig. 10 for example, the current focus of interaction is Lisp. Model is as yet untaught (top right segment), is very relevant (bottom segment) and the system believes that the student knows something about it (top left segment). Model, Syntax and Evaluation are the three most closely connected concepts. ILOs such as Prolog are more distant.

If the author wishes to change connections, etc. in the structure, she can edit the representation directly as described in the section on the concept map (above).

Scripting. If an author designs a course and tests it, playing the part of the student by entering answers to questions etc. then a certain amount of work has been done to carry out that test. If the test proves unsatisfactory, the author will probably modify the information she has provided to the system. After modification it is likely that the author will wish to retest the course in a similar situation. To avoid repeating all the work of the previous test, ECAL offers a scripting facility. The script system can watch what a person does and record it. This sequence of actions can then be repeated-either a step at a time or as a batch of operations which return the author to a given point in the course. This facility enables an author to build up a set of virtual students on which to test the course before offering it to real students. ECAL provides two default scripts which respectively answer all questions correctly or incorrectly.

CONCLUSIONS

In this paper we have described a system intended to bridge the gap between CAL and ITS. It is essentially very simple, but clarifies a number of issues in the use of computers in education. At present there are approx. 7 implementations of the system by various groups. The most complete

Pascal Basic

Fig. 10. Hot concept map.

Bridging the gap between CAL and ITS 81

implementation is designed for an IBM-PC compatible machine, and can run with 500 K of memory.

Future plans for the system include using it as an implementation bridge between an advanced CAL authoring environment (AGD) and a tool for generating knowledge representations for ITSs (KRAS). This will extend ECAL to being a full multi-media system.

1. 2. 3.

4.

5. 6. 1.

Elsom-Cook, M. T., Overview of the ECAL system. Open University CITE Report 43 (1988). Elsom-Cook M. T., The ECAL presentation system. Open University CITE Report 89 (1988). O’Malley C. E., Elsom-Cook M. T. and Ridwan E., The ECAL authoring environment. In Proceedings of Third International Symposium on Computer and Information Science (ISCIS III), izmir, Turkey (1988). O’Malley C. E. and Sharples M., Tools for management and support of multiple constraints in a writer’s assistant. In People and Computers: Designing for Usability (Edited by Harrison M. D. and Monk A. F.). Cambridge University Press (1986). Spensley F., Dominie: the trainer interface. OU CITE Report 44 (1988). Spensley, F., Authoring issues for intelligent tutoring systems. OU CITE Report (1989). Elsom-Cook M. T.. Design considerations of an intelligent tutoring system for programming languages. Ph.D. thesis, University of Warwick (1984).

8. Posner G. J. and Rudnitsky A. N.. Curriculum Design. Longmans, London (1985).

REFERENCES