the clinical use of motivational messages: an inquiry into the validity of the arcs model of...

Instructional Science 19:467-500 (1990) 467 © Kluwer Academic Publishers, Dordreeht - Printed in the Netherlands

The clinical use of motivational messages: an inquiry into the validity of the ARCS model of motivational design

JAN VISSER 1 & JOHN M. KELLER 2 1 UNESCO, P.O.Box 4775, Harare, Zimbabwe. 2Department of Educational Research, Florida State University, Tallahassee, FL 32306, U.S.A.

Abstract. Instruction, even when prepared according to sound instructional design principles, often does not stimulate students' motivation to leam. A motivational intervention, developed according to the processes outlined in the ARCS model of motivatictaal design, was implemented and tested following the conventions of an embedded single-case study. It was tested with a group of 15 adults participating in a staff development course conducted within the Mozambique Ministry of Education. Results, based on concurrent and discriminant analysis c~ both quantitative and qualitative data, indi- catexi that the primar/treatment, motivational messages, had positive effects on student attitude and performance. It may be concluded that the specific intervention and the ARCS model which provided the basis for designing it provide a sound basis for positively influencing student motivation to learn.

Introduction

Designing instructionally sound courses and materials is but one step in the direction of solving learning problems. However excellent such instructional materials and educational programs may be, they will not be adcquatsly attended to by the learner unless he or she is motivated to do so. Consequently, a fundamental question is, "How do we motivate learners?" and more importantly still, "How do we keep them motivaWd?"

Over the past decades, much attention has been given to improving the ways in which instructional materials and processes arc designed and developed. A whole series of models has been devised to that effect (Andrews and Goodson, 1980). In these efforts, various learning domains have been stressed, often with a predomi- nant emphasis on cognitive skills, whether as an independent area, or as a support for learning in other domains. Relatively little attention, however, has been given to the motivational requirements of the learner. This area has been identified as a major omission (e.g. Keller, 1979; Reigeluth, 1979) and the desirability of addressing it has been stressed (e.g. Martin and Driseoll, 1984). However, in the practice of the instructional designer, motivational requirements of the learner are often at most a generic concern to which some general attention is given but for which the designer lacks the tools that would help to analyze this area and to devise strategies that respond to specific problems. Briggs (1980) observes in this context that "our theories or models of design do not take enough account of motivation, even though we may acknowledge that motivation effects are stronger than IxeaUnent effects" (p. 49).

468

In recent years there have been more concerted efforts to understand and influence student motivation (e.g. Brophy, 1983; Keller, 1979, 1983; Wlodkowsky, 1981), but a great deal of work is needed to operationalize and validate these approaches (Keller, 1987a). The intention of the present embedded single-case (Yin, 1984) exploratory study was to conlribute to this effort and even extend it by developing and testing a set of motivational strategies in a cross-cultural setting with employed adults enrolled in a professional skills development course.

There are many ways to document the power of personal motivation on individual performance. One need but read accounts like John Holt's (1978) Never

too late or explore the lives of people like Eduardo Chivambo Mondlane - an illiterate Mozambican herds boy until the age of 12, a Doctor of Philosophy of Northwestern University at the age of 39 and founder and first president of the Mozambican liberation movement FRELIMO at the age of 42 - to become con- vinced of how powerful a force motivation is in learning. Given motivation, many learners are able to perform learning tasks, even in totally inadequate instructional environments and in the absence of properly la'ained teachers or skillfully designed materials. The first author, in his career as an educator in developing countries, particularly in Africa, has been witness to innumerable examples of the above. It is probably justified to say that in many cases, if learning did take place - and indeed it did - it must have occurred despite the adverse instructional circumstances. On the other hand, particularly in the other one third of the world, are those cases where excellent instructional conditions surround the learner, but where, despite all this, no learning is accomplished.

The primary emphasis in instructional design has normally been on factors directly influencing learning effectiveness and not on factors that stimulate the motivation to learn. Nevertheless, there have been some important contributions. Improved instruction can result in higher success rates which contribute to improved motivation. Another result of instructional design is to get rid of what is not required to be learned, i.e., to prevent learners from becoming demotivated by having to attend to tasks that would be perceived as irrelevant. In other words, instructional design procedures help us enhance the relevance of instruction.

Also Mager's (Mager and McCann, 1961, Gagnt, 1965) classical intervention at Varian, Inc., as well as his subsequent work in this regard (Mager and Clark, 1963), clearly show that the motivation and direction resulting from a person's being made aware of what s/he should be able to do can be more important for learning than actual instruction. In fact, what Mager did was simply create Gagnt 's (1985) first two inslructional events, 'gaining attention' and 'informing learners of the objective', leaving it to the learner to identify/select appropriate resources and structure and complete the remaining part of the learning process. Given the availability of resource materials, motivation is not just a necessary condition for learning to take place, in cases like the above where no particular instruction was delivered, it may even be a sufficient one.

469

In the psychological literature, motivation has been linked to the principles of tension or need reduction in Freudian psychoanalytic theory and Hullian drive theory; to expectancy of goal attainment and the incentive value of the goal in Lewin's field theory, Atkinson's achievement motivation theory and Rotter's theory of social learning; and to the striving for self-understanding and self- realization within one's environment in attribution theory by Heider and Kelley as well as in humanistic psychology by Maslow, Rogers and AUport (Weiner, 1980). None of these theoretical approaches has been able to present a full picture of why humans, or organisms in general, behave as they do. They each highlight particular aspects of human behavior as seen from particular points of view. Weiner stresses that they are not commensurate and should therefore not be looked upon from a perspective of "hierarchical ordering of 'Truth'" (p. 6). They should rather be seen as complementary.

Considerations like the ones inherent in the above mentioned theoretical approaches have led others to devise ways of enhancing people's motivation, particularly with respect to the accomplishment of learning tasks. Two different conceptions can be distinguished here. One such conception looks upon motivation as a general state to be achieved prior to the implementation of learning tasks in order for learning to be efficient and effective. This leads to approaches such as conducting motivation workshops (McClelland and Steele, 1972) or executing motivational (metacognitive) ,,;kills training interventions (McCombs, 1983, 1984). In the other conception, motivation is seen as a continually changing set of factors influencing the individual's learning behavior before and during the learning task. Instructional interventions based on this latter conception (Keller, 1984, 1987a, b; Wlodkowski, 1981, 1985) are oriented towards strategies that aim at setting the motivational factors at appropriate levels during instruction for optimal learning. There is evidence that'both types of approaches result in positive effects. They should probably be seen as complementary rather than as pre-emptive. Both the pre-instructional motivational disposition and the control of motivational factors during instruction are impoltant for efficiency and effectiveness of learning.

The problem addressed in this paper pertained more to the latter conception of motivation: How can people be helped to overcome their motivational restraints in a given instructional context? In the particular case of this study, the people under consideration were adults and the instructional context refers to an inservice continuing education program of personal and professional development.

The two aforementioned approaches to address this problem show similarities as well as differences. Both Wlodkowski's (1981, 1985) time continuum model of motivation and Keller's (1983, 1984) ARCS model analyze the motivational requirements of the learner in terms of different components. Wlodkowski (1981) distinguishes between six different motivational factors, i.e. attributes, needs, stimulation, affect, competence and reinforcement, that influence the learning process in its different phases. These factors are presumed to influence motivation

470

in a fixed sequence and a model is provided which links the six factors to motivational strategies, from a variety of schools of motivation theory, that can be readily implemented by the instructor.

Keller (1984), on the other hand, provides a problem solving model which, first of all, has the instructional designer or instructor analyze the motivational requirements of the audience in terms of a more compact set of only four factors, i.e. attention (A), relevance (R), confidence (C) and satisfaction (S). Each of these factors encompasses a host of specific variables and applied strategies that assist designers and teachers in identifying and solving motivational problems. The components of each factor can be organized into several different types of subcat- egorization schemas (e.g. Keller, 1984, 1987a), but the most commonly used one (Table 1) has three subcategories and associated process questions (Keller, 1987a) for each of the four factors.

Unlike the assumption in Wlodkowski's time continuum model, these factors are not considered by Keller to be specifically linked to particular phases of the learning process. It is recognized that, although the emphasis may shift in the

Table 1. Motivational categories of the ARCS model

Categories and subcategories Process questions

Attention

A.I Perceptual arousal A.2 Inquiry arousal A.3 Variability •

Relevance

R.1 Goal orientation

R.2 Motive matching

R.3 Familiarity

Confidence

C.1 Learning requirements •

C.2 Success opportunities

C.3 Personal control

Satisfaction

S.1 Natural consequences

S.2 Positive consequences •

s.3 Equity

What can I do to capture the learner's interest? How can I stimulate an attitude of inquiry? How can I maintain the learner's attention?

How can I best meet my learners' needs? (Do I know their needs?) How and when can I provide my learners with appropriate choices, responsibilities and influences? How can I tie the instruction to the learner's experiences?

How can I assist in building a positive expectation for success? How will the learning experience support or enhance the students' beliefs in their competence? How will the leamers dearly know their success is based en their efforts and abilities?

How can I provide meaningful oppommities for learners to use their newly acquired knowledge/skill?

What will provide reinforcement to the learner's successes?

How can I assist the students in anchoring a positive feeling about their accomplishments?

471

course of instruction (e.g., more attention getting strategies at the beginning and more satisfaction producing strategies at the end), requirements relating to any one of the four motivational components may occur at any time, whence the need for a flexible response and a problem solving attitude on the part of the designer or instructor. Accordingly, the ARCS model includes a systematic approach to motivational design which follows a general problem solving and design process (Keller, 1987b). The process begins with an extensive audience analysis which forms the basis for identifying motivational objectives. These objectives guide the design, development and implementation of motivational strategies.

Although the ARCS model primarily addresses the motivational requirements of the learner in the context of a specific lesson or course of instruction, its flexibility allows one to take into account more long term motivational goals as well. As is pointed out by Bohlin (1987), the motivational dimensions specified by the ARCS model refer to internal states of the learner. This poses the problem that, unlike some of the factors in Wlodkowski's model, they cannot be manipulated directly. Instead, external events have to be created that prompt the learner into modifying his/her internal motivational disposition. Conceptually, the problem solving approach inherent in the application of the ARCS model is more complex than the algorithmic approach advocated by Wlodkowski. On the other hand, the modification of the internal motivational disposition of the learner, aimed at by the ARCS model, may be considered to touch on longer lasting effects, as the outcomes relate to changes the learner has brought about him or herself internally. Bohlin (1987) refers to the "adaptability to a wide variety of teaching methods and processes" (,p. 14) as the relative strength of the ARCS model in a cross- cultural comparison with methods of designing motivating instruction presented in the Soviet literature. He claims this to be of particular importance in the context of Western educational programs, where an "extreme variance in classroom processes is encouraged to accommodate a wide range of learning styles and to develop individuality" (p. 14).

The study reported on in this paper was carded out in Mozambique, a context far removed from both the Western and the Soviet instructional experiences, although, interestingly, influences from both these worlds can certainly be found in the educational framework of the country in question, a young nation which has gone through a period of armed slruggle before attaining its independence. It was the process of cultural and intellectual emancipation that accompanied the liberation struggle that helped shape future educational practice against the background of traditional and colonial influences, while taking into account educational experiences accumulated in the socialist as well as in the Western world.

The specific purpose of the present study was to test the application of a motivational intervention, the clinical use of motivational messages, based on the ARCS model, in the context of a training workshop carded out among profession- als employed by an educational R and D institute in the capital of Mozambique.

472

The term clinical in the above context is deliberately chosen in order to emphasize the problem solving nature of the approach, based on continuous diagnosis of the audience and subsequent formulation or reformulation of a solution. This is in contrast to the more typically applied approaches in which solutions are prede- fined and applied in a general way.

In the training environment referred to above, adverse motivational conditions provided an adequate setting for this study to be considered a critical case in the sense defined by Yin (1984, p. 42). It would allow a test of the theoretical propositions of the ARCS model in a situation where it would be difficult to attribute motivational effects to external events other than the application of the motivational intervention based on that model.

Situation

Contextual factors

There are several characteristics of the overall society that are also characteristic of the R and D Institute in which this study occurred. Their description helps put this study in perspective. The following brief summary of these societal characteristics defines the boundaries for its "analytical generalization" (Yin, 1984, p. 39). It also contributes to understanding the overall motivational climate surrounding the intervention.

Mozambique, situated on the East coast of Africa, attained independence from Portugal in 1975 after ten years of armed liberation struggle and approximately 500 years of colonial domination. A great many languages are spoken within the borders of Mozambique. For political reasons, related to the building of a unified nation, Portuguese was chosen as the official language, hence also as the manda- tory language of instruction at all levels.

Educationally the picture of Mozambique was rather grim immediately after independence: 93% of the population was illiterate in 1975. Although no up-to- date statistics are available (the latest census was in 1980), it is believed that presently between 70% and 75% of the population is still illiterate. The 3.6 million people in the age range of 7 to 19 years old have to be attended to by approximately 25000 teachers. Of these, 80% are elementary school teachers for grades I to 4, more than haft of whom are virtually untrained and possess barely more knowledge than they are supposed to teach.

The political and development options taken by Mozambique attribute high priority to the interests of society at large as compared to the interests of the individual. Most people working in education would have chosen other careers had they been given the option. As in many other countries the social status of the teaching profession is not particularly high and teachers have to work under

473

extremely difficult circumstances (high student-teacher ratio, almost non-existent material conditions, lack of adequate training). Furthermore, the demands of the work are much greater than can possibly be met by a handful of people.

In addition, the country is economically in a bad shape due to military destabil- ization by externally supported groups of armed bandits and as a consequence of

a number of natural disasters. Elementary questions of survival - food, clothing,

housing, transport, etc. - are foremost in the minds of the people, including the subjects of this study.

The post-independence efforts to bring about change have resulted, among other things, in the implementation of a new National System of Education for which new curricula are being developed, teachers are being trained, retrained and upgraded (partly by means of distance education) and instructional materials are being produced. The R and D institute which provided the setting for our study,plays a vital role in guiding the process of introducing the new system, particularly as regards the development of curricula and accompanying instructional materials and in developing alternative means for the delivery of instruction.

Course characteristics

The motivational intervention tested in this study was inserted in a course on systematic instructional design taught within the R and D Institute. The aim of the course was to develop competence in the systematic design of instructional materials among relevant staff and to instil favorable attitudes towards the adoption of this new approach which was to replace the intuitive way in which materials had heretofore been developed. The design model used in this course was pilot tested approximately one year earlier for a distance education audience served by the institute (Visser, 1985). Two of the institute's staff participated in this study.

The instructional system design course consisted of an intensive ten-week term of study. During that period, two weekly sessions of three hours each were conducted, partly in an instructor-led lecture format and partly in a facilitator- conducted workshop fashion with a strong emphasis on the latter mode of delivery. The total of 20 course sessions was preceded by an introductory session held approximately two weeks before the course began. Besides, participants had structured access to the facilitator during an additional weekly period of half a day if they wished to receive further assistance. All sessions were conducted at the institute during working hours.

Course requirements

In addition to their attendance at the 20 course sessions, the participants were expected to engage in the following activities:

474

• Autonomous study, either individually or in a self-organized study group, of a self- instructional text (Dick and Carey, 1979 and 1985) in accordance with a given plan, which included rigorous evaluation deadlines. This text was supplied to all subjects in Spanish. Reference copies in English were available.

• Individual and/or group study of course handouts (in Portuguese), complementary to the above text.

• Active participation in discussing, in different group settings, of relevant content matter previously studied.

• Participation, either individually or in group settings, in activities aimed at getting experience in applying knowledge related to the area of study involved.

• Participation in three test sessions that evaluated their mastery of content matter related to the above activities.

• Completion by each individual of a systematic instructional design project resulting in a suitably validated instructional package on a topic of the participant's choice and completed in accordance with set deadlines and given evaluative criteria.

• Oral presentation and discussion of the results of the individual instructional design projects.

• Preparation by each person and presentation in two different installments, of a full written report of the individual instructional design project.

• Participation in formative evaluation of the course.

It was also made clear to the participants that they would be expected to dedicate themselves, over and above their presence in the course sessions, to at least ten hours per week of independent study and project assignments. This turned out to have been a very low estimate. Most subjects had to spend about double that time on their performance of the required tasks.

Motivational conditions

Completion by the participants of the above tasks was expected to be influenced by a variety of conditions that would directly and indirectly affect their motivation. These conditions were identified by systematically reviewing the various motivational factors contained in the ARCS model and identifying positive and negative aspects of the psychological, social and physical environment of the audience. The description given below was based on the first author's knowledge of the instructional environment, rooted in more than twelve years of working experience in the education and lraining sector in Mozambique. An instructor without that pre-knowledge of the situation would have had to engage in systematic collection of data from the students and persons who are familiar with the

475

instructional environment. Such data are then interpreted in terms of the four dimensions of the ARCS model.

It should be noted that motivational conditions and the way they are perceived by the learners are assumed in this study to change and particularly also to be changed as a consequence of systematic motivational intervention. Continuous reassessment of these conditions is therefore necessary right from the beginning, and at regular intervals during, the instructional intervention. This process, desig- nated 'Audience Analysis', will be dealt with in the Method section of this paper.

Following is a summary of the most salient motivational conditions identified:

1) Most learners had not themselves chosen to participate in the course.

2) Most learners were expected to engage in the learning task not out of individual commitment, but rather on the basis of the existing tendency to tacitly accept tasks because society expected them to do so.

3) No material reward whatsoever was attached to completing the course successfully.

4) Remuneration in the work environment in which the training was carded was among the lowest in the world for the professional category in question.

5) While there was some potential for ideological motivation among the participants, an l 1-year post-independence period with few concrete signs of achieved progress limited the possibilities ,to motivate individuals on ideological grounds.

6) Difficult material circumstances, severely affecting participants' quality of life, was expected to easily divert the learners' attention from their course assignments.

7) Learners were expected to perceive the defined results and outcomes of the course as highly demanding in comparison with their past exposure to training interventions.

8) Required student behavior' during the course contradicted established, though not approved, behavior on the job, characterized by lack of punctual- ity and assiduity.

9) Required learner behavior would clash with personal and professional obligations.

10) The prospect of change in job behavior, an inherent corollary of attaining the training objectives, would produce feelings of uneasiness in the learners.

11) Some learning experiences, particularly as regards formative evaluation of the product of the design effort, were bound to be painful and initially frustrating.

12) There was a possible threat to authority for those participants, in directive positions, who were participating in the course on an equal basis with their subordinates.

476

13) Studying from materials in Spanish, an unknown language with only a certain degree of similarity with the official language, Portuguese, constituted an important obstacle.

14) Participants would be required to be highly inventive and creative in finding solutions to organizational problems related to their design assignment.

In summary, the above statements regarding the motivational analysis of the audience should be seen as indicative, rather than as exhaustive. They will, however, convince the reader that a motivational intervention was in order in the given context and that without such an intervention the course might well have failed completely.

Approach

Because of its flexibility and its problem solving characteristics, the ARCS model was used in the above context, to design and develop an intervention, i.e., the clinical use of motivational messages, to help adult learners in an inservice continuing education program of personal and professional development attain adequate levels of motivation with respect to the four components specified by the model. The design process outlined in the ARCS model (1987b) was used to diagnose specific motivational problems and prepare motivational interventions in the form of written messages, usually in the format of a greeting card with an attention-getting cover and a written message on the inside. These messages were be distributed in the work place during the interval between weekly class sessions and pertained, directly or indirectly, to the tasks or conditions surrounding the tasks assigned at that time.

In doing so, it was considered that the successful design and development of the intervention would not only contribute to the solution of existing problems of severe motivational restraints within the referred context, but that it would, at the same time, lend further support to the validity of the model on which the innova- tion was based. As a further point of interest it was considered that such a model validation would broaden the perspective of the applicability of ARCS, taking into account that the socio-cultural setting chosen for the appiication of the intervention would be fundamentally different from the one in which the ARCS model was developed. While working on the development of the intervention, motivational requirements of the learner, both as an individual and as a group member, were addressed. This latter area has so far received little attention in instructional design theory, as well as in related practice (Martin and Driscoll, 1984). However, audience motivational analysis is an important step in the ARCS model and is specifically referred to by Bohlin (1987), in his cross-cultural comparison, as a field deserving more attention in Western models.

477

Research questions

A case study is the study of a bounded system in its own habitat (Stake, 1988). The questions underlying case studies are therefore distinct from the questions that are asked in research settings where control over events is a methodological requirement. Furthermore, in conceptualizing a case study it is recognized that "the boundaries between phenomenon and context are not clearly evident" (Yin, 1984, p. 23). Considering the real-life context of the issues under scrutiny in a case study, as well as the relatively loose distinction between these issues and their habitat, it is not uncommon and certainly in keeping with the requirement that the 'wholeness' of the case be preserved, that the perspective of inquiry changes while a case study is being carded out. While emphasizing the unity of the bounded system, attention should be confined "to those aspects that are relevant to the research problem at the time" (Stake, 1988, p. 258).

At the outset of the present study, the basic research question was a develop- mental one. Would it be possible to develop a strategy that would help learners in the adverse motivational circumstances described above to overcome such restraints? If such a strategy could indeed be developed, what would be its most efficient and effective form and how could it best be applied?

Once developed and while still subject to changes that would allow the strategy to improve gradually, the research concern regarding the use of the motivational messages shifted to collecting specific evidence that could support the proposition that the strategy is effective and that could be used formatively in the process of enhancing it.

More specifically, the intervention was expected to work against the tendency of participants to drop out from the course and to procrastinate. It should also help people in making correct decisions about the adequate allocation of time and attention to the performance of assigned tasks, the degree of difficulty of which would be perceived as different by different subjects and to engage in the active procurement and use of resources for that purpose. High levels of productivity and positive affect should be important dimensions of the overall result.

The exploration of the above questions and concerns, as well as of the validity of the stated propositions was inserted in an embedded single-case research design (Yin, 1984), having as its research object the interaction of the motivational messages strategy to the motivational behavior of the aforementioned group of participants in an instructional design skills training course. Outcomes of the application of the motivational intervention were studied at the level of individual motivation as well as at that of the group as a whole and with respect to its consequences in instructional attainment. The intervention, as described below, was multifaceted in nature, but consisted primarily of the application of a set of motivational diagnostic activities, the results of which were analyzed in terms of the dimensions of the ARCS model, followed by motivational feedback delivered

478

in a variety of formats, the most common of which was a written motivational message. The activities of the intervention and of the administration of the entire course in which it occurred were documented and data were collected to assist in forming inferences as to the effectiveness of the intervention and hence the validity of its underlying model of analysis, the ARCS model.

Method

Subjects

The persons used in this study were 15 adults participating in the instructional design course mentioned earlier. All participants were members of the staff of one autonomous R and D institution within the national Ministry of Education, responsible for curriculum development and instructional materials production for a variety of subject areas and at different levels of a recently introduced new National System of Education. Their levels of professional responsibility ranged from Director of the Institute and Heads of Department, to that of producer of instructional printware, radio programs and/or materials for other AV-media.

Participants had been selected from among the total staff of the institute, primarily on the basis of the expected positive influence of the training on their work, as perceived by their superiors. With the exception of four participants who held directive posts, the other participants would not necessarily share this perception. Their selection to participate in the course, as well as their original appointment to the staff of the institute, was done primarily on the basis of national interest rather than personal choice.

Considering these conditions it seems justified to claim a certain representa- tiveness, albeit not in statistical terms, of this group as regards what Stake (1986, 1988) calls 'naturalistic generalizations' toward audiences whose learning requirements and learning engagements are not derived from self-chosen professional options. Involuntary audiences such as these present interesting motivational challenges to the instructional designer and the instructor.

An analysis of personal data regarding the participants, collected by means of self-report, showed a positively skewed age distribution with M = 31.3 yrs., S.D. = 7.0 yrs. According to the same data the number of years of professional schooling - generally after grade 9, in two cases after grade 11 - again was a positively skewed distribution with M = 4.1 yrs and S.D. = 2.4 yrs. The majority were trained as lower secondary or elementary school teachers. Four participants held university degrees at the BAfBSc level, one had the equivalent of an MA. Their content specializations ranged over a wide variety of areas of almost all regular school subjects, including such areas as civics and political/ideological training. About half of them spoke Portuguese, the official language, as their mother

479

tongue, the other half had a non-Portuguese mother tongue. Roughly one third were born in an urban environment, two thirds in a rural or semi-rural environment. They were all of Mozambican nationality, one having been naturalized. Nine subjects had not participated in training activities, in which set performance standards had to be reached, for periods of two to five years, three had not done so for between five and ten years and three for more than ten years.

Motivational intervention

Considering the audience motivational conditions as analyzed earlier, it was concluded that it would be wise to keep track of the motivational requirements of the learners while they were participating in the course. It was assumed that motivational requirements would change over time, as well as across learners, as a function of factors both external and internal with respect to the course.

Apart from keeping track of the learners' motivational requirements, a way would have to be found to respond to these requirements in a flexible manner, so as to be able to address each learner's needs at the most appropriate moment. As the most important part of the learning was expected to take place in the absence of the instructor/facilitator, it was considered that often the learner's perceived motivational needs would be highest exactly when the instructor/facilitator would not be there and the learner had to do it all alone. Also, motivation being an internal state of the learner and not something the instructor/facilitator could accomplish, the intervention should be an unobtrusive one, leaving it to the learner to create his or her own motivational conditions.

The above considerations led to the idea of the clinical use of motivational messages, i.e. simple, written messages that could be distributed to the learner at the moment when he or she was most likely to need such an extra impulse. The content of the messages and the timing of their distribution, would be derived on the basis of the following inputs:

1) The student's progress as regards the different tasks that were required to be performed. It could be foreseen what type of particular difficulties a learner might face when working on his or her autonomous study assignments, as contrasted, for instance, to the feelings of uneasiness that would most likely accompany the need to make an early contact with representatives of the target audience at a moment when most of the instructional design work would still have to be done, or to his or her frustrations when attempting to carry out an instructional 'analysis, or when finding a meticulously elabo- rated first draft of an instructional package torn to pieces in a formative evaluation.

2) Anonymous self-report by the student, on a periodical basis, as regards his or her feelings towards progress made in carrying out the different assignments

480

as well as regarding the course in general. To that effect, students would respond to an open-ended questionnaire distributed weekly at the beginning of each second session. In this questionnaire the student was asked to give a spontaneous reaction to the following three questions:

- What were, in order of importance, the three factors that, during the past week, most severely damaged your motivation while carrying out your assignments?

- And what were, in order of importance, the factors that had the most positive influence on your motivation?

- What, if anything, would most urgently require attention at this moment in order to improve the quality of the course?

3) Spontaneous reactions of the student concerning his or her progress as regards the various assignments as well as information spontaneously provided by the student regarding motivational aspects. Both positive and negative factors, internal as well as external to the course environment, were taken into account.

4) Comments solicited by the facilitator in follow-up contacts with the students, made whenever appropriate, in order to clarify aspects still unclear after analysis of the data from any of the above mentioned inputs.

5) Evaluation results regarding the student's performance on the different tasks.

These various inputs would be analyzed and interpreted in terms of the ARCS model, identifying learners' motivational requirements as regards the four dimensions specified in the model, i.e. attention, relevance, confidence and satisfaction. These aspects would be related both to the learner as an individual and to his or her being a member of the group of participants in the course, as well as to his or her potential membership of the larger community of instructional designers in general.

The analysis was carried out at least once a week, following the collection of data by means of the questionnaire. However, particularly in periods when it was expected that developments would be more rapid and more varied across learners, a more frequent analysis was made. Data pertaining to the various inputs would be considered collectively, so that, whenever possible, the motivational messages would respond, in a balanced way, to the entire range of factors that required attention at a given moment. This process resulted in messages being given out at an average rate of at least two a week. They would be timed to coincide with the moments when the learner most needed them.

In order for the motivational messages to work effectively they were never referred to by that name and used in as casual a way as possible. It was important that they were perceived as something entirely natural in the process of communi- cation between the instructor and the learners.

481

Audience analysis

Earlier in this paper, under 'Situation', a description was given of the motivational conditions surrounding the students' participation in the course. That description provides the overall framework within which the audience analysis referred to below was carded out.

In line with the findings of arousal theory, indications of the level of motivation in terms of the four different dimensions identified in the ARCS model can be thought of as points on an inverted U-curve (Keller, 1987b), relating performance and arousal. For each dimension there is an optimum which should be sought, in order to attain the best possible learning performance. It is quite possible and perhaps even likely, that values of arousal with respect to attention (A), relevance (R), confidence (C) and satisfaction (S) would vary considerably across learners at a particular moment of instruction. Figure 1 shows an analysis of the four dimensions in question at the beginning of the course which provided the setting for the present study, comparing two groups of learners: two students who had participated in the try-out of the systematic instructional design model the previous year (the insiders) and the remaining 13 students who had not had that opportunity (the outsiders).

In the present case, the information presented in Figure 1 was hypothetical at the start, based on logical analysis conducted by the first author, using his already existing extensive knowledge of the student population and the circumstances of their participation. In settings where the researcher or practitioner lacks this first- hand knowledge, the ARCS model can be used as a basis for collecting data from the students and others who have personal knowledge of them (Bickford, 1988) and for interpreting those data in terms of the four ARCS dimensions. In either case, continued data collection will refine and further validate the initial perception of the audience.

Insiders Outsiders

~'~ high c ~ S 2A high R A" '~ ' ' " ' '~2f ! ! " ~ s A

low R1 low I Rfr : : t low high low high

Motivational Arousal Figure 1. Mcgivational profiles of insiders and outsiders

482

Whether the audience analysis as initially conducted has provided a correct representation of the audience's motivational situation can be found out by looking at the results of motivational interventions based upon it. For instance, if on the basis of the analysis of the available data, it were concluded that a section of the audience should be expected to be underattentive and an intervention were applied aiming at improving their levels of attentiveness, the result should be a more adequate disposition to learn among that section of the audience.

Should it have been predicted that members of the audience were attributing too high a degree of relevance to what they were learning, causing them to be overanxious in their attempts to obtain good learning results, it should be expected that their learning performance would improve if a motivational intervention were applied that demonstrated the relativity of the importance of what they were learning and to the fact that not everything would be lost should they not immediately obtain the required learning results.

In the analysis reported on here, established and validated on the basis of the procedures outlined above, all learners were seen to be underconfident at the start, though a somewhat higher degree of confidence was atlributed to the insiders, in view of their familiarity with at least a number of external aspects of the approach to be learned.

Some aspects regarding relevance were the same for both groups of learners, other aspects were different. In Figure 1, this bi-polar situation is represented by two points, R1 and R2, on each of the curves. Goal, or ends, orienled relevance (see Table 1, R.1), such as expectations regarding possibilities for career development as a result of the course, is considered to have been at an approximately equally low level for both groups of learners. This aspect of relevance is therefore represented by a point R1 in the same position on both curves in Figure 1. Motive matching, or process oriented relevance (see Table 1, R.2), on the other hand, was analyzed as being on entirely different levels for the two groups: those who had accompanied the try-out and had witnessed its dramatic results, might have over- estimated the value of the approach, as they had experienced little regarding the tremendous effort involved in attaining such results; the outsiders must at that instance still have been moderate non-believers, who would yet have to see what truth there was in the propagandistic words of the insiders. This results in entirely different positions of R2 on the curves for the two groups of learners in Figure 1.

Similarly, the noted difference in attention level can be explained. Through their enthusiasm for the known results of the new approach, the insiders were likely to be somewhat hyperactive in their attention. By being overeager and trying to pay attention to everything they would have difficulty focusing their attention on the central topic. In contrast, the outsiders would still play a waiting game and be somewhat underattentive.

483

Finally, to be in the course could be considered to be potentially satisfying to the insiders, because they had seen the positive outcomes of the earlier course. Ultimately, their actual satisfaction with the course would depend upon the appropriate use of motivational feedback. The outsiders, in contrast, could be expected to have negative satisfaction potential (e.g. "Even if I succeed at this, I 'm still not going to like it.") given that most of them had no choice in their selection and would not view the course as a needed intervention in their busy lives.

Motivational objectives

The aim of applying the motivational messages strategy was to prompt participants into acquiring and maintaining levels of motivation, along the dimensions given above, that would optimize their learning. Ideally, their levels of attending to the learning tasks, of perceiving the relevance of what they were studying and what they were preparing themselves for, of developing confidence in their capa- bility to complete the study assignments and to apply the skills they were being trained on in the context of their daily work and of feeling satisfied with their engagement in this learning experience should all be on or near the top of the inverted u-curves for the various motivational dimensions mentioned before. Below follows a description of what this would mean in the context of the learning experience the course participants were supposed to go through.

Through the intervention, learners should be led to view the total learning task as a finite set of portions, each of which they would be able to engage in and complete successfully, provided an adequate study effort would be put into it. As the extent of this study effort was for most learners normally a considerable one, they should not confide in their ability to cope with the learning tasks independently of such an effort. They should discover that they would be able to access and organize all necessary resources required for a successful instructional design effort and that the learning experience they were going through was transferable to their professional field of activity.

~ a m e r s should look upon systematic instructional design as a sound alternative to their regular practice of designing instructional materials, worth investing the learning effort in, but without perceiving it as a panacea, or as something without which they would not be able to survive professionally. Though there were few prospects for short-term career improvement as a consequence of their taking part in the course, participants could be led to perceive their successful completion of the course as a relevant stepping stone in a long-term perspective of career and personal development.

An adequate level of attention would be reached if learners would daily be spending time, individually and collectively, on the execution of given assignments, if they-would attend class sessions, being properly prepared and participating in an active manner, if they would not be postponing their course work

484

to moments close to the given deadlines and if their attention would not be diverted from current tasks to concerns regarding the end result of the course. Trying to reflect on what they had learned in the context of their regular tasks at the institute would be seen as another strong indicator of adequate attention.

The motivational intervention should result in learners becoming aware that completion of difficult tasks can as such be a gratifying experience, particularly when these tasks are perceived as being relevant. It should furthermore lead them to derive satisfaction from their becoming integrated in a recognized professional field. Perceived recognition from their peers, supervisors and the instructor for their learning accomplishments, as well as feelings of togetherness with colleagues who were going through similarly difficult learning experiences, are other aims of the intervention.

Motivational design (motivational strategies)

The 20 regular work sessions of the course were preceded by an introductory one, two weeks earlier. During that session the selected participants were made aware of the implications to them of their participating in the course. Procedures of the course, as well as the meaning to the learners of what they would study, were carefully explained and discussed. Much attention was paid in that context to participants' obligations as students. It was made clear to them that these would interfere heavily, not only with their professional duties, but also with their personal life. They were then given the choice to either withdraw themselves from the course, or to sign a contract, committing themselves to fulfill their obligations as a learner. If they so wished, they were given a few days to decide. It should be noted that the same contract was also signed by the instructor, who would bind himself regarding the fulfillment of specified instructional duties, guidance and facilitation, as well as availability for consultation outside the class periods.

On the same occasion all selected participants received the initial set of instructional materials, including the self-instructional text, which they were told they could keep for future reference if they completed the course successfully. The reinforcement value of this condition may become clear if it is known that in the given context virtually no reading material, let alone adequate academic textbooks, can be obtained by the general public. Besides the textbook, the participating subjects also received a Gantt chart, indicating the sequencing of their learning tasks over time and deadlines by which various course products were due.

The introductory session to the course was planned to put the initial motivation of all learners within the desired range of arousal, both regarding the learners' expectancy of goals to be attained and with respect to the value attributed by them to those goals. Another goal of this session and the associated contract was to create positive satisfaction potential. In the graphs of Figure 1 this range is marked by the two dashed vertical lines.

485

Signing the contract would get people committed to attaining the overall goal, i.e. it would create the opportunity for them to provide themselves with their own competency related relevance. At the same time, this being a group act, it would give them the satisfaction of having publicly been granted and of having con- sciously accepted, the privilege of becoming a member of a group destined to make a difference within their working environment. Besides this perceived enhancement of their professional status, also the book offer would contribute to their motivation in terms of ends oriented relevance and expected satisfaction.

The fact that the contract was also signed by the instructor/facilitator, expressing a commitment on his part to be available to help the learners both in and outside the programmed learning environment, should have contributed to giving them a greater feeling of confidence, as well as to having strengthened their perception of professional and personal togetherness.

Furthermore, explanations and discussion in the introductory session should have contributed to enhancing the process oriented relevance of the course for the outsiders and to providing a more realistic view in this regard to the insiders, bringing for both groups of learners this dimension of motivation more in range of the desired optimum. As a matter of fact, feelings regarding the relevance of the course content would become more substantial during the training intervention, particularly as a function of the progressive discovery that learning would contribute to competence in areas participants had hitherto had problems with.

Finally, regarding the attention dimension of motivation, the distribution of the Gantt chart, which defines activities to be undertaken and the time available for each of them, would relocate attention for both groups of learners at appropriate levels of inquiry arousal. Handing out the Gantt chart also provided the learners with an opportunity to define intermediate goals and to divide the learning task into acceptable portions, which should have raised their confidence level as well.

After the introductory session, the learners' motivation was monitored on a weekly basis, using the anonymous open-ended questionnaires and the other indi- cators of motivational needs. Information derived from this monitoring process was interpreted along lines similar to the ones that underlie the development of the above rationale. On the basis of this interpretation, motivational messages were designed and distributed to the participants to help them overcome expected motivational restraints at the moment when they would most need such help.

The format of the motivational message most frequently used was that of a computer produced greeting card, normally including some kind of attention get- ter on the outside and a short statement, varying in length between 10 and 200 words, on the inside. Other formats, such as that of a letter, were also used.

As the objective was to reach the learner at the most appropriate moment, the distribution of the motivational messages was sometimes delayed for some learners relative to others. Occasionally, also, messages were personalized, or distributed to only part of the audience and some individual messages were designed. Usually messages were handed to the participants personally, but often

486

they were distributed anonymously, or the individual would simply find the message on his or her desk when coming to work.

Following the contract, which can be looked upon as an interactive motivational message that addressed needs regarding goal orientation and personal control, 22 more motivational messages were designed and distributed during the ten week workshop. Using the categorization of Table 1, Table 2 gives an overview of the different messages distributed, their format and the motivational factors they addressed.

It should be noted that the process of arriving at a particular message is partly analytical and partly heuristic. The ARCS model is used to analyze, on the basis of the regularly collected data, the motivational requirements of the learners in terms of the four dimensions of the model. However, knowing what these requirements are, there is normally not just one solution to satisfy them. This is where creativeness and sometimes luck, comes in. In fact, an almost infinite number of different messages could be written to address a given situation. What comes out of it will depend, to an extent, on personal judgement. However, whatever the result of this heuristic step, the designer of the message will not deliver it before s/he has verified that its contents and meaning are in accordance with the result of the ARCS analysis. To exemplify this, one case will be discussed now.

Getting to grips with the process of instructional analysis was found to be something very difficult for the learners. At the time they were learning to carry it out, it was difficult for them to see that the considerable effort would be worth- while. A message was designed, presenting a real case in which the application of instructional analysis had resulted in dramatic learning improvements and it was suggested that students would be able to attain similar improvements in their instructional design work and thereby gain professional prestige. However, at the time when the message was planned to be delivered, president Samora Machel had died in a plane crash, changing completely the motivational context. An entirely different message had to be designed to address the new situation.

While the above example is an extreme one, resulting in the replacement of a message, normally the process of checking a message's content against the ARCS analysis would result in adjuslment, rather than substitution.

None of the motivational messages (Table 2) addressed the attention subcate- gory A3, variability. However, this motivational factor was an overall concern in designing the various messages. It should therefore be looked upon as a characteristic of the complete set of them, rather than as a feature of any particular message. A variety of approaches was used to maintain curiosity and interest. At the same time, a feature was added to provide a sense of consistency. Each message, in some way or another, would always carry the logo ESMI, derived from the initials of the Portuguese translation for Systematic Design of Instructional Materials. An unobtrusive graphic pattern was used that would make the messages easily identifiable, as well as easy to identify with.

487

Table 2 (part 1). Motivational messages: their content, format and motivational categories and subeategories addressed

Content Format Attention Relevance Confidence Satisfaction

AI A2 A3 R1 R2 R3 C1 C2 C3 SI $2 $3

Alert first test is due; announce- greeting X ment oppommity for discussion card of content matter concerned. (general)

Feedback first test; analysis test results & their interpretation; indication about what next; encouragement to those who did less well, indicating possible strategies for improvement; intermediate goal setting; alert for next assignment due.

Expression of grief at the death of President Samora Machel in plane crash & encouragement to persevere.

Presentation of real ease showing dramatic results of instructional analysis & allusion that learner will be able to attain similar results by applying systems approach and thereby gain professional prestige.

Feedback second test; indication of intermediate goals attained & their significance; attention to social integration in professional environment; indication of areas to be attended to; further intermediate goal setting; alert for assignments due; offer availability facilitator for individual help by phone or appointment.

Appeal to arrive in time for class; reference to responsibility student in social context of class.

Summary grand idea behind last third of autonomous study assignment; indication of aspects to pay particular attention to; reiteration offer students may phone-in for further help.

letter (per- X sonalized)

greeting X card

(general)

greeting card

(general)

letter (personalized)

mini-poster (general)

letter (general)

X X

X X X X X

X X

X X X X

X X

X X X X X X X X

X

X X X X X

488

Table 2 (part 2). Motivational messages: their content, format and motivational categories and subeategories addressed


A1 A2 A3 R1 R2 R3 C1 C2 C3 S1 $2 S3

Feedback third test; identification of areas that need further attention; student responsibility to draw own conclusions; reference to tragic external events (death president); appreciation of students dedication under circumstances; fuaher intermediate goal setting; perseverance strategies; encouragement.

Reminder to be in time for class; reference to student responsibility with respect to colleagues.

letter(per- X X X X X X X sonalized)

greeting card

(limited indiv. distr.)

Announcement social event mark- greeting hag completion first haft of card course. (general)

Advice regarding preparation first part of project report; reiteration phone-in possibility for help; encouragement; contextual interpretation of difficulties encountered.

greeting card

(general)

Mid-course evaluation of results greeting obtained; student to draw own card conclusions; indication of areas (personal that require attention; commit- hand- ment facilitator to provide help; written reference to feedback given message) regarding first part of report; encouragement,

Presentation of newspaper clip- greeting ping interpretable in terms of card acquired knowledge (general)

Presentation of real world ease of application of acquired knowledge with positive success; inter- pretafion; importance of approach for professional working in field of education.

greeting card

(general)

X X

X X

X X X X

X X X X X X X X X

X X X X

X X X X X X X X

489

Table 2 (part 3). Motivational messages: their content, format and motivational categories and subcategories addressed


A1 A2 A3 R1 R2 R3 C1 C2 C3 S1 $2 $3

Announcement new deadlines for assignments

mini-poster X (general)

Three quarters of course corn- letter pleted; need to participate (general) actively continues to be fundamental.

Alert regarding frustrations expected to follow formative evaluation; importance leamer feedback; teacher no longer ultimate authority; guidance as regards canying out formative evaluation; need to develop capacity to correct oneself.

Further guidance formative evaluation; implicitly made clear that student now masters the .approach; reference to future applications by student as profes- siunal instructional designer.

greeting card

(general)

greeting card

(general)

"Viva ESMI" (ESMI = mini-poster X Systematic Design of (graphic- Instructional Materials). ally person-

alized)

Announcement final session; greeting show of inmrest on part of facili- card tator as regards final products; (general) expectations as regards contributions students in final session.

Seasons greetings (several weeks after completion course, coincid- ing with end of year); reference to new professional status.

greeting X card

(general)

Final feedback; quantitative eval- greeting uation of student's accomplish- card (co- ment& qualitative appraisal of signed by student's potential and director participation, inst. &

stamped)

X

X

X

X X

X

X

X

X X

X

X

X

X

X

X X X

X

X

X X

X X

490

Measures

Multiple sources of evidence were used in collecting data about the effect of the ARCS-based motivational messages slrategy. The following ways of measuring these effects were used:

1) The weekly questionnaires, designed in the first place to measure participants' motivational requirements with respect to future learning tasks, could also be used to determine what influenced students' motivation in relation to past learning events. They therefore provided indications regarding the role played by the motivational messages.

2) At the end of the course, all participants were asked to respond anonymously to three questionnaires.

a) The first of these questionnaires asked the participants to rate on a 5-point scale the perceived importance of each of the 15 components of the instructional strategy of the course. Among these components, the use of the motivational messages was mentioned explicitly, as well as related components like the provision of feedback by the facilitator (basically by means of the motivational messages) and the participants' providing feedback to the facilitator through the weekly questionnaires. Besides, the respondents were encouraged to comment on each of the instructional strategy components, as well as on the instructional strategy as a whole.

b) In an identical way, in a second questionnaire, students made an evaluation of their perceived importance of each of the seven media used in the course, as well as of the media-mix as a whole. The motivational messages were mentioned explicitly as one of the items to be evaluated. Related media, like the facilitator, the Gantt chart and the contract signed at the beginning of the course, were also mentioned.

c) In the third questionnaire, participants were asked to list in order of importance the five most important things they had learned during the course, as well as five aspects they would think required attention as regards possible improvement of the instructiorml intervention. 3)

In collaboration with an outside evaluator, an open-ended round table discussion, lasting 65 minutes, was conducted and taped. All subjects participated. They were encouraged to comment on any aspect of the course.

4) During the course, both spontaneous and solicited remarks made by students were registered.

5) Note was taken also of immediate, observable effects of particular messages, such as the use of the possibility to phone-in or see the facilitator for specific help, or the students' completing assignments in accordance with sugges- tions provided in the motivational messages.

491

6) Participants were also observed to determine how they would react to the medium as such. Would they read the messages? Would they throw them away after having read them? If they kept them, would they treat them with the same care as the more traditional instructional materials? Would they indicate that they were looking forward to any following messages? Would they be seen discussing them with their colleagues?

7) The instructional outcome of the course, as well as the students' dedication to attaining the course goals, would certainly also be attributable to the motivational intervention. However, this measure is less exclusively dependent on the use of the motivational messages.

Procedure

During the introductory session participants were told that they would be required to participate in activities aimed at constantly improving the quality of the instruction. As part of that effort they would have to fall out questionnaires and interact with the instructor regarding their study behavior. By signing their contract they had accepted this condition. They were not made aware that the information they were providing would be used concurrently for formal research purposes, nor that the course they were participating in was part of a research effort.

The data collected in the weekly questionnaires were anonymous. To further assure respondents in that regard, questionnaires were distributed, completed and collected in a way that would allow participants to convince themselves that ano- nymity of their responses was ensured. They were also regularly given feedback on the results of the overall analysis of the questionnaires and changes in instructional strategy decisions derived from the analysis results were discussed with them during class sessions. This was done not only for obvious pedagogical purposes, but also to make clear that their frequent contributions to the process of data collection were indeed used for the stated purpose of course improvement.

The three end-of-course questionnaires, as well as the round table discussion, were handled in a similar fashion, the stated purpose of their use being the improvement of the course should it be offered to future audiences. The presence of the outside evaluator in the round table discussion should have been perceived as natural, as he had participated in a number of previous course activities and his services as a subject matter expert (in biology) and as a methodological advisor (regarding systematic instructional design) had been used by various students.

As one of the researchers (the first author) performed the role of instructor in the teaching-learning environment in which the motivational intervention was applied, observations were made from a participant point of view. In order not to reveal his role as participant-observer, the inslructor did not record observations in the presence of the subjects whose motivational behavior was being studied.

492

Results and discussion

An accessible, extensive, data base, available for inspection and purposes of further research, was built in the process of carrying out the inquiry. The results presented below are grounded in the evidence available in the data base.

Both on the basis of observation and as substantiated by report and self-report, the motivational intervention is considered to have been successful. The results of the course (Measure 7 above) surpassed by far the initial expectations of the directorate of the institute, as well as those of the participants and the facilitator. Contrary to previous experiences, all participants participated actively in all sessions, some of them, in directive positions, going as far as cancelling other appointments normally considered as being among their highest priorities. With the exception of three cases where circumstances beyond the control of the participants forced them to postpone part of the formative evaluation of their instructional packages, all participants succeeded in completing all assignments according to schedule, given a tolerance for lateness of no more than two or three days. Even during the dramatic period following the sudden death of a beloved president, activities were temporarily delayed for no more than an average of two days. Also, expected delays were always immediately and spontaneously reported to the facilitator. Students sometimes spontaneously reported that they worked during entire 'free' weekends to complete assignments. In a number of cases these self-report data could be confirmed by researcher observations. In other cases they could be cross-checked with information provided by spontaneous independent reporting by other participants. Some students had to surmount tremendous problems of transport and walked for hours, just to be able to carry out the formative evaluation part of their design project. Performance results obtained in the course, as measured against pre-established criteria, using a 100-point scale, ranged from 64 (in the 'satisfactory' category) to 92 (in the 'outstanding' category), with a medium of 75 and a standard deviation of 8.3. These descriptive d a!~ should be interpreted against the backgrotmd of a course which, by objective comparison, based on duration and workload, as well as by subjective appreciation, was considered the most demanding ever given in the institute.

In the weekly questionnaires (Measure 1), participants refer to the motivational messages, either explicitly or by referring to their content. Also, a number of responses can by inference be interpreted as referring to the motivational messages. Over the total period of the course and across learners, a total of 194 anonymous responses was collected as regards the question what factors had, on different occasions, had the most positive influence on the subject's motivation (not all participants could always come up with three such factors on each occasion the questionnaire was administered). These responses were coded according to whether they referred to the motivational messages in a direct way (by mentioning them), whether they did so by referring to their content, or whether they

493

did so inferentially. Table 3 shows the results, specifying the various references in accordance with the order of importance attributed to them by the participants. With respect to the interpretation of these results it should be considered that the stimulus that prompted the responses was an entirely open one, not restricting the range of possible answers in any respect but that of mentioning motivational factors. Moreover, it will be recalled that great care was taken to use the motivational messages as unobtrusively as possible. This meant, among other things, that they were never identified by the name "motivational messages", consequently making it difficult for the learners to refer back to them explicitly. When they did so they might for instance call them "those little things that were being passed around". In view of these considerations it is of interest to note the con- centration of responses (slightly more than 20%) around the issue of motivational messages.

In the responses that mention the motivational messages directly, participants referred to the perceived good quality of the messages and specifically to the following characteristics:

- They provide constant motivation.

- Their distribution is systematic.

- They are opportune. They always arrive at exactly the right moment.

- They are both informative and motivational.

It should be mentioned here that on one occasion an individually distributed motivational message was referred to as the single most demotivating factor by one subject. Because of circumstantial clues provided in the responses to the questionnaire, it was possible to trace the respondent and to discover that the message had been given to her inappropriately. The problem could be solved. The incident shows that the messages are effective, though in the wrong direction if not applied correctly.

Table 3. Number of times (out of total of 194) reference was made to motivational messages in weekly questionnaires by category and by perceived importance (1 is highest)

ORDER OF IMPORTANCE

TYPE OF REFERENCE

Direct mention Reference to content By inference

1 1 9 7

2 2 11 7

3 2 1 1

TOTALS: 5 21 15

494

Data from the end-of-course questionnaires (Measure 2) are presented in Table 4 and Table 5. They refer to all but two of the participants of whom one was ill during the final course assessment and the other was on an assignment abroad at that time. Questionnaires 2a and 2b were completexl with an interval of four days between them. Questionnaire 2c was completed at home during those four days.

The following meanings were defined for the ratings on the 5-point scale used in these instruments. They were stated in the introduction to the questionnaires and orally explained to the participants before they filled out the questionnaires.

(1) Of no, or almost no, importance. Could probably be done away with.

(2) Makes a moderate contribution to attaining course objectives. Not having it would influence the course result, but not dramatically.

(3) Is important. Not having it would significantly influence the result of the

course.

(4) Is very important. Not having it would have a dramatic influence on the course results.

(5) Extremely important. Without it attainment of the major course objectives would be impossible.

Questionnaire 2a evaluated perceived importance of 15 instructional strategy components, most of them referring to aspects that would be perceived by the respondents as "instructional" (e.g. autonomous study, design project, report writing, exercises, etc.). The average mean rating for the various strategy components was 3.82 (S.D. = .68). Table 4 presents the mean ratings for the three Motivational Message related components. Also presented are the proportions of respondents that rated the component as having had a significant, dramatic, or absolutely fundamental influence (rating levels 3, 4 and 5).

Table 4. Partidpant ratings of perceived importance of Motivational Message related instructional strategy components

STRATEGY COMPONENT PERCEIVED IMPORTANCE

mean rating on proportion of ratings 5-point scale at levels 3, 4, or 5

Provision of feedback by facilitator to participants

Provision of motivational messages by facilitator

Participation of participants in providing feedback to fadlitator

4.15 1.00

2.54 .54

3.92 .92

495

On the surface, it would seem that provision of motivational messages was not viewed as being as important as the other strategies. However, due to the fact that in the questionnaire the provision of feedback, which was almost exclusively hart- died through the motivational messages, appears as an item separate from the motivational messages as such, it is likely that responses to the latter item refer particularly to motivational messages that did not communicate feedback and were thus perceived as being motivational in a less concrete sense of the word. As the aim had been to make the messages as unobtrusive as possible, it should not be a surprise that respondents attribute less importance to them. This should be even less surprising if it is considered that the evaluation was done in a comparative context in which some very obtrusive components, such as being tested on three occasions on one's learning of an entire textbook written in a foreign language, or writing an extensive report, were included as well.

It is interesting in this connection to note that qualitative and therefore less comparative, comments of respondents, particularly for the lower ratings, sometimes deviate from their numerical ratings. For example, here are some (translated) remarks of participants who rated their perceived importance of the motivational messages as 2: "(They) were quite motivating and served to call my attention at every moment of my learning." "The motivational messages helped to remind us to carry out the tasks, to keep us attentive, to mobilize us. They had a decisive influence." "Sometimes I felt frustrated, but some of them raised my morale." A subject who rated his/her perceived importance of the motivational messages as 3, observed: "The messages were quite useful. At one moment I had lost courage and wanted to leave the course because of the difficulties I had."

The data collected on questionnaire 2b confh-m those discussed in relation to questionnaire 2a. They are presented in Table 5. To facilitate their interpretation, it is noted that a total of seven media was evaluated by the respondents. The mean ratings of these media averaged 3.78 (S.D. = .92). The other three media were: textbook, activities manual, complementary handouts.

Table 5. Participant ratings of perceived importance of Motivational Message related media

MEDIUM PERCEIVED IMPORTANCE

mean rating on

5-point scale proportion of ratings

at levels 3, 4, or 5

Gannt chart

Contract

Motivational messages

Facilitator

3.38

2.62

2.54

4.62

. 6 2

.46

.46

1.00

496

The mean rating of the perceived importance of motivational messages was, coincidentally, 2.54, although with a different distribution and in reference to comparisons of other media used in the course. The facilitator is presented here as a component of the media-mix. In view of the verbal observations made by the respondents, the rating attributed to this medium should be considered as a result of his use of the various other media concerned, including the motivational messages. This is in line with Heinich's (1970) new paradigm of instructional management, in which the teacher is conceived not as the sole deliverer of instruction, but as one of the media whose action could be aided by other media. It is reminded in this context that straight instructional tasks constituted only a minor part of the facilitator's activities.

Questionnaire 2c received, by its nature, open-ended, qualitative responses. They confn'm the students' perception of the course as highly demanding, exces- sively short, executed during the wrong time of the year, using materials, though of high quality, difficult to work through because of their having been written in a language the students were not familiar with, but highly rewarding in terms of its ultimate outcome. Positive references were also made in the responses to this questionnaire to the way the course was organized, alluding to the motivational messages as well, to the role of the facilitator and to that of the contract. One student said, "Only sign when you know what you are up to." Another said, "The contract has kept me going. The facilitator developed the work in such a way that, ever since he made us put down our signatures, we were carded to the fmish." One respondent, comparing this course with previous experiences, remarked that, if this time it went remarkably better, the problem, on other occasions, must "not have been the course; it was us". This seems to indicate that, indeed, motivation had a great influence.

The comments obtained spontaneously and in the round table discussion (Measures 3 and 4) resulted in data confirming the findings reported in relation to Measures 1 and 2. It was recognized, sometimes emphasized, that the motivation does not come from the messages as such. Motivation comes from within, or from the perceived intrinsic interest of what is being learned, or from the expected value of the learning result for one's professional contributions to society. However, no doubt is left by the great majority of responding participants, that these messages played an important, if not crucial role in mobilizing and stimulating them in developing their motivation. The fact that they were always there at exactly the fight moment received particular attention from the respondents.

Particular reference is made to the contract signed at the beginning of the course. This interactive motivational message was referred to in the following ways: "It was enough to make us reach the end." "It led to a continued physical presence of everyone." "I never participated in a course in which I did not miss classes." "It influenced me such that I walked the entire distance from Mahotas

497

[approximately 10 kin] to be able to hand in the report." "The paper as such was not important, but the attitude which it gave rise to, which was necessary to sign it, was important."

As regards the observable reactions to the messages fMeasures 5 and 6), it was found that students did indeed follow-up on the content of the messages. Though, according to their own observations, it had never been their habit to trouble their instructor outside class hours with questions, they now made frequent use of the possibility to receive help by telephone or by direct contact with the facilitator. The motivational messages were invariably read. People would normally keep them and care for them, carrying them around for reference while working on their assignments, discussing them with their colleagues. It was often found and expressed by participants, that they had some kind of expectancy regarding messages to come. They were looking forward to them, wondering what the next one would be.

Conclusion

The present field based exploratory study has, by its very nature, the limitations inherent in its design, that of an embedded single-case study ('gin, 1984), based on action research carried out in the context of an existing instructional environment. On the other hand, carrying out the study in a real world context, rather than in a laboratory, has the advantage of enhancing the external validity of its results. Furthermore, the context chosen, that of instructees whose incorporation in the instructional target group had been beyond the control of the participants themselves, but had been determined by assignment based on the interests of the organization, or as in this case the society, supports the natnralistic gener~liT_abfl- ity (Stake, 1986 and 1988) of its results. It is also noted that the ARCS model is a problem solving model aiming at applications in the real world rather than the laboratory. Testing its validity would therefore naturally have to be done in the context for which it was developed, i.e. that of that same real world.

The results presented in this study give a clear indication of the effectiveness of the clinical use of motivational messages in the context under consideration. However, the first author was, in addition to being one of the researchers, the designer and implementer of the intervention. While it is impossible to completely control for this influence, an effort was made to document all aspects of the intervention, including the process and outcome measures, to facilitate replication of the study with other researchers and designers. Both authors are continuing their study of this motivational strategy, respectively in a different training program in Mozambique, as well as in the context of a grsd~l~te study program in the United States of America. Results so far obtained are in agreement with those reported in the present study (Bickford, 1988).

498

The clinical use of motivational messages is a pro-active strategy, originated on the basis of the ARCS model, to address people's needs as regards the mobili- zation of their motivational potential. The intimate link between the strategy and the model which is used in the development of the medium in question, lends support to the contention that not only was the strategy effective, but so was the model. Though the ARCS model was initially developed with the instructional context of the United States in mind, its validation in the context reported on here shows its applicability in a wider, socially and culturally different, context.

The third research question asked in this study, regarding what would be the most efficient and effective form of the developed strategy and how it could best be applied, addresses the question of optimality (Reigeluth, 1989). Inspection of the data base shows that the design and delivery process of the motivational messages adapts itself to the circumstances to which the motivational intervention responds, resulting therefore in continuous improvement. A particular and dramatic, instance of this "improvability' of the tendency of the strategy to sirive for the optimum, was cited earlier under Motivational Design. Although the results of the present study indicate that indeed the clinical use of motivational messages seeks optimization of results, further research is considered necessary to address this question more rigorously. A follow-up study has meanwhile been conducted providing more extensive supporting evidence in this regard (Visser, 1990).

Despite the positive results obtained in the present study, it is felt that further research is needed and further development of the model is in order. While identifying the range of motivational factors to be addressed by a particular motivational message, it was found somewhat cumbersome to express in terms of the subeategories of the ARCS model motivational factors related to such concepts as solidarity and, in general, interests related to the group rather than the individual. It is our view that the problem is related to the existence of differences between the sociocultural circumstances prevailing in the context within which the ARCS model was developed and those found in the country where the present study was conducted. The high emphasis placed on the interest of the individual in countries like the USA is quite different from the dominant model of social interaction in Mozambique and many other parts of the world where both traditional values and the political orientation emphasize values related to the well-being of the group, the community and society at large. In line with the conclusions of Bohlin (1987), some further refinement of the ARCS model, in the light of the above difficulty, seems to be desired. Particular attention should be given in this context to the development of subcategories that would facilitate describing an individual's motivation as regards relevance and satisfaction in a context where these concepts are derived from feelings related to a person's incorporation in group, communal, or societal efforts to reach common aims. Also, research regarding the delivery of motivational messages through different media and using different formats, should constitute a natural complement of the present study.

499

Regarding the application of the clinical use of motivational messages, it is believed that the strategy has an interesting potential for the motivational

enhancement of distance learning, particularly in situations and geographical regions where there is a great lack o f interpersonal contact between the learner and those who guide him or her. Distance learning contexts in developing countries usually show this characteristic. It is o f interest in this connection that the design of motivational messages does not assume high competence, on the part o f the designer, as regards the content matter being studied. In the case o f many dis-

tance education systems existing in the Third World, important roles are played by peer tutors (Young, Perraton, Jenkins and Dodds, 1980), who are non- professional teachers who may be only slightly more competent than the learners, but have a great degree o f homophyly with the distance students and often work with them on a voluntary and/or part-time basis. Their task is to a high degree a motivational one. Short training courses or seminars are frequently organized for

these people and could, with relatively little effort, develop their competencies in the design and timely delivery of motivational messages. Alternatively, or in

addition to the above, motivational messages could be pre-designed for expected circumstances and be personalized and delivered in accordance with observed motivational requirements.

A final caveat is in order. The approach presented in this study is, in part, an

analytic one. However, there is an important prerequisite to be fulfilled before the motivational designer should assume the analytic mode. Dedicated human rela- tionships are at the basis o f a person's ability to motivate another human being. They are a necessary, though rarely also a sufficient, condition to help the other person overcome his or her motivational restraints.

References

Andrews, D. and Goodson, L. (1980). A comparative analysis of models of instructional design. Journal of Instructional Development, 3(4), 2-16.

Bickford, N. (1988). The systematic application of principles of motivation to the design of print materials. Unpublished doctoral dissertation, Florida State University, Tallahassee, FL.

Bohlin, R. M. (1987). Motivation in instructional design: comparison of an American and a Soviet model. Journal of Instructional Development, •0(2), 11-14.

Briggs, L. J. (1980). Thirty years of instructional design: one man's experience. Educational Technology, 29(2), 45-50.

Brophy, J. (1983). Conceptualizing student motivation. Educational Psychologist, 1, 200-215. Dick, W. and Carey, L. (1979). Dise~o s£¢temdtico de la instruccidn. Bogota, Colombia: Voluntad. Dick, W. and Carey, L. (1985). The systematic design of instruction. Glennview, IL: Scott, Foresman. Gagnt, R. M. (1965). The analysis of instmctiomd objectives. In R. Glaser (Ed.), Teaching machines

and programmed learning, II: Data and directions (pp. 21-65). Washington, DC: Department of Audiovisuad Instruction, National Education Foundation.

Gagnt, R. M. (1985). The conditions qf learning and theory of instruction (Fourth Edition). New York: Holt, Rinehart and Winston.

Heinich, R. (1970). Technology and the management of instruction. Washington, DC: Association for Educational Commtmications and Technology.

500

Holt, J. (1980). Never too late: my musical life story. New York, NY: Dell Publishing Co. Keller, L M. (1979). Motivation and instructional design: a theoretical perspective. Journal of

Instructional Development, 2 (4), 26-34. Keller, J. M. (1983). Motivational design of instruction. In C. M. Reigeluth (Ed.), Instructional-design

theories and models: an overview of their current status. Hillsdale, NJ: Lawrence Erlbaum Associates.

Keller, L M. (1984). Use of the ARCS model of motivation in teacher training. In K. E. Shaw (Ed.), Aspects of educational technology. Volume XVIL Staff development and career updating. London: Kogan Page.

Keller, J. M. (1987a). Strategies for stimulating the motivation to leam. Performance and Instruction Journal, 26(8), 1-7.

KeUer, J. M. (1987b). The systematic process of motivational design. Performance and Instruction Journal, 26(9), 1-8.

Mager, 1L F. and Clark, C. (1963). Explorations in student controlled instruction. Psychological Reports, 13, 71-76.

Mager, R. F. and McCann, J. (1961). Learner controlled instruction. Palo Alto, CA: Varian Associates.

Martin, B. and Driscoll, M. P. (1984). Instructional theories: maximizing their strengths for application. Performance and Instruction Journal, 23(6), 1-4.

MeClelland, D. C. and Steele, R. S. (1972). Motivation workshops: a student workbook for experien- tial learning in human motivation. New York, NY: General Learning Press.

McCombs, B. L. (1983, April). Motivational skills training: helping smdems adapt by taking personal responsibility and positive self-control. Paper presented at the meeting'of the American Educational Research Association, Montreal

MeCombs, B. L (1984). Processes and skills underlying continuing intrinsic motivation to learn: toward a definition of motivational skills training. EducationalPsychologist, 19(4), 190-218.

Reigeluth, C. M. (1979). In search of a better way to organize instruction: the elaboration theory. Journal of Instructional Development, 20), 8-15.

Reigeluth, C. M. (1989). Educatiomd technology at the crossroads: new mindsets and new directions, Educational Technology Research and Development, 37, 67-80.

Stake, R. E. (1986). An evolutionary view of educational improvement. In E. R. House (Ed.), New directions ofeducatlanal evaluation. Philadelphia, PA: The Falmer Press.

Stake, R. E. (1988). Case study methods in educational research: seeking sweet water. In R. M. Jaeger (gd.), Complementary tncthods for research in education. Washington, DC: American Educational Research Association.

Visser, J. (1985). The systematic design of an instructional module in elementary science education for a Mozambican adult audience. Unpublished manuscript. Florida State University, Instructional Systems Program, Tallahassee, FL.

Visser, J, (1990), Enhancing learner motivation in an instructor-facilitated learning context. Unpublished doctoral dissertation, Florida State University, Tallahassee, F1.

Weiner, B. (1980). Human motivation. New York: Holt, Rinehart and Winston. Wlodkowski, R. J. (1981). Making sense out of motivation: a systematic model to consolidate motiva-

tional constructs across theories. Educational Psychologist, 16(2), 101-110. Wlodkowski, R. J. (1985). Enhancing adult motivation to learn: a guide to improving instruction and

increasing learner achievement. San Francisco, CA: Jossey-Bass Publishers. Yin, R. K. (1984). Case study research: design and methods. Beverly Hills, CA: Sage Publications. Young, M., Perraton, H.0 Jenkins, J. and Dodds, T. (1980). Distance teaching for the Third World: the

lion and the clockwork mouse. London, UK: Rontledge and Kegan Paul.

the clinical use of motivational messages: an inquiry into the validity of the arcs model of...

Documents