generating meaningful hypotheses with aptitude-treatment interactions

13
Generating Meaningful Hypotheses with Aptitude-Treatment Interactions PERRIN E. PARKHUST Knowledge of individual difference variables that statistically interact with a particular instructional setting is useful in adapting a treatment to a student. In a paper, "How Can In- struction be Adapted to Individual Differences?", Cronbach (1967) suggests that the most adequate way of coping with individual differences might be to alter instructional methods to fit the aptitude patterns of the learner (e.g., student dif- ferences in IQ, verbal ability, etc.). This procedure would not be useful unless an aptitude-treatment-interactionI exists; i.e., students high in a given ability (aptitude) achieve better under one method of instruction than they do under an- other, and the reverse trend holds true for another treatment with students of low ability. However, for the most part, research efforts dealing with aptitude-treatment-interactions (ATI) have failed to produce consistent results that are statistically significant and/or 1Many of the statistical and methodological issues of ATI research, be- yond the scope of this paper, are dealt with by other authors in this special AVCR issue. Perrin E. Parkhurst is senior engineering psychologist, Advanced Instruc- tional System, McDonnell Douglas Corporation, Lowry AFB, Denver, CO 80230. AVCR VOL. 23, NO. 2, SUMMER 1975 171

Upload: perrin-e-parkhust

Post on 22-Aug-2016

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Generating meaningful hypotheses with aptitude-treatment interactions

Generating Meaningful Hypotheses with Aptitude-Treatment Interactions

PERRIN E. PARKHUST

Knowledge of individual difference variables that statistically interact with a particular instructional setting is useful in adapting a treatment to a student. In a paper, "How Can In- struction be Adapted to Individual Differences?", Cronbach (1967) suggests that the most adequate way of coping with individual differences might be to alter instructional methods to fit the aptitude patterns of the learner (e.g., student dif- ferences in IQ, verbal ability, etc.). This procedure would not be useful unless an aptitude-treatment-interaction I exists; i.e., students high in a given ability (aptitude) achieve better under one method of instruction than they do under an- other, and the reverse trend holds true for another treatment with students of low ability.

However, for the most part, research efforts dealing with aptitude-treatment-interactions (ATI) have failed to produce consistent results that are statistically significant and/or

1Many of the statistical and methodological issues of ATI research, be- yond the scope of this paper, are dealt with by other authors in this special AVCR issue.

Perrin E. Parkhurst is senior engineering psychologist, Advanced Instruc- tional System, McDonnell Douglas Corporation, Lowry AFB, Denver, CO 80230.

AVCR VOL. 23, NO. 2, SUMMER 1975 171

Page 2: Generating meaningful hypotheses with aptitude-treatment interactions

AV C O M M U N I C A T I O N REVIEW 1975 S U M M E R : 1 7 2

DEFINITION OF ATI

FIGURE 1 Plot Illustrating

an Aptitude- Treatment- Interaction

meaningful to educational practitioners. This paper at- tempts to: 1) provide a brief description of the ATI concept; 2) review examples of selected ATI studies; and 3) present two conflicting theories (about realism and relevant cues in visual treatments) that might be resolved using the ATI ap- proach. An aptitude variable can be any personological or organis- mic variable upon which individuals differ (e.g., IQ, anxiety, dogmatism, etc.). A treatment is any instructional strategy or combination of instructional strategies that structures in- formation for the purpose of having students learn that in- formation. An aptitude-treatment-interaction exists when, as a result of a given treatment, individuals at one end of an aptitude variable perform at one level on a criterion measure. Also, individuals at the other end of the aptitude variable per- form at a significantly different level on the criterion measure and the reverse trend holds true for a second treatment. (For an expanded discussion see Snow & Salomon, 1968; Bracht, 1970; and Rhetts, 1974.)

In Figure 1 T a would be used for those individuals high on aptitude "X", while T b would be used for those individuals low on aptitude "X". In short, aptitude-treatment-inter- action analysis or ATI (Snow, Tiffin, & Seibert, 1965; Cron- bach, 1967; and Cronbach & Snow, 1969) focuses attention on the question of whether a particular instructional method

LOW HIGH APTITUDE "X"

Page 3: Generating meaningful hypotheses with aptitude-treatment interactions

GENERATING MEANINGFUL HYPOTHESES WITH ATI : 1 7 3

EXAMPLES OF ATI STUDIES

tends to be facilitating or inhibiting for a particular area of difference among learners. Educational psychologists have met with some degree of suc- cess in delineating the nature and dimensions of various fac- tors of individual differences; e.g., IQ, anxiety, reading com- prehension, etc. Cooper and Gaeth (1967) reported inter- actions between factors including a method of presenting au- diovisual materials and other factors such as IQ, age, dif- ficulty of material, and reading ability. James (1962) found preferences for a particular mode of instruction were unre- lated to ability, although performance was. He found that su- periority of reading was greater for students who scored higher on the criterion. Knight and Sassenrath (1966) re- ported that students with high achievement motivation per- formed better on three criterion variables than did students with low achievement motivation. Skanes et al. (1974) inves- tigated ATIs between IQ and ability to transfer information with and without the use of pretests. They reported an IQ- by-pretest interaction. Educational practitioners, however, have lagged far behind in their efforts to identify and imple- ment teaching strategies based on those differences in order to facilitate learning of various educational objectives.

In a study by Pascal (1971) students had an option of selecting one of three instructional modes: lecture, lec- ture with discussion, or independent reading. According to Pascal, students who chose the independent study option indicated 1) a greater need for autonomy, 2) greater flexibility, 3) a higher tolerance for ambiguity, and 4) a greater preference for abstract and scientific thinking, than those students who selected the lecture option. Also, students preferring the independent study option seemed more likely to have had prior experience with that mode of instruction. Pascal made no direct reference to student learning; however, he did report that the inde- pendent study students indicated doing nonrequired read- ing and enjoyed writing papers. Students favoring the lec- ture-discussion option indicated that they enjoyed bull sessions and had more frequent informal verbal exchanges with their peers than either of the other two groups.

In general, options offered do seem to appeal to different types of individuals with different habits, experiences, and

Page 4: Generating meaningful hypotheses with aptitude-treatment interactions

AV COMMUNICATION REVIEW 1975 SUMMER : 1 7 4

abilities. James (1962) and Pascal and McKeachie (1970) report that, while students generally have a preference for one option over another, the effects of receiving their pre- ferred option over an alternative are not as definitive as they had originally anticipated. This finding suggests that student preference might not be a viable individual difference vari- able.

Campeau (1965) found that students scoring high on test anxiety learned more in an instructional program with feed- back provisions, while those low in anxiety learned more in a program without feedback. Another researcher (Lublin, 1965) reported that college students scoring high on autonomy-need (seeking independence) achieved higher scores with a self-paced instructional strategy than those with low autonomy-need.

Buckland (1967) in a study designed to investigate differ- ent ways of responding (writing, thinking, and reading) in a linearly programed instructional sequence, examined the gen- eral mental ability variable, IQ. On the immediate posttest there were no significant differences among response modes reported at either high or low IQ level. The delayed posttest yielded a significant IQ-by-response-mode interaction. Among high IQ students, those employing written responses were superior in achievement to those employing reading re- sponses. Low ability students suffered a decrement in achievement for all response modes except reading.

Doty and Doty (1964), using the Guilford-Zimmerman Temperament Survey (sociability scale), investigated the ef- fectiveness of media in relation to five student characteristics: GPA (grade point average), creativity, achievement need, social need, and attitude toward instructional media. These researchers reported significant correlations between scores on an achievement test in a programed unit and GPA, creativ- ity, and social need. When effects of GPA were partialed out, significant correlations were obscured between achievement on programed instruction and social need.

Moore, Smith, and Teenan (1965), using attitudes and achievement as the prime criterion measures, investigated selected traits of the individual as they relate to success in individualized learning. They reported that individuals who were most successful with programed instructional mate-

Page 5: Generating meaningful hypotheses with aptitude-treatment interactions

GENERATING MEANINGFUL HYPOTHESES WITH ATI : 175

A SPECIFIC EXAMPLE OF THE

GENERAL PROBLEM

rials were high hostile pressure learners (express fear of fail- ure) and high need achievers.

The ATI approach described thus far focuses on the use of a two-way interaction between aptitude and treatment variables. However, recent researchers have suggested that studies including an interaction analysis of a task variable, along with the treatment and aptitude variables, might yield greater benefits for educational practitioners.

Pascal (1971) has suggested that future studies should give thought to a three-way analysis design in order to examine trends in interactions among several variables. Referring to his own (1971) study and that of Pascal and McKeachie (1970), h e states: "Both of these studies suggest that a three- way analysis design be used to test for possible interactions between personality factors, the preference factor (receiving or not receiving preferred option) and instructional methods and their effects on cognitive and affective outcomes" (Pas- cal, 1971, p. 16).

Rhetts (1974) has challenged ATI researchers to examine three elements: 1) the nature of the task, 2) the learner, and 3) treatment variables. He states, " . . . research demonstrates that learner, task, and treatment characteristics can combine in interaction with one another to produce complex perform- ance differences. These findings, in turn, underline the need for investigators interested in instructional design to utilize an Attribute by Treatment interaction design. However, re- searchers cannot simply 'throw together' combinations of the three variables. A task-first approach . . . and a two-stage design for data collection (Rhetts, 1972) should improve sig- nificantly the yield from such research" (p. 347). To this point, only the very general nature of ATI investiga- tions has been examined. This section of the paper presents a brief discussion of two competing theoretical frameworks dealing with different approaches to instructional treatments and suggests a rationale for an ATI approach as a possible solution to this dilemma.

An ongoing controversy exists in the literature regarding the role of realism in visualized instructional materials. A number of theorists, writing during the 1940s and early 1950s, suggested the existence of a realism continuum along which all materials could be scaled in relation to their in-

Page 6: Generating meaningful hypotheses with aptitude-treatment interactions

A V C O M M U N I C A T I O N REVIEW 1975 S U M M E R : 176

structional effectiveness. Specifically, these theories include: Dale's (1946) cone of experience, the inconicity theory of Morris (1946), Carpenter's (1953) sign similarity theory, and Gibson's (1954) theory of pictorial perception. Dwyer (1967, 1972) has referred to these collectively as "realism theories." Each of these realism theories hypothesizes that the more realistic an instructional treatment is, the more effec- tively it will facilitate student learning.

However, . more recent researchers, Travers et al. (1964, 1970) and Dwyer (1967, 1972), have disagreed with these theories on the grounds that more realistic materials will tend to present too many irrelevant cues which will interfere with learning. Miller et al. (1957) suggest that too many irrele- vant cues may actually compete with one another for atten- tion and subsequent information storage. Broadbent (1958, 1965) and Travers et al. (1964) view human information processing as a single-channel limited-capacity system and that in order for information to be stored, many irrelevant cues must be filtered. Dwyer (1972) has published data indi- cating that, for some educational objectives, simplified or less realistic visual instructional materials are more effective in facilitating student achievement than more complex photo- graphic or detailed drawings of the same object. Dwyer found strong evidence to indicate that the most realistic visual illus- trations were not necessarily the most effective instruc- tionally, thereby lending experimental support to the point of view hypothesized by Travers et al. (1964, 1970) and Broadbent (1958, 1965). Each of these "relevant cue" theories (i.e., those of Travers, Miller, Broadbent, and Dwyer) hy- pothesizes that, on some educational tasks, realism in visuals accompanying instructional materials may be edited with no decrement in student learning.

Since the theoretical approaches of the realism and relevant cue theorists contradict one another, perhaps an examina- tion of individual differences among students might be one way of attempting to explain the conflicting results. Such an approach would permit consideration of an aptitude vari- able, a task variable, and a treatment variable in the analysis.

Studies dealing with students' prior knowledge of subject matter, as an aptitude variable, have yielded interesting re- sults. Cooper and Gaeth (1967), Tobias and Abramson

Page 7: Generating meaningful hypotheses with aptitude-treatment interactions

GENERATING MEANINGFUL HYPOTHESES WITH ATI : 177

Building the Hypothesis

(1971), and Abramson and Kagen (1975) conducted studies

in which familiarity of content material was one variable un-

der investigation. These researchers report that the presence

of interactions clearly supports the notion of an ATI analy-

sis approach to experiments involving students ' level of prior

knowledge as one of the variables.

The ATI approach would seem to be a logical step toward

solving the dilemma of the realism theorists and the relevant

cue theorists. The reader is cautioned not to infer that

knowledge of how one individual difference variable interacts

in the instructional setting is transferable to how other vari-

ables might interact. Only by assessing the trends of m a n y

such variables will strides be made towards discovering a

" theory of instruction. ''2

The following section hypothesizes a relationship between

visual realism accompanying instructional materials and

performance on one particular educational task, for students

who differ with respect to prior knowledge of subject mat-

ter. 3 The discussion that follows is an a t tempt to provide

a basis for the nonintuit ive design of a "media mix ''4 along

three dimensions, thereby setting forth one possible concep-

tual f ramework for responding to Rhetts ' (1974) earlier men-

tioned challenge. Operat ing within this f ramework, the ex-

perimenter could assess the interaction of an aptitude vari-

able, a visual realism variable, and a task variable.

2Not to be confused with a theory of learning. The latter is descriptive; i.e., it describes, after the fact, the conditions under which some behavior was acquired. The former is prescriptive; i.e., it sets forth rules concerning or specifying the most effective way a particular type of student achieves knowledge or masters a skill.

3Since this introductory paper is meant to deal with basic concepts only, more sophisticated hypotheses dealing with more than one treatment variable, in combination with more than one level of educational task, in combination with the aptitude variable have been omitted. These considera- tions should not be neglected in the design of future studies.

4A term adapted from the field of business (marketing communications) where a "marketing mix" is organized for the introduction of each new product. This mix involves assessing the strengths of such elements as: di- rect mail, radio, TV, newspapers, point of purchase displays, magazines, etc., and mixing the proper amounts of each, to facilitate sales to a previous- ly identified audience.

Page 8: Generating meaningful hypotheses with aptitude-treatment interactions

AV COMMUNICATION REVIEW 1975 SUMMER : 1 7 8

Research into self-paced instruction has given educators some indication of its overall effectiveness in teaching cer- tain information. However, with few exceptions, relatively little research has been conducted relating directly to the use of visual illustrations in improving the effectiveness of self- paced instruction for students differing on some aptitude variable. Figure 2 presents a hypothetical plot of a situation in which students, who differ in prior knowledge of subject matter, receive one of three kinds of visuals accompanying an instructional unit.

The criterion measure has purposely been labeled "high level educational objective." Successfully completing such an objective would involve a task similar to a comprehension or synthesis test, designed to measure the degree to which in- terrelationships between material or facts can be explained. Simply stated, the proposed hypothesis is that realistic vis- uals will facilitate learning only for students who are rela- tively unfamiliar with a particular content area, while there will be no significant differences among visual treatment groups for students who are relatively familiar with a par- ticular content area.

We can generally assume that individuals who have a high

FIGURE 2 High Level

Educational Objective-

Hypothesized Plot for High and

Low Level of Prior Knowledge

Students Receiving 1"1, Tz,

or T3

IOO

CRITERION MEASURE

HIGH LEVEL

ED. OBJ.

r3

i! ii ili : ng O L()W HIGH

ENTRY BEHAVIOR (APTITUDE)

Page 9: Generating meaningful hypotheses with aptitude-treatment interactions

GENERATING MEANINGFUL HYPOTHESES WITH ATI : 1 7 9

level of entry behavior are going to perform (achieve) well no matter what type of instruction they receive. To insist upon a significant difference of means within the "high" group as a prerequisite to reporting the results seems to run the risk of obscuring what might otherwise be extremely useful infor- mation. As a case in point, refer again to Figure 2. In this hypothesized example, Ts > T2 > TI for the low prior knowledge group. No significant differences are hypothe- sized among treatments for the high prior knowledge group.

As a rationale for the findings of the proposed hypothesis, the following explanation is submitted. Students who per- ceive the more realistic illustrations as possessing an abund- ance of irrelevant learning cues (students with high subject matter expertise), may consider the illustrations to be of low information value and may be reluctant to "study" or inter- act-with the more realistic visuals. This high-level-of-prior- knowledge group could be expected to extract relevant learn- ing cues from simple illustrations and integrate that infor- mation with previous knowledge to achieve the stated higher level educational objective.

For those students with less subject matter expertise, real- istic visual presentations may be perceived as possessing more inherent information. Students in this group are likely to interact or "study" the visuals for a longer period of time. They are also likely to attend to the additional learning cues and extract the relevant information in order to achieve the stated higher level educational objective. For this same group (low level of prior knowledge), the less realistic visuals may transmit less relevant information, no matter how long the students are permitted to view and interact with their respective illustrations. Knowledge of these kinds of inter- actions might help resolve the dilemma of the "realism" theorists and the "relevant cue" theorists. Furthermore, when posttests both of an immediate and delayed nature are added, data would be provided so that designers of mediated instruc- tional materials could make strides toward: 1. compensating for individual differences by identifying a

"media mix" which would facilitate learning of a given task by a given segment of the population measured on one dimension of human variability;

2. building a basis for rational and informed selection of

Page 10: Generating meaningful hypotheses with aptitude-treatment interactions

AV COMMUNICATION REVIEW 1975 SUMMER : 1B0

C O N C L U S I O N

alternative methods of self-paced instruction, by educa- tors, for their students;

3. contributing to our knowledge of the extent to which in- creased realism in visualized instruction facilitates student learning of specific educational tasks.

If experimenters are to see the results of their research prop- erly interpreted and applied in a practical learning environ- ment, more care must be given to formulating research ques- tions and designing studies that can reflect the true purpose behind the definition of ATI, i.e., identifying different treat- ments that can facilitate learning for different types of indi- viduals so that students m a y be differentially assigned to al- ternative instructional treatments based on empirical evidence.

In the not too distant future, some educators will no doubt be professionally trained to assist in formulating educational prescriptions for individuals. Therefore it follows that these professionals must have knowledge of an individual's needs, psychological makeup, style of processing information, cog- nitive and affective measures, and personal or individual learning factors. This information would serve as a basis for prescribing learning experiences that meet previously speci- fied objectives.

This forecast suggests examination of a prescriptive or di- agnostic approach to educational research. It implies that such research should concern itself with studies designed to fit a decision theory or adaptive model rather than being restricted only to mean difference hypothesis testing models. Regression analysis techniques, which test regions of sig- nificance around regression slopes, provide a valuable added dimension to existing techniques in ATI studies. A strict analysis of mean differences of a heterogeneously grouped sample of students could obscure differences in treatments that might possibly have significance for an individualized instructional method, program, strategy, or model.

The answers to instructional questions raised by individual variability are not to be found in unique groupings of stu- dents but in unique instructional practices or strategies. The question of responsibility (accountability) for facilitating learning in an individualized instructional program must undergo considerable reexamination. It is clearly impossible for the teacher, even the most dedicated, to satisfy all the

Page 11: Generating meaningful hypotheses with aptitude-treatment interactions

GENERATING MEANINGFUL HYPOTHESES WITH ATI : 181

individual learning requirements of all the students in all cur- riculum areas. To assume such capabilities in the design of any individualized instructional program is to court failure.

A major implication of the studies reviewed is that the future investigation of ATIs is a promising field of research, with potentially significant educational applications. It seems reasonable to suggest that learning can be enhanced by assigning students to instructional materials known to have been designed on the basis of optimal relation to the student's individual aptitude or ability patterns. The question of how, and on what individual difference variables these instruc- tional materials are designed to maximize learning for a group of individuals, depends largely upon the conceptual base underlying the experiment itself.

REFERENCES Abramson, T., & Kagen, E. Familiarization of content and different response modes in programmed instruction. Journal of Edu- cational Psychology, 1975, 67, 83-88.

Bracht, G. H. Experimental factors related to aptitude-treatment in- teractions. Review of Educational Research, 1970, 40, 627-645.

Broadbent, D. E. Perception and communication. New York: Per- gamon, 1958.

Broadbent, D. E. Information processing in the nervous system. Science, 1965, 3695, 457-462.

Buckland, P. R. The response in a linear program: Its mode and importance. Programmed Learning and Educational Tech- nology, 1967, 4 (February), 47-$1.

Campeau, P. L. Level of anxiety and presence or absence of feed- back in programmed instruction. USOE, NDEA Title VII Project No. 1155. Palo Alto, Calif.: American Institutes for Research, February 196S.

Carpenter, C. R. A theoretical orientation for instructional film re- search. AV Communication Review, 1953, 1, 38-52.

Cooper, J. C., Jr., & Gaeth, J. H. Interactions of modality with age and with meaningfulness in verbal learning. Journal of Educa- tional Psychology, 1967, 58, 41-44.

Cronbach, L. J. How can instruction be adapted to individual dif- ferences? In R. M. Gagn~ (Ed.), Learning and individual dif- ferences. Columbus, Ohio: Charles E. Merrill, 1967. Pp. 23-29,

Cronbach, L. J., & Snow, R. E. Individual differences in learning ability as a function of instructional variables. Final Report, USOE Contract No. OEC4-6-061269-1217. Stanford Univer- sity, 1969.

Page 12: Generating meaningful hypotheses with aptitude-treatment interactions

AV COMMUNICATION REVIEW 1975 SUMMER : 182

Dale, E. Audio-visual methods in teaching. New York: Dryden, 1946.

Dory, B., & Doty, L. A. Programmed instructional effectiveness in relation to certain student characteristics. Journal of Educational Psychology, 1964, 334-338.

Dwyer, F. M. Adapting visual illustrations for effective learning. Harvard Educational Review, 1967, 37, 250-263.

Dwyer, F. M. A guide for improving visualized instruction. State College, Pa.: Learning Services, 1972.

Gibson, J. J. A theory of pictorial perception. AV Communication Review, 1954, 2, 2-23.

James, N. E. Personal preference for method as a factor in learning. Journal of Educational Psychology, 1962, 53, 43-47.

Knight, H. R., & Sassenrath, J. M. Relation of achievement motiva- tion and test anxiety to performance in programmed instruc- tion. Journal of Educational Psychology, 1966, 57, 14-17.

Lublin, S. C. Reinforcement schedules, scholastic aptitude, auto- nomy need, and achievement in a programmed course. Journal of Educational Psychology, 1965, 56, 295-302.

Miller, N. E. (Ed.) Graphic communication and the crisis in Educa- tion. AV Communication Review, 1957, 5(3), 1-120.

Moore, J. W., Smith, W. I., & Teenan, R. Motivation variables in programed learning, NEM 795, 1965.

Morris, C. W. Signs, language, and behavior. New York: Prentice Hall, 1946.

Pascal, C. E. Individual differences and preference for instructional methods. Canadian Journal of Behavioural Science, 1973, 5, 253-262. Also, Montreal: McGill University, 1971. (ED 059 971.)

Pascal, C. E., & McKeachie, W. J. Offering course options: Person- ality, option preference, and course outcomes. Paper presented at the annual meeting of the American Educational Research Association, Minneapolis, March 1970.

Rhetts, J. E. Attribute-treatment interactions and individualized in- struction: A conceptual framework and example from the project PLAN. In L. Sperry (Ed.), Learning, performance and individualized differences: Essays and Readings. Glenview, Ill.: Scott Foresman, 1972. Pp. 269-285.

Rhetts, J. E. Task, learner, and treatment variables. Journal of Educational Psychology, 1974, 66, 339-347.

Skanes, G. R., Sullivan, A. M., Rowe, E. J., & Shannon, E. Intelli- gence and transfer: Aptitude by treatment interactions. Journal of Educational Psychology, 1974, 66, 563-568.

Snow, R. E., & Salomon, G. Aptitudes and instructional media. AV Communication Review, 1968, 16, 341-357.

Snow, R. E., Tiffin, J., & Seibert, W. F. Individual differences and instructional film effects. Journal of Educational Psychology, 1965, 56, 315-326.

Page 13: Generating meaningful hypotheses with aptitude-treatment interactions

GENERATING MEANINGFUL HYPOTHESES WITH ATI : 183

Tobias, S., & Abramson, T. Interaction among anxiety, stress, re- sponse mode and familiarity of subject matter on achievement from programmed instruction. ]ournal of Educational Psy- chology, 1971, 62, 357-364.

Travers, R. M. W. Man's information system. Scranton, Pa.: Chandler Publishing, 1970.

Travers, R. M. W., McCormick, M. C., Van Mondfrans, A. D., & William, F. E. Research and theory related to audiovisual infor- mation transmission. Salt Lake City: University of Utah, Bureau of Education Research, 1964.