document resume author durham, gigi title note · document resume ed 374 408 cs 011 837 author...

DOCUMENT RESUME

ED 374 408 CS 011 837

AUTHOR Durham, GigiTITLE Toward a Systematic Method of Measuring Free Recall

from Printed News Stories.PUB DATE Aug 94NOTE 31p.; Paper presented at the Annual Meeting of the

Association for Education in Journalism and MassCommunication (77th, Atlanta, GA, August 10-13,1994).

PUB TYPE Speeches/Conference Papers (150) ReportsDesc-iptive (141)

EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS *Evaluation Methods; Higher Education; Media

Research; News Writing; Pilot Projects; *Recall(Psychology); Reliability; Schemata (Cognition);*Text Structure; Validity

IDENTIFIERS Journalism Research; *News Stories; Text Factors

ABSTRACTThis paper proposes a systematic method of measuring

subjects' free recall from printed hard news stories, based on schematheories of cognition. Citing literature that demonstrates the roleof text structures and text schemas in the recall of written text,the paper suggests incorporating these processes into the assessmentof recall of news. In the paper, the first steps toward developingsuch a measure are taken; the procedure hinges on parsing stimulusand response passages according to a schematic news structure. Apilot study of the instrument's reliability and validity (involving69 undergraduate students) is included in the paper, although furtherwork neecis to be done to refine the instrument. Contains 63references and 2 figures illustrating examples of text structures.The comprehension measure is attached. (Author/RS)

***********************************************************************

Reproductions supplied by EDRS are the best that can be madefrom the original document.

***********************************************************************

. .

Toward a systematic method of measuring free recallfrom printed news stories

Gigi Durham, Ph.D.Assistant Professor

Department of JournalismUniversity of Texas at Austin

Austin, TX 78712(512) 471-1980 (work)/(512) 443-5121 (home)

U.S. DEPARTMENT OF EDUCATIONOffice Ot Educational Research and lreoichieinno:

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

IffThis document has been reproduced asreceived from the person or organizationoriginating it.

Minor changes have been made toimprove reproduction quality.

Points et view or opinions slated in thisdocument do not necessarily representofficial OERI position or policy

"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

0.r

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

Paper presentul in the Theory and Methodology Division of the Association forEducation in Journalism and Mass Communication, 1994, Atlanta, GA

Running head: Measuring recall

BEST COPY AVAILABLE

Toward a systematic method of measuring free recallfrom printed news vies

Gigi Durham

This paper proposes a systematic method of measuring subjects' free recall from printed

hard news stories, based on schema theories of cognition. Citing literature that

demonstrates the role of text structures and text schemas in the recall of written text, the

author suggests incorporating these processes into the assessment of recall of news. In

this paper, the first steps toward developing such a measure are taken; the procedure

hinges on parsing stimulus and response passages according to a schematic news

structure. A pilot study of the instrument's reliability and validity is included, although

further work needs to be done to refine the instrument.

Measuring recallPage 1

Toward a systematic method of measuring free recall from printed news stories

In the past quarter-century of mass communication scholarship, a substantial body of

research has focused on audience recall of the content of news stories. The bulk of these

studies deal primarily with the quantification of news recall (e.g. Booth, 1970; Neuman,

1976; Gunter, 1980, 1981; Findahl & Hoijer, 1975, 1981, 1985; Edwardson, Grooms &

Pringle, 1976; Edwardson, Grooms & Proud love, 1981; Edwardson, Kent & McConnell,

1985; Fumham & Gunter, 1989; De Fleur & Cronin, 1991; Wicks, 1992; De Fleur,

Davenport, Cronin & De Fleur, 1992; Beentjes, Vooijs & Van der Voot, 1993; Feccorro

& De Fleur, 1993). Remarkably, in each of these and other studies, different

measurement instruments have been developed to assess subjects' recall of the stimulus

news stories. Many of these measures are loosely based on standard tests of aided and

unaided recall, also referred to as recognition and recall. While these measures have

each been shown to be reliable within the discrete contexts of the studies, no attempt

has been made, thus far, to develop a theoretically derived, reliable, valid and systematic

method of accurately measuring memory for news that could be used with consistent

results across a variety of experimental methods.

The goal of this paper is to present the first steps in developing such a measure .

The method presented in this paper based on the information-processing model of

schema theory.

Schema Theory and the Recall of Printed Text

Research indicates that recall is an interactive operation in which a reader brings

his/her prior knowledge and beliefs into play while encoding new information and latk r

activates that knowledge for retrieval of the information (e.g. Bobrow & Norman, 1975;

Schank & Abelson, 1977; Pearson, Hansen & Gordon, 1979; Graesser & Nakamura,

4


1982; Freebody & Anderson, 1983; Stahl & Jacobson, 1986). The theoretical position

on which these findings are predicated is known as schema theory.1

Schema-based models of information processing have led to increased understanding

of the recall of written text. Central to the development of such models is the evolution

of the construct of the schema as a paradigm for cognition.

The psychological concept of the schema emerged initially as a reaction to the

traditional associationist models of memory and learning (e.g. Ebbinghaus, 1964), in

which recall occurred simply as a response to a stimulus. The associationist model

gradually gave way to the trace theory of mental representation, which evolved into

schema theory as we know it today. (For a more complete account of the history of

schema theory, see Hastie, 1981). The notion of a schema was first used in studies of

memory and remembering and later applied in the study of reasoning, learning, language

processing, problem solving, reading, and countless other cognitive and psychosocial

processes.

In general terms, a schema may be defined as a dynamic, generic mental framework

for the hierarchical representation of knowledge. Anderson (1977) asserts, "A schema

represents generic knowledge; that is, it represents what is believed to be generally true

of a class of things, events, or situations" (p. 2). Graesser and Nakamura (1982), in an

extensive exposition on the role of schemas in comprehension and memory, define

schemas as "generic knowledge structures that guide the comprehender's interpretations,

inferences, expectations, and attention. A schema is generic in that it is a summary of

the components, attributes and relationships that typically occur in specific exemplars"

(pp. 60-61).

In the schematic view of cognition, incoming information is encoded and stored via

an appropriate schema or pre-existing mental knowledge structure.

1In this paper, the word "schema" will be pluralized as "schemas," per the style used by Mandler (1984, p.2. note 2), rather than the traditional "schemata."


Thus, in the context of recall of written text, schema theory is predicated upon the

notion that knowledge is organized into dynamic knowledge structures in the brain that

are activated during the reading process, as well as during other types of cognitive

processing. As Adams and Collins (1977) have observed:

The goal of schema theory is to specify the interface between the reader and the

textto specify how the reader's knowledge interacts with and shapes the

information on the page and to specify how that knowledge must be organized to

support the interaction. (p. 5)

In relation to the processing of mass media news messages, several studies have

shown that prior knowledge is vital to comprehension and recall of news (cf. Findahl

Hoijer, 1981; Findahl & Hoijer, 1985). In other words, the existence of a schema for a

news topic or for some other aspect of a news story will improve recall of the story. As

Findhal and Hoijer point out:

Schema theory stresses the organization of earlier knowledge in memory in

general or prototypical schemata, representing standard situations, events, or

structures. Two different kinds of schemata have been proposed: one deals with

knowledge about recurrent events and situations . . . ; the other deals with

knowledge about the typical structure of stories . . In news comprehension and

recall, both kinds of schemata (about recurrent events and about the structure of

news items) are probably activated. (1985, p. 390).

In a departure from the strictly numbers-oriented tradition of this line of research in

mass communication, Woodall, Davis, and Sahin (1983) proposed a theoretical

framework for memory and understanding of news based on principles of episodic

memory and on the trace theory of memory and understanding. Another pioneer in this

domain, Doris Graber (1988), conducted in-depth interviews with 21 subjects to study

their schema-based strategies for processing information gained mainly from the news


media. These theoretical perspectives are strongly tied to the schematic model of

information processing.

Text Structures and the Reading Process

In his groundbreaking book Remembering (1932) , Frederic Bartlett of Cambridge

University, England, proposed that recall of written material depends on a reader's

schema for the structure of a written passage. Such structures, also known as text

grammars, have been theorized to be fundamental in the organization of all written text

(Rumelhart, 1975; Rumelhart, 1977; Mandler & Johnson, 1977; Meyer, 1977a, 1977b;

Stein & Glenn, 1979; Marshall & Glock, 1979; Taylor, 1980; Taylor & Samuels, 1983;

Page & Stewart, 1985; Waters & Hou, 1987; Griffith & Ripich, 1988; Cook & Mayer,

1988; Troyer, 1992).

News stories possess a structure or text grammar uniquely their own. As van Dijk

(1988a) observed, "[I]n English we may use the term 'news story,' and this suggests that

news might be a special kind of narrative. Yet, we also know that itdiffers from the kind

of stories we tell in everyday conversations or in children's books or in novels. Hence,

we must specify why and how news stories are different." (p. 1).

That news stories follow a set structural pattern is a notion that has been intuitively

acknowledged for many years. In the textbook Writing for Mass Comm,;nication ,

Hutchison (1986) points out that all hard news stories 2 should have a formal structure,

beginning with a lead:

In a good lead, the important things come first. They provide the umbrella under

which all details of the story will fit comfortably. . . . The details usually flow from

the lead in order of descending importance into the succeeding paragraphs. A

2The inverted pyramid story structure is characteristic of hard news stories, i.e. stories that are factualaccounts of events, usually with a time element. Soft news stories (news stories with a human interestfocus, written in a lighter vein) or feature stories often do not follow the inverted pyramid structure. Thepresent research is therefor confined to hard news stories.


simple news story about a minor traffic accident or a minor house fire will look

like an inverted pyramid. (p. 125).

In another popular journalism textbook, Fed ler (1989) describes the structure of a

hard news story thus:

The lead in an inverted pyramid story summarizes the topic, and each of the

following paragraphs presents some additional information about it: names,

descriptions, quotations, conflicting viewpoints, explanations, background data

and so forth. Most paragraphs are self-contained units that require no further

explanation, and only the summary of the entire story appears in the lead. News

stories end with their least important details. (pp. 135-136).

The structuring of hard news stories as inverted pyramids is so entrenched in the

newswriting process that Tuchman (1978) claims that most news stories consist of

prestnictured patterns of words into which reporters insert "factoids."

The traditional inverted pyramid structure of news stories corresponds to the concept

of a schematic structure or text grammar. Van Dijk (1983, 1988a, 1988b) observed that a

news story can be viewed in terms of schematically structured discourse. Using cognitive

models, van Dijk examined media discourse and its representation in memory, and his

analysis of hundreds of international newspaper stories led him to postulate an

underlying structure for printed news:

The overall organization of news discourse reflects the importance of

macrostructures. These will typically be expressed by titles or headlines, by initial

or final summaries, or by leads . . . The lead, often printed in bold type . . . will

express, in a first few sentences (which are, by definition, "thematic sentences"),

the full macrostructure of the news discourse. Following sentences will then

progressively specify further details of the events, with the less important ones at

the end (with the practical consequences that these can, if necessary, be cut by

the editor). Unlike argumentatively structured discourse, such as the scholarly

, MinE


paper, where the important conclusion comes at the end, news in the daily press

is organized by the principle of relevance or importance, along a dimension of

decreasing prominence with respect to the macrostructure. (van Dijk, 19f'3,

p. 35).

Newsom and Wollert (1988, p. 120) assert that most hard news stories have the

following elements:

1. The lead (the main point)

2. Secondary points in a tie-in transition

3. Elaboration on the main point

4. Support for the lead

5. Background

6. Development of the main idea

7. Details

They offer two diagrammatic representations of the inverted pyramid: the

"traditional" inverted pyramid (Figure 1.1) and the "modified" inverted pyramid (Figure

1-2), represented on the following pages.

LEAD:

TIE-IN:

BODY:


Who, what, when, where, why & how (16 to 25 words)

One sentence connecting one element of the lead to the body

Development of the most important

Second most important element of

Further development of most important element

Other elements

The least important facts in the storynothing new introduced

H elements of lead

LEAD 5W Sz. H

Tie-in

Most important element

Second most important element

Other

Least importantelement

Figure 1-1. Newsom and Wollert's modified inverted pyramid

U

LEAD:


Major theme, could be significance of event, rather than factMay be two sentencesMay not include 5W & H

TIE-IN: The leftovers of the 5W & H not mentioned in the lead

1ST GRAPH: Explication of the lead incident, quote, meaning or background ofeventhow something came to be

2ND GRAPH: Additional information about most important fact of lead.Something to give credibility or significance to lead information

3RD GRAPH: Secondary theme or supporting documentation for the lead

4TH GRAPH: Any other details, in order of significance to lead

LEAD

Documentation orexplicationbackground or history

Elaboration of lead

Secondary themeor

supporting facts, quotes

Least significant details

Figure 1-2. Newsom and Wollert's modified inverted pyramid


Each element in these structures roughly corresponds to the concept of a "basic node"

in the Mandler and Johnson story grammar (Mandler & Johnson, 1977)the elements

comprising the surface structure of a story, comprised of a sentence or several sentences.

The basic nodes govern the way a story is written or rewritten after having been recalled.

For the purposes of this study, Newsom and Wollert's modified inverted pyramid

structure will be adopted as the typical structure for a breaking hard news story in an

American newspaper. The modified inverted pyramid includes quotes and the possibility

of a secondary theme in the story, which more complex news stories often contain; the

traditional pyramid does not accommodate these elements. In this respect, the Newsom

and Wollert pyramid is a more useful descriptor than the Fed ler pyramid, which makes a

provision for quotes but not for a second theme within a story.

Defining Recall

Recall is the dependent variable of interest in this study. Van Dijk (1987) observes

that "one result of understanding a text is a representation of the meaning of the text in

(episodic) memory" (p. 165), the direct implication being that text comprehension

always results in the storage of information in long-term memory for later retrieval, i.e.

text comprehension always precedes long-term recall of text information. Voss (1984)

corroborates this notion. He writes

While reading, the individual is assumed to interpret the text contents in terms

of his or her own knowledge, interests, and attitude. During the interpretive

process the individual develops a representation of the contents of the text.

Learning is thus presumed to involve the storage of information via the development of

the representation (p. 197) (emphasis added)

As Belli (1986) observes, rival psychological theories have resulted in very different

interpretations of the memory process. The mechanistic model of memory, for example,


views it as a passive process, whereas schema-based models regard memory as an active,

adaptive operation. The latter position will be adopted for the purposes of this study.

The term "recall" possesses different meanings in different contexts. In the mass

communication literature, recall is further classified into aided and unaided recall.

Facorro and De Fleur (1993) note, "In unaided recall, the subject reports to the

experimenter all the facts of the stimulus material that he or she can remember. in aidea

recall, cues are provided" (p. 592). British psychologist Martin Le Voi refers to unaided

recall as "free recall" (1986, p. 105) and describes the process as happening in a situation

where "the subject is free to recall any items .. . and create and use helpful cues in any

way he or she wishes" (p. 105). Generally, unaided or free recall means the unprompted

remembrance of information; aided recall, on the contrary, refers to the process of

remembering information in a situation where prompts or cues are provided.

In most studies of recall from printed news sources, research questions are generally

framed in terms of long-term benefits to readers from the assimilation of information in

news stories. Thus. free recall is of greater interest than aided recall. The measure

proposed in this paper ;s geared to the evaluation of free recall of news story content.

Measuring Free Recall

Free recall is often measured very informally, usually by means of a request to "write

down brief descriptions" of what is recalled (Gunter, 1980) or requests for lists of facts

remembered from stimulus stories (Wicks, 1993).

This paper proposes a more formal method of measuring recall from a stimulus news

passage, based on a procedure developed by Meyer (1975), adapted by Taylor (1980) and

Taylor and Samuels (1985) and later used by McGee (1982) for scoring recall of

expository (nonfictional) texta method similar to the scoring procedure followed by

Mandler and Johnson (1977) for measuring recall of narrative (fictional) text. The

,,.'4725.=2GIVG. ..LZtt.alAdriZZ2=.71,ILL=ZZZIA:AV


procedure is based on schema theory and the adherence of text passages to schematic

text grammars.

The measurement of recall inevitably involves a comparisonthe recalled text is

compared to the original stimulus passage, and a reca 4 score is awarded based on the

similarity of the two.

In a schema-based method, the proportion of elements from the initial passage

recalled per structural node would be calculated. The recall score would be based on the

overall proportion of the stimulus passage recalled. The use of a text grammarin this

case the inverted pyramid structurerenders such a measure more rigorous in its

capacity to account for each significant element of the story, making use of the reader's

schematic processing of the text. In addition, the sequencing of elements is built into

the structure of the stimulus passage as well as that of the recalled passage so that the

measure can, if desired, also provide an assessment of subjects' sensitivity to news

structures.

Since van Dijk (1983, 1988a, 1988b) has established that most printed hard news

stories conform to a specific text grammar (or "superstructure" in his terms), the first step

in developing a rigorous measurement instrument is the parsing of a given stimulus

passage according to that grammar.

The stimulus passages is then administered to subjects in the context of an

experiment; afterward, subjects are asked to write down an account of the stimulus

passage, keeping as close to the original version as possible.

In scoring, the recalled passage is also parsed according to the news text grammar,

and the number of propositions remembered from each terminal node in the original text

is noted. If less than half of an original sentence is remembered, the sentence is not

counted as having been recalled. If about half the sentence is remembered, it is counted

as 0.5. If more than half of the sentence is remembered, it is counted as a full sentence.

Depending upon the intent of the study, propositions recalled outside of their nodes


could be scored as though they had been recalled in the proper sequence, given an

alternate scoring scale, or discounted.

An Example

To test a recall measure based on this system, a sample news story was administered

to 69 undergraduates. Subjects were given sufficient time to read the passages; the

stimuli were then removed.

Subjects were next asked to perform a series of distractor tasks, including filling out a

multiple-choice questionnaire requesting demographic information and the West

Informal Reading Inventory (West, 1978) to assess their reading level.

The stimulus passage, taken from an Associated Press wire story, was as follows:

PLAN GIVES BIG CATS BOOST

Breeding program for panthers OK'd

MIAMI (AP)Ten Florida panthers roaming the wilds of South Florida are about to be

chosen for a new life in captiv. that may have important consequences for the survival

of the endangered species.

A captive-breeding program has been approved by federal and state officials to boost the

shrinking panther population from an estimated 30 to 50 in the wild.

"There are certain purists who say, 'Let them die a natural death out in the wild'," said

John Christian of the U.S. Fish and Wildlife Service. "We need to consider their views,

but on the other hand we are charged with halting the extinction of the species and

moving toward its recovery."

"It's no question the population is stressed. When you get down t o a total population of

30 to 50 animals, you get to the point of facing the brink of extinction," said Dennis

Jordan, Fish and Wildlife's Florida panther coordinator. "We consider we have one

viable sustaining population now in South Florida and none anywhere else."

-1S,731.==TA-M0,11,- -747 -Aac- -


But some wildlife managers say the Florida panther, a type of cougar, is in nowhere near

the danger of the California condor or the black-footed ferret when their entire

population was rounded up for captive breeding.

The goal of the new program is 500 breeding adults in captivity and three wild colonies

in 20 years using high-tech methods such as radio-telemetry collars and possibly in vitro

techniques.

The nocturnal cats, with adults weighing 60 to 120 pounds, are smaller and darker than

most cougars and have a unique tail crook and a cowlick in the middle of their backs.

The panther, which hunts deer and smaller game, is a solitary hunter that needs lots of

room at least 40 square miles for a female and more than 200 square miles for each

male, with little overlap.

The panther once ranged from Louisiana to South Carolina, but widespread hunting and

urban sprawl have pushed it into the Everglades and the undeveloped center of South

Florida.

Without help, experts estimate, the panther will vanish in 25 to 40years.

After the distractor tasks, subjects were asked to recall the stimulus passages and

write them down using language as close to the originals as possible. Finally, a measure

of reading comprehension of the passage was administered. This instrument (see

Appendix A) was based nn Pearson and Johnson's description of textually explicit and

scriptally implicit questions (Pearson & Johnson, 1978) as well as on the comprehension

questions used in Johns' (1988) Basic Reading Inventory, a standardized informal reading

evaluation instrument.

Recalls were scored by parsing the subject-generated passages and recording the

number of text elements recalled from the original from each node in the appropriate

text grammar. An example of one such recalled passage and its scoring is provided

below.

1 el0


PLAN GIVES BIG CATS BOOST,

Panther breeding OK'd

10 Florida panthers will be taken into captivity in an effort to increase their

population from 30 to 50. By the year 2000 they want 500 in captivity and 3 wild

colonies. The cats require 200 square miles for males and 40 square miles for females for

the nocturnal hunters, with little overlap.

This isn't so bad as the California Condor or blackfooted ferret whose entire

population was rounded up for captive breeding.

So-and-so, from the Fish and Wildlife Service, said, "Some people say we sheuld just

leave them alone to die in the wild but we're also charged with bringing them back from

extinction."

The panther, which once roamed in an area from Louisiana to South Carolina has

declined in population due to hunting and urban sprawl (and is now confined to the

Everglades and the unpopulated area of central South Florida).

The original stimulus passage was parsed as follows:

Node I. (Lead) Ten Florida panthers roaming the wilds of South Florida are about to be

chosen for a new life in captivity that may have important consequences for the survival

of the endangered species. (1 sentence)

Node 2. (Tie-in) A captive-breeding program has been approved by federal and state

officials to boost the shrinking panther population from an estimated 30 to 50 in the

wild. (1 sentence)

Node 3. (Elaboration of lead) "There are certain purists who say, let them die a natural

death out in the wild'," said John Christian of the U.S. Fish and Wildlife Service. "We

need to consider their views, but on the other hand we are charged with halting the

extinction of the species and moving toward its recovery." (2 sentences)

Node 4. (Support for the lead) "It's no question the population is stressed. When you

get down to a total population of 30 to 50 animals, you get to the point of facing the


brink of extinction," said Dennis Jordan, Fish and Wildlife's Florida panther coordinator.

"We consider we have one viable sustaining population now in South Florida and none

anywhere else." (2 sentences)

Node 5. (Background) But some wildlife managers say the Florida panther, a type of

cougar, is in nowhere near the danger of the California condor or the black-footed ferret

when their entire population was rounded up for captive breeding. (1 sentence)

Node 6. (Development of the main idea) The goal of the new program is 500 breeding

adults in captivity and three wild colonies in 20 years using high-tech methods such as

radio-telemetry collars and possibly in vitro techniques. (1 sentence)

Node 7. (Details) The nocturnal cats, with adults weighing 60 to 120 pounds, are

smaller and darker than most cougars and have a unique tail crook and a cowlick in the

middle of their backs.

The panther, which hunts deer and smaller game, is a solitary hunter that needs lots of

roomat least 40 square miles for a female and more than 200 square miles for each

male, with little overlap.

The panther once ranged from Louisiana to South Carolina, but widespread hunting and

urban sprawl have pushed it into the Everglades and the undeveloped center of South

Florida.

Without help, experts estimate, the panther will vanish in 25 to 40 years. (4 sentences)

In the recalled story, the number of sentences remembered from each terminal node

in the stimulus text structure is noted, per the scoring system described on page 11. Thus,

in the recalled passage provided above, the first sentence the subject wrote was: "10

Florida panthers will be taken into captivity in an effort to increase their population

from 30 to 50." The subject has recalled the first part of the sentence but not the

second, and therefore is given 0.5 for partial recall of the sentence comprising the first

node.


In the first sentence, the subject recalled part of the tie inthe goal of the program

to boost the panther population from 30 to 50 in the wild. Again, the subject is given

partial credit for recalling about half of the sentence in the node. (0.5)

The subject's second sentence is "By the year 2000, they want 500 in captivity and 3

wild colonies." This sentence is not very similar to anything in the second node, but it is

similar to the sentence in the sixth node, the development of the main idea. The subject

is given 0.5 for partial recall of the sixth node, because in this example sequencing is not

being considered as part of the recall evaluation.

The subject then wrote, "The cats require 200 square miles for males and 40 square

miles for females for the nocturnal hunters, with little overlap." This is similar to the

second sentence in the seventh node. Again, the subject is given 0.5 for that sentence.

"This isn't so bad as the California Condor or blackfooted ferret whose entire

population was rounded up for captive breeding." The subject has almost exactly

recalled Node 5, and is given a full point (1.0) for this node.

So-and-so, from the Fish and Wildlife Service, said, "Some people say we should just

leave them alone to die in the wild but we're also charged with bringing them back from

extinction." This captures part of Node 3. The first sentence in this node was almost

completely remembered, with the exception of the name of the quoted speaker, so tha

subject receives 1.0 point. The subject remembered about half of the second sentence, so

0.5 was awarded. The total score on this node is 1.5.

The subject's final sentence was, "The panther, which once roamed in an area from

Louisiana to South Carolina has declined in population due to hunting and urban sprawl

(and is now confined to the Everglades and the unpopulated area of central South

Florida)." This is a complete sentence from Node 7, so the subject receives a score of

1.0 for this sentence.

1.i


The subject's recall score is computed as follows, based on the proportion of each node

recalled:

Node 1. 0.5/1 = 0.5

Node 2. 0.5/1 = 0.5

Node 3. 1.5/2=0.75

Node 4. 0/2 = 0.0

Node 5. 1/1 = 1.0

Node 6. 0.5/1 = 0.5

Node 7. 2/4 = 0.5

Total score = 0.5 + 0.5 + 0.75 + 0 + 1 + 0.5 + 0.5 = 3.75/7 = 54%

Reliability and Validity of the Measurement Instrument

Validity

"The validity of a measuring instrument may be defined as the extent to which

differences in scores on it reflect true differences among individuals on the characteristics

that we seek to measure" (Selltiz, Wrightsman & Cook, 1976, p. 169). In other words,

the validity of a measure refers to the extent to which it is a true gauge of the construct it

is supposed to measure. Here, the crucial question would be whether the recall

instrument described above was an accurate measures of the subjects' memory for the

stimulus passage.

Content Validity

Content validity is an estimate of the extent to which the measurement instrument is

an adequate sample of the domain or process being measured. Content validity is often

assessed by submitting the measurement instrument to the scrutiny of experts, who ,....tty

that all facets of the construct or domain under investigation are represented in the

instrument. The instruments used in this experiment possessed considerable content

cr. ". na


validity because they were derived from the stimulus passage itself, were constructed

following the methods used by earlier investigators of similar phenomena, and measured

recall of each structural proposition within each passage. The measure thus represented

an adequate sample of the processes under investigation.

Construct Validity

In this study, the dependent variable "recall" is a construct or abstraction that

describes a trait possessed by the subjecti.e., the ability to remember and to understand

text. Construct validation refers to the process of estimating to what extent the

measurement instruments measure these latent traits.

Construct validation may be accomplished by means of examining patterns of

correlation of a measure with other validated measures of the same trait (convergent

validity) and by showing that the trait as measured by the instrument in question can be

differentiated from other traits or constructs (discriminant validity).

Curtis and Jackson (1962) have suggested that high correlations between measures

intended to measure different but theoretically related constructs provide evidence of

convergent validity. In this case, comprehension and recall of the stimulus passage could

be expected to vary together: they are theoretically related but conceptually distinct

constructs. The construct validity of the recall measure was thus estimated by measuring

the degree of correlation between subjects' scores on a measure of comprehension and

the measure of recall based on the same stimulus passage. The Pearson correlation

coefficient was found to be 0.33 (N=69); this correlation was statistically significant

(1)=.003).

Reliability

Reliability of a measurement instrument refers to the steadiness of scores on the

instrument. Reliability may be measured in terms of stability or consistency of scores

over time; internal consistency, sometimes called homogeneitythe similarity of items


within a test or other instrument; and equivalence, or consistency across different forms

of the same instrument.

Reliability of the instrument was measured using the alternate-forms method.

Twenty-four undergraduate students were asked to participate in the reliability study.

The students responded to the various measures in the experimental sequence described

above. Two days later, the experiment was repeated with the same class; however,

students were given different stimulus passages on the second day. Thus, they were

effectively given alternative forms of a single test.

A coefficient of stability and equivalence was computed according to the formula:

p AB = A -MA) (B -Md

SDA SDB

where A represents a subject's score on the first test (Test A) and B represents the

subject's score on the second test (Test B), MA represents the mean score on Test A, MB

represents the mean score on Test B, SDA represents the standard deviation of scores on

Test A, and SDBrepresents the standard deviation of scores on Test B. (For a more

detailed explanation of the alternative forms method of reliability assessment, see Walsh

& Betz, 1985, pp. 50-51, and Horvath, 1985, pp. 71-85).

The reliability coefficient computed for the recall measure was 0.64. This rather low

coefficient could be attributed to the small size of the sample used in the reliability test.

Selltiz, et al. (1976) point out that low reliability coefficients are not necessarily

indicators of low validity of measurement instruments (pp. 194-197). They argue that in

some cases, fluctuations in scores on measurement instruments from one test

administration to another or even within a test are desirable in that tests that produce

extremely homogenous results are not as useful for making fine discriminations among

responses and may in fact reflect a high degree of content error. They note that the


assessment of reliability and validity occurs along a continuum from convergence of

scores to divergence, depending on the correlations being computed, and that "if a

measure can be shown to be reasonably valid . . . it must ipso facto be reasonably reliable,

since a measure with a large error component could not show such consistent

relationships" (p. 197).

Discussion

As noted earlier in this paper, research on recall of printed text has been greatly

influenced by schema theories. In the case of news stories, there is empirical evidence to

show that hard news stories usually follow a prescribed text structure and can easily be

parsed into fairly standard sequential elements. It would appear that the schematic

processing of text is a primary influence in the recall of a printed news story's content, as

it has been shown to be in the case of other types of printed text. This study represents

an attempt to take advantage of the schematic structuring of hard news stories to devise a

systematic method of assessing memory for news.

This research contributes to the development of a theoretically rigorous model for

understanding the process of recalling news and using it in the evaluation of memory for

news content. However, this inquiry is still in its preliminary stages, and a more rigorous

investigation of news text structures and the role of schemas in the processing of news

messages needs to be undertaken before a standard measurement tool can be created.

The me, iurement method described herein needs to be retested and refined, particularly

in terms of its reliability, so that it can be applied to all hard news stories across a variety

of experimental methods.


Appendix A

The Comprehension Measure

1) How many Florida panthers are left in the wild?

2) What is the goal of the new captive breeding program?

3) Briefly describe how the range of the Florida panther has changed over time.

4) Two men were quoted in the story you just read. With what federal agency were

they affiliated?

5) To what other North American big cats might the Florida panther be related?

6) Why is the Florida panther facing extinction?

7) What is meant by the term "wildlife conservation"?

8) Why are conservation efforts critically important in today's world?

9) What was they key point of this story?

10) How well do you feel you understood this story?

24

1.

41


References

Adams, M. J., & Collins, A. (1977). A schema-theoretic view of reading. (Report No.

BBN-3548). Urbana, IL: University of Illinois, Center for the Study of Reading.

(ERIC Document Reproduction Service No. ED 142 971).

Anderson, R. C. (1977). Schema-directed Processes in lanquaP comprehension.

(Technical Report No. 50). Urbana, IL: University of Illinois, Center for the Study

of Reading. (ERIC Document Reproduction Service No. ED 142 977).

Bartlett, F. C. (1932). Remembering: A study in experimental and social psychology (5th ed.) .

Cambridge, England: Cambridge University Press.

Beentjes, J.W.J., Voojis, M.W., & Van Der Voot, T.H.A. 1(993). Children's recall of

televised and printed news as a function of test expectation. Journal of Educational

Television, 19, 3-13.

Belli, R.F. (1986). Mechanist and organicist parallels between theories of memory and

science. Journal of Mind and Behavior, 7, 63-86.

Bobrow, G. H., & Norman, D. A. (1975). Some principles of memory schemata. In D.

G. Bobrow & A. Collins (Eds.), Representation and understanding: Studies in cognitive

science, (pp. 131-149). New York: Academic Press.

Booth, A. (1970). The recall of news items. Public Opinion Ouarterly, 34, uA-610.

Cook, L.K. & Mayer, R.F. (1988). Teaching readers about the structure of scientific text.

Journal of Educational Psychology, 80, 448-56.

Curtis, L. F., & Jackson, E. F. (1962). Multiple indicators in survey research. American

Journal of Sociology, 68, 195-204.

2 5


De Fleur, M.L. & Cronin, M.M. (1991). Completeness and accuracy of recall in the

diffusion of the news from a newspaper vs. a television source. Sociological Inquiry,

61, 148-166.

Defleur, M.L., Davenport, L., Cronin, M., & DeFleur, M. (1992). Audience recall of

news stories presented by newspaper, computer, television and radio. Journalism

Quarterly, 69, 1010-1022.

Ebbinghaus, H. (1964) Memory: A contribution to experimental psychology. New York:

Dover.

Edwardson, M., Grooms, D., & Pringle, P. (1976). Visualization and TV news

information gain. Journal of Broadcasting, 20, 373-380.

Edwardson, M., Grooms, D., & Proudlove, S. (1981). Television news information gain

from interesting video vs. talking heads. Journal of Broadcasting, 25, 15-24.

Edwardson, M., Kent, K., & McConnell, M. (1985). Television news information gain:

Videotex vs. a talking head. Journal of Broadcasting & Electronic Media, 29, 367-378.

Faccorro, L.B., & DeFleur, M.L. (1993). A cross-cultural experiment on how well

audiences remember new stories from newspaper, computer, television and radio

sources. Journalism Quarterly, 70, 585-601.

Fedler, F. (989). Reporting for the print media (4th ed.) . San Diego: Harcourt Brace

Jovanovich.

Findahl, 0., & Hoijer, B. (1975). Effect of additional verbal information on retention of

a radio news program. Journalism Ouarterly, 52, 493-498.

Findahl, 0., & Hoijer, B. (1981). Studies of news from the perspective of human

comprehension, Mass Communication Review Yearbook, 6, 379-396..


Findahl, 0., & Hoi, -r, B. (1985). Some characteristics of news memory and

comprehension. Journal of Broadcasting & Electronic Media, 29, 379-396.

Freebody, P., & Anderson, R. C. (1983). Effects of vocabulary difficulty, text cohesion,

and schema availability on reading comprehension. Reading Research Quarterly , 18,

277-294.

Furnham A., & Gunter, B. (1989). The primacy of print: Immediate cued recall of news

as a function of the channel of communication. Journal of General Psychology, 116,

305-310.

Graber, D. A. (1988). Processing the news: How people tame the information tide (2nd ed.) .

New York: Longman.

Graesser, A. C., & Nakamura, G. V. (1982). The impact of a schema on comprehension

and memory. In H. Bower (Ed.), The psychology of learning and motivation, 16, 59 -

109.

Griffith, P.L., & Ripich, D.N. (1988). Story structure in hearing-impaired learning-

disabled and nondisabled children. American Annals of the Deaf, 133, 43-50.

Gunter, B. (1980). Remembering television news: Effects of picture content. Journal of

General Psychology, 102, 127-133.

Gunter, B. (1981). Forgetting the news. Intermedia, 9, 4143.

Hastie, R. (1981). Schematic principles in human memory. In E. T. Higgins, C. P.

Herman, & M. P. Zanna (Eds.) Social cognition: The Ontario symposium: Vol. I

Hillsdale, NJ: L. Erlbaum.

Horvath, M. J. (1985). Statistics for educators . Seattle, WA: Special Child Publications.

Hutchison, E. R. (1986). Writing for mass communication. New York: Longman.


Le Voi, Martin E. (1986). Encoding and retrieval in recognition and recall. In G. Cohen,

M. W. Eysenck, & M. E. Le Voi (Eds.), Memory: A Cognitive Approach, (pp. 104-

161). Milton Keynes, England: Open University Press.

Ivlandler, J. M., & Johnson, N. S. (1977). Remembrances of things parsed: Story

structures and recall. Cognitive Psychology , 9, 111-151.

Mandler, J.M. (1984). Stories, scripts, and scenes: Aspects of schema theory. Hillsdale, NJ:

Lawrence Erlbaum.

Marshall, N., & Glock, M. (1979). Comprehension of connected discourse. Rending

Research Quarterly, 16, 10-56.

McGee, L. (1982). Awareness of text structure: Effects of children's recall of expository

text. Reading Research Quarterly , 17, 581-590.

Meyer, B. J. F. (1975). The organization of prose and its effects on memory. Amsterdam:

North Holland Publishing.

Meyer, B. J. F. (1977a). The structure of prose: Effects on learning and memory and

implications for educational practice. In R. C. Anderson, R. Spiro, & W. E.

Montague (Eds.), Schooling and the acquisition of knowledge (pp. 179-200). Hillsdale,

NJ: Lawrence Erlbaum.

Meyer, B. J. F. (1977b). What is remembered from prose: A function of passage structure.

In R. 0. Freedle (Ed.), Discourse production and comprehension: Vol. 1 (pp. 307-336).

Norwood, NJ: Ablex.

Neuman, W. R. (1976). Patterns of recall among television news viewers. Public Opinion

Quarterly, 40, 115-123.

Newsom, D., & Wollert, J. A. (1988). Media writing: Preparing information for the mass

media (2nd ed) . Belmont, CA: Wadsworth Publishing Co.


Page, J.L., & Stewart, S.R. (1985). Story grammar skills in school-age children. Topics in

Language Disorders, 5, 16-30.

Pearson, P. D., Hansen, J., & Gordon, C. (1979). The effects of background knowledge

on young children's comprehension of explicit and implicit information. Journal of

Reading Behavior, 11, 201-209.

Rumelhart, D. E. (1975). Notes on a schema for stories. In D. G. Bobrow & A. M.

Collins (Eds.), Representation and understanding: Studies in cognitive science (pp.

211236). New York: Academic Press.

Rumelhart, D. E. (1977). Understanding and summarizing brief stories. In D. LaBerge &

S. J. Samuels (Eds.), Basic processes in reading: Perception and comprehension, (pp.

265-303). Hillsdale, NJ: Lawrence Erlbaum.

Schank, R. C., & Abelson, R. (1977). Scripts plans. aoals and understanding. Hillsdale, NJ:

Lawrence Erlbaum.

Stahl, S. R., & Jacobson, M. G. (1986). Vocabulary difficulty, prior knowledge, and text

comprehension. Journal of Reading Behavior, 18, 309-323.

Wicks, R. H. (1986, August). Methodological approaches to schema measurement: Applying

script' measures to mass media information processing. Paper presented at the meeting

of the Association for Education in Journalism and Mass Communication,

Norman, OK.

Wicks, R.H. (1992). Improvement over time in recall of media information: An

exploratory study. Journal of Broadcasting and Electronic Media, 36, 287-302.

Woodall, W. G., Davis, D. K., & Sahin, H. (1983). From the boob tube to the black box:

TV news comprehension from an information processing perspective. In D. C.

Whitney & E. Wartella (Eds.), Mass communication review yearbook: Vol. 4, (pp.

173-194). Beverly Hills, CA: Sage.


Selltiz, C., Wrightsman, L. S., & Cook, S. W. (1976). Research methods in social relations

(3rd ed.) . New York: Holt, Rinehart, & Winston.

Stein, N. L., & Glenn, C. G. (1979). An analysis of story comprehension in elementary

school children. In R. 0. Freed le (Ed.), Advances in discourse processes: New

directions in discourse processing: Vol II (pp. 53120). Norwood, NJ: Ablex.

Taylor, B. M. (1980). Children's memory for expository text after reading. Reading

Research Ouarterly, 13, 399411.

Taylor, B. M., &-Samuels, S. J. (1983). Children's use of text structure in the recall of

expository material. American Educational Research Journal, 20, 517-528.

Tuchman, G. (1978). Making news. New York: Free Press.

Waters, H.S., & Hou, F.-T. (1987). Children's production and recall of narrative

passages. Journal of Experimental Child Psychology, 44, 348-63.

Troyer, S.J. (1992, December). The effects of text structure on fifth graders' comprehension.

Paper presented at the annual meeting of the National Reading Conference, San

Antonio, Texas.

van Dijk, T. A. (1983). Discourse analysis: Its development and application to the

structure of news. Journal of Communication, 33, 20-43.

van Dijk, T. A. (1987). Episodic models in discourse processing. In R. Horowitz and S. J.

Samuels (Eds.), Comprehending oral and written language, (pp. 161-196). San Diego,

CA: Academic Press.

van Dijk, T. A. (1988a). News as discourse. Hillsdale, NJ: Lawrence Erlbaum.

van Dijk, T. A. (1988b). News analysis: Case studies of international and national news in

the press. Hillsdale, NJ: Lawrence Erlbaum.

0


Voss, J. F. (1984). On learning and learning from text. In H. Mandl, N. L. Stein, & T.

Trabasso (Eds.), Learning and comprehension of text, (pp. 193-212). Hillsdale, NJ:

Lawrence Erlbaum.

West, E. (1978). The use of an unobtrusive screening device to approximate reading levels of

adults. The 27th Yearbook of the National Reading Conference.

Walsh, W. B., & Betz, N. E. (1985). Tests and assessment. Englewood Cliffs, NJ:

Prentice-Hall.

31

e,A.,01,,Vele,-_,..0'6t/ei.,- .

document resume author durham, gigi title note · document resume ed 374 408 cs 011 837 author...

Documents