historical problem solving a study of the cognitive processes used in the evaluation

7/30/2019 Historical Problem Solving a Study of the Cognitive Processes Used in the Evaluation

http://slidepdf.com/reader/full/historical-problem-solving-a-study-of-the-cognitive-processes-used-in-the-evaluation 1/15

Journal of Educational Psychology1991, Vol. 83, No. 1,73-87

Copyright 1991 by the American Psychological Association, Inc.

0022-0663/91/S3.00

Historical Problem Solving:A Study of the Cognitive Processes Used in the Evaluation

of Documentary and Pictorial Evidence

Samuel S. WineburgD e p a r t m e n t of Educational Psychology, College of Educa t ion , Universi ty of Washington

History teachers are frequently urged to use primary sources in their classrooms. Yet littleresearch exists to guide them, for history has been virtually ignored by researchers interested in

cognition and instruction. The present study explored how people evaluate primary and secondarysources when considering questions of historical evidence. A group of working historians and

high school seniors "thought aloud" as they reviewed a series of written and pictorial docum entsabout the Battle of Lexington. Differences were found in how each group reasoned abouthistorical evidence. It is suggested th at these differences are due in part to beliefs that frame the

act of historical inquiry.

Over 70 years ago J. Carleton Bell asked in the pages of this

journal: "What is the historic sense? How can it be developed?These are questions in which the educational psychologist is

interested and which it is incumbent upon him to answer"(1917, p. 317). In the years since Bell asked these questions,we have witnessed an ex plosion of research on school learning.This explosion, part of the "cognitive revolution" in psychol-ogy (Gardner, 1985), has shed light on students' thinking in

such areas as arithmetic (Nesher & Katriel, 1977; Resnick,1982), algebra (Sleeman, 1984), geometry (Greeno, Magone,& Chaiklin, 1979), biology (Carey, 1985), physics (diSessa,1985; McCloskey, 1983), and computer science (Sleeman,Putnam, Baxter, & Kuspa, 1986). These citations represent a

tiny sample of an expansive literature on the cognitive psy-

chology of school subjects. But amid this efflorescence of

research, the subject matter of history has been ignored.The situation is not much better in the cognitive literature

on expertise. Although there are detailed descriptions of the

problem solving of mathematicians (Schoenfeld, 1985), radi-ologists (Lesgold, Feltovich, Glaser, & Wan g, 1 981), physicists(Chi, Feltovich, & Glaser, 1981), physicians (Kuipers, Mos-

This article is based on a dissertation submitted to the Area of

Psychological Studies, School of Education, Stanford University, in

partial fulfillment of the requirements of the doctoral degree. Thiswork was supported by a predoctoral fellowship from the Spencer

Foundation, and that support is gratefully acknowledged.I express my indebtedness to the members of my committee—Lee

S. Shulman, Chair, R ichard E. Snow, and David B. Tyack—for theiradvice and encouragement. I also thank Earl Butterfield, Pam Gross-man, Mike Martinez, Susan Monas, Catherine Crain-Thoreson, Suz-

anne Wilson, and two anonymous reviewers for their com ments on

an e arlier version of this article. I also extend thanks to David D eHart,Amy Julian, N. L. Gage, Greg Hancock, Daniel Hardebeck, LorryHyink, Alan Klockars, John R ossi, and Bonnie Taylor for their helpand advice.

Correspondence concerning this article should be addressed to

Samuel S.Wineburg, Educational Psychology, 312 Miller Hall, DQ-

12, University of W ashington, Seattle, Washington 981 95.

kowitz, & Kassirer, 1988), chess masters (Chase & Simon,

1973), and others (see Chi, Glaser, & Farr, 1988, for review),knowledge of what historians do is limited to what they revealin the handbooks they write for novices (e.g., Barzun & Graff,

1962; Cantor & Schneider, 1967; Carr, 1962; Clark, 1967;

Commager, 1966; Gottschalk, 1958; Gray, 1959; Shafer,1969). But as a window through which to view historicalcognition, these books, prescriptive in nature and basedlargely on self-reports, are of limited value. This is so for at

least three reasons. First, research on other professions has

shown that a wide gap often separates the practices recom-mended by textbooks and those actually carried out by prac-titioners (cf. Elstein, Shulman, & Sprafka, 1978). Second,there is ample evidence demonstrating that people are lessthan accurate reporters of their own cognitive processes, par-

ticularly when these processes have long faded from short-term memory (Chapman & Chapman, 1968; Kahneman,Slovic, & Tversky, 1982; Nisbett & Ross, 1980; Nisbett &

Wilson, 1977). Finally, because expertise is characterized by

fluidity and automaticity (Chi et al., 1988), key features of

cognitive performance are often inaccessible to consciousreview and reflection. Thus, textbooks on historical methodmay tell us more about what historians say they do or whatthey think they ought to do than what they actually do.

Should those interested in historical cognition simply drawfrom the stock of general knowledge about learning? Doingso, in fact, contradicts findings that stress the domain speci-ficity of knowledge (Glaser, 1984; Voss, Vesonder, & Spilich,

1980). The disciplines that lend us school subjects possessdistinctive logics and modes of inquiry (Resnick, 1985;

Schwab, 1978). To presume that competence in any one of

these logics leads automatically to competence in another is

to believe that transfer of training comes easily and effort-lessly, a belief unsupported bydecades of research. To under-stand the "historic sense," we must study people as theyengage in the process of historical inquiry. •

Historical inquiry differs considerably from problem solv-ing in well-structured domains. For example, in doma ins suchas geometry or physics, goals are given to individuals, who

then transform problems to arrive at solutions. But in history,

73



74 SAMUEL S. WINEBURG

goals remain vague and indefinite, open to a great deal of

personal interpretation. In solving well-structured problems,

subjects know whether they have succeeded because "a test

exists . . . that will determine whether an object proposed as

a solution is in fact a solution" (Newell & Simon, 1972, p.

73). But no such tests exist in history. Even the point at which

a historian can say "I know enough to render an account" is

ill-defined, appealing to criteria much different from tests ofwell-structured problems (cf. Fain, 1970).

Indeed, to cast historical understanding as "problem solv-

ing" may itself be problematic (Wineburg, 1989), for historical

understanding can be thought of as beginning where problem

solving in other domains ends. In history, outcomes are often

known—the Babylonians sacked the First Temple in 586

B.C.E.; Sioux Indians routed Custer's 7th Cavalry in 1876;

Ayatollah Ruhollah Khomeini deposed the Shah of Iran in

1979. Rather than arriving at a solution by maneuvering

through a "problem space" of preexisting templates, patterns,

and moves, historians may be said to dwell in an "explanation

space" (Wilensky, 1983) in which they already possess the

"solution" but must "reconstruct the goal and state of the

world from it" (p. 10).

How might such "reconstructions" take place? The present

study tried to find out by examining working historians and

high school history students as they reviewed a set of historical

documents. The characterization of cognitive performance by

sampling the ends of a continuum of expertise has proven to

be a theoretically rich and useful research strategy (e.g., Chi

et al., 1981; Larkin, McDermott, Simon, & Simon, 1980).

However, such comparisons have not escaped criticism. As

Schoenfeld (1985) noted, these designs are fraught with diffi-

culty, for not only do experts and novices differ in the

knowledge and skill they bring to the task, but they differ in

other areas as well—comfort with the researcher, experience,

maturity level, and so on. Such contrasts, then, are better seenin the context of suggesting hypotheses and yielding careful

descriptions of cognitive phenomena than in rigorously esting

the effects of background knowledge or strategy use.

Although clues exist concerning how people might remem-

ber historical facts (e.g., Frisch, 1989), this study began with

a more basic question: How are historical "facts" arrived at

in the first place? A group of general questions, rather than a

set of explicit hypotheses, provided the guideposts for this

study: (a) How do people construct an understanding of

historical events from a group of fragmented and contradic-

tory documents? (b) What heuristics or rules of thumb help

individuals fill in the gaps left by such documents? and (c)

What beliefs do people hold about history that help or hinder

their ability to make sense of historical evidence?

Method

Subjects

Historians. Eight historians (H1-H8; 6 men and 2 women [H5

and H6]) were recruited from universities in the San Francisco Bayarea (see Table 1). Four of these historians (H1 -H4 ) were "American-ists," or historians who had graduate specializations in Americanhistory and had taught history at the college level; four (H5-H8) were

"non-Americanists," with specializations in other areas. Six historianspossessed a doctoral degree; two others were graduate students in theadvanced stages of their doctoral work.

Students. Eight students (S1 -S8; M = 16 years, 7 mon ths; 4 menand 4 women [SI, S3, S5, and S8]) were recruited from two highschools in the San Francisco Bay area. T hree considerations guidedtheir selection: (a) that all students had taken 1 lth-grade Americanhistory the previous academic year, (b) that all were reading at or

above grade level (asdetermined by teacher recom mendations andtheir Scholastic Aptitude Test scores), and (c) that all scored 50% or

above on a pretest, administered during students' regularly scheduledhistory class, composed of items drawn from the N ational Assessmentof Educational Progress (NAEP) examination in American history(Ravitch & Finn, 1987). The items selected for this pretest wereamong the most difficult on the NAEP exam, with item difficultiesof .33 or less. Students' mean SAT score was 1,227 (SD = 95), whichput this sample of students in the top 20% of college-bound highschool students (cf. Waddill, McDaniel, & Einstein, 1988). Students'mean grade point average (GPA) was 3.54 (SD = .35); two studentshad perfect 4.0 GPAs. All students planned to go to a 4-year collegeupon graduation. Students, but not historians, received a $25 hon-

orarium for their participation in the study.

Materials

A set of eight written and three pictorial documents related to theBattle of Lexington, the opening volley of the Revolutionary War,

was assembled. The written docum ents (see Appendix) included twodiary entries, an excerpt from an autobiography, a formal deposition,a newspaper report, and a letter of protest, all of which were writtenfairly close to the time of the battle. Also included were twodocu-ments written much later: a selection from a historical novel (Fast,1961) and an excerpt from a high school textbook (Steinberg, 1963).

Three paintings of the battle were assembled. These depictionswere selected so that n o one of them completely m atched the descrip-tions of the battle in the written documents. Of the three pictures,the 1775 depiction (reprinted in Tourtellot, 1959) best matched the

consensus of the written accounts, but it too was flawed. This pictureshows the colonists in a state of disarray (the impression conveyed byDocuments 1 and 2), but it also depicts a British officer giving the

command to fire, something explicitly denied by Documents 4 and

6. Finally, no wall appears in the painting, a feature referred to by

Documents 4, 5, 6, an d 8.

The 1859 painting (reprinted in McDowell, 1967) contains fouruncorroborated features. First, a woman appears in the center of thepicture, but no mention was made of women at the ba ttle site. Second,the minutem en are shown positioned on a hill, a feature tha t conflictswith the geography of Lexington. Third, a cloud of musket smokerises from the back row of British troops, an unlikely event becauseit would have meant that these soldiers would have been firing overthe heads of their own men. Fourth, some colonists seem to be

loading or even reloading their muskets, an implausible rendering if

one accepts the British account of a single causality (Document 4).However, unlike the first painting, this picture shows a wall.The 1886 painting (reprinted in McDowell, 1967) shows a pitched

battle between the British troops and the American colonists, a

depiction that led Murdock (1925) to characterize it as the "myth of

Lexington." In this "mythic" version of events, the minutemen,instead of fleeing or dispersing as they themselves claimed in theirdeposition (Docum ent 2), are depicted as standing tall and defiant in

the face of overwhelming od ds. In addition, the minutemen are cladin the uniforms of the Continental Army, a historical anachronismbecause the Continental A rmy did not yet exist.

Seven features important in assessing the accuracy of these pictureswere identified. They included, from the 1775 depiction, (a) the



HISTORICAL PROBLEM SOLVING 75

Table 1

Backgrounds of Eight Historians (H1-H 8)

Historian

HI

H2

H3

H4

H5

H6

H7

H8

Degree

Ph D

PhD

Ph D

Doctoral candidate

Ph D

Ph D

Ph D

Doctoral candidate

Doctoralinstitution

StanfordUniversity

University of

Wisconsin—MadisonUniversity of

Wisconsin—Madison

University ofCalifornia,Berkeley

HarvardUniversity

University ofWisconsin—Madison

StanfordUniversity

University ofCalifornia,Berkeley

Specialization

AmericanEducation

Western United

States

Americanbusiness

Native Americans

Japan

17th-centuryEngland

British socialhistory

Medieval Islam

Primarylanguages

English

English,

Spanish

English

English

Japanese

English,French

English

Arabic,Hebrew

TaughtU.S. history

Yes

Yes

Yes

Yes

No

No

No

No

Note. H1-H8 = Historians 1-8.

absence of a wall; from the 1859 depiction, (b) the appearance of thewom an, (c) the depiction of a hill, and (d) the smoke rising from the

rear group of soldiers; from the 1775 and the 1859 depictions, (e) theBritish officer giving the command to fire; from the 1886 depiction,(f) soldiers wearing the uniforms of the Continental Army; and, fromthe 1859 and 1886 depictions, (g) the similarity of the m ain buildingson Lexington Green, which would suggest that the earlier paintingserved as a template for the latter one.

Procedure

The procedure was the same for historians and students unlessotherwise noted .

Think-aloud. Subjects practiced the think-aloud procedure usinga three-digit multiplication program and a series of anagrams, as

recommended by Ericsson and Simon (1984, p. 376). When com-

fortable using this technique, subjects were told that they would be

presented with a series of documents about the Battle of Lexingtonand that their goal was "to try to understand what happened at

Lexington G reen on the morning of April 19, 1775."Presentation of the texts. Subjects read the documents aloud (cf.

Bereiter & Bird, 1985). They were encouraged to say everything tha tcame to mind as they read but were given no specific prompts or

probes for when to comment. Only when they fell silent for several

seconds were they asked "W hat are you thinking?" or "Why did youpause?"Each written document was presented twice: first in its complete

form and then, broken up into individual sentences, each appea ringon a separate 5 in. x 7 in. index card. This method had the advantageof allowing subjects to first see documents as they might appear in a

book of historical sources (e.g., Force, 1847); also, during the secondpresentation, it encouraged readers to slow down the reading processto make it more amena ble to thinking aloud. After reading Docum ent1 for the second time, subjects were given two reference sheetscontaining all eight written documents. Subjects were told that the

purpose of these sheets was to eliminate the need to "flip back throughthe index cards" if they wanted to refer to an earlier document.

Picture evaluation. When subjects had completed reading the

eight written docum ents, they were shown copies of the three paint-ings, with the nam e of the artist and the date of the painting deleted.They were asked to think aloud as they reviewed these paintings.When subjects finished commenting, they were asked which of thepictures "most accurately depicts what happened on LexingtonGreen ." They were also asked to date each picture.

Ranking task. After evaluating the pictures, subjects were askedto rank the written documents in order of their "trustworthiness as

sources for understanding what happened on Lexington Green."

Identification of erms. After the ranking task, subjects were askedto identify 12 terms (names, events, and concepts drawn from the

Colonial period) as a rough measure of background knowledge.(Students completed this task as part of the written pretest describedabove.) Th e following term s were identified: (a) Olive Branch Petition;(b) George Grenville; (c) virtual representation; (d) Salutary Neglect;(e) Townshend Acts; (f) Quebec Act; (g) Proclamation of 1763; (h)

Pontiac; (i) Battle of Saratoga; 0) "one by land, two by sea"; (k)

internal taxation; and (1) Fort Ticonderoga.

Data Analysis

All sessions were audiotaped and transcribed. Protocol analysistook on different forms for different aspects of the task.

Picture evaluation. Protocols from the picture evaluation task

were separated into coding units consisting of independent subject/predicate clauses. A coding scheme was developed, consisting of fourcategories:1

1. Description: Included descriptive statements that made no ref-

erence to the purpose or function of the feature being described.2. Reference: Included statements that referred back to the written

documents or related some aspect of the pictures to the subjects'"mental model" (Gentner & Stevens, 1983) of the event. Also in-

cluded were statements that referred one picture to another for the

purpose of either c orroboration or discorroboration.

1 Details of this coding scheme can be obtained from the author.




3. Analysis: Included statements that related to the point of view,intentions, goals, or purposes of the pa inting or its artist. Also includedwere unprompted estimates about the dates of the paintings.

4. Qualification: Included statements that qualified other state-ments. For example, if after identifying one of the pictures as themost accurate, the subject added "But I do n't like the fact tha t thereis no w all," this would be coded as a qualification. Q ualifications alsoincluded statements about the limitations of historical knowledge or

the limitations in learning about the battle from the pictures (e.g.,"None of these pictures can tell us how the firing started.").

Subjects' inferences about their cognitive processes (e.g., "What Iam now going to do is think about this aspect"), statements abouttheir likes and dislikes (e.g., "The color in this one is more appeal-ing"), or comm ents that were not directly related to the task at hand(e.g., "This reminds me of something I saw when I was younger")were coded under a miscellaneous category. After protocols werecoded, miscellaneous statements were eliminated from later analyses.

Reliability in applying the c oding scheme was assessed between theauthor and a second rater unfamiliar w ith the study and blind to thebackgrounds of the subjects. Interrater agreement of 90% wasachieved. Disagreements were resolved through discussion.

Document evaluation. Protocol analysis for the written docu-ments followed a strategy similar to that described by Bereiter andBird (1985). Line by line coding, with the goal of producing acomputer simulation of cognitive performance, was abandoned infavor of more macroscopic coding, with the goal of identifyingheuristics that carry promising instructional implications. Accordingto Bereiter and Bird (1985, p. 134), the goal of such coding is not toarrive at a model that a ccounts for allaspects of cognitive perform ancebut "to identify content that is [(a)] valuable from the standpoint ofpeople competent in the subject.. .[and (b) ] . . . likely to be learnableby intended studen ts (allowing for simplifications, as long as these donot undercut the first criterion)."

Protocols were first reviewed informally and inductively, and cat-egories of heuristics were developed. These categories were the n testedby applying them to uncoded protocols. Categories were added,retained, or deleted on the basis of a decision rule that instances ofeach heuristic had to be present in at least four of the eight protocolsof historians. In all, three heuristics survived the full coding of theprotocols: c orroboration, sourcing, and contextualization.2 Reliabilityin coding for the presence or absence of heuristics was assessedbetween the author and a second rater blind to the backgrounds ofthe subjects. Interrater agreement was 96%.

Results

Identification Items

Students identified an average of 1.8 of the 12 terms (SD= .5). Historians identified an average of 7.1 of the terms (SD= 4). As expected, Americanists (M = 10, SD = 2.16) knewmore terms than non-Americanists (M = 4.25, SD = 3.3), p< .05, Mann-Whitney test.

Picture Evaluation

Historians' picture evaluation protocols contained signifi-cantly more statements overall (M = 52 for historians vs. M= 28 for students), F(\, 14) = 16.47, MSC = 594.14, p < .001.Table 2 shows the means and standard deviations for thenumber of statements in each of the four response categories.Significant differences were found in three of the categories:

analysis, /(14) = 2.65, p < .05; reference, f(14) = 2.15, p <.05; and qualification, t(7.55)

3= 5.21, p< .01 (for description,

p>.20).Although significant differences were not found in the

number of descriptive statements made by the two groups,the quality of these statements did differ, particularly in theextent to which the groups described features that had a

bearing on the historical accuracy of the paintings. Of a totalof 56 possible m entions of the key features in t he paintings (8historians/students x 7 features), historians noted 25 (45%)and students noted 4 {!%), t(H) = 4.18, p < .001.

With respect to the most accurate painting, the pictureselected most by historians was chosen least by students. Fou rhistorians but only one student saw the 1775 painting (whichshowed the minu temen fleeing the battlefield) as most accu-rate. Six students and two historians selected the 1886 paint-ing. Two historians and one student selected the 1859 paint-ing. Moreover, histo rians' selections came w ith qualifications,but only rarely did stude nts qualify their choice (see Table 3).Students seemed to base their selection on the quality of theartwork (especially its realism and detail); historians focusedon the correspondence between the visual representations andthe written documents. Historians tended to see a progressionin the pictures, with the latest depiction (the 1886 painting)representing the final stage in the evolution of a historicalmyth. N o student com mented on this mythical aspect. Sevenof eight historians placed the pictures in the correct chrono-logical sequence, whereas only one of eight students did so, p< .01, Fisher's exact test.

Historians who selected the same picture as most accurategenerally did so for similar reasons, such as the presence of awall in the 1859 depiction or the depiction of the colonistsfleeing in the 1775 painting. But a comparison of the re-sponses of a historian and a student shows how the same

answer can be undergirded by dramatically different—evencontradictory—reasons. Both H5 and S6 selected the 1859picture, both scored the same on the identification task (2 of12 terms correct), and both identified the same key featuresin their protocols—the lack of a wall and the presence of awoman. Table 4 displays excerpts from their protocols.

Within one paragraph, H5 referred to the written docu-ments five times—that Lexington was a "redcoat riot," areference to Document 4; that the British did not maintaintheir lines, another reference to Docum ent 4; that there wasfiring from buildings, a reference to D ocum ent 5; and thatneither the women nor the hill were mentioned in the set ofwritten d ocum ents. Her choice came with the qualificationsthat characterized the responses of historians—that the pic-tures do not show how the firing started and that the uncor-roborated details in the 1859 picture posed a problem. She

2 A fourth heuristic, the consideration of absent evidence, seemedto play an important role in the understanding of two historians bu twas eliminated because it did not meet the decision rule. For infor-mation about this heuristic, see Wineburg (1990).

3 Satterthwake's solution (Satterthwaite, 1946) was used to estimatedegrees of freedom because of heterogeneity of variances betweenstudents and historians in this category.




Table 2Means and Standa rd Deviations for the Number of Statements in Each of the FourResponse Categories in the Picture Evaluation Task

Subject

Historians (n = 8)

MSDAmericanists (n = 4)

MSD

Non-Americanists (n = 4)MSD

Students (n = 8)MSD

Description

20.88.5

17.34.6

24.310.7

16.88.1

Category

Analysis

7.64.4

7.05.1

8.34.3

2.82.8

Reference

15.58.3

13.510.7

17.56.0

7.17.3

Qualification

8.13.8

8.253.2

8.04.8

1.00.8

chose this depiction, but her choice was characterized byhesitancy and tentativeness.

S6 confidently chose the same picture for the very reasonsthat made H5 unsure and hesitant. H5 selected the 1859picture despite the hill, which she said did "n ot exist anywhereon Lexington Green." But for S6, this hill, a detail that hadnot been mentioned or alluded to in any of the writtendocuments, was a necessity. S6's selection seemed to havebeen guided by a general schema of battlefield encountersrather than, as in H5's case, specific details about the battlefrom the written documents.

Ranking of Documents

Figure 1 shows subjects' rankings of the trustworthiness ofthe written docum ents. K endall's coefficient of concordance

was used to measure the amount of agreement among eachgroup: for historians, W= .69, x 2 (7) = 38.92, p < .001; forstudents, W= .25, x 2 (7) = 13.83, p = .054. The amount ofagreement among historians of different specializations wasvirtually iden tical, W= .73 for Americanists and W= .74 fornon-Americanists. The average correlation (Spearman rho)between any two raters was compu ted (Hays, 19 72, p. 803),yielding r = .14 for students and r = .65 for historians.

Figure 1 shows no overlap between historians and studentson the highest ranked document: All eight historians rankedLt. Barker's diary (Document 4) as the most or second mosttrustworthy docu ment, whereas no student ranked it this high.The two groups also diverged in ranking Steinberg's (1963)

textbook excerpt (Document 7). All but one of the historiansrated this document last or next to last, in part because of thedocum ent's unsubstantiated claim that the minutem en "stoodtheir ground." On the other h and, three of eight students (SI,S2, and S4) ranked the textbook as the most or second mosttrustworthy document.

Heuristics

Three heuristics were identified from a review of the pro-tocols: (a) corroboration, the act of comparing documentswith one another; (b) sourcing, the act of looking first to the

source of the document before reading the body of the text;and (c) contextualization, the act of situating a document in

a concrete tem poral a nd spatial context. These heuristics canbe thought of as sense-making activities, for they help theiruser resolve contradictions, see patterns, and make distinc-tions among different types of evidence. The use of theseheuristics, however, does not guarantee success because thereis much personal leeway in deciding when they are appropri-ate and what conclusions to draw from them. What followsis an explanation of each heuristic and a description of howeach contributed to historical understanding.

Heuristic 1: Corroboration. Corroboration, in the wordsof Barbara T uchm an (1981), is the "great corrective" withoutwhich historical practice would "slip easily into the invalid"(p. 34). Stated as a heuristic, corroboration could be formu-lated as "Whenever possible, check important details against

each other before accepting them as plausible or likely."The differences between historians and students in the use

of corroboration can be seen best at two key junctures: (a) thedetermination of the size of the colonial force that assembledon Lexington Green, a nd (b) the evaluation of the statement(Document 7) that the minutemen "stood their ground."Table 5 displays the responses by historians and students atthese two juncture s. In evaluating the description of the 2 00 -300 minutemen in Document 4, seven of eight historiansreferred back to Docum ent 2. (Although Docum ent 2 did notdirectly state the size of the minutemens' force, it was signedby 34 men, a factor that these 7 historians took into consid-eration). In addition to noting the discrepancy of the "200-

300" estimate with this earlier information, other historiansdid such things as note the plausibility of this informationbased on the absence of precise details in the colonists' ac-count ( HI ); another (H6) generated a research question basedon finding out who the ad ditional "nonsigners" might be; an danother (H8) created a scenario to explain why a Britishofficer might inflate battlefield numbers. Students did littlewith this information. Only SI noted the discrepancy, but didnot follow up on it. S2 commented on an irrelevant feature.S3 misunderstood the phrase, and S5 noted this detail but didnot corroborate it with the earlier information. Four otherstudents offered no com ments on this information.




Fable 3

Analysis of Historians' and Students' Answers in the Picture Selection Task

SubjectPainting selectedas most accurate Qualifications of selected painting Characterization of 1886 painting

HistorianH I

H2

H3

H4

H7

S2

S3S4

S5

1775

1859

1886

1775

1886

1886

17751886

1886

S6S7S8

185918861886

Does not show British out of control asin Docum ent 4; shows them firing on

comm and, which also conflicts withDocument 4.Can't tell if minutemen are behind a

wall; British officer has arm raised,which conflicts with documents. Nopicture is concerned with getting it"exactly the way it was." (t: 30)

Uniforms are anachronistic

British troops are too orderly; not com-fortable with the position of the horseof British officer.

H5

H6

1859

1775

Hill and woman are problematic; nopicture tells how battle started.

No w all is shown.

Does not show how battle started, doesnot say who fired first or whatprompted the confrontation.

H8

StudentSI

1775

1886

Does not show impetuosity of theBritish.

Uniforms are anachronistic.

Pitcairn's horse is out of place.

"Courageous heroic colonists firingback in the face of an overwhelming

number of British." (t: 36)

"Made to glorify colonists." (t: 30)

" 'Hollywood' theatrical aspects."(t: 27-28)

"People who survived . . . would realizethat a picture like this was fairly fool-ish, having those m en in uniforms.So the generation that participated in[the battle] would have to be gone."(t: 33)

"No one is running away . . . a sense ofreal courage, or sacrifice. It makesme think of a commemorative kindof painting." (t: 47)

"Typ i ca l . . . motif, where the Britishare all lined up in rows, and the pa -triots are sort of in disarray . . . suf-fering perhaps a bit more." (t: 27)

"Idealized . . . the myth has been cre-ated." (t: 29)

"The painting looks clearer to me , itdepicts the scene more accurately, soit would be taking it more from anunbiased point of view." (t: 11)

"Just radiates an image that they are

determined to fight, that they areready to fight." (t: 15)

"Seems to m e to be depicted a little bitbetter, the men lined up better."(t: 19)

"They show a lot more of the Ameri-can people, at least they seem to bemore toge the r.... They are gatheredthere for a reason." (t: 23)

"Seems it's more hum an-like." (t: 21)

"Shows more of the people . . . [it's]more like telling a story because youcan see the people a little bit better."(t: 28)

Note. H1-H8 » Historians 1-8; S1-S8 = Students 1-8. Numbers in parentheses with the notation " t:" refer to the page of the transcript onwhich the quotation appears.

The responses in the second example follow a similar

pattern. The version of events in which the minutemen "stood

their ground" was rejected by seven of eight historians, all of

whom relied on discorroborating evidence from the other

documents. But among students, only SI mentioned other

accounts, saying that the textbook agreed with "the British

view" (t: 18),4

when, in fact, the London Gazette (Document

5) reported that the colonists "went off in great confusion."

S2 and S8 characterized this information as "the facts" (t: 7)

and "really straightforward" (t: 12). Five other students com-

mented on aspects that were not central to determining the

accuracy of this description.

4 This notation refers to the page of the transcript on which the

quotation appears.




Table 4Comparison of Historian 5 and S tudent 6 in Selecting 1859 Depiction as "Most Accurate Picture"

Subject Response

Historian 5

Student 6

You get the idea from all of the descriptions, whether American or British, that the Britishsoldiers . . . couldn't control themselves. It was a riot—y ou've heard of the Chicagopolice riot, well this was a redcoat riot. In all these [other paintings] the British havemaintained their lines, and you get the idea that they did not maintain their lines from

the accounts The thing is that none of these [paintings] tells us how the battlestarted It's possible that [the 1859 depiction] is the mo st accura te because theyseem to be firing from a building and there was some indication they would be firingfrom buildings. Howeve r, that was only from the British side Now given the factthat there are quite a few women in this, and no wom en are made mention in thedocum ent, that is something of a problem, but it also implies that it's kind of acitizenry army, and so that may be accurate, (t: 36)

[The 1859 depiction is most accurate] because it gives sort of .. . an advantageousposition, where they are sort of on a hill and I presume somewhere over here is a wall Iguess The minutem en are going to be all scrambled, going to be hiding behind thepoles and everything, rather than staying out here facing them . . . You know there's gotto be like a hill, and they're thinking they got to hide beh ind som ething, get at a placewhere they can't be shot besides being on low ground, and being ready to kill. Theirmentalities would be ludicrous if they were going to stand, like, here in [the 1775depiction], ready to be shot, (t: 22)

Note. Num bers in parentheses with the notation " t:" refer to the page of the transcript on which the quo tation appears.

Another m easure of corroboration was the numb er of timessubjects looked back to a previous document, because thepurpose of a lookback was usually to corroborate a statementor fact. Historians looked back more often than students, M= 6 for historians (« = 6)5 versus M = 2 for students (n = 8),p < .05, Mann-W hitney test.

Heuristic 2: Sourcing. Stated most simply, the "sourcingheuristic" could be formulated thus: "When evaluating his-torical documents, look first to the source or attribution ofthe document." Historians used this heuristic 98% of thetime; students used it 31 % of the time. In terms of readingthe attribution first (as opposed to reading the attribution

before reaching the end of the document), all eight historiansdid this at least once; only three of eight students did so, p <.025, Fisher's exact test.

To understand the importance of the sourcing heuristic, itis instructive to look at instances when it was no t used. S5read through Lt. Barker's diary (Document 4) thinking thatit was written by a minuteman, and only when she reachedthe attribution did she comment, "Oh my God, it's British"(t: 8). Likewise, at the beginning of Document 6 (Ezra Stiles'sdiary), S3 spent considerable energy trying to determine au-thorship, but it was not until the middle of the documen t thatshe figured out that it "might be from the colonists' point ofview" (t: 9). In such instances, the nonuse of the sourcingheuristic impeded the construction of meaning. Needlessprocessing was devoted to establishing local coherence orpronou n reference instead of piecing together a mental modelof the event. Because key information was contained in theattribution, the act of locating the document in a temporaland spatial context had to wait until after the reader reachedthe end of the document. It is no wonder that readers stillpuzzling about the authorship of documents did not engagein more sophisticated processes.

Often, the historians' first step in approaching the text wasto deploy the sourcing heuristic. By knowing a document'sauthor and the place and date of its creation, the historian

could develop hypotheses about what would be in the bodyof the document, the stance it might take, and its truthfulnessor accuracy. After reading the attribution for Document 3(but before beginning the body of the document), H6 ex-plained:

Knowledge of the source helps you unde rstand, helps you predictwhat you might find . . . how reliable it might be, or unrelia-ble. . . . Long before I was attracted to history, I loved readingthe historical novels of Howard Fast and found them absolutelygripping. And since then, I've heard a lot about his biases....You know he was a very strong and believing Comm unist . . .[but] by this time, I think that was no longer true, (t: 28)

Already, H6 had generated a set of hypotheses to take tothe text. At the most basic level, she identified the genre ofthe text and knew tha t it should be regarded differently froma primary source. Second, she knew when the document waswritten and thus could place it in a temporal context. Third,she recognized the author and was familiar with the biases hemight take to his subject. H6 did not know whether or notthe auth or's ideology would be im printed on this text, but itremained in her mind as a distinct possibility. In this instance,the background knowledge about au thor an d text genre acti-vated by this heuristic provided the historian with a richframework for encoding this docum ent.

The sourcing heuristic is crucial in providing such antici-

patory frameworks for the subsequent encoding of text. Inaddition to cueing readers about authorship, the sourcingheuristic alerts them to the genre of text and activates a set oftextual schemata (Anderson, 1977) that help readers weightextual information and determine its probity (Gottschalk,

5 Lookbacks were not recorded for the first two historians inter-viewed, H4 and H8. On the basis of data collected from these twohistorians, the observation and recording of lookbacks was integratedinto the research protocol. Lookbacks were subsequently recorded forthe other six historians and for all eight students.




• • • • • A

• •

ooo

OCX)

A

OO O

AAAA

O

A

O

A

O

A

O

O

A - Historians

A

O

AAA

OO O

A A

O O

AA

O

o

A

O

O O O

AA

O• *f

AA A

O

A

A

O O

O - Students

ooo

o

A

AAAA

AA

OO O

A

O

A

A

A

A

OO O

AA

OO

A

O

A

OO

A

OO

A

OO

AA A

O

AAA

OO O

O

O O

O

A

O

AA A

OO

AAAA

O

2

3

4

5

6

7

Bark er Stiles Lister Mulliken Warren Nunn Fast Steinberg

(Doc. #4) (Doc. #6) (Doc. #8) (Doc. #2) (Doc.#l) (Doc. #5) (Doc. #3) (Doc. #7 )

Document

Figure 1. Rankings of the trustworthiness of the eight written documents. (Doc. = Document.)

1958). The following are H4's initial comments after readingthe source of Docum ent 7:

So I look an d read "The United States: Story of a Free People, ahigh-school textbook," "I n April 1775" right off the bat I knowwhat this is going to be—the narrative form, something about"April 1775." Had i t been "i n April" I might've taken it as moreof a precise thing, from somebody like Gross.6 Gross would begoing day by day. So starting ou t with "April 1775" is startingwith such a broad thing that I'm not surprised it's a text[Textbooks] don't get into a lot of detail and they tend to be alittle bit patriotic, an d second, they also tend to be very politicaland tend to give information that can be answered in a multiple-choice exam. So I expect to find this all laid ou t very clearly forme ; something I can give a quick answer to back. It will tend toblur over . . . things that aren't that clear an d make them clearbecause you simply can't be that vague on a multiple-choice test.

The confirmation that the passage comes from a textbookleads to the activation of a TEXTBOOK SCHEMA thatcarries with it a broad set of expectations. As Figure 2 shows,H4's schema for a U.S. history textbook provides an elaborateframework for the encoding of this document. In this case,almost all of the nodes of this schema became instantiatedwith specific textual material.

Heuristic3: Contextualization. Stated in its simplest form,the contextualization heuristic would read: "When trying toreconstruct historical events, pay close attention to when theyhappened and where they took place." The "when" of thisheuristic refers to the act of placing events in chronological

sequence. This can be as simple as placing X before Y or assophisticated as determining how long it took for a report

filed in Boston on April 26, 1775, to reach the offices of aLondon newspaper—a computation made by H3 when read-ing the excerpt from the London Gazette (Document 5). Thecontextualization heuristic trains historians' attention on whatprecedes and follows events, on how long they lasted, and onthe amount of time between their occurrence and their re-cording by witnesses (cf. Winks, 1968). The "where" of thisheuristic is concerned with situating events in concrete spacesand determining the conditions of their occurrence—issuesof geography, weather, climate, a nd landscape.

Two points in the written docum ents highlight the central-ity of this heuristic for reconstructing historical events. Thefirst example comes in Document 2, when the minutemennote that they were informed at "about one or two o'clock in

the mo rning" that the British were coming, but then go on toreport that they were dismissed by their captain only to bemustered again 3 hr later. Among students, this informationevoked no special response. Only S3 paused, saying: "I'mtrying to figure out if the one or two in the morning is stillApril 19" (t: 4) but she did not elaborate. Of the other students,only S7 used this information to attempt a reconstruction ofthe conditions that prevailed tha t m orning: "I can kind of see

6 Here H4 refers to the monograph by Robert A. Gross (1976),The Minutemen an d Their World.




Table 5

Use and No nuse of the Corroboration Heuristic by Historians and Students

Subject Response

Example 1: "We saw a num ber of people, I believe between 200 and 300, formed in a comm on in the middle of the town ." (Document 4)

HistorianHI

H2

H3

H4H5

H6

H7

H8

StudentSI

S2

S3S5

"He sees between 200 and 300 formed in a comm on in the middle of town; [the

colonists] say, they don't say how many were there It just says the deposition of34 people. I don't have a sense of how many people were there so he might be rightand because of the absence of the information in the second docume nt, the estimateof 200 to 300 m ight be correct." (t: 16)a

"The 200 to 300 is a larger body than the colonists indicated in their two d ocum ents."(t: 15)

"Certainly a figure never mentioned in any of the American documents ... a largernumber than I would've imagined." (t: 12)

"He do esn't know exactly how many, b ut it's a sense of a large crowd." (t: 8)"Which is quite a different nu mber from the num ber I suspected from the other

accou nt." (t: 16)"So when we go back and we know there was a deposition of 34 so, you know, again

we'd want to know what those other people—were they all minutem en?" (t: 19)"Two hundred to 300 seems to be quite a lot, perhaps more tha n w as suggested

earlier." (t: 12)"Certainly a very different num ber from the docume nt of the minutemen , but we know

. . . that w henever battles are being described it's always in the interest of the autho r

to describe the enemy as being greater because it shows how m uch more valor youhave. So it's muc h m ore imp ortant to defeat 200 or 300 ." (t: 13)

"So which one do you believe, 200 or I forget how many in the other docum ent, howmany in the other docum ent, [it] said there were 300?" (t: 10)

"Inexact numbers, but if he had said between 200 and 300, it might have lent morecredence to it, but he said 'I believe.'" (t: 14)

"So does that m ake 500?" (t: 10)"Before he said hundreds, th at's right." (t: 9)

Example 2: "The 'rebels' were ordered to disperse. They stood their ground." (Docum ent 7)

HistorianHI "It's not clear that they were ordered to disperse, the depositions don't indicate that, the

British accounts do indicate that. Let me check back to Barker [Docum ent 4] for asecond—yeah, Barker doe sn't even say there was any dispersal." (t: 29)

H2 "Every thing I've read so far, it sounded like it was rather confusing and raggle-taggle.

But 'they stood their ground.' Now this really is giving motive and interpretation."(t: 25)H3 "Th at's contrary to the other docum ents you gave me which indicated, in fact, that

they were dispersing." (t: 21)H4 "Th at's not clear from the docum ents, there seems to be some confusion about whether

they actually stood their ground or started moving." (t: 15)H5 "The American accounts said they weren't even there yet 'Stood their ground' is a

very different connotation from being just a bunch of rebels who w on't disperse, youknow, rowdies out the re." (t: 31)

H6 "T hat's mayb e one of the few things here that is said that certainly is boldly inconsist-ent with what we've heard.... The accounts who saw this mentioned that they had,in fact, dispersed when they were ordered to disperse." (t: 40)

H7 "Here we have a sense of purpose." (t: 20)H8 "Non e of the stories we saw said th a t! .. . What, this is the seventh docume nt? Not one

of those six docum ents said they 'stood their ground .'" (t: 23)Student

51 "So tha t's the only part where this textbook agrees with the British view because they

say they 'stood their ground' but they also say they were fired on." (t: 18)52 "It seems in a way just reporting the facts, 'The rebels were ordered to disperse. They

stood their ground,' just concise, journalistic in a way, just saying what happenedthere." (t: 29)

54 "It doesn't sound right, the English were firing and the rebels quote unquote stayedhere."(t: 16)

55 "Now this is the first docum ent that makes it sound like they really had a right to dothat.... Nice short sentence to get the point across." (t: 19, 20)

56 "Obviously they stood their ground or else they wou ldn't have gotten shot." (t: 19)57 "So these are the same guys as in the other accou nt." (t: 9)58 "T ha t's really straightforward." (t: 22)

Note. H1-H8 = Historians 1-8; S1-S8 = Students 1-8." Notation refers to the page of the transcript on w hich the quotation appears.




High-SchoolHistory Textbooks

Aspects Purposes

Broad Time PoliticalFrame Focus

To instill To make it easyPatriotism to write multiple-

/ choice items

nothing explicit!but nothing j

contradictory

"embattledfanners"

I "patriots of all INew England"

To treatindeterminate

events as certain

inferred reference

specific reference

Figure 2. A partial representation of Historian 4's textbook schema instantiated with material fromthe textbook excerpt (Document 7).

the battle scene, troops, it's kind of dark, lots of gunshots" (t:

3).When reading history, it makes sense for readers to note

aspects of time and place. But the deployment of the contex-tualization heuristic goes beyond "taking no te." For example,H8 used the information about time in Document 2 toreconstruct the intelligence network of the minutem en, mak-ing inferences about when the colonists must have learnedthat the British were setting out from Boston (t: 8). HI usedthis information to reconstruct what eyewitnesses would havebeen able to make out in the darkness of this New Englandmorning, a scenario that led him to dou bt the quality of thistestimony (t: 7-8). H3 and H4 took a different tack. H3 usedthe heuristic to reconstruct the fears that must have grippedthe minutemen as they waited for the British to arrive, notinghow this 3-hr waiting period provided the right conditions forthe minutemen to overreact (t: 7). Similarly, H4 connectedthe waiting period to feelings he himself experienced at ananti-apartheid demonstration he attended several years ear-lier:

They woke us up in the middle of the night saying that theyheard that the KKK was about to come down and beat everybodyup and end this thing. And this went on for about an hour and

they finally sent someone up to the Greek Theater and foundout it was just a fraternity. But again, what I'm remembering isall of these people suddenly woken up in the middle of the night,in the dark, huddled together, and feeling very nervous about it,with some people stirring up their courage saying, "We're notgoing to let them do this to us" type of thing. I wonder howmuch of that might have been going on in [the minutemens']lives What other sources can I get that try to get at that—can I get their own personal letters to see if that's going on? (t:7)

From personal experience, H4 knew that waiting under acloud of danger is "enough time to let your fears run awaywith you" (t: 7). The scenario he constructed is a clue to how

historians "find" new research questions (cf. Getzels, 1979):Could he find new documents that would attest to the colo-nists' fears d uring this w ait?

It is interesting to note an additional way this informationwas used as historians proceeded through the task. Threehistorians (but no students) commented on the phrase "bay-onets glittering in the sunshine" in Document 3. H3's com-ment was most explicit: "Five o'clock in the morning, evenwith no daylight savings time, I don't—what's the date—April, no way the sun would be up at 5 o'clock in themorning" (t: 10). By carefully situating events in time, thesethree historians were able to use information from Docum ent2 to call into question the veracity of this later account.

Another key instance in which the contextualization heu-ristic came into play was when historians encountered Lt.Barker's unexpected admission (Document 4) that his men,disobeying orders, charged the colonists and put them toflight. This informa tion set off a kind of search proced ureamong four historians (but not among any students): Whatmight be the cause of this uncharacteristic loss of order bycrack British troops? These historians found part of the answerby using the contextualization heuristic to erect scenariosabout "cold and wet" men (H5, t: 17) in "waterlogged shoes"

(H2, t: 14) who were "tense on the m arch" (H8, t: 14) throughcold Massachusetts marshes. These historians then consideredthe effect this might h ave on the soldiers, a consideration tha thelped explain the British officer's admission that his menran amok. H1 's response to this information from Barker wasthe m ost elaborate:

One has to try to put themselves in the minds and the bodies ofthe British. They're starting out early in the morning, they mustbe walking quickly; I'd have to figure out how many milesbetween the barracks where [the British commander] and histroops left and how fast they were walking, because that ...might help explain if they were really fatigued and then the




adrenaline started to flow in the battle, that they may have lostcontrol. They may have been angry—a whole range of otherkinds of things. So the physical dimensions of when they left,the fact that they had to go through a river up to the middles oftheir bodies means tha t they were wet, I suppose, the entire time.So if I wanted to get a sense of the veracity of these accounts,then I would want to pick up on some of these details, (t: 17-18)

Discussion

The findings of this study are predictable but not trivial: Agroup of historians read a set of historical doc umen ts in m oresophisticated ways than a group of high school students. Butto say that historians "did better" simply because they werehistorians is to substitute ascription for explanation. As Larkinet al. (1980) noted: "We 'explain' superior problem-solvingskill by calling it 'talent,' 'intu ition,' 'judgment,' and 'imagi-nation.' Behind such words... lies a reality we must discoverif we are to understand expert performance" (p. 1335). Themost pressing questions for a psychology of school subjectsrevolve around understanding what enabled these historiansto take a set of fragmented and contradictory docum ents andbuild a complex understanding of the events at Lexington:What did they do , think, know, and believe that allowed themto see patterns where a group of able high school seniors sawonly a collection of details?

My approach to these questions has been somewhat differ-ent from previous studies using the expert/novice design.Most theorizing about problem solving has rested on well-structured problems such as those used by Chi et al. (1981)in their pioneering work on physics problem solving. Theseresearchers explained differences betw een novices and ex perts,in part, as differences in the number and organization of

problem templates (preset mental structures for solving prob-lems) possessed by the different groups. Successful perform-ance was based first on possessing the appropriate problemtemplate, and then consisted of the "activation and confir-mation of an appropriate principle-oriented knowledge struc-ture Once activated, the schema itself specifies further

(schema-driven) tests for its appropriateness" (p. 149).Regarding this study, can one say that historians possessed

and activated an appropriate principle-oriented knowledgestructure, perhaps a "Lexington" schema? Probably not, be-cause the non-Americanists knew little about Lexington priorto the task, and even the Americanists, who had read mono-graphs about this period and had taught U.S. history to their

students, had never considered aspects of this battle in thedetail demand ed by this task. Perhaps, then, one can say thatthese historians possessed a more general "RevolutionaryBattle" schema that helped them wend their way throughthese documen ts. But even this seems unlikely, as Lexingtonwas not Saratoga, not Trenton, not Oriskany, and certainlynot Yorktown.

In this context, exp ertise seemed to rest less on bringing th eright problem schema to the task and more on constructing acontext-specific schema tailored to this specific event (cf.Spiro, Vispoel, Schmitz, Samarapungavan, & Boerger, 1987).There is no "problem isomorph" for the Battle of Lexington;

what happened on that April morning was a unique combi-nation of circumstances and conditions that will never berepeated. Certainly, many schemata were activated by histo-rians in the course of reading these documents. But theactivation of schemata did no t lead to the type of automatic,schema-driven processing described in the literature on phys-ics problem solving. To be able to reason thoughtfully about

the accuracy of these do cumen ts, historians needed to build—node by node—an elaborate model of this event.

In identifying and isolating some of the cognitive processesused by historians and students, I have run th e risk of decon-textualizing them, that is, making them appear as separateprocesses and responses rather than as manifestations of abroader, more sweeping set of beliefs. Possibly, two fairlygeneral but very different orientatio ns to the task underlie thedifferences between historians and students. For historians,the task as posed seemed unreasonable to begin with: Indeed,the question "Which painting most accurately depicts whathappened at Lexington?" elicited comments such as "Whatdid go on?" and "This can't be do ne." Nonetheless, historianssettled into doing what historians do—they puzzled aboutdiscrepancies, they compared the pictures with the writtendocuments, they corroborated and discorroborated key fea-tures, and they tried to represent what could and what couldnot be know n. For h istorians, the picture evaluation task wasan exercise in exploring the limits of historical knowledge.The end result was more a suggestion than an answer, morea forced choice from flawed alternatives than a committeddecision executed with resolve.

Students, on the other hand, generally sized up the picturesand made a selection without regret or qualification. Forthem, the picture evaluation task rarely entailed shiftingthrough the written documen ts, puzzling about the inten tionsor goals of the artist, or reflecting back on what they had read .

Rather, students responded as if the three pictures were anal-ogous to three options on a multiple-choice test. Therefore,to postulate that students believed in a single "correct answer"helps explain why they did not qualify their answers orcompare the pictures with the written docum ents. To select apicture without "looking back," one must believe that a two-dimensional image can capture the multilayered, contradic-tory accounts of the encounter at Lexington.

Similarly, the use and nonuse of the sourcing heuristiccould be seen as a reflection of differences in each group'sconception of text—differences that addressed not just whata text says, but what a text is. In reading texts from top tobottom, from the first word in the upper-lefthand corner tothe last word in the bottom -righthand corner, students seemedto view texts as vehicles for conveying information in whichthe attribution was just the last thing to be read, one more bitof information to be added to the other bits that had beengathered. But for historians, who used the attribution to erectelaborate scenarios about authors and the circumstances ofdocument generation, the attribution was not another bit ofinformation, but the "bit" from which all else emanated.Historians seemed to view texts not as vehicles but as people,not as bits of information to be gathered but as social ex-changes to be understood. Viewed in this light, the sourcingheuristic is not really a rule of thumb or problem-solving




strategy as much as it is the manifestation of a belief systemin which texts are defined by their authors.

When texts are thus viewed, what is said becomes insepa-rable from who says it. For some students, however, the detailsof authorship seemed incidental to considerations of a docu-ment's worth. One can see this contrast in perspectives byjuxtaposing the responses of S6 and HI to the excerpt from

Howard Fast's (1961) novel (Docum ent 3). When S6 initiallyevaluated the excerpt, he knew something was wrong: "Youcan't really believe exactly what they're saying, it's going tobe— the details are going to be off' (t: 12). But by the tim e hereached the sixth document, elements from Fast's accounthad already been integrated into his understanding of theevents at Lexington. In other words, his reservations aboutFast fell away as he moved through the task, and details fromFast's account were remembered even when their author wasnot.

Contrast this with Hi's reaction to the claim in Document8 that the colonists were drawn up in "regular order." En-countering this claim, HI remembered that another one ofthe accounts described the battle formation. He then flippedthrough the documents until he reached Document 3, andthen broke out in laughter: "Oh, that's from Fast! Forget it! Ican't hold on to Fast, I can't do that. But it's funny, it stuckin my mind" (t: 18). This is the opposite of what occurredwith S6. A detail is first remembered, but the historian cannotremember its source. This recognition sends the historiansearching for the source of this detail, and, when reunitedwith its author, the detail is rejected. The reason is that thehistorian knows that there are no free-floating details, onlydetails tied to witnesses, and if witnesses are suspect, so aretheir details.

Differences in the use of the corroboration heuristic couldbe viewed as reflecting differences in beliefs abo ut the natur e

of historical evidence. For historians, corroboration was in-dispensable because every account was seen as reflecting aparticular point of view. The question pu t by the historian tothe source was not "Is the source biased?" but "How does asource's bias influence the quality of its report?" Students, onthe other han d, seemed to view bias as binary, an attribute ofsome texts but not of others. For example, S7 was confusedby Barker's account and wanted to withhold judgment untilhe could find an "unbiased report" (t: 6). Two other studentslocated this "unbiased report" in the textbook excerpt. ForS4, the textbook was not slanted like the other accounts; inhis words, it was "straight information" (t: 18). For S2, thetextbook was "just reporting the facts" (t: 14). For thesestudents, the textbook— not any of the eyewitness accounts—constituted the " primary " source.

Surely background knowledge contributed to the differ-ences described here. However, the concep tualization of back-ground knowledge as discrete names, dates, and facts, theform of background knowledge measured by the pretest inthis study, is certainly too narrow. H5 and S6 knew the samenum ber of answers on this pretest (2 or 12), bu t when it cameto evaluating pictures (see Table 4), H5, a specialist in Japa-nese history, ended up with an elaborate und erstanding of theevents at Lexington, whereas S6 floundered. Thrown intounfamiliar territory, H5 could find routes and pathways be-

cause she knew how to use the disciplinary equivalent of acompass. In some respects, her expertise lay not in what sheknew, but in what she was able to do when she did notknow—a type of knowledge not measured by conventionalpaper-and-pencil assessments (e.g., Ravitch & Finn, 1987).

What seemed to distinguish historians from students wasnot whether they could identify Fort Ticonderoga or the

Townshend Acts but broader, m ore sweeping ways of know-ing and thinking about historical evidence, or what Schwab(1978) called "syntactic knowledge"—knowledge of how toestablish warrant and determine the validity of competingtruth claims in a discipline. Such knowledge may have con-tributed to the differences in each group's rankings of thewritten accounts. Historians knew to regard a diary entrydifferently from a deposition written for promulgation; theyknew to weigh an account written a day after the eventdifferently from one written 7 years later. Students, in somerespects, may have simply appealed to different syntacticknowledge—a knowledge system in which textbooks serve asthe arbiters of historical questions.

In making these comments, my intention is not to dismissthe differences between non-Americanists, who lacked de-tailed knowledge about the Am erican Revolution, and Am er-icanists, who had taught this period and were intimatelyfamiliar with it. But in the overall context of this task, anexercise aimed at reconstructing what happened on a partic-ular m orning in history, historians without large amo unts offactual knowledge could make do with what they had. Thisresult is partially a function of the task, for had the goal beendifferent, had the medievalist or the Japanese specialist beenasked to place the events at Lexington into the broadersociocultural milieu of the 18th century, surely their lack offactual knowledge would have shown through more clearly.

Conclusion

Like historical knowledge, the findings of this study areindeterminate. Given the inherent problems of expert/novicecomparisons, it is impossible to determine whether the differ-ences described here are a function of differences in belief orsomething else entirely. It can be argued that experts andnovices do not represent different stages on the same contin-uum because they are not drawn from the same populationin the first place. In other words, the differences betweenstudents and historians may, in fact, be evident in certaincognitive activities but have their roots elsewhere. Thus, anyfacile claim about the roots of these differences should beavoided.

In the meantime, it can be said with some assurance thatable high school students can know a lot of history but stillhave little idea of how historical knowledge is constructed. Itis doubtful that teaching these students m ore facts about theAmerican Revolution w ould help them d o better on this taskwhen they remain ignorant of the basic heuristics used tocreate historical interpretations, when they cannot distinguishamong different types of historical evidence, and when theylook to a textbook for the "answer" to historical questions—even when that textbook contradicts primary sources fromboth sides.




Can we improve students' ability to deal with the contra-dictory and complex nature of history by teaching them touse the strategies and heuristics exhibited by historians? Ordo these cognitive processes make sense only when under-girded by a broader set of beliefs about historical inquiry? Inlight of the attention being paid to the history curriculum(e.g., Bradley Commission on History in Schools, 1988; Gag-

non, 1988; Ravitch & Finn, 1987) and a host of unsubstan-tiated claims about how students learn this subject (e.g.,Ferrell, 1990; Howard & Mendenhall, 1982), the questionsraised by J. Carleton Bell in 1917 have lost none of theirurgency.

References

Anderson, R. C. (1977). The notion of schemata and the educationalenterprise. In R. C. Anderson, R. S. Spiro, & W. E. Montagne(Eds.), Schooling and the acquisition of knowledge (pp. 415-431).Hillsdale, NJ: Erlbaum .

Barzun, J., & Graff, H. F . (1962). Th e modern researcher. New York:Harcourt, Brace, & World.

Bell, J. C. (1917). Editorial: The historic sense. Journal of EducationalPsychology, 8, 317-318.

Bennett, P. S. (1970). What happened at Lexington Green? MenloPark, CA: Addison-Wesley.

Bereiter, C , & Bird, M. (1985). Use of thinking aloud in identificationand teaching of reading comprehension strategies. Cognition andInstruction, 2, 131-156.

Bradley Comm ission on History in Schools. (1988). Building a His-tory Curriculum. New York: Educational E xcellence N etwork.

Cantor, N . F., & Schneider, R . I. (1967). How to study history. NewYork: Crowell.

Carey, S. (1985). Conceptual change in childhood. Cambridge, MA:MIT Press.

Carr, E. H. (1962). What is history? New York: Knopf.

Chapm an, L. J., & Chapm an, J. P . (1968). Illusory correlation as anobstacle to the use of valid psychodiagnostic signs. Journal ofAbnormal Psychology, 76, 271-280.

Chase, W. G., & Simon, H . A. (1973). Perception in chess. CognitivePsychology, 4, 55-81.

Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorizationand representation of physics problems by experts and novices.Cognitive Science, 5, 121-152.

Chi, M. T. H., Glaser, R., & Farr, M. J. (1988). The nature ofexpertise. Hillsdale, NJ: Erlbaum.

Clark, G. K. (1967). Th e critical historian. New York: Basic Books.Comm ager, H. S. (1966). Th e nature and stud y of history. Columbus,

OH: Merrill.Dana, R. H., Jr. (1877). A British officer in Boston. Atlantic Monthly,

39 , 389-401.Dexter, F. B. (Ed.). (1901). The literary diary of Ezra Stiles. NewYork: Charles Scribner.

diSessa, A. A. (1985). Learning about knowing. In E . L. K lein (Ed.),Children an d cognition (pp. 9 7-12 4). San Francisco: Jossey-Bass.

Elstein, A. S., Shulman, L. S., & Sprafka, S. A. (1978). Medicalproblem solving: An analysis ofclinical reasoning. Cambridge, MA:Harvard University Press.

Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbalreports as data. Cambridge, MA: MIT Press.

Fain, H. (1970). Between philosophy and history: The resurrection ofspeculative philosophy of history w ithin the analytic tradition.Princeton, N J: Princeton U niversity Press.

Fast, H. (1961). April morning. New York: Crown.Ferrell, R. H. (19 90). History and the public schools. Organization of

American Historians Newsletter, 18 , 18, 21.Force, P. (1847). American archives. Washington, D C: Congress.Frisch, M. (1989). American history and the structures of collective

memory: A modest exercise in empirical iconography. Journal ofAmerican History, 75, 1130-1155.

Gagnon, P. (1988, Novem ber). Why study history? Th e Atlantic, pp .

43-66.Gardner, H. (1985). The mind's new science. New Y ork: Basic Books.Gentner, D., & Stevens, A. L. (1983). Mental models. Hillsdale, NJ:

Erlbaum.Getzels, J. W. (1979 ). Problem finding: A theoretical no te. Cognitive

Science, 3, 167-172.Glaser, R. (1984). Education and thinking: The role of knowledge.

American Psychologist, 39 , 93-104.Gottschalk, L. (1958). Understanding history: A primer of historical

method. Chicago: University of Chicago Press.Gray, W. (1959). Historian's handbook. Boston: Houghton Mifflin.Greeno, J. G., Magone, M. E., & Chaiklin, S. (1979). Theory of

constructions and set in problem solving. Memory & Cognition, 7,445-461.

Gross, R. A. (1976). The minutemen and their world. New York: Hilland Wang.

Hays, W. L. (1972). Statistics for the social sciences. New York: Holt,Rinehart & Winston.

Howard, J., & Mendenhall, T. (1982). Making history come alive.Washington, D C: Council for Basic Education.

Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment underuncertainty: Heuristics and biases. Cambridge, England: Cam-bridge University Press.

Kuipers, B., Moskowitz, A. J., & Kassirer, J. P. (1988). Criticaldecisions under uncertainty: Representation and structure. Cogni-tive Science, 12 , 177-210.

Larkin, J., McDermott, J., Simon, D., & Simon, H. (1980). Expertand novice performance in solving physics problems. Science, 208,1335-1342.

Lesgold, A. M., Feltovich, P. J., Glaser, R., & W ang, Y. (1981 ). Th eacquisition of perceptual diagnostic skill in radiology (TechnicalReport No. PD S-1). Pittsburgh: University of Pittsburgh, L earningResearch and Development Center.

Lister, J. (1931). Concord fight. Cambridge, MA: Ha rvard U niversityPress.

McCloskey, M. (1983). Naive theories of motion. In D. Gentner &A. L. Stevens (Eds.), Mental models (pp. 299-324). Hillsdale, NJ:Erlbaum.

McDowell, B. (1967). The Revolutionary War. Washington, DC:National Geographic Society.

Murdock, H. (1925). Th e nineteenth ofApril 1775. Boston: HoughtonMifflin.

Nesher, P., & K atriel, T. (1977). A semantic analysis of addition andsubtraction problems in arithmetic. Educational Studies in Math-

ematics^, 251-269.Newell, A., & Simon, H. A. (19 72). Human problem solving. Engle-wood Cliffs, NJ: Prentice-Hall.

Nisbett, R. E., & Ross, L. (1980). Human inference: Strategies andshortcomings in human inference. Englewood Cliffs, NJ: Prentice-Hall.

Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we canknow: Verbal reports on mental processes. Psychological Review,84 , 231-259.

Ravitch, D. R., & Finn, C. E. (1987). What do our 17-year-olds know?A report on the first na tional assessment of history and literature.New York: Harper & Row.

Resnick, L . B. (1982). Syntax and semantics in learning to subtract.




In T. Carpenter, J. Moser, & T. A. Romberg (Eds.), Addition andsubtraction: A cognitive perspective (pp. 136-156). Hillsdale, NJ:Erlbaum.

Resnick, L. B. (1985). Education an d learning to think. Unpublishedmanuscript, University of Pittsburgh, Learning Research and De-velopment Center.

Satterthwaite, F. E. (19 46). An approxim ate distribution of estimatesof variance comp onents. Biometrics Bulletin, 2, 110-114.

Sawtell, C. C. (1968). The nineteenth of April, 1775: A collection offirst hand accounts. Lincoln, MA: Sawtells of Somerset.

Schoenfeld, A. H. (1985). Mathematical problem solving. San Diego,CA: Academic Press.

Schwab, J. J. (1978). Education and the structure of the disciplines.In I. Westbury & N. J. Wilkof (Eds.), Science, curriculum, andliberal education (pp. 229-272). Chicago: University of ChicagoPress.

Shafer, R. J. (1969). A guide to historical method. Homewood, IL:Dorsey.

Sleeman, D. H. (1984). Basic algebra revisited: A study with 14-year-

olds. Unpublished m anuscript, Stanford University, H euristic Pro-gramming Project, Stanford, CA.

Sleeman, D., Putnam , R., Baxter, J., & Kuspa, L. (1986). Pascal andhigh school students: A study of errors. Journal of EducationalComputing Research, 2, 5-23.

Spiro, R. J., Vispoel, W. P., Schm itz, J. G., Sam arapungavan, A ., &

Boerger, A. E. (1987). Knowledge acqu isition for application: Cog-nitive flexibility and transfer in complex content domains. In B.K. Britton & S. W. Glynn (Eds.), Executive control processes inreading (pp. 177-199 ). Hillsdale, NJ: Erlbaum.

Steinberg, S. (1963). The U nited States: Story ofa free people. Boston:Allyn and Bacon.

Tourtellot, A. B. (1959). Lexington and Concord: The beginning ofthe war of the American Revolution. New York: Norton.

Tuchman, B. (1981). Practicing history. New York: Knopf.Voss, J. F., Vesonder, B. T., & Spilich, G. J. (19 80). Text generation

and recall by high-knowledge and low-knowledge individuals. Jour-

nal of Verbal Learning an d Verbal Behavior, 19 , 651-667.Waddill, P. J., McD aniel, M. A., & Einstein, G. O. (1988). Illustrators

as adjuncts to prose: A text appropriate processing approach.Journal of Educational Psychology, 80, 457-464.

Wilensky, R. (1983). Planning and understanding: A computationalapproach to human reasoning. Reading, MA: Addison-Wesley.

Wineburg, S. S. (1989). Rem embrance of theories past. EducationalResearcher, 18 , 7-10.

Wineburg, S. S. (1990). Historical problem solving: A study of thecognitive processes used in the evaluation of documentary evidence.Unpublished doctoral dissertation, Stanford University, Palo Alto,CA.

Winks, R. W. (Ed.). (1968). Th e historian as detective. New York:Harper and Row.

Appendix

Set of Written Documents Used in This Study

(The only reference information provided to subjects was the italicized descriptions at the end of the document passages. Material inparentheses following document passages is provided for bibliographic purposes.)

Document 1

In 1775, Benjamin Franklin was the colonial representative inLondo n. After the events in Lexington and Concord, the M assachusettsProvincial Congress put together 21 sworn depositions about the eventsan d sent them to Franklin with the following cover letter:

To the inhabitants of Great Britain: In Provincial Congress, Water-town, April 26, 1775

Friends and fellow subjects: Hostilities are at length commencedin the Colony by the troops under command of General Gage; andit being of the greatest importance that an early, true, and authenticaccount of this inhuman proceeding should be known to you, theCongress of this Colony have transmitted the same, and from wantof a session of the honorable Continental Congress, think it properto address you on the alarming occasion.

By the clearest depositions relative to this transaction, it will appear

that on the night preceding the nineteenth of April instant, ... theTown of Lexington ... was alarmed, and a company of the inhabit-ants mustered on the occasion; that the Regular troops, on their wayto Concord, marched into the said town of Lexington, and the saidcompany, on their approach, began to disperse; that notwithstandingthis, the regulars rushed on with great violence, and first beganhostilities by firing on said Lexington Company, whereby they killedeight and w ounded several others; that the Regulars continued theirfire until those of said company, who were neither killed norwounded, had made their escape.

These, brethren, are marks of ministerial vengeance against thiscolony, for refusing, w ith her sister colonies, a subm ission to slavery.But they have not yet detached us from our Royal Sovereign. We

profess to be his loyal and dutiful subjects, and so hardly dealt with

as we have been, are still ready, with our lives and fortunes, to defendhis person, family, crown, and dignity. Nevertheless, to the persecu-tion and tyranny of his cruel ministry we will not tamely submit;appealing to Heaven for the justice of our cause, we determine to dieor be free. Joseph Warren [President pro tern](Document reprinted in Bennett, 1970)

Document 2We Nathaniel Mulliken, Philip Russell, [followed by the names of

32 other men present on Lexington Green on April 19, 1775],... allof lawful age, and inhabitants of Lexington, in the County of Middle-sex, . . . do testify and declare, that on the nineteenth of April instant,about one or two o'clock in the morning, being informed that... abody of regulars were marching from Boston towards Concord, . . .we were alarmed and having met at the place of our company'sparade [Lexington Green], were dismissed by our Captain, JohnParker, for the present, with orders to be ready to attend at the beatof the drum, we further testify and declare, that about five o'clock inthe morning, hearing our drum beat, we proceeded towards theparade, and soon found that a large body of troops were marchingtowards us, some of our com pany were coming up to the parade, andothers had reached it, at which time the com pany began to disperse,whilst our backs were turned on the troops, we were fired on by them,and a number of our men were instantly killed and wounded, not agun was fired by any person in our company on the regulars to ourknowledge before they fired on us, and they continued firing until wehad all made our escape. Lexington, April 25 , 1775, Nathaniel Mul-liken, Philip Russell, [and the other 32 m en] [Duly sworn to by 34




minutemen on April 25 before three justices o f the peace]

(Docum ent reprinted in Sawtell, 1968)

Document 3

Major Pitcairn screamed at us: "Lay down your arms, you lousybastards! Disperse, you lousy peasant scum!" . . . At least, those werethe words that I seem to remem ber. Others remem bered differently;but the way he screamed, in his strange London accent, with the

motion and excitement, with his horse rearing and kicking ... withthe drums beating again and the fixed bayonets glittering in thesunshine, it's a wonder that any of his words remain w ith u s . . . . Westill stood in our two lines, our guns butt end on the ground or heldloosely in our hands. Major Pitcairn spurred his horse and racedbetween the lines. Somewhere, away from us, a shot sounded. Aredcoat soldier raised h is musket, leveled it at Father, an d fired. Myfather clutched at his breast, then crumpled to the ground like an

empty sack Then the whole British front burst into a roar ofsound and flame and smoke. Excerpt from the novel April Morning,by Howard Fast, published in 1961

Document 4

19th. At 2 o'clock we began our march by wading through a verylong ford up to our middles; after going a few miles we took three or

four people who were going off to give intelligence; about five mileson this side of a town called Lexington, which lay in our road, weheard there were some hun dreds of people collected together intend-ing to oppose us and stop our going on; at 5 o'clock we arrived there,and saw a n umbe r of people, I believe between 200 a nd 300, formedin a com mon in the m iddle of the town; we still continued advancing,keeping prepared against an attack though without intending to attackthem; but on our coming near them they fired one or two shots, uponwhich our men without any orders, rushed in upon them, fired andput them to flight; several of them were killed, we could not tell howmany, because they were got behind walls and into the woods; Wehad a m an of the 10th light Infantry w ounded, nobody else hurt. Wethen formed on the Com mon, but with some difficulty, the m en wereso wild they could hear no orders; we waited a considerable timethere, and at length proceeded on our way to Concord. Entry for

April 19th, 1775 , from the diary of Lieutenant John Barker, an officerin the British army.(Document reprinted in Dana, 1877)

Document 5

Lieutenant Nunn, of the Navy arrived this morning at Lord Dart-mouth's and brought letters from General Gage, Lord Percy, andLieutenant-Colonel Smith, containing the following particulars ofwhat passed on the nineteenth of April last between a de tachmen t ofthe K ing's Troops in the Province of Massachusetts-Bay and severalparties of rebel provincials Lieuten ant-Colon el Sm ith finding,after he had advanced some m iles on his march, that the country hadbeen alarmed by the firing of guns and ringing of bells, dispatchedsix companies of light-infantry, in order to secure two bridges ondifferent roads beyond C oncord, who, u pon their arrival at Lexington,

found a body of the country people under arms, on a green close tothe road; and upon the King's Troops marching up to them, in orderto inquire the reason of their being so assembled, they went off ingreat confusion, and several guns were fired upon the King's troopsfrom behind a stone wall, and also from the meeting-house and otherhouses, by which one man was wounded, and Major Pitcairn's horseshot in two places. In consequence of this attack by the rebels, thetroops returned the fire and killed several of them . After which thedetachment marched on to Concord without any thing further hap-

pening. Newspaper account from The London Gazette, June 10,1775(Document reprinted in Bennett, 1970)

Document 6

There is a certain sliding over and indeterminateness in describingthe beginning of the firing. Major Pitcairn who was a good man in abad cause, insisted upon it to the day of his death, that the colonistsfired first He does not say that h e saw the colonists fire first. Ha d

he said it, I would have believed him, being a man of integrity andhonor. He expressly says he did no t see who fired first; and yetbelieved the peasants began. His account is this—that riding up tothem he ordered them to disperse; which they not doing instantly, heturned about to order his troops so to draw out as to surround anddisarm them. As he turned he sa w a gun in a peasant's hand frombehind a wall, lash n the pan without going off. and instantly or verysoon two or three guns went off by which he found his horse woundedand also a man near him wounded. These guns he did not see, butbelieving they could no t come from his own people, doubted not a ndso asserted that they came from our people; and that thus they beganthe attack. The impetuosity of the King's Troops were such that apromiscuous, uncommanded but general fire took place, which Pit-cairn could not prevent; though he struck his staff or sword down-wards with all earnestness as a signal to forbear or cease firing. This

account Major Pitcairn himself gave Mr. Brown of Providence whowas seized with flour and carried to Boston a few days after the battle;and Gov. Sessions told it to me. From the diary of Ezra Stiles,president of Yale College, entry for August 21, 1775(Document reprinted in Dexter, 1901)

Document 7

In April 1775, General Gage, the military governor of Massachu-setts, sent out a body of troops to take possession of military storesat Concord, a short distance from Boston. At Lexington, a handfulof "embattled farmers," who had been tipped off by Paul Revere,barred the way. The "rebels" were ordered to disperse. They stoodtheir ground. The English fired a volley of shots that killed eightpatriots. It was not long before the swift-riding Paul Revere spreadthe news of this new atrocity to the neighboring colonies. The patriotsof all of New England, although still a handful, were now ready tofight the English. From The United States: Story of a Free People, ahigh school textbook by Samuel Steinberg, Allyn and Bacon, publish-ers, 1963

Document 8

To the best of my recollection abo ut 4 o'clock in the morning beingthe 19th of April the 5 front companies was ordered to load whichwe did It was at Lexington when we saw one of their companiesdrawn up in regular order. Major Pitcairn of the Marines second incommand called to them to disperse, but their not seeming willinghe desired us to mind our space which we did when they gave us afire then run off to get behind a wall. We had one man wounded ofour Company in the leg, his name was Johnson, also Major Pitcairn's

horse was shot in the flank; we returned the ir salute, and before weproceeded on our march from Lexington I believe we killed andwounded either 7 or 8 men. Ensign Jeremy Lister, youngest of theBritish officers at Lexington, in a personal narrative written in 1782(Docum ent reprinted in Lister, 1931)

Received April 23, 1990Revision received July 30, 1990

Acce pted August 14, 1990 •