a questionnaire to measure ehildren's awareness of ... · a questionnaire to measure...

Maribeth Cassidy Schmitt

A questionnaireto measure ehildren'sawareness of strategiereading proeessesSchmitt an assistant professorof education at DePauw Universityin Greencastle, Indiana, isinterested in helping children learnto apply metacognitive strategiesduring their reading.

G ood readers are actively involved inthe comprehension process. They se-lect and use appropriate strategies

and monitor their comprehension as they readto help them understand and remember infor-mation. Some of the strategies good readersemploy include previewing the text, hypothe-sizing about the content, generating ques-tions, and summarizing periodically.

Several studies have shown that childrencan be taught metacomprehension strategies.For example, Paris. Cross, and Lipson (1984)taught children how and when to set purposesfor reading, activate background knowledge,attend to main ideas, draw inferences, andmonitor comprehension. Also, Baumann.Seifert-Kessell, and Jones (1987) used a think

aloud technique to help children monitor theircomprehension during reading.

Because it has been shown that awarenessof metacomprehension strategies is character-istic of good comprehenders {e.g., Paris &Jacobs, 1984; Schmitt. 1988), it would beuseful for teachers to evaluate their students'awareness of those strategies. Such knowledgecould be used informally to design a readingprogram that includes explicit instruction inmetacomprehension skills (e.g., Paris et al.,1984) or is structured so that it fosters the de-velopment of such skills (e.g.. Duffy et al.,1987; Schmitt, 1988; Schmitt & Baumann,1986).

This article describes a multiple-choicequestionnaire-the Metacomprehension Strat-egy Index (MSI) —that teachers can use toevaluate middle and upper elementary stu-dents' knowledge of strategic reading proc-esses. The questionnaire was originallydeveloped to measure strategic awareness ofstudents who participated in a metacompre-hension training study (Schmitt, 1988), andadaptations of the index have been used in sev-eral additional studies (e.g., Baumann et al.,1987; Lonberger, 1988). It is currently beingused at the Benchmark School in Pennsylva-nia in a three-year research and developmentstudy funded by the McDonnell Foundation.This article describes the MSI and supporting

454 The Reading Tbacher March 1990

validity and reliability data. Also discussedare suggestions on how teachers can interpretand use the information derived from theMSI.

Description of the MSIThe MSI is a 25-item. 4-option, multiple-

choice questionnaire (see the Appendix) thatasks students about the strategies they coulduse before, during, and after reading a narra-tive selection. The MSI assesses students'awareness of a variety of metacomprehensionbehaviors that fit within six broad categories:(a) predicting and verifying, (b) previewing,(c) purpose setting, (d) self questioning, (e)drawing from background knowledge, and (f)summarizing and applying fix-up strategies.The response for each item that is indicative ofmetacomprehension strategy awareness is un-derlined in the Appendix.

Table 1 correlates individual MSI items tothe six categories. For example, option C initem 1—"Make some guesses about what Ithink will happen in the story"-is a predic-tion. The strategies assessed by the MSI areconsistent with those taught in several meta-comprehension instructional studies (e.g.,Braun, Rennie, & Labercane, 1986; Palincsar& Brown, 1984; Paris et al., 1984; Risko &Feldman, 1986).

Reliability and validity of the MSIThe MSI has been shown to be a reliable

measure of metacomprehension strategyawareness. Lonberger (1988) reported an MSIinternal consistency value of ,87 using the Ku-der-Richardson Formula 20. To increase over-all reliability of the MSI, it was designed tohave several questions address each strategycluster.

Validity data for the MSI come from sev-eral sources. I (Schmitt, 1988) compared itwith The Index of Reading Awareness (IRA),a self-report measure of awareness of the needto evaluate, plan, and regulate reading proc-esses (Paris, et al., 1984; Paris & Jacobs,1984). A statistically significant correlationwas found between the MSI and the IRA (r =.48, p < .001), suggesting both instrumentsare measuring similar constructs. In addition,I found that in my experimental study (Sch-mitt, 1988), students who received training inmetacomprehension strategies scored signifi-cantly higher on the MSI than students in an

Table 1Strategies measured by the MSI

Predicting and verifyingPredicting the content of a story promotes active compre-hension by giving readers a purpose for reading (i.e., toverify predictions). Evaluating predictions and generatingnew ones as necessary enhances the constructive natureof the reading process.

Item nos. 1,4. 13, 15, 16, 18,23

PreviewingPreviewing the text facilitates comprehension by activatingbackground knowledge and providing information for mak-ing predictions.

Item nos. 2, 3

Purpose settingReading with a purpose promotes active, strategic reading.

Item nos, 5, 7, 21

Self questioningGenerating questions to be answered promotes activecomprehension by giving readers a purpose for reading(i.e., to answer the questions).

Item nos. 6, 14, 17

Drawing from background knowledgeActivating and incorporating information from backgroundknowledge contributes to comprehension by helping read-ers make inferences and generate predictions.

Item nos, 8,9,10, 19,24,25

Summarizing and applying fix-up strategiesSummarizing the content at various points in the storyserves as a form of comprehension monitoring. Rereadingor suspending judgment and reading on when comprehen-sion breaks down represents strategic reading.

Item nos. 11, 12,20,22

instructed control group. I also found statisti-cally significant correlations between the MSIand two comprehension measures, an errordetection task (r = .50, p < .001) and a clozetask {r = .49, p < .001). These data providefurther evidence of the relationship betweenperformance on the MSI and tasks (error de-tection, cloze) commonly used to measurestudents' metacomprehension ability.

Classroom use and interpretationof the MSI

Using and adapting the MSI. The MSIcan be administered in two ways: (a) teachersmay choose to read the questions and possibleanswers aloud to the students; or (b) they mayallow the students to read and answer thequestionnaire silently. I recommend the

A questionnaire to measure children's awareness 455

Table 2Class record for fourth graders' performance on the MSI

MSI Data

P /V(7)

Pre(2)

Pur(3)

Que(3)

B/K(6)

S/FU(4)

Total(25)

Teacher

observations

Linda A, 6 2 2 3 5 4 22

Emily B. 2 0 1 1 1 1

Dwayne B. 6 2 2 2 4 3 19

Constance C, 3 0 1 2 1 2

89 A strong, competent reader performing ata high level on all reading comprehensiontasks.

12 Struggles with most reading tasks; tends toover rely on the graphophonic cue system,

17 A capable student and a good decoder, buthe has difficulty in many comprehensiontasks; does not always seem to apply skillswell,

67 A good reader with no apparent problems inreading comprehension.

Key P / V = Predicting and verifying; Pre = Previewing. Pur = Purpose selling, Que = Self questioning; B / K = Drawing from background knowl-edge; S / FU = Summarizing and applying fix-up strategies; °/oile = comprehension score from a standardized tesi Number of items within eachmetacomprehension category is indicated by parentheses.

former procedure if it is a teacher's judgmentthat students' limited decoding ability or slowreading rate would interfere with their abilityto perform on the questionnaire.

The MSI is designed to measure strate-gies specific to narrative text comprehensiononly. Because previewing, predicting, summa-rizing, questioning, etc. are strategies that canbe used for both narrative and expository text,the MSI can be adapted easily to measure ex-pository text strategies or content-specificstrategies. Following is an example of an item(#2) that has been rewritten to be appropriatefor use with a content textbook:

Before I begin reading, it's a good idea to:a. Look at the Illustrations to see what the

chapter will be about.b. Decide how long it will take me to read

the chapter.c. Sound out the words I don't know.d. Check to see if the information is making

sense.

Interpreting the MSI. The results of theMSI can be used to help teachers design pro-grams of reading comprehension instructionfor individual students. Table 2 presents a por-tion of a teacher-constructed class record forthe MSI for a hypothetical fourth-grade class.On the record are the students' performanceon each of the six clusters of items, a totalMSI score, and, for comparison purposes, the

national percentile rank for the comprehen-sion subtest of a recently-administered stand-ardized achievement test. MSI results can beinterpreted both quantitatively and qualita-tively. Following are descriptions of the deci-sion making processes for several students inthis fourth-grade class.

Linda A. Linda performed at a high levelon the MSI, selecting 22 out of 25 responsesthat are indicative of metacomprehensionawareness. Further, her standardized testscore suggests that she is a skilled comprehen-der, and her teacher's opinion of Linda's abili-ties was consistent with both of these findings.Thus, the MSI served to affirm the teacher'sbelief that Linda was a competent, strategicreader.

Emily B. Emily's performance on theMSI suggests low strategic awareness, and herperformance on the standardized test suggestsa low general performance in reading compre-hension. Emily's teacher sees her strugglingwith reading comprehension and also notesthat Emily tends to focus on accurate wordpronunciation during reading rather than read-ing for meaning. As a result, Emily's miscuestend to be semantically unacceptable thoughthey are good phonic representations of thewords she attempts to pronounce. Consistent

456 The Reading Teacher March 1990

with this finding is Emily's tendency to selectresponses on the MSI that were related toword identification (e.g., she chose "Soundout words I don't know" "Check to see if thewords have long or short vowels in them").

In addition, Emily tended to select re-sponses that were inappropriate with the phaseof reading. For example, for item 6, whichprobed for a before reading behavior, Emilyselected "Retell all of the main points that havehappened so far," a during or after reading be-havior. This suggests that she is unawareof when to select specific strategies for use.Emily's teacher concludes that she wouldbenefit from a program of instruction that in-volves explicit teaching of metacomprehen-sion strategies and when to apply thosestrategies; the program of instruction shouldalso enable Emily to achieve a balance indrawing from graphophonic and meaningclues.

Dwayne B. Dwayne performed at an ac-ceptable level on the MSI, selecting 19 of 25items that indicate strategy awareness. How-ever, his overall comprehension performanceon the standardized test was in the low range,and his teacher also observes Dwayne havingcomprehension problems. She notes furtherthat Dwayne does not always seem to applycomprehension strategies well; he tends toperform acceptably during the skill lessons,but he has difficulty calling up strategies thathe presumably has learned. Dwayne's teacherconcludes that although he seems to knowwhat he should be doing strategically duringreading, he has difficulty actually implement-ing reading strategies. Therefore, she has de-cided that Dwayne would benefit frominstruction that involves teacher modeling andguided practice of strategy application. Fur-ther, she plans to arrange for some one-on-oneinstruction with Dwayne, so he can receiveon-the-spot instruction and application ofcomprehension strategies as he is actuallyreading stories.

Constance C. Constance achieved a highaverage score on the standardized comprehen-sion test, and her teacher observes that she hasno difficulty understanding what she reads.However, Constance's performance on theMSI is relatively low (9 of 25 expected re-sponses). Therefore, Constance's teacher is ina quandary about what to do. After some

thought, she has decided not to engage in anyspecial intervention for Constance, as she be-lieves that Constance is a student whose strate-gic knowledge is tacit information; that is, shehas the strategic abilities but is not con-sciously aware of them or their use. Theteacher will observe Constance carefully,however, and provide instruction as required ifshe begins to demonstrate difficulties in read-ing comprehension.

Cautions about interpreting the MSIIt should be understood that although

there is support for the reliability and validityof the MSI, it is a self-report instrument, and,as such, has the limitations associated withsuch instruments. For example, one cannottell from the instrument alone whether or notchildren actually do the behaviors they saythey do (as is the case for Dwayne). Teacherobservation is required to verify the say/do re-lationship.

Also, while the overall reliability of theMSI is good, one should be cautious when in-terpreting individual item clusters, as they canbe highly unreliable. For example, just be-cause a student may have answered both pre-viewing items with the strategic response andfailed to do so for all three purpose settingitems, it does not mean that the student is ableto preview text but is unable to establish pur-poses for reading. There are too few items tomake these kinds of sweeping statements.

Finally, as with any assessment instru-ment, the MSI should not be used in isolation.Rather, teachers should consider it as onesource of information about students' readingabilities that must be viewed in conjunctionwith other sources of information.

Instructional techniquesMethods for promoting awareness and

use of metacomprehension activities aboundin the professional literature (e.g., see Irwin &Baker, 1989; McNeil, 1987; Wilson & Gam-brell, 1988). A few techniques will be men-tioned here as possible options for teacherswanting to increase students' strategic readingskills.

Students can benefit from observingteachers modeling the thinking processes theyare using to make sense of text (e.g.,Baumann et al., 1987; Bereiter &. Bird, 1985;Davey, 1983). These processes should include


previewing, predicting, questioning, usingfix-up strategies for comprehension break-downs, and summarizing as a monitoringstrategy.

Comprehension skill instruction shouldfocus on teaching the skills as strategies forgetting meaning. Including conditional knowl-edge about the skills (i.e., when they shouldbe used and why they are important or rele-vant) will promote use of them in a strategicmanner (Baumann & Schmitt, 1986; Duffy etal., 1987). For example, instruction in how toidentify implicit main ideas should include theinformation that it is most useful as a strategywhen the text is expository and that it is an im-portant skill to learn because it will help stu-dents learn and remember the informationmore easily.

Directed reading activities can be con-ducted in a manner that promotes metacom-prehension skills and fosters independent,strategic reading. For example, an elaborateddirected reading activity involves allowingstudents to activate their own background, setpurposes for reading, generate prequestions,hypothesize, verify or reject hypotheses, andsummarize where appropriate (Schmitt, 1988;Schmitt &. Baumann, 1986).

ConclusionThe MSI provides one means to evaluate

elementary school students' awareness of pre-reading, during reading, and postreadingmetacomprehension strategies for readingnarrative prose. The information gleanedfrom the instrument may be used to identifystudents* levels of awareness of metacompre-hension. It also can be used to interpret quali-tatively the kinds of strategies studentsconsider to be important, and to evaluate theirawareness of the need to match strategies tothe appropriate reading phase (T^efore, duringor after reading). This information may beuseful 10 teachers in designing a comprehen-sive reading program that fosters metacom-prehension strategy awareness andcompetence.

ReferencesBaumann, J,F,. Seifert-Kessell, N., & Jones. L, (1987, De-

cember). Effects of think-aloud instruction on elemen-tary students' ability to monitor their comprehension.Paper presented al the National Reading Conference,St, Petersburg. FL,

Baumann, J,R, & Schmitt, M,C, (1986), The what, why,how, and when of comprehension instruction. TheReading Teacher. 39. 640-646,

Bereiter. C , & Bird. M, (1985), Use of thinking aloud inidentification and teaching of reading comprehensionstrategies. Cognition and Instruction. 2, 131-156,

Braun, C, Rennie, B,J., & Labercane. G,D. (1986), A con-ference approach to the development of metacognitivestrategies, Solvir}g problems in literacy: Learners,teachers, and researchers. The 35th Yearbook of theNational Reading Conference. 204-209.

Davey, B, (1983), Think aloud—Modeling the cognitiveprocesses of reading comprehension. Journal of Read-ing, 27. 44-47,

Duffy. G,. Roehler. L,R., Sivan. E,. Rackliffe, G,, Book, C,Meloth, M.M,. Vavrus. L,G,. Wesselman, R,. Putnam,J,, & Bassiri, D, (1987), Effects of explaining the rea-soning associated with using reading strategies, Read-ing Research Quarterly 22, 347-368

Irwin, J,W,, & Baker, I. (1989), Promoting active readingcomprehension strategies: A resource book for teach-ers. Englewood Cliffs, NJ: Prentice Hall,

Lonberger. R, (1988, February), Effects of training in a self-generated learning strategy on the prose processingabilities of 4th and 6th graders. Paper presented at theannual meeting of the Eastern Education Association,Savannah. GA,

McNeil, J,D. (1987), Reading comprehension: New dimen-sions for classroom practice (2nd ed). Glenview, IL:Scott, Foresman,

Palincsar, AS,. & Brown. AL, (1984), Reciprocal teachingof comprehension fostering and monitoring activities.Cognition and Instruction, 1, 117-175,

Paris, S,G,. Cross, DR,, & Lipson, M.Y, (1984), Informedstrategies for learning: A program to improve children'sreading awareness and comprehension. Journal of Ed-ucational Psychology, 76. 1239-1252,

Paris, S,G,, & Jacobs, J.E, (1984), The benefits of in-formed instruction for children's reading awarenessand comprehension skills. Child Development. 55,2083-2093,

Risko, V,J,, & Feldman. N, (1986). Teaching young reme-dial readers to generate questions as they read, Read-ing Psychology, 23, 54-64,

Schmitt, M.C, (1988). The effects of an elaborated di-rected activity on the metacomprehension skillsof third graders. Dialogues in literacy research. The37th Yearbook of the National Reading Conference.167-181.

Schmitt, M,C,. & Baumann, J.F, (1986), How to incorpo-rate comprehension monitoring strategies into basalreader instruction. The Reading Teacher, 40. 28-31,

Wilson. R.M,, & Gambrell. L.B. (1988), Reading compre-hension in the elementary school: A teacher's practicalguide. Newton, MA: Allyn & Bacon,

458 The Reading Tfeacher March 1990

APPENDIX

Metacomprehension strategy index

Directions: Think about what kinds of thingsyou can do to help you understand a story bet-ter before, during, and after you read it. Readeach of the lists of four statements and decidewhich one of them would help you the most.There are no right answers. It is just what youthink would help the most. Circle the letter ofthe statement you choose.

I. In each set of four, choose the onestatement which tells a good thing todo to help you understand a story het-ter before you read it.

1. Before I begin reading, it's a good idea to:

A. See how many pages are in thestory.

B. Look up all of the big words in thedictionary.

C. Make some guesses about what Ithink will happen in tbe story.

D. Think about what has happened sofar in the story.

2. Before I begin reading, it's a good idea to;

A. Look at the pictures to see what thestory is about.

B. Decide how long it will take me toread the story.

C. Sound out the words I don't know.D. Check to see if the story is making

sense.


A. Ask someone to read the story tome.

B. Read the title to see what the story isabout.

C. Check to see if most of the wordshave long or short vowels in them.

D. Check to see if the pictures are inorder and make sense.


A. Check to see that no pages aremissing.

B. Make a list of the words I'm not sureabout.

C. Use the title and pictures to help memake guesses about what will hap-pen in the story.

D. Read the last sentence so I will knowhow the story ends.


A. Decide on why I am going to readthe story.

B. Use the difficult words to help memake guesses about what will hap-pen in the story.

C. Reread some parts to see if I can fig-ure out what is happening if thingsaren't making sense.

D. Ask for help with the difficultwords.


A. Retell all of the main points thathave happened so far.

B. Ask myself questions that I wouldlike to have answered in the story.

C. Think about the meanings of thewords which have more than onemeaning.

D. Look through the story to find all ofthe words with three or moresyllables.


A. Check to see if I have read this storybefore.

B. Use my questions and guesses as areason for reading the story.

C. Make sure I can pronounce all of thewords before I start.

D. Think of a better title for the story.


A. Think of what I already know aboutthe things I see in the pictures.

B. See how many pages are in thestory.

C. Choose the best part of the story toread again.

D. Read the story aloud to someone.


A. Practice reading the story aloud.B. Retell all of the main points to make

sure I can remember the story.C. Think of what the people in the

story might be like.D. Decide if I have enough time to read

the story.

^Underlined responses indicate metacomprehen-sion strategy awareness.

A questionnaire to measure ehildren's awareness 459


A. Check to see if I am understandingthe story so far.

B. Check to see if the words have morethan one meaning.

C. Think about where the story mightbe taking place.

D. List all of the important details.

II. In each set of four, choose the onestatement which tells a good thing todo to help you understand a story het-ter whiie you are reading it.

11. While Fm reading, it's a good idea to:

A. Read the story very slowly so that Iwill not miss any important parts.

B. Read the title to see what the story isabout.

C. Check to see if the pictures haveanything missing.

D. Check to see if the story is makingsense by seeing if I can tell what'shappened so far.

12. While I'm reading, it's a good idea to:

A. Stop to retell the main points to seeif I am understanding what has hap-pened so far.

B. Read the story quickly so that I canfmd out what happened.

C. Read only the beginning and the endof the story to find out what it isabout.

D. Skip the parts that are too difficultfor me.


A. Look all of the big words up in thedictionary.

B. Put the book away and fmd anotherone if things aren't making sense.

C. Keep thinking about the title and thepictures to help me decide what isgoing to happen next.

D. Keep track of how many pages Ihave left to read.


A. Keep track of how long it is takingme to read the story.

* Underlined responses indicate metacomprehen-sion strategy awareness.

B. Check to see if I can answer any ofthe questions I asked before Istarted reading.

C. Read the title to see what the story isgoing to be about.

D. Add the missing details to thepictures.


A. Have someone read the story aloudto me.

B. Keep track of how many pages Ihave read.

C. List the story's main character.D. Check to see if my guesses are right

or wrong.

16. While I'm reading, it's a good idea to:A. Check to see that the characters are

real.B. Make a lot of guesses about what is

going to happen next.C. Not look at the pictures because

they might confuse me.D. Read the story aloud to someone.


A. Try to answer the questions I askedmyself.

B. Try not to confuse what I alreadyknow with what Fm reading about.

C. Read the story silently.D. Check to see if I am saying the new

vocabulary words correctly.


A. Try to see if my guesses are going tobe right or wrong.

B. Reread to be sure I haven't missedany of the words.

C. Decide on why I am reading thestory.

D. List what happened first, second,third, and soon.


A. See if I can recognize the new vo-cabulary words.

B. Be careful not to skip any parts ofthe story.

C. Check to see how many of the wordsI already know.

D. Keep thinking of what I alreadyknow about the things and ideas inthe story to help me decide what isgoing to happen.

460 The Reading Teacher March 1990


A. Reread some parts or read ahead tosee if I can figure out what is hap-pening if things aren't making sense.

B. Take my time reading so that I canbe sure I understand what is happen-ing.

C. Change the ending so that it makessense.

D. Check to see if there are enough pic-tures to help make the story ideasclear.

III. In each set of four, choose the onestatement which tells a good thing todo to help you understand a story het-ter after you have read it.

21. After I've read a story it's a good idea to:

A. Count how many pages I read withno mistakes.

B. Check to see if there were enoughpictures to go with the story to makeit interesting.

C. Check to see if I met my purpose forreading the story.

D. Underline the causes and effects.

22. After I've read a story it's a good idea to:A. Underline the main idea.B. Retell the main points ofthe whole

story so that I can check to see if Iunderstood it.

C. Read the story again to be sure I saidall ofthe words right.

D. Practice reading the story aloud.


A. Read the title and look over the storyto see what it is about.

B. Check to see if I skipped any of thevocabulary words.

C. Think about what made me makegood or bad predictions.

D. Make a guess about what will hap-pen next in the story.


A. Look up all of the big words in thedictionary.

B. Read the best parts aloud.C. Have someone read the story aloud

to me.D. Think about how the story was like

things I already knew about before Istarted reading.

25. After I've read a story it's a good idea to:A. Think about how I would have acted

if I were the main character in thestory.

B. Practice reading the story silentlyfor practice of good readmg.

C. Look over the story title and pic-tures to see what will happen.

D. Make a list of the things I under-stood the most.

*Under!ined responses indicate metacomprehen-sion strategy awareness.

Personal literacy statementJean Fritz

The one thing I liked about the British school which I attended as a child was that from a very earlyage we had to memorize poetry: "The splendor falls on castle walls," "Hark, hark, the lark." Such aglory of words! They transported me from the schoolroom, from the everydayness of everyday life,from the limitations of my self. I've been in love with words ever since and stand in awe of the powerthey can wield. My mother, who had been a Latin teacher, shared this love, and I remember particu-larly the joy in words that she took when she read aloud Kipling's "The Butterfly that Stamped."

Jean Fritz is an award-winning author of historical biographies and novels for young people. Hermany works include Homesick: My Own Story, The Double Life of Pocahontas, and The CabinFaced West.


a questionnaire to measure ehildren's awareness of ... · a questionnaire to measure...

Documents