Assessing Academic Literacy: The role of text in comprehendingThe role of text in comprehending

written language

Barbara Foorman Ph DBarbara Foorman, Ph.D.Florida Center for Reading g

ResearchFl id St t U i it

1Florida State University

What are the Issues?• Academic literacy assumes grade-level

fi iproficiency.• On the 2007 Reading NAEP, 33% below

basic in G4; 26% below basic in G8.• For minorities, the % below basic on the 2007 ,

Reading NAEP are: 53% in G4 & 45% in G8 for Blacks; 50% in G4 and 42% in G8 for ;Hispanics.

• NCLB requires that students at-risk for2

NCLB requires that students at risk for reading disability receive intervention.

Goals for This PresentationGoals for This Presentation

Explain relation of academic literacy toExplain relation of academic literacy to academic languageDefinitions of reading comprehensionDefinitions of reading comprehensionCharacteristics of text difficultyMeasuring text difficultyAssessing academic literacyg y

3

Academic Language is at the Core of Literacy Instruction

Word Meaningsa. because it allows literate

people to discuss literary products; previouslyproducts; previously referred to as extended discourse or d t t li d ldecontextualized language.

b. because contextual cues and shared assumptions pare minimized by explicitly encoding referents for pronouns actions and

4Text

pronouns, actions, and locations

13 higher-SES childrenSES children(professional)

23 iddl /l23 middle/lower-SES children(working class)

6 welfare 6 welfare childrenchildrenchildrenchildren

5Age of child in monthsAge of child in months

Hart & Risley, 1995

child Language ExperienceLanguage Experience

ssed

to c

Professional

ds a

ddre

s

W ki l

tive

wor

d Working-class

cum

ulat

Welfare

stim

ated

6

Es Age of child in monthsHart & Risley, 1995

Quality Teacher Talky(Snow et al., 2007)

R d• Rare words • Ability to listen to

children and to extendchildren and to extend their comments

• Tendency to engage y g gchildren in cognitively challenging talkP t t• Promotes emergent literacy & vocabulary & literacy success in

7

literacy success in middle grades

Home & Schoolexperiences: ages 3-6 Skills developed: ages 3-6 School performance

Literacy Understandingliteracy

Kindergartenand first grade

Print focus Print

gradereading

Conversation Conversational Instruction andConversation

Extended

Conversationallanguage Practice in reading

Extendeddiscourseforms andnonfamiliaraudiences

Decontextualizedlanguage

Reading comprehensionIn Grade 4

8(Snow, 1991)

T bl 3Table 3

% Independent Reading

Words Read Per g

Minutes Per Day Year 98 65.0 4,358,000 90 21.1 1,823,000 80 14 2 1 146 00080 14.2 1,146,00070 9.6 622,000 60 6.5 432,000 50 4.6 282,00050 4.6 282,00040 3.3 200,000 30 1.3 106,000 20 0.7 21,000

Variation in Amount of Independent Reading

10 0.1 8,0002 0.0 0

9

(Cunningham & Stanovich, 1998, adapted from Anderson, Wilson, & Fielding,1988)

Is Literacy Enough? (Snow et al 2007)Is Literacy Enough? (Snow et al., 2007)

For adolescents oral language and literacyFor adolescents, oral language and literacy skills need to be adequate, but also need:

• Caring adult(s) at home• Caring adult(s) at home• Caring adults at school who provide

id b t h t t l ( ftguidance about how to meet goals (often need smaller school)

• Minimal risk: Not many school transitions; minimal family disturbances.

10

What is Reading Comprehension?What is Reading Comprehension?

• “the process of simultaneously extracting• the process of simultaneously extracting and constructing meaning through interaction and involvement with writteninteraction and involvement with written language” (RAND, 2002, p. 11)

• “Reading is an active and complex processReading is an active and complex process that involves– Understanding written textg– Developing and interpreting meaning; and– Using meaning as appropriate to type of text,

11

purpose, and situation” (NAEP Framework, 2009)

Word recognition, vocabulary,background knowledge, strategyuse, inference-making abilities,

Text structure, vocabulary, genrediscourse, motivating features,print style and font , g ,

motivationprint style and font

TEXT READER

ACTIVITY

Purpose, social relations,school/classroom/peers/families

Environment, cultural norms

12A heuristic for thinking about reading comprehension (Sweet & Snow, 2003).

Understanding what h b d th has been read; the application to written t t ftext of:(a) nonlinguistic( t l) k l d (conceptual) knowledge (b) general language

h i killcomprehension skills(Rayner, Foorman, Perfetti, Pesetsky, & Seidenberg, 2001)

13

g, )

Skilled ReadingSkilled ReadingThe Reading The Reading PillarPillar

(NRC, 1998)

Skilled ReadingSkilled ReadingPillarPillar

FluencyFluencyConceptual

Speed and ease of reading ith

ComprehensionComprehensionConceptual Knowledge/vocabulary

Strategic processing of text

reading with comprehension

Word RecognitionWord RecognitionDecoding using alphabetic principle

text

Emergent ReadingEmergent ReadingPrint Awareness & Letter

Knowledge

Motivation to Read

p p

Decoding using other cues

Sight Recognition

14

Motivation to Read

Oral Language including

Phonological Awareness

What Makes a Text Difficult?

15

Components of Reading Comprehension (Perfetti, 1999)

C h i PComprehension Processeses

Situation Model

General Knowledge

Linguistic System

Infe

renc

e

Text Representation

Parser

Linguistic SystemPhonologySyntaxMorphology

Parser

Meaning and Form Selection LexiconMeaningWord

Representation

Word Identification

MeaningMorphologySyntax

OrthographicUnits

PhonologicalUnits

OrthographyMapping tophonology

16Visual Input

Vocabulary Demands in 6 G1 Basals (Foorman et al., 2004)

Table 4 Representation of Oral and Written Vocabulary in Program (Types) A B C1 C2 D E

LWV Levels

Freq. % Freq. % Freq. % Freq. % Freq. % Freq. %

2 889 (51.99) 897 (52.55) 891 (52.72) 1101 (48.96) 196 (64.47) 586 (55.44) 4 609 (35 61) 575 (33 68) 592 (35 03) 785 (34 90) 102 (33 55) 375 (35 48)4 609 (35.61) 575 (33.68) 592 (35.03) 785 (34.90) 102 (33.55) 375 (35.48)6 104 (6.08) 107 (6.27) 113 (6.69) 197 (8.76) 2 (.66) 53 (5.01) 8 47 (2.75) 35 (2.05) 33 (1.95) 64 (2.85) 1 (.33) 17 (1.61) 10 18 (1.05) 24 (1.41) 16 (.95) 28 (1.24) 1 (.33) 8 (.76) 12 25 (1.46) 41 (2.40) 23 (1.36) 45 (2.00) 2 (.66) 10 (.95) 13 9 (.53) 11 (.64) 14 (.83) 15 (.67) 6 (.57) 16 9 ( 53) 17 (1 00) 8 ( 47) 14 ( 62) 2 ( 19)16 9 (.53) 17 (1.00) 8 (.47) 14 (.62) 2 (.19)

Total

1710

1707

1690

2249

304

1057

Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD

SF1 53.24 (10.29) 52.64 (10.83) 53.78 (9.72) 51.91 (10.06) 61.42 (9.12) 55.38 (10.10)

Note. LWV = Living Word Vocabulary (Dale & O’Rourke, 1981).

17

SFI = Standard Frequency Index Zeno et al., 1995).

Relation of Frequency in Corpus to

Grade 1 Frequency in Zeno et al. (1995)

18

Some “rare” (G1 Basal) and “not-so-rare” (elementary literature) Wordsrare (elementary literature) Words

WORD LWV Level Basal f/100 Lit. f/1,000,000

craft 6 .001684892 4.952

d 6 002813969 11 638due 6 .002813969 11.638

elk 6 .002813969 7.429

exhausted 6 .002813969 7.429

fifth 8 .002813969 23.029

fi 8 002813969 5 448fins 8 .002813969 5.448

flung 6 .002813969 13.371

gathering 6 .002813969 16.343

generally 6 .002813969 11.886

greatl 8 002813969 12 133greatly 8 .002813969 12.133

hooks 12 .002813969 5.200

hops 12 .002813969 5.200

horned 6 .002813969 5.200

household 6 002813969 10 648household 6 .002813969 10.648

illness 6 .001684892 5.695

jersey 6 .002813969 10.648

kingdom 6 .002813969 20.800

layer 6 002813969 25 257

19

layer 6 .002813969 25.257

leash 8 .001684892 11.390

least 6 .002813969 139.904

lights 13 .002813969 97.314

Representation of Opportunity W d A B lWords Across Basals

Number of Programs

LWV L l T t lLWV Level1 2 3 4

Total

6 87 33 9 0 1298 14 12 4 2 32

10 11 2 3 0 1612 22 2 5 0 2913 3 1 0 1 516 4 1 0 0 5

Total 141 51 21 3 216

20

Total 141 51 21 3 216

Opportunity Words in Grade 1 Basalsd k l h d b illi t ti id fl hi l ktad creak glossary perch sped brilliant timid flahing plankton

amuse creamy gown phrase spoiled celebrated typical foal ticking

arch create granite poetry squad coral vacuum fro

attract crib grief poisonous squire draws vegetation gracious

backwards determination gust porcupine sturdy dune yourselves handlesg p p y y

blues device haze potter survive elegant alas hatching

blur display holly pox swap fins bog heather

boar doe horned prey swoop gerbil brute hooks

boast dose illness prickly tattered gruff cam hops

bony driftwood item pueblo thankful hermit cove mantis

breed elk jumper pulp ties heron dialogue mats

bronze establish kicks radar towering huff flora maze

burrow exhausted leapt relate turquoise lance framework minded

career fangs lent relay twinkle polar guinea riocareer fangs lent relay twinkle polar guinea rio

cement fearless listener resist veterinarian promises hangs senora

chops fig llama rhinoceros wag ramp hemisphere slanted

chowder flapped magnificent rhythm walrus reed lulu sneaking

clam fled marine rover wee reef ping stacks

clippers foil mercury rum whaling returns squid swish

clumsy frisky meter sculpture whew ribbons stripes tad

cocoon furthermore mi seller whoa rushes taps taro

con gallery mobile shack wraps scurry blasted taut

i ll ld h k l i b i kli

21

conservation galley mold shaken wrestle si boa twinkling

construction garlic outdoor shrug yelp stated buster amazon

contented genius overcome slimy zoom stirring chameleon digs

craft gigantic packet sow thud chi splitting

C l i V b lConclusions on Vocabulary• Publishers need to provide teachers with

cumulative vocabulary lists• These need to be made available

electronically to textbook adopters and should include information on:– Frequency in text and lesson number– Separate entry for each definition used– Derivational forms

P i t d d f i th l t22

– Printed word frequency in other relevant corpora

Conclusions on VocabularyConclusions on Vocabulary

• Instruction needs to target oral language development from pre-school through high school

• Printed word frequency and age of acquisition are useful tools for guiding selection of lexical entries to be taught

• Assessment of vocabulary for the purpose of Reading First should focus on the link between assessment and instruction

23

between assessment and instruction

Summary and ConclusionsSummary and Conclusions• Programs differ substantially in the

composition of their print materials for Gradecomposition of their print materials for Grade 1 students

• Length of texts, grammatical complexity,Length of texts, grammatical complexity, numbers of unique and total words, repetition of words, coverage of important vocabulary

• Differences exist in the decodability of types and tokens

Generally there is greater decodability for tokens– Generally there is greater decodability for tokens than types,

– most programs show improvements for types later i th

24

in the year

Summary and ConclusionsSummary and ConclusionsPrograms vary in the approach they take to g y pp y

achieve decodability and in the degree to which materials can be expected to yield accuracy in reading. - Vary in phonic elements taught - Vary in opportunity to practice words

containing these elements- Within 6-week blocks, 70% of words are

singletons in 4 of the 6 basalsV i li h li ti ll t ht d

25

- Vary in reliance on holistically-taught words

Implications for fluencyp y• “…for dysfluent readers, the texts that are

read and reread for fluency practice needread and reread for fluency practice need to have sufficiently high percentages of words within the word zone fluencywords within…the word zone fluency curriculum and low percentages of rare words, especialy multisyllabic ones” ( p. , p y y ( p18)

• “Repetition of core words makes science ptext ideal for fluency practice in the primary grades” (p. 11)

26Hiebert (2007)

Word Zone Fluency CurriculumHigh Freq Words Phonics/Syllable MorphologicalHigh-Freq Words Phonics/Syllable Morphological

A 300 most freq accuracy rate of 40%

Short/long vowels Simple, inflected endings (ed, ing, s, es,’s)

in first grade in Seymour et al., 2003).

B 500 most freq Short & long & r-controlled vowels

C 1,000 most freq All monosyllabic

D 1,000 most freq 2-syllable compound words with at least 1 root from 1,000 most f t d

Prefixes: un, aSuffixes: er, est,ly, y (doubling)

frequent wordsy, y ( g)

E 2,500 most freq

27F 5,000 most freq

Jabberwocky (Lewis Carroll 1872)Jabberwocky (Lewis Carroll, 1872)

‘Twas brillig, and the slithy tovesg yDid gyre and gimble in the wabe:All mimsy were the borogoves,y gAnd the mome raths outgrabe.

“Beware the Jabberwock, my son!The jaws that bite, the claws that catch!jBeware the Jubjub bird, and shun The frumious Bandersnatch!”

28And four more stanzas From Through the Looking-Glass and What Alice Found There

Discussionscuss o

You know how to pronounce the words inYou know how to pronounce the words in Jabberwocky; some are real English words.

1 Which ones are real English words?1. Which ones are real English words?2. What is the distinction between those that

t l E li h d d th th tare actual English words and those that aren’t?

3. Do the two paragraphs differ in these distinctions?

29

Alice’s reactionAlice s reaction“It seems very pretty ” she said when sheIt seems very pretty, she said when she

had finished it, but it’s rather hard to understand!” (You see she didn’t like tounderstand! (You see she didn t like to confess, even to herself, that she couldn’t make it out at all ) “Somehow it seems tomake it out at all.) Somehow it seems to fill my head with ideas—only I don’t exactly know what they are! Howeverexactly know what they are! However, somebody killed something: that’s clear at any rate—”

30

any rate

NAEP 2009 Reading FrameworkNAEP 2009 Reading Framework

Characteristics of text difficulty:Characteristics of text difficulty:

• Vocabulary reported out separatelySubscales for literary & informational• Subscales for literary & informational text

• Grade-level standards for text type

31

2009 NAEP FrameworkLiterary Text Informational TextLiterary Text

● Fiction● Literary Nonfiction

Informational Text● Exposition ● Argumentation and Persuasive

● Poetry Text● Procedural Text and Documents

Cognitive Targets Distinguished by Text TypeCognitive Targets Distinguished by Text Type

Locate/Recall Integrate/Interpret Critique/Evaluate

32

AchievementLevel

Literary Informational

Achievement Levels for Grade 4 NAEP Reading

Advanced G4 students at the Advanced level should be able to:

•Interpret figurative language•Make complex inferencesId tif i t f i

G4 students at Advanced level should be able to:

•Make complex inferences•Evaluate the coherence of a text

•Identify point of view•Evaluate character motivation•Describe thematic connections across literary texts.

•Explain author’s point of view•Compare ideas across texts

Proficient G4 students at the Proficient level should be able to:

•Infer character motivation•Interpret mood or tone•Explain theme

G4 students at Proficient level should be able to:

•Identify author’s implicitly stated purpose•Summarize major ideas•Find evidence in support of an argumentp

•Identify similarities across texts•Identify elements of author’s crafts

pp g•Distinguish between fact and opinion•Draw conclusions

Basic G4 students at the Basic level should be able to:

•Locate textually explicit information, such as plot, setting, and character•Make simple inferences

G4 students at the Basic level should be able to:

•Find the topic sentence or main idea•Identify supporting details•Identify author’s explicitly stated purpose

33

•Identify supporting details•Describe character’s motivation•Describe the problem•Identify mood

•Make simple inferences

2009 NAEP Framework2009 NAEP FrameworkEnglish Mathematics History Science

literary informational or expository, Informational or

text typeliterary informational or

technical, symbolic, diagrams

expository, argumentative, persuasive

Informational or technical, diagrams

plot, setting, sequence, cause sequence, cause sequence, cause

text structure

characterization, point of view, verse, rhyme

and effect, problem and solution, supporting ideas and evidence, graphical features

and effect, problem and solution, author’s perspective supporting ideas

and effect, problem and solution, supporting ideas and evidence, graphical featuresstructure graphical features supporting ideas

and evidence, contrasting viewpoints, graphical features

graphical features

author’s craft

diction, dialogue, symbolism, imagery, irony, figurative language

rhetorical structure, examples, logical arguments

figurative language, rhetorical structure, examples, emotional appeal

rhetorical structure, examples, logical arguments

34

language

35

What Does Mean to be Proficient?What Does Mean to be Proficient?

• W score cutpoints on NAEP and state testsW score cutpoints on NAEP and state tests communicate grade-level proficiency or benchmark performancebenchmark performance.

• State curriculum standards need to be aligned with benchmarks/proficiency levelsaligned with benchmarks/proficiency levels.

• Are states’ proficiency levels comparable to NAEP’ ?NAEP’s?

36

% Proficient on State vs NAEP Reading 2005% Proficient on State vs NAEP Reading 2005

State 4-state 4-NAEP DIFF 8-state 4-NAEP DIFFState DIFF DIFF

ME 53 35 -18 44 38 - 6

MO 35 33 - 2 33 31 - 2

WY 47 34 -13 39 36 - 3WY 47 34 13 39 36 3

TX 79 29 -50 83 26 -57

GA 87 26 -61 83 25 -58

NC 84 29 -55 89 27 -62

37

C 8 9 55 89 6

[Porter, 2007]

Most state testing systems do not assess college and work readiness

• 26 states require students to pass an exam before they graduate highexam before they graduate high school.*

• Yet most states have testing systems• Yet most states have testing systems that do not measure college and work readiness **readiness.

* d i li h h l d

38

*Source: Center on Education Policy, State High School Exit Exams: States Try Harder, But Gaps Persist, August 2005.**Source: Achieve Survey/Research, 2006.

Graduation exams in 26 states establish the performance “floor”

Figure reads: Alaska has a mandatory exit exam in 2005 and is withholding diplomas from students based on exam performance. Arizona is phasing in a mandatory exit exam and plans to begin withholding diplomas based on this exam in 2006. Connecticut does not have an exit exam, nor is it scheduled to implement one.

39

have an exit exam, nor is it scheduled to implement one.

Source: Center on Education Policy, based on information collected from state departments of education, July 2005.

How challenging are state exit exams?How challenging are state exit exams?

• Achieve conducted a study of graduation• Achieve conducted a study of graduation exams in six states to determine how high a bar the tests set for studentsa bar the tests set for students.

• The results show that these tests tend to measure only 8th 9th or 10th grademeasure only 8th, 9th or 10th grade content, rather than the skills students needs to succeed in college and theneeds to succeed in college and the workplace.

40

The tests Achieve analyzedyFirst

Graduating

StateGrade Given Reading Writing Math

Class Facing Requirement

Florida 10th • • 2003

Maryland End of course • • • 2009

Massachusetts 10th • • • 2003Massachusetts 10th • • • 2003

New Jersey 11th • • • 2003

Ohio 10th • • 2007

Texas 11th • • • 2004

41Source: Achieve, Inc., Do Graduation Tests Measure Up? A Closer Look at State High School Exit Exams,2004.

Students can pass state English tests with skills ACT expects of 8th & 9th graders

ACT PLAN ACT

(11th/12th)

0 1 2 3 4 5 6

ACT EXPLORE (8th/9th)

ACT PLAN (10th)

(11th/12th)

FL

MDMD

MA

NJ

OH

TX

42Source: Achieve, Inc., Do Graduation Tests Measure Up? A Closer Look at State High School Exit Exams,2004.

% Students Proficient on FCAT(L l 3 d b )(Level 3 and above)

Grade 2001 2006 DifferenceGrade 2001 2006 Difference3 57 75 184 53 66 134 53 66 135 52 67 156 52 64 126 52 64 127 47 62 158 43 46 38 43 46 39 28 40 12

43

10 37 32 -5

Is 10th Grade FCAT Too Hard?Is 10 Grade FCAT Too Hard?

• The St Petersburg Times article (4/15/07)The St. Petersburg Times article (4/15/07) concluded correctly that the 10th Grade FCAT is harder than the 10th grade NRTFCAT is harder than the 10 grade NRT.

• Conclusion based on fact that Level 3 (proficient) performance is 56th %ile(proficient) performance is 56th %ile nationally at Gr 7; 80th %ile at Gr 10O “Wh it til hi h h l t• Or “Why wait until high school to implement world class standards?”

44

Absolute level of reading proficiency nationally

cien

cy

9

10 Grade level standard on the FCAT

ding

pro

fic

7

8

evel

of r

ea

5

6

Abs

olut

e le

3

4

A

3 4 5 6 7 8 9 10

45

Passage Length in Wordsg gGrade FCAT range FCAT average NAEP range NAEP average

3 100-700 3504 100-900 400 200-8005 200-900 4506 200-1000 5007 300-1100 6008 300-1100 700 400-10008 300 1100 700 400 1000

9 300-1400 80046

9 300 1400 80010 300-1700 900 500-1500 (12) 1000 (12)

% of Passage Typesg yp

Grade FCAT Literary FCAT Informa- NAEP Literary NAEP Informa-Texts tional Texts Texts tional Texts

3 60% 40%4 50% 50% 50% 50%4 50% 50% 50% 50%5 50% 50%6 50% 50%7 40% 60%8 40% 60% 45% 55%9 30% 70%

4710 30% 70% 30% (12) 70% (12)

FCAT Test Designg

• Cognitive Complexity (Webb’s Depth ofCognitive Complexity (Webb s Depth of Knowledge)

• Content Categories for Reading• Content Categories for Reading- Words & phrases in context- Main idea, plot, & author’s purpose- Comparison; cause/effectp ;- Reference & Research – locate, organize, interpret synthesize & evaluate information

48

interpret, synthesize, & evaluate information

To Make Proficiency Standards M i f l d F iMeaningful and Fair

• Agree on target for proficiency (e gAgree on target for proficiency (e.g., college readiness)

• Align elementary middle and high school• Align elementary, middle, and high school targetsAli i l t d d• Align curriculum standards

• Evaluate dimensionality of tests and prepare instruction accordingly

• Equate state tests with NAEP to guarantee 49

q gcomparability and equity

From Barbara Tuckman’s The Zimmerman Telegram…

The first message of the morning watch plopped out of the pneumatic tube into a wire basket with no more premonitory rattle than usual The duty officer at thepremonitory rattle than usual. The duty officer at the British Navel Intelligence twisted open the cartridge and examined the German wireless intercept it contained

ith t ti thi f l i ifi Whwithout noting anything of unusual significance. When a glance showed him that the message was in non-navel code, he sent it in to the Political Section in the inner room and thought no more about it. The date was January 17, 1917, past the halfway mark of a war that had already ground through thirty months of reckless

50

had already ground through thirty months of reckless carnage and no gain.

What Makes This Text Difficult?What Makes This Text Difficult?

• Consider the text type and structureConsider the text type and structure• Consider prior knowledge

C id th b l• Consider the vocabulary• Consider the discourse features—linguistic

markers for coherence, coreference, deixis• Other factors?

51

Instructional ConsiderationsInstructional Considerations• Text Type/Structure

persuasive text– persuasive text • anti-war sentiment, “thirty months of reckless carnage and no gain”• indictment of war bureaucracy

– narrative structure– narrative structure – historical non-fiction

• Prior Knowledge• Prior Knowledge– World War I

• text references: war, 1917, British, German, duty officerearly 20th century communications– early 20th century communications

• text references: telegram, pneumatic tube, wire basket, wireless intercept – Zimmerman telegram

• text references: German wireless, non-naval code

52

text references: German wireless, non naval code

53

Instructional Considerations(continued)

• Vocabulary – academic languageg g

• examined, significance, “ground through”– generative words

• premonitory, carnage, interceptTi 3 b l ( ilit d i )– Tier 3 vocabulary (military domain)

• “morning watch,” non-naval code, German wireless, pneumatic tube

• Linguistic Markers (Coherence Relations)Linguistic Markers (Coherence Relations)– pronouns

• duty officer = he, him – co-references

• German wireless intercept = the message – deixis

• “in the inner room”chronology

54

– chronology • “When a glance showed him that the message was in non-navel code,…”

Instructional Delivery

• Model strategies (activating background knowledge, questioning, searching for information, summarizing, organizing graphically, identifying story structure (e.g., Guthrie et al., 2004; Brown, Pressley et al., 1996)

• Keep the focus on the meaning of the text through high quality discussion.

• Model “thinking like an historian” (e.g., sourcing) to provide a purpose for reading

55

g) p p p g(Biancarosa & Snow, 2004).

Measuring Text Difficultyg y• Teacher judgment• Readability: Tuchman passage ranges from

8.4 on Dale-Chall to 13.3 on the Flesch-Kincaid & Fry; 13.5 on Lexiles.

• Latent semantic analysisy• Natural language processing (e.g.,

McNamara 2001)McNamara, 2001)• Text equating to control passage difficulty

56

Limitations of readabilityy• Circular use• Capture surface features only• Measurement error on specialized textp

- Primary grade textPoetry- Poetry

- Technical documents (e.g., train h d l t f )schedules; tax forms)

57

How Do We Assess Academic Li ?Literacy?

58

Discussion of Academic Literacy AAssessment

• What are the important knowledge and skills to assess in K-3?

• What are the important knowledge and skills to assess in 4-12?

• What kind of text should be used?• What kind of outcome measures should be• What kind of outcome measures should be

used?

59

Converging EvidenceConverging Evidence

V lid d li bl di t f i k fValid and reliable predictors of risk for reading difficulty are:

P i ( l K)Print concepts (early K)Letter name knowledge (early K)Ph l i l d l tt d (K 1)Phonological awareness and letter sounds (K-1)Rapid naming of letters (end of K to early G1)Word recognition (G1 and beyond)Word recognition (G1 and beyond)Vocabulary

60(Fletcher et al., 2002; Scarsborough, 1998; Torgesen, 2002)

Assessing written language• Use various formats to assess:

--multiple choicep--cloze--mazemaze--question/answer

constructed response--constructed response--retelling

t ifi ti--sentence verification• Report achievement in language proficiency

l l t h t ELL (F i 2008)61

levels to chart ELLs progress (Francis, 2008)

New PK-12 Florida ReadingAssessment SystemAssessment System

• Instructionally useful; free to FL schools in 2009-2010• Includes vocabulary and comprehensionIncludes vocabulary and comprehension • Computer administered in grades 3-12• Screening, progress monitoring, & diagnostic assessments;

data available in the Progress Monitoring & Reportingdata available in the Progress Monitoring & Reporting Network (PMRN)

• Screen is empirically linked to the Florida Comprehensive Assessment Test (FCAT) or outcome measureAssessment Test (FCAT) or outcome measure

• Targeted diagnostic inventories administered to students not meeting expectations are linked to Florida standards and provide information for guiding instructionprovide information for guiding instruction

• Reading comprehension & oral reading fluency passages are equated for difficulty to allow for accurate progress monitoring

62

monitoring• Instructional level passages provided

New Reading Assessmentsg

• PK: print knowledge, phonological awareness, p g , p g ,vocabulary, math (linked to K screening)

• K-2: phonemic awareness, letter knowledge, decoding, di fl b l li t i diencoding, fluency, vocabulary, listening or reading comp.

• 3-12: adaptive complex & low level reading comp., fluency, word analysis, skill assessmentfluency, word analysis, skill assessment

• K-12: Informal reading inventories• Lexile scores in grades 3-12 allow matching students to

text and access to online libraries• Identifies risk of reading difficulties and reading

disabilities63

disabilities

New Reading Assessmentsg

64

Th k !Thank you!bfoorman@fcrr.org

www.fcrr.org65

g

References• Biancarosa, G., & Snow, C.E. (2004). Reading next—A vision for action and research in middle and high

school literacy: A report to Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education.

• Brown, R., Pressley, M., Van Meter, P., & Schuder, T. (1996). A quasi-experimental validation of transactional strategies instruction with low-achieving second grade readers. Journal of Educational Psychology, 88, 18 3718-37.

• Foorman, B.R., Francis, D.J., Davidson, K., Harm, M., & Griffin, J. (2004). Variability in text features in six grade 1 basal reading programs. Scientific Studies in Reading, 8(2), 167-197.

• Guthrie, J.T., Wigfield, A., Barbosa, P., Perencevich, K.C., Tabada, A., Davis, M.H., Scafiddi, N.T., & Tonks, S. (2004). Increasing reading comprehension and engagement through Concept-Oriented Reading Instruction Journal of Educational Psychology 96(3) 403 423Instruction. Journal of Educational Psychology, 96(3), 403-423.

• Hiebert, E.H. (2007). A fluency curriculum and the texts that support it. In P. Schwanenflugel & M. Kuhn (Eds.), Creating a literacy curriculum: Fluency instruction. New York: Guilford Press.

• National Assessment Governing Board (in press). 2009 NAEP Reading Framework. Washington, D.C.: Author. Retrieved March 26, 2007 from http://www.naepreading.org/.

• National Research Council (1998) Preventing Reading Difficulties in Young Children Committee on the• National Research Council (1998). Preventing Reading Difficulties in Young Children. Committee on the Prevention of Reading Difficulties in Young Children, Commission on Behavioral and Social Science and Education. In C.E. Snow, M.S. Burns, and P. Griffin (Eds.). Washington, DC: Nat’l Academy Press

• Perfetti, C.A. (1991). Representation and awareness in the acquisition of reading competence. In L. Rieben & C. Perfetti (Eds.), Learning to read: basic research and its implications (pp. 33-44). Hillsdale, NJ: Erlbaum.

• Porter A (2007) NCLB lessons learned: Implications for reauthorization In A Gamoran (Ed ) Will “No ChildPorter, A. (2007). NCLB lessons learned: Implications for reauthorization. In A. Gamoran (Ed.), Will No Child Left Behind “ help close the poverty gap? Washington, DC: Brookings Institution.

• RAND Reading Study Group (2002). Reading for understanding: Toward a R&D program in reading comprehension. Arlington, VA: RAND.

• Snow, E., Porche, M., Tabors, P., & Harris, S. (2007). Is literary enough? Baltimore, MD: Brookes.• Snowling M J & Hulme C (2005) The science of reading: A handbook NY: Blackwell

66

Snowling, M.J., & Hulme, C. (2005). The science of reading: A handbook. NY: Blackwell.• Sweet, A.P., & Snow, C.E. (2003). Rethinking reading comprehension. NY: The Guilford Press.• Zeno, S.M., Ivens, S.H., Millard, R.T., Duvvuri, R. (1995). The educator’s word frequency guide. NY:

Touchstone Applied Science Associates, Inc.