the lexile framework for reading: integrating measurement and instruction copenhagen sunday, august...
TRANSCRIPT
The Lexile Framework for Reading: Integrating Measurement and Instruction
COPENHAGENSunday, August 14, 2011
A. Jackson StennerChairman & CEO, MetaMetrics
Research Professor, UNC-Chapel Hill
The Lexile Framework is NOT:
• A reading program
• A test or method of assessment
• Educational software
What is The Lexile Framework for Reading?
• An educational tool – a scale - that links reading material and readers under a common metric – the Lexile.
− Measures the complexity of reading material
− Measures the reading level of individual readers
The Lexile Framework for Reading
A student’s Lexile measure is her reading level.
A book’s Lexile measure is its reading difficulty.
an 1170L reader
an 1170L book
A Lexile is a measure of both reading level and text complexity, on the same scale.
Why The Lexile Framework?
• LINK reader to text under a common measure
• FORECAST levels of comprehension
• TRACK growth over time
Where are Lexile Measures Found?
• All major norm-referenced tests are linked• Over 30 million students received a Lexile measure last
school year• Over 450 publishers• Nearly 115,000 book titles to search at www.lexile.com• Over 400 million Lexiled articles
Where are Lexile Measures Found?Formative and Norm-Referenced Assessments
Where are Lexile Measures Found?Library Resources
The Lexile Scale
How high does the Lexile scale go?
How low does the Lexile scale go?
The Lexile Scale
• Lexile measures typically range from below 200L for beginning readers to above 1700L for advanced readers.
• Text below 200L represents beginning-reading material. A student’s Lexile measure of 0L or below is coded BR, signifying ‘Beginning Reading’.
Lexile Measures & Text Complexity
“Quantitative dimensions of text complexity refer to those aspects of text complexity, such as word length or frequency, sentence length, and text cohesion, that are difficult if not impossible for a human reader to evaluate efficiently, especially in long texts, and are thus today typically measured by computer software.”
-from the Common Core State Standards
Determining Text Complexity
• Sentence Length (syntactic complexity)Longer sentences are more complex and require more short-term memory to process.
Determining Text Complexity
• Vocabulary (semantic difficulty) Less familiar words impede reading fluency and affect comprehension.
Analyzing Text Readability
• Read and examine Camping is Fun. • Easy or difficult to read?
Why?
We went camping. On Monday, we put up the tent. The tent fell down! On Tuesday, we hiked. It rained andwe got wet! On Wednesday, we fished. Uncle Pete caught a boot! On Thursday, we left food on the table. The raccoons ate it! On Friday, we cookedmarshmallows. They fell in the fire! On Saturday, we heard a strangenoise. We were too scared to sleep!On Sunday, we went home. "Did you like camping?" asked Uncle Pete. "Yes!" we said. "Camping is fun!"
Camping is Fun(M. Garcia)
We went camping. On Monday, we put up the tent. The tent fell down! On Tuesday, we hiked. It rained andwe got wet! On Wednesday, we fished. Uncle Pete caught a boot! On Thursday, we left food on the table. The raccoons ate it! On Friday, wecooked marshmallows. They fell in the fire! On Saturday, we heard a strange noise. We were too scared to sleep!On Sunday, we went home. "Did you like camping?" asked Uncle Pete. "Yes!" we said. "Camping is fun!"
Camping is Fun(M. Garcia)
200L
Analyzing Text Readability
• Read and examine an excerpt of a speech from Lee Kuan Yew: The Separation of Singapore from Malaysia
• Difficult or easy to read? Why?
Speech from Lee Kuan Yew:Separation of Singapore from Malaysia
"For me, it is a moment of anguish. All my life, my whole adult life, I believed in merger and unity of the two territories. ... Now, I, Lee Kuan Yew, Prime Minister of Singapore, do hereby proclaim and declare on behalf of the people and the Government of Singapore that as from today, the ninth day of August in the year one thousand nine hundred and sixty-five, Singapore shall be forever a sovereign democratic and independent nation, founded upon the principles of liberty and justice and ever seeking the welfare and happiness of the people in a most and just equal society.“
Speech from Lee Kuan Yew:Separation of Singapore from Malaysia
"For me, it is a moment of anguish. All my life, my whole adult life, I believed in merger and unity of the two territories. ... Now, I, Lee Kuan Yew, Prime Minister of Singapore, do hereby proclaim and declare on behalf of the people and the Government of Singapore that as from today, the ninth day of August in the year one thousand nine hundred and sixty-five, Singapore shall be forever a sovereign democratic and independent nation, founded upon the principles of liberty and justice and ever seeking the welfare and happiness of the people in a most and just equal society.“
1550L
Why 75% Comprehension?
Research suggests that at 75%…• A reader can have a successful reading
experience without frustration or boredom• A reader can achieve “functional
comprehension” of the text• A reader will be sufficiently challenged (by
vocabulary and syntax) to improve
75% is the “right amount of challenge”
Managing Comprehension• Readers may experience frustration when…
−Text readability is 100L+ above their Lexile level.
• Readers may experience ease when…−Text readability is 50-100L below their Lexile
level.• Readers may experience growth when…
−Text readability is within their Lexile range.
General Reading Recommendation: Targeted text range of 100L below to 50L
above the student’s Lexile level
Note: This range may vary based on text type, reading context and purpose,
reading strategies and support, and reader motivation.
Finding targeted text on ecosystems:
Books: Lexile Measure:Inside Ecosystems and Biomes 700LExploring Ecosystems with Max Axiom, Super Scientist 750LEarth’s Ecosystems IG810LEcosystems 860LPopulations and Ecosystems 880LBiomes and Ecosystems 970LThe Eastern Forest 1010LThe Deserts of the Southwest 1080LThe Prairie 1140LEndangered Species 1160L
Time Reading and Reading Achievement
from Anderson et al., 1988, Table 3, N = 155. Reading material included books, magazines, and newspapers.
Fifth-Grade Students
Percentile Rank
Minutes of Text Reading
per Day
Estimated Number of Words
Read per Year
989070502010
67.333.316.99.22.41.0
4,733,0002,357,0001,168,000601,000134,00051,000
College and Career Readiness SkillsReading Demand of Newspapers
• USA Today (1200L)• Associated Press (1310L)• Chicago Tribune (1310L)• Wall Street Journal (1320L)• Washington Post (1350L)• NY Times (1380L)• Reuters (1440L)
College and Career Readiness SkillsReading Demand of International Newspapers
• The Egyptian Gazette (Egypt) 1440L• Oman Daily Observer (Oman) 1430L• Financial Times (Great Britain) 1430L• Straits Times (Singapore) 1410 L• Gulf Times (Qatar) 1420L• China Daily (China) 1400L• France Daily (France) 1400L• The Moscow Times (Russia) 1400L• The Australian (Australia) 1390L• German Times (Germany) 1390L• Copenhagen Post (Denmark) 1390L• Irish Times (Ireland) 1380L• Santiago Times (Chile) 1380L• Jerusalem Post (Israel) 1370L
FREE Lexile Toolswww.lexile.com
• Lexile Find a Book
• Lexile Analyzer
www.Lexile.com
Find-a-Book
Select reading interest:
Search results:
Create a reading list:
Edit reading list to personalize:
Search for books:
Search results:
Link to public library
Link to public library
Tools - Lexile Analyzer
The Lexile Analyzer
The Lexile Analyzer
Copy the article
The Lexile Analyzer
Paste the text into the Word
document
The Lexile Analyzer
Remove titles, pictures, graphs,
illustrations, captions, sub-headings, and
bylines
The Lexile Analyzer
Be sure to remove
hyper-links
The Lexile Analyzer
Be sure to save as ‘Plain
Text’
Oasis A student-centered personalized
learning platform– Does not require teacher direction
Differentiated activities Embedded assessment in the
background Web-enabled with minimal bandwidth
requirement Built on the principles of deliberate
practice…
Demo Accounts for Copenhagen
45 demo accounts have been created inside Oasis for the folks in Copenhagen.
All the usernames are: rasch# where the # is a number from 1 to 45 (eg., rasch1, rasch15, rasch42).
All the passwords are: jakob Oasis can be accessed by browsing to:
http://www.alearningoasis.com and clicking on the green “launch” button. If anyone has issues logging in, they can contact Sean Hanlon – [email protected]
Reader Ability
Temperature
Three well researched constructs
Reader ability
Text Complexity
Comprehension
Reading is a process in which information from the text and the
knowledge possessed by the reader act together to
produce meaning.
Anderson, R.C., Hiebert, E.H., Scott, J.A., & Wilkinson, I.A.G. (1985) Becoming a nation of readers: The report of the Commission on ReadingUrbana, IL: University of Illinois
An Equation
=Reader Ability
Text ComplexityComprehension -
Conceptual
Statistical
ExpectedRawScore
=i
e (RA – TC )i
1 + e (RA – TC i)
RA = Reading Ability
TC = Text Calibrations
A causal model relating reader ability, text complexity, and comprehension.
Measures reader ability and text complexity, on a common scale—the Lexile scale.
Allows educators to forecast the level of success a reader is likely to experience with a particular text.
The Lexile Framework for Reading
Eight Features of the Causal Model Relating Text Complexity, Reader Ability, Task Difficulty and Comprehension1. The model is individual centered. The focus is on explaining
variation within person over time.
2. In this framework the measurement mechanism is well specified and can be manipulated to produce predictable changes in measurement outcomes (e.g. percent correct).
3. Item parameters are supplied by substantive theory and, thus, person parameter estimates are generated without reference to or use of any data on other persons or populations. Therefore, effects of the examinee population have been completely eliminated from consideration in the estimation of person parameters for reader ability.
4. The quantitivity hypothesis can be experimentally tested by evaluating the trade-off property for the individual case. A change in the person parameter can be off-set or traded-off for a compensating change in text complexity to hold comprehension constant. The trade-off is not just about the algebra.
5. When uncertainty in item difficulties is too large to ignore, individual item difficulties may be a poor choice to use as calibration parameters in causal models. As an alternative we recommend, when feasible, averaging over individual item difficulties to produce “ensemble” means. For example empirical text complexities can be excellent dependent variables for testing causal theories.
Eight Features of the Causal Model cont’d.
6. Causal Rasch models are individual centered and are explanatory at both within-subject and between-subject levels. The attribute on which I differ from myself a decade ago is the same attribute on which I differ from my brother today.
8. When data fit a Rasch model, differences between person measures are objective. When data fit a causal Rasch model absolute person measures (reader abilities) are objective (i.e. independent of instrument).
9. Causal Rasch models make possible the construction of generally objective growth trajectories. Each trajectory can be completely separated from the instruments used in its construction and from the performance of any other persons, whatsoever.
Eight Features of the Causal Model cont’d.
May 2016(12th Grade)
1200
1000
1400
1600
Text Demands forCollege and Career
May 2007 – April 2011
347 Encounters138,695 Words3,342 Items983 Minutes
Student 1528
7th GradeMaleHispanicPaid Lunch
Expected: 73.5%Observed: 71.7%
Theoretical versus Empirical Text Complexity for
719 Articles*
Reliability = 0.997
SEM = 12.8L
r = 0.968
r” = 0.969
R2” = 0.938
RMSE” = 89.6L
* Inclusion criteria: 50 encounters and 1,000 items
How Temperature and Pressure Relate Under Constant Volume
Temperature Volume Pressure
2000°K 20 Liters 20.0 atm
1000°K 20 Liters 10.0 atm
500°K 20 Liters 5.0 atm
250°K 20 Liters 2.5 atm
125°K 20 Liters 1.25 atm
Comprehension Rates for Readers of the Same Ability with Texts of Different Complexity or How Reader Ability and Comprehension Rate Relate Under Varying Text Complexity
ReaderAbility
TextComplexit
yText Titles
Comprehension Rates
1000L
1000L
1000L
1000L
1000L
500L
750L
1000L
1250L
1500L
The Magic School Bus, Inside the
Earth (Cole)
The Martian Chronicles (Bradbury)
The Reader’s Digest
The Call of the Wild (London)
On Equality Among Mankind
(Rousseau)
96%
90%
75%
50%
25%
Comprehension Rates for Fixed Reader Ability
Comprehension Rates for Readers of Different Ability with Texts of the Same Complexity or How Reader Ability and Comprehension Rate Relate Under Constant Text Complexity
Reader Ability Classroom Textbook Comprehension Rates
500L750L
1000L1250L1500L
1000L1000L1000L1000L1000L
25%50%75%90%96%
Comprehension Rates for Fixed Text Complexity
How Many Ways Can We Say X Causes Y? X “elicited a greater”
YX “impacts” Y
X “accounts for” Y X “has been linked to” Y
Y “is the result of” X X “didn’t diminish” Y
Y “because of” X Y “depends on” X
X “has led to” Y X “largely motivates” Y
Y “stemmed from” X X “proved critical to” Y
X “fosters” Y X “changes” Y
X “triggers” Y X “affects” Y
To causally explain a phenomenon [a measurement outcome] is to provide information about the factors [person processes and instrument mechanisms] on which it depends and to exhibit how it depends on those factors. This is exactly what the provision of counterfactual information…accomplishes: we see what factors some explanandum M [measurement outcome, raw score] depends on (and how it depends on those factors) when we have identified one or more variables such that changes in these (when produced by interventions) are associated with changes in M (Woodward, 2003, p.204).
Ensemble Calibration
The Psychometric Engine of Oasis
Oasis Items Single Instance No individual item statistics Theoretical estimates of text
complexity become empirical estimates of text complexity
Protocol for determining response illustrations and foils
Machine-generated with no human review
Item Distributions
Items are presumed to be sampled from a distribution with a mean difficulty and an imputed spread
This distribution is called an ensemble
Traditional Rasch Algorithm Iterative process Two stages of iteration
– Begin with asssumed starting values for person abilities and item difficulties (usually item difficulties)
– Estimate person abilities given item difficulties.– Fix these estimated person abilities and estimate new
item difficulties given these person abilities– Iterate until a stopping criterion is met
Ensemble Calibration Algorithm Same iterative technique as traditional
method– Fix person abilities to determine ensemble
means– Fix resulting ensemble means to determine
person abilities– Iterate until stopping criterion is met
What is Different? Individual item difficulties neither
available nor needed Start iterative procedure with person
abilities from Oasis P-delta-sigma table to account for
spread of distributions Spread of ensemble effects success rate
P-delta-sigma table A success rate (p) is determined by the difference in
the ensemble mean for a distribution and the person ability (delta) and the spread of the ensemble (sigma)
Normal distribution assumption for the ensemble of item difficulties
Table used instead of integration over the random term for programming simplicity– Dimensions: (delta -1000 to +1000 by fives,
Sigma 10 to 260 by fives)
Formula for p-delta-sigma
p-delta-sigma = uee
eu
u
u
2
2
2
21
1
Δ=delta=difference between person ability and ensemble calibration
σ=sigma=standard deviation of the ensemble
u=variable of integration=random difference of item difficulty from ensemble mean
Spread of Ensemble
Ensembles vary in mean but not in SD
SD is uniform for this implementation
With embellishment a method for
unequal variances could be derived
ClosingNo matter how it is sliced and diced, analyses of joint and conditional probability distributions yield no more than patterns of association. Nothing in the response data nor Rasch analyses of these data exposes the processes (features of the object of measurement) or mechanisms (features of the instrument) that are hypothesized to be conjointly causal on the measurement outcomes.
