ATOS LexileWhich Readability Formula Is Best?

s p e c i a l r e p o r tvs

.

2

Introduction

Readability formulas are helpful tools used by teachers and librarians to match students with appropriate books. Today’s educators have many different formulas from which to choose. This paper compares ATOS (Advantage-TASA Open Standard), a scientifically research-based readability formula developed by Renaissance Learning, and the Lexile Framework, a readability formula developed by MetaMetrics, Inc.

Key Findings

ATOS is a superior tool for helping educators match books to students because it is

1. Based on extensive research

2. More accurate than Lexile

3. An open standard—meaning it is free and easier to use

ATOS is especially well suited to teachers working with nonfiction, emergent-reader, and high-interest/low- readability level material.

Background on the ATOS Readability Formula

In 1998, Renaissance Learning, in conjunction with Touchstone Applied Science Associates (TASA), began developing a new, improved readability formula to give teachers and students a better tool for estimating text difficulty. The goals of the project were to improve the reliability and accuracy of reading estimates, and to develop a formula specifically designed for matching books to students.

To achieve these goals, it was necessary, among other things, for the formula to deal with evaluating fiction versus nonfiction, books for emergent readers, and high-interest/ low-readability level books for older readers, all of which have posed problems for readability formulas in the past. It was also Renaissance Learning’s goal to create an “open” formula—one that would be available to the educational community free of charge, in an easy-to-use format, and able to be used with any nationally normed reading test.

The resulting project is believed to be the largest and most comprehensive study of readability ever conducted (Milone, 2009).

It produced three different formulas, each of which is used for a different purpose:

• ATOS for Books Readability Formula—a grade-level scale for measuring book readability levels. This formula is used to establish levels for books used with Renaissance Learning’s popular Accelerated Reader (AR) progress-monitoring and personalized reading- practice software, and is available free to publishers and others wanting an improved readability measure for books.

• ATOS for Text Readability Formula—also a grade-level scale for use with other reading materials such as magazine articles, textbooks, and passages from reading tests. Educators may submit passages of text for free analysis through the Renaissance Learning website: www.renlearn.com/ar/overview/atos

• ATOS Readability Formula Conversion Scales—formulas that convert the ATOS grade-level scale to 100- and 2000-point scales similar to those used by TASA’s Degrees of Reading Power (DRP) test and the Lexile Framework, respectively.

Read on to learn why ATOS is the better tool for helping educators match books to students and build a lifelong love of reading.

1ATOS Is Based on Extensive ResearchATOS was developed using comprehensive tools and data, including:

• The largest set of reading test items ever used— more than 650 leveled items from a combination of Renaissance Learning’s STAR Reading (a norm- referenced, computerized reading test) and TASA’s Degrees of Reading Power norm-referenced test.

• The world’s largest word-frequency corpus—474 million words representing all the text of 28,000 K–12 popular trade books in the Renaissance Learning quiz library. Many of these books were published within five years of the ATOS project, making this corpus more relevant than those used in other readability research that date back several decades and include materials other than K–12 trade books.

• An improved and expanded graded-vocabulary list—starting with TASA’s existing list showing the words most often found in books of different grade levels, and expanded based on study of the Renaissance Learning corpus.

3

• The National Reading Practice Database from Renaissance Learning—containing the AR records of more than 30,000 students who read and tested on more than 950,000 actual trade books. Thus, for the first time, substantial data on actual book reading was incorporated into the design of a readability formula.

In Accelerated Reader, level of success in reading practice is measured by percent correct on reading practice quizzes, which is in effect a measure of comprehension. Renaissance Learning recommends that students average 85 percent correct or higher on these quizzes. This recommendation has been validated by several large research studies. For instance, Topping and Sanders (2000) analyzed Tennessee Value-Added Assessment System data and found a positive effect on the teacher effectiveness index when students averaged 85 percent correct or higher.

More recently, researchers at Renaissance Learning validated the 85-percent-correct recommendation by analyzing the latest reading practice and achievement data from the new National Reading Practice Database—which included 50,000 students and 3 million books (Paul, 2003). This research showed that percent correct on AR reading practice quizzes is more important to student reading achievement than the amount of reading practice or the readability level of books. At all ability levels, students experience greater normal curve equivalent (NCE) reading gains as percent correct increases (see Table 1).

Another study used hierarchical linear modeling to replicate Paul’s 2003 findings. Borman and Dowling (2004) found that a high average percent correct on AR quizzes, over the course of the school year, was associated with better reading achievement at the end of the school year. In addition, Bolt

(2004) found that the benefits of a high success rate on AR quizzes held regardless of the core reading curriculum used in the classroom.

Despite the strength of the evidence supporting a high comprehension rate, some reading programs, including the Lexile Framework, target a lower level of comprehension. Specifically, Lexile targets 75 percent comprehension with its book-level recommendations—a benchmark that is not supported by research and, in fact, is described by the publisher as “an arbitrary, but useful, choice of 75% comprehension” (Stenner & Stone, 2004, p. 21).

2 ATOS Is More Accurate Than LexileATOS improves readability scoring of several types of books where previous formulas have

been less accurate and less helpful to teachers—such as nonfiction, emergent-reader, and high-interest/low- readability level books. It also correlates well with other readability formulas.

ATOS employs the three statistics researchers have found to be most predictive of reading difficulty: number of words per sentence, number of characters per word, and average grade level of the words in the book. The formula eliminates sampling error because all books are leveled based on a computerized, full-text scan of the book. That is, ATOS calculates the readability of the entire book, not just a few samples of text within it.

Unlike ATOS, the Lexile Framework is originally based on average word frequency found in the American Heritage Intermediate Corpus (Carroll, Davies, & Richman, 1971) and the average number of words per

Student Achievement Level

(Percentile Range)

Average Percent Correct Range

Below 65% 65%–74% 75%–84% 85%–94% 95%–100%

0–20 –2.08 –0.74 1.57 5.01 3.44

21–40 –3.13 –1.18 0.85 4.72 6.35

41–60 –4.66 –1.06 0.23 5.29 6.77

61–80 –3.95 –2.78 –0.30 3.21 6.08

81–100 –5.72 –4.78 –2.18 2.11 4.93

Table 1. Students Experience Greater NCE Gains as Percent Correct Increases Grades 2–12; N=45,670

4

sentence. Because average word frequency is not as accurate a predictor of semantic difficulty as either word length or word grade level, Lexile is likely to be less accurate than ATOS.

Compared to other readability formulas, ATOS works especially well with books for emergent readers. Other formulas often overstate the difficulty of emergent-reader books. For example, Five Little Kittens, by Nancy Geller Jewell, is a 32-page picture book. The sentences are slightly long and the word “kitten” is used repeatedly. The publisher labels this book as appropriate for grades K–3, ages 5–8. This book has a Flesch-Kincaid level of 4.6 and a Lexile level of 970, which equates to grades 6–7. This is the same range that includes books such as Dogsong by Gary Paulsen and Exploring the Titanic by Robert Ballard.

The ATOS level for Five Little Kittens is a much more reasonable 2.6. By measuring vocabulary level and adjusting for extreme sentence length, ATOS produces a more accurate readability level than either Flesch-Kincaid or Lexile.

Although MetaMetrics, Inc., claims Lexile to be the only system that puts reader and text on the same scale, Renaissance Learning has for years published tables that put reader and text on the same scale.

Teachers match books to students using Renaissance Learning’s Goal-Setting Chart. This ATOS companion provides suggested ranges of book difficulty for students’ grade-equivalent scores in reading. Because it is based on grade-equivalent score, any norm-referenced standardized test of reading achievement can be used. (For a free copy of this chart, visit http://doc.renlearn.com/KMNet/R003792601GF8A5B.pdf.)

While the Lexile Framework also provides a chart showing recommended reading ranges, the Lexile ranges are much narrower than those on Renaissance Learning’s Goal-Setting Chart, especially for grades 5 and above. This is due to Lexile’s adoption of the arbitrary 75% correct rule rather than basing their recommendations on empirical evidence, as we do with ATOS. This will likely result in frustration and lack of motivation on the part of students as Lexile ranges do not provide the flexibility to read a wide range of books. Table 2 shows a comparison of Renaissance Learning ZPD ranges and Lexile ranges.

The reading ranges on the Goal-Setting Chart are based on the theoretical concept of zone of proximal development, or ZPD (Vygotsky, 1978). In independent, literature-based reading, ZPD is the range of books that will challenge a student without causing frustration or

Grade Equivalent

Renaissance Learning ZPD Range

Lexile Range Converted to ATOS Scale for Comparison

Low High Low High

5.0 3.4 5.4 3.8 6.4

5.5 3.7 5.7 4.0 6.8

6.0 4.0 6.1 4.3 7.4

6.5 4.2 6.5 4.6 8.0

7.0 4.3 7.0 4.9 8.5

7.5 4.4 7.5 5.0 8.7

8.0 4.5 8.0 5.2 9.1

9.0 4.6 9.0 5.8 10.1

10.0 4.7 10.0 6.3 10.9

11.0 4.8 11.0 6.6 11.3

12.0 4.9 12.0 6.9 11.7

Table 2. Renaissance Learning’s ZPD Ranges Allow Students to Read a Wider Range of Literature

5

loss of motivation. The ranges become increasingly broad as students move up in grade equivalent. This recognizes that much good literature does not have a very high reading level. Nor does it contain leveled vocabulary, so even if books are easy for students to read/comprehend, they are still exposed to new vocabulary. In addition, once students become proficient readers, it is important they read a wide variety of books. The use of wide reading ranges ensures students have access to a large selection of good books that are interesting to them.

The ZPD ranges on the Goal-Setting Chart were developed and updated over the years using actual student reading data (Renaissance Learning, 2003). The ranges were also validated through a sophisticated analysis of reading data from more than 50,000 students (Borman & Dowling, 2004). This research showed that students who read within their ZPD ranges experience greater gains in reading achievement than students who read below their ZPD ranges. Students who read above their ZPD can also experience gains, as long as they are reading with a high level of comprehension.

3 ATOS Is an Open StandardThe ATOS Readability Formula is an “open standard,” meaning that teachers can use grade-

equivalent scores from any nationally normed reading test to determine each student’s reading level.

It also means ATOS analysis is free of charge. (Lexile charges state departments of education and test publishers for the use of the Lexile scale.)

With an open standard, teachers have the flexibility to use one readability formula for textbooks and another for literature-based reading. In addition, systems such as Renaissance Learning’s ZPD tables, which express book readability on a grade-level scale, allow teachers to use their students’ grade-equivalent scores from any test to match students with appropriate books.

The ATOS formula’s ease of use is especially helpful in communications with students and parents. For example, in the ATOS system, a reading level of 4.5 translates simply to “fourth year, fifth month.” The Lexile Framework—a closed standard—uses a non-grade-level scale that is more difficult to compare and understand, and makes more work for teachers and parents. For instance, a student with a reading level of 4.5 might be assigned an enigmatic “700” in Lexile.

Because Lexile is a closed standard, teachers and librarians in districts adopting this formula may be forced to use only those tests and reading materials that report readability levels according to the Lexile formula. Obviously, this arrangement limits the choices in reading materials available to teachers and students.

To also help educators and in keeping with the concept of open standards, Renaissance Learning has developed conversion charts that correlate ATOS for Books with a 100-point scale like the one used by DRP and a 2000-point scale like the one used by Lexile. Table 3 shows sample ATOS levels and the corresponding 100-point and 2000-point scale values. For the full conversion table, see The Development of ATOS: The Renaissance Learning Readability Formula (Milone, 2009): http://research.renlearn.com/research/pdfs/334.pdf

ATOS Levels

ATOS 100-Point Scale

Values

ATOS 2000-Point Scale

Values

0.3 21 23

0.5 27 31

0.7 30 39

1.0 34 51

1.3 37 120

1.5 38 157

1.7 40 190

2.0 41 241

2.3 43 301

2.5 44 348

2.7 44 381

3.0 46 440

3.3 47 491

3.5 47 530

3.7 48 561

4.0 49 619

4.3 50 669

4.5 50 701

4.7 51 731

Table 3. ATOS Conversion Chart to 100- and 2000-Point Scales

6

Finally, while Lexile is proprietary, ATOS is available for educators to use on any text for which a readability level is needed.

Conclusion

Readability formulas are important tools. They give teachers and librarians a place to start in their task of matching books to students, and we hope, creating a love of reading in all students. The ATOS for Books Readability Formula, designed especially to accurately level trade books, is a superior tool for accomplishing this task—especially for teachers working with nonfiction, emergent-reader, or high-interest/low- readability material.

Learn More About ATOS

Download a copy of The Development of ATOS: The Renaissance Learning Readability Formula from http://research.renlearn.com/research/pdfs/334.pdf, or contact Renaissance Learning for a print copy.

Table 3. (continued)

ATOS Levels

ATOS 100-Point Scale

Values

ATOS 2000-Point Scale

Values

5.0 52 781

5.3 53 829

5.5 53 860

5.7 54 888

6.0 55 921

6.3 56 951

6.5 57 979

6.7 57 999

7.0 58 1029

7.3 60 1061

7.5 60 1080

7.7 61 1098

8.0 61 1121

8.3 62 1149

8.5 62 1161

8.7 62 1178

9.0 63 1201

9.3 63 1237

9.5 64 1258

9.7 64 1272

10.0 65 1293

10.3 65 1314

10.5 65 1328

10.7 66 1342

11.0 66 1364

11.3 66 1385

11.5 67 1399

11.7 67 1413

12.0 67 1434

12.3 68 1455

12.5 68 1469

12.7 68 1483

13.0 69 1504

7

References

Bolt, D. (2004). HLM analysis of effect of Reading Renaissance implementation on various reading curricula. Unpublished manuscript, University of Wisconsin–Madison.

Borman, G. D., & Dowling, N. M. (2004). Testing the Reading Renaissance program theory: A multilevel analysis of student and classroom effects on reading achievement. Unpublished manuscript, University of Wisconsin–Madison. Available online from

http://www.education.wisc.edu/elpa/people/faculty/Borman/BormanDowling2004_RdgRenProg.pdf

Carroll, J. B., Davies, P., & Richman, B. (1971). The American heritage intermediate corpus. New York: American Heritage Publishing Company.

Milone, M. (2009). The development of ATOS: The Renaissance readability formula. Wisconsin Rapids, WI: Renaissance Learning, Inc.

Paul, T. D. (2003). Guided independent reading: An examination of the Reading Practice Database and the scientific research supporting guided independent reading as implemented in Reading Renaissance. Wisonsin Rapids, WI: Renaissance Learning, Inc.

Available online from http://research.renlearn.com/research/pdfs/165.pdf

Renaissance Learning. (2003). The research foundation for Accelerated Reader goal-setting practices. Wisconsin Rapids, WI: Author.

Stenner, A. J., & Stone, M. H. (2004, May). Does the reader comprehend the text because the reader is able or because the text is easy? Paper presented at the International Reading Association, Reno-Tahoe, NV. Durham, NC: Metametrics. Available online from

http://www.lexile.com/m/resources/materials/Stenner_Does_the_Reader_Comprehend_the_Text....pdf

Topping, K. J., & Sanders, W. L. (2000). Teacher effectiveness and computer assessment of reading: Relating value-added and learning information systems data. School Effectiveness and School Improvement, 11(3), 305–337.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

1L2211.1209.NS.1M

R35200AR, ATOS, Accelerated Reader, Advanced Technology for Data-Driven Schools, Renaissance, Renaissance Learning, and STAR Reading are trademarks of Renaissance Learning, Inc., and its subsidiaries, registered, common law, or pending registration in the United States and other countries. Degrees of Reading Power, DRP, and TASA are registered trademarks of Touchstone Applied Science Associates, Inc. The Lexile Framework is a registered trademark of MetaMetrics, Inc.

For more information, or for additional copies of this report, contact:

Educational Research DepartmentPO Box 8036 • Wisconsin Rapids, WI 54495-8036(800) 338-4204 • www.renlearn.com

© 2006 Renaissance Learning, Inc. All rights reserved.