policy conversations - the ounce of prevention fund · 2018. 2. 12. · rethinking state education...

IntroductionIn the last decade-plus, statewide accountability systems have emerged as a strategy for improving child outcomes, particularly for low-income and minority children. It is clear that state accountability systems have changed the behavior of schools. But to date, accountability* efforts in both early learning and K–12 public school systems have not set the right goals for educators. Moreover, both early learning and K–12 have struggled to generate the capacity needed to improve schools† at scale, and the strategies currently being used for improvement have frequently not had the intended effect. For a state accountability system to truly succeed, it must both set the right goals and provide the right supports for achieving those goals. This paper proposes a policy framework that redefines statewide accountability based on the best elements of existing early learning and K–12 accountability systems in order to strengthen state approaches to educational improvement.

* In this paper, the term “accountability” draws from Martha Thurlow’s definition: “Educational accountability targets either the processes or results of education. A desired goal is identified and measures are identified for determining whether the goal is met.” Thurlow, Martha. (2009). “Accountability.” The Gale Group. http://www.education.com/reference/article/accountability/.

† In this paper, the term “school” refers to both K–12 public schools and school- or center-based early learning providers with an educational focus.

Change policy, cause a ripple effect.

A framework for RETHINKING STATE EDUCATION ACCOUNTABILITY AND SUPPORT

from birth through high schoolelliot regenstein

rio romero-jurado1

policy conversationsConversation No. 3 Version 1.0 June 3, 2014

2the ounce a framework for rethinking state education accountability and support from birth through high school v. 1.0

Conversation No. 3 Version 1.0 June 3, 2014

policy conversations

The broad goals for state accountability systems are widely agreed upon: Accountability systems are supposed to measure the professional practice of schools and then help schools improve their practices as a means of achieving better student outcomes. But to date no consensus has emerged about how to measure practice, how to help schools improve, or what student outcomes should be measured. While there is not yet clear national agreement on how accountability systems should work, there are promising trends to improve state accountability systems in both early learning and K–12 (which to date have been entirely separate systems). States can build on the best ideas in both early learning and K–12 accountability systems to create a single state education-accountability system from birth through high school—one that sets the right goals and identifies the supports needed to help achieve them.

In both early learning and K–12, recent federal efforts have helped catalyze state-level progress in goal-setting and the development of better supports. In early learning accountability, there has been an increasing focus on child outcomes as an essential goal, and greater attention to the need for metrics based on professional practice—metrics that can be linked to child outcomes. In K–12 accountability, there have been meaningful changes in what child outcomes are being sought. As for supports, the federal Early Learning Challenge grant pushed states to strengthen their early learning quality-improvement processes; at the K–12 level, federal waivers of the Elementary and Secondary Education Act (ESEA) have prompted shifts in thinking about school improvement. There is promise in these efforts, but they also have significant limitations.

While existing separate initiatives have led to incremental progress, states may be able to make greater advances by combining the best ideas of early learning and K–12.* Through this combination, states could create birth-to-high-school accountability systems that really do set the right goals and provide the right supports. The framework for a combined system could include states using the following elements in their accountability systems:

• Performance metrics that set the right goals for educators at both the early learning and K–12 levels. These goals will include metrics in two major categories: professional practice and child outcomes. Using professional-practice metrics will provide diagnostic information that can be used to improve practice, and child outcomes metrics can take a broader view of success than just proficiency in English language arts and math.

• Performance measurements† to track the metrics that include observations, which will provide information that school leaders and teachers can use to improve practice. This also will require child

* In this paper, the term “early learning” means school-based or center-based programs for children from birth through kindergarten entry that provide standards-based education; while many other programs for young children can and do provide early learning, this paper focuses on the subset of early learning programs that are or could be appropriately subjected to education accountability.

† In this paper, the term “metrics” means data that captures an outcome, and “measurements” means the method by which that metric is collected. For example, “student math achievement” is a metric, and “standardized math assessments” is a measurement by which that metric could be established.




outcome measurements that go beyond standardized assessment* results to measure skills like executive function and cognitive processes that underlie lifelong learning.

• Performance tiers that provide transparent and easily understood information for parents about school quality but that are not used to determine punitive consequences for schools.

• Supports that are based on a detailed diagnosis of school performance and child need and are designed to improve professional practice and child outcomes. At the early learning level this may require fundamental shifts in resource allocation, given the number of children who currently do not have access to high-quality early learning.

Accountability systems are a promising approach because they have a demonstrated record of influencing decision-making at both the policy and practice level. If better accountability systems lead to improved decision-making about the use of resources, then they can play a key role in improving child outcomes. If a major purpose of early learning and K–12 accountability systems is to guide resource use in a way that improves educator practice and child outcomes, perhaps this moment of rapid change in both systems is an opportunity to bring the best thinking of each together to develop a unified accountability system that gives more complete information about what’s happening in our schools and helps drive the change needed to make things better.

This paper first lays out its proposed framework for how education accountability and support systems could work. It then discusses existing accountability and support systems in early learning and K–12, identifying specific ways existing systems need to be improved. It concludes by identifying steps to begin improving and unifying state education accountability and support systems.

* In this paper, the term “assessment” reflects the US Department of Education’s definition of “assessment” in the Race to the Top program: An assessment is “designed to measure a student’s knowledge, understanding of, and ability to apply critical concepts through the use of a variety of item types and formats.” US Department of Education. (December 2009). “Race to the Top Program Executive Summary.” http://www2.ed.gov/programs/racetothetop/executive-summary.pdf.

http://www2.ed.gov/programs/racetothetop/executive-summary.pdf




I) A Proposed Theory of Action for Birth-Through-High-School Education AccountabilityState education-accountability systems can support improved outcomes,2 and it is clear that they have the power to significantly change behavior.3 Because state-level metrics drive local-level action, states need a set of metrics that provide local actors with the right incentives. That will require using new metrics and measurements that capture a better set of desired outcomes than the outcomes currently driving state accountability systems. States also need to report results in ways that better inform the public and act on results in a tailored manner that leads to improvement—focusing state resources on helping the children most likely to need dramatic change in their schooling to succeed. An accountability system of this kind would not only change behavior, it would lead to improved outcomes for students.4 This paper is meant to offer a starting point for discussion of what a more effective accountability system might look like.

A) The Right Performance MetricsBecause accountability metrics drive behavior, it is essential that the metrics be focused on the right goals. Stated broadly, the goal of the education system is that children achieve positive outcomes and that they are supported in doing so by adults engaged in high-quality professional practices. This high-level statement addresses both the end (child outcomes) and the means (professional practices). For states to develop metrics that support this statement of the goal will require a combination of metrics that address the ends and the means, not just one or the other.5

Existing accountability systems have suffered from a tendency to focus on only outcomes or practice and to take an inappropriately narrow approach within their area of focus.

• K–12 accountability has focused on child outcomes but has defined those outcomes very narrowly. It relies primarily on outcomes measured by standardized tests in a limited range of subjects—which is an indirect way to improve the practice of educators,6 and has also meant that strong accountability results are frequently highly correlated with wealth.7 Given the limited scope of standardized testing for accountability purposes, there is also significant concern that the accountability system focused solely on child outcomes is not even fully capturing the child outcomes desired from the system.8

• Early learning historically has focused on professional practices through input metrics—that is, metrics that do not directly measure child outcomes (although some are linked to improved child outcomes through research). But while early learning accountability has focused on adult practices, in many instances it has done so through checklists of activities that are not necessarily linked to improved teaching.9 This has resulted in programs improving their accountability ratings without either improving their teaching practice or producing better child outcomes.

New accountability systems should include metrics addressing both child outcomes and professional practice—and in doing so can build on lessons learned from ongoing improvement efforts in both early learning and K–12.




1) The Right Performance Metrics: Professional Practice

If a major goal of accountability is to lead to improved practices in schools, then accountability systems should actually measure school and professional practice directly. There is an emerging literature about what school practices are most important for success. A landmark study by the University of Chicago Consortium on Chicago School Research identified five essential supports for school improvement (referred to as the “five essentials” or “5E”)—which are organizational features of schools that, when present, improve the chance of positive student outcomes. These features are:

1. A coherent instructional guidance system

2. Professional capacity

3. Strong parent-community-school ties

4. Student-centered learning climate

5. Leadership to drive change

Research has shown that a substantial, sustained weakness in even one of the essential features significantly threatens the ability of the school to improve—and if three or more of the five essentials are strongly present, the school is at least ten times more likely to improve over time.10 These supports can provide a starting point for states in developing research-based metrics of professional practice. Research by the Ounce of Prevention Fund and the Consortium on Chicago School Research is also under way to specifically apply the 5E in the early learning context.11

The Five Essential Supports for School Improvement (referred to as the “five essentials” or “5E”) are based on longitudinal research by the University of Chicago Consortium on Chicago School Research that determined five key ingredients or “essential supports” to school success. Schools strong in most of the essential supports were at least ten times more likely to achieve significant academic gains and experience improved student attendance and engagement compared with schools showing strengths in only one or two areas. The 5E are:

1. A coherent instructional guidance system: Schools have strong classroom curriculum alignment where instruction is academically challenging, engaging students, and provides adequate tools for teachers to advance learning.

2. Professional capacity: Schools recruit quality teachers and staff and provide ongoing professional development that fosters strong commitment to improvement and capacity to work together.

3. Strong parent-community-school ties: Schools create a welcoming environment for parents and families, and build partnerships with local community institutions.

4. Student-centered learning climate: Schools provide a safe, stimulating, and nurturing learning environment for all students.

5. Leadership to drive change: The principal actively works with teachers to develop a clear, strategic vision for school success that is inclusive and focused on improving instructional practice.14

The five essentials provide a research-based framework for whole-school improvement. The 5E assessment tool, which draws from the research framework, reliably measures and predicts which schools will demonstrate success or stagnate over time.15 The state of Illinois is now using the 5E as part of a statewide survey of students and teachers. State legislation in 2011 required the Illinois State Board of Education (ISBE) to implement a survey that captured the learning conditions and climate of schools. ISBE partnered with the University of Chicago to develop the Illinois 5Essentials Student Survey for students in grades 6–12 and the 5Essentials Teacher Survey for pre-kindergarten–12th grade certified teachers.16 While school districts in other states have used the survey, Illinois is the first to administer it statewide across all districts. The survey results are intended to help frame and direct school improvement planning.17




States may also want to work on aligning their school accountability metrics with the practice metrics used in their statewide teacher evaluation systems, given the extensive work states have done in recent years to improve teacher evaluation.12 Interestingly, at the K–12 level, the work on teacher evaluation has been in some ways the reverse of what has been proposed here—in teacher evaluation, states have sought to integrate child-outcome metrics to a process that historically has not included them,13 as opposed to adding professional-practice metrics to a process that has been based solely on child-outcome metrics.

States will undoubtedly need to proceed carefully in this area, given that the use of metrics focused on professional practice has not been a focus of accountability systems in the past. In doing so, states should be mindful that there is a value to minimizing the number of professional-practice metrics: If there are too many metrics for schools to easily keep track of, all of the metrics will lose value. This has been a key lesson in some early learning accountability systems where the sheer volume of requirements minimizes the importance of any individual requirement, even when some requirements are clearly more important than others.

2) The Right Performance Metrics: Child OutcomesChild outcomes should be an important part of any state accountability system, but “child outcomes” and “child standardized test scores” are not the same thing. For example, at the high school level, graduation rates are a form of child outcome that is already included in accountability systems.18 The positive outcomes that policy leaders and parents want from the education system are broader than just high scores on standardized tests,19 and the child outcomes for which we have accountability metrics should be broader than standardized tests to reflect that.20 A move to a new accountability system could force a productive conversation about exactly what child outcomes states desire from and prioritize for their schools; clearly, English-language arts and math achievement are important, but just as clearly they are not the only outcomes a state might want from its education system. States also might reasonably differ between themselves on what outcomes they want, with some states emphasizing English-language arts and math and others focusing on a broader range of metrics.

The right child outcomes metrics will likely vary across age spans. For example, even if there were no federal requirement to do so, many states would likely include high school graduation rates in their high school accountability, but it is likely that no states would include high school graduation rates in their elementary school accountability.21 In the earlier years, child outcomes should be tied to developmental milestones, with recognition that in the younger years children develop different skills at different rates and that wide variance is appropriate and expected.22 In that spirit, it will be important for child outcomes to incorporate child growth, which is likely to be a better measure of school performance than proficiency.23 Overall, the approach to child outcomes as accountability metrics should continue evolving toward a broader range of outcomes that focus increasingly on growth and go beyond assessment scores.




Unfortunately at this time, neither early learning nor K–12 offer significant lessons in how to mix a range of metrics reflecting different desired child outcomes. States will need to identify the best possible child-outcome metrics and balance those with professional-practice metrics. States may choose to have different balances for different age spans—most likely with a gradual shift in balance toward child outcomes as children get older.

B) The Right Performance Measurements

If an accountability system is going to include metrics focused on professional practice and child outcomes, then the measurements it uses must address both. Measures that are meant to capture whether schools are engaged in high-quality professional practice require external observation—which to date has not played a significant role in K–12 accountability, but has played a meaningful and growing role in early learning accountability. As for child outcomes, historically, standardized tests have been the dominant child-outcome measurements, despite widespread discomfort with their role. In measuring both professional practice and child outcomes, there are lessons to be learned from existing practice here and abroad that can help update measurements to use in state education-accountability systems.

1) The Right Performance Measurements: Professional PracticeMeasuring professional practice for school accountability is already common practice in England, where a school inspectorate is a key part of school improvement.24 Since the early 1990s, the English public school system has put in place independent inspectors to evaluate the overall quality of its schools in an effort to hold them accountable and promote improvement while also offering parents more information. External inspectors make on-site visits generally once in each three- to six-year inspection cycle, although the frequency of inspection varies based on past school performance. The inspections typically occur on short notice so as to limit last-minute school preparations for the visits. Inspections focus on reporting student learning progress, the quality of instruction, the behavior of students and safety climate of the school, and the effectiveness of school leadership and management. The inspections utilize a blend of metrics to assess school performance, including test scores, classroom observation, and staff, student, and parent feedback. Importantly, these areas of focus are aligned with those described in the five essentials. To measure learning and development in early learning, inspectors are also advised to study a best practices framework for child development that lays out critical areas of learning for young children (noting that these are not a prescribed set of expectations).25

The inspector decides overall school performance based on a grading scale of (1) outstanding, (2) good, (3) requires improvement, and (4) inadequate; schools receiving a failure rating may be subject to increased and intensive inspections.26 After inspections are conducted, inspectors provide schools feedback about areas that need improvement at the student and school levels and offer suggestions on what can be done to improve. Information is then released to the public on the Internet through a full report that includes a description of the school, the inspection’s findings, and the school’s overall grade.27 The results from these inspections




have been shown to have overall positive impact on school improvement with regard to higher academic achievement, quality of instruction and supports, improved management and leadership, and enhanced connections among teachers and students.28

While the English system may be the best known in the United States, inspection-based accountability is also used elsewhere. For example, the Dutch education system has implemented an inspectorate process for assessing schools in a way that shares information with the public about school performance and identifies poor performing schools in need of additional inspections. The goal is not to penalize the school but to respond to the inspectorate findings by providing resources to build school capacity to overcome the barriers identified.29

While English- and Dutch-style inspections are not currently a feature of K–12 accountability in the United States, the practice of external review is already common in US early learning accountability:

• As part of child care licensing, reviewers typically observe early childhood programs for compliance with basic structural and environmental quality features including health, safety, and physical environment.30

• Accreditation of child care programs—which is voluntary and aims to meet higher standards than licensing rules—generally includes a site visit by an external observer in addition to a self-assessment component.31

• Trained independent assessors have also been used by state quality rating systems to make on-site visits and administer standardized assessment tools like the Classroom Assessment Scoring System (CLASS) and Environment Rating Scales (ERS) to rate and verify a program’s level of quality (see sidebar, next page).

• States have used external observers to measure preschool quality and support improvement efforts.32

• Head Start programs are also subject to monitoring reviews that require assessors to use research-based observations for measuring dimensions of program quality, such as classroom environment, teacher-child interaction, and teacher instructional practice.33

Over time, some states have moved from assessors reviewing a checklist of basic aspects of quality to evaluating key elements of quality care and instructional practice.34

An important assumption underlying this approach is that quality teaching can be defined and measured.35 This is an area where state efforts to develop new teacher evaluation systems may be instructive, and there are teacher evaluation rubrics that can be used and improved upon.36 (See sidebar, page 10) In measuring the quality of teaching, it is important to note that instructional practice must be age-appropriate, and that in early learning, instructional practice includes substantial play-based teaching and learning.37

Accreditation processes are another possible model for the measurement of professional- practice




quality in schools serving all age ranges. Even though external review is not generally required at the K–12 level by state accountability systems, many schools use outside accreditors to evaluate their professional practice. In early learning, quality rating and improvement systems (QRIS) can give credit to programs for being externally accredited, in some instances by automatically assigning them high program ratings.43 While K–12 schools may not receive an equivalent benefit, some of them still use accreditation as part of a continuous quality-improvement process.44

An accountability system based on outside observations of schools and educator practice would be more expensive than existing accountability; maintaining a staff of well-trained observers is an expense states currently are not bearing.45 States can mitigate this to some degree by focusing observations on the schools likely to need them most,46 but this work cannot be done without repurposing resources. If improved accountability leads to improved graduation rates and college success, then the up-front costs will have a long-term payoff. But it must be acknowledged that in many states the up-front cost of measuring performance would be a barrier.

2) The Right Performance Measurements: Child Outcomes

Child outcomes as measured by standardized tests are the key drivers of accountability in K–12 education, with high school graduation rates also playing a role in high school accountability.59 These metrics have been used in part because measurements for them are readily available; indeed, in some instances it appears that metrics are used primarily because they are easily measured, not because they meaningfully capture student progress or predict later learning.60

Classroom Assessment Scoring System (CLASS) and Environment Rating Scales (ERS)

The Classroom Assessment Scoring System was developed at the Curry School Center for Advanced Study of Teaching and Learning at the University of Virginia as part of a national study on early learning. The CLASS, currently distributed by Teachstone, is an observational tool used to assess classroom quality in grades pre-k to 12. It is currently implemented by the Office of Head Start and also in state and local efforts to measure and enhance early education and care programs.38 The instrument—which includes versions specifically for infants and toddlers as well as preschoolers—focuses primarily on measuring teacher behaviors and teacher-child interactions shown to be linked to student achievement and development.39 The CLASS categorizes teacher-student interactions into three domains (emotional support, classroom organization, instructional support) and also further organizes interactions into multiple dimensions. These are dimensions of teaching and learning with an empirical link to academic and social outcomes for children.40 Information and results from the tool are used to support teacher professional development needs, improve instructional practice, and guide school and program-wide goals and policies. The CLASS tool has been validated and tested reliable in more than 3,000 classrooms.41

The Environment Rating Scales, designed by the Frank Porter Graham Child Development Institute at the University of North Carolina at Chapel Hill, assess the overall process quality of early learning and school-age care programs. The observational tool evaluates quality by measuring physical learning environment in a broad sense. The instrument, researched-tested as valid and reliable, focuses on assessing the following: physical environment, basic care, curriculum, interaction between staff and children and among children themselves, schedule and program structure, and parent and staff education. ERS is currently implemented for program improvement and evaluation purposes related to early learning licensing, accreditation, credentialing, and quality-rating systems.42




If states have a conversation about which metrics matter to them, they may find that of the metrics that matter, most do not have measurements attached to them—for example, civic engagement or art and music proficiency. A new state of the art is emerging in accountability measurements, but it may take states time to develop measurements that meet their needs.

Early learning accountability systems have not historically included child outcomes, although leaders in the field have called for it to do so.61 Leading researchers in the field have written that it is inappropriate to use child-assessment results for program accountability with children 8 years old or younger.62 States seeking to include child outcomes in early learning accountability should therefore identify child-outcome metrics that can be measured through developmentally-appropriate measurement tools. In some cases, states may identify appropriate measurements that are already available that they would like to include in early learning accountability; in other instances, they may not, creating pressure to develop new measurements. This too is an area where further development is needed.

A large and growing body of research demonstrates that early learning educators’ approach to interacting with students in the classroom is highly predictive of children’s achievement.47 Children in the early childhood years learn within the context of trusting, supportive relationships—with caregivers, parents, and teachers.48 Research therefore shows that effective practice in early education involves positive and emotionally supportive teacher-child interactions in which teachers demonstrate respect, responsiveness, sensitivity, enjoyment, and regard for students’ perspectives.49

Effective practice is also language- and content-rich and intentional, with a focus on developing young children’s higher-order reasoning skills through the use of evidence-based, developmentally appropriate curricula and strategies focused on interactions between teacher and child, including open-ended questions, conversation, modeling, variety, play-based engagement, and promotion of students’ interests.50 These interactions help facilitate joint attention—paying close attention to what has captured a child’s interest and staying with the child’s interest through commentary and shared physical manipulation of classroom tools.51 Joint attention not only supports language and vocabulary development but is also an important way adults show children they are interested in them and their ideas.52 Infants and young toddlers

might not be able to say what they are interested in, so effective teachers in infant and toddler classrooms need to be keen observers, paying careful attention to what children look at, react to, and point to—the behaviors that communicate their interests, likes, and dislikes.53 Effective teachers follow a child’s eye gaze and talk specifically about what the child is looking at.54

Building on this research, effective, comprehensive early learning curricula often structure time in the day for preschoolers to act on their interests by choosing areas in the classroom to play, materials to incorporate into their play, and scenarios to act out in their play with other children, under educators’ guidance.55 During these particular times in the day, teachers create episodes of shared attention with preschoolers, following their ideas, interests, and scenarios, and offering commentary, content-specific vocabulary, and questions to help children expand the complexity of their vocabulary, syntax, and play.56 Effective early learning educators gently guide children’s learning by populating the classroom with enriching materials and facilitating engagement with peers and adults, creating opportunities for exploration, discovery, and social development.57 While providing ample opportunities for such child-directed activities, early learning classrooms should be well-organized, with clear expectations, efficient behavior management, and predictable routines and transitions.58

Best Practice Pedagogy in Early Learning




C) The Right Performance Tiers

State accountability systems at both the K–12 and early learning levels sort schools into performance tiers, which serves several primary purposes. One is the important value of providing information for parents and the public about the quality of programs.63 The standards for inclusion in higher tiers set expectations for schools by defining a pathway to improvement and what is supposed to be achieved.64 A somewhat different purpose has been that tiers organize the provision of incentives and supports for schools, with different incentives and supports tied to placement in particular tiers.65

Changing the metrics of accountability does not substantially change how states should approach organizing tiers for the purpose of informing the public and setting expectations for schools. Incorporating professional-practice metrics should increase the connection between performance tiers and the actual quality of instruction being provided. However, setting performance tiers and weighing multiple metrics is already a complex exercise and is in many ways more art than science.66 It may be reasonably straightforward for states to identify the schools in their top performance tier as those with both outstanding professional practice and excellent child outcomes—and it may also be reasonably straightforward to identify the schools in their lowest performance tier as those where results are just the opposite. Sorting out schools in the middle is already a challenge, however, and even with improved metrics it will likely take some time for states to sort out how best to differentiate among schools in the middle tiers.

Importantly, the use of the professional-practice metrics in accountability should relieve pressure on performance tiers to serve their other function, identifying school support needs. Existing systems group together into tiers schools with similar test scores that may have very different challenges because existing accountability measures do not identify what practices a particular school needs to improve. When consequences are based on tiers, the consequences are not necessarily well suited to the problems a particular school faces—to date, in many instances, there has been a misalignment between the reasons a school was identified for interventions and the interventions actually being used.67 School inspections would provide much richer information about what behaviors need to be changed and would allow states to mobilize supports to address the actual problems.68 Consequences based on performance tier have been a relatively blunt instrument; integrating professional practice into accountability should substantially increase the precision with which supports are designed to lead to improved professional practice.

D) The Right Supports

Developing the right system of supports in states will require an ongoing process that integrates substantial feedback from the field, but a focus on practice-oriented metrics and measurement increases the chances that the right supports will be mobilized. Diagnostic information from school observations can be useful to improvement whether or not the school becomes the recipient of state improvement assistance. States will likely be most effective and efficient if they focus their resources




on the schools with the lowest professional capacity, which the new metrics will actually identify. States can also then focus on ensuring that any resources provided align with core issues of a school so that they meaningfully address the challenges at hand.69 Ideally, supports are based on the diagnosis of need at a school, are designed to build the capacity of the school to address those needs, and include a focus on improving learning and instruction.70

While the focus of accountability should be on supports, those supports must be backstopped by consequences. Many schools given awards to undergo major improvement still face challenges to making significant gains.71 Both early learning and K–12 accountability systems have included different provisions for significant consequences when schools are incapable of improving even with support.72 Punitive measures should be used only after schools have been given a reasonable chance to improve and failed to do so.73 However, if a school cannot perform even after receiving assistance, then it is appropriate to insist on major changes in school management for the school to continue receiving government funding.74

In Early Learning, Supports Are Necessary But Not Sufficient

Supports will be essential to education accountability in both early learning and K–12, but it is important to acknowledge that while children are guaranteed access to K–12 public schools that are designed to provide an educational experience, in early learning there is no such guarantee. Accordingly, many children do not have access to early learning at all. Moreover, historically many of the programs serving children birth through age five have not been designed or funded sufficiently to provide a high-quality educational experience.

In recent years, significant progress has been made in providing educational opportunities to young children. The need for high-quality early learning is well established.77 All states now have learning standards for 3- and 4-year-olds, and most have learning standards for infants and toddlers,78 which establishes a framework for birth-to-five education. However, the funding stream serving the most children in this age group—federal and state child care dollars—is primarily a work support and is neither designed nor funded to provide standards-based education.79 Even many state pre-k programs* are not funded adequately to provide a quality educational experience.80 The low per-child funding rate for these programs has made it difficult to deliver high quality, in part because this funding approach remains insufficient to hire and retain the best teachers.81

Because many early learning programs have not been provided adequate funding to support high-quality professional practice, the imposition of true educational accountability on early learning systems is likely to show that the quality of professional practice needs substantial improvement in many early learning programs. Given that, we strongly believe that states should do more than just provide targeted assistance to early learning programs that receive low ratings in accountability systems—they should fundamentally change how they fund early learning to ensure that programs provide a high level of professional practice, particularly programs serving children who without state support would have no access to standards-based early education. While that fundamental change in how states approach early learning is by no means an inevitable outcome of shifting to a birth-through-high-school education-accountability system, we are hopeful that it would be a major result of increased education accountability in the pre-kindergarten years.

* State pre-k programs are generally defined as state-funded, standards-based education programs serving 3- and 4-year-olds.




Integrating professional practice into accountability would improve the ability of states to identify and provide the right supports for schools, shifting the focus of accountability systems from identifying problems to solving them. But it must be acknowledged that problem solving at the state level is not an exercise that has historically been successful at scale. Indeed, state-level problem solving is often hampered by the fact that the term “state” oversimplifies in early learning what is often a complex web of agencies and offices; early learning governance is frequently spread across multiple agencies,75 and low-performing school districts are often subject to oversight by multiple offices of the state education agency,76 meaning that “state” assistance is not always so simple to mobilize. Moreover, state offices typically have insufficient capacity to help the full range of schools that need assistance, and frequently lack the capacity to develop the nuanced understanding of individual communities necessary to be an effective partner to schools in a wide range of local contexts. States also have not always designed supports that account for the fact that improvement efforts generally will not succeed without strong school-level leadership. As states seek to build their capacity to help local schools, they should focus on what is actually achievable from the state level given their role, and what state-level changes will be needed to accomplish that which is achievable.

proposed birth to high school education accountability

Ages 0–3 Ages 3–5 Grades K–2 Grades 3–8 High School

Metrics

• A mix of metrics balancing child outcomes with professional practice

• Child outcomes will vary across age spans

• Balance between child outcomes & professional practice will vary across

age spans

Measurements• Measurements of child outcomes that are research-based and age-appropriate

• School observations used to measure professional practice

Tiers

• Tiers that communicate to the public the quality of outcomes and practice at a school

• The highest tier will be reserved for schools that score highly on both, and the lowest tier for schools that score poorly on both

• Over time it will become possible to draw more meaningful distinctions in the middle tiers

Supports • Supports to schools designed to address issues identified by the observation of professional practice




II) What’s Happening Today in Education AccountabilityHaving broadly defined what birth-through-high-school education accountability and supports ought to look like, it is important to consider how existing accountability systems match up with that definition. Early learning and K–12 accountability are currently two entirely separate systems, both of which are evolving rapidly. While they are evolving in ways that bring them marginally closer to the model defined above, both still have a long way to go—and in most states, they are evolving separately. Indeed, in many states, child care and the accompanying QRIS are administered by a state agency other than the state education agency responsible for K–12 accountability.82 This only reinforces the divide between these two systems.

This section gives an overview of key themes in early learning and K–12 accountability. Because these systems are administered separately in most states, there may be few leaders in positions of authority who are intimately familiar with both. This overview is intended to help leaders and advocates understand similarities and differences between the two systems and identify areas in which each could evolve to better reflect the principles articulated in section I.

key structural differences for accountability between early learning and k–12

Early Learning K–12

Metrics

• Focused on inputs and structural

criteria

• Not directly tied to child outcomes

• Limited range of student outcomes

• Not directly tied to practices,

processes, or instruction

Measurements• Objective criteria

• Observation measures

• Assessments

• Graduation rates

Tiers • Point, block, or hybrid/combination• Reward, priority, and focus schools

• Letter grades

Supports

• Typically financial incentives based on child care reimbursement

• Professional development and training

• On-site technical assistance

• Improvement grants

• Focused on building capacity and

changing practices

• May be a governance change

• May be focused on a specific provider

(school) or on a more systemic level

(district)




A) Early Learning Accountability

In many ways it is misleading to describe “early learning accountability” as if it were a single enterprise. Historically, Head Start, state pre-k, and child care programs have all been subject to different accountability. Head Start has a thorough set of program standards, and programs are monitored to determine whether they are meeting those standards;83 the lowest-rated programs are now required to recompete for their grants.84 State pre-k programs generally have design requirements of their own that funded programs are required to meet,85 but states have not historically provided public rankings of pre-k program quality. Child care accountability varies substantially from state to state, but in recent years an increasing number of states have used QRIS to grade and rank programs, with incentives built in for program improvement.

Quality rating and improvement systems* are the early learning accountability methodology with a framework most like K–12’s, and not coincidentally are also the methodology growing rapidly across states and early learning programs.87 A significant driver of QRIS development in the last few years has been the federally-funded Race to the Top-Early Learning Challenge. The Challenge began in 2011 as a competition

* While the term “quality rating and improvement system” is used to describe an overall approach, most individual state systems are given specific brand names, such as Better Beginnings (Arizona), Paths to QUALITY (Indiana), Keystone STARS (Pennsylvania), and BrightStars (Rhode Island).

Note: The chart above outlines the general accountability oversight structures currently in place for K–12 and early learning. In early learning, individual providers often use multiple funding streams and are therefore held accountable to multiple agencies. Also in the chart above, a local “super grantee” refers to a grantee overseeing multiple Head Start delegate agencies that contract services. For example, in 2013, the Los Angeles County Office of Education contracted with 24 school districts and private nonprofit agencies operating more than 300 sites to provide early learning services including Head Start and Early Head Start.86 The dotted line indicates that in some state pre-k programs, accountability flows directly from the state to schools.

CURRENT ACCOUNTABILITY OVERSIGHT STRUCTURES FOR K-12 AND EARLY LEARNING

FEDERAL

K-12 Head Start Child Care State Pre-K

STATE

LOCAL

SCHOOL

Department of Education

SEA

Office of Head Start,Department of Healthand Human Services

LEA Local“super grantee”

Office of Child Care,Department of Healthand Human Services

Public SchoolPublic School orPrivate Provider

Public School orPrivate ProviderPrivate Provider

State Human Services,SEA, or Early Learning

Agency

SEA or StateHuman Services

Agency

District




for states to develop early learning systems, with QRIS a major focus of the grant.88 As defined in the Challenge, QRIS use the framework of accountability described above: It has metrics and measurements meant to sort providers into performance tiers, with the tiers having consequences (both positive and negative) for performance. The competition rewarded states that committed to expanding QRIS to preschool and Head Start.89 In total, 20 states have been awarded over $1 billion dollars to conduct their work.90

The changes to QRIS brought through the Early Learning Challenge have in some ways moved states closer to the right approach to accountability in early learning. But the metrics used in QRIS still have an insufficient focus on pedagogy, and the support infrastructure is still lacking. For early learning to reach its full potential to improve long-term outcomes, both of these issues must be addressed.

1) Early Learning Accountability: Current Performance Metrics

Current metrics used in quality rating and improvement systems generally focus on program quality components that are externally observable (which can include document reviews).94 In many cases, the lowest level of QRIS is tied to child care licensing requirements, which are typically focused on health and safety.95 Common criteria include staff qualifications and professional development, curriculum and learning activities, administration and business practices, family engagement, staff-child ratios and group size, the use of child assessments, and health and safety.96 “Teaching and learning” is often a component of the ratings—which typically include the quality of instruction, learning environment, and teacher-child interactions.97

Many of these metrics are rooted in child care, but QRIS are evolving to draw criteria from multiple programs (child care, Head Start, and preschool) to facilitate alignment and increase participation. Head

The Scope of QRIS Coverage

In those states that have QRIS (and not all do91) it is frequently a mark of some quality for schools to be included in QRIS even at the lowest tier, as some programs—even programs that receive state support—do not achieve that level of quality.92 Moreover, participation in QRIS is frequently voluntary.93 Because QRIS include programs under multiple funding streams (child care, Head Start, and state pre-k)—each of which may have different QRIS participation rates in a given state, to say nothing of across states—the scope of QRIS’ coverage is not at all comparable to the scope of accountability coverage in K–12.

Of course, some difference in the scope of coverage is to be expected given the different natures of the systems. Public K–12 systems are obligated to accept all children who want to attend them, and compulsory attendance laws require children to attend; in early learning, schools have no such obligation, and parents have no obligation to send their children to school. In addition, early learning takes place in a wide variety of settings, many of which draw on multiple funding streams (including parent payments).

States are currently working to expand the scope of their QRIS coverage even as they try to improve the structural elements of QRIS. This expansion work would likely need to continue if states moved from redesigning QRIS to creating a unified education-accountability system.




Start programs are able to participate in close to all statewide QRIS, and state-funded pre-kindergartens are eligible in approximately two-thirds of QRIS.98 In some states, pre-kindergarten programs are required to reach specific levels of quality within QRIS to receive public funding.99 States have also made special efforts to ensure that their QRIS reflect the developmental needs of children from birth through age three.100 States including infant and toddler quality indicators have integrated specific infant and toddler training and professional development, appropriate teaching and curricula for birth to age three, lower staff-child ratios and group size, increased family involvement and information sharing regarding child routines, program policies that promote continuity of care, and use of assessments targeted for infants and toddlers.101

Metrics measuring the qualifications of personnel have played an increasingly essential role in QRIS, but challenges remain to building a sustainable, high-quality early learning teaching force.102 The education level of the early learning teaching force is far below that of K–12 teachers: In K–12, essentially all teachers have at least a bachelor’s degree, while in early learning, such degrees are not required in Head Start, some state pre-k programs, and child care.103 Supports such as professional development, mentoring, and paid training may be more readily available in K–12, and many community-based early learning programs lack the infrastructure and adequate funding to offer benefits that enable teachers to effectively apply their knowledge and continue to develop their practice.104 Additionally, differences in compensation and perceptions of the role of teachers in K–12 compared to early learning have contributed to persistent issues facing the early learning workforce, such as high turnover rates and limited career opportunities in the field.105 Changes in state accountability systems to focus on professional practice could help to drive improvement in these areas.

While in some states the existing QRIS are relatively complex, there have been efforts to move toward simplifying the metrics to focus on a handful of simple but powerful standards.106 In addition, the Early Learning Challenge required states to conduct validations of QRIS to determine whether they had the intended impact.107 These forces could combine to make the metrics of QRIS evolve relatively rapidly over the next few years.

2) Early Learning Accountability: Current Performance MeasurementsOne key difference in performance measurements between early learning and K–12 is the performance of children on summative assessments.* There are no assessment-based output criteria suitable for use in the QRIS years.108 Accordingly, QRIS have included a more substantial focus on process.109

As states and localities have developed and adopted QRIS, these systems have focused on improvement around early learning inputs and child outcomes. The general underlying theory is that higher quality programs will contribute to improved child functioning. Rather than using one main measurement

* In this paper, “summative assessment” refers to evaluating student learning at the completion of an instructional unit by assessing learning against a specific standard or benchmark. Eberly Center for Teaching Excellence. (n.d.) “What is the Difference Between Formative and Summative Assessment?” Carnegie Mellon University. http://www.cmu.edu/teaching/assessment/basics/formative-summative.html.




to assess program performance, QRIS typically incorporate multicomponent assessments designed to measure process quality and make quality programs more transparent and better understood. These assessments include capturing the level of teacher-child interaction through the use of the CLASS observation tool110 and assessing quality of environment, materials and activities, and caregiver interaction using ERS.111 To support a process of continuous improvement, QRIS provide programs feedback and technical assistance based on assessment findings and offer a variety of incentives to make enhancements.112

3) Early Learning Accountability: Current Performance TiersQRIS typically use star rating levels (one star through five star), but they also use other terminology to describe their tiers.113 No state currently uses letter grades to describe its tiers.114 The goal of the rating system is to make it easy for families to understand the quality of programs to help guide their choices for enrolling their children.115

There are three primary methods for sorting programs into tiers: point systems, block systems, and hybrid systems.116 These methods operate as follows:

• Point systems: A program earns points within each category of standards, with the total point score determining the level of rating. For example, a program is awarded points in each quality criterion and must reach a minimum number of total points to achieve the level and move to the next—but can achieve that overall point total even if the program is substandard on some criteria. Additional points may be provided in some states if a provider attains outside accreditation.117

• Block systems: A program must meet all standards in each quality level to progress to the next level. For example, a program can move to level 3 once it achieves all of the requirements for levels 1 and 2.118 Many states require a program to meet licensing standards to be included in the system at level 1.

• Hybrid or combination systems: A program is rated using the block approach and point system. The initial levels are blocks, while the higher levels are achieved through a point system. For example, a program might have to meet all standards in level 2 to advance to level 3 but could advance to level 4 by acquiring enough points in some criteria without scoring highly in others.119

Different states use different systems, with the most common form of assigning ratings being the block system. According to the National Center on Child Care Quality Improvement and the QRIS National Learning Network, currently 18 states (Arkansas, Illinois, Indiana, Kentucky, Maine, Maryland, Massachusetts, Mississippi, Montana, New Hampshire, New Mexico, North Dakota, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, and Virginia) plus the District of Columbia use a block system, nine states (Arizona, Georgia, Idaho, Kansas, Michigan, New York, North Carolina, Texas, and Vermont) use a point system, and ten states (Colorado, Delaware, Iowa, Louisiana, Minnesota, Nevada, Ohio, Tennessee, Washington, and Wisconsin) operate a hybrid system.120




4) Early Learning Accountability: Current SupportsBecause QRIS is based on the child care system—in which providers are given variable reimbursements by the state—the primary method of support through QRIS is often higher reimbursement rates for programs in higher performance tiers, creating a defined spectrum of consequences for each level.121 States also provide participating programs with training and professional development generally aligned with QRIS requirements, in addition to on-site assistance and quality-improvement grants. The training opportunities made available cover several content areas, ranging from curriculum to language and literacy to environment assessments. The on-site technical assistance helps programs navigate the QRIS process and the implementation of improvement efforts. Meanwhile, improvement grants allow programs to purchase materials, resources, or services to help advance quality-building efforts.122

In QRIS, because many of the criteria are inputs or structural issues, the solution to identified deficiencies is frequently straightforward. For example, if a program’s staff is not adequately qualified, the solution to the problem is obvious: hire qualified staff. The same is true of many other common deficiencies, such as offering technical assistance and financial supports to help programs with limited capacity put quality-improvement plans into action.123 However, the fact that QRIS is able to identify remediable deficiencies does not mean it is able to solve them. In many instances, the differential reimbursement provided is not adequate for programs to improve quality.124 Moreover, in some instances, the relationship between these discrete improvements and better child outcomes is not clear.

QRIS supports also must be flexible because they reflect the fact that children enrolled in early learning are frequently outside a traditional public school setting. Accordingly, a significant focus of support efforts has been on continuous quality improvement processes.125

B) K–12 Accountability

For more than a decade, K–12 accountability has been shaped by the No Child Left Behind Act (NCLB), which reauthorized the Elementary and Secondary Education Act (ESEA)—and, more recently, by the US Department of Education’s waivers from that act. Under NCLB, the federal government provided a framework with performance metrics for schools, measurements, performance tiers, and supports.126 The waiver process initiated in 2011 by the Department of Education did not change that framework; instead, it allowed for changes in the metrics, measurements, tiers, and supports.127 As of this writing, 43 states, the District of Columbia, and Puerto Rico have received waivers; the Bureau of Indian Education and two states (Iowa and Wyoming) have requests under review; two states (North Dakota and Vermont) submitted requests that were later withdrawn; and three states (California, Montana, and Nebraska) had not applied.128

The department has announced plans to extend the waivers for two additional years.129 An extra one-year extension to implement the teacher-evaluation portion of the waivers was recently approved for Missouri and Nebraska,130 while the state of Washington’s request was not extended. Washington’s waiver flexibility will end with the 2013–2014 school year, making it the first state to have its waiver flexibility revoked.131




A primary goal of the ESEA waiver process was to allow states to redefine their accountability systems and move away from No Child Left Behind’s requirements of reaching 100% proficiency based on state assessments.132 The waiver process, administered by the Department of Education, defines a set of criteria that states must use for accountability systems in order to obtain a waiver.133 The criteria are based largely on accountability principles proposed by the Council of Chief State School Officers in its “Principles and Process for State Leadership on Next-Generation Accountability Systems.”134 These systems are aligned to college- and career-ready standards, focused on student outcomes (including performance, growth, and graduation), and continue NCLB’s requirement of disaggregating data based on student characteristics.135

1) K–12 Accountability: Current Performance MetricsAccountability under ESEA is based on student outcomes, primarily test results and graduation rates.136 Under NCLB—the first generation of federal accountability to focus on outcomes—the required metrics were proficiency in English-language arts and math, based on assessments measuring student achievement.137 States were required to achieve 100% proficiency by 2014 and set interim goals for proficiency for each year in between.138 States were allowed to create their own definitions of proficiency, and the definitions they came up with varied widely.139 Regardless of definition, however, as of 2014, no state had achieved 100% proficiency.140

The limitations of a proficiency-based system quickly became apparent. Because schools were focused on a single performance bar, studies showed that some schools had little incentive to focus resources on children who were already above that bar—or who were so far below it they had little hope of meeting it.141 Moreover, schools were not given credit for progress they made if a child did not achieve proficiency. For example, if a new 7th-grade student showed up at a school reading at a 2nd-grade level, and in one year the school raised his reading level to 6th-grade, that extraordinary accomplishment would not be recognized by NCLB; it instead required that the school be held accountable for failing to have him reading at a 7th-grade level.

The waivers allow for accountability to include a mixture of proficiency and growth, to give credit where schools helped students improve their performance even if that improvement still left them short of proficiency.142 Waiver states have added student growth (based on assessment results) into their accountability systems.143 Importantly, this change represents a step toward the kind of accountability called for in this paper; the addition of growth measures is meant to serve as a proxy for the quality of professional practices, on the theory that outstanding professional practice may lead to growth even when it does not lead to proficiency. While this is still an indirect method of measuring something that could and should be measured directly, it does suggest that the approach recommended here—balance professional-practice metrics with child-outcome metrics—is one that is already emerging in K–12 accountability.

Another important nascent change in the waivers is that they have allowed states to expand accountability to additional subjects beyond English-language arts and math.144 Some states have used




the opportunity to expand accountability systems to include social studies or science (or both).145 Still, a recent analysis showed that only 18 of the 42 waiver states have incorporated additional measures that go beyond traditional NCLB-era measures.146 Here too there is room for greater movement in this direction, but the waivers represent a first step away from NCLB’s limited focus on English-language arts and math.

Just as early learning metrics can differ depending on the age of the children served, so too does K–12 accountability use different metrics for different ages. Within K–12, the waiver process contemplates different methods of rating elementary schools and high schools, with graduation rate a key component of high school accountability.147 As with NCLB, graduation rates are an element of high school accountability, although the waiver process has reduced their significance.148

2) K–12 Accountability: Current Performance MeasurementsWhile the metrics in K–12 accountability may have changed, the measurements have not: standardized tests remain the dominant measurement in K–12 accountability. The waivers have allowed states to use standardized test scores from subjects other than English-language arts and math, but the ESEA waivers do not, by and large, represent a major advance in the use of accountability measures other than test scores.149 States have gone beyond raw proficiency scores to develop multi-indicator indices that provide a more well-rounded picture of school performance,150 but those indices are still based on standardized test scores.

The standardized tests at the heart of K–12 accountability are rapidly changing. The widespread adoption of Common Core standards has pushed states to update and revise the standards-based accountability assessments they have in place. The Department of Education has supported two consortia, known as the Partnership for Assessment of Readiness for Colleges and Careers (PARCC) and SMARTER Balanced,151 to develop new Common Core-aligned assessments; other test vendors have also worked on developing new Common Core-aligned accountability assessments.152 In many states, these new assessments are scheduled to roll out over the course of the next few years.153 Some states have decided to take a gradual approach to full implementation of these assessments, while others have chosen to withdraw from PARCC and SMARTER Balanced and will use other assessments.154

As noted above, at the high school level, graduation rates are also an important part of accountability. The process of calculating graduation rates has been standardized across states.155

3) K–12 Accountability: Current Performance TiersWhile federal and state accountability measures apply to and have consequences for school districts and schools, tiering efforts have focused primarily on schools. Waiver accountability requires states to designate “priority schools” and “focus schools” whose performances need improving, as well as high-performing “reward schools.”




• Priority schools: Priority schools are among the lowest performing in the state. The total number of priority schools must be at least 5% of all Title I schools statewide.156 (The Title I program provides eligible schools serving a high at-risk population with financial assistance to meet the needs of their students.157) A priority school is defined as one of the following:

• A school among the bottom 5% of Title I schools in the state based on school-wide academic achievement scores158

• A Title I or Title I-eligible high school with a graduation rate below 60% consistently over a period of years

• A Tier I or Tier II school currently implementing a School Improvement Grant (SIG) intervention model. Tier I and Tier II SIG schools are considered by the state to be persistently low achieving.159

• Focus schools: Focus schools are Title I schools with significant achievement gaps. Focus schools must include at least 10% of Title I schools statewide. A focus school is defined as one of the following:

• A school with the largest achievement gaps or difference in graduation rates between the highest- and lowest-performing subgroup of students

• A school with subgroups of students with low academic performance or poor graduation rates160

• Reward schools: Reward schools represent Title I schools with a track record of top performance or high progress. A reward school is defined as one of the following:

• A school that has been identified as having the highest absolute performance for all students and also subgroups on statewide assessments, and at the high school level is among Title I schools with exemplary graduation rates

• A school that is among the 10% of Title I schools showing the most progress in improving academic achievement gaps among all students and subgroups and also demonstrated increased high school graduation rates161

Waivers have significantly changed the metrics used for tiering under state accountability systems, moving the focus from absolute performance (whether or not a school met a state-defined standard) to relative performance.162 As noted above, states have also used metrics other than test scores and graduation rates to identify priority and focus schools, although test scores continue to be the primary metric.163

At the state level, an increasing number of tiering systems are using letter grades.164 With its A+ Plan for Education, Florida became the first state to institute an A–F grading accountability system under Governor Jeb Bush as part of the push for standards-based reform. The grading of schools on an A–F scale based on




student achievement sought to spur school improvement in a manner that rewarded academic success for schools as well as put into place sanctions and forms of assistance for low-performing “D” and “F” schools.165 Over the years, the letter-grading scale has been used as a model for 15 other state initiatives, despite mixed reviews.166 It is worth noting that even the states using letter grades have different criteria for sorting schools into those tiers, including utilizing measures on student learning growth, high school graduation rates, college and career readiness, and dual enrollment;167 thus, the use of letter grades creates a common tiering and labeling approach but not a common set of metrics.

4) K–12 Accountability: Current SupportsBecause K–12 accountability is based on outcomes, the supports necessary to achieve those outcomes are often difficult to pinpoint. In general accountability metrics are used to make an initial categorization of schools, but the actual nature of support to the school must be based on a more individualized and targeted diagnostic review.168 The ESEA waiver process represents another stage in the evolution of thinking about school improvement, anticipating that states will identify priority and focus schools and design interventions that meet the needs of schools in each of those categories.169 Both prior to and after the implementation of waivers, state lists of the lowest-performing schools subject to supports and interventions include a disproportionate number of schools with significant percentages of students in poverty.170

A critical aspect of K–12 supports is that they do not generally flow directly to schools—they flow to school districts. Generally, efforts by school districts to effectively implement interventions and distribute resources to low-performing schools are limited by lack of capacity, integration and alignment, and buy-in to produce school success.171 Additionally, district-implemented improvement strategies can be disconnected from the daily activities of schools; without adequate capacity, initiatives to aid schools often fall short of making significant gains and meeting the needs of schools and their students.172

It is also important that much of the conversation about the consequences of K–12 accountability has focused less on support than on negative consequences for schools that fail to perform. The last few years have seen a sea change in how districts and schools approach school turnaround,173 driven in large part by federal School Improvement Grants that force schools to use one of four turnaround models.174 These consequences are in many instances interventions aimed at changing the oversight of the school—or shutting it down.175 Consistent with this approach, ESEA waiver accountability requires significant consequences for schools that are among the lowest performing, including closure or major governance changes. These approaches are new enough that no definitive long-term study of their impacts has been possible, although states have struggled with their implementation.176

At the K–12 level, accountability systems have historically offered few positive incentives for higher-level achievement. The ESEA waiver process attempts to address this issue by requiring states to identify reward schools that will receive benefits for their high performance or high progress.177 While limited analysis exists on the effectiveness of these rewards, the offerings given to schools range from

monetary rewards to recognition certificates, and they often vary by state.178




III) Creating the Next Generation of Education AccountabilityTo move from the existing accountability systems described in section II to the proposed accountability system described in section I will require substantive changes to existing accountability systems and likely process changes at the state level. This section summarizes what those substantive changes should look like, with some broad recommendations for processes that states can use to achieve change.

A) The Changes That Need to Be Made

In each key element of the accountability system described in section I—metrics, measurements, tiers, and supports—changes need to be made at both the birth-to-five and K–12 levels. In all areas, our view is that the needed changes will not only improve performance compared to existing accountability systems in each sector but will help create a more seamless continuum of accountability. This subsection summarizes our proposed changes for each of those elements.

1) The Changes That Need to Be Made: Performance MetricsMovement is already under way toward accountability systems that measure both professional practice and child outcomes.

• Early learning accountability is increasing its focus on metrics that measure professional practice, and K–12 accountability is adding growth measures—an indirect measure of professional practice.

• Early learning accountability is more explicitly focusing on metrics that are linked directly to child outcomes, although for the most part it still does not measure child outcomes directly. K–12 accountability has focused for many years on child outcomes.

This shows that the approach advocated here is one that is already gaining traction.179 There are several challenges, of course.

First, the metrics used for professional practice are either insufficiently precise and focused (in early learning) or indirect (in K–12). In early learning, many states are already in the process of pursuing more precise and focused metrics, and the work under way is likely to yield useful lessons for next-generation systems. In K–12, the push to use growth measures represents a step in the right direction, but a fundamental change is still needed to use direct measures of professional practice. The 5E are a research-based resource that can be used to develop these metrics, but more work is clearly needed to develop professional-practice metrics that could be integrated directly into state accountability systems—and that recognize differences in practice across age spans.

Second, the metrics for child outcomes need substantial work. In early learning, the child metrics that are appropriate for use in school accountability tend not to focus directly on standards-based learning—for example, health outcomes. This challenge is discussed further under “Performance Measurements” on




page 26. In K–12, an emerging trend is to broaden the metrics used to go beyond child outcomes on English-language arts and math tests, but even here the trend is primarily limited to standardized assessments in other academic subjects. Broader thinking may be needed to identify child outcomes that more accurately capture what we want from our education system.180 Moreover, the current trend toward measuring growth should continue, as child growth is likely to be a better measure of school performance than proficiency.181

A potential strategy in this area, particularly at higher grade levels, is to leverage some of the rubrics states have developed for planning school improvement and turnaround efforts. In some instances, states have used test scores to identify which schools will receive intensive supports but then developed diagnostic tools to determine what intensive supports or interventions are needed.182 Some of the elements articulated in those diagnostic tools may speak to one of the 5E, or represent a key value of school improvement of the state that would be worth elevating to an accountability metric.

One other legal requirement that is important to the use of child-outcome metrics is that NCLB accountability required the reporting of child outcome by subgroup (race, income, special education status, and limited English language proficiency), which has helped identify racial achievement gaps in many communities.183 The ESEA waiver process has changed the approach to subgroups, with states combining racial subgroups together for reporting purposes.184 The impact of these changes is unclear. While this issue is not explored in depth here, it is important to note that even in accountability systems remodeled in the way proposed here, child-outcome metrics could be reported by subgroup, and differences in subgroup scores could be used for school accountability purposes.

To date, federal law has discouraged states from developing a thoughtful mix of child outcome and professional practice metrics. Accountability systems should include child-outcome measures to ensure a continued focus on achieving results but should do so in a manner that is broadly inclusive of the many

performance metrics

Proposed Early Learning and K-12A mix of metrics balancing child outcomes withprofessional practice.

Child outcomes will vary across age spans. Balancebetween child outcomes & professional practice willvary across age spans.

Requires rethinking definitions of child outcomesand professional practices, and making dramaticshifts in metrics for all age spans.

Current Early LearningFocused on inputs andstructural criteria. Not directlytied to child outcomes.

Current K-12Student outcomes.Not directly tied to practices,processes, or instruction.




outcomes we collectively want for children.185 Because of the importance of age-appropriate metrics, the mix of metrics should change for different age groups, with some metrics cutting across ages and others (like high school graduation rates) used where appropriate.186 The balance between professional-practice metrics and child-outcome metrics may also vary across age groups. While it is possible to identify principles that should inform state action in this area, it must be acknowledged that a great deal of work would be needed at the state level to give these principles real-world effect.

2) The Changes That Need to Be Made: Performance Measurements

In both early learning and K–12, there is a history of using accountability measurements that are relatively cheap at scale. To some degree, the impacts that accountability systems have had so far reflect the fact that state systems have gotten what they paid for. If the theory of action is that improved professional practice will lead to improved child outcomes—and the research is quite clear that it does—then changing adult practice may require us to measure that practice directly rather than indirectly. But we must continue to measure child outcomes and must acknowledge that by measuring a limited set of child outcomes we may influence adult behaviors in ways that are detrimental to achieving a broader set of child outcomes that we actually care about.

Accountability for K–2

Currently there is no real accountability for the early elementary years (K–2); these grades are generally included in elementary schools that are rated based on their performance on tests beginning in 3rd grade. State accountability systems do not generally measure anything relating directly to K–2 performance. A birth-through-high-school approach with a continuum of metrics—including metrics based on professional practice—could create real and unprecedented accountability for K–2. Indeed, states may want to consider giving K–2 its own set of metrics, distinct from what came before and what comes afterward. Improved metrics in K–2 might help drive better supports in those years and in the ones that precede them.

performance measurements

Proposed Early Learning and K-12Measurements of child outcomes that are research-based and age-appropriate.

School observations used to measure professionalpractice.

Moves the focus away from high-stakesassessment to other measurable child outcomes,and requires substantially ramping up theuse of observational assessments ofprofessional practice.

Current Early LearningObjective criteria.Observation measures.

Current K-12Assessments.Graduation rates.




Early learning has recognized the importance of measuring professional practice and begun to integrate it more aggressively into accountability systems.187 The same is not true of K–12 at scale, although there are elements of this approach in use—specifically, the review teams that some states use to assess schools for turnaround after those schools have been identified through low test scores.188 The approach is certainly not novel in other countries; as discussed in section I, England and the Netherlands use an inspectorate to rate professional practice in schools.189 This form of measurement may have greater up-front costs than what states currently spend on accountability, but those costs need not be astronomical—and because the close examination of professional practice is likely to yield more immediately actionable information than test scores, the investment may prove to be worth it.

Significant work is also needed in measuring child outcomes. This is certainly true at the early learning level because the research on developmentally appropriate assessment has shown that high-stakes assessment is not appropriate for children prior to 3rd grade.190 Some existing measurements that can be used may include those focused on health outcomes, but that may feel too attenuated for states in their education-accountability systems. But at all ages, it is important that high-stakes assessments not be the only child-outcome measures. Some child-outcome measurements beyond high-stakes assessment are already in use, like graduation rates; thoughtful states may identify more that are appropriate for inclusion in accountability systems.

3) The Changes That Need to Be Made: TiersWhile different metrics should be used for different ages (as they are today), a set of common performance tiers across age spans could improve the quality of public reporting and help reinforce connectivity between early learning and K–12. While not all states use QRIS as an early learning accountability system, the majority do, and these quality-rating systems generally use a tiering approach. At the K–12 level, ESEA waivers demand tiering to a degree by requiring states to designate reward, focus, and priority schools;191 states generally go beyond those minimal requirements with additional tiers and increasingly are using A–F grading systems.192 These approaches can be consolidated into a common tiering framework, which can help give more consistent information to the public across age spans.

One major current purpose of tiering is to drive consequences by setting incremental targets for improvement tied to increased flexibility and support (or at least the avoidance of negative consequences). In QRIS, this frequently occurs through increased payments to highly-rated programs.193 The ESEA waiver requirements are another example of this, as the waivers define a discrete universe of actions states must take to address performance in focus and priority schools.194 If states incorporate metrics and measures that actually diagnose school needs, then the need for tiering to define supports is reduced dramatically; the supports should be based on the diagnosis. States may still want to tie positive consequences to achieving higher tiers, but these could be more generalized rewards, as these higher-performing schools are likely to be able to take advantage of additional resources or flexibility.




In a new model, performance tiering would primarily serve a public information purpose: It would be a way for schools to set a clear path to improvement and expectations for performance, and for parents and the public to understand the quality of a school, which can help inform a community’s approach to a school and a parent’s decision on whether to enroll a child there. Indeed, one of the main purposes for the development of QRIS has been to build parent awareness around understanding and selecting quality early learning programs. Some states post a summary of program ratings in an effort to help parents make more-informed decisions and to increase the demand for higher quality early learning programs.195 Similarly, in K–12, accountability ratings are meant to inform parents of the quality of their school, and the move to A–F accountability has been fueled in part by a desire for school ratings that are easy for parents to understand.196 Based on existing infrastructure, this public information function will likely play out very differently between early learning and K–12; early learning is a market-based system in which many providers have never had attendance boundaries, whereas in K–12, the nature of school choice can vary dramatically from jurisdiction to jurisdiction.197 Parents weigh a variety of factors, including quality, in choosing early learning programs,198 but some recent studies have indicated that key factors related to early learning’s market-based structure—such as affordability, location, and availability of programs—are also closely tied to parents’ decision-making.199 Regardless of how many viable choices a parent has, transparent information about the quality of the school can help improve the exercise of that choice and inform efforts by parents and the community to improve the quality of their schools.

Schools as the Unit of Accountability

Federal and state systems treat individual schools as an important unit of accountability. While there is a logic to that, it can create disconnects as students progress and interventions at one level bear no relationship to the academic program at other levels. For this reason, some commentators have advocated creating accountability and implementing supports in “clusters” that are vertically aligned across schools and grade levels.202 K–12 does have district-level accountability, although that does not always force connectivity across grade levels.

School-level accountability raises complex questions of how responsible schools should be for activities that go on elsewhere. For example, should preschools be responsible for the performance of their students in elementary school? In the case of early learning, supporters and critics of increasing funding for early learning use long-term results to justify their positions, but that has not led to the inclusion of long-term metrics in early learning accountability. That is partly because of the intractable problem that by the time those long-term results are available, it is far too late to provide useful information to parents or meaningful supports or consequences for the preschools. But the issue of accountability for later performance can also be applied to elementary schools, middle schools, and high schools; indeed, one potential metric of high school accountability is college remediation rate.203

The issue of accountability for later performance can be flipped to raise the question of whether high schools should be given responsibility for the performance of feeder schools, to encourage working with them to improve performance—a principle that could in theory also be applied to middle schools, elementary schools, and even preschools. This approach would raise enormous challenges of governance and authority. To date, systems have generally shied away from attempting to create accountability across administrative units, but these approaches may begin to emerge in the coming years.204




There is no accepted state-of-the-art formula for setting performance tiers, and states will need to experiment in this area.200 As noted in section I, one thing that should be consistent across all age spans is that the highest and lowest rating tiers should be reserved for those schools whose indicators are all telling a consistent story. For example, in K–12, some schools have been given low ratings even though their students have shown impressive growth, which means the school’s letter grade may give a misleading impression of the professional practice in the school. An increased focus on the quality of practice and a broader set of metrics should make it easier to reserve the lowest ratings for those schools where student performance is weak and adult professional practice is unlikely to help students improve.201 Many states that have used voluntary QRIS will need to reconsider the relationship between the lowest tier of the accountability system and the providers not currently in the accountability system. Beyond the lowest tiers, it will likely take years for states to sort out how they want to differentiate among schools in the middle tiers and how those tiers can be used to convey credible information about performance to the general public.

4) The Changes That Need to Be Made: Supports and ConsequencesExisting support systems have generally grouped schools into performance bands (e.g., the lowest-performing 5% of schools) and prescribed a set of supports and consequences for schools in that band. A move to accountability based on diagnosis of professional practice replaces this approach with one that is more surgical—an approach in which the supports are specifically based on the school’s identified needs. This approach is impossible in an accountability system based solely on student test scores, which do not provide that kind of diagnostic information.

In early learning, states have already started integrating holistic measures of professional performance into QRIS and are attempting to design supports based on those, particularly in the Early Learning Challenge states.205 Those states are required to perform validations of their QRIS over the course

tiers

Proposed Early Learning and K-12Tiers that communicate to the public the quality ofoutcomes and practice at a school.

The highest tier will be reserved for schools thatscore highly on both, and the lowest tier for schoolsthat score poorly on both.

Over time it will become possilbe to draw more meaningful distinctions in the middle tiers.

Requires alignment of definitions of qualityacross age spans, and the development of newmethodologies to assign schools into tiers.

Current Early LearningPoint, block orhybrid/combination

Current K-12Reward, priority, and focusschoolsLetter grades




of the grant, which could yield valuable information about whether the new supports actually drive improvement.206 In K–12, there are also elements of this approach in states that have developed diagnostic tools for school performance.207 In both early learning and K–12, however, these deeper diagnostics are generally used only for a subset of schools.208

In some instances, the lowest-performing schools may show they are unable to improve their performance even with the targeted supports. Existing accountability systems in K–12 have included provisions that when schools do not improve, they undergo governance changes or have their funding pulled.209 The same is true in early learning, where some states are now restricting government funding to programs that achieve a minimum threshold of quality.210 This kind of consequence seems appropriate only if the school has been given genuine support to address any identified deficiencies but has still shown itself unable to make changes in a reasonable period of time.211 While shutting off government funding can be disruptive to families, if a school cannot perform even after receiving assistance then it is appropriate to insist on major changes in school management for the school to continue receiving government funding.

State resources for school improvement are limited, and states’ track record of success is also limited. A benefit of better system-wide diagnostics may be that states can focus their efforts on those schools that need the most help with professional practice and design their supports more effectively as well.212 This is likely to make state resource use more effective and efficient, a must given how little capacity states have had in this area. States must also be thoughtful about how to leverage local resources (particularly school district resources), which may be a more promising role for states than as large-scale direct service providers.

At the state level, in both early learning and K–12, there has been concern that accountability systems are labeling low performers without leading to any improvement in performance.213 While targeted supports for low performers may always be needed, the bigger issue is whether states are systemically and sufficiently supporting their educational system to produce the results they want. This has obvious implications in the early learning context, where historically states have funded programs that are not designed to provide standards-based education and that frequently reach only a small percentage of the population. But it also has implications in K–12, where states sometimes make policy decisions that are in significant tension with the goals of their accountability system.

Moreover, it is essential that school improvement not be thought of simply as a school-system responsibility. All schools are in communities, and while state supports may be valuable they are no replacement for community engagement in schools. The role of school districts in this work will vary; districts have different levels of capacity to provide support, and in early learning, many schools are not directly supported by their local district—although ideally they will have productive working relationships with it (or with multiple districts, depending on location and attendance patterns). But this work should go beyond school districts and parents to engage additional community leaders and partners. States should attempt to design support systems that reflect the value of community engagement in school improvement and acknowledge the diverse needs of different schools and communities.214




Many of the elements of successful schools are elements that state policy and supports can facilitate, but none of the elements of successful schools can be mandated or imposed by states. For state support systems to be successful, they must be premised on the idea that states can use data to inspire action for improvement, but that in many instances the action for improvement will be taken by actors who are not state employees (or even school employees). But states do have a critical role in defining the policies that lead to strong schools and instructional systems, and in many cases can provide valuable support capacity that allows local leaders to act more effectively. Better metrics, measurements, and performance tiering should help state and local leaders be more thoughtful about their improvement efforts and lead to more efficient resource use on behalf of improving student outcomes.

B) How the Changes Will Happen

In most states there is no history of dramatic improvement driven by an accountability system at either the early learning or K–12 level. QRIS and ESEA waivers are both new enough that it is far too early to declare success or failure regarding their implementations. Both appear to be based on a sound theory of action grounded in identifying needs and targeting resources to address those needs. But there is no doubt that if they experience widespread success in multiple states it will be a development without precedent in the history of education policy in this country.

supports

Proposed Early Learning and K-12Supports to schools designed to addressissues identified by the observation ofprofessional practice.

Will require a shift from relying onbroad categories of activity to moretargeted and customized supports.

Current Early LearningTypically financial incentivesbased on child carereimbursementProfessional developmentand trainingOn-site technical assistanceImprovement grants

Current K-12Focused on building capacityand changing practicesMay be a governance changeMay be focused on a specificprovider (school) or on a moresystemic level (district)




While states may have a long way to go to develop successful birth-through-high-school accountability, there are some important steps they can take to get there:

• Identify key entities and stakeholders. First, states must decide what entities are necessary to develop a comprehensive birth-through-high-school accountability system. As noted, existing QRIS and ESEA waiver systems are likely in separate agencies.215 This means that governors and legislative leaders will likely be required to develop a combined system, whether through a state P-20 Council216 or legislative study group. Different states have different infrastructure in place for addressing issues that cut across early learning and K–12; some states may have no effective entity set up to discuss the creation of a system like this. If there is no existing forum, then a governor or legislature would need to set up a dedicated study committee.

• While the K–12 accountability system will be housed at the state education agency, governance structures in early learning vary substantially, which means that in most states, two or more agencies will need to be actively involved in developing a more seamless accountability continuum. And regardless of how a state’s executive branch is structured, the dynamic between the executive and legislative branch will also be important to consider in developing unified accountability.

• Identify technical experts. While high-level political leadership and stakeholder support would be needed to advance the process of developing a comprehensive accountability system, it is important that in most states this exercise will involve revamping something that already exists, not building something new. That can cut both ways politically—sometimes it is easier to create new systems than modify old ones—but at least states will already have some amount of technical expertise in all the elements of their systems (metrics, measurements, tiering, and supports) because they have been responsible for operating systems. As with the development of the existing systems, the process for revamping accountability will require high-level buy-in and technical expertise to develop new metrics, measurements, tiers, and supports. States may wish to draw on national technical experts (as many have done in the creation of QRIS and waiver systems), or they can start with the confidence that there are already people in their community who know how to make these systems work and can modify them based on an assignment given by the governor and/or legislature. States will also need to examine their data systems to ensure that those systems can support a redesigned accountability system.

• Commit to investing in system improvement with an understanding that savings are garnered in the long run. There is no question that the proposed use of deeper diagnostic measures in accountability will increase the expense of administering an accountability system and demand a kind of oversight capacity states may not be used to providing.217 It will also increase substantially the amount of actionable information states have about school improvement, which could easily lead to pressure for greater spending on school supports. The tradeoff if a change like this is executed should




be that (a) accountability will be a seamless progression rather than disconnected criteria across age spans, and (b) accountability systems will be much more effective at driving improvement, so that professional-practice and child outcomes will improve consistently. In the long run, accountability systems should lower overall costs by improving performance across the board—especially in earlier years—in ways that reduce long-term remedial expenses.218 Of course, the benefits of increasing high school graduation rates and long-term performance are well known.219 States should only try to design a next-generation accountability system if they are prepared to make the investments necessary to make it work, with the expectation that a well-designed investment will lead to long-term payoffs.

• Be aware of and primed for changes in standards and practice. The Early Learning Challenge and the ESEA waiver process have moved state accountability conversations in positive directions. But in all states, as noted, the adoption of a common framework will require real changes in accountability at all levels of the age spectrum. How great the gap is between the existing system and a model system will likely vary from state to state, and within states from age span to age span and grade range to grade range (birth to 3 years, 3 to 5 years, grades K–2 and 3–8, and high school). States must be prepared for the idea that developing a new accountability system may shift how resources are allocated among those groups, and they would need to prepare transition plans to ensure that strong educational practice can continue at all age and grade levels.

These recommendations are all for state leaders, but the federal government also has an essential role to play here. For states to succeed with new accountability systems, it is important that these processes be state-driven and not federally mandated. While the Early Learning Challenge and ESEA waiver processes have helped spark movement toward a better system, neither one really contemplates a complete birth-through-high-school overhaul, and the development of a true next-generation accountability system might bump up against limitations within the challenge and ESEA. There are some ways the federal government could give states the leeway and support they need to update their accountability systems.

• For better or worse, developing next-generation systems will be a multiyear process; by the time a new system is designed and adopted, the first wave of Early Learning Challenge grants will likely be over.220 By then Congress may have reauthorized ESEA, but if not, then the emergence of comprehensive plans for birth-to-high-school accountability in states might influence Congress to reauthorize ESEA in a manner that could account for next-generation accountability systems.

• In the short term, if states are interested in developing systems of the kind proposed here, the ESEA waivers, administered by the Department of Education, would likely have to be modified to accommodate them. If the department is interested in allowing states to attempt accountability systems of this kind—or at least willing to indulge the possibility—it could review and update its waiver requirements to ensure it is not preventing states from developing such systems.




• In addition to their potential use in ESEA, the accountability principles articulated here could potentially be useful in the reauthorization of important legislation that funds early learning programs, including Head Start and child care. The US Department of Health and Human Services could also integrate some of these principles into its administrative oversight of those programs.




ConclusionAccountability systems should be based on what we want the accountable parties to be providing. In education, the accountable parties are schools, and they should be providing high-quality professional practice that leads to better child outcomes. The metrics we use to measure progress toward these goals are imperfect, and the supports we provide to meet these goals have been inadequate. There is widespread recognition that we can and must do better, and that just leaving schools to their own devices without additional support is unlikely to substantially improve student outcomes.

Accountability must continue to evolve. The early learning field gained lessons in the administration of early learning programs that led to the development of QRIS, yet research has indicated that the approach currently being used in QRIS may not be measuring the practices that lead to improved child outcomes.221 The K–12 field learned lessons in NCLB that are now being applied in ESEA waivers accountability. While many strong ideas are embedded in existing state QRIS and ESEA waiver accountability systems, one major goal in the next decade should be to learn lessons from their implementation that help them evolve into even better systems. Some of these lessons can be learned even if the systems are kept separate, but creating a single system could create greater opportunity to learn across age spans. The building blocks of unified accountability are already being put into place, and states can combine them to create a more seamless approach to improving student outcomes.

If states are committed to having accountability systems—and in the short term they clearly are—there is much they can do to make them better. They can set metrics and use measurements that evaluate both professional practice and a broader range of child outcomes, giving schools useful information for improvement and helping adults focus on the full spectrum of outcomes public education is supposed to lead to. They can give parents better information about school performance while targeting resources more effectively to improve low-performing schools. They can gain new information that will allow them to make better system-wide decisions about resource use and implementation. And they can bring together systems that are currently separate, providing a strong reminder that it is the same children who are being served from birth through high school graduation. These unified accountability systems are not guaranteed to produce better professional practices or improved child outcomes, but they would offer a better framework for doing so than the systems in use today.

Join the Conversation at theOunce.org/policyconversationsOunce Policy Conversations are published by the Ounce of Prevention Fund to stimulate and facilitate discussion on topics of interest in early learning policy. We welcome feedback on this paper, and may address that feedback in subsequent versions of this paper or in other papers. If you have comments or questions, please visit our webpage at theOunce.org/policyconversations or contact Elliot Regenstein at [email protected].

ounceofprevention.org/policyconversations

www.ounceofprevention.org/policyconversations

mailto:[email protected]




Endnotes1. The authors are grateful to their many colleagues at the Ounce of Prevention Fund who reviewed and

commented on drafts of this paper. A special thanks to Ounce colleagues Ann Hanson and Brooke Fisher for their contribution to the sidebar on best practice pedagogy in early learning. The authors are also thankful for input from their external reviewers: Jon-Paul Bianchi, Laura Bornfreund, Justin Cohen, Libby Doggett, Barbara Gebhard, Stacie Goffin, Joan Lombardi, Kelly Maxwell, Laura McSorley, Anne Mitchell, Patrick Murphy, Michael Petrilli, Tom Schultz, Alison Segal, Nancy Shier, Helene Stebbins, Louise Stoney, Albert Wat, Sarah Weber, Marcy Whitebook, Conor Williams, Karen Yarbrough, and Margie Yeager. The views expressed in this paper are those of the authors and do not necessarily represent those of the reviewers.

2. Anh, T., and Vigdor, J. (2013). “Were All Those Standardized Tests for Nothing? The Lessons of No Child Left Behind,” 12. American Enterprise Institute. http://www.aei.org/files/2013/05/17/-vigdor-and-ahn-nclb-sanctions-paper_15005080098.pdf.

3. Cawelti, G. (2006). “The Side Effects of NCLB.” Educational Leadership, Vol. 64, No. 3. http://www.csun.edu/~krowlands/Content/SED610/NCLB/Side%20effects%20of%20nclb.pdf; Dillon, S. (March 26, 2006). “Schools Cut Back Subjects to Push Reading and Math.” New York Times. http://www.nytimes.com/2006/03/26/education/26child.html?pagewanted=all&_r=0; Hayhurt, E. (2013) “Have Federal Sanctions Helped Failing Schools? The Impact of No Child Left Behind in Texas.” CMC Senior Theses, Paper 564. http://scholarship.claremont.edu/cgi/viewcontent.cgi?article=1706&context=cmc_theses; Schneider, M. (2011). “The Accountability Plateau.” Thomas B. Fordham Institute. http://www.edexcellence.net/sites/default/files/publication/pdfs/20111215-The-Accountability-Plateau_7.pdf.

4. For a helpful overview of the theory of action behind accountability, see Williams, C. (August 5, 2013). “The Way We Talk: Accountability.” New America Foundation, Early Ed Watch blog. http://earlyed.newamerica.net/blogposts/2013/the_way_we_talk_accountability-89267.

5. Large Countywide and Suburban District Consortium. (2014). “21st Century Education Accountability: Recommendations for a New Federal Framework.” http://blogs.edweek.org/edweek/District_Dossier/Consortium%20-%20%20Recommendations%20for%20a%20New%20Federal%20Accountability%20Framework%20February%202014.pdf; Wisconsin Policy Research Institute. (2013). “Pathways to Success for Milwaukee Schools,” 30–31. http://www.aei.org/files/2013/05/29/-pathway-to-success_170815996739.pdf.

6. Tucker, M. “The Failure of Test-Based Accountability.” (February 27, 2014). Education Week, Top Performers blog. http://blogs.edweek.org/edweek/top_performers/2014/02/the_failure_of_test-based_accountability.html?cmp=ENL-EU-NEWS2.

7. Cody, A. (August 9, 2013). “From School Grades to Common Core: Debunking the Accountability Scam.” Education Week, Living in Dialogue blog. http://blogs.edweek.org/teachers/living-in-dialogue/2013/08/from_school_grades_to_common_c.html; Wiggins, G. (November 14, 2013). “Genuine vs. Sham Accountability.” Granted, and …Thoughts on Education blog. http://grantwiggins.wordpress.com/2013/11/14/genuine-vs-sham-accountability/. See also Huffington Post. (November 19, 2013). “How Poverty Impacts Students’ Test Scores, In 4 Graphs.” http://www.huffingtonpost.com/2013/11/19/poverty-test-scores_n_4298345.html?ir=Education&ref=topbar; Ladd, H. (2012). “Education and Poverty: Confronting the Evidence.” Journal of Policy Analysis and Management, Vol. 31, No. 2, 203–227. http://dukespace.lib.duke.edu/dspace/bitstream/handle/10161/6650/Ladd%20-%20Education%20and%20Poverty.pdf?sequence=1; Weiss, E. (October 14, 2011). “Test Score Gaps Refuse to Budge, Plead Poverty.” Economic Policy Institute, Working Economics blog. http://www.epi.org/blog/test-score-gaps-refuse-budge-claim-poverty/.

8. Hall, D. (2013). “A Step Forward or a Step Back? State Accountability in the Waiver Era,” 5. Education Trust. http://www.edtrust.org/sites/edtrust.org/files/A_Step_Forward_Or_A_Step_Back.pdf; Tucker, M., Darling-Hammond, L., and Jackson, J. (October 9, 2013). “Fewer, Better Tests Can Boost Student Achievement.” Education Week. http://www.edweek.org/ew/articles/2013/10/09/07tucker.h33.html.

http://www.aei.org/files/2013/05/17/-vigdor-and-ahn-nclb-sanctions-paper_15005080098.pdf

http://www.csun.edu/~krowlands/Content/SED610/NCLB/Side%20effects%20of%20nclb.pdf

http://www.csun.edu/~krowlands/Content/SED610/NCLB/Side%20effects%20of%20nclb.pdf

http://www.nytimes.com/2006/03/26/education/26child.html?pagewanted=all&_r=0

http://www.nytimes.com/2006/03/26/education/26child.html?pagewanted=all&_r=0

http://scholarship.claremont.edu/cgi/viewcontent.cgi?article=1706&context=cmc_theses

http://scholarship.claremont.edu/cgi/viewcontent.cgi?article=1706&context=cmc_theses

http://www.edexcellence.net/sites/default/files/publication/pdfs/20111215-The-Accountability-Plateau_7.pdf

http://www.edexcellence.net/sites/default/files/publication/pdfs/20111215-The-Accountability-Plateau_7.pdf

http://earlyed.newamerica.net/blogposts/2013/the_way_we_talk_accountability-89267

http://blogs.edweek.org/edweek/District_Dossier/Consortium%20-%20%20Recommendations%20for%20a%20New%20Federal%20Accountability%20Framework%20February%202014.pdf



http://www.aei.org/files/2013/05/29/-pathway-to-success_170815996739.pdf

http://blogs.edweek.org/edweek/top_performers/2014/02/the_failure_of_test-based_accountability.html?cmp=ENL-EU-NEWS2

http://blogs.edweek.org/edweek/top_performers/2014/02/the_failure_of_test-based_accountability.html?cmp=ENL-EU-NEWS2

http://blogs.edweek.org/teachers/living-in-dialogue/2013/08/from_school_grades_to_common_c.html

http://blogs.edweek.org/teachers/living-in-dialogue/2013/08/from_school_grades_to_common_c.html

http://grantwiggins.wordpress.com/2013/11/14/genuine-vs-sham-accountability/

http://www.huffingtonpost.com/2013/11/19/poverty-test-scores_n_4298345.html?ir=Education&ref=topbar

http://www.huffingtonpost.com/2013/11/19/poverty-test-scores_n_4298345.html?ir=Education&ref=topbar

http://dukespace.lib.duke.edu/dspace/bitstream/handle/10161/6650/Ladd%20-%20Education%20and%20Poverty.pdf?sequence=1

http://dukespace.lib.duke.edu/dspace/bitstream/handle/10161/6650/Ladd%20-%20Education%20and%20Poverty.pdf?sequence=1

http://www.epi.org/blog/test-score-gaps-refuse-budge-claim-poverty/

http://www.epi.org/blog/test-score-gaps-refuse-budge-claim-poverty/

http://www.edtrust.org/sites/edtrust.org/files/A_Step_Forward_Or_A_Step_Back.pdf

http://www.edweek.org/ew/articles/2013/10/09/07tucker.h33.html




9. In recent years, there has been steady movement to strengthen accountability metrics in early learning that go beyond checklists and promote tools that measure effective practice. See Stoney, L. (April 26, 2012). “Effective QRIS Technical Assistance and Coaching: Challenges and Opportunities.” Education Week, Sara Mead’s Policy Notebook blog. http://blogs.edweek.org/edweek/sarameads_policy_notebook/2012/04/effective_qris_technical_assistance_and_coaching_challenges_and_opportunities.html; Severns, M. (November 8, 2011). “New Rules Make Quality Count in Head Start Programs.” New America Foundation, Early Ed Watch blog. http://earlyed.newamerica.net/blogposts/2011/finally_making_quality_count_in_head_start_programs-60131; Bodrova, E., Leong, D., and Shore, R. (March 2004). “Child Outcome Standards in Pre-K Programs: What Are Standards; What Is Needed to Make Them Work?” National Institute for Early Education Research, Preschool Policy Matters. http://nieer.org/resources/policybriefs/5.pdf.

10. Bryk, A. (2010). “Organizing Schools for Improvement.” Phi Delta Kappan, Vol. 91, No. 7, 24–25. http://www.carnegiefoundation.org/sites/default/files/elibrary/bryk_organizing-schools_pdk.pdf.

11. University of Chicago Consortium on Chicago School Research and the Ounce of Prevention Fund. (2013). “Measuring Organizational Support in Early Childhood Education Settings: Adapting the Five Essential Supports Surveys for Early Learning Programs.” Proposal submitted to the Institute for Education Sciences.

12. Doherty, K., and Jacobs, S. (2013). “State of the States 2013: Connect the Dots: Using Evaluations of Teacher Effectiveness to Inform Policy and Practice.” National Council on Teacher Quality. http://www.nctq.org/dmsView/State_of_the_States_2013_Using_Teacher_Evaluations_NCTQ_Report; Hull, J. (2011). “Building a Better Evaluation System.” Center for Public Education. http://www.centerforpubliceducation.org/Main-Menu/Staffingstudents/Building-A-Better-Evaluation-System/Building-A-Better-Evaluation-System.html.

13. Bornfreund, L. (2013). “An Ocean of Unknowns: Risk and Opportunity in Using Student Achievement Data to Evaluate PreK–3rd Grade Teachers.” New America Foundation. http://newamerica.net/publications/policy/an_ocean_of_unknowns; Center on Public Education. (2013). “Trends in Teacher Evaluation: How States Are Measuring Teacher Performance.” http://www.centerforpubliceducation.org/teacherevalreview.

14. Sebring, P., Allensworth, E., Bryk, A., Easton, J., and Luppescu, S. (2006). “The Essential Supports for School Improvement.” Consortium on Chicago School Research. http://ccsr.uchicago.edu/sites/default/files/publications/EssentialSupports.pdf.

15. Ibid.; UChicago Impact. (n.d.). “5Essentials.” http://www.crpe.org/sites/default/files/UChicago%20Impact%205E%20onesheet_2012.pdf.

16. Chicago Consortium on School Research. ( January 21, 2014). “Illinois 5Essentials FAQ.” 5Essentials Client Services. http://help.ccsrsurvey.uchicago.edu/customer/portal/articles/780471-illinois-5essentials-faqs.

17. Illinois State Board of Education, Division of Public Education. (2013). “Illinois 5Essentials Survey: Fact Sheet.” http://www.isbe.net/performance/pdf/5essentials/fact-sheet-1013.pdf.

18. US Department of Education. (February 10, 2012). “ESEA Flexibility Request,” 107–110. http://www.ed.gov/esea/flexibility/documents/esea-flexibility-request.doc; No Child Left Behind Act of 2001, Pub. Law No. 107–110. ( January 8, 2002). 107th Cong. ( January 8, 2002). http://www2.ed.gov/policy/elsec/leg/esea02/107-110.pdf.

19. For example, a recent Fordham Institute survey found that many parents value career education, instruction in citizenship and democracy, teaching the ability to work with people of diverse backgrounds, and arts and music instruction. Zeehandelaar, D., and Northern, A. (2013) “What Parents Want: Education Preferences and Trade-offs.” http://www.edexcellence.net/publications/what-parents-want.html.

20. As Robert Pondiscio, Executive Director of CitizenshipFirst, has said, “I have made no secret of my discontent with the effects of standardized testing, while maintaining support for the principle of accountability.” Pondiscio, R. (March 11, 2014). “Testing, Truth, and Consequences.” Education Week, Bridging Differences blog. http://blogs.edweek.org/edweek/Bridging-Differences/2014/03/testing_truth_and_consequences.html.

http://blogs.edweek.org/edweek/sarameads_policy_notebook/2012/04/effective_qris_technical_assistance_and_coaching_challenges_and_opportunities.html

http://blogs.edweek.org/edweek/sarameads_policy_notebook/2012/04/effective_qris_technical_assistance_and_coaching_challenges_and_opportunities.html

http://earlyed.newamerica.net/blogposts/2011/finally_making_quality_count_in_head_start_programs-60131

http://earlyed.newamerica.net/blogposts/2011/finally_making_quality_count_in_head_start_programs-60131

http://nieer.org/resources/policybriefs/5.pdf


http://www.carnegiefoundation.org/sites/default/files/elibrary/bryk_organizing-schools_pdk.pdf

http://www.nctq.org/dmsView/State_of_the_States_2013_Using_Teacher_Evaluations_NCTQ_Report

http://www.nctq.org/dmsView/State_of_the_States_2013_Using_Teacher_Evaluations_NCTQ_Report

http://www.centerforpubliceducation.org/Main-Menu/Staffingstudents/Building-A-Better-Evaluation-System/Building-A-Better-Evaluation-System.html

http://www.centerforpubliceducation.org/Main-Menu/Staffingstudents/Building-A-Better-Evaluation-System/Building-A-Better-Evaluation-System.html

http://newamerica.net/publications/policy/an_ocean_of_unknowns

http://newamerica.net/publications/policy/an_ocean_of_unknowns

http://www.centerforpubliceducation.org/teacherevalreview

http://ccsr.uchicago.edu/sites/default/files/publications/EssentialSupports.pdf

http://ccsr.uchicago.edu/sites/default/files/publications/EssentialSupports.pdf

http://www.crpe.org/sites/default/files/UChicago%20Impact%205E%20onesheet_2012.pdf

http://help.ccsrsurvey.uchicago.edu/customer/portal/articles/780471-illinois-5essentials-faqs

http://www.isbe.net/performance/pdf/5essentials/fact-sheet-1013.pdf

http://www2.ed.gov/policy/elsec/leg/esea02/107-110.pdf

http://www.edexcellence.net/publications/what-parents-want.html

http://blogs.edweek.org/edweek/Bridging-Differences/2014/03/testing_truth_and_consequences.html




21. The question of whether elementary schools should be responsible for high school graduation rates is a potentially interesting one, noted in the sidebar on page 28. But even if a state wanted to include high school graduation rates in elementary school accountability, the practical challenges—including the time lag in collecting data—would be enormous.

22. Shonkoff, J., and Phillips, D. (2000). From Neurons to Neighborhoods: The Science of Early Childhood Development. Washington, DC: National Academy Press. http://www.nap.edu/openbook.php?record_id=9824; Kagan, S., Moore, E,, and Bredekamp, S. (1995). “Reconsidering Children’s Early Development and Learning: Toward Common Views and Vocabulary.” National Education Goals Panel. http://eric.ed.gov/?id=ED391576.

23. Petrilli, M. (August 14, 2013). “The Problem with Proficiency.” Thomas B. Fordham Institute, Flypaper blog. http://www.edexcellence.net/commentary/education-gadfly-weekly/2013/august-15/the-problem-with-proficiency.html. While growth is the better measure, proficiency is still important. Finn, C., Jr. (August 19, 2013). “Let’s Hear It for Proficiency.” Thomas B. Fordham Institute, Flypaper blog. http://www.edexcellence.net/commentary/education-gadfly-daily/flypaper/2013/lets-hear-it-for-proficiency.html.

24. Hussain, I. (2013). “The School Inspector Calls: Low Ratings Drive Improvements for Schools in England.” Education Next, Vol. 13, No. 3. http://educationnext.org/the-school-inspector-calls/; Jerald, C. (2012). “On Her Majesty’s School Inspection Service.” Education Sector. http://www.educationsector.org/sites/default/files/publications/UKInspections-RELEASED.pdf.

25. Office for Standards in Education, Children’s Services and Skills. (2012). “The Framework for School Inspection.” http://webarchive.nationalarchives.gov.uk/20130401151715/https://www.education.gov.uk/publications/eOrderingDownload/090019.pdf.

26. Hussain. “The School Inspector Calls”; Robelen, E. ( January 11, 2012). “School Inspections Eyed as State Strategy in Post-NCLB Era.” Education Week, Curriculum Matters blog. http://blogs.edweek.org/edweek/curriculum/2012/01/school_inspections_an_idea_who.html.

27. Office for Standards in Education, Children’s Services and Skills. (2014) “Understanding a School Inspection Report.” http://www.ofsted.gov.uk/schools/for-schools/understanding-school-inspection-report.

28. Ryan, K., Gandha, T., and Ahn, J. (2013) “School Self-Evaluation and Inspection for Improving US Schools?” National Education Policy Center. http://nepc.colorado.edu/files/pb-inspectorate-final.pdf; Jerald, C. “On Her Majesty’s School Inspection Service”; Crone, T., Coghlan, M., Wade, P., and Rudd, P. (2009). “Evaluation of the Impact of Section 5 Inspections—Strand 3.” National Foundation for Educational Research. http://www.nfer.ac.uk/nfer/publications/SFO01/SFO01.pdf.

29. Tucker, M. “Accountability: What the Top Performers Do.” Education Week, Top Performers blog. http://blogs.edweek.org/edweek/top_performers/2014/03/accountability_what_the_top_performers_do.html?cmp=ENL-EU-NEWS3; Dutch Education Inspectorate, Ministry of Education, Culture and Science. (n.d.). “The Dutch Inspectorate of Education.” http://www.onderwijsinspectie.nl/english

30. Zellman, Gail and Karoly, Lynn A. (2012) “Moving to Outcomes: Approaches to Incorporating Child Assessments into State Early Childhood Quality Rating and Improvement Systems.” 10. RAND. http://www.rand.org/pubs/occasional_papers/OP364.html.

31. Child Care Aware of America. “Accreditation.” (n.d.). http://childcareaware.org/parents-and-guardians/parent-information/accreditation.

32. Ackerman, Debra J. (2014) “State-Funded PreK Policies on External Classroom Observations: Issues and Status.” Educational Testing Service. http://qrisnetwork.org/sites/all/files/resources/[email protected]/2014-05-12%2007:19/State-Funded%20PreK%20Policies%20on%20External%20Classroom%20Observations%20-%20Issues%20and%20Status.pdf.

33. Head Start Resource Center. “An Overview of CLASS.” (2009). http://www.headstartresourcecenter.org/events/?sub=168&subsub=169.

http://www.nap.edu/openbook.php?record_id=9824

http://eric.ed.gov/?id=ED391576

http://www.edexcellence.net/commentary/education-gadfly-weekly/2013/august-15/the-problem-with-proficiency.html

http://www.edexcellence.net/commentary/education-gadfly-weekly/2013/august-15/the-problem-with-proficiency.html

http://www.edexcellence.net/commentary/education-gadfly-daily/flypaper/2013/lets-hear-it-for-proficiency.html

http://educationnext.org/the-school-inspector-calls/

http://www.educationsector.org/sites/default/files/publications/UKInspections-RELEASED.pdf

http://www.educationsector.org/sites/default/files/publications/UKInspections-RELEASED.pdf

http://webarchive.nationalarchives.gov.uk/20130401151715/https://www.education.gov.uk/publications/eOrderingDownload/090019.pdf

http://webarchive.nationalarchives.gov.uk/20130401151715/https://www.education.gov.uk/publications/eOrderingDownload/090019.pdf

http://blogs.edweek.org/edweek/curriculum/2012/01/school_inspections_an_idea_who.html

http://www.ofsted.gov.uk/schools/for-schools/understanding-school-inspection-report

http://nepc.colorado.edu/files/pb-inspectorate-final.pdf

http://www.nfer.ac.uk/nfer/publications/SFO01/SFO01.pdf

http://blogs.edweek.org/edweek/top_performers/2014/03/accountability_what_the_top_performers_do.html?cmp=ENL-EU-NEWS3



http://www.rand.org/pubs/occasional_papers/OP364.html

http://childcareaware.org/parents-and-guardians/parent-information/accreditation

http://qrisnetwork.org/sites/all/files/resources/[email protected]/2014-05-12%2007:19/State-Funded%20PreK%20Policies%20on%20External%20Classroom%20Observations%20-%20Issues%20and%20Status.pdf



http://www.headstartresourcecenter.org/events/?sub=168&subsub=169




34. Zellman, G., and Karoly, L. “Moving to Outcomes: Approaches to Incorporating Child Assessments into State Early Childhood Quality Rating and Improvement Systems.”

35. Danielson, C. (2013). “The Framework for Teaching Evaluation Instrument.” Danielson Group. http://www.danielsongroup.org/userfiles/files/downloads/2013EvaluationInstrument.pdf; Guernsey, L., and Ochshorn, S. (2011). “Watching Teachers Work: Using Observation Tools to Promote Effective Teaching in the Early Years and Early Grades.” New America Foundation. http://earlyed.newamerica.net/sites/newamerica.net/files/policydocs/Watching_Teachers_Work.pdf.

36. For example, states can offer model rubrics for teacher evaluation; states that have done so include Colorado (http://www.cde.state.co.us/educatoreffectiveness/smes-teacher) and Massachusetts (http://www.doe.mass.edu/edeval/model/).

37. Clements, D., and Sarama, J. (March 3, 2014). “Play, Mathematics, and False Dichotomies.” National Institute for Early Education Research, Preschool Matters blog. http://preschoolmatters.org/2014/03/03/play-mathematics-and-false-dichotomies/; Schweinhart, L. (March 1, 2014). “Taking the Long View on Play in Early Childhood Education.” National Institute for Early Education Research, Preschool Matters blog. http://preschoolmatters.org/2014/03/01/taking-the-long-view-on-play-in-early-childhood-education/; Hirsh-Pasek, K., and Golinkoff, R. (2008). “Why Play=Learning.” Encyclopedia on Early Childhood Development. http://www.child-encyclopedia.com/documents/Hirsh-Pasek-GolinkoffANGxp.pdf.

38. Hamre, B. (n.d.). “Using Classroom Observation to Gauge Teacher Effectiveness: Classroom Assessment Scoring System.” Center for Advanced Study on Teaching and Learning. http://www.gse.harvard.edu/cepr-resources/files/news-events/ncte-conference-class-hamre.pdf.

39. Center for Advanced Study on Teaching and Learning. (n.d.). “Classroom Assessment Scoring System.” http://curry.virginia.edu/research/centers/castl/class.

40. Pianta, R. (2012). “Implementing Observation Protocols: Lessons for K-12 Education from the Field of Early Childhood.”Center for American Progress. http://www.americanprogress.org/wp-content/uploads/issues/2012/05/pdf/observation_protocols.pdf; Burchinal, M., Vandergrift, N., Pianta, R., and Mashburn, A. (2010). “Threshold Analysis of Association Between Child Care Quality and Child Outcomes for Low-Income Children in Prekindergarten Programs.” Early Childhood Research Quarterly, Vol. 25, 166–176. http://eric.ed.gov/?id=EJ874842.

41. Teachstone Training LLC. (n.d.). “The CLASS Tool.” http://www.teachstone.com/about-the-class/.

42. Frank Porter Graham Child Development Institute. (n.d.). “Environment Rating Scales.” University of North Carolina at Chapel Hill. http://ers.fpg.unc.edu/.

43. National Center on Child Care Quality Improvement. (October 2013). “QRIS Standards, Levels, and Rating Systems.” Administration for Children and Families. https://occqrisguide.icfwebservices.com/files/QRIS_Levels_Rating.pdf; QRIS National Learning Network. (n.d.). “Glossary of Terms.” http://qrisnetwork.org/glossary.

44. AdvancED. (2014). “What Is Accreditation?” http://www.advanc-ed.org/what-accreditation; New England Association of Schools and Colleges. ( July 2006). “The Impact of Accreditation on the Quality of Education: Results of the Regional Accreditation and Quality of Education Survey, NEASC 2005.” http://www.neasc.org/downloads/SURVEY_REPORT_IN_FULL.pdf.

45. Mead, S. (August 28, 2013). “Do QRIS Improve Student Outcomes?” Education Week, Sara Mead’s Policy Notebook blog. http://blogs.edweek.org/edweek/sarameads_policy_notebook/2013/08/do_qris_improve_student_outcomes.html.

46. Large Countywide and Suburban District Consortium. “21st Century Education Accountability.”

47. Sabol, T., Soliday Hong, S., Pianta, R., and Burchinal, M. (August 23, 2013). “Can Rating Pre-K Programs Predict Children’s Learning?” Science, Vol. 341, No. 6148, 845–846. http://www.sciencemag.org/content/341/6148/845.

http://www.danielsongroup.org/userfiles/files/downloads/2013EvaluationInstrument.pdf

http://earlyed.newamerica.net/sites/newamerica.net/files/policydocs/Watching_Teachers_Work.pdf

http://www.cde.state.co.us/educatoreffectiveness/smes-teacher

http://www.doe.mass.edu/edeval/model/

http://preschoolmatters.org/2014/03/03/play-mathematics-and-false-dichotomies/

http://preschoolmatters.org/2014/03/03/play-mathematics-and-false-dichotomies/

http://preschoolmatters.org/2014/03/01/taking-the-long-view-on-play-in-early-childhood-education/

http://www.child-encyclopedia.com/documents/Hirsh-Pasek-GolinkoffANGxp.pdf

http://www.gse.harvard.edu/cepr-resources/files/news-events/ncte-conference-class-hamre.pdf

http://curry.virginia.edu/research/centers/castl/class

http://www.americanprogress.org/wp-content/uploads/issues/2012/05/pdf/observation_protocols.pdf

http://www.americanprogress.org/wp-content/uploads/issues/2012/05/pdf/observation_protocols.pdf

http://eric.ed.gov/?id=EJ874842

http://eric.ed.gov/?id=EJ874842

http://www.teachstone.com/about-the-class/

http://ers.fpg.unc.edu/

https://occqrisguide.icfwebservices.com/files/QRIS_Levels_Rating.pdf

https://occqrisguide.icfwebservices.com/files/QRIS_Levels_Rating.pdf

http://qrisnetwork.org/glossary

http://www.advanc-ed.org/what-accreditation

http://www.neasc.org/downloads/SURVEY_REPORT_IN_FULL.pdf

http://blogs.edweek.org/edweek/sarameads_policy_notebook/2013/08/do_qris_improve_student_outcomes.html

http://blogs.edweek.org/edweek/sarameads_policy_notebook/2013/08/do_qris_improve_student_outcomes.html

http://www.sciencemag.org/content/341/6148/845.summary




summary; Mashburn, A. et al. (2008). “Measures of Classroom Quality in Prekindergarten and Children’s Development of Academic, Language, and Social Skills.” Child Development, Vol. 79, No. 3, 732–749.

48. Dealy, K. (2008). “Foundations: How States Can Plan and Fund Programs for Babies and Toddlers.” Ounce of Prevention Fund. http://www.ounceofprevention.org/news/pdfs/Foundations.pdf; Miller, S. (2008). “Secure Attachment.” Ounce of Prevention Fund. http://www.ounceofprevention.org/research/pdfs/SecureAttachment.pdf.

49. La Paro, K. et al. (2009). “Quality in Kindergarten Classrooms: Observational Evidence for the Need to Increase Children’s Learning Opportunities in Early Education Classrooms.” Early Education and Development, Vol. 20, No. 4, 657–692; Pianta, R., and Hamre, B. (2009). “Conceptualization, Measurement, and Improvement of Classroom Processes: Standardized Observation Can Leverage Capacity.” Educational Researcher, Vol. 38, No. 2, 109–119; Sabol et al. “Can Rating Pre-K Programs Predict Children’s Learning?”; Dealy, “Foundations”; Miller, “Secure Attachment”; Dealy, K., Pacchiano, D., and Shimpi, P. (n.d.). “The Language of Babies, Toddlers, and Preschoolers: Connecting Research to Practice.” Ounce of Prevention Fund. http://www.ounceofprevention.org/languageofbabies/index.html#/1/. Continuity of care in which children remain with the same teacher over time is also important for establishing the strong relationships that promote attachment between teacher and child, and thus provide the foundation for learning. See Miller, “Secure Attachment.”

50. La Paro et al. “Quality in Kindergarten Classrooms”; Dealy, “Foundations”; Miller, “Secure Attachment”; Dealy et al. “The Language of Babies.”

51. Dealy, “The Language of Babies.”

52. Ibid.

53. Ibid.

54. Ibid.

55. Dealy, “Foundations”; Hirsh-Pasek, K., Golinkoff, R., Berk, L., and Singer, D. A Mandate for Playful Learning in Preschool: Presenting the Evidence. (2009). New York, NY: Oxford University Press.

56. Dealy, “Foundations.”

57. Hirsh-Pasek et al. A Mandate for Playful Learning.

58. La Paro et al. “Quality in Kindergarten Classrooms”; Pianta and Hamre, “Conceptualization, Measurement, and Improvement.”

59. Ujifusa, A. ( January 14, 2014). “All 50 States Now Using Testing or Student Performance for School Accountability.” Education Week, State EdWatch blog. http://blogs.edweek.org/edweek/state_edwatch/2014/01/all_50_states_now_using_testing_or_student_performance_for_school_accountability.html?cmp=ENL-EU-NEWS3; Lovell, P., Cardichon, J., and Jones, F. (2013) “The Effect of ESEA Waiver Plans on High School Graduation Rate Flexibility.” Alliance for Excellent Education. http://all4ed.org/wp-content/uploads/2013/06/ESEAWaivers.pdf.

60. National Education Association. (2011) “Beyond Two Test Scores: Multiple Measures of Student Learning and School Accountability.” http://www.nea.org/assets/docs/PB38beyondtwotestscores2011.pdf; Committee for Economic Development. ( January 2001). “Measuring What Matters: Using Assessment and Accountability to Improve Student Learning.” http://www.ced.org/pdf/Measuring-What-Matters.pdf.

61. National Early Childhood Accountability Task Force. (2007). “Taking Stock: Assessing and Improving Early Childhood Learning and Program Quality.” http://www.pewtrusts.org/uploadedFiles/wwwpewtrustsorg/Reports/Pre-k_education/task_force_report1.pdf.

62. Meisels, S. (2006). “Accountability in Early Childhood: No Easy Answers.” Herr Research Center for Children and Social Policy, Erikson Institute. http://www.isbe.state.il.us/earlychi/pdf/meisels_accountability.pdf; Shepard, L.,

http://www.sciencemag.org/content/341/6148/845.summary

http://www.ounceofprevention.org/news/pdfs/Foundations.pdf

http://www.ounceofprevention.org/research/pdfs/SecureAttachment.pdf

http://www.ounceofprevention.org/research/pdfs/SecureAttachment.pdf

http://blogs.edweek.org/edweek/state_edwatch/2014/01/all_50_states_now_using_testing_or_student_performance_for_school_accountability.html?cmp=ENL-EU-NEWS3

http://blogs.edweek.org/edweek/state_edwatch/2014/01/all_50_states_now_using_testing_or_student_performance_for_school_accountability.html?cmp=ENL-EU-NEWS3

http://all4ed.org/wp-content/uploads/2013/06/ESEAWaivers.pdf

http://www.nea.org/assets/docs/PB38beyondtwotestscores2011.pdf

http://www.ced.org/pdf/Measuring-What-Matters.pdf

http://www.pewtrusts.org/uploadedFiles/wwwpewtrustsorg/Reports/Pre-k_education/task_force_report1.pdf

http://www.isbe.state.il.us/earlychi/pdf/meisels_accountability.pdf




Kagan, L., and Wurtz, E. (1998). “Principles and Recommendations for Early Childhood Assessments.” Goal 1 Early Childhood Assessments Resource Group, National Education Goals Panel. http://govinfo.library.unt.edu/negp/reports/prinrec.pdf.

63. Foundation for Excellence in Education. (2013). “Accountability for School Learning: A–F School Grades.” http://static.excelined.org/wp-content/uploads/School-Grading-PB_OVRVW-P1-3.pdf?__hstc=76856138.085c0f1c0bbd282238b6755de0db2a7a.1382991083267.1382991083267.1382991083267.1&__hssc=76856138.7.1382991083268&__hsfp=2274203314; Thomas, M. (October 17, 2013). “Excellence in Education Strategy Session: Accountability 2020: Adopting and Readying Transparent A–F Grading Systems for the Next Generation of Schools.” Foundation for Excellence in Education, Edfly Blog. http://excelined.org/2013/10/excellence-education-strategy-session-accountability-2020-adopting-readying-transparent-f-grading-systems-next-generation-schools/.

64. Mitchell, A. (2005). “Stair Steps to Quality: A Guide for States and Communities Developing Quality Rating Systems.” United Way Success by 6. http://www.earlychildhoodfinance.org/downloads/2005/MitchStairSteps_2005.pdf.

65. National Center on Child Care Quality Improvement. (2013) “QRIS Financial Incentives.” Administration for Children and Families. https://occqrisguide.icfwebservices.com/files/QRIS_Financial_Incentives.pdf; Associated Press. (October 28, 2013). “Making the Grade, A–F: Lawmakers Weigh New School Accountability System.” Escanaba (MI) Daily Press. http://www.dailypress.net/page/content.detail/id/544597/Making-the-grade--A-F--Lawmakers-weigh-new-school-accountability-system.html?nav=5003; DiCarlo, M. (February 11, 2013). “Why Did Florida Schools’ Grades Improve Dramatically Between 1999 and 2005?” Albert Shanker Institute, Shanker Blog. http://shankerblog.org/?p=7642.

66. Petrilli, M. (2013). “Quality Control in a Local Marketplace,” in Pathway to Success in Milwaukee Schools, ed. Kozak, W. Wisconsin Policy Research Institute. http://www.aei.org/files/2013/05/29/-petrilli-quality-control-in-a-local-market_162508122005.pdf.

67. McNeil, M. (February 21, 2014). “New NCLB Waiver Reports Show More Issues With Struggling Schools, New Tests.” Education Week, Politics K-12 blog. http://blogs.edweek.org/edweek/campaign-k-12/2014/02/kansas_oklahoma_south_dakota_g.html?cmp=ENL-EU-NEWS3.

68. Cody, A. (September 15, 2013). “‘Reign of Error’ Reviewed: Ravitch Rises.” Education Week, Living in Dialogue blog, http://blogs.edweek.org/teachers/living-in-dialogue/2013/09/review_of_reign_of_error.html?cmp=ENL-EU-VIEWS2.

69. Sebring et al. “Essential Supports.”

70. Council of Chef State School Officers. (2011). “Roadmap for Next-Generation State Accountability Systems,” 3. http://www.ccsso.org/Documents/2011/Roadmap_for_Next-Generation_Accountability_2011.pdf; Fullan, M. April 2011). “Choosing the Wrong Drivers for Whole System Reform,” 17. http://www.edsource.org/today/wp-content/uploads/Fullan-Wrong-Drivers1.pdf; Johnson-Staub, C. (2011). “Meeting the Early Learning Challenge: A Checklist for High Quality QRIS.” Center for Law and Social Policy. http://www.clasp.org/admin/site/publications/files/Meeting-the-Early-Learning-Challenge-A-Checklist-for-a-High-Quality-QRIS.pdf; Darling-Hammond, L. and Weingarten, R. (May 20, 2014). “It’s Time for a New Accountability in American Education.” Huffington Post. http://www.huffingtonpost.com/linda-darlinghammond/its-time-for-a-new-accoun_b_5351475.html.

71. Klein, A. (February 26, 2014). “School Improvement Grant Reanalysis Shows Smaller Gains Than First Reported.” Education Week, News in Brief. http://www.edweek.org/ew/articles/2014/02/26/22brief-b2.h33.html.

72. American Institutes for Research. (2011). “Reauthorizing ESEA.” http://www.air.org/files/0669_PG_SchoolTurnaround_Online_d41.pdf; US Department of Health and Human Services, Administration on Children and Families. (September 22, 2010 ). “Designation Renewal of Head Start Grantees.” Federal Register, Vol. 75, No. 183, 57704. http://www.gpo.gov/fdsys/pkg/FR-2010-09-22/pdf/2010-23583.pdf; US Department of Education. (2011). “An Overview of School Turnaround.” http://www2.ed.gov/programs/sif/sigoverviewppt.pdf.

http://govinfo.library.unt.edu/negp/reports/prinrec.pdf

http://static.excelined.org/wp-content/uploads/School-Grading-PB_OVRVW-P1-3.pdf?__hstc=76856138.085c0f1c0bbd282238b6755de0db2a7a.1382991083267.1382991083267.1382991083267.1&__hssc=76856138.7.1382991083268&__hsfp=2274203314



http://excelined.org/2013/10/excellence-education-strategy-session-accountability-2020-adopting-readying-transparent-f-grading-systems-next-generation-schools/



http://www.earlychildhoodfinance.org/downloads/2005/MitchStairSteps_2005.pdf

http://www.earlychildhoodfinance.org/downloads/2005/MitchStairSteps_2005.pdf

https://occqrisguide.icfwebservices.com/files/QRIS_Financial_Incentives.pdf

http://www.dailypress.net/page/content.detail/id/544597/Making-the-grade--A-F--Lawmakers-weigh-new-school-accountability-system.html?nav=5003

http://www.dailypress.net/page/content.detail/id/544597/Making-the-grade--A-F--Lawmakers-weigh-new-school-accountability-system.html?nav=5003

http://shankerblog.org/?p=7642

http://www.aei.org/files/2013/05/29/-petrilli-quality-control-in-a-local-market_162508122005.pdf

http://blogs.edweek.org/edweek/campaign-k-12/2014/02/kansas_oklahoma_south_dakota_g.html?cmp=ENL-EU-NEWS3

http://blogs.edweek.org/edweek/campaign-k-12/2014/02/kansas_oklahoma_south_dakota_g.html?cmp=ENL-EU-NEWS3

http://blogs.edweek.org/teachers/living-in-dialogue/2013/09/review_of_reign_of_error.html?cmp=ENL-EU-VIEWS2

http://blogs.edweek.org/teachers/living-in-dialogue/2013/09/review_of_reign_of_error.html?cmp=ENL-EU-VIEWS2

http://www.ccsso.org/Documents/2011/Roadmap_for_Next-Generation_Accountability_2011.pdf

http://www.edsource.org/today/wp-content/uploads/Fullan-Wrong-Drivers1.pdf

http://www.edsource.org/today/wp-content/uploads/Fullan-Wrong-Drivers1.pdf

http://www.clasp.org/admin/site/publications/files/Meeting-the-Early-Learning-Challenge-A-Checklist-for-a-High-Quality-QRIS.pdf

http://www.clasp.org/admin/site/publications/files/Meeting-the-Early-Learning-Challenge-A-Checklist-for-a-High-Quality-QRIS.pdf

http://www.edweek.org/ew/articles/2014/02/26/22brief-b2.h33.html

http://www.air.org/files/0669_PG_SchoolTurnaround_Online_d41.pdf

http://www.air.org/files/0669_PG_SchoolTurnaround_Online_d41.pdf

http://www.gpo.gov/fdsys/pkg/FR-2010-09-22/pdf/2010-23583.pdf

http://www2.ed.gov/programs/sif/sigoverviewppt.pdf




73. Samuels, C. (April 3, 2013). “Some Head Start Providers Lose Their Federal Aid.” Education Week, Early Years blog. http://blogs.edweek.org/edweek/early_years/2013/04/some_head_start_providers_to_lose_their_federal_aid.html; National Council on Teacher Quality. (2011). “Recommendations for the Reauthorization of ESEA.” http://www.nctq.org/p/publications/docs/nctq_eseaReauthorization.pdf.

74. Shea, J., and Liu, M. (2010). “School Improvement Grants: Take 2.” Mass Insight Education. http://www.massinsight.org/publications/stg-resources/125/file/1/pubs/2010/12/07/STG_-_School_Improvement_Grants_Take_2_Dec_2010__.pdf.

75. Regenstein, E., and Lipper, K. (2013). “A Framework for Choosing a State-Level Early Childhood Governance System,” 9–13. BUILD Initiative. http://www.buildinitiative.org/Portals/0/Uploads/Documents/Early%20Childhood%20Governance%20for%20Web.pdf.

76. Mass Insight Education. (2012). “Being Bold: An Assessment of Turnaround Initiatives in Select School Districts and States.” http://www.massinsight.org/publications/turnaround/162/file/1/pubs/.

77. Heckman, J. (2012). “The Case for Investing in Disadvantaged Young Children.” Big Ideas for Our Children: Investing in Our Nation’s Future, Vol. 55, 49–56. http://www.heckmanequation.org/content/resource/case-investing-disadvantaged-young-children; Grunewald, R., and Rolnick, A. (2006). “A Proposal for Achieving High Returns on Early Childhood Development.” Federal Reserve Bank of Minneapolis. http://www.minneapolisfed.org/publications_papers/studies/earlychild/highreturn.pdf; Karoly, L., and Kilburn, M. (2005). “Early Childhood Interventions: Proven Results, Future Promise.” RAND. http://www.rand.org/content/dam/rand/pubs/monographs/2005/RAND_MG341.pdf.

78. Petersen, S., Jones, L., and McGinley, K. (2008). “Early Learning Guidelines for Infants and Toddlers: Recommendations to States.” ZERO TO THREE. http://main.zerotothree.org/site/DocServer/Infant-Toddler_Early_Learning_Guidelines_Factsheet.pdf?docID=11941&JServSessionIda004=l2r7remp81.app33d.

79. Adams, G., and Rohacek, M. (2002). “More Than a Work Support? Issues Around Integrating Child Development Goals Into the Child Care Subsidy System.” Early Childhood Research Quarterly, Vol. 17, 418–440. http://www.urbaninstitute.org/UploadedPDF/1000449_ECRQtrly.pdf.

80. Barnett, W., and Hustedt, J. (2011). “Improving Public Financing for Early Learning Programs.” National Institute for Early Education Research. Preschool Policy Brief, Vol. 23. http://nieer.org/resources/policybriefs/24.pdf.

81. Barnett, W., Carolan, M., Fitzgerald, J., and Squires, J. (2012). “The State of Preschool 2012: State Preschool Yearbook. National Institute for Early Education Research. http://nieer.org/sites/nieer/files/yearbook2012.pdf; Barnett, W. (2003). “Low Wages = Low Quality. Solving the Real Preschool Teacher Crisis.” National Institute for Early Education Research, Preschool Policy Matters, http://nieer.org/resources/policybriefs/3.pdf.

82. Regenstein and Lipper. “A Framework for Choosing,” 9–13.

83. US Department of Health and Human Services. (2009). “Head Start Program Performance Standards 45 CFR Chapter XIII.” Administration for Children and Families. https://eclkc.ohs.acf.hhs.gov/hslc/standards/Head%20Start%20Requirements/45-cfr-chapter-xiii/45-cfr-chap-xiii-eng.pdf.

84. US Department of Health and Human Services. (November 9, 2011). “Head Start Program.” Federal Register, Vol. 76, No. 217, 70010. http://eclkc.ohs.acf.hhs.gov/hslc/standards/Head%20Start%20Requirements/1307/Part%201307-FRNotice_2011-28880.pdf.

85. Barnett et al. “The State of Preschool 2012.”

86. Zellman and Karoly. “Moving to Outcomes,” 10.

87. Los Angeles County Office of Education. (2014). “Head Start-State Preschool Division. Annual Report 2013.” http://www.prekid.org/userfiles/file/xx%20HS%20Annual%20Report%202013_WEB-B.pdf

http://blogs.edweek.org/edweek/early_years/2013/04/some_head_start_providers_to_lose_their_federal_aid.html

http://blogs.edweek.org/edweek/early_years/2013/04/some_head_start_providers_to_lose_their_federal_aid.html

http://www.nctq.org/p/publications/docs/nctq_eseaReauthorization.pdf

http://www.massinsight.org/publications/stg-resources/125/file/1/pubs/2010/12/07/STG_-_School_Improvement_Grants_Take_2_Dec_2010__.pdf

http://www.massinsight.org/publications/stg-resources/125/file/1/pubs/2010/12/07/STG_-_School_Improvement_Grants_Take_2_Dec_2010__.pdf

http://www.buildinitiative.org/Portals/0/Uploads/Documents/Early%20Childhood%20Governance%20for%20Web.pdf

http://www.buildinitiative.org/Portals/0/Uploads/Documents/Early%20Childhood%20Governance%20for%20Web.pdf

http://www.massinsight.org/publications/turnaround/162/file/1/pubs/

http://www.heckmanequation.org/content/resource/case-investing-disadvantaged-young-children

http://www.heckmanequation.org/content/resource/case-investing-disadvantaged-young-children

http://www.minneapolisfed.org/publications_papers/studies/earlychild/highreturn.pdf

http://www.rand.org/content/dam/rand/pubs/monographs/2005/RAND_MG341.pdf

http://main.zerotothree.org/site/DocServer/Infant-Toddler_Early_Learning_Guidelines_Factsheet.pdf?docID=11941&JServSessionIda004=l2r7remp81.app33d

http://main.zerotothree.org/site/DocServer/Infant-Toddler_Early_Learning_Guidelines_Factsheet.pdf?docID=11941&JServSessionIda004=l2r7remp81.app33d

http://www.urbaninstitute.org/UploadedPDF/1000449_ECRQtrly.pdf


http://nieer.org/sites/nieer/files/yearbook2012.pdf


https://eclkc.ohs.acf.hhs.gov/hslc/standards/Head%20Start%20Requirements/45-cfr-chapter-xiii/45-cfr-chap-xiii-eng.pdf

https://eclkc.ohs.acf.hhs.gov/hslc/standards/Head%20Start%20Requirements/45-cfr-chapter-xiii/45-cfr-chap-xiii-eng.pdf

http://eclkc.ohs.acf.hhs.gov/hslc/standards/Head%20Start%20Requirements/1307/Part%201307-FRNotice_2011-28880.pdf

http://eclkc.ohs.acf.hhs.gov/hslc/standards/Head%20Start%20Requirements/1307/Part%201307-FRNotice_2011-28880.pdf

http://www.prekid.org/userfiles/file/xx%20HS%20Annual%20Report%202013_WEB-B.pdf




88. US Department of Education. (2013). “Race to the Top—Early Learning Challenge.” http://www2.ed.gov/programs/racetothetop-earlylearningchallenge/index.html.

89. Section 14005 “State Applications” states, “The State will take action to design and implement an integrated system of high-quality early learning programs and services.” In “State Fiscal Stabilization Fund,” American Recovery and Reinvestment Act of 2009, Sections 14005, 14006, and 14013, Title XIV (Public Law 112-10, February 17, 2009). http://www2.ed.gov/policy/gen/leg/recovery/statutory/stabilization-fund.pdf.

90. Nine states—California, Delaware, Maryland, Massachusetts, Minnesota, North Carolina, Ohio, Rhode Island, and Washington—were first-round winners and granted the full amount they applied for (except for California). Five more states—Colorado, Illinois, New Mexico, Oregon, and Wisconsin—were second-round winners and received 50% of what they originally applied for. Another round of funding in 2013 provided the states that did not initially receive full funding—California and the five second-round winners—with another 25% of their originally requested grant. US Department of Education. (August 2, 2013). “Six States Awarded Supplemental Race to the Top–Early Learning Challenge Grants.” http://www.ed.gov/news/press-releases/six-states-awarded-supplemental-race-top-early-learning-challenge-grants. Most recently, six states were awarded grants in December 2013: Georgia, Kentucky, Michigan, New Jersey, Pennsylvania, and Vermont. US Department of Education. (December 19, 2013). “Six States Awarded Race to the Top–Early Learning Challenge (RTT-ELC) Grants to Build Statewide Systems of High-Quality Early Learning.” http://www.ed.gov/news/press-releases/six-states-awarded-race-top-early-learning-challenge-rtt-elc-grants-build-statew.

91. As of February 2014, six states do not have a fully implemented QRIS (Missouri requires legislative action, while Connecticut, Nebraska, South Dakota, West Virginia, and Wyoming are in the planning stages). QRIS National Learning Network. (2014). “Current Status of QRIS in States.” http://qrisnetwork.org/sites/all/files/maps/QRIS%20Map,%20QRIS%20National%20Learning%20Network,%20www.qrisnetwork.org%20%5BRevised%20February%202014%5D.pdf.

92. National Center on Child Care Quality Improvement. (2011). “Overview of the QRIS Resource Guide,” 8–10. Office of Child Care. http://qrisnetwork.org/sites/all/files/resources/gscobb/2011-12-15%2006:10/QRISResourceGuide.pdf.

93. Ibid., 18–19.

94. For a deeper description of how this works, see Pianta. “Implementing Observation Protocols” and Tout, K. (2010). “The Child Care Quality Rating System Assessment: Compendium of Quality Rating Systems and Evaluations.” Office of Planning, Research and Evaluation, Administration for Children and Families. http://www.acf.hhs.gov/sites/default/files/opre/qrs_compendium_final.pdf.

95. National Center on Child Care Quality Improvement. “Overview of the QRIS Resource Guide,” 4.

96. Ibid., 8–10.

97. Blair, J. “States View Teacher Quality as Pre-K Rating Factor.” ( July 11, 2013). Education Week, Vol. 32, No. 35, 31 and 34. http://www.edweek.org/ew/articles/2013/06/12/35preschool.h32.html; National Center on Child Care Quality Improvement. (2011). “Common Categories of QRIS Quality Standards.” Office of Child Care. https://occqrisguide.icfwebservices.com/files/QRIS_Standards_Categories.pdf; California Early Learning Quality Improvement System Advisory Committee. (2010) “Dream Big for Our Youngest Children,” 13–15. http://www.cde.ca.gov/sp/cd/re/documents/fnlrpt2010.pdf; Smith, S., Robbins, T., Stagman, S., and Kreader, J. (2012). “Practices for Promoting Young Children’s Learning in QRIS Standards.” National Center for Children in Poverty. http://www.nccp.org/publications/pdf/text_1070.pdf.

98. Tout. “Child Care Quality Rating System Assessment.”

99. National Center on Child Care Quality Improvement. “Overview of the QRIS Resource Guide”; Tout. “Child Care Quality Rating System Assessment”; Mitchell, A. (2009). “Quality Rating and Improvement Systems as the

http://www2.ed.gov/programs/racetothetop-earlylearningchallenge/index.html

http://www2.ed.gov/policy/gen/leg/recovery/statutory/stabilization-fund.pdf

http://www.ed.gov/news/press-releases/six-states-awarded-supplemental-race-top-early-learning-challenge-grants

http://www.ed.gov/news/press-releases/six-states-awarded-supplemental-race-top-early-learning-challenge-grants

http://www.ed.gov/news/press-releases/six-states-awarded-race-top-early-learning-challenge-rtt-elc-grants-build-statew

http://www.ed.gov/news/press-releases/six-states-awarded-race-top-early-learning-challenge-rtt-elc-grants-build-statew

http://qrisnetwork.org/sites/all/files/maps/QRIS%20Map,%20QRIS%20National%20Learning%20Network,%20www.qrisnetwork.org%20%5BRevised%20February%202014%5D.pdf



http://qrisnetwork.org/sites/all/files/resources/gscobb/2011-12-15%2006:10/QRISResourceGuide.pdf


http://www.acf.hhs.gov/sites/default/files/opre/qrs_compendium_final.pdf

http://www.edweek.org/ew/articles/2013/06/12/35preschool.h32.html

https://occqrisguide.icfwebservices.com/files/QRIS_Standards_Categories.pdf

http://www.cde.ca.gov/sp/cd/re/documents/fnlrpt2010.pdf

http://www.nccp.org/publications/pdf/text_1070.pdf




Framework for Early Care and Education System Reform,” 6. BUILD Initiative. http://www.earlychildhoodfinance.org/downloads/2009/QRISasSystemReform_2009.pdf.

100. Mayoral, M. (2013). “Supporting Babies Through QRIS: Inclusion of Infant and Toddler Quality Standards.” ZERO TO THREE. http://www.zerotothree.org/public-policy/building-early-childhood-systems/qris/quality-rating-and-improvement-system-standards.pdf; Schmit, S., and Matthews, H. (2013). “Better for Babies: A Study of State Infant and Toddler Child Care Policies.” Center for Law and Social Policy. http://www.clasp.org/admin/site/publications/files/BetterforBabies2.pdf; National Infant and Toddler Child Care Initiative. (2010). “State Quality Rating and Improvement Systems (QRIS): Inclusion of Infant/Toddler Quality Indicators.” Office of Child Care. http://www.zerotothree.org/public-policy/state-community-policy/nitcci/qris-state-breakdown-of-infant-toddler-quality-indicators.pdf.

101. Mayoral. “Supporting Babies Through QRIS”; National Infant and Toddler Child Care Initiative. “State Quality Rating and Improvement Systems.”

102. Committee on Early Childhood Care and Education Workforce. (2012). “The Early Childhood Care and Workforce: Challenges and Opportunities.” Institute of Medicine and National Research Council of the National Academies. http://books.nap.edu/catalog.php?record_id=13238.

103. Ryan, S., and Whitebook, M. (2012). “More Than Teachers.” In Pianta, R., Barnett, S., Justice, L., and Sheridan, S. (eds.). Handbook of Early Childhood Education (New York: Guilford Press), 92–110; Austin, L., Whitebook, M., Connors, M., and Darah, R. (2011). “Staff Preparation, Reward, and Support: Are Quality Rating and Improvement Systems Addressing All of the Key Ingredients Necessary for Change?” Center for the Study of Child Care Employment. http://qrisnetwork.org/sites/all/files/resources/gscobb/2011-09-14%2006:29/report.pdf; “Critical Facts About the Early Childhood Workforce,” National Association for the Education of Young Children. http://www.naeyc.org/policy/advocacy/ecworkforcefacts; Barnett et al. “The State of Preschool 2012,” 18; Schmitt, S. (2013). “Head Start Participants, Programs, Families, and Staff in 2012” Center for Law and Social Policy.

104. Ryan and Whitebook. “More Than Teachers.”

105. Ibid.

106. Mitchell, A., and Stoney, L. ( July 8, 2013). “Comments on the Proposed CCDF Regulations.” Alliance for Early Childhood Finance. www.earlychildhoodfinance.org/downloads/2013/CCDFCommentsLS%26AWLetterheadJuly2013.docx.

107. “The extent to which the State has a High-Quality Plan to demonstrate the relationship between quality ratings and the learning outcomes of children served by Early Learning and Development Programs, by validating that the tiers in the State’s Tiered Quality Rating and Improvement System accurately reflect differential levels of quality, are related to progress in learning and development, and build toward school readiness,” as stated in (B)(5) of the Race to the Top–Early Learning Challenge. US Department of Education. “Race to the Top–Early Learning Challenge 2013 Application for Initial Funding”; Zellman, G., and Fiene, R. (2012). “Validation of Quality Rating and Improvement Systems for Early Care and Education and School-Age Care.” Office of Planning, Research, and Evaluation, Administration for Children and Families. https://www.acf.hhs.gov/sites/default/files/opre/val_qual_early.pdf.

108. Meisels. “Accountability in Early Childhood”; Lombardi, J. “Additional Comments.” In National Early Childhood Accountability Task Force. “Taking Stock.” For purposes of this paper, we take no position on the propriety and quality of the assessments currently used in 3rd grade and higher to satisfy ESEA’s accountability requirements.

109. “QRISs differ in a key way from K–12 accountability efforts because they focus almost exclusively on inputs into caregiving and caregiving processes rather than on the outcomes of the process, which for K–12 accountability systems are measures of student performance on standardized assessments.” Zellman and Karoly. “Moving to Outcomes,” 10.

http://www.earlychildhoodfinance.org/downloads/2009/QRISasSystemReform_2009.pdf

http://www.zerotothree.org/public-policy/building-early-childhood-systems/qris/quality-rating-and-improvement-system-standards.pdf

http://www.zerotothree.org/public-policy/building-early-childhood-systems/qris/quality-rating-and-improvement-system-standards.pdf

http://www.clasp.org/admin/site/publications/files/BetterforBabies2.pdf

http://www.clasp.org/admin/site/publications/files/BetterforBabies2.pdf

http://www.zerotothree.org/public-policy/state-community-policy/nitcci/qris-state-breakdown-of-infant-toddler-quality-indicators.pdf

http://www.zerotothree.org/public-policy/state-community-policy/nitcci/qris-state-breakdown-of-infant-toddler-quality-indicators.pdf

http://qrisnetwork.org/sites/all/files/resources/gscobb/2011-09-14%2006:29/report.pdf

http://www.naeyc.org/policy/advocacy/ecworkforcefacts

http://www.earlychildhoodfinance.org/downloads/2013/CCDFCommentsLS%26AWLetterheadJuly2013.docx

http://www.earlychildhoodfinance.org/downloads/2013/CCDFCommentsLS%26AWLetterheadJuly2013.docx

https://www.acf.hhs.gov/sites/default/files/opre/val_qual_early.pdf

https://www.acf.hhs.gov/sites/default/files/opre/val_qual_early.pdf




110. QRIS National Learning Network. (August 7, 2011). “Webinar: QRIS and CLASS: Emphasizing Effective Teacher–Child Interaction to Improve Child Outcomes.” http://qrisnetwork.org/member/calendar/event/110817/qris-and-class-tm-emphasizing-effective-teacher-child-interaction-impro.

111. Frank Porter Graham Child Development Institute. “Environment Rating Scales.”

112. Zellman Karoly. “Moving to Outcomes.”

113. Examples of states using “levels”: Arkansas, Indiana, Iowa, Kentucky, Maryland, Massachusetts, and Rhode Island. States using “steps”: Maine and Mississippi. New Hampshire uses the term “licensed plus.” The District of Columbia uses “bronze,” “silver,” and “gold.” Tennessee uses “minimum,” “better,” and “higher-quality” standards. Mitchell, A. (2012). “Quality Rating Improvement Systems: A State-by-State Listing of QRIS Websites.” QRIS National Learning Network. http://www.qrisnetwork.org/sites/all/files/resources/gscobb/2012-05-28%2008:04/WebsitesforQRIS.pdf; National Center on Child Care Quality Improvement. “QRIS Standards, Levels, and Rating Systems.”

114. National Center on Child Care Quality Improvement. “QRIS Standards, Levels, and Rating Systems.”

115. Mitchell, A. “Quality Rating and Improvement Systems,” 4.

116. US Department of Health and Human Services. (n.d.). “Standards and Criteria.” Administration for Children and Families. https://occqrisguide.icfwebservices.com/index.cfm?do=question&sid=3&qid=244; National Center on Child Care Quality Improvement. “QRIS Standards, Levels, and Rating Systems.”

117. National Center on Child Care Quality Improvement. “QRIS Standards, Levels, and Rating Systems”; QRIS National Learning Network. “Glossary of Terms.”

118. National Center on Child Care Quality Improvement. “QRIS Standards, Levels, and Rating Systems”; QRIS National Learning Network. “Glossary of Terms.”

119. Colorado Department of Human Services. (2013). “Welcome to Colorado’s Quality Rating and Improvement System: The Next Generation.” http://www.colorado.gov/cs/Satellite/CDHS-ChildYouthFam/CBON/1251583639294; National Center on Child Care Quality Improvement. “QRIS Standards, Levels, and Rating Systems”; QRIS National Learning Network. “Glossary of Terms.”

120. National Center on Child Care Quality Improvement. “QRIS Standards, Levels, and Rating Systems”; QRIS National Learning Network. (n.d.). “State Profiles.” http://qrisnetwork.org/lt/continuous-quality-improvement-quality-rating-and-improvement-systems-qris-learning-table/stateprofiles; Barry, J. (February 3, 2014). “Bright and Early North Dakota.” North Dakota Department of Human Services. http://www.nd.gov/dhs/info/testimony/2013-2014-interim/economic/2-3-bright-early.pdf.

121. Center for Child Care Workforce. (n.d.). “Quality Rating and Improvement Systems.” http://www.ccw.org/index.php?option=com_content&task=view&id=189&Itemid=96; Satkowski, C. (2009). “A Stimulus for Second-Generation QRIS.” New America Foundation. http://newamerica.net/publications/policy/stimulus_second_generation_qris; Smith et al. “Practices for Promoting Young Children’s Learning in QRIS Standards”; National Center for Child Care Quality Improvement. (2013). “QRIS Elements.” Administration for Children and Families. https://childcare.gov/sites/default/files/254_1302_qris_elements.pdf.

122. Tout. “Child Care Quality Rating System Assessment.”

123. Mitchell. “Stair Steps to Quality.”

124. Maxwell, K. (2012). “Discussion Topic: QRIS Validation Issues.” Frank Porter Graham Child Development Institute. http://www.fpg.unc.edu/node/227; Schulman, K., Matthews, H., Blank, H., and Ewen, D. (2012). “A Count for Quality: Child Care Center Directors on Rating and Improvement Systems.” National Women’s Law Center and Center for Law and Social Policy. http://www.nwlc.org/sites/default/files/pdfs/ACountforQualityQRISReport.pdf.

http://qrisnetwork.org/member/calendar/event/110817/qris-and-class-tm-emphasizing-effective-teacher-child-interaction-impro

http://qrisnetwork.org/member/calendar/event/110817/qris-and-class-tm-emphasizing-effective-teacher-child-interaction-impro

http://www.qrisnetwork.org/sites/all/files/resources/gscobb/2012-05-28%2008:04/WebsitesforQRIS.pdf

http://www.qrisnetwork.org/sites/all/files/resources/gscobb/2012-05-28%2008:04/WebsitesforQRIS.pdf

https://occqrisguide.icfwebservices.com/index.cfm?do=question&sid=3&qid=244

http://www.colorado.gov/cs/Satellite/CDHS-ChildYouthFam/CBON/1251583639294

http://www.colorado.gov/cs/Satellite/CDHS-ChildYouthFam/CBON/1251583639294

http://qrisnetwork.org/lt/continuous-quality-improvement-quality-rating-and-improvement-systems-qris-learning-table/stateprofiles

http://qrisnetwork.org/lt/continuous-quality-improvement-quality-rating-and-improvement-systems-qris-learning-table/stateprofiles

http://www.ccw.org/index.php?option=com_content&task=view&id=189&Itemid=96

http://newamerica.net/publications/policy/stimulus_second_generation_qris

http://newamerica.net/publications/policy/stimulus_second_generation_qris

https://childcare.gov/sites/default/files/254_1302_qris_elements.pdf

http://www.fpg.unc.edu/node/227

http://www.nwlc.org/sites/default/files/pdfs/ACountforQualityQRISReport.pdf




125. Wiggins, K., and Mathias, D. (2013). “Continuous Quality Improvement: An Overview Report for State QRIS Leaders.” BUILD Initiative. http://qrisnetwork.org/sites/all/files/session/resources/Continuous%20Quality%20Improvement,%20An%20Overview%20Report%20for%20State%20QRIS%20Leaders.pdf.

126. No Child Left Behind Act of 2001; Education Week. (September 19, 2011). “No Child Left Behind.” http://www.edweek.org/ew/issues/no-child-left-behind/.

127. US Department of Education. “ESEA Flexibility Request.”

128. US Department of Education. (May 1, 2014). “ESEA Flexibility: States Approved for ESEA Flexibility and States with ESEA Flexibility Under Review.” http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html; US Department of Education. (2012). “The Opportunity of ESEA Flexibility.” http://www2.ed.gov/policy/elsec/guid/esea-flexibility/resources/esea-flex-brochure.pdf; McNeil, M. (August 20, 2013). “Pennsylvania Gets its No Child Left Behind Act Waiver, Just in Time.” Education Week, Politics K–12 blog, http://blogs.edweek.org/edweek/campaign-k-12/2013/08/pennsylvania_gets_its_no_child.html?cmp=ENL-EU-NEWS2. While the state of California did not apply, a group of districts in California did apply and receive a waiver. US Department of Education. (August 6, 2013). “Obama Administration Approves NCLB Waiver Requests for California CORE Districts.” http://www.ed.gov/news/press-releases/obama-administration-approves-nclb-waiver-request-california-core-districts.

129. McNeil, M. (August 29, 2013). “Arne Duncan Attaches More Strings to NCLB Waiver Renewals.” Education Week, Politics K–12 blog. http://blogs.edweek.org/edweek/campaign-k-12/2013/08/barring_reauthorization_this_m.html.

130. Klein, A. (December 4, 2013 ). “Two States Approved for ESEA Teacher Evaluation Extension Waiver.” Education Week, Politics K–12 blog. http://blogs.edweek.org/edweek/campaign-k-12/2013/12/two_states_approved_for_esea_t.html.

131. Duncan, A. (April 24, 2014). “Washington Extension Determination Letter.” US Department of Education, letter to Superintendent Randy Dorn. https://www2.ed.gov/policy/eseaflex/secretary-letters/wad6.html; Klein, A. (April 24, 2014). “Arne Duncan Revokes Washington State’s NCLB Waiver.” Education Week, Politics K–12 blog. http://blogs.edweek.org/edweek/campaign-k-12/2014/04/washington_state_loses_waiver_.html?cmp=ENL-EU-NEWS2.

132. As of February 2014, states have not reached 100% proficiency in math and reading as required under NCLB. US Department of Education. “ESEA Flexibility Request”; Guisbond, L. (February 3, 2014 ). “NCLB Crashed and Burned. When Will We Ever Learn?” Washington Post. http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/02/03/nclb-crashed-and-burned-when-will-we-ever-learn/; National Public Radio. (December 26, 2013). “Schools Should Be Examined in 2014.” http://www.npr.org/2013/12/26/257255808/future-series-examining-education.

133. The ESEA waiver process required states to address four principles: College- and Career-Ready Expectations for All Students; State-Developed Differentiated Recognition, Accountability, and Support; Supporting Effective Instruction and Leadership; and Reducing Duplication and Unnecessary Burden. US Department of Education. ( June 7, 2012). “ESEA Flexibility.” http://www.ed.gov/esea/flexibility/documents/esea-flexibility-acc.docThis paper addresses only the second principle, State-Developed Differentiated Recognition, Accountability, and Support, which focuses on district- and school-level accountability; it does not address the third principle, Supporting Effective Instruction and Leadership, which focuses heavily on personnel evaluation and individual accountability.

134. Council of Chief State School Officers. (2011). “Principles and Processes for State Leadership on Next-Generation Accountability Systems.” http://www.ccsso.org/Documents/2011/Principles%20and%20Processes%20for%20State%20Leadership%20on%20Next-Generation%20Accountability%20Systems%20%28Final%29%20%282%29.pdf.

135. Ibid.

136. Ujifusa. “All 50 States Now Using Testing”; Council of Chief State School Officers. “Principles and Processes.”

http://qrisnetwork.org/sites/all/files/session/resources/Continuous%20Quality%20Improvement,%20An%20Overview%20Report%20for%20State%20QRIS%20Leaders.pdf

http://qrisnetwork.org/sites/all/files/session/resources/Continuous%20Quality%20Improvement,%20An%20Overview%20Report%20for%20State%20QRIS%20Leaders.pdf

http://www.edweek.org/ew/issues/no-child-left-behind/

http://www2.ed.gov/policy/elsec/guid/esea-flexibility/resources/esea-flex-brochure.pdf

http://www2.ed.gov/policy/elsec/guid/esea-flexibility/resources/esea-flex-brochure.pdf

http://blogs.edweek.org/edweek/campaign-k-12/2013/08/pennsylvania_gets_its_no_child.html?cmp=ENL-EU-NEWS2

http://blogs.edweek.org/edweek/campaign-k-12/2013/08/pennsylvania_gets_its_no_child.html?cmp=ENL-EU-NEWS2

http://www.ed.gov/news/press-releases/obama-administration-approves-nclb-waiver-request-california-core-districts

http://www.ed.gov/news/press-releases/obama-administration-approves-nclb-waiver-request-california-core-districts

http://blogs.edweek.org/edweek/campaign-k-12/2013/08/barring_reauthorization_this_m.html

http://blogs.edweek.org/edweek/campaign-k-12/2013/08/barring_reauthorization_this_m.html

http://blogs.edweek.org/edweek/campaign-k-12/2013/12/two_states_approved_for_esea_t.html

http://blogs.edweek.org/edweek/campaign-k-12/2013/12/two_states_approved_for_esea_t.html

https://www2.ed.gov/policy/eseaflex/secretary-letters/wad6.html

http://blogs.edweek.org/edweek/campaign-k-12/2014/04/washington_state_loses_waiver_.html?cmp=ENL-EU-NEWS2

http://blogs.edweek.org/edweek/campaign-k-12/2014/04/washington_state_loses_waiver_.html?cmp=ENL-EU-NEWS2

http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/02/03/nclb-crashed-and-burned-when-will-we-ever-learn/

http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/02/03/nclb-crashed-and-burned-when-will-we-ever-learn/

http://www.npr.org/2013/12/26/257255808/future-series-examining-education

http://www.npr.org/2013/12/26/257255808/future-series-examining-education

http://www.ccsso.org/Documents/2011/Principles%20and%20Processes%20for%20State%20Leadership%20on%20Next-Generation%20Accountability%20Systems%20%28Final%29%20%282%29.pdf






137. No Child Left Behind Act of 2001.

138. Davidson, E., Reback, R., Rockoff, J., and Schwartz, H. (2013). “Fifty Ways to Leave a Child Behind: Idiosyncrasies and Discrepancies in States’ Implementation of NCLB.” http://www.columbia.edu/~ekd2110/Fifty_Ways_4_5_2013.pdf; No Child Left Behind Act of 2001.

139. Carey, K. (2007). “The Pangloss Index: How States Game the No Child Left Behind Act.” Education Sector. http://www.educationsector.org/sites/default/files/publications/The%20Pangloss%20Index.pdf. Regenstein, E., and Hubbard, A. (2013). “Early Learning at the Turbulent Dusk of the Pangloss Era. Ounce of Prevention Fund. http://www.ounceofprevention.org/national-policy/conversations/PanglossPaper-040913-version1-0.pdf.

140. Guisbond. “NCLB Crashed and Burned.”

141. Neal, D., and Schanzenbach, D. (2007). “Left Behind by Design: Proficiency Counts and Test-Based Accountability. National Bureau of Economic Research. Working Paper No. 13293. http://www.nber.org/papers/w13293.


143. Chubb, J., and Clark, C. (2013). “The New State Achievement Gap: How Federal Waivers Could Make It Worse—or Better,” 14. Education Sector. http://www.educationsector.org/sites/default/files/publications/NewStateAchieveGap-RELEASED.pdf; Ayers, J., and Owen, I. (2012). “No Child Left Behind Waivers: Promising Ideas for Second Round Applications,” 24. Center for American Progress. http://www.americanprogress.org/wp-content/uploads/issues/2012/07/pdf/nochildwaivers.pdf; Domaleski, C., and Perie, M. (2012). “Promoting Equity in State Education Accountability Systems.” National Center for the Improvement of Educational Assessment. http://www.nciea.org/publication_PDFs/Promoting%20Equity%20CSDMP110712.pdf; Mead, S. (February 13, 2012.). “ESEA Flexibility Waivers and Growth.” Education Week, Policy Notebook blog. http://blogs.edweek.org/edweek/sarameads_policy_notebook/2012/02/esea_flexibility_waivers_and_growth.html.


145. Riddle, W. (2012). “Major Accountability Themes of Second-Round State Applications for NCLB Waivers.” Center on Education Policy. http://www.cep-dc.org/publications/index.cfm?selectedYear=2012; Ayers and Owen. “No Child Left Behind Waivers.”

146. McNeil, M. (April 10, 2014). “Many States Left Key NCLB Flexibility on the Table.” Education Week. http://www.edweek.org/ew/articles/2014/04/10/28multiple.h33.html?cmp=ENL-EU-NEWS1; Polikoff, M., McEachin, A., Wrabel, S., and Duque, M. (2013). “The Waive of the Future? School Accountability in the Waiver Era.” In press at Educational Researcher. http://www-bcf.usc.edu/~polikoff/Waivers.pdf.


148. “While there are certainly examples of promising reforms being realized through the waiver process … the progress made by the [Department of Education’s] 2008 graduation rate regulations in holding schools accountable for how many students they actually graduate—the ultimate goal of K–12—may be threatened in several states.” Lovell et al. “The Effect of ESEA Waiver Plans”; Miller, G. (September 21, 2012). “Representative Miller Letter to Secretary of Education Arne Duncan.” http://democrats.edworkforce.house.gov/sites/democrats.edworkforce.house.gov/files/documents/112/pdf/letters/9-21-12_Accountability_ltr_to_Arne_Duncan.pdf.

149. Yettick, H. (February 20, 2014). “Analysis Finds NCLB Waivers Too Often Maintain Flawed Accountability Practices.” Education Week, Inside School Research blog. http://blogs.edweek.org/edweek/inside-school-research/2014/02/waivers_missed_opportunities.html?cmp=ENL-EU-NEWS2; Hall. “A Step Forward or a Step Back?” 5.

150. Chubb and Clark. “The New State Achievement Gap,” 15.

151. Partnership for Assessment of Readiness for Colleges and Careers. http://www.parcconline.org/; Smarter

http://www.columbia.edu/~ekd2110/Fifty_Ways_4_5_2013.pdf

http://www.educationsector.org/sites/default/files/publications/The%20Pangloss%20Index.pdf

http://www.ounceofprevention.org/national-policy/conversations/PanglossPaper-040913-version1-0.pdf

http://www.nber.org/papers/w13293

http://www.educationsector.org/sites/default/files/publications/NewStateAchieveGap-RELEASED.pdf

http://www.educationsector.org/sites/default/files/publications/NewStateAchieveGap-RELEASED.pdf

http://www.americanprogress.org/wp-content/uploads/issues/2012/07/pdf/nochildwaivers.pdf

http://www.americanprogress.org/wp-content/uploads/issues/2012/07/pdf/nochildwaivers.pdf

http://www.nciea.org/publication_PDFs/Promoting%20Equity%20CSDMP110712.pdf

http://blogs.edweek.org/edweek/sarameads_policy_notebook/2012/02/esea_flexibility_waivers_and_growth.html

http://blogs.edweek.org/edweek/sarameads_policy_notebook/2012/02/esea_flexibility_waivers_and_growth.html

http://www.cep-dc.org/publications/index.cfm?selectedYear=2012

http://www.edweek.org/ew/articles/2014/04/10/28multiple.h33.html?cmp=ENL-EU-NEWS1

http://www-bcf.usc.edu/~polikoff/Waivers.pdf

http://democrats.edworkforce.house.gov/sites/democrats.edworkforce.house.gov/files/documents/112/pdf/letters/9-21-12_Accountability_ltr_to_Arne_Duncan.pdf

http://democrats.edworkforce.house.gov/sites/democrats.edworkforce.house.gov/files/documents/112/pdf/letters/9-21-12_Accountability_ltr_to_Arne_Duncan.pdf

http://blogs.edweek.org/edweek/inside-school-research/2014/02/waivers_missed_opportunities.html?cmp=ENL-EU-NEWS2

http://blogs.edweek.org/edweek/inside-school-research/2014/02/waivers_missed_opportunities.html?cmp=ENL-EU-NEWS2

http://www.parcconline.org/




Balanced Assessment Consortium. http://www.smarterbalanced.org/.

152. Chingos, M. (2013). “Standardized Testing and the Common Core Standards: You Get What You Pay For? Brown Center on Education Policy, Brookings Institution. http://www.brookings.edu/research/reports/2013/10/30-standardized-testing-and-the-common-core-chingos; Educational Testing Service. (n.d.). “An Unprecedented Opportunity Demands a Unique Response.” https://www.ets.org/k12/commonassessments.

153. Educational Testing Service. “An Unprecedented Opportunity.”

154. Lu, A. ( January 24, 2014). “States Reconsider Common Core Tests.” Pew Charitable Trusts Stateline. http://www.pewstates.org/projects/stateline/headlines/states-reconsider-common-core-tests-85899535255; Ujifusa, A. (November 21, 2013 ). “Louisiana K–12 Chief Sets 10-Year Timeline for Common Core Accountability.” Education Week, State EdWatch blog. http://blogs.edweek.org/edweek/state_edwatch/2013/11/la_k-12_chief_announces_10-year_timeline_for_common-core_accountability.html.

155. US Department of Education. “ESEA Flexibility Request”; National Governors Association. (2006). “Graduation Counts: Compact and Task Force Report.” NGA Center for Best Practices. http://www.nga.org/files/live/sites/NGA/files/pdf/0602GRADGUIDANCE.PDF.

156. US Department of Education. “ESEA Flexibility.”

157. US Department of Education. “Title I—Improving the Academic Achievement of the Disadvantaged.” In Elementary and Secondary Education Act of 1965 (20 U.S.C. 6301 et seq.). http://www2.ed.gov/policy/elsec/leg/esea02/pg1.html.


159. A Tier I school is among the bottom of Title I schools in improvement, corrective action, or restructuring phase, or is a Title I high school that has graduation rates under 60% over several years. Tier II schools are the lowest 5% of secondary schools in the state that are eligible but do not receive Title I funds, or are high schools with consistently low graduation rates. US Department of Education. (November 1, 2010). “Guidance on Fiscal Year 2010 School Improvement Grants. Under 1003(g) of the Elementary and Secondary Education Act.” Office of Elementary and Secondary Education. https://ed.gov/programs/sif/sigguidance11012010.pdf.


161. Ibid.

162. Hyslop, A. (2013). “It’s All Relative: How NCLB Waivers Did—and Did Not—Transform School Accountability.” New America Foundation. http://newamerica.net/publications/policy/its_all_relative.

163. Polikoff, et al. “Waive of the Future?”

164. Riddle, W. (2012). “Major Accountability Themes of Approved State Applications for NCLB Waivers,” 8–9. Center on Education Policy. http://www.cep-dc.org/displayDocument.cfm?DocumentID=387; Riddle. “Major Accountability Themes of Second-Round State Applications for NCLB Waivers,” 11; Riddle, W., and Kober, N. (2012). “What Impact Will NCLB Waivers Have on the Consistency, Complexity, and Transparency of State Accountability Systems?” 12. Center on Education Policy. http://www.cep-dc.org/publications/index.cfm?selectedYear=2012.

165. Rouse, C., Hannaway, J., Goldhaber, D., and Figlio, D. (2007). “Feeling the Florida Heat? How Low Performing Schools Respond to Voucher and Accountability Pressure.” National Center for Analysis of Longitudinal Data in Education Research. http://static.excelined.org/wp-content/uploads/2007.-Feeling-the-FL-Heat-Natl.-Center-for-Analysis-of-Long.-Data-in-Ed-research.-Rouse.-Hannaway.-Goldhaber.-Figlio.pdf.

166. Polikoff et al. “Waive of the Future?”; Strauss, V. (November 23, 2013). “What Virginians Don’t Know About New A–F School Grading System.” Washington Post. http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/11/23/what-virginians-dont-know-about-new-a-f-school-grading-system/; DiCarlo. “Why Did Florida Schools’ Grades

http://www.smarterbalanced.org/

http://www.brookings.edu/research/reports/2013/10/30-standardized-testing-and-the-common-core-chingos

http://www.brookings.edu/research/reports/2013/10/30-standardized-testing-and-the-common-core-chingos

https://www.ets.org/k12/commonassessments

http://www.pewstates.org/projects/stateline/headlines/states-reconsider-common-core-tests-85899535255

http://blogs.edweek.org/edweek/state_edwatch/2013/11/la_k-12_chief_announces_10-year_timeline_for_common-core_accountability.html

http://blogs.edweek.org/edweek/state_edwatch/2013/11/la_k-12_chief_announces_10-year_timeline_for_common-core_accountability.html

http://www.nga.org/files/live/sites/NGA/files/pdf/0602GRADGUIDANCE.PDF

http://www2.ed.gov/policy/elsec/leg/esea02/pg1.html

http://www2.ed.gov/policy/elsec/leg/esea02/pg1.html

https://ed.gov/programs/sif/sigguidance11012010.pdf

http://newamerica.net/publications/policy/its_all_relative

http://www.cep-dc.org/displayDocument.cfm?DocumentID=387

http://www.cep-dc.org/publications/index.cfm?selectedYear=2012

http://static.excelined.org/wp-content/uploads/2007.-Feeling-the-FL-Heat-Natl.-Center-for-Analysis-of-Long.-Data-in-Ed-research.-Rouse.-Hannaway.-Goldhaber.-Figlio.pdf

http://static.excelined.org/wp-content/uploads/2007.-Feeling-the-FL-Heat-Natl.-Center-for-Analysis-of-Long.-Data-in-Ed-research.-Rouse.-Hannaway.-Goldhaber.-Figlio.pdf

http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/11/23/what-virginians-dont-know-about-new-a-f-school-grading-system/

http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/11/23/what-virginians-dont-know-about-new-a-f-school-grading-system/




Improve?”

167. Foundation for Excellence in Education. (2012). “Alabama A–F School Grading (HB 588).” http://excelined.org/downloads/school-grading-and-rewards-hb-588-2012/. Also see Foundation for Excellence in Education. (2011). “Indiana A–F School Grading (Rule Language, 511, IAC 6.2-6).” http://static.excelined.org/wp-content/uploads/A-F-School-Grading-Rule-Language-511-IAC-6.2-6.pdf; Foundation for Excellence in Education. (2012). “Ohio A–F School Grading (HB 555).” http://static.excelined.org/wp-content/uploads/A-F-grades-HB-555-2012.pdf.

168. Ayers and Owen. “No Child Left Behind Waivers,” 27.

169. US Department of Education. (February 10, 2012). “ESEA Flexibility: Review Guidance.” http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html.

170. Cody. “From School Grades to Common Core”; DiCarlo, M. (November 1, 2012). “The Structural Curve in Indiana’s New School Grading System.” Albert Shanker Institute, Shanker Blog. http://shankerblog.org/?p=7090; Cousins, C. (May 3, 2013). “Now That Maine Schools Have Been Graded, What Will Be Done?” Bangor Daily News. http://bangordailynews.com/2013/05/03/opinion/now-that-maine-schools-have-been-graded-what-will-be-done/; Education Trust-West. (2009). “Access Denied: 2009 API Rankings Reveal Unequal Access to California’s Best Schools.” Equity Alert. http://www.edtrust.org/sites/edtrust.org/files/publications/files/Access%20Denied.pdf; National Center for Education Statistics. (2013). “A First Look: 2013 Mathematics and Reading.” Institute of Education Sciences, US Department of Education. http://nces.ed.gov/nationsreportcard/subject/publications/main2013/pdf/2014451.pdf; Education Week. (2011). “Achievement Gap.” http://www.edweek.org/ew/issues/achievement-gap/.

171. Calkins, A., Guenther, W., Belfiore, G., and Lash, D. (2007). “The Turnaround Challenge.” Mass Insight Education & Research Institute. http://www.massinsight.org/publications/turnaround/51/file/1/pubs/2010/04/15/TheTurnaroundChallenge_MainReport.pdf.

172. Honig, M. (2013). “From Tinkering to Transformation: Strengthening School District Central Office Performance.” American Enterprise Institute. http://www.aei.org/outlook/education/k-12/system-reform/from-tinkering-to-transformation-strengthening-school-district-central-office-performance/; Mass Insight Education. “Being Bold.”

173. Calkins et al. “The Turnaround Challenge.”

174. US Department of Education. (November 1, 2010). “Guidance on Fiscal Year 2010 School Improvement Grants Under Section 1003(g) of the Elementary and Secondary Education Act of 1965.” http://www2.ed.gov/programs/sif/sigguidance11012010.pdf.


176. McNeil, M. ( January 14, 2014). “Caution Flags Waived for Six Waiver States.” Education Week. http://www.edweek.org/ew/articles/2014/01/15/17waiver.h33.html; Smarick, A. (November 19, 2013). “The Disappointing but Completely Predictable Results From SIG.” Thomas B. Fordham Institute, Flypaper blog. http://www.edexcellence.net/commentary/education-gadfly-daily/flypaper/2012/the-disappointing-but-completely-predictable-results-from-SIG.html; US Department of Education. (November 21, 2013). “School Improvement Grants Assessment Results: Cohorts 1 and 2.” http://www2.ed.gov/programs/sif/sigassessmentrpt-cohort1-2.pdf.


178. Klein, A. (May 6, 2013). “Rewards for Schools Key Facets of NCLB Waivers.” Education Week. http://www.edweek.org/ew/articles/2013/05/06/30rewards.h32.html?tkn=ORCCyi%2B56x%2BmtiOj5ON6%2BTUNK8%2BJNb4%2FH3Jj&cmp=clp-sb-ascd.

179. A call for an improved accountability framework was recently made by a consortium of 15 superintendents representing large suburban districts. The consortium proposed a framework in which learning is measured using multiple metrics that capture a broader range of student learning. Maxwell, L. (February 11, 2014). “Large-District

http://excelined.org/downloads/school-grading-and-rewards-hb-588-2012/

http://excelined.org/downloads/school-grading-and-rewards-hb-588-2012/

http://static.excelined.org/wp-content/uploads/A-F-School-Grading-Rule-Language-511-IAC-6.2-6.pdf

http://static.excelined.org/wp-content/uploads/A-F-School-Grading-Rule-Language-511-IAC-6.2-6.pdf

http://static.excelined.org/wp-content/uploads/A-F-grades-HB-555-2012.pdf

http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html


http://bangordailynews.com/2013/05/03/opinion/now-that-maine-schools-have-been-graded-what-will-be-done/

http://bangordailynews.com/2013/05/03/opinion/now-that-maine-schools-have-been-graded-what-will-be-done/

http://www.edtrust.org/sites/edtrust.org/files/publications/files/Access%20Denied.pdf

http://nces.ed.gov/nationsreportcard/subject/publications/main2013/pdf/2014451.pdf

http://nces.ed.gov/nationsreportcard/subject/publications/main2013/pdf/2014451.pdf

http://www.edweek.org/ew/issues/achievement-gap/

http://www.massinsight.org/publications/turnaround/51/file/1/pubs/2010/04/15/TheTurnaroundChallenge_MainReport.pdf

http://www.massinsight.org/publications/turnaround/51/file/1/pubs/2010/04/15/TheTurnaroundChallenge_MainReport.pdf

http://www.aei.org/outlook/education/k-12/system-reform/from-tinkering-to-transformation-strengthening-school-district-central-office-performance/

http://www.aei.org/outlook/education/k-12/system-reform/from-tinkering-to-transformation-strengthening-school-district-central-office-performance/

http://www2.ed.gov/programs/sif/sigguidance11012010.pdf

http://www2.ed.gov/programs/sif/sigguidance11012010.pdf

http://www.edweek.org/ew/articles/2014/01/15/17waiver.h33.html

http://www.edexcellence.net/commentary/education-gadfly-daily/flypaper/2012/the-disappointing-but-completely-predictable-results-from-SIG.html

http://www.edexcellence.net/commentary/education-gadfly-daily/flypaper/2012/the-disappointing-but-completely-predictable-results-from-SIG.html

http://www2.ed.gov/programs/sif/sigassessmentrpt-cohort1-2.pdf

http://www2.ed.gov/programs/sif/sigassessmentrpt-cohort1-2.pdf

http://www.edweek.org/ew/articles/2013/05/06/30rewards.h32.html?tkn=ORCCyi%2B56x%2BmtiOj5ON6%2BTUNK8%2BJNb4%2FH3Jj&cmp=clp-sb-ascd






Superintendents Lay Out Their Vision for ESEA Renewal.” Education Week, District Dossier blog. http://blogs.edweek.org/edweek/District_Dossier/2014/02/large_district_superintendents.html?r=159485657&cmp=ENL-EU-NEWS2&preview=1.

180. For example, the Oakland, California, Unified School District set comprehensive school-quality metrics for itself that included ensuring thriving students and healthy communities; quality learning experiences for all students; safe, supportive and healthy learning environments; learning communities focused on continuous improvement; meaningful student, family, and community engagement and partnerships; effective school leadership and resource management; and a high quality central office that is in service of quality schools. Its rationale for including these broader metrics was in response to two central concerns: (1) Current accountability systems that focus on high-stakes testing give little incentive for schools to improve and (2) Limited information exists on the effectiveness of district supports and services. The district sought to build a transparent system of accountability that provided a clear understanding of how effective schools and systems work. Oakland Unified School District. (2011). “Community Schools, Thriving Students: A Five-Year Strategic Plan.” http://www.thrivingstudents.org/sites/default/files/Community-Schools-Thriving-Students-Strategic-Plan.pdf.

181. Petrilli. “The Problem with Proficiency.”

182. New York State Education Department. (2013). “Diagnostic Tool for Effectiveness.” Office of Accountability. http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/home.html; Hurlburt, S., Therriault, S., and Le Floch, K. (2010). “School Improvement Grants: Analyses of State Applications and Eligible and Awarded Schools.” US Department of Education, National Center for Education Evaluation and Regional Assistance. http://ies.ed.gov/ncee/pubs/20124060/pdf/20124060.pdf.

183. “NCLB shined a bright light on the underperformance of low-income students and students of color, and it directed energy and attention to doing right by them.” Ushomirsky, N., Hall, D., and Haycock, K. (September 13, 2011). “Getting It Right: Crafting Federal Accountability for Higher Student Performance and a Strong America,” 2. Education Trust. http://www.edtrust.org/dc/publication/getting-it-right-crafting-federal-accountability-for-higher-student-performance-and-a.

184. Hall. “A Step Forward or a Step Back?” 4–5.

185. For example, Sam Chaltain argues that we should also measure social and emotional skills and creativity. Chaltain, S. (August 27, 2013). “How Should We Evaluate Our Preschools?” Education Week, Of, By, For: In Search of the Civic Mission of K–12 Schools blog, http://blogs.edweek.org/edweek/civic_mission/2013/08/how_should_we_evaluate_our_preschools.html. Martha Zaslow, Director of the Office for Policy and Communications of the Society for Research in Child Development and a Senior Scholar at Child Trends, noted the importance of thinking beyond child outcomes to include the professionalization of the field, family engagement, and systems building. Zaslow, M. “Thinking of QRIS.” ( January 6, 2014). Presentation at National Women’s Law Center National and State Child Care Advocates Meeting. For K–12, the Large Countrywide and Suburban District Consortium recommends measuring college- and career-readiness skills, which includes a mastery of deep content knowledge, dual-language ability and fluency, higher-order thinking, and the possession of noncognitive competencies, such as perseverance, self-regulation, and confidence. Large Countywide and Suburban District Consortium. “21st Century Education Accountability.”

186. For example, Achieve Inc. recommends that accountability systems include metrics focused on college and career readiness: obtaining a rigorously defined diploma, assessment scores that indicate college and career readiness, earning college credit while in high school, and minimizing postsecondary remediation rates. Achieve Inc. “Closing the Expectations Gap: 2013 Annual Report,” 27. http://www.achieve.org/files/2013ClosingtheExpectationsGapReport.pdf. These metrics could be integrated into state accountability for high schools as part of an overall continuum, in combination with a focus on professional practice.

187. Twenty states participating in the QRIS require the use of the Environment Rating Scale. Also, as more states develop and establish comprehensive data systems, the use or planned use of the Classroom Assessment and Scoring System (CLASS) into a QRIS or a Race to the Top–Early Learning Challenge application has expanded. Berlin, R. (2012). “Use of CLASS in Quality Rating and Improvement Systems.” http://qrisnetwork.org/

http://blogs.edweek.org/edweek/District_Dossier/2014/02/large_district_superintendents.html?r=159485657&cmp=ENL-EU-NEWS2&preview=1



http://www.thrivingstudents.org/sites/default/files/Community-Schools-Thriving-Students-Strategic-Plan.pdf

http://www.thrivingstudents.org/sites/default/files/Community-Schools-Thriving-Students-Strategic-Plan.pdf

http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/home.html

http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/home.html

http://ies.ed.gov/ncee/pubs/20124060/pdf/20124060.pdf

http://www.edtrust.org/dc/publication/getting-it-right-crafting-federal-accountability-for-higher-student-performance-and-a

http://www.edtrust.org/dc/publication/getting-it-right-crafting-federal-accountability-for-higher-student-performance-and-a

http://blogs.edweek.org/edweek/civic_mission/2013/08/how_should_we_evaluate_our_preschools.html

http://blogs.edweek.org/edweek/civic_mission/2013/08/how_should_we_evaluate_our_preschools.html

http://www.achieve.org/files/2013ClosingtheExpectationsGapReport.pdf

http://www.achieve.org/files/2013ClosingtheExpectationsGapReport.pdf

http://qrisnetwork.org/resource/2012/use-class%E2%84%A2-quality-rating-and-improvement-systems




resource/2012/use-class%E2%84%A2-quality-rating-and-improvement-systems; National Center for Child Care Quality Improvement. “QRIS Elements”; Tout. “Child Care Quality Rating System Assessment.”

188. Corbett, J. (2011). “Lead Turnaround Partners.” Center on Innovation & Improvement. http://www.adi.org/about/downloads/LeadPartners.pdf; Kutash, J., Nico, E., Gorin, E., Rahmatullah, S., and Tallant, K. (2010). “The School Turnaround Field Guide.” Wallace Foundation. http://www.wallacefoundation.org/pages/turnaround-actors-school-turnaround-field-guide.aspx.

189. Hussain. “The School Inspector Calls,” 66; Rothstein, R. (2009). “What’s Wrong With Accountability by the Numbers?” American Educator, American Federation of Teachers. http://www.aft.org/pdfs/americaneducator/spring2009/ROTHSTEIN%282%29.pdf; Dutch Education Inspectorate, Ministry of Education, Culture and Science. “The Dutch Inspectorate of Education.”

190. National Research Council of the National Academies. (2008). Early Childhood Assessments: Why, What, How. Snow, C., and Van Hemel, S. (eds.). Washington, DC: National Academies Press. http://www.nap.edu/openbook.php?record_id=12446; Shepard et al. “Principles and Recommendations for Early Childhood Assessments.”


192. Oklahoma Center for Education Policy and Center for Educational Research and Evaluation. (2013). “Oklahoma School Grades: Hiding ‘Poor’ Achievement.” http://blogs.edweek.org/edweek/state_edwatch/Oklahoma%20A-F%20--%20Hiding%20Poor%20Achievement%20.pdf; Biddle, R. (2013). “The Tony Bennett Letter Grade Fiasco: Another Reason Why Accountability Matters.” Dropout Nation. http://dropoutnation.net/2013/07/31/the-tony-bennett-letter-grade-fiasco-is-a-lesson-about-accountability-for-reformers-everywhere/; Ruddock, N. ( January 4, 2013.) “A–F School Grades Will Result in Greater Accountability and Parent Involvement.” Foundation for Excellence in Education, EdFly Blog. http://excelined.org/2013/01/a-f-school-grades-will-result-in-greater-accountability-and-parental-involvement/.

193. National Center on Child Care Quality Improvement. “QRIS Financial Incentives.”


195. National Center on Child Care Quality Improvement. “Overview of the QRIS Resource Guide”; Zellman, G., and Perlman, M. (2010). “Child Care Quality Rating and Improvement Systems in Five Pioneer States: Implementation Issues and Lessons Learned.” RAND. http://www.rand.org/content/dam/rand/pubs/monographs/2008/RAND_MG795.pdf.

196. Commonwealth of Virginia. (November 21, 2013). “State Board of Education Approves A–F Grading System.” Press release. Office of Governor Bob McDonnell. http://www.doe.virginia.gov/news/news_releases/2013/nov21_gov.shtml; Ruddock. “A–F School Grades.”

197. Council of Chief State School Officers. (2013). “School Choice in the States: A Policy Landscape.” http://www.ccsso.org/Documents/2013/Choice_by_State_2013.pdf; Education Commission of the States (ECS). (2013). “ECS State Policy Database Choice of Schools.” http://www.ecs.org/ecs/ecscat.nsf/WebTopicView?OpenView&count=-1&RestrictToCategory=Choice+of+Schools--Choice/Open+Enrollment.

198. Forry, N., Tout, K., Rothenberg, L., Sandstrom, H., and Vesely, C. (2013). “Child Care Decision-Making Literature Review.” OPRE Brief 2013-45. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, US Department of Health and Human Services. http://www.acf.hhs.gov/sites/default/files/opre/child_care_decision_making_literature_review_pdf_version_v2.pdf.

199. Sandstrom, H., Giesen, L., and Chaudry, A. (2012). “How Contextual Constraints Affect Low-Income Working Parents’ Child Care Choices.” Urban Institute, Brief 22. http://www.urban.org/UploadedPDF/412511-How-Contextual-Constraints-Affect-Low-Income-Working-Parents-Child-Care-Choices.pdf.

200. Petrilli, M. ( July 31, 2013 ). “The Tony Bennett Flap: School Grades, Stakes, and Signals.” Thomas B. Fordham

http://qrisnetwork.org/resource/2012/use-class%E2%84%A2-quality-rating-and-improvement-systems

http://www.adi.org/about/downloads/LeadPartners.pdf

http://www.adi.org/about/downloads/LeadPartners.pdf

http://www.wallacefoundation.org/pages/turnaround-actors-school-turnaround-field-guide.aspx

http://www.aft.org/pdfs/americaneducator/spring2009/ROTHSTEIN%282%29.pdf



http://blogs.edweek.org/edweek/state_edwatch/Oklahoma%20A-F%20--%20Hiding%20Poor%20Achievement%20.pdf

http://blogs.edweek.org/edweek/state_edwatch/Oklahoma%20A-F%20--%20Hiding%20Poor%20Achievement%20.pdf

http://dropoutnation.net/2013/07/31/the-tony-bennett-letter-grade-fiasco-is-a-lesson-about-accountability-for-reformers-everywhere/

http://dropoutnation.net/2013/07/31/the-tony-bennett-letter-grade-fiasco-is-a-lesson-about-accountability-for-reformers-everywhere/

http://excelined.org/2013/01/a-f-school-grades-will-result-in-greater-accountability-and-parental-involvement/

http://excelined.org/2013/01/a-f-school-grades-will-result-in-greater-accountability-and-parental-involvement/



http://www.doe.virginia.gov/news/news_releases/2013/nov21_gov.shtml

http://www.ccsso.org/Documents/2013/Choice_by_State_2013.pdf

http://www.ccsso.org/Documents/2013/Choice_by_State_2013.pdf

http://www.ecs.org/ecs/ecscat.nsf/WebTopicView?OpenView&count=-1&RestrictToCategory=Choice+of+Schools--Choice/Open+Enrollment

http://www.ecs.org/ecs/ecscat.nsf/WebTopicView?OpenView&count=-1&RestrictToCategory=Choice+of+Schools--Choice/Open+Enrollment

http://www.acf.hhs.gov/sites/default/files/opre/child_care_decision_making_literature_review_pdf_version_v2.pdf

http://www.acf.hhs.gov/sites/default/files/opre/child_care_decision_making_literature_review_pdf_version_v2.pdf

http://www.urban.org/UploadedPDF/412511-How-Contextual-Constraints-Affect-Low-Income-Working-Parents-Child-Care-Choices.pdf

http://www.urban.org/UploadedPDF/412511-How-Contextual-Constraints-Affect-Low-Income-Working-Parents-Child-Care-Choices.pdf




Institute, Flypaper blog. http://educationnext.org/the-tony-bennett-flap-school-grades-stakes-and-signals/.

201. Parent information about schools is often tied to parental choice. For example, a study on parents from the Charlotte-Mecklenburg, North Carolina, area found 16% of parents given notifications of school proficiency scores responded by choosing a school different from their NCLB school. Hastings, J., and Weinstein, J. (2007). “No Child Left Behind: Estimating the Impact on Choices and Student Outcomes.” http://www.econ.yale.edu/~jh529/papers/Hastings&Weinstein_NCLB.pdf; Schneider, J., and Nathan, A. (October 29, 2013). “How to Measure School Quality.” Education Week. http://www.edweek.org/ew/articles/2013/10/30/10schneider.h33.html?tkn=YZWFoq6bh0sq9GNpASXtdfCuRWFbSU%2BBWydo&cmp=ENL-EU-NEWS2; RAND. (2009). “Increasing Participation in No Child Left Behind School Choice.” http://www.rand.org/content/dam/rand/pubs/research_briefs/2009/RAND_RB9424.pdf.

202. Cohen, J. ( January 16, 2013). “Using Turnaround as an Entry Point to Forming a Portfolio District.” Michael & Susan Dell Foundation blog. http://www.msdf.org/blog/2013/01/turnaround-schools-as-an-entry-point-to-forming-a-portfolio-district/; Mass Insight Education. “Being Bold.”

203. Achieve Inc. “Closing the Expectations Gap.”

204. In Florida, preschools are held accountable for the performance of their students on kindergarten entry assessments. Florida Department of Education. (November 8, 2013). “Kindergarten Assessment: History and Legislative Authority.” Office of Early Learning. http://www.floridaearlylearning.com/sites/www/Uploads/HistoryOfKindergartenAssessmentInFlorida.pdf. Without taking a position on the propriety of holding one provider accountable for the performance of graduates enrolled with a subsequent provider, we disagree with Florida’s approach because of its inappropriate use of kindergarten assessment as an accountability measure. See Meisels. “Accountability in Early Childhood.”

205. Stoney, L. (2012). “Unlocking the Potential of QRIS: Trends and Opportunities in the Race to the Top–Early Learning Challenge Applications.” QRIS National Learning Network. http://www.qrisnetwork.org/sites/all/files/resources/gscobb/2012-03-07%2008:29/LouiseStoneyMemo.pdf; Johnson-Staub. “Meeting the Early Learning Challenge.”

206. Tout, K., and Starr, R. (2013). “Key Elements of a QRIS Validation Plan: Guidance and Planning Template.” Office of Planning, Research, and Evaluation, Administration for Children and Families. http://www.acf.hhs.gov/sites/default/files/opre/key_elements_of_a_qris_validation_plan_final_2_21_13.pdf.

207. Schwartz, H., Hamilton, L., Stecher, B., and Steele, J. (2011). “Expanded Measures of School Performance.” RAND. http://www.rand.org/content/dam/rand/pubs/technical_reports/2011/RAND_TR968.pdf; Petrides, L., and Nodine, T. (2005). “Anatomy of School System Improvement: Performance-Driven Practices in Urban School Districts.” New Schools Venture Fund. http://www.newschools.org/files/District_Performance_Practices.pdf.

208. National Center for Child Care Quality Improvement. “QRIS Elements”; Smith, S., Robbins, T., Stagman, S., and Kreader, J. (2012). “Practices for Promoting Young Children’s Learning in QRIS Standards.”

209. “An SEA may also implement interventions aligned with the turnaround principles as part of a statewide school turnaround strategy that allows for State takeover of schools or for transferring operational control of the school to another entity such as a recovery school district or other management organization.” US Department of Education. “ESEA Flexibility Request”; Hyslop, A. (2013). “It’s All Relative: How NCLB Waivers Did—and Did Not—Transform School Accountability.” New America Foundation; Smith, N. (April 2013). “Redefining the School District in Tennessee.” Thomas B. Fordham Institute. http://www.edexcellence.net/sites/default/files/publication/pdfs/20130423-Redefining-the-School-District-in-Tennessee-FINAL_7.pdf.

210. National Child Care Information Center. (2011). “Overview of the QRIS Resource Guide.” http://qrisnetwork.org/sites/all/files/resources/gscobb/2011-12-15%2006:10/QRISResourceGuide.pdf; National Center on Child Care Quality Improvement. (n.d.). “QRIS Resource Guide: Initial Design.” US Department of Health and Human Services, Administration on Children and Families. https://occqrisguide.icfwebservices.com/index.cfm?do=question&sid=1&qid=159. Some states require participation in QRIS in ways that automatically place licensed providers at the first level (Iowa, Maine, Maryland, Montana, New Mexico, Oklahoma, Rhode Island, and

http://educationnext.org/the-tony-bennett-flap-school-grades-stakes-and-signals/

http://www.econ.yale.edu/~jh529/papers/Hastings&Weinstein_NCLB.pdf

http://www.econ.yale.edu/~jh529/papers/Hastings&Weinstein_NCLB.pdf

http://www.edweek.org/ew/articles/2013/10/30/10schneider.h33.html?tkn=YZWFoq6bh0sq9GNpASXtdfCuRWFbSU%2BBWydo&cmp=ENL-EU-NEWS2

http://www.edweek.org/ew/articles/2013/10/30/10schneider.h33.html?tkn=YZWFoq6bh0sq9GNpASXtdfCuRWFbSU%2BBWydo&cmp=ENL-EU-NEWS2

http://www.rand.org/content/dam/rand/pubs/research_briefs/2009/RAND_RB9424.pdf

http://www.rand.org/content/dam/rand/pubs/research_briefs/2009/RAND_RB9424.pdf

http://www.msdf.org/blog/2013/01/turnaround-schools-as-an-entry-point-to-forming-a-portfolio-district/

http://www.msdf.org/blog/2013/01/turnaround-schools-as-an-entry-point-to-forming-a-portfolio-district/

http://www.floridaearlylearning.com/sites/www/Uploads/HistoryOfKindergartenAssessmentInFlorida.pdf

http://www.qrisnetwork.org/sites/all/files/resources/gscobb/2012-03-07%2008:29/LouiseStoneyMemo.pdf

http://www.qrisnetwork.org/sites/all/files/resources/gscobb/2012-03-07%2008:29/LouiseStoneyMemo.pdf

http://www.acf.hhs.gov/sites/default/files/opre/key_elements_of_a_qris_validation_plan_final_2_21_13.pdf

http://www.acf.hhs.gov/sites/default/files/opre/key_elements_of_a_qris_validation_plan_final_2_21_13.pdf

http://www.rand.org/content/dam/rand/pubs/technical_reports/2011/RAND_TR968.pdf

http://www.newschools.org/files/District_Performance_Practices.pdf

http://www.edexcellence.net/sites/default/files/publication/pdfs/20130423-Redefining-the-School-District-in-Tennessee-FINAL_7.pdf

http://www.edexcellence.net/sites/default/files/publication/pdfs/20130423-Redefining-the-School-District-in-Tennessee-FINAL_7.pdf








Tennessee) or require QRIS in order to receive public funding (Maine, New Mexico, North Carolina, Oklahoma, and Wisconsin). Vermont’s pre-kindergarten programs are required to be licensed and participate in QRIS, and Maine’s Head Start programs are also required to join the state’s QRIS.

211. As Matthew DiCarlo of the Albert Shanker Institute has written, “You shouldn’t want to radically restructure a school that is compelling strong growth from its students, even if those students still score lower than their peers elsewhere.” DiCarlo, M. (August 5, 2013). “Under the Hood of School Rating Systems.” Albert Shanker Institute, Shanker Blog. http://shankerblog.org/?p=8695.

212. States are already trying to assess what they can accomplish through turnaround efforts. Better information should help them improve this process. Cohen, J., and Segal, A. (2014). “The State of the State: New SEA Structures for a New Approach to Turnaround.” In Morando Rhim, L., and Redding, S. (eds.). (2014). The State Role in School Turnaround. San Francisco: WestEd. http://centeronschoolturnaround.org/staterole/.

213. DeWitt, P. (May 7, 2013). “Can Michael Fullan Save California From NCLB?” Education Week, Finding Common Ground blog. http://blogs.edweek.org/edweek/finding_common_ground/2013/05/can_michael_fullan_save_california_from_nclb.html; DeWitt, P. (September 3, 2011). “Impressive Empathy: An Interview With Michael Fullan.” Education Week, Finding Common Ground blog. http://blogs.edweek.org/edweek/finding_common_ground/2011/09/impressive_empathy_an_interview_with_michael_fullan.html; Fullan. “Choosing the Wrong Drivers,” 8–9.

214. McAlister, S. (2013). “Why Community Engagement Matters in School Turnaround.” Voices in Urban Education: Annenberg Institute for School Reform at Brown University. http://vue.annenberginstitute.org/issues/36/why-community-engagement; National Education Association. (2008). “Parent, Family, Community Involvement in Schools.” http://www.nea.org/assets/docs/PB11_ParentInvolvement08.pdf.

215. Regenstein and Lipper. “A Framework for Choosing.”

216. Education Commission of the States. (n.d.) “P–16/P–20 Councils.” http://www.ecs.org/html/educationissues/HighSchool/highschooldb1_intro.asp?topic=p-20.

217. “[O]ur system for handling state accountability and grading systems is amateurish and half-baked.” Hess, F. (September 9, 2013). “Tony Bennett Exonerated in Indiana Grading Furor.” Education Week, Rick Hess Straight Up blog. http://blogs.edweek.org/edweek/rick_hess_straight_up/2013/09/tony_bennett_exonerated.html.

218. National Early Childhood Accountability Task Force. “Taking Stock”; Lynch, R. (2005). “Early Childhood Investment Yields Big Payoff.” Policy Perspectives, West Ed. http://www.wested.org/online_pubs/pp-05-02.pdf.

219. Alliance for Excellent Education. (2013). “Saving Futures, Saving Dollars: The Impact of Education on Crime Reduction and Earnings.” http://all4ed.org/wp-content/uploads/2013/09/SavingFutures.pdf; Tucci, T. ( June 2011). “Education and the Economy: Boosting State and Local Economies by Improving High School Graduation Rates.” Alliance for Excellent Education, Issue Brief. http://all4ed.org/wp-content/uploads/EdEconomy_seb_leb.pdf; Messacar, D., and Oreopoulos, P. (2012). “Staying in School: A Proposal to Raise High School Graduation Rates.” Hamilton Project, Brookings Institution. http://www.brookings.edu/~/media/research/files/papers/2012/9/27%20compulsory%20schooling/thp_messacaroreopoulos_compschool_discpaper.pdf .

220. For the first wave of Early Learning Challenge grants, “grantees must complete all activities outlined in their approved applications and obligate all RTT–ELC funds by the end of the four-year grant period that began on January 1, 2012.” US Department of Education. (May 12, 2012). “RTT-ELC FAQs Combined.” http://www2.ed.gov/programs/racetothetop-earlylearningchallenge/faq.html.

221. Sabol et al. “Can Rating Pre-K Programs Predict Children’s Learning?”; Willingham, D. (August 26, 2013). “Preschool Quality Can Be Measured, but States Aren’t Getting It Right.” Daniel Willingham Science and Education Blog. http://www.danielwillingham.com/1/archives/08-2013/1.html.


http://centeronschoolturnaround.org/staterole/

http://blogs.edweek.org/edweek/finding_common_ground/2013/05/can_michael_fullan_save_california_from_nclb.html

http://blogs.edweek.org/edweek/finding_common_ground/2013/05/can_michael_fullan_save_california_from_nclb.html

http://blogs.edweek.org/edweek/finding_common_ground/2011/09/impressive_empathy_an_interview_with_michael_fullan.html

http://blogs.edweek.org/edweek/finding_common_ground/2011/09/impressive_empathy_an_interview_with_michael_fullan.html

http://vue.annenberginstitute.org/issues/36/why-community-engagement

http://vue.annenberginstitute.org/issues/36/why-community-engagement

http://www.nea.org/assets/docs/PB11_ParentInvolvement08.pdf

http://www.ecs.org/html/educationissues/HighSchool/highschooldb1_intro.asp?topic=p-20

http://blogs.edweek.org/edweek/rick_hess_straight_up/2013/09/tony_bennett_exonerated.html

http://www.wested.org/online_pubs/pp-05-02.pdf

http://all4ed.org/wp-content/uploads/2013/09/SavingFutures.pdf

http://all4ed.org/wp-content/uploads/EdEconomy_seb_leb.pdf

http://all4ed.org/wp-content/uploads/EdEconomy_seb_leb.pdf

http://www.brookings.edu/~/media/research/files/papers/2012/9/27%20compulsory%20schooling/thp_messacaroreopoulos_compschool_discpaper.pdf

http://www.brookings.edu/~/media/research/files/papers/2012/9/27%20compulsory%20schooling/thp_messacaroreopoulos_compschool_discpaper.pdf

http://www2.ed.gov/programs/racetothetop-earlylearningchallenge/faq.html

http://www.danielwillingham.com/1/archives/08-2013/1.html