ccsso criteria for procuring and evaluating high-quality ... · and evaluating high-quality...
TRANSCRIPT
CCSSO Criteria for Procuring and
Evaluating High-Quality Assessments –
Effective Uses by States
Prepared by Student Achievement Partners for NCSA 2016
PAGE 2
Learning Objectives
• Participants will have a high-level
understanding of the
ELA/literacy and Mathematics
alignment criteria (Sections B
and C) from the CCSSO “Criteria
for Procuring and Evaluating
High-Quality Assessments.”
• Participants will learn about
specific ways states have
incorporated the Criteria into
their assessment programs.
PAGE 3
Sections of the Criteria for Procuring and
Evaluating High-Quality Assessments
A. Meet Overall Assessment Goals and Ensure
Technical Quality
B. Align to Standards – English Language
Arts/Literacy
C. Align to Standards – Mathematics
D. Yield Valuable Reports on Student Progress and
Performance
E. Adhere to Best Practices in Test Administration
F. State Specific Criteria (as desired)
http://www.ccsso.org/Documents/2014/CCSSO%20Criteria%20for%20High%20Quality%20Assessment
s%2003242014.pdf
PAGE 4
Section B. Align to Standards – ELA/Literacy
Criterion B.1: Assessing student reading and writing
achievement in both ELA and literacy
Criterion B.2: Focusing on complexity of texts
Criterion B.3: Requiring students to read closely and use
evidence from texts
Criterion B.4: Requiring a range of cognitive demand
Criterion B.5: Assessing writing
Criterion B.6: Emphasizing vocabulary and language skills
Criterion B.7: Assessing research and inquiry
Criterion B.8: Assessing speaking and listening
Criterion B.9: Ensuring high-quality items and a variety of
item types
PAGE 5
Format of the CCSSO Criteria for Procuring
and Evaluating High-Quality Assessments
B.5 Assessing writing:
Assessments
emphasize writing
tasks that require
students to engage
in close reading
and analysis of
texts so that
students can
demonstrate
college- and
career-ready
abilities.
Test blueprints and other specifications as well as exemplar test
items for each grade level are provided, demonstrating the
expectations below are met.
Writing tasks reflect the types of writing that will prepare students
for the work required in college and the workplace, balancing
expository, persuasive/argument, and narrative writing, as state
standards require. At higher grade levels, the balance shifts towards
more exposition and argument.
For example, for common core aligned assessments, goals include:
o Taking all forms of the test together, writing tasks are
approximately one-third each exposition, argument, and
narrative (some tasks may represent blended structures), with
the balance shifting towards more exposition and argument
at the higher grade levels.
Tasks (including narrative tasks) require students to confront text or
other stimuli directly, to draw on textual evidence, and to support
valid inferences from text or stimuli.
PAGE 6
Criterion B.1
Assessing student reading and writing
achievement in both ELA and literacy:
The assessments are English language arts and
literacy tests that are based on an aligned balance of
high-quality literary and informational texts.
Key phrase: “aligned balance”
Key phrase: “high-quality”
PAGE 7
Aligned Balance of High-Quality Texts
Aligned Balance:
• CCR standards, and thus, CCR assessments, call for
increased emphasis on informational text as
students move through the grade levels so they will
be prepared for the kinds of texts most often
encountered in college and the workplace.
High Quality:
• Texts should be worthy of students’ time and
attention. Well-crafted texts support the kind of
close reading and deep analysis required by CCR
standards. The evidence descriptors for B.1 speak
to texts being either previously published or of
publishable quality.
PAGE 8
B.1: Informational Text Quality
Non-CCR Text CCR-Aligned Text
Are you tired and sleepy when you wake up in the morning? Do you think your school starts far too early in the morning for you to learn effectively? What would you say if you learned that researchers think the same thing?It’s true. Recent research has shown that teenagers benefit when high school starts later in the day. Researchers have found that teenagers, an age group more likely to stay awake late into the night, benefit when school starts at a time allowing them to sleep later in the morning.
…To help sleepy teens, some school districts have tried delaying the opening of the high school day. Educational researcher Kyla Wahlstrom, from the University of Minnesota, has been following districts that changed their start times, tracking the effect on schools and students. The Minneapolis school district, for example, changed its start time from 7:20 to 8:40 a.m., giving its 12,000 high schoolers an extra hour and twenty minutes each morning. Wahlstrom says the students have benefited from the change…
“What Time Should High Schools Start?” Commissioned for Assessment
“High Schools Staring Later to Help Sleepy Teens’ by Michelle Trudeau (g. 7 text)
PAGE 9
Criterion B.2
Focusing on complexity of texts:
The assessments require appropriate levels of text
complexity; they raise the bar for text complexity
each year so students are ready for the demands of
college- and career-level reading no later than the end
of high school. Multiple forms of authentic, previously
published texts are assessed, including written,
audio, visual, and graphic, as technology and
assessment constraints permit.
Key phrase: “appropriate levels”
Key phrase: “raise the bar”
PAGE 10
Emphasis on Text Complexity
Appropriate Levels:
• The evidence descriptors for this criterion outline
an expectation for a process of determining text
complexity, using both quantitative and qualitative
tools and transparency around the associated
results.
Raise the Bar:
• The evidence descriptors note that texts should
increase in complexity as students move through
the grades, meeting college- and career- readiness
standards by the end of high school.
PAGE 11
Criterion B.3
Requiring students to read closely and use
evidence from texts:
Reading assessments consist of test questions or
tasks, as appropriate, that demand that students read
carefully and deeply and use specific evidence from
increasingly complex texts to obtain and defend
correct responses.
Key phrase: “carefully and deeply”
Key phrase: “use specific evidence”
PAGE 12
Reading Closely and Using Evidence
Reading Carefully and Deeply:
• The evidence descriptors for B.3 detail that test
questions should arise from and require close
reading and analysis of text and focus on the
central ideas and important particulars of the text.
• Also, the criterion addresses the need for questions
to assess the depth and specific requirements
delineated in the standards at each grade level.
PAGE 13
Reading Closely and Using Evidence
Using Specific Evidence:
• The evidence descriptors call for many test
questions to require students to directly point to
textual evidence in support of a claim or inference.
This aligns to research-based CCR standards, as the
ability to develop claims and support them with
evidence from text(s) is crucial for college and
career readiness.
PAGE 14
B.3: Traditional vs. CCR Item
Traditional CCR
Click on the sentence in paragraph 19 that the author uses to show what Gram thinks about Old Faithful.
Paragraph 19 included here for reference:
More steam, boiling and hissing, and a huge jing-bang spray of water surged out, climbing and climbing, and then more and more, until it looked like a whole river of water was shooting straight up into the air. “It looks like an upsidey-down waterfall!” Gram said. All the while there was a walloping hissing, and I could have sworn the ground rumbled and trembled underneath us. The warm mist blew toward us and people started backing away.
CA: “It looks like an upsidey-down waterfall!” Gram said.
When the family first arrives at Old Faithful, Sal says, “I was afraid Gram was going to be disappointed because it didn’t look like much at first.” Circle three paragraphs that show that Gram was not disappointed in Old Faithful.
CA: Students should circle three paragraphs from the following options: 20, 22, 24, 25, or 26.
Associated text: Walk Two Moons by Sharon Creech
PAGE 15
Criterion B.4
Requiring a range of cognitive demand:
The assessments require all students to demonstrate
a range of higher-order, analytical thinking skills in
reading and writing based on the depth and
complexity of college- and career-ready standards,
allowing robust information to be gathered for
students with varied levels of achievement.
Key word: “range”
PAGE 16
Cognitive Demand
Range of Demand:
• The evidence descriptors for B.4 describe the
requirement that assessments should include a
range of cognitive demand that is appropriate to
the CCR standards themselves.
PAGE 17
Criterion B.5
Assessing writing:
Assessments emphasize writing tasks that require
students to engage in close reading and analysis of
texts so that students can demonstrate college- and
career-ready abilities.
Key phrase: “analysis of texts”
PAGE 18
Writing to Sources
Analysis of Texts:
• The criteria outline a shift from a traditional
emphasis on writing that calls for students to use
only their prior knowledge or experience. Instead,
the criteria, like CCR standards, place a premium
on writing to sources. This approach requires that
students use evidence from texts to present careful
analyses, well-defended claims, and clear
information.
PAGE 19
Criterion B.6
Emphasizing vocabulary and language skills:
The assessments require students to demonstrate
proficiency in the use of language, including
vocabulary and conventions.
Key word: “proficiency”
PAGE 20
Vocabulary and Language
Vocabulary:
• The evidence descriptors for vocabulary focus on
general academic (Tier 2) words, asking students to
use context to determine meaning, and assessing
words that are important to the central ideas of the
text.
Language:
• The evidence descriptors for language questions
state that these items should mirror real-world
activities (e.g., actual editing or revision, actual
writing) and focus on common student errors and
those conventions most important for readiness.
PAGE 21
Criterion B.7
Assessing research and inquiry:
The assessments require students to demonstrate
research and inquiry skills, demonstrated by the
ability to find, process, synthesize, organize, and use
information from sources.
Key phrase: “use information from sources”
PAGE 22
Criterion B.8
Assessing speaking and listening:
Over time, and as assessment advances allow, the
assessments measure the speaking and listening
communication skills students need for college and
career readiness.
Key word: “communication”
PAGE 23
Criterion B.9
Ensuring high-quality items and a variety of item
types:
High-quality items and a variety of types are
strategically used to appropriately assess the
standard(s).
Key phrase: “high-quality”
Key phrase: “strategically used”
Key word: “variety”
PAGE 24
Use of Item Types
High Quality:
• The previous eight criteria underlie this one. For
example, based on the criteria, reading items are
high quality when they are based on complex text,
and require close reading, analysis, and use of
evidence. Another example: questions and prompts
calling for writing are considered high quality when
they are text based.
Variety of Item Types Strategically Used:
• A variety of item types, used in meaningful ways, is
requisite to assess the full depth and complexity of
CCR standards.
PAGE 25
Summary of Section B of the CCSSO Criteria for
Procuring and Evaluating High-Quality
Assessments
Section B is based on key findings about college and career
readiness in ELA/literacy. To be CCR, students must:
1. Be adept at reading high-quality informational texts as well as
literature (B.1).
2. Regularly encounter texts with appropriate and challenging
text complexity (B.2).
3. Continually read and think deeply (B.3, B.4).
4. Use and cite textual evidence in support of claims and
inferences (B.3, B.5, B.9).
5. Be proficient in vocabulary, language, and research (B.6.
B.7), as well as speaking and listening (B.8).
PAGE 26
Section C. Align to Standards - Mathematics
Criterion C.1: Focusing strongly on the content most
needed for success in later
mathematics
Criterion C.2: Assessing a balance of concepts,
procedures, and applications
Criterion C.3: Connecting practice to content
Criterion C.4: Requiring a range of cognitive demand
Criterion C.5: Ensuring high-quality items and a
variety of item types
PAGE 27
Criterion C.1
Focusing strongly on the content most needed for
success in later mathematics: The assessments help
educators keep students on track to readiness by
focusing strongly on the content most needed in each
grade or course for later mathematics.
Key phrase: “focusing strongly”
PAGE 28
C.1.1 –Major, Supporting, and Additional Clusters for
Grade 5
Emphases are given at the cluster level. Refer to the Common
Core State Standards for Mathematics for the specific standards
that fall within each cluster.
Major Clusters
5.NBT.A – Understand the place value system.5.NBT.B – Perform operations with multi-digit whole numbers and with decimals to hundredths.5.NF.A – Use equivalent fractions as a strategy to add and subtract fractions.5.NF.B – Apply and extend previous understandings of multiplication and division to multiply and divide fractions.5.MD.C. – Geometric measurement: understand concepts of volume and relate volume to multiplication and to addition.
Supporting Clusters
5.MD.A – Convert like measurement units within a given measurement system.5.MD.B – Represent and interpret data.
Additional Clusters
5.OA.A – Write and interpret numerical expressions.5.OA.B – Analyze patterns and relationships. 5.G.A. – Graph points on the coordinate plane to solve real-world and mathematical problems.5.G.B – Classify two-dimensional figures into categories based on their properties.
PAGE 29
C.1 – Criteria evaluation of focus for CCSS-
aligned assessments
• For common core aligned assessments, the assessments
focus strongly on the content most important for students
to master in order to reach college and career readiness:
– In elementary grades, approximately three-quarters or
more of the points in each grade align exclusively to the
Major work of the grade;
– In middle school grades, approximately two-thirds or
more of the points in each grade align exclusively to the
Major work of the grade; and
– In high school, approximately half or more of the
points in each course align exclusively to prerequisites
for careers and a wide range of postsecondary studies.
PAGE 30
Criterion C.2
Assessing a balance of concepts, procedures, and
applications: The assessments measure conceptual
understanding, fluency and procedural skill, and
application of mathematics, as set out in college- and
career-ready standards.
PAGE 31
C.2 – The Importance of Balance
Conceptual Understanding
Procedural Skill and Fluency
Application
Achievement
PAGE 32
C.2 – Criteria Evidence Descriptors
What does “balance” mean for mathematics
assessments?
Two evidence descriptors for Criterion C.2 explain:
• The distribution of score points reflects a balance
of mathematical concepts, procedures/fluency, and
applications, as the state’s standards require.
• All students, whether high performing or low
performing, are required to respond to items within
the categories of conceptual understanding,
procedural skill and fluency, and applications, so
that they have the opportunity to show what they
know and can do.
PAGE 33
Criterion C.3
Connecting practice to content: The assessments
include brief questions and also longer questions that
connect the most important mathematical content of
the grade or course to mathematical practices, for
example, modeling and making mathematical
arguments.
Key word: “connect”
PAGE 34
C.3 – The Importance of Connections
• Focus and practices are not separate ideas. They
work together. Focus ensures we have time to treat
the material in-depth. The practices are partly what
"in-depth" looks like according to the standards. So
practices have to connect to content - preferentially
to the most important content - or else they aren't
consistent with the standards' design for higher
achievement.
What does “connect” mean for mathematics
assessments?
PAGE 35
C.3 – Criterion Evidence Descriptors
• Assessments for each grade and course
meaningfully connect mathematical practices and
processes with mathematical content (especially
with the most important mathematical content at
each grade), as required by the state’s standards.
• Explanatory materials (citing test blueprints and
other specifications) describe the connection, for
each grade or course, between content and
mathematical practices and processes.
PAGE 36
C.3 – Criterion Required Evidence for CCSS
• Every test item that assesses mathematical
practices is also aligned to one or more content
standards (most often within the major work of the
grade)
PAGE 37
Criterion C.4
Requiring a range of cognitive demand: The assessments
require all students to demonstrate a range of higher-order,
analytical thinking skills in mathematics based on the depth
and complexity of college- and career-ready standards,
allowing robust information to be gathered for students with
varied levels of achievement. Assessments include questions,
tasks, and prompts about the basic content of the grade or
course as well as questions that reflect the complex
challenge of college- and career-ready standards.
Key word: “range”
PAGE 38
C.4 – Evidence Descriptors
• Test blueprints and other specifications are
provided to demonstrate that the distribution of
cognitive demand for each grade level is sufficient
to assess the depth and complexity of the state’s
standards, as evidenced by use of a generic
taxonomy (e.g., Webb’s Depth of Knowledge) or,
preferably, classifications specific to the discipline
and drawn from mathematical factors, such as:
– Mathematical topic coverage in the task;
– Nature of reasoning;
– Nature of computation;
– Nature of application;
– Cognitive actions
PAGE 39
CCSSO Criterion C.5
Ensuring high-quality items and a variety of item
types: High-quality items and a variety of item types
are strategically used to appropriately assess the
standard(s).
Key word: “strategically”
PAGE 40
C.5 – Evidence Descriptors
What does “strategically” mean for mathematics
assessments?
• Specifications are provided to demonstrate that the
distribution of item types for each grade level and
content area is sufficient to strategically assess the
depth and complexity of the standards being addressed.
• To support claims of quality the following are provided:
– The list and distribution of the types of work
students will be asked to produce
– Exemplar items for each item type used in each
grade band
– Rationales for the use of the specific item types
– Specifications showing the proportion of item types
on a form
PAGE 41
State Panelists
• Irene Hunting, Deputy Associate Superintendent,
Assessment Section, Arizona Department of
Education
• Rebecca Kockler, Assistant Superintendent of
Academic Content, Louisiana Department of
Education
• Tracy Gruber, Education Programs Supervisor,
Nevada Department of Education
• Miah Daughtery, K-12 Literacy Coordinator,
Tennessee Department of Education
PAGE 42
Circling Back to Our Learning Objectives
• We hope you now feel more familiar with the
specific criteria (Sections B and C) from the CCSSO
“Criteria for Procuring and Evaluating High-Quality
Assessments.”
• Additionally, we hope that you have heard about
some actionable ways to incorporate the criteria
into your assessment programs to reach the goal of
producing high-quality, college- and career-
readiness assessments.