common core institute #2 assessment literacy. agenda welcome announcements and introductions...

Common Core Institute #2

Assessment Literacy

Agenda• Welcome• Announcements and Introductions• Introduction to Assessment• Assessment Types• Assessment System

– Break• Technical Quality• Assessment and Accountability• Building an Assessment Plan

– Lunch• Q & A/Closing Remarks/Announcements

What is ASSESSMENT?• The systematic collection, review and use of

information about educational programs to improve student learning (Palomba & Banta)

• A process embedded within larger systems in a school/district/state

• Noun: tools and strategies used to collect evidence (Heritage & Herman)

• Verb: process of using evidence to improve decision-making (Heritage & Herman)

Purpose• Assessments are always developed with

purpose in mind:– What is to be assessed?– Why is assessment taking place?– How will student learning be measured?– How will the results of the assessment be used?

Types of Assessment• Summative Assessment• Interim Assessment• Formative Assessment

Tiers of Assessment

Formative (minute-by-minute, integrated into the lesson)

Interim (instructional, evaluative, predictive)

Summative

Scop

e an

d D

urati

on

Frequency of Administration

Wide

Narrow

Less Often

MoreOften

Adapted from Perie et al.(2009) and Talbot (2011)

Summative Assessments• Given once at end of a longer period of time• Are typically large scale assessments, which are

given state wide or nationally, but may be district wide or semester exams

• Generally used to evaluate performance against a defined set of content standards

(ASSESSMENT OF LEARNING--Stiggins, Arter, Chauppuis and Chauppuis , 2006)

Summative Assessments• Can meaningfully aggregate/disaggregate and

report results• Use results for accountability and policy

information• Least flexible

Arkansas Assessment Overview 2011-2012

Grade Tested

Criterion-Referenced Tests NAEP Norm-Referenced and National Tests

KEnglish Language Development Assessment

Kindergarten Screening: Qualls Early Learning Inventory (QELI)

1 English Language Development Assessment  

Norm-Referenced Test: Iowa Test of Basic Skills (ITBS)

Reading, Language and Mathematics

2 English Language Development Assessment  

Norm-Referenced Test: Iowa Test of Basic Skills (ITBS) Reading, Language and Mathematics

3

Augmented Benchmark Exam: Literacy and Math

Alternate Portfolios for IEP English Language Development Assessment

 Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics

4

Augmented Benchmark Exam: Literacy and Math

Alternate Portfolios for IEP English Language Development Assessment

Long-Term Trend (9 year olds)

Reading and Math

Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics

5

Augmented Benchmark Exam: Literacy, Math, and Science

Alternate Portfolios for IEP English Language Development Assessment

 Augmented Benchmark Exam: Abbreviated ITBS Reading, Language, Mathematics and

Science

6

Augmented Benchmark Exam: Literacy and Math

Alternate Portfolios for IEP English Language Development Assessment

 Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics

Arkansas Assessment Overview 2011-2012

Grade Tested

Criterion-Referenced Tests NAEP Norm-Referenced and National Tests

7

Augmented Benchmark Exam: Literacy, Math, and Science

Alternate Portfolios for IEP English Language Development Assessment

 Augmented Benchmark Exam: Abbreviated ITBS Reading, Language, Mathematics and

Science

8

Augmented Benchmark Exam: Literacy and Math

Alternate Portfolios for IEP English Language Development Assessment

Long-Term Trend (13 year olds) Reading and

Math

Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics

ACT EXPLORE

9

End-of Course Algebra I Grade 9 Math Portfolio for IEP

English Language Development Assessment

 Iowa Test of Educational Development (ITED) Complete Battery

Reading and Mathematics

10

End-of-Course Geometry End-of-Course Biology

Grade 10 Science Portfolio for IEP English Language

Development Assessment

 ACT PLAN/

PSAT

11

Grade 11 Literacy Grade 11 Literacy Portfolio for IEP

English Language Development Assessment

Long-Term Trend (17 year olds)

Reading and Math

ACT** SAT** **AP (Advanced Placement)

12English Language Development Assessment

ACT** SAT** **AP (Advanced Placement)

Interim Assessments• Fall between formative and summative assessments• Given periodically throughout the year or course• Are typically medium-scale assessments, which are

given at the district or school level• Generally used to evaluate performance against a

defined set of content standards(ASSESSMENT OF LEARNING--Stiggins, Arter, Chauppuis and Chauppuis , 2006)

Interim Assessments• Can meaningfully aggregate/disaggregate and

report results• Use results for program evaluation, resource

allocation, and instructional information on past practice

• More flexible than summative

Formative Assessment• Occurs moment-to-moment as part of instruction• Is a process used by teachers and students

frequently and is embedded in the current unit of instruction

• Are small scale, short cycle assessments given in the classroom to diagnose where students are in their learning

(ASSESSMENT FOR LEARNING--Stiggins, Arter, Chauppuis and Chauppuis, 2006)

Formative Assessment• Cannot aggregate/disaggregate and report

results• Use results to provide descriptive feedback or

to modify instruction• Most flexible

Formative AssessmentFormative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing

teaching and learning to improve students’ achievements of intended instructional outcomes (FAST SCASS, October 2006)

Five Critical Features to Guide Educators Toward Effective Use of

Formative Assessment• Learning Progressions – clearly articulates the trajectory

along which students are expected to progress• Learning Goals and Success Criteria – immediate goals to

be learned• Descriptive Feedback – ideas, strategies, and tasks students

can use to close the “gap” between his or her current learning level and the next level

Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Five Critical Features to Guide Educators Toward Effective Use of

Formative Assessment• Self and Peer Assessment – Students evaluate

their own learning and their peers learning based upon a success criteria

• Collaboration – All participants share responsibility for learning

Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Formative Assessment Cycle

Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

What are the major differences aMONG the assessment types?

Dimensions of AssessmentType of Assessment

Assessment for Learning Assessment of Learning

Dimension Formative Interim Summative

Purpose • Instructional • Most designed for managerial uses• Some designed for instructional

uses

• Managerial

Implementation • Driven by moment-to-moment decisions; generated or selected by teacher; individualized

• Regulated by protocols developed in or out of the classroom; teacher-generated or externally generated

Timing • During instruction• High frequency

• After instruction or during a break in instruction

• Moderate frequency

• After instruction• Low frequency

Scope • Narrow; one or very few learning objectives at a time

• Moderate; a manageable number of objectives

• Broad; comprehensive set of objectives

Audience • Classroom (teachers and students) • Administration and/or classroom • Public• Administration• Classroom

Adapted from Talbot (2011)

The Assessment System

Comprehensive Assessment System

• Consists of three types of assessments– Summative– Interim– Formative

• Assessment of learning and assessment for learning

Assessment System

Assessments in the Assessment System

Learning Goals/Progressions

Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).

The Three C’s• Comprehensiveness• Coherence• Continuity

Pellegrino, J., Chudowsky, N. & Glaser, R. Knowing What Students Know. National Research Council.

Building Complementary Assessment System

• Role of Classroom Assessment• Role of District Assessment• Role of State Assessment

Role of Classroom Assessment• Monitor ongoing progress• Make ongoing adjustments

– Adapting and responding to learning needs– Use different teaching methods– Use different teaching materials

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Role of District AssessmentFor Districts

• Monitor progress• Identify needed changes in:

– policies/procedures– funding– school organization

• Guide curriculum changes• Guide professional

development

For Schools• Set school improvement

goals• Determine staffing needs• Guide instructional changes• Guide professional

development

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Role of State AssessmentFor States

• Monitor standards• Monitor equity of opportunity• Guide changes in:

– laws and regulations– policies and funding– organizational structures

• Set improvement goals• Guide professional

development

For Districts• Monitor standards• Anchor district standards and

assessments• Guide changes in:

– board rules– policies and funding– organizational structures

• Monitor school improvement efforts

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Break:We will return in 15 minutes

Technical Quality

Why does technical quality matter?• Test construction• Test administration• Usefulness of results

– Credibility– Accuracy of score interpretation– Sound basis for adjusting instruction

Adapted from Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Assessment Design and Format• Items, tasks, strategies, rubrics must be

purposely designed to serve purpose• Various assessment formats and contexts:

– Paper-pencil, performance– Selected, constructed, extended response– Formal “tests,” analysis of work, homework

• Format does NOT = quality• Good assessments DO well serve their purpose(s)Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on

Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).

Credibility

• Valid and reliable for purpose intended

• Ensuring fairness and eliminating bias

Validity and Reliability• Validity is measuring what is intended to be

measured and using data to make accurate inferences.

• Reliability is consistency of test results from one test to another, from one part to another, from one time to another, and from one score to another.

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Bias and Fairness• Bias is giving or not giving an advantage to

one group or another.• Fairness refers to the purposes and uses of

tests in ways that are clearly communicated, ethical, and valid for all students to demonstrate their knowledge and skills.

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Accommodations and Validity• Individuals with Disabilities Education Act (IDEA)

Section 300.160(b)--Accommodation guidelines–(1) A State (or, in the case of a district-wide assessment, an LEA) must develop guidelines for the provision of appropriate accommodations.–(2) The State's (or, in the case of a district-wide assessment, the LEA's) guidelines must:

• (i) Identify only those accommodations for each assessment that do not invalidate the score; and

• (ii) Instruct IEP Teams to select, for each assessment, only those accommodations that do not invalidate the score.

• Elementary and Secondary Education Act (ESEA), Title ISection 200.2(b)(2) requires that a State’s assessment system be designed to be valid and accessible for use with the widest possible range of students, including students with disabilities and students with limited English proficiency.

Dos and Don’ts When Selecting Accommodations

Do…make accommodation decisions based on individualized needs.

Don’t…make accommodations decisions based on whatever is easiest to do (e.g., preferential seating).

Do…select accommodations that reduce the effect of the disability to access instruction and demonstrate learning.

Don’t…select accommodations unrelated to documented student learning needs or are intended to give students an unfair advantage.

Do…be certain to document instructional and assessment accommodation(s) on the IEP or 504 plans.

Don’t…use an accommodation that has not been documented on the IEP or 504 plans.

Do…be familiar with the types of accommodations that can be used as both instructional and assessment accommodations.

Don’t…assume that all instructional accommodations are appropriate for use on assessments.

Do…be specific about the “Where, When, Who, and How” of providing accommodations.

Don’t…simply indicate an accommodation will be provided “as appropriate” or “as necessary.”

Thompson, S., Morse, A., Sharpe, M. & Hall, S. Accommodations Manual: How to Select, Administer, and Evaluate Use of Accommodations for Instruction and Assessment of Students with Disabilities. Council of Chief State School Officers, Assessing Special

Education Students State Collaborative.

Dos and Don’ts When Selecting Accommodations

Do…refer to state accommodations policies and understand implications of selections.

Don’t…check every accommodation possible on a checklist simply to be “safe.”

Do…evaluate accommodations used by the student. Don’t…assume the same accommodations remain appropriate year after year.

Do…get input about accommodations from teachers, parents, and students, and use it to make decisions at IEP team or 504 planning committee meetings.

Don’t…make decisions about instructional and assessment accommodations alone.

Do…provide accommodations for assessments routinely used for classroom instruction.

Don’t…provide an assessment accommodation for the first time on the day of a test.

Do…select accommodations based on specific individual needs in each content area.

Don’t…assume certain accommodations, such as extra time, are appropriate for every student in every content area.

Thompson, S., Morse, A., Sharpe, M. & Hall, S. Accommodations Manual: How to Select, Administer, and Evaluate Use of Accommodations for Instruction and Assessment of Students with Disabilities. Council of Chief State School Officers, Assessing Special

Education Students State Collaborative.

Accuracy of Score Interpretation• Use results based on the design and intended

use of assessment• Avoid overinterpretation/misuse of results• Factors affecting trend lines: change in test

blueprint; change in content standards; change in scale; change in cut scores; change in student population

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

What is the relationship between the assessment system and

accountability?

Accountability System• Data system for reporting selected pieces of data

about student learning and factors affecting learning to determine success of the educational system

• Used as means of holding schools and their supports (district, state, legislation, policies, procedures, practices, funding) accountable for how well students are learning

• Assessment is one part of the accountability system

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Accountability System

Tiers of AccountabilityState District Schools Students

How well the laws and funding passed by legislature have

positively impacted student achievement of the standards

How well students have achieved standards

How well students have achieved the standards

How well students have achieved the standards

How well the State Department’s policies and procedures have

resulted in student achievement of the standards and high functioning

schools

How well funds are used to carry out laws, policies, and procedures in supporting student achievement

How effectively the school has maximized students’ opportunity to

learn by keeping them in school

How diligent students have been in using the opportunities to learn

provided by the school

How well qualified teachers are to teach the standards

How effectively the school has created a safe and positive learning

environment

How well schools have used given resources to maximize students’

opportunity to learn

Adapted from Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Building an assessment plan

Sample Assessment PlanCentral Bucks School District, Doylestown, PA

Sample Assessment PlanCentral Bucks School District, Doylestown, PA

Sample Assessment Plan

Sample Assessment Plan

Sample Assessment Plan

Assessment Plan TemplateAssessment Cycle Assessment type(s) Assessment tools Expectations for

student use of dataPrimary Expectations for teacher use of data

Expectations for school and/or district level use of data

Emphasis area for professional development

Continuous or Daily            

Weekly or Monthly            

Unit            

Quarterly            

Annual            

Ultimate goal of a coherent assessment approach: Create an aligned system of assessment so that all assessments are measuring agreed upon learning goals and discrete measurement takes place along the learning progression embedded within each goal.

Courtesy of Syracuse School District in Syracuse, NY, and the New York Comprehensive Center. http://www.nycomprehensivecenter.org/docs/form_assess/SyracuseAssessmentFramework.pdf

Ultimate goal of a coherent assessment approach:

Create an aligned system of assessment so that all assessments are measuring agreed upon learning

goals and discrete measurement takes place along the learning progression

embedded within each goal.

Assessments in the Assessment System

Learning Goals/Progressions

Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).