Building Systems of Assessments that Support Deeper Learning

Virg

inia

Sta

ndar

ds o

f Lea

rnin

g In

nova

tion

Com

mitt

ee, J

uly

15, 2

014

The Changing Demand for Job Skills (United States)2

1960 1970 1980 1990 2000 2006 200935

40

45

50

55

60

65

70

Routine manualNonroutine manualRoutine cognitiveNonroutine analyticNonroutine inter-personal

Mean task input in percentiles of 1960 task distribution

Source: Autor, David H. and Brendan M. Price. 2013. "The Changing Task Composition of the US Labor Market: An Update of Autor, Levy, and Murnane (2003)." MIT Mimeograph, June.

3

Current Tests Focus on Lower-Level Skills

• RAND study of 17 states’ tests: Only 2% of math items and 21% of ELA items assess higher-order skills.

• After NCLB, most states phased out performance assessments that could assess higher-order skills.

2000 2003 2006 2009 2012400

420

440

460

480

500

520

540

493

483

474

487

481

499

491 489

502

497

504

495498

502498

US Trends on PISA, 2000-2012

MathScienceReading

2012 PISA

Reading Singapore Japan Korea Finland Canada Ireland Poland Estonia Lichtenstein AustraliaUS is #21

Mathematics Singapore Korea Japan Lichtenstein Switzerland Netherlands Estonia Finland Canada Poland

US is # 32

ScienceSingapore Japan Finland EstoniaKoreaVietnamPoland Canada Lichtenstein GermanyUS is #23

6

High Achievers Use:-- Open-ended essays and problems to be solved and explained-- Performance tasks that require students to design and conduct investigations, collect data, analyze and present findings in writing, orally, and with technology

Other Nations Assess the Full Range of College & Career Ready Skills

7

Research finds that these assessments:

• Can be scored reliably by teachers with moderation and auditing systems

• Enable teachers to learn about the standards, curriculum, and teaching

• Support more rigorous instruction and more intensive learning

SINGAPORE EXAMINATIONS AND ASSESSMENT BOARD 8

To Assess Experimental Skills and Investigations, Students…

Identify a problem, design and plan an investigation, evaluate their methods and techniques

Follow instructions and use techniques, apparatus and materials safely and effectively

Make and record observations, measurements, methods, and techniques with precision and accuracy

Interpret and evaluate observations and experimental data

SINGAPORE SCHOOL-BASED SCIENCE PRACTICAL ASSESSMENT

GCSE ICT Task (England)

Litchfield Promotions promotes bands and puts on performances in England. They need to be sure that each performance will make enough money to cover all the costs and make a profit. Many people need to be paid: the bands; sound engineers; and, lighting technicians. There is also the cost of hiring the venue.

Litchfield Promotions needs to create an ICT solution to ensure that they have all necessary information and that it is kept up to date. Their solution will show income, outgoings and profit.

Candidates need to: 1) Work with others to plan and carry out research to investigate how similar companies have produced a solution. 2) Clearly record and display your findings. 3) Recommend and evaluate a solution addressing the task requirements.4) Produce a design brief explaining & critiquing your solution.

1. Common standards are grounded in a thoughtful, lean curriculum, which informs teacher planning, but is not a straitjacket.

It is managed as part of an integrated system of curriculum, assessment, instruction, and teacher development.

Teaching and Learning are supported by a coherent SYSTEM

2. Teachers are integrally involved in the development of curriculum at the school level and the development and scoring of moderated assessments.

Teachers are Deeply Involved

Teacher Collaboration in Design, Review, Scoring, and Evaluation

3. Open-ended assessments include evidence of actual student performance on challenging tasks that evaluate applications of knowledge and skills.

Performance is Valued

4. Assessment measures are structured to continuously improve teaching and learning.

The goal is Assessment of, as, and for Learning

Policy Contexts for Performance Assessments

• Formative tools for diagnosis and instruction• Components of summative assessments• Scored component of (end of) course exams• Assessments for proficiency-based decisions • Graduation portfolios, including exhibitions• Digital portfolios for college admissions/ placement

16

Queensland’s System of Curriculum and Assessments

Pre-Secondary Level Senior Level(Grades 11-12)

Curriculum Guidance Essential Learnings: Scope and sequence guides, unit templates, plus assessable elements and quality descriptors (rubrics)

Syllabi for each subject outlining content & assessments

External tests National tests of literacy and numeracy at grades 3, 5, 7, 9 -- Centrally scored

Queensland Core Skills Test, grade 12 (used for moderation)

State guided / Locally administered performance tasks

Queensland Comparable Assessment Tasks (QCAT): Common performance tasks locally administered & scored at grades 4, 6, and 9

Course assessments, outlined in syllabus -- locally scored / externally moderated

Locally developed assessments

Local performance assessment systems - locally scored and externally moderated - all grades

Graduation portfolios -- locally scored / externally moderated

17

Extended Experimental Investigation at the Senior Level (Grade 11-12) Over four or more weeks, students must develop and conduct an extended experimental investigation to investigate a hypothesis or to answer a practical research question. Experiments may be laboratory or field based The outcome of the investigation is a written scientific report of 1500 to 2000 words. The student must: • develop a planned course of action• clearly articulate the research question and provide a statement of purpose for the investigation• provide descriptions of the experiment• show evidence of student design• provide evidence of primary and secondary data collection and selection• execute the experiment(s)• analyze data• discuss the outcomes of the experiment• evaluate and justify conclusion(s)

Assessment Continuum

C-PASCollege ReadyAssess-ments

Narrow Assessment Assessments of Deeper LearningNarrow Assessment Assessments of Deeper Learning

CCSS Assessments

(SBAC &PARCC)

Common Performance

Tasks (Ohio, NH,New York)

Student-Designed Projects

(Envision, NY

Performance Standards Consortium,

Singapore, IB)

Traditional Tests

Descriptions

Standardized, multiple-

choice tests of routine skills

Standardized performance

tasks (1-2 weeks)that include structured

inquiry and demand more

integrated skills,

including collaboration

Examples

Standardized tests with m-c & open-ended items + short

(1-2 day) performance tasks of some applied skills

Performance tasks that require

students to formulate and carry out their own inquiries,

analyze & present

findings, and (sometimes)

revise in response to feedback

Longer, deeper investigations,(2-3 months) &

exhibitions, including graduation portfolios, requiring

students to initiate, design,

conduct, analyze, revise,

and present their work in

multiple modalities

19

5 CRITERIA FOR HIGH-QUALITY SYSTEMS OF

ASSESSMENT

1. ASSESS HIGHER ORDER SKILLS

21

2. Assess Critical Abilities with High-Fidelity

Research and analysisExperimentation and evaluationWritten communication (reading, writing)Oral communication (speaking, listening) Use of technologyCollaboration, Modeling, design, and problem solving

22

3. Be Internationally Benchmarked

• Rigorous content• Productive tasks assessing 21st century skills• High performance standards

23

4. Be Instructionally Sensitive and Educationally Valuable

• Underlying concepts should be teachable and learnable in school contexts

• Not depending on tricky interpretations that mostly reflect test-taking skills

• Not based on students’ ‘cultural capital’ or outside-of-school experiences

• Assessments should engage students in instructionally valuable activities

• Results from the tests should provide instructionally useful information.

24

5. Be Valid, Reliable and Fair

• Measure well what they purport to measure• Accurately evaluate students' abilities • Assess reliably across contexts and scorers • Be unbiased • Be accessible to a diverse learners• Be used appropriately • Support positive outcomes for students and

instruction.

25

To Accomplish this, New Systems of Assessment Should:

• Include locally implemented performance tasks as well as “on-demand” tests that are more sophisticated

• Involve teachers in design, review, and scoring of both kinds of assessments

• Provide information useful for developing curriculum and teaching

• Keep track of learning in many ways that support student agency and reflection

• Be used for improvement, not punishment

How is a “System of Assessment” built? Put the individual student in the center and consider:

• What are the knowledge, skills, and dispositions critical to completion of K-12 education and postsecondary success?

• How might students, teachers, and policymakers keep track of different kinds of learning?

• What student measures do we already – or plan to – collect?

• How could we develop a profile of student success?

< Knowledge, Skills, and Dispositions >

< Student

Measures

>

Interactive Elements of an Assessment System

28

Standardized Tests (with Performance Components)

Performance-Based Assessments / Portfolios

Used to validate local assessment results

Used to enrich test results and inform teaching

29

Steps in Creating a System of Assessments

• Define college and career readiness • Evaluate the gap between the system as it now exists and

the desired system • Identify the information needed for different users:

-- Policymakers (state / local)-- Students and parents-- Teachers-- Higher education and employers

• Identify purposes for state and local assessments

30

Creating a System of Assessments (2)

• Consider a continuum of assessments that address different purposes

• Develop assessments that can provide a profile of student abilities and accomplishments

• Connect these assessments to curriculum, instruction, and

professional development in a productive teaching and learning system

• Create an accountability system that encourages the kinds of learning and practice that are needed to reach the goals of college and career readiness.

31

Tasks for States and Districts

Develop a System of State

and Local Assessments that Meets Multiple

Needs

Create Means for High-Quality Assessment

Development, Review, and Moderated

Scoring

Increase Local Capacity through

Deep Participation in Assessment for

Learning and Professional

Learning Communities

Design an Accountability System that

Ensures Supports,

Professional Capacity, and

Useful Information

The CCSSO Innovation Lab NetworkPerformance Assessment Project:

• Works with states and districts to develop an Assessment Bank of exemplary performance tasks that evaluate CCSS standards and 21st century skills.

• Helps states and districts develop and pilot tasks and assessment strategies to measure the identified standards, support professional development for teachers, and build state and local capacity.

• Supports states as they develop / refine a policy framework for integrating this wider array of strategies into an overall system of assessments.

Stanford Center for Assessment

Learning and Equity (SCALE)

University of Oregon Education

Policy Improvement Center (EPIC)

U. of Kentucky Center for

Innovation and Learning

NY Performance Standards

Consortium

Center for Collaborative

Education

Innovation Lab Network Partners 33

34

A Performance Assessment Task Bank

35

Examples of Tasks - MathematicsSuggested

CourseGrade

Level(s) Task Title Task Overview

Algebra 9-10 The Rising Cost of Education

Students will act as a reporter for the US News and World Report magazine, who is writing an article about the rising cost of obtaining a college education. In order to be able to write the article students will first collect and analyze data on the cost of a college education. They will create equations and graphs showing the rising cost of education at different types of colleges including an in-state college, a community college, an out-of-state college, and an Ivy League college.

Algebra 2 11 Which is a Better Deal?

Getting a “good deal” is a challenge for every consumer. In this task, students will strengthen their skills to make well-informed purchasing decisions as they might in the real world. Students will identify multiple options of a good or service they want to purchase and use mathematical reasoning skills to evaluate their options. Students will produce a lab report detailing their analysis and reasoning.