innovation lab network performance assessment project performance task review process & quality...

Innovation Lab Networkperformance assessment project

Performance Task Review Process

& Quality Criteria OverviewKatie Wilczak

kwilczak@stanford.edu

Support states in the assessment of deeper learning by:Providing high quality performance tasks aligned to CCSS and NGSS in an online resource bankDeveloping educator capacity to effectively use performance tasks through:

a) Piloting tasks at school sitesb) Training educators to design, review,

implement, and score performance tasksHelping states, individually and collectively, in designing policy frameworks that support performance assessment implementation

ILN Performance Assessment Project Goals

3

Perf

orm

ance

Ass

essm

ent R

esou

rce

Bank

Inno

vatio

n La

b N

etw

ork

Perf

orm

ance

As

sess

men

t Pro

ject

Curriculum Embedded Performance Tasks

Performance assessments (typically culminating or end of unit) that Require application and integration of knowledge,

skills, and “Critical Abilities” (larger grain size tasks)

Follow a series of learning experiences in the classroom directly linked to the assessment

Are scaffolded by the teacher, with opportunities for feedback (peer, teacher), self-assessment, revision

Aligned to curriculum and core content in the discipline

Critical Abilities in the CCSS/NGSS

Research and analysis Experimentation and evaluation Written communication (reading, writing) Oral communication (speaking, listening) Use of technology Collaboration, Modeling, design, and problem solving

Not includedTask certified & ready for teacher use!

Task Review Process Overview1

2

3 4

5

6 7

8

9

Explanation of Steps

1. Tasks tracked by SCALE on Google docs; stored in Dropbox

2. All tasks must have: rubric, student instructions/prompt, any mentioned materials

3. Is the task a curriculum-embedded task? Or is it a unit/on-demand assessment?

4. Task and materials organized and uploaded to online system by EPIC

5. Reviewer evaluates task and makes comments in online system

6. Top-tier tasks reformatted into common project template by EPIC/SCALE staff

7. Reviewer evaluates task and makes modifications/completes materials

8. SCALE content leads resolve any conflicts between raters; conduct random backreads

9. Task checked for formatting, completeness; metadata form completed by SCALE

Overview ofPerformance Task Quality Criteria

Key Categories

Focus on Deeper Learning Alignment to Standards Student Choice and Agency Relevance and Authenticity Suitable for Diverse Populations Design of Student Task Curriculum Embedded

Performance Task Quality Criteria “Yes” – the criterion is met; it does

not require any revision to meet the criterion

“Yes, with Modifications” – the criterion can be met with minor edits/revisions

“No” – the criterion can only be met with major revision of the task

Key Shifts in the CCSS

1. Regular practice with complex text and its academic language.2. Reading, writing, and speaking grounded in evidence from text, both literary and informational.3. Building knowledge through content-rich nonfiction.

Key Transferable Skills & Concepts Some examples in ELA – Key Concepts

Language can reflect culture or shape culture Understanding the meaning of a text is enhanced

by understanding the biographical, historical, cultural, or social context in which it was created.

Some examples in ELA – Key Skills Critically analyze and compare multiple pieces of literature

with different perspectives on the same topic Analyze literary/rhetorical devices (author’s craft) Use a lens or theory (e.g., psychoanalytic, feminist,

Marxist) to interpret a piece of literature Write with attention to audience and purpose

What do we mean by “authentic” A topic or subject matter relevant to the

real world or to students’ lived experiences

Authentic in purpose and/or audience (e.g., a presentation to a real audience)

Meaningful to students

Bias, Stereotypes, or Sensitive Topics Avoid topics, texts, or tasks that may introduce

bias or offends students, in ways that disadvantages them in completing the taskE.g., a personal essay about a vacation trip; a task that tells students that everyone is an immigrant and instructs students to investigate their immigration history.

Topics, texts, or tasks that may advantage some sub-groups may be okay (e.g., multi-cultural literature)

Sensitive topicse.g., bullying, war, death (it may be okay, provided that there is a note to teachers about how to deal with emotional sensitivity of students)

2 Reviews Per Task

Primary Review Review task against Quality Criteria and

provide suggestions for modifications. Provide holistic recommendation for task.

Secondary Review Review task against Quality Criteria and

provide suggestions for modifications. Provide holistic recommendation for task. Compare your ratings to those from a primary

reviewer. Modify task using comments and suggestions

from your review as well as prior reviews.

Task Outcomes (Holistic Recommendation)

Included in task bank

Task Categories

Accepted as is or with minimal modifications. The task either already meets all of the Quality Criteria that are starred and a number of other criteria, and/or could meet all of the starred Quality Criteria with minimal revisions, taking 2 hours or less to make.

Accepted as a promising learning task. The task does not meet our criteria but it could be useful as a learning task in classrooms. For example, the task may not represent key learning shifts in the Common Core or NGSS, may not be useful as an assessment of learning, or may not require rigorous disciplinary thinking, but it would still be useful as a learning task for students to build specific targeted skills, e.g., oral presentation, time management.

Not included. The task does not meet the vast majority of our criteria.

What counts as a minor edit/revision? Change the wording of the prompt slightly

to clarify or focus the prompt Add or change the scoring rubric to align

better with the CCSS and the task Make an extension a requirement of the

task. (Add a prompt.) (Add or swap out a text.)Minor edits/revision should not take

more than two hours of work.

What DOES NOT count as a minor edit/revision?

Change the topic or focus of the task Change the core of what students are

asked to do or produce Replace all texts Rewrite a major portion of the task Re-organize the entire task

Non-Negotiables for top-tier

With minor edits/revision, it must…Require application of complex skills (Critical abilities, 21st Century Skills, Key Cognitive Strategies)Address key, transferable concepts and skills in a discipline(s)Be aligned to key skills and major claims in the CCSSGenerate student responses that can be scored using CCSS/NGSS aligned rubricsAllow for a variety of responses (open-ended)Be authentic, relevant, or meaningful to studentsBe free of bias, stereotypes, or other highly sensitive topics that may disadvantage subgroupsHave clear student directions/prompts

Common Template

*Only tasks accepted into the top-tier bin are reformatted.

ILN Performance Task Online Review System

Benefits of the Online Review System

Superior, centralized document management

Automated process to receive multiple reviews from multiple content experts

Status reports on outcomes of all tasks Automatically sends task to qualified

reviewer(s)

Online task database

performanceassessment.stanford.edu Tasks searchable by:

Grade level Subject & course Duration Level of collaboration Critical Abilities CCSS/NGSS/C3 alignment Accompanying materials (e.g., student work

samples) Source/contributor

Review ofDLSAI Tasks

Projectile Motion

Look at the original version of the task Take 5-10 minutes to begin completing

the Quality Review View SCALE/consultant completed Quality

Review & final ILN version of task Questions? Feedback?

*Remember that the goal of the ILN review is to categorize and improve tasks, NOT to provide feedback to the author/contributor

Other DLSAI Tasks

In small groups, review the other completed Quality Criteria reviews Social Entrepreneurship Don’t Believe Everything You Think Eugenics World Hunger The Bus Stops Here

What patterns do you notice? How would you change/improve the Quality Criteria?

Review Tasks

Use QC tool to review your tasks