how’s it growing? enhancing data literacy and engagement through data visualization

How’s it Growing?Enhancing Data Literacy

and Engagement through Data Visualization

Juan D’BrotExecutive Director

Office of Assessment and Accountability

October 22, 2012

The Overview1. What is Growth and a Growth Model?2. Growth at the school level 3. Applying growth (FAIPs)4. Drilling back down to the student level 5. Applying growth in the classroom6. Where to go next to drive instructional decisions

Why a Growth Model?

•AYP vs. Continuous Improvement (non- accountability) •Enhancing Data Literacy•Enhancing Engagement•Future-oriented Capacity Building– SMARTER Balanced Assessment Consortium– ESEA Reauthorization

Why NCLB Promotes Insensitivity to Individual Student Growth and School Improvement

No Content Mastery

Full Content Mastery

No Content Mastery

Full Content Mastery

Student Proficiency

AYP (status) Model

Growth (improvement) Model

Access to Full Spectrum of Student Performance and Improvement

Compliance vs. Continuous Improvement

Adequate Yearly Progress Improvement and Growth

•Snapshot•Non-comparable trends•Funding considerations•“Outcome”

•Multiple Snapshots•Comparable trends•(More) instructional consideration•“Process”

Transitioning to Growth

• Past: Contextualizing prior performance

• Present: Understanding current student and school growth

• Future: Building capacity for understanding and applying growth post-WESTEST 2

The WV Growth Model

•What are we trying to answer?•Purpose •How it works•What it yields?•How we interpret it•What levels (overview)– Past, present, future– Student, Grade, School, District, State– Prior growth, observed growth, projected growth (Growth

to Standard)

Asking the Right Questions…

• Until you’ve defined the question, you cannot examine the appropriate data.

• Why?– Different data answer different

questions – Different questions lead to different

conversations

Fact: …increased use of student assessment data as a “significant factor” in (insert favorite accountability topic here…)

• But what student data? That leads us to start asking questions…

• But what questions?

• Whose questions?

Whose Questions?• Researchers

• Administrators

• Teachers

• Parents

• Policymakers

• Business Leaders

• Community Members

• Guardians

WHY?To answer questions about the

student

How?By not declaring a verdict

(status). Instead, by changing the

conversation (to growth and progress)…

Fact: Assessment Systems try to be Everything to Everyone• Danger: answering too many

questions with the same data– How to Differentiate Instruction– Determining Student Proficiency– Measuring Class Performance– Determining AYP

• Possible Solution: Can we come up with a common question across stakeholders? But first…

The WV Growth Model: Our Purpose

• To provide an answer to a few common questions: 1. “How much did my student grow this

year?” 2. “What does this growth mean

compared to everyone else?”3. “Is it enough growth?”

What do these three questions have in common?

Information about students

Status vs. Growth

• Before growth, let’s review terms

– Status: A snapshot measure of a single point in time – “Proficiency”

– Growth: Multiple snapshots across many points in time

Status vs. Growth A Balanced Approach

Low Growth High GrowthQuadrant 2

High achieving but potentially declining: previously

unrecognized under status model

Quadrant 1High achievement and growth: A potential site of interest for best

practices

Quadrant 3Low achievement and growth:

Legitimately requires improvement

Quadrant 4Low achieving but improving:

previously unrecognized under the current status model. May be a potential site of interest for best

practices.

High Performing

(Status)

Low Performing

(Status)

Let’s Consider Growth at a High Level…

Measuring Growth: A Primer

• What is the likelihood of current (and future) performance given past performance?

• Based on students who have similar academic histories (academic cohort)

• Uses all available data to answer these three questions:– Am I proficient? (Status)– How much did I grow? (SGPs)– Is it enough growth? (Growth to Standard)

Our Growth Options (Primer, cont’d)• How much growth? – Starts with the

student. Can be aggregated to drive school conversations– High Growth (a good surprise)– Typical Growth (a statistical expectation)– Low Growth (a bad surprise)

• Is it enough growth? – Can drive student learning conversations– Catching Up– Keeping Up – Falling Behind

Questions - Revisited1. “How much did my student grow this year?”

(Time 2 – Time 1) – We can see this today

2. “What does this growth mean compared to everyone else?” (Normative Component)

3. “Is it enough growth?” (Criterion Component - CSOs)

For more information…Please visit the WVDE’s Growth Model webpage:

http://wvde.state.wv.us/growth/

Understanding Grade Level and School Growth Reports

Viewing Aggregated Growth at the School Level

• What to consider?– High vs. Low Achievement

• Individual Student Performance• Overall Percent Proficient

– High vs. Low Growth• Individual Student Growth• Overall Growth for a School

• An example of a grade and school level report

Grade Level Bubble Chart

• Growth Percentile vs. Scale Score (Achievement) for all students within a grade that have growth data

• Shows distribution of a group of students

• Hover capability to view student information at each data point (bubble)

Above Mastery

Mastery

Partial Mastery

Novice

Scale Score

Individual Student Growth Percentile

Each bubble is a single student who is plotted by:

Higher Observed Growth of a Single Student in a School

High

er P

erfo

rman

ce o

f a S

ingl

e St

uden

t

John Smith:2010-11 student Growth Percentile (Target): 95 (14)2010-11 Achievement Level (Scale Score): Distinguished (757)

Distinguished

Above Mastery

Mastery

Partial Mastery

Novice

Why are we interested in these data?

1. Easy to read overview of grade level performance

2. Growth + performance is more telling than performance alone

3. Ease of access can lead to immediate interest into student performance

4. Helps support our next logical question…

School Growth and Achievement Bubble Plots

School Growth and Achievement Bubble Plots

Enhancing Interest in the Story Behind the Data

Growth Model Interest in Data Examining Individual Performance Reviewing Individual Item Analysis Consideration of how to more effectively use of interim assessments and formative processes Validating what we see students learning and understanding

Teacher expertise leads to professional judgment about instruction

Examining District-LevelStudent Reports

• Log in to WVEIS and go to the Private Data Site

• At the bottom of the screen, you will see a link titled “Growth Model”

• Click Growth Model, select your district and select a school of interest.

• Click Go!

Growth Legend

Content Areas

Math (pdf) Reading (pdf)

Math (pdf) Reading (pdf)

Math (pdf) Reading (pdf) Click Here for Bubble Plots

Growth LegendSubgroup Comparison

Content Areas

Understanding Individual Student WESTEST 2 Growth

& Achievement Report

Achievement: A Quick Review• Current WESTEST 2 Student Report

shows: 1. Single Scale Score in each content area2. Level of Achievement (Performance

Levels)• Distinguished• Above Mastery• Mastery• Partial Mastery• Novice

The WV Growth Model: Our Purpose

• To provide answers to a few common questions: 1. “How much did the individual student

grow this year?” 2. “What does this growth mean

compared to everyone else?”3. “Is it enough growth?”

What do these three questions have in common?

Information about students

Measuring Growth: A Primer

• What is the likelihood of current (and future) performance given past performance?

• Based on students who have similar academic histories (academic cohort)

• Uses all available data to answer these three questions:– Am I proficient? (Status)– How much did I grow? (SGPs)– Is it enough growth? (Growth to Standard)

Our Growth Options (Primer, cont’d)• How much growth? – Starts with the

student. Can be aggregated to drive school conversations– High Growth (a good surprise)– Typical Growth (a statistical expectation)– Low Growth (a bad surprise)

• Is it enough growth? – Can drive student learning conversations– Catching Up– Keeping Up – Falling Behind

Hello. It’s nice to meet you.

Hello, Growth Report

Meet the Student Growthand Achievement Report

Novice Partial Mastery Partial Mastery Mastery

Name of Student Name of School

Interpretation Guide

Suggested Uses

Growth and Achievement for Math

Growth and Achievement for R/LA

Novice Partial Mastery Partial Mastery Mastery

Legend for Math

Legend for R/LA

Scale Scores for Math

Scale Scores for R/LA

Grade Level and School Year

Grade Level and School Year

Student Achievement Levels

Student Achievement Levels

WESTEST 2 Achievement Levels

WESTEST 2 Achievement Levels

Novice Partial Mastery Partial Mastery Mastery

GrowthLevelsGrowthPercentiles

The colored arrows between the dots represent growth from one grade to the next.

The color of the arrow signifies the level of growth.

High growth is indicated in green, typical growth in white, and low growth in red.

The fan represents how the student might score next year under several growth scenarios.

How much growth?

Is it enough growth?

How much growth?Growth Percentile – 1% to 99%• 66%-99% = High Growth = Green Arrow

- Good surprise - the student exceeded what was expected• 35%-65% = Typical Growth = White Arrow

- Statistical expectation - the student grew as

expected• 1%-34% = Low Growth = Red Arrow

- Bad surprise – The student grew less than was expected

Is it enough growth?Will a student be proficient next year?• Determined at only one grade level at a time• Depends on the achievement level where the student

was at a particular grade level• The fan can help us understand potential stories for a

given student• What level of growth is required for a student who

is proficient to keep up with proficiency next year? • What level of growth is required for a student who

is not proficient to catch up to proficiency next year?• What level of growth for any student, would cause

a student to fall or stay below proficiency next year?

Prior Proficient: Student is Keeping Up

Prior Proficient: Student is Not Keeping Up/Falling Behind (Math)

Prior Proficient: Student Keeping Up (Math)/Will Fall Behind (R/LA)

What about this student?

Novice

What about this student?

Turning to Companion Reports

–WVEIS-based Reports»Teacher-based Growth Report»CIAS/CSO Analysis»Teacher-based CSO Analysis»Student’s Individual Responses

A Sample WVEIS on the WEB Teacher Report

WESTEST 2 Growth Model Report

WESTEST 2 CSO Analysis

WESTEST 2/CSO Analysis

Item Analysis and Objectives

• Go to WVEIS private data site– Bottom left panel: Electronic CSR– Bottom left panel: WESTEST2/CSO Analysis

• Click on CSOs• See standard and number of students at each item

– What about after CIAS?• Think about the objectives themselves– Alignment

Item Analysis and Objectives

• Danger:– Objective-level sampling– Overemphasis on a single objective

• Consideration:– Consider learning progressions– Foundational objectives– Reporting (e-CIAS) is only the first step– What are the linked objectives and what tools

can help you analyze student mastery of those skills?