data tools in a pa standards aligned system session facilitators : pde pvaas statewide team kristen...

Data Tools in a PA Standards Aligned System

Session Facilitators:

PDE PVAAS Statewide Team•Kristen Lewald, Ed. D.•Jennifer Ross•Jim Bohan, Ed. D

Evidence to Action: K-12•Cindy Goldsworthy, Ed. D.

PDE SAS InstituteHershey, PADec 5-8, 2010

Session Description

• This session will highlight foundational practices and protocols for engaging teachers in data-driven decision-making at the school, grade and student levels using PA’s data tools.

• Throughout the session participants will view actual video clips of team meetings with teachers engaged in meaningful data analyses connected to instructional planning.

• Participants will deepen their understanding of the connection between these practices and Pennsylvania’s Standards-Aligned System (SAS Portal).

• Participants are asked to bring:– laptops with wireless access

– user names/passwords to eMetric and PVAAS

Session Focus• School Level – Not reviewing Getting

Results (www.pasip.org)• Session is not designed to be an

introduction to PSSA reporting or PVAAS reporting.– We will do our best to fill in/review info to

assist with understanding for those new to these types of reporting.

– Help your table mates!

• If you need training on PSSA Reporting/PVAAS reporting, let us know and we will connect you to professional development resources.– Ex. IU PVAAS Contacts; Archived Webinars

Session Agenda

• 8:15AM -1:30PM–AM Break

–Lunch, 11:30-12:00

• 30 minutes

Session Materials

• Powerpoint• Sample Data Packets

– Grades 4-8 and Grades 9-12

• Foundational Questions Cards• Foundational Questions One Pager (2 versions)• Data Tool Reference Guide• PVAAS Color One Pager• Online: Student Data Mtgs DVD + Supplemental

PPT– Do NOT attempt to download during this session

• End of session- flash drive with ALL materials including video clips and entire DVD/PPT and supplemental PPT!!!!

Table Team and Roles

• Table Team Introductions– Name, Organization, Role

• Roles for Session1.Spokesperson – IU Consultant

2.Select timekeeper, task master and documenter!

Annual Building-Wide Planning Process

School Structures for Data-Informed Decision Making

Periodic Grade-Level Planning Process

Student-Planning Process

District-Level Support(Budgetary Support, Professional Development, Resources and Time)

ACHIEVEMENT AND GROWTHScatter plots

Growth Standard Scatter plots

Grades 4 through 8 – Math and Reading

Growth Standard Scatter plotGrades 4 through 8 – Math and Reading

School A (2.3, 87)Coordinate Value Description Interpretation

PVAAS Growth Value

2.3

Cohort made more than one year’s growth

Congratulations! The instructional program is resulting in students performing AND growing well!Percent

Proficient or Advanced 87

87% of students Proficient or Advanced

School B (0.8, 95) Coordinate Valu

eDescription Interpretation

PVAAS Growth Value

0.8

Cohort made slightly more than one year’s growth

Congratulations! The instructional program resulted in students performing AND growing well!Percent Proficient

or Advanced95


School C (-1.2,90) Coordinate Valu

eDescription Interpretation

PVAAS Growth Value

-1.2Cohort did not make one year’s growth

Caution! Students are performing well but have slipped over the past year. The student group has not made a year of growth. This may impact achievement over time.

Percent Proficient or Advanced


School D (0,75) Coordinate Value Description Interpretation

PVAAS Growth Value

0Cohort made one year’s growth

Students maintained their position in the state data base over the past year. However, as AYP targets increase, this group may be at risk for not meeting the AYP targets.



School E (1.7,58) Coordinate Value Description Interpretation

PVAAS Growth Value

1.7Cohort made more than one year’s growth

While the cohort did not meet the AYP target, the positive PVAAS Growth value indicates that the students are on a positive trajectory. Determine what may have yielded more than a year’s growth.



School F (-2.1,49) Coordinate Value Description Interpretation

PVAAS Growth Value


This cohort performed well below the AYP target. The PVAAS Growth value indicates that this cohort lost even more position over the last year. Significant focus on school improvement should be considered.



School G (-1.7,72) Coordinate Value Description Interpretation

PVAAS Growth Value


While this cohort performed above the AYP target, the PVAAS Growth value indicates that the cohort is losing position. As the AYP targets increase, the cohort may not meet or exceed the AYP target.



All Schools: Possible Next Steps

Question – check all grades/subjects Source Report

1. Is this performance consistent with the prior 2 years’ performances?

eMetricGetting Results 3-year Portrait

2. Is there any indication of subgroup achievement gaps?

eMetricReporting Categories(p. 5 in both)

3. How did the performance category distribution from this year compare to last year?

eMetricMatched Comparison(p. 6 in 4-8 only)

4. Are all predicted performance groups showing a year or more of growth?

PVAAS

Performance Diagnostic(p.14 in grade 4-8; p.11 in grade 11)

5. Are all subgroups showing a year or more of growth? PVAASPerformance Diagnostic

6. What proportion of our students are not likely to be proficient on a future PSSA exam?

PVAASProjection Summary – Proficient(p.18 in 4-8; p.14 in 11)

7. What proportion of our students could meet an advanced level of performance?

PVAAS Projection Summary - Advanced

Different Schools Call for Different Questions

• What different questions/issues/additional data/emphasis for analyses might School B ask and investigate as compared to School C?– Using Grade 4-8 sample data packet

School B School C

Group 1: School B compared to School CSchool B:• How can we make sure we keep getting these results!•Which grades/subjects have we been getting high growth?• Which performance groups of students have been getting high growth?•What practices MAY be contributing to our results?•Make sure TEACHERS answer these questions• Data: PVAAS Projections to Proficient and Advanced (to keep the expectation for high levels of performance); Student Lists of PVAAS Projections to Proficient and Advanced (70-100% AND 40-70%)•Data: eMetric Reporting Categories reports, multiple years, any areas to fine tune?•Data: eMetric Matched Comparison; who moved a whole performance level? Why? Any patterns to learn from?•Other?:

School C:• How do we keep our achievement levels from slipping due to a lack of growth?• Why are we NOT growing our high performing students?• Are we growing any performance groups of students?• Do the administrators and teachers believe high achievers can make a year of growth? Are there any excuses people are sharing? Data: eMetric Individual Performance Single Year to show RANGE of scores; Scatter plots to show that others are yielding more• Data: PVAAS Projections to Proficient and Advanced (to set the expectation for high levels of performance); Student Lists of PVAAS Projections to Proficient and Advanced (70-100% AND 40-70%)• Data: eMetric Reporting Categories reports, multiple years; any patterns emerging?•Data: eMetric Scaled Score and Performance Levels, how many are P v. A,?; could more be advanced/stay advanced?•Other?:

School B School C

Predictive Method Scatter plots

Grade 11 – Math and Reading; Writing, Science

Click on each underlined point/school to go to interpretation.

Predictive Scatter plot

School A (3.5,1600) Coordinate Value Description Interpretation

Standardized School Effect

3.5

Significant evidence that the cohort exceeded the expected level of performance.

Students were predicted to score well above the state mean and the school yielded growth significantly above that expectation.

Mean Predicted Scaled Score

1600Students were predicted to score a mean of 1600 on this exam.

School B (-1.7,1480) Coordinate Value Description Interpretation


-1.7Evidence that the cohort performed at the expected level of performance.

Students were predicted to score slightly above the state mean, and the school fostered growth to meet that expectation.



School C (-5.1,1620) Coordinate Value Description Interpretation


-5.1

Significant evidence that the cohort fell below the expected level of performance.

Students were predicted to score well above the state mean, and the school yielded growth significantly below that expectation.



School D (4.0,1275) Coordinate Value Description Interpretation


4.0

Significant evidence that the cohort exceeded the expected level of performance.

Students were predicted to score slightly below the state mean, and the school yielded growth significantly above that expectation.



School E (1.9,1195) Coordinate Value Description Interpretation


1.9Evidence that the cohort performed at the expected level.

Students were predicted to score well below the state mean, and the school yielded growth that met that expectation.



School F (-3.3,1305) Coordinate Value Description Interpretation


-3.3

Significant evidence that the cohort fell below the expected level of performance.

Students were predicted to score slightly below the state mean, and the school yielded growth significantly below that expectation.



Possible Next Steps

Question – check all grades/subjects Source Report

Is this performance consistent with the prior 2 years’ performances?

eMetricGetting Results 3-year Portrait

Is there any indication of subgroup achievement gaps? eMetric Reporting Categories

Are all predicted performance groups showing a year or more of growth?

PVAASPerformance Diagnostic

Are all subgroups showing at least or more than a year of growth?

PVAASPerformance Diagnostic

What proportion of students are not likely to be Proficient on a future exam? What proportion are likely to be advanced?

PVAASProjection Summary – Proficient

Different Schools Call for Different Questions

• Tables are numbered off by 1 and 2• Groups 1 and 2:

– What different questions/issues/additional data might School __ ask and investigate as compared to School __?

• Use your Grade 9-11 data packets as a starting point

• 5 minutes• Be prepared to report out

Group 1: Grade 9-11 Schools B and C

School B:• • • • • • • • • • • • • •

School C:• • • • • • • • • • • • • •

Group 2: Grade 9-11 Schools E and F

School E:• • • • • • • • • • • • • •

School F:• • • • • • • • • • • • • •

Engaging TEACHERS: Using Data

• Table Spokesperson- time to start working!• Do 2 quick “whip arounds” at your table to

get answers to these 2 questions:– How are TEACHERS in your school(s)

engaged in school level data analyses?– How is data used to address people’s belief

systems as to why they are not yielding the desired results? – to create ownership

• 3 minutes• Be prepared to share with the whole group

Engaging Teachers

• Let’s Watch a Team!!!!!• Building-wide Team, K-6

– Regular and Special Ed– Principal at the table– 2 facilitators– Reviewing annual results

• You will not be able to hear details of conversations– Watch

• Question to ask during video clip– Are the teachers engaged? Why?

• CV CLIP: Data Work Discussion (1:15)

Engaging Teachers

• No formal training on any data tools• Specific Data Packet Provided

– Everyone has a data packet– Line of Inquiry (the questions to ask) for EACH

set of reports is provided

• Work in pairs……cannot hide in a pair• Debrief after each section to summarize

findings, clarify, uncover issues/beliefs• Document findings

Teacher Ownership

• All team members have chance to report out• No one is presenting results to the teachers• Teachers and principal are discussing THEIR

results• Let’s Watch!• CV CLIP: Data Discussion (1:05 - stop before

completed)

Teacher Ownership• Ownership of Results Discussion

– Positive– Needs Improvement

• Do they see/believe there is room for improvement?• Data may not take away the argument/beliefs that they

do not own the problem, but it can minimize the focus– Need ownership of both

• Reminder needed about owning areas for improvement……as well as strengths

– If not, areas for improvement tend to be linked to student issues, family issues, socioeconomic issues, and subgroup issues – issues outside of our sphere of influence

THE HEALTH OF OUR STANDARDS-ALIGNED SYSTEM

Additional Data!

Foundational Question #1

Is there strong, observable evidence that the standards-aligned curriculum and effective instructional practices are consistently implemented across all classrooms?

Indicators:•There is a curriculum that specifies the competencies to be taught which are aligned to the standards.•The curriculum is consistently implemented across all classrooms. •Effective instructional practices are implemented across all classrooms.

Foundational Question #2Is there strong, observable evidence that school staff regularly uses standards-aligned assessments to monitor student achievement and adjust instructional practices?

Indicators:•Standards-aligned benchmark assessments are used to monitor the progress of student achievement and to make data-informed decisions.•Formative assessments are used by teachers and students during instruction to provide feedback to adjust ongoing teaching and learning.•Standards-aligned summative assessments are used to make data-informed decisions.•Diagnostic assessments are used as needed to ascertain students’ strengths, weaknesses, knowledge, and skills.•The school has an assessment calendar which includes assessment dates and dedicated time for school staff to analyze data and determine the effectiveness of instructional practices in a timely manner.


Is there strong, observable evidence that struggling students are identified early and are supported by an intervention system with procedures for monitoring effectiveness?

Indicators:•A universal screening process is in place to proactively identify students at risk.•Diagnostic assessments are used to target appropriate interventions for students academically at risk. •An intervention infrastructure exists in the school which identified appropriate materials and resources linked to student needs at all three RTII tiers. •The school staff conducts ongoing monitoring of student progress to determine the effectiveness of interventions and to adjust them as needed, based on the data.


Is there strong, observable evidence that all students have access to standards-aligned curriculum and challenging expectations?

Indicators:•All students have the opportunity to demonstrate proficiency in a grade level core curriculum, supported with interventions as needed. •All classroom teachers design and differentiate instruction that reflects challenging learning expectations for all students. •School staff use evidence-based interventions aligned with specific student needs.


Is there strong, observable evidence that student needs drive decisions about teacher placement?

Indicators:•All teachers are highly qualified in their content areas.•Expert teachers and specialists are matched with the students most in need.


Is there strong, observable evidence that professional development is focused, strategic and implemented with fidelity?

Indicators:•Professional development is based upon a needs assessment that is aligned with the school’s instructional priorities.•Professional development is planned across a 3 to 5 year span and is limited to no more than six priorities. •Building administrators actively participate in professional development activities with their staff and can articulate key features of the initiative, including what it looks like in the classroom.•Professional development is monitored for fidelity of implementation.•The majority of professional development resources (time, money and people) are used to support on-going implementation.


Is there strong, observable evidence that teachers and administrators meet regularly to reflect on the progress of students learning using multiple data sources and professional practice?Indicators:•The school has regularly scheduled time for teachers and administrators to reflect on professional practice as it relates to the progress of student learning.•School staff uses data meeting protocols.•School staff reviews student work to collaboratively identify practices to improve student performance.


Is there strong, observable evidence that teachers and administrators receive timely, effective support and intervention as needed?

Indicators:•All new teachers and administrators complete a required induction and mentoring program.•There is a system in place that provides support and intervention for individuals whose practice is in need of improvement.


Is there strong, observable evidence that the school’s resources effectively address the school’s instructional priorities?

Indicators:•Data informs the allocation and alignment of resources (infrastructure: facilities, technology, human resources, scheduling, etc.).•Funding sources support the integration of resources to address the school’s instructional priorities. •Teachers have access to sufficient materials to support their instructional practice.


Is there strong, observable evidence that the school’s administrative team leads to the implementation of a standards-aligned system?Indicators:•The administrative team participates in the school-based professional development to determine the specific educator behaviors that will indicate the professional development is effective in changing practice.•The administrative team holds all staff members accountable for consistent implementation of the standards-aligned curriculum.•The administrative team is actively involved in the analysis of student achievement data at the building level.•The administrative team has a working knowledge of standards-aligned curricula for all grades within the school.


Is there strong, observable evidence that each member of the school community promotes, enhances and sustains a shared vision of positive school climate?

Indicators:•School rules establish clear and consistent boundaries that support a positive school climate.•School rules are consistently enforced by all staff.•The school leadership team gathers and shares accurate and reliable data about school climate from students, staff and parents for continuous improvement. •A behavior support team exists that responds promptly to students with chronic problem behaviors. •The school environment is clean and well maintained.•Teachers and students are recognized for accomplishments.


Is there strong, observable evidence that school staff monitor attendance and student engagement and respond with classroom and school-wide interventions when students are chronically absent or disengaged?

Indicators:

•Teachers and school administrators design specific classroom and school-wide practices to reengage students who have become disengaged.

•School policies promote frequent monitoring of student attendance data with a system for intervening with students that are chronically absent.

Foundational Questions

• Partners or Trios– Review the foundational questions and make note of SAS

Portal Connection/Resources• Any additional connections

– Answer the 3 questions below– 5 minutes– Large group debrief

1.Who answers these questions for your school?

2.Are your teachers engaged in responding to these questions? How?

3.How do you address the issue of evidence for each question?

Engaging Teachers: Getting Closer to the Truth

• Strategy to Engage Teachers• Uncovering the Truth• Use question cards

– Each of you get a set to take with you– Also on jump drive

• Let’s Watch!• CV Clips:

– Question Discussion (1:35)– Prioritizing (0:30)


• Using question cards• Ask whole group the question

– Clarify as needed• Example: If question is about

“diagnostic assessments,” you may need to define what is meant by that term in the question or how it will be defined for the discussion.

– Lots of pausing– Facilitator role – they answer their

own questions


• Sort questions into 2 groups1. Doing well, needs some fine tuning

2. In need of improvement

• Can sort on chart, poster board, electronically• Must generate evidence for reason

question/foundational practice goes in category 1 or category 2

• If NO evidence exists….In need of improvement • After all questions have been asked and

categorized – prioritize areas of focus– What will yield greatest results?

Local Experience

• Lori Stollar, Lincoln IU 12

• Sharing her experiences with school-wide teams

answering these questions

School Level Planning

Questions? Comments?




Focus of Analysis: Grade/Course

Purpose of Analysis– Plan to Improve Student Achievement and At

Least Meet AYP Targets by Grade Level– Analyze Grade-Level Results

• Not student level, different meeting purpose– Set Grade-Level Goals– Ensure Instructional Program

Coherence/Alignment• Curriculum• Instruction• Assessment• Supports (infrastructure, including scheduling, staffing, etc.)

– Monitor Implementation and Effectiveness of Grade-Level Program

Grade Level/Course Specific Teams

• Why???– Do not just jump to student level and

miss grade level discussions– Trees v. leaves

• Curriculum, Assessment, Instruction, Organization Issues

– May address student needs versus one student at a time

• How do our grade level results compare to others/ Across subjects?

HOW DID WE DO?WHAT DO WE NEED TO DO?

Grade/Course Level Teaming

Data Packets: Grade LevelTimingTiming SourceSource ReportReport

Initial Meeting – Looking Back

eMetric

eMetric

PVAASPVAAS

PVAASPVAAS

eMetric/PVAASeMetric/PVAAS

Performance Levels

Reporting Categories

Value-Added ReportValue-Added Report

Performance Diagnostic ReportPerformance Diagnostic Report

PSSA/PVAAS Scatter PlotsPSSA/PVAAS Scatter Plots

Initial Meeting– Looking Forward

PVAASPVAAS

4Sight Baseline

4Sight Baseline

CDT Baseline

Projection Summary ReportProjection Summary Report

Performance Levels

Subscale Reports

Initial Meeting– Subgroup Comparisons

Same as above Same as above

Periodic Meetings 4Sight

4Sight

CDTs

Local Formative

Assessment

Performance Levels

Subscale Reports

70PDE SAS Institute, December 2010

eMetric Reporting Categories: Multi Year

PVAAS Value-Added: How do we compare to others?

PVAAS Performance Diagnostic Report

PVAAS Projection Summary: Who is on the right trajectory? Who needs support?

PVAASCustom

Diagnostic Report

Seeing that Others are Yielding Different Results

Let’s Talk More About:PVAAS Performance Diagnostic Report

Divide and Conquer!...aka, Jigsaw

PDE SAS Institute, December 2010

78

•Count off at your table by 6’s•Each person reviews one PVAAS Performance Diagnostic Report as indicated on the next 6 slides•Respond to these 3 questions:

•What pattern does this report reveal?•How would this pattern impact the achievement gap if applied to a cohort over time?•What questions would a grade level team ask as they reviewed this report? What would they want to investigate further?

•Different results mean different questions/focus

1: Patterns of Growth:1: Patterns of Growth:Upward ShedUpward Shed

This pattern occurs when school learning experiences benefit higher-achieving students more than their lower-achieving peers. This tends to occur where the educational program is benefitting those students at the higher end of the achievement spectrum. If this pattern continues over time with the same student cohort, the achievement gap will widen.

2: Patterns of Growth:2: Patterns of Growth:Downward Shed (1)Downward Shed (1)

This pattern occurs when school learning experiences benefit lower-achieving students more than their higher-achieving peers. This pattern may occur more often in schools with a “pure” AYP focus. The educational program may be focused on low-achieving students whose needs are more obvious than those students at the upper end of the achievement spectrum. In this case, many of the high-achieving students are not being challenged. Maintaining this pattern in the earlier grade levels can results in fewer higher-achieving student s in later grade levels.

3: Patterns of Growth:3: Patterns of Growth:Downward Shed (2)Downward Shed (2)

This pattern occurs when the educational program benefits all students as all students are making positive gains. However, the lowest achieving students are making the most gains. Over time this pattern has the potential to close the achievement gap.

4: Patterns of Growth:4: Patterns of Growth:Tent or TeepeeTent or Teepee

This pattern occurs when the educational program benefits middle-achieving students more than their lower-achieving and higher-achieving peers. For many reasons, the system is not appropriately addressing the needs of all students, but rather has a focus on the middle-achieving students.

5: Patterns of Growth:5: Patterns of Growth:Flat (1)Flat (1)

This pattern occurs when the educational program benefits all students of all achievement levels similarly. This pattern illustrates that all students are gaining more than a year’s worth of growth for a year of schooling.

6: Patterns of Growth:6: Patterns of Growth:Flat (2)Flat (2)

This pattern is of greatest concern as all students are showing negative gains. In other words, all blue bars (including the red whisker line) are well below the green reference line, or one year’s worth of growth (value of 0). In this case, all students are making less than one year’s worth of growth for one year of schooling.

Line Of Inquiry:Questions for Grade-Level Teams

There are two main questions to ask: Are we serving all students adequately? Are we serving all students equitably?

Questions to ask about the entire grade level:

1. Where are we doing well? Which groups are showing a significant positive gain, or are showing significantly higher than expected scores?

2. Where are we not doing as well? Which groups are showing negative gain, or are showing significantly lower than expected scores?

3. What patterns are you seeing across the groups?

4. Is there a change from previous reports? Does this tell you anything about the school system, specifically the curriculum and instruction in the subject area of interest?

Grade Level Teams

• Give them the data needed AND the questions to ask

• Patterns and trends• Then apply this/keep these findings in

mind throughout the year when using benchmark as assessments, diagnostics assessments, etc.

Grade Level Teams in Action• 2 hour session• 6th grade teams from each elementary

– Approx 4 per team– Regular Ed– Special Ed– Principal– Assistant Supt of Elementary

• Started with some intro to growth• They reviewed their own results

– Provided interpretation

• Let’s watch several clips:– PM Clips: _______

PVAAS Custom Diagnostic Report

What is it?– Procedure to examine subgroup growth patterns

based on user-defined educational criteria for groups of 15 or more students.

How might a school use this report?– Explore effects of intervention programs, etc.– Explore effects of varied curricular and/or

instructional experiences.Cautions

– This report does not infer any causal relationships between any educational variables and student growth.

– This report should NOT be used to estimate teacher effect on learning.

Custom Diagnostic Report

Student Selection Screen

Custom Diagnostic Report: Let’s Try It!


Grade/Course Level PlanningQuestions? Comments?



STUDENT LEVEL PLANNINGMeeting Student Needs

Purpose of Analysis: The Student

• Purpose of Analysis:– Plan to meet proficiency or beyond

– Student level goal setting

– Monitor student level learning and response to instruction and intervention

– Adjust instructional practices

What Does Student-Level Planning Look Like Now?

– Does it occur Quarterly? Monthly? • Elementary, Middle, High School

– Who is at the table?• Principals• Teachers (Regular Educators, Special Educators)• Others

– Is there an established protocol?• Which students are reviewed?

– Who leads process?– What data are used?– Does this result in an action plan?– Is the action plan followed? – Is the process monitored to ensure strategic use of

time?

Student Level Planning

• Data Profiles- Multiple Data Sources• Lines of Inquiry• Entrance/Exit Criteria• Ownership by each team member• Engaging team members

– The right people at the table

• Building Leadership• Time…Time….Time• Video clips of actual team meetings

– grade 5 teams

Quick Review: PVAAS Student Projection Report

See example:Grade 4-8 packet - pg. 20Grade 11 packet - pg.16

PVAAS Fall 2009 Reporting Projections to Basic, Proficient, and Advanced

Grade Last Tested

Math and Reading: 3 4 or 5 4 5 or 6 5 6 or 7 6 7 or 8 7 8 8 11

Writing: 3 5 4 5 5 8 6 8 7 8 8 11

Science:

3 4 4 NA 5 8 6 8 7 8 8 11

Grade Last Tested Grade Last Tested

How do I know that the projectionis accurate?

• The projection is precise, and is created using a model that has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office).

• The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study.(http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html)

• Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score.

http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html

Why is there no individual measure of student growth?

PVAAS does NOT estimate growth for one student because…

• Observed scores are comparisons of “snapshots in time” that are very unreliable…

• The error in an estimate for a data set with only one record is too large to make the estimate meaningful…

• Error depends on variation in data AND the sample size (the number of student records in the data set).

Multiple Data Sources:Elementary Scenario/Reading

Student NOW in 5th Grade

PSSA

Grade 4 Proficient Range: 1255-1468

1210

4Sight Baseline

Grade 5 Proficient Range: 1275-1496

1200

PVAAS PVAAS ProjectionsProjections

To Basic:To Basic:

To Proficient:To Proficient:

To Advanced:To Advanced:

93.6%93.6%

64.8%64.8%

5.5%5.5%

DIBELS

ORF

Grade 5 – Low Risk

104+ at beginning of year

124+ at end of year 89

Some Risk


Multiple Data Sources:High School Scenario/MathStudent NOW in 9th Grade

PSSAGrade 8 Proficient Range

1284-1445 1450

4Sight Baseline

Grade 9 Proficient Range 1304-1508 1507

PVAAS PVAAS ProjectionsProjections

(11(11thth Grade) Grade)

To Basic:To Basic:

To Proficient:To Proficient:

To Advanced:To Advanced:98.7%98.7%

89.9%89.9%

45.7%45.7%

Grade 8 Final Grades

Final Exam: 91%

Course Grade: 87%


Line Of Inquiry:Questions for Student-Level Planning

There are two main questions to ask: Is this student currently on a trajectory to be

Proficient or Advanced on the next PSSA assessment?

Are we satisfied with the probability that this student may be at least Proficient on his/her next PSSA assessment?

What about working towards Advanced?

Line Of Inquiry:Questions for Student-Level Planning

Additional questions to ask:1. Does the school have the appropriate courses available to meet

this student’s needs? Is the appropriate time allocated for each subject area?

2. What is the schedule of courses? Is it fitting this student’s needs?

3. Does this student have access to the appropriate interventions and resources he/she needs? If yes, is the student participating in those interventions? What are we doing to ensure the student is taking advantage of the available resources?

4. Are we providing enough challenging instruction to this student?5. Are we providing materials at the appropriate instructional level

and with appropriate rigor and relevance?6. What supports are available to the parents of this student?

Turn to your table team and discuss:

• Does your school have a structure/system for student level planning/student-focused data meetings?– How frequently?

• Who is at the table?


• Focus on one district’s student data planning system as the springboard for our discussions– Elementary School– Middle School– High School

• NOT the only way, not the perfect model, not saying any district must do it this way – only intended to serve as a REAL example to spark today’s discussions and inquiry

Frequency: Proficiency Planning

Principal Leadership is EssentialFrequency:

•Grades K & 1: Monthly•Grades 2-5: Quarterly•Middle School: Three times per year•High School: Three times per year•This same cycle of meetings occurs

for reading and math separately

PDE PVAAS Statewide TeamWinter 2010

Comprehensive System for Students

Grade 4 Example:• Individual Student Planning Meetings

− 4X/year reading; 4X/year math− Classroom teacher; one at a time

• “Pouring Over Data”(POD) meetings− 2X/month reading; 2X/month math− Small group of teachers− PLC format – to study instruction

• Data Meeting− 2X/month for social/emotional needs

Progress Monitoring

• All classroom teachers in this school district, K-5, engage in literacy progress monitoring– Weekly, bi-weekly, every 3 weeks

basis

– Required for all students below benchmark

– Some teachers have moved to progress monitoring all students

PDE PVAAS Statewide TeamWinter 2010

Building Admin Leadership in Student Planning Process

• Present

• Participating

• Priority

• Let’s listen to the principal!

WHAT IS IN YOUR DATA PROFILES?


Example of Student Data Profile

Examples of local/district data management tools or warehouses such as, but not limited to:– CDA– EdInsight (OnHand Schools)– eScholar– Performance Tracker– TetraData– Others

Elementary Profile

Elementary Profile Continued on page 2

Student Data Profiles

• Technology as a tool to enhance the process

• Let’s listen to the principal!– DT Clip # ( )

Middle School Profile: Math

Middle School Profile: Reading

High School Profile DRAFT: Math

High School Profile DRAFT: Reading

Student Data Profiles

Table Discussion:1.How do these data profiles compare to the data your teams use during student data planning meetings?

• Be prepared to share with the large group

PROTOCOLSLINE OF INQUIRYQUESTIONS TO ASK


Lines of Inquiry – Questions to Ask

Table Discussion:

1.Who facilitates your student level planning meetings?

2.How do team members know what questions to ask?

3.Be prepared to share with the large group

ELEMENTARYLines of Inquiry

Let’s Take a Look!

• What can you expect to see?– Who is at the table?– Data?– Focus?– Meeting sequence?– Results of meeting?

• DT CLIP: Student #1 ( insert time)

Individual Proficiency Meetings (IPM)Initial Meeting of the Year

Format ASlide 1 of 4

--Continuation of table on previous slide--



\Progress Monitoring Meetings for previous students

Round 2 and 3Format B

Slide 1 of 4

Progress Monitoring Meetings for previous students

Round 2 and 3Format B

Slide 2 of 4

Progress Monitoring Meetings for previous students Round 2 and 3

Format B Slide 3 of 4

Continuation of table from previous page

Progress Monitoring Meetings for previous students Round 2 and 3

Format B Slide 4 of 4

Individual Proficiency Initial and Progress Monitoring Meetings

Diagnostic Questions for DIBELS subtests 1 of 3


Diagnostic Questions for DIBELS subtests 2 of 3

Derry Township Individual Proficiency Initial and Progress Monitoring MeetingsDiagnostic Questions for DIBELS subtests3 of 3


Diagnostic Questions for DIBELS subtests (Set 2) Slide 1 of 3

Let’s Listen In!

• Let’s listen to one more data team meeting!

MIDDLE SCHOOLLines of Inquiry

Middle/High School Student Meetings

• Current Achievement Levels?– Scores/Data

• Tier One Status?• Interventions Tier 2/3 Response to Intervention?• Actions Needed?

Tier 1: In preparation for the upcoming IPMs, we will want to have discussion about individual students’ challenges with reading that you observe as you

work with them in the regular classroom. Please be prepared to

discuss any of the following aspects of the reading process as you observe

the student’s performance in your class.

Fluency

• What have you observed in this student with regards to the student’s:– Appropriate phrasing, expression, and pace in

reading? Does the student’s reading sound smooth? Is it easy to listen to?

– Accuracy in reading (i.e. does the student skip or insert words/phrases; misread words)– therefore causing interference in meaning?

Vocabulary

• What have you observed in the student with regard to his/her:• Understanding of the meaning/context of

specific vocabulary? (could be vocabulary in directions or in the actual content area)

• Accurate use of words in speaking/writing or accurate understanding of words?

• Ability to connect meanings of related words? (ex. understand that “Commutative” is related to the word “commute” & how it is related)

• Background knowledge – does the lack of understanding interfere with ability to understand things read?

Comprehension

• What have you observed in the student with regard to his/her:• Ability to get “the gist” of what he/she

reads independently? (i.e. can he/she summarize, restate, find the main idea, take notes on critical information) or from class discussions and lectures?

• Ability to make connections to other information/understandings and make inferences?

Follow-up

• After the IPMs, we will follow-up by team discussing and identify strategies that will allow those students who are struggling to be more successful in your class AND to make progress in reading.

Tier 1: IPMs Student Reporting SheetStudent:

•No concerns – Let’s monitor•Things specifically being done in class that are helpful to this struggling learner:•Small group instruction one to one instruction paired assistance •Hands on work read directions aloud read tests/quizzes aloud•Oral assessments opportunity for test retakes graphic organizers/study guides •Anticipatory guides opportunities for review modified assignments/tests

HIGH SCHOOLLines of Inquiry

High SchoolUniversal Screening Process & Entrance Criteria

PSSA-PVAAS-4Sight

IPM #1


Student Level PlanningQuestions? Comments?



Your Thoughts from Today’s Session

Index Card:

• 2 “Take Aways”

• 1 Next Step

• Share via whip-around with your table team

• Leave card on table

data tools in a pa standards aligned system session facilitators : pde pvaas statewide team kristen...

Documents

students proficient

percent proficient

session participants

pas data tools

datadriven decisionmaking

meaningful data analyses

year of growth

session description