pvaas f2f overview: evaluating growth, projecting performance
DESCRIPTION
PVAAS F2F Overview: Evaluating Growth, Projecting Performance. Fall 2013. Goal of Session. By the end of the session, participants will be able to answer the following questions: What is PVAAS? How does PVAAS measure growth? What does PVAAS tell us about growth in our district and school(s)? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/1.jpg)
PVAAS F2F Overview:Evaluating Growth,
Projecting Performance
Fall 2013
![Page 2: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/2.jpg)
F2F Overview 2013
Goal of SessionBy the end of the session, participants will be able to answer the
following questions:
• What is PVAAS?
• How does PVAAS measure growth?
• What does PVAAS tell us about growth in our district and school(s)?
• What does PVAAS tell us about how students are projected to perform on a future state assessment?
2
![Page 3: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/3.jpg)
F2F Overview 2013
WHAT IS PVAAS?
3
![Page 4: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/4.jpg)
F2F Overview 2013 4
What is PVAAS?
• Longitudinal analysis of existing state assessment data
• Reliable estimates of academic growth & projections of future performance
• Effectiveness of the educational system
![Page 5: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/5.jpg)
F2F Overview 2013 5
What is PVAAS?
• Growth of a group of students over time• NOT a simple comparison of test scores
Grade 3
Grade 4
Grade 5 Grade 6
![Page 6: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/6.jpg)
F2F Overview 2013
PVAAS Addresses Complex Issues!
Research-based Statistical Modeling
•Minimizes the effects of measurement error associated with any one test scoreMeasurement
Error
•Accounts for mobility of studentsMobility
•Compensates for missing dataMissing Data
6
![Page 7: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/7.jpg)
F2F Overview 2013
Demographics and Growth
As long as you can sufficiently account for the prior achievement of the student,…
7
…there is NO significant relationship between demographics and student progress, or growth!
![Page 8: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/8.jpg)
F2F Overview 2013
Prior AchievementHow can we adequately account for prior achievement?
• The SAS EVAAS team has found that the only way to adequately account for prior achievement is to use ALL available test data for each student to dampen the effects of measurement error.
• PVAAS does NOT need to adjust for the demographic status of students directly since it is accounting for students’ prior achievement.– In other words, demographics are inherently controlled for in
PVAAS by the use of all longitudinal data on all students.
8
![Page 9: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/9.jpg)
ActivityWork with a partner to decide if each statement is true or false.
9F2F Overview 2013
![Page 10: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/10.jpg)
F2F Overview 2013 10
PVAAS Analyses
PSSA Writing
Keystone Exams:Algebra I, Biology,
Literature
PSSA Science
PSSA Reading
PSSA Math
LOOKING AHEADPlanning for
Students’ Needs:Student
Projections to Future Tests
LOOKING BACKEvaluating School
Effectiveness:Value Added &
Diagnostic Reports
What is PVAAS?
![Page 11: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/11.jpg)
F2F Overview 2013
PVAAS Value-Added ReportingHow is growth measured in PVAAS?
11
![Page 12: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/12.jpg)
PVAASValue-Added Reporting
Testing Subjects – Grades Methodologies
Assessments in consecutive
years
Mathematics, 4-8Reading, 4-8 Growth Standard
Assessments not in consecutive
years
Writing – 5 & 8 Science – 4 & 8
Keystones – Algebra I, Biology, & Literature
Predictive
12F2F Overview 2013
![Page 13: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/13.jpg)
F2F Overview 2013
Measuring Academic Growth
Students don’t all start the year at the same place academically
13
![Page 14: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/14.jpg)
F2F Overview 2013
Measuring Academic Growth
• Standard for PA Academic Growth– Regardless of their entering achievement level, students
should not lose ground academically
• Is it not a reasonable goal to maintain the achievement level of a group of students?– Not necessarily increase their achievement level, but
maintain their achievement level.
14
![Page 15: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/15.jpg)
F2F Overview 2013 15
PVAAS Value-AddedGrowth Indicators
![Page 16: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/16.jpg)
F2F Overview 2013
Measuring Growth in PVAAS• Remember, PVAAS does NOT compare the
achievement of one group of students in a particular subject and grade with the previous year’s students in the same subject and grade.
• PVAAS does NOT use the percentages of students at various academic performance levels to measure growth.
• PVAAS uses students’ scale scores within those performance levels to better estimate growth.
16
![Page 17: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/17.jpg)
F2F Overview 2013 17
Growth Measures Not Reportedfor Prior Years
• PA’s state assessment system is transitioning to include Keystone end-of-course exams and statewide assessments aligned to the PA Common Core over the next several years.
• Growth Measures in prior years will NOT be included in fall 2013 reporting (SY12-13).
• Anticipated that Growth Measures from up to 3 previous years with 3-year averages will be yielded and reported beginning with fall 2015 reporting (SY14-15).
![Page 18: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/18.jpg)
F2F Overview 2013
PVAAS Value-AddedGrowth Measures
• For Science, Writing and Keystone content areas, Growth is measured in scale score points.
• The Growth Measure is NOT reported in years and cannot be referenced in terms of number of year’s growth.
• The Growth Measure is NOT reported in percentages and cannot be referenced in terms of an increase or decrease in percentage points.
18
![Page 19: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/19.jpg)
F2F Overview 2013
Purpose of Value-Added & Performance Diagnostic Reports
Assessing curricular needs across grades, subjects, and Keystone content areas
Evaluating instructional needs of students at various achievement levels and in different subgroups
Ensuring instructional coherence/alignment• Curriculum Instruction Assessment• Supports, including scheduling and staffing needs
Monitoring implementation and effectiveness of grade-level or course-specific programs
19
![Page 20: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/20.jpg)
F2F Overview 2013
What Are Projections?
Given a specific set of circumstances…
…what’s the most likely outcome?
20
![Page 21: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/21.jpg)
F2F Overview 2013
Setting Expectations
How do teachers set expectations for their students?
• Informationo What do they know about this
student?
• Experienceo How have students like this one
performed in the past?
21
![Page 22: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/22.jpg)
F2F Overview 2013 22
PVAAS ProjectionsWhat are they based on?
Expectations based on what we know:
About this student:
»Prior test scores
»Across subjects
![Page 23: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/23.jpg)
F2F Overview 2013 23
PVAAS ProjectionsWhat are they based on?
About other students:
» Prior test scores across subjects
» Scores on the test to which we are projecting
![Page 24: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/24.jpg)
F2F Overview 2013
Given this student’s testing history, across subjects…
…what is the student likely to score on an upcoming test, assuming the average schooling experience?
What Are PVAAS Projections?
24
![Page 25: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/25.jpg)
F2F Overview 2013
Student Projection Methodology
What data are used in calculating projections?
25
Projection TO…. Data Used to Calculate ProjectionPSSA Math PSSA Math and ReadingPSSA Reading PSSA Math and ReadingPSSA Science PSSA Math, Reading, and Science (in grades available)PSSA Writing PSSA Math, Reading, and Writing (in grades available)Keystone content areas (Algebra I, Literature, Biology)
PSSA Math, Reading, and Science (in grades available)
![Page 26: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/26.jpg)
F2F Overview 2013
Student Projection Methodology
Which students have projections? In what subjects?
– Projections are available for all students who have a minimum of 3 data points. The exception to this are projections to grade 4 where only 2 data points from grade 3 are needed.
– The next two slides detail the grade levels, subjects, and Keystone content areas in which projections are reported.
26
![Page 27: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/27.jpg)
PVAAS Projection Reporting: Projections to Keystones (Basic, Proficient, Advanced)
27
• Anticipating projections to future Keystone exams will be available to students who last tested in grade 5
• Remember, projections to a Keystone exam are NOT grade specific
• The projection probability indicates the probability of a student reaching the indicated performance level on a future Keystone exam – not necessarily at a specific grade level.
F2F Overview 2013
![Page 28: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/28.jpg)
F2F Overview 2013
PVAAS Student Projection Report
28
![Page 29: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/29.jpg)
F2F Overview 2013
PVAAS Student Projection Report
29
![Page 30: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/30.jpg)
F2F Overview 2013
PVAAS Student Projection Report
30
![Page 31: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/31.jpg)
F2F Overview 2013 31
LOOKING AHEAD
LOOKING BACK
![Page 32: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/32.jpg)
F2F Overview 2013
PA’s Intent for PVAAS
• Measure of Growth to Add to Existing Achievement Measures
• Indicator of Likelihood of Future Student Performance to Add to Current Achievement Data
• Another Tool for Continuous Improvement
• Not to Be Used in Isolation
32
![Page 33: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/33.jpg)
F2F Overview 2013 33http://www.education.state.pa.us/pvaas
Additional PVAAS Supports
![Page 34: PVAAS F2F Overview: Evaluating Growth, Projecting Performance](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681619f550346895dd159d6/html5/thumbnails/34.jpg)
www.pde.state.pa.us
PENNSYLVANIAValue Added Assessment System
______________________________________________________________________
PVAAS
333 Market Street Harrisburg, PA 17126