Download - Using Data to Improve Student Achievement
Using Data to Using Data to Improve Student Improve Student
AchievementAchievement
Summer 2006 PreschoolSummer 2006 PreschoolCSDCCSDC
OutcomesOutcomes Know why we need to look at dataKnow why we need to look at data Identify two types of testsIdentify two types of tests Understand three types of scoresUnderstand three types of scores Understand Summative & Formative Understand Summative & Formative
AssessmentsAssessments Be able to interpret Summative Be able to interpret Summative
Assessment ReportsAssessment Reports Know how to use data in instructional Know how to use data in instructional
planning for increased student planning for increased student learninglearning
OutcomesOutcomes
Understand how students are placed Understand how students are placed into High School Intensive Reading into High School Intensive Reading coursescourses
Review materials published by Great Review materials published by Great SourceSource
Align assessments included within Align assessments included within curriculum to drive instructioncurriculum to drive instruction
Develop lesson plansDevelop lesson plans
Guide to the ProgramGuide to the Program 1) Plan1) Plan
Lesson Plan BooksLesson Plan Books 2) Teach2) Teach
Teacher’s GuideTeacher’s Guide Overhead TransparenciesOverhead Transparencies Website – www.greatsource.comWebsite – www.greatsource.com
3) Practice3) Practice Student Applications BookStudent Applications Book Independent PracticeIndependent Practice WebsiteWebsite
Great Source Material Great Source Material OverviewOverview
Lesson Plan BookLesson Plan Book Teacher’s Guide (all levels)Teacher’s Guide (all levels) Student Application BookStudent Application Book Student Application Book, Teacher’s EditionStudent Application Book, Teacher’s Edition Reader’s Handbook (all levels)Reader’s Handbook (all levels) Test BookTest Book Overhead Transparencies (all levels) Overhead Transparencies (all levels) WebsiteWebsite SourcebookSourcebook Sourcebook, Teacher’s EditionSourcebook, Teacher’s Edition Sourcebook Florida Diagnostic Tests Sourcebook Florida Diagnostic Tests
(optional/available by site)(optional/available by site)
Lesson Plan BookLesson Plan Book
Gives day-by-day and week-by-week Gives day-by-day and week-by-week lesson planslesson plans
Shows how to use Shows how to use Reader’s HandbookReader’s Handbook to set-up a complete reading to set-up a complete reading curriculum curriculum
Curriculum plan suggests year-long Curriculum plan suggests year-long planplan
Individual lesson plans outline weekly Individual lesson plans outline weekly and daily lesson plansand daily lesson plans
Teacher’s GuideTeacher’s Guide
Walks through each lesson in the Walks through each lesson in the Reader’s HandbookReader’s Handbook
Highlights what to teachHighlights what to teach Suggests ways to extend the lessonSuggests ways to extend the lesson
““How to Use a Teacher’s Guide How to Use a Teacher’s Guide Lesson”Lesson”
Pages 16 – 20Pages 16 – 20
Student Application BookStudent Application BookStudent Application Book, Student Application Book,
TETE
Extends the lessons with a new Extends the lessons with a new selection for students to work throughselection for students to work through
Lessons let students apply the reading Lessons let students apply the reading strategies and tools to a new selection, strategies and tools to a new selection, give them guided practice, and help you give them guided practice, and help you assess their understandingassess their understanding
Reader’s HandbookReader’s Handbook
HANDBOOK- NOT A TEXTBOOK!HANDBOOK- NOT A TEXTBOOK! What is the purpose?What is the purpose?
Guides students as they read informational Guides students as they read informational texttext
Students should use this book to:Students should use this book to: Look up information prior to or while Look up information prior to or while
readingreading Develop new strategies to improve readingDevelop new strategies to improve reading Get to know different types of textGet to know different types of text
Test BookTest Book
Contains two types of tests for each Contains two types of tests for each topictopic
Assesses students’ understanding of Assesses students’ understanding of skills and strategiesskills and strategies
Tests can be used as diagnostic, Tests can be used as diagnostic, formative, or summative assessmentsformative, or summative assessments
Overhead TransparenciesOverhead Transparencies
Display key concepts presented in Display key concepts presented in the handbookthe handbook
48 color transparencies48 color transparencies
WebsiteWebsite
www.greatsource.comwww.greatsource.com Great Source Homepage Great Source Homepage
www.greatsource.com/rehandwww.greatsource.com/rehand Reader’s Handbook WebsiteReader’s Handbook Website
www.greatsource.com/floridawww.greatsource.com/florida Florida Diagnostic Sourcebook – Florida Diagnostic Sourcebook –
provides 9provides 9thth grade SR and ER grade SR and ER
SourcebookSourcebookSourcebook TESourcebook TE
Incorporates four approaches:Incorporates four approaches: ComprehensiveComprehensive Strategy IntensiveStrategy Intensive Literature BasedLiterature Based InteractiveInteractive
Can be used as a formative assessment Can be used as a formative assessment tooltool
Each of the 24 selections includes: Each of the 24 selections includes: Before, During, and After Reading Before, During, and After Reading strategies, vocabulary and assessmentsstrategies, vocabulary and assessments
Sourcebook Florida Sourcebook Florida Diagnostic TestsDiagnostic Tests
PretestPretest Interim Test OneInterim Test One Interim Test TwoInterim Test Two Interim Test ThreeInterim Test Three PosttestPosttest
Why Look at Data?Why Look at Data?
The purpose of data is to The purpose of data is to
give educators give educators
INSIGHT!INSIGHT!
Types of TestsTypes of Tests
Norm-Referenced Test (NRT)
Criterion-Referenced Criterion-Referenced Test Test (CRT)(CRT)
What is a Norm-What is a Norm-Referenced Test (NRT)?Referenced Test (NRT)?
A standardized A standardized assessment in which assessment in which all students all students perform under perform under the same conditionsthe same conditions
It compares the performance of a It compares the performance of a student or student or group of students to a national group of students to a national sample of students at the same grade sample of students at the same grade and age, called the norm groupand age, called the norm group
What is a Criterion-What is a Criterion-Referenced Test (CRT)?Referenced Test (CRT)?
An assessment comparing one student's An assessment comparing one student's performance to a specific learning performance to a specific learning objective or performance standardobjective or performance standard and and notnot to the performance of other students. to the performance of other students.
It tells us how well students are It tells us how well students are performing on specific goals or content performing on specific goals or content standards rather than how their standards rather than how their performance compares to a national or performance compares to a national or local norming group.local norming group.
Summary NRT and CRTSummary NRT and CRT
TYPE Norm-referenced
Test (NRT) Criterion-ref erenced
Test (CRT)
DEFINITION
Shows how a student does in relation to a norm group.
Shows how a student does in relation to a standard.
EXAMPLES
FCAT NRT SAT/ 10 TerraNova PSAT/ NMSQT
FCAT SSS Classroom Tests AP Exams District Math
Assessments
Types of ScoresTypes of Scores
Raw Score (RS)Raw Score (RS)
The number of items a student The number of items a student answers correctly on a test.answers correctly on a test.
John took a 20 item mathematics test John took a 20 item mathematics test (where each item was worth one (where each item was worth one point) and correctly answered 17 point) and correctly answered 17 items. items.
His His raw scoreraw score for this assessment is for this assessment is 17.17.
Scale Score (SS)Scale Score (SS)
Mathematically converted raw scores Mathematically converted raw scores based on level of difficulty per based on level of difficulty per question question
For FCAT-SSS, a computer program is For FCAT-SSS, a computer program is used to analyze student responses and used to analyze student responses and to compute the scale scoreto compute the scale score
Scale Scores reflect a more accurate Scale Scores reflect a more accurate picture of the student’s achievement picture of the student’s achievement levellevel
Developmental Scale Developmental Scale Score (DSS)Score (DSS)
ReadingReadingGrade Level
1Level
2Level
3Level
4Level
58 886-1695 1696-
18811882-2072
2073-2281
2282-2790
9 772-1771 1772-1971
1972-2145
2146-2297
2298-2943
10 844-1851 1852-2067
2068-2218
2219-2310
2311-3008
Current Current GradeGrade
DSSDSS
Student Student AA
99 22242224
Student Student BB
1010 18351835
Student Student CC
1111 16751675
Grade Grade EquivalenEquivalen
tt
1010thth
88thth
66thth
High School High School Intensive Reading Intensive Reading
PlacementPlacement
If If LANGUAGE!LANGUAGE! Placement Placement Assessment Assessment
indicates placement indicates placement into into LANGUAGE!LANGUAGE! Levels A/B, C/D or Levels A/B, C/D or
E/F– E/F–
2 periods2 periods
If If LANGUAGE!LANGUAGE! Placement Placement Assessment Assessment
indicates placement indicates placement beyond beyond
LANGUAGE!,LANGUAGE!,
Great Source -Great Source -
1 period1 period
Incoming L1 & L2 students on 2006 FCAT
Note: Incoming 11th & 12th graders who scored above 1926 will be able to take a 10th grade fluency assessment to be placed out of the Intensive Reading requirement..
Gain ScoresGain Scores
Commonly referred to as “Learning Gains”
The amount of progress a student makes in one school year.
Learning Gains: Who Qualifies?
All students with a pre- and post-test, including all subgroups (ESE, LEP, etc.).
All students with matched, consecutive year (i.e. 2005 & 2006) FCAT SSS results, grades 4-10, who were enrolled in the same school surveys 2 & 3 (FTE).
Learning Gains: Which Scores?
Gains apply in reading and math, not writing or science.
Pre-test may be from same school, same district, or anywhere in the state.
Learning Gains:What equals Adequate
Yearly Progress (AYP)? A. Improve FCAT Achievement Levels from 2005
to 2006 (e.g. 1-2, 2-3, 3-4, 4-5)
OR
B. Maintain “satisfactory” Achievement Levels from 2005-2006 (e.g. 3-3, 4-4, 5-5)
OR
C. Demonstrate more than one year’s growth within Level 1 or Level 2 - determined by DSS Cut Points (not applicable for retained students)
Developmental Scale Score
Gains Table (DSS Cut Points)Students achieving within Level 1 (or within Level 2) for two consecutive years
must gain at least one point more than those listed in the table in order to satisfy the “making annual learning gains” component of the school accountability system.
Grade Level Change Reading Mathematics
3 to 4 230 162
4 to 5 166 119
5 to 6 133 95
6 to 7 110 78
7 to 8 92 64
8 to 9 77 54
9 to 10 77 48
Learning Gains: Retainees
A retained student can only be counted as making adequate
progress if he/she:
Moves up one level. (e.g. 1-2, 2-3, 3-4, 4-5)
Maintains a level 3, 4, or 5.
Learning Gains: Activity
Using the data on the following table, determine:
which students made a learning gain
what percentage of the teacher’s students made a learning gain
Data Display for Data Display for FCAT Reading ResultsFCAT Reading Results
Student 04/05 Grade Level
05/06Grade Level
Pre-test Achievement Level
Pre-test DSS
Post-test Achievement Level
Post-test DSS
Learning Gain Determination
A 7 8 Level 1 Level 2 Yes or NoReason: A, B, or C
B 7 8 Level 4 Level 4 Yes or NoReason: A, B, or C
C 7 8 Level 2 1598 Level 2 1743 Yes or NoReason: A, B, or C
D 8 8 Level 1 Level 2 Yes or NoReason: A, B, or C
E 8 8 Level 3 Level 3 Yes or NoReason: A, B, or C
F 8 8 Level 1 1486 Level 1 1653 Yes or NoReason: A, B, or C
G 7 8 Level 5 Level 4 Yes or NoReason: A, B, or C
Teacher Learning Gains Based on Data Display
5 out of 7 students made learning gains. 71% of this teacher’s students made learning gains
and add points towards the school’s grade. No points are given to the school for Student F
because he was retained and stayed within level 1 – even though he made significant gains in DSS points.
No points are given to Student G because he decreased a level.
Total Number of Students with a Pre and Post-test who qualify for learning gain calculations:
Reason AIncreased 1 or more Achievement Levels
Reason BMaintains “satisfactory” levels (3, 4, or 5)
Reason CDSS Target Gain (More than a year’s growth)
7 2 2 1
Class Record Sheet Class Record Sheet for for
Learning GainsLearning Gains
Microsoft Word Document
Types of DataTypes of Data
Results (Summative)
Data used to make decisions about student achievement at the end of a period of instruction.
Process (Formative)
Data gathered at regular intervals during the instructional period; used to provide feedback about student progress and to provide direction for instructional interventions.
Summative DataSummative DataContinuedContinued
FCAT FCAT Great SourceGreat Source
Florida Diagnostic Tests (Sourcebook)Florida Diagnostic Tests (Sourcebook) Post TestPost Test
Reader’s Handbook Test BookReader’s Handbook Test Book Multiple Choice/Short Answer test pages for Multiple Choice/Short Answer test pages for
each topic.each topic. Multiple topic tests can be combined in order Multiple topic tests can be combined in order
to create a semester exam.to create a semester exam. Teacher’s Guide Source BookTeacher’s Guide Source Book
Fourth black-line master page of each story.Fourth black-line master page of each story.
A Closer Look at Results A Closer Look at Results DataData
Examples:
FCAT Parent Report
Group ActivityGroup ActivityHow do parents get these reports?How do parents get these reports?
Did this student pass the 10Did this student pass the 10thth grade test? grade test?
In what grade did this student first achieve In what grade did this student first achieve grade level mastery?grade level mastery?
Which content area had the most questions?Which content area had the most questions?
Using the bar graph, how does this student’s Using the bar graph, how does this student’s achievement compare to grade level?achievement compare to grade level?
A Closer Look at A Closer Look at Formative DataFormative Data
Quizzes
Chapter Tests
DIBELS
District Math Assessments
Formative Data ExamplesFormative Data ExamplesGreat SourceGreat Source
DefinitionDefinition: : Data gathered at regular intervals Data gathered at regular intervals during the instructional period; used to provide during the instructional period; used to provide feedback about student progress and to provide feedback about student progress and to provide direction for instructional interventions.direction for instructional interventions.
Florida Diagnostic Tests (Sourcebook)Florida Diagnostic Tests (Sourcebook) Interim TestsInterim Tests
Reader’s HandbookReader’s Handbook Student Application BookStudent Application Book Lesson Plan BookLesson Plan Book
SourcebookSourcebook Teacher’s GuideTeacher’s Guide Student TextStudent Text
What tools do we have?What tools do we have?
FCAT Inquiry (Summative)FCAT Inquiry (Summative) Teacher Tools for Data Collection Teacher Tools for Data Collection
(Can be Summative or Formative)(Can be Summative or Formative)
•Histogram•Pareto Chart•Run Chart•Scatter Diagram•Item Analysis
HistogramHistogram
Bar chart representing a frequency distribution
of student scores
Heights of the bars represent
number of students scoring
at same level/score
Used to Monitor progress
Histogram: Mastery of FCAT Subtest Content Categories
0
2
4
6
8
10
12
14
16
F1 F2 F3 F4 FA FBSubtests
Nu
mb
er o
f S
tud
ents
Using Data Inquiry to Using Data Inquiry to Determine MasteryDetermine Mastery
FCAT Data InquiryFCAT Data Inquiry
Histogram: Grade Distribution on Final Exam (Summative Test) 9th Grade Reader’s Handbook Classes
0
10
20
30
40
50
60
70
0-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100Grade
Fre
qu
ency
Run Chart
Use to:
Monitor progress over time
Display data in simplest form
Run Chart: Words Correct Per Minute on Weekly Fluency Test
100
110
120
130
140
150
160
170
180
1 2 3 4 5 6 7 8 9Week
Nu
mb
er o
f w
ord
s
Class Goal: By the end of 9 weeks, 100% of our class will have an average
of at least 75% on our weekly benchmark reading passages.
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9Week
Per
cen
t w
/ avg
. of
at le
as75
%
Class Run Chart: Percent of Students Averaging at Least 75%
Scatter DiagramScatter Diagram
Scatter Diagram: Fluency Rates vs. Gain Points
50
60
70
80
90
100
78 88 98 108 118Gain Points
WC
PM
Scatter Diagram: Number of Siblings vs Grade Point Average
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
0 1 2 3 4 5 6
Number of Siblings
Gra
de
Po
int
Ave
rag
e
Scatter Diagram: Days of Attendance vs # of Mistakes on Sourcebook Diagnostic Posttest
0
3
6
9
12
15
18
0 18 36 54 72 90 108 126 144 162
Days of Attendance
Mis
take
s
180
Item Analysis
Use to:
Determine mastered content
Determine most common mistakes
Classroom Test AnalysisBENCHMARK
ASSESSEDITEM
#NUMBER CORRECT
NUMBERINCORRECT
NUMBERPARTIALCREDIT
NUMBERDISTRACTORA/1
NUMBERDISTRACTORB/2
NUMBERDISTRACTORC/3
NUMBERDISTRACTORD/4
NUMBERNOANSWER
1
2
3
4
5
6
7
8
9
10
Additional Additional Item Analysis ChecklistItem Analysis Checklist
Item Analysis ChecklistItem Analysis Checklist
Classroom Test Analysis
Student q1 q2 q3 q4 q5 q6 q7 q8 q9 q10 q11 q12 q13 q14 q15 q16 q17 q18 q19 q20
1 B C B B D C A A B C D C C A B C A D A D 2 B A A B D A B B A A D A C A B A A D A D 3 B C A B A B D A C A D C C A B C A D A D 4 A C A B D C A A A B D C C B C C A D A D 5 B C B B D A A A B A D C C A C C A C A A 6 B B A B C A D D D B A C C A B B A D A D 7 B B A A D C C D A C D C C C B C A D A D 8 B C A A D C C A C D D C C A B C A D A D 9 B C C B D C D A B D D C C A B C A A A D 10 B D A B B D D A A A D C C A B C A D A C 11 C C A B D B A C B C D B C B B D A D A D 12 B C A B D A C C B B D C C A A C A D A D 13 B A C B D C D A C B D C C A B C A D A D 14 B C A B D C B A A C D C C A B C A B A D 15 B B A B A B A B D A D C C A B C A D A A 16 A C A B D B C A A C D C C A B C A D A D 17 B B B B D C D C C D D C C A B C A D A D 18 B C A B D A B A A B D C C A B B A D A D 19 B C A B D A D A B A D C C A B C A D A D 20 B C A B A C B A C B D C C D B C A D A D
Correct Answer
B C A B D C D A A C D C C A B C A D A D
Questions 1-5 Introduction Questions 6-10 Reading Process Questions 11-15 Reading Know-How Questions 16-20 Improving Vocabulary
Item Analysis Activity
Pareto ChartUse to:
Rank issues in order of occurrence
Decide which problems need to be addressed first
Find the issues that have the greatest impact
Monitor impact of changes
Pareto Chart: Introduction, Reading Process, Reading Know-How, Improving Vocabulary
0
10
20
30
40
50
60
70
80
90
100
Reading Process Introduction Reading Know-How
Improving Vocabulary
Category
Pe
rce
nt
0
10
20
30
40
50
60
70
80
90
100
Cu
mu
lati
ve p
erc
en
tag
e
Data analysis provides:
Insight
and
Questions
Adapted from Getting Excited About Data, Edie Holcombwww.corwinpress.com
• What question are we trying to
answer?• What can we tell from the data?• What can we NOT tell from the
data? What else might we want to
know?• What good news is here for us to
celebrate?• What opportunities for improvement
are suggested by the data?
Questions to Ponder…
Action
Answers!
Provides
What information have I gained from my
data? What interventions can I put
In place?
Implement the plan.
Analyze the results.
Make improvements.
Steps to Improvement
PLAN
DO
STUDY
ACT
Personal Action Plan
What data can I access?
What tools can I use to help me monitor progress toward our class goals?
What/who else do I need to help me?
What is my start date?
How will I evaluate the results?
P
DS
A
Lesson PlanningLesson Planning
2 approaches can be used when 2 approaches can be used when planning your Great Source Reader’s planning your Great Source Reader’s Handbook lessons. You can organize Handbook lessons. You can organize your year by using:your year by using: The Teacher’s Guide and Lesson Plan bThe Teacher’s Guide and Lesson Plan b
ook ook Sunshine State Standards BenchmarksSunshine State Standards Benchmarks