group 2 program evaluation
TRANSCRIPT
-
7/29/2019 Group 2 Program Evaluation
1/26
1
Jenna Hutto
Kayse Morris
Niki Taylor
Program Evaluation
FRIT 8425
Fall 2012Georgia Southern University
December 3, 2012
-
7/29/2019 Group 2 Program Evaluation
2/26
2
Executive Summary Page 3
Introduction to report Page 4
Focus of the evaluation Page 6
Brief overview of evaluation plan and procedures Page 8
Presentation of evaluation results Page 9
Conclusions and recommendations Page 15
Appendices Page 18
-
7/29/2019 Group 2 Program Evaluation
3/26
3
The purpose of this evaluation is to determine the impact of the program Read Naturally,
and its role in aiding student achievement based on oral reading fluency and comprehension.
This program evaluation was conducted at Bacon County Elementary school with ten students
from each third, fourth, and fifth grades. Currently this program is offered to students who
score below the twenty fifth percentile based on Lexile scores. In determining the
effectiveness, the following evaluation questions were posed:
Do students who use Read Naturally show significant gains in reading fluency andcomprehension?
Do teachers who implement Read Naturally see improvement in students on an academiclevel?
Data sources were used from the Read Naturally program at Bacon County Elementary in
Alma, Ga. These reports included students weekly probes based on achievement, areas of
weakness, and grade level. We evaluated CRCT scores and instructional folders to review
approaches for response to interventions. Students who were placed in the lower twenty fifth
percentile were nominated for tier three interventions and the Read Naturally, program
based on success rates. Students final recommendations for promotion were reviewed before
making a final decision on probing subjects. Questionnaires were distributed to classroom
teachers and parents of students who participated in the program.
-
7/29/2019 Group 2 Program Evaluation
4/26
4
Since the introduction of Read Naturally, in the fall of 2009, at Bacon Country
Elementary, the program has demonstrated success based on individual student achievement.
This program has helped many students enhance their reading capabilities. Read Naturally,
focuses on the five major points of reading: vocabulary, fluency, phonics, phonemic awareness,
and comprehension.
Read Naturally's structured intervention programs combine teacher modeling,
repeated reading, and progress monitoring three strategies that research has shown are
effective in improving students' reading proficiency. Using audio support and graphs of their
progress, students work with high-interest material at their skill level.
(http://www.readnaturally.com)
Read Naturally, also focuses on primary steps when assessing and aiding in better
reading.
1.) Select a story2.) Key Words3.) Prediction4.) Cold Timing5.) Graph the cold timing score6.) Read along7.) Practice
http://www.readnaturally.com/http://www.readnaturally.com/ -
7/29/2019 Group 2 Program Evaluation
5/26
5
8.) Answer the questions9.) Pass10.) Graph the hot timing score11.) Retell12.) Practice and pass the words list
-
7/29/2019 Group 2 Program Evaluation
6/26
6
The goal of Read Naturally, since 1991, is to help students become better readers. The
program is designed to help students increase reading fluency by using three main elements,
teacher modeling, repeated reading, and progress monitoring.
The program has been implemented into the regular school day schedule in 30 minute
sessions for 14 weeks at Bacon County Elementary School. The students selected to participate
in the program are students that receive a low score on the AimsWeb Screening or are
currently in any tier of intervention. AimsWeb is a universal screening of oral reading fluency
in which students read a passage to check for a students accuracy and determine their number
of correctly words read per minute. Students on tier intervention were returning users of the
program. Some of the students were new to the program Read Naturally.
Read Naturally is an individualized program. It allows students to progress at their own
pace through different reading levels. Students participating the program become more
independent and develop a feeling of ownership for their success. The program offers high-
interest nonfiction stories that are age-appropriate depending on the reader.
The purpose of the evaluation of the program Read Naturally is to focus on reading
accuracy, reading fluency, and reading comprehension. The following questions were used to
guide the evaluation:
1. Is Read Naturally a suitable program for students?
-
7/29/2019 Group 2 Program Evaluation
7/26
7
2. Do students who use Read Naturally show significant gains in reading fluency andcomprehension?
3. Do teachers who implement Read Naturally see improvement in students on anacademic level?
-
7/29/2019 Group 2 Program Evaluation
8/26
8
To assess the impact of Read Naturally on students at Bacon County Elementary School,
information was collected detailing student gains between cold timing scores and hot timing
scores and the perceptions of the stakeholders over a 14 week period. The information was
collected through student evaluations, student questionnaires, parent questionnaires, teacher
questionnaires, and system data analysis.
The evaluators collected qualitative data by providing evaluations and questionnaires
to students, parents, and teachers. The program system data analysis was collected to show
quantitative data. The evaluators examined the data of 60 students, 10 third graders, 10 fourth
graders, and 10 fifth graders. Again, some students were familiar with the program and some
were new to it.
Data of the 60 students was used to determine the gains between the cold and hot
timing scores. Periodic reports were collected for the 60 students. The reports presented the
students goal while working with the Read Naturally program (Appendix A,B,C). Fluency
details were also collected for each of the 60 students (Appendix A,B,C). Student evaluations
were distributed to the 60 students from third, fourth, and fifth grade (Appendix F).
Questionnaires were also distributed to the students (Appendix E) and their parents by the
evaluators (Appendix D). The data collected in the program evaluation of Read Naturally was
obtained from the 60 students at Bacon County Elementary School.
-
7/29/2019 Group 2 Program Evaluation
9/26
9
The counselor and RTI coordinator at our school analyze previous CRCT test scores and
reading fluency scores. Also, they meet with teachers and discuss students that need an
intervention in reading. After completing a list of which students meet the requirements for
interventions, the intervention team sends home letters explaining that the students need help
in the area of reading. The letter also explains they will be using Read Naturally, a computer
based program, for 30 minutes a day. The child has to remain in the intervention until they
meet the grade level goal. Below are the recommended goals for fluency for grades three
through five.
Below are the charts that illustrate the information for ten students from each grade
level. It depicts the starting date, ending date, starting WPM rate, ending WPM rate, stories
read, and comprehension correct.
Recommended Reading Fluency Rates in Connected Text
Grade Fall Winter Spring
3rd 79 84-93 100-114
4th 90-99 98-112 105-118
5th 105 110-118 118-128
-
7/29/2019 Group 2 Program Evaluation
10/26
10
After examining the third grade results, we noticed that 1 out of 10 students did not
increase in reading fluency and that 3 out of 10 students do not meet the standard of 80% on
comprehension correct. We recommended that teachers who are implementing Read Naturally
pay attention to what the students are doing to maybe cause no increase in WPM or less
percentage in comprehension. It may be due to the students not properly knowing how to use
the program or making mistakes. This may be something that can be improved with the help of
the teacher applying help or it may be that the student just does not understand and is missing
a step prior to being placed on Read Naturally. Another problem was that most of the third
grade students, with the exception of three, did not meet the recommended WPM goal for
winter which is 84-93. However, looking at their starting WPM goal, they are well below grade
level already. In the fall they should have been reading 79 words and most of them started at
either 50 or 60. Therefore, this tells me some of students still improved a tremendous amount.
3rd Grade
StudentStarting
Date
Ending
Date
Starting
Goal
(WPM)
Ending
Goal
(WPM)
Stories
Read
Comprehension
Correct
1 8/15/11 11/30/12 50 80 36 76%
2 8/15/11 11/28/12 50 100 22 101%3 8/15/11 11/30/12 50 80 41 87%
4 8/15/11 11/30/12 50 50 61 80%
5 8/15/11 11/30/12 50 70 50 83%
6 8/15/11 11/30/12 50 75 13 71%
7 8/15/11 11/30/12 65 125 52 82%
8 8/15/11 11/30/12 50 100 36 82%
9 8/15/11 11/30/12 50 60 34 70%
10 8/15/11 11/30/12 85 90 30 85%
-
7/29/2019 Group 2 Program Evaluation
11/26
11
When examining the fourth grade results, all of the students increased in WPM and only
three of the students did not reach the recommended goal of 80% in comprehension answered
correctly. Once again, this could be a problem that can be helped by the teacher or may be that
the student needs other intervention. However, most of the students in fourth grade did reach
the recommended reading fluency for winter which was 98-112. This shows better
improvement than third grade. Also, more of the students in fourth reached the 80% goal of
comprehension. However, there was still one student who fell short of the WPM goal and the
comprehension goal. This student may need a different type of reading intervention. Keep in
mind some students to not cooperate and learn well on computers. Therefore, this student
may need to be in some type of reading intervention class.
4th Grade
StudentStarting
Date
Ending
Date
Starting
Goal
(WPM)
Ending
Goal
(WPM)
Stories
Read
Comprehension
Correct
1 8/15/11 11/30/12 65 90 93 70%2 8/15/11 11/30/12 85 115 33 84%
3 8/15/11 11/30/12 80 100 30 84%
4 8/15/11 11/30/12 65 100 86 79%
5 8/15/11 11/30/12 85 100 72 81%
6 8/15/11 11/30/12 80 120 41 81%
7 8/15/11 11/30/12 85 130 52 82%
8 8/15/11 11/30/12 50 90 30 83%
9 8/15/11 11/30/12 65 100 92 80%
10 8/15/11 11/30/12 75 79 52 78%
-
7/29/2019 Group 2 Program Evaluation
12/26
12
Fifth grade results look awesome. Eight out of the ten students started well below WPM
for their grade level. However, after completing the program, eight out of the ten students
were well above the WPM goal for winter which is 110-118. Also, only three out of the ten
students had a comprehension that was below 80%. Several of the students had a large gain in
WPM after using Read Naturally and maintained an 80% or above on comprehension
questions correct. By the Spring fluency check, most of the fifth grade students should be
reading at or above average recommended WPM, which is 118-128. Four of the students are
already above the recommended WPM for Spring. The other six will have time to continue to
increase their WPM in reading fluency and hopefully reach the grade level goal.
5th Grade
StudentStarting
Date
Ending
Date
Starting
Goal
(WPM)
Ending
Goal
(WPM)
Stories
Read
Comprehension
Correct
1 8/15/11 11/30/12 50 120 30 87%
2 8/15/11 11/30/12 130 140 68 77%
3 8/15/11 11/30/12 120 130 22 87%
4 8/15/11 11/30/12 75 110 44 84%
5 8/15/11 11/30/12 100 100 77 80%
6 8/15/11 11/30/12 70 110 75 90%
7 8/15/11 11/30/12 85 120 106 77%
8 8/15/11 11/30/12 75 110 140 84%
9 8/15/11 11/30/12 80 115 57 80%
10 8/15/11 11/30/12 80 80 29 77%
-
7/29/2019 Group 2 Program Evaluation
13/26
13
Stakeholder Perceptions
After discussing the program with teachers, parents, students, and staff members, the
overall information indicates that Read Naturally has had a positive impact on the students
who participated. Some of the strengths they named were self-paced, easy to use/navigate,
and computer correction. These were some ofthe most important strengths. They believe
that Read Naturally being self-paced helps the students not feel intimated by others. Another
great feature is that they read on their own so other students cannot hear them or worry about
who is more advanced in the program.
The parents really liked the fact that the program was offered during the school day and
that students could use the program while at school without getting further behind on their
actual classwork. They also liked that while working individually on computers, there was also
a teacher present so if the students needed guidance. Another feature the parents really liked
about Read Naturally is that it gave the students immediate feedback on the story and words
read per minute. Also, the program allows them to read, reads to them, and allows them to
read again. It is teaching them how to read the passage, not just letting them read it alone.
Some features that the teachers and staff liked are the read- word and comprehension
questions. If while reading a student comes to a word they do not know, they can simply click
on the word and the program will tell them how to pronounce the word. Also, the
comprehension questions are multiple choice and if the student gets the question wrong, it will
not allow them to move on to the next question, it brings the story back up and allows them to
go back in the story and find the correct answer. Even though they answered incorrectly and
the question counts against them, they have to find the correct answer before they can move to
-
7/29/2019 Group 2 Program Evaluation
14/26
14
the next question or next step of the program. This teaches them skills of how to read back in
the story to find an answer.
The students were also asked what they thought about Read Naturally and how it has
benefited them in reading. The answers speak for themselves. Some students said I can read
better and I can read faster now. The students who really struggle in reading and want to
increase their reading speed and accuracy were overjoyed working on the program that it
helped them succeed. The only downfall was that some students who are on it very frequently
said that it does get boring because you are doing the same thing each day even if you are
reading different stories.
Program limitations cited by this group were concerned with student ability and
staffing concerns. More students could benefit from this program but without the staff to allow
the students to participate in the Read Naturally program, some of the students are not chosen
because they dont quite make the requirements for the program. Also, some students just do
not have the ability to use the program and actually increase in words per minute (fluency)
and comprehension.
Overall, the program definitely has more positives than negatives. There are more
strengths in the program data and the surveys from teachers, parents, and students than
downfalls in the program. This program may not be perfect and may not help every student
succeed in reading faster and more accurately, but it is a great start to help a child improve
their reading skills. Read Naturally focuses on increasing fluency and comprehension and at
Bacon County Elementary School it has done that in many ways. The program has more than
proved its effectiveness to our staff, parents, and students.
-
7/29/2019 Group 2 Program Evaluation
15/26
15
Based upon the evaluation of the program data relating to student needs and gains of the
current program Read Naturally, this evaluation study concludes:
Program Strengths:
The teachers manual is well-organized with clear objectives linking research with practice,plus detailed information for proper implementation and effective use of the program.
Students receive explanations as to why they perform various activitiescold timing,reading along, and practice readings.
Progress monitoring drives instructional practice, helps with instructional decisions, andalerts teachers to reading problems.
Comprehension questions and the retell step are included with every story, remindingstudents that their ultimate goal is to understand what they read.
Students are instructionally engaged in the act of reading and their time on task is high. Students work in a self-directed manner, freeing the teacher to work with other students.
Teacher assistance is required for the pass step.
Timed reading and graphs of results can be very motivating for students. Most students increase in WPM over the time period while using Read Naturally.
-
7/29/2019 Group 2 Program Evaluation
16/26
16
Program Weaknesses:
Lack of ample time and staff to productively implement Read Naturally as recommended. As seen in the results, it does not always work on every student, there are exceptions. Amount of comprehension questions answer correctly is not as high at least 80% for every
child.
Recommendations:
Work with scheduling at the school to allow more time for students to work on ReadNaturally each week.
Work individually with the students who have not increased in WPM to see what they maybe doing or what their problem may be.
Try to help the students who are not increasing to at least 75% on comprehensionquestions.
Therefore, for the most part, Read Naturally is a very effective program. As with every program
though, it has its weaknesses. For the most part, the school needs to get a different schedule
and try to incorporate the students using Read Naturally more during the week. Also, the
school and teachers need to figure out what individual students are not increasing in WPM are
struggling with. It may be that they do not fully understand the program or are having trouble
navigating, especially if they were not taught properly. Also, the students who do not have a
comprehension of at least 80%, which is 12 students out of the 30, may have some issues also.
The staff members who are with the students when they are working on Read Naturally may
have to work closer one-on-one and watch to see what the students are missing. As always,
-
7/29/2019 Group 2 Program Evaluation
17/26
17
there are some students who need further instruction or help than others. Therefore, the lower
comprehension and no increase in WPM may be a problem that can also be improved.
-
7/29/2019 Group 2 Program Evaluation
18/26
18
Appendix A3rd Grade Results Letter and Fluency Graph
Student 1
-
7/29/2019 Group 2 Program Evaluation
19/26
19
-
7/29/2019 Group 2 Program Evaluation
20/26
20
Appendix B
4th Grade Results Letter and Fluency Graph
Student 1
-
7/29/2019 Group 2 Program Evaluation
21/26
21
-
7/29/2019 Group 2 Program Evaluation
22/26
22
Appendix C
5th Grade Results Letter and Fluency Graph
Student 1
-
7/29/2019 Group 2 Program Evaluation
23/26
23
-
7/29/2019 Group 2 Program Evaluation
24/26
24
Appendix D
Parent Questionnaire
-
7/29/2019 Group 2 Program Evaluation
25/26
25
Appendix E
Student Questionnaire
1.) Do you enjoy reading?
2.) Do you believe Read Naturally, has helped yourreading accuracy and fluency?
3.) How often do you read?
4.) When do you read best?
5.) How has Read Naturally, helped you?
-
7/29/2019 Group 2 Program Evaluation
26/26
Appendix F
Student Evaluation