eco suggestions on indicators c3 and b7 kathy hebbeler , eco lynne kahn, eco/nectac
DESCRIPTION
ECO Suggestions on Indicators C3 and B7 Kathy Hebbeler , ECO Lynne Kahn, ECO/NECTAC August 13, 2009. Take-home points for today. The “change” in the measurement table involves 2 new calculations from data already being reported - PowerPoint PPT PresentationTRANSCRIPT
ECO Suggestions on Indicators C3 and B7
Kathy Hebbeler, ECOLynne Kahn, ECO/NECTAC
August 13, 2009
Take-home points for today
• The “change” in the measurement table involves 2 new calculations from data already being reported
• The change reduces the possible number of targets for these Indicators.
• The calculation described in the Indicator Measurement Table can be done by an ECO calculator.
• The “summary statements” provide data describing program effectiveness.
2
Timeline
• February 2007 – states report data on indicators C3 and B7 for the first time
• February 2010 – states are to set targets for these indicators
C3 and B7: Three Child Outcomes
• Children have positive social-emotional skills (including social relationships)
• Children acquire and use knowledge and skills (including early language/communication [and early literacy])
• Children use appropriate behaviors to meet their needs
Reporting Categories (Measures) for C3 and B7
Percentage of children who: a. Did not improve functioningb. Improved functioning, but not sufficient to move nearer
to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged
peers but did not reach itd. Improved functioning to reach a level comparable to
same-aged peerse. Maintained functioning at a level comparable to same-
aged peers3 outcomes x 5 “measures” = 15 numbers
The Problem
• Data reported includes– 5 progress categories – For each of 3 outcomes– Total of 15 numbers reported each year
• OSEP heard: Too many targets for one Indicator
• OSEP asked ECO for a recommendation
6
The Solution
“Summary Statements”
(a way to reduce the data so states did not have to set 15 targets for the
indicator)
Broad input to develop the Summary Statements
• ECO presented options to states and ECO work groups via conference calls
• Posted on the ECO web site for comments• OSEP put the summary statements out for
public comment• Comments came in that were thoughtful, but not
necessarily consistent with one another
8
• Paper documenting the process and alternatives considered on the ECO website
• Setting Targets for Child Outcomes
9
The concepts are easier than the words or the formulas
• Summary Statement 1: How many children changed growth trajectories during their time in the program?
• Summary Statement 2: How many children were functioning like same aged peers when they left the program?
Illustration of 5 Possible Develomental Trajectories (i.e, the OSEP Reporting Categories)
0
10
20
30
40
50
60
70
1 6 11 16 21 26 31 36 41 46 51 56Age in Months
Sco
re
Maintained functioning comparable to age peers
Achieved functioning comparable to age peers
Moved nearer functioning comparable to age peers
Made progress; no change in trajectory
Did not make progress
The “change” to the Measurement Table=The Summary Statements
• Of those children who [entered the program] below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 6 years of age or exited the program.
• The percent of children who were functioning within age expectations in each Outcome by the time they turned 6 years of age or exited the program.
12
• Measurement for Summary Statement 1:
Percent = # of infants and toddlers reported in progress category (c) plus # of infants and toddlers reported in category (d) divided by [# of infants and toddlers reported in progress category (a) plus # of infants and toddlers reported in progress category (b) plus # of infants and toddlers reported in progress category (c) plus # of infants and toddlers reported in progress category (d)] times 100.
13
Summary Statement 1
c + d_________________________
a+ b + c + dX 100
• Measurement for Summary Statement 2: Percent = # of infants and toddlers reported in progress category (d) plus [# of infants and toddlers reported in progress category (e) divided by the total # of infants and toddlers reported in progress categories (a) + (b) + (c) + (d) + (e)] times 100.
15
Summary Statement 2
d + e_____________________________
a+ b + c + d + eX 100
A shortcut to the calculations
17Summary Statements Calculator -April 14, 2009
Example of State Indicator Data for 2008-2009
Positive social-emotional skills (including social relationships):
Number of
children
% of children
a. Percent of infants and toddlers who did not improve functioning
40 4
b. Percent of infants and toddlers who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers
150 15
c. Percent of infants and toddlers who improved functioning to a level nearer to same-aged peers but did not reach
270 27
d. Percent of infants and toddlers who improved functioning to reach a level comparable to same-aged peers
300 30
e. Percent of infants and toddlers who maintained functioning at a level comparable to same-aged peers
240 24
Total N=1000 100%
SS1-What are we calculating??
19
Prog cat
# %
a 40 4b 150 15c 270 27d 300 30e 240 24
760 (a, b, c, and d) or 76% of the children entered or exited the program functioning below age expectations
240 (e) or 24% of the children entered and exited functioning at age expectations
SS1 (continued)
20
Prog cat
# %
a 40 4
b 150 15
c 270 27d 300 30e 240 24
570 (c and d) of the 760 (a, b, c, and d) changed their growth trajectories (made greater than expected progress)
270 +300= 570 760
= 75%
SS2-What are we calculating??
21
Prog cat
# %
a 40 4
b 150 15
c 270 27
d 300 30e 240 24
300+240= 540 1000
= 54%
30% of the children reached age expectations by exit and 24% of the children entered and exited at age expectations
Summary Statement 1 is one type of evidence of program effectiveness
• Required Summary Statement 1:
Of those children who entered the program below age expectations in Outcome A, the percent who substantially increased their rate of growth by the time they exited the program= 75%
• 75% of the children who were below age expectations made greater than expected gains in their social relationships, made substantial increases in their rates of growth. i.e. changed their growth trajectories
22
Summary Statement 2 is another type of evidence of program effectiveness
• Required Summary Statement 2:
The percent of children who were functioning within age expectations in Outcome A when they exited the program= 54%
• 54% of the children were functioning like same age peers in their social relationships when they exited the program.
23
Questions and comments?
24
More Information re Target Setting
• Conference calls,
• Individualized TA
• Resources at
The-ECO-Center.org25
What else can we say?
• 96% of children participating in Part C made progress in their social relationships while they were enrolled.
• The 4% of children who did not make progress included children with the most severe disabilities and/or degenerative conditions. Can you describe them?
26
• 24% of the children participating in Part C were functioning at age expectations at entry and at exit in their social relationships.
• Can you describe them?
27