linda goodman connecticut birth to three system early childhood outcomes conference july 31, 2010 1
TRANSCRIPT
Linda GoodmanConnecticut Birth to Three System
Early Childhood Outcomes ConferenceJuly 31, 2010
1
Results and Performance Accountability, Decision-making and Budgeting
Mark FriedmanFiscal Policy Studies
InstituteSanta Fe, New Mexico
www.resultsaccountability.comwww.raguide.org
Use #1
2
“All Performance Measures
that have ever existed
in the history of the universe
involve answering two sets of
interlocking questions.”
3
HowMuchdid we do?
( # )
HowWelldid we do it?
( % )
Quantity Quality
4
Effort
Effect
HowMuch
HowWell
5
How much did we do?
How welldid we do it?
Is anyonebetter off?
Quantity Quality
E
ffect
E
ffort
# %
6
How much service did we deliver?
How welldid we deliver it?
How much change/effect did we produce?
What quality of change/ effect did we produce?
Quantity Quality
E
ffect
E
ffort
Ou
tpu
t
Inp
ut
7
How much did we do?
Education
How well did we do it?
Is anyone better off?
Quantity Quality
E
ffect
E
ffort
Number ofstudents
Student-teacherratio
Percent of 9th graders whoenter college oremployment after graduation
Number of 9th graders whoenter college oremployment after graduation
8
How much did we do?
Part C
How well did we do it?
Is anyone better off?
Quantity Quality
E
ffect
E
ffort
Number of Eligible Children
Percentage of children under 12 months
Number of children exiting whose development was accelerated
Percent of children exiting whose development was accelerated
9
How much did we do?
Not All Performance Measures Are Created Equal
How well did we do it?
Is anyone better off?
LeastImportant
Quantity Quality
E
ffect
Eff
ort
2nd MostImportant
3rd MostImportant Most
Important
10
How much did we do?
The Matter of Control
How well did we do it?
Is anyone better off?
MoreControl
Quantity Quality
E
ffect
E
ffort Less
Control
Partnerships needed to improve performance
11
“We haven’tgot the money, so we’ve gotto think.”
Lord Rutherford1871 - 1937
“Turning the curve”
12
Contributionrelationship
Alignmentof measures
Appropriateresponsibility
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATION ACCOUNTABILITY
Healthy Births Rate of low birth-weight babiesStable Families Rate of child abuse and neglectChildren Succeeding in School Percent graduating from high school on time
CUSTOMERRESULTS
# Foster ChildrenServed
% withMultiplePlacements
# RepeatAbuse/Neglect
% RepeatAbuse/Neglect
PERFORMANCE ACCOUNTABILITYChild Welfare Program
POPULATIONRESULTS
13
Quality of Life Result: All Connecticut children are healthy and ready for school success at age 5, contributing to a reduction over time in Connecticut’s achievement gap at Grade 4.
Program Contribution to Result: By providing family-centered early intervention services, the program strengthens the capacity of Connecticut’s families to meet the developmental and health-related needs of their infants and toddlers who have delays or disabilities, thereby ensuring that more of these children are ready for Kindergarten. Activities include coaching caregivers to embed intervention in the child’s daily natural routines and learning opportunities.Partners: 44 contracted providers programs; local school districts; pediatricians and family medicine practitioners, hospital NICU staff, Office of Policy and Management; Department of Public Health; Department of Insurance; State Department of Education; Department of Children and Families; Board of Education and Services for the Blind; Children’s Trust Fund, UCEDD, Early Childhood Cabinet; HMO Association Members and Anthem; United Way; State ICC; Local ICCs;
Connecticut Part C Example
14
Performance Measure 1 Percentage of infants and toddlers with disabilities or developmental delays who improve so that at exit, they function at age level in three skill areas
Skill Areas
Story Behind the baseline: The three skill areas on the graph are: (1) social/emotional; (2) acquisition and use of knowledge (including early language and literacy); and (3) use of appropriate behavior to meet their needs.
Each bar represents the percentage of children that performed at age level in each skill area at the time they exited the Birth to Three System. Data began in FY07 and some variation is due to the fact that it has taken three years for the children with the most significant disabilities, identified at birth, to exit the system.
Children that received services for at least six months prior to exit
15
Proposed actions to turn the curve: Since Birth to Three only enrolls children with significant delays or disabilities, it is not possible that 100% of children will exit the program at age-level. However, we do expect these percentages to increase from their current levels. The research shows that improvement is most highly correlated with the ability of each provider to ensure that parents and childcare givers know how to incorporate intervention techniques into daily routines so that each child gets the maximum amount of practice, all day, every day. This is a different approach than out-patient rehabilitation services, in which short sessions in therapy rooms are expected to lead to generalization and carryover into other environments. That approach does not work well for infants and toddlers. Since each local program is responsible for overseeing the delivery of early intervention services, the next step will be to rank each program on its child outcome data and then focus on improving the low-performing programs through technical assistance and additional training for their personnel.
16
Performance Measure 2: Percentage of families who report that, as a result of receiving services from the Birth to Three System, they are better able to help their children develop and learn.
Story behind the baseline: The blue bars on the graph show the percentage of families who strongly or very strongly agreed with the statement: “Birth to Three has helped me to help my child develop and learn.” The trend in this positive family response has increased from 73% to 78% in the past three years, which shows the programs’ effectiveness in achieving their mission of helping families to facilitate their children’s development. The percentage of families indicating “agree, strongly agree, or very strongly agree” for this measure was 87.5%
17
Proposed actions to turn the curve: This data, like the child outcome data, will be publicly posted by program and used to monitor any low-performing programs and drive improvement in how well staff work with families. Additional training for service coordinators will be offered both through Birth to Three and through the UCEDD.
18
Focused Monitoring
National Center for Special Education Accountability Monitoring
Alan CoulterJane Nell Luster
Use #2
19
There is something very sad in the disparity between our passion for figures and our ability to make use of them once they are in our hands.
-M.J. Moroney, 1951
20
Why?Monitoring resources are in shorter and
shorter supply yet there is a greater emphasis on general supervision
It is not possible for EI programs or districts to concentrate on the hundreds of requirements in IDEA at all times
Programs cannot work to improve hundreds of things at the same time
Focusing on important result-oriented priorities can more readily lead to improved results for young children
21
Six Components of Focused Monitoring
1. Stakeholder participation and decision-making
2. Data availability, analysis, and use3. Integration with the system of general
supervision4. Off-site and on-site monitoring processes5. Corrective actions, incentives, and sanctions6. Evaluation
22
Principles of Focused Monitoring1. A limited number of priorities are chosen by
a diverse group of stakeholders2. A limited number of indicators are identified
within each priority area3. The system is data and information-based
and is verifiable4. Data-based information is used to allocate
monitoring resources in the direction of most need
23
Principles of Focused Monitoring5. The monitoring agency provides supports
and imposes sanctions upon programs in order to achieve corrective actions
6. Standard, uniform benchmarks are used for inquiry when making monitoring decisions
7. There is a relationship between monitoring and corrective actions – solutions are linked to identified problems.
8. The system includes clear, known triggers for interventions and sanctions
24
Principles of Focused Monitoring9. Limited resources are allocated to the areas
of greatest need, which are determined by identifying what is most likely to lead to improvements in child performance
10.Available information is used to select priorities that will prove child and family outcomes
11.Monitoring strategy is systemic12.Corrective actions ensure a change in
behavior that results in improved child and family outcomes
25
Principles of Focused Monitoring13.Monitoring staff are well trained and
engage in continuous professional development
14.There is third party evaluation of the monitoring/enforcement system
26
Selecting Priorities & Key Performance IndicatorsPriority areas should align with SPPBest not to select compliance indicators
if possibleIndicators must be measureableData must be available (preferably at
least quarterly) otherwise indicator must be deferred until the required data is available
27
Many priorities...Determining priority indicators in 2004
IFSP• child and family outcomes met• resources• objective timelines met• planning
Child Find• identification by diagnosed
condition by age• correct assessment, all needs
identified• referral age• timely referral new money • families not accepting services
Transition• date of transition plans• date of referral to LEA• date of transition conference with
LEA• Inservice training on Part B
requirements• the timely preparation of families
for all transitions• training personnel and parents
about advocacy for Part B and C• adequate information shared from
Part C to Part B
Service provision• timelines of objectives met• withdrawals• competent providers• appropriate services and
support• resources/other families• measurable progress• services delivered match IFSP
Family involvement• define• direct/indirect• opportunity to be involved• knowledgeable to help their
child• information and decision-
making• advocacy
28
Little data….Determining priority indicators in 2004
IFSP• child and family outcomes met• resources• objective timelines met• planning
Child Find• identification by diagnosed
condition by age• correct assessment, all needs
identified• referral age• timely referral new money • families not accepting services
Transition• date of transition plans• date of referral to LEA• date of transition conference with
LEA• Inservice training on Part B
requirements• the timely preparation of families
for all transitions• training personnel and parents
about advocacy for Part B and C• adequate information shared from
Part C to Part B
Service provision• timelines of objectives met• withdrawals• competent providers• appropriate services and
support• resources/other families• measurable progress• services delivered match IFSP
Family involvement• define• direct/indirect• opportunity to be involved• knowledgeable to help their
child• information and decision-
making• advocacy
29
But in 2009… things changed Priority: Child & Family Outcomes Indicator: Families are more confident and gain new skills
which help their children develop and learn.
DATA FOR INDICATORresponses to the family survey measure #32 (Over the past year Birth to Three services have helped me and/or my family figure out solutions to problems as they come up)child outcome data (C3b) showing the percent of children who closed or reduced the “gap” in their acquisition and use of knowledge and skills (including early language/communication).
30
But in 2009… things changed Priority: Child & Family Outcomes Indicator: Families are more confident and gain new skills
which help their children develop and learn.
Selection criteria Programs that are more than 2 SD below the statewide mean are listed as “ very low”Programs that are 1 – 2SD below the statewide mean are listed as “low”. 0 – 1SD below the statewide mean is “mid”The first programs selected will be those that are very low in at least one measure. The next programs selected will be those that are low in at least one measure.
31
How might that look?Both High One
Mid/One High
Two Mid One or more low
One or more very low
Cheshire Abilis Rehabilitation Assoc.
Ahlbin Center Bisantz & Assoc.
Early Connections
Building Bridges
CES REM
Hill Health Children’s Therapy
Child & Family Network
Key Services
Kennedy Center CREC Easter Seals
Oak Hill Education Connections
Family Junction
LEARN SARAH
HARC
McLaughlin
Wheeler32
Program is notified that they’ve been selected for focused monitoring
At desk audit, monitoring team studies data, looks deeper into the data and related data and forms a hypothesis. (e.g. High rate of staff turnover is why children in this program are not making as much progress as children in similar programs.) Discusses with program administrator
Monitoring team goes on-site, follows established protocol that includes looking at records, talking to families, talking to staff, and sometimes talking to LEAs.
What’s the Monitoring Process?
33
Protocol excerptsWhat to look for Records Famil
yStaff Dat
aLEA
Other
Quality evaluation/assessment reports describing strengths and needs
× × ×
Families know what their service coord. can do for them
× ×
Families are connected to outside supports
× × ×
IFSP is routines-based × × ×
Families are using strategies between visits
× × ×
Where to look
34
Protocol excerptsWhat to look for Records Famil
yStaff Dat
aLEA
Other
Staff can demonstrate/describe coaching vs. consulting vs. teaching
× ×video?
Timing of visits is related to outcomes
× ×
Family outcomes are addressed
× × × ×
Updated assessments are used at transition conferences
× × × ×
Families discuss child’s progress, strengths & needs with LEA
× × ×
Where to look
35
What’s the Monitoring Process?Monitoring team members keep administrator
informedMonitoring team meets at the end of each day to
discuss. When all data is gathered, monitoring team meets
to discuss hypothesis and any other issues they have discovered
Monitoring team meets with administrator(s) for exit interview. Nothing said should be a surprise. TA offered
Final report is issued, including findings36
What’s the Process?Improvement plan (or compliance
agreement) written to correct any non-c0mpliance and improve performance. Strategies may include training and TA.
Accountability and Monitoring staff track improvement plan timelines and data until data indicates that non-compliance is corrected and performance has improved.
37