ECO FAMILY EXPERIENCES AND OUTCOMES MEASUREMENT SYSTEM

Lauren Barton, ECO at SRI

Siobhan Colgan, ECO at FPG

Kathy Hebbeler, ECO at SRI

Melissa Raspa, ECO at RTI

Alice Ridgway, Connecticut

Chelsea Guillen, Il l inois

Lisa Backer, Minnesota

Robin Nelson, Texas

GOALS FOR TODAY• Introduce the Family Experiences and

Outcomes Measurement System• Hear feedback from states who helped

us refine the framework and piloted the self-assessment

• Discuss applications and uses with participants

HAVE YOU BEEN WONDERING…

• How well do we measure the experiences of families in our program or state?

• How can we interpret our family outcomes data?

• How do we know if we are heading in the right direction?

• How can we prioritize improvements to our family outcomes measurement system?

Family Experiences and Outcomes Measurement System

INTRODUCING…

PROCESS OF DEVELOPMENT

• Parallels the Child Outcomes Measurement Framework & Self-assessment

• ECO workgroup• Partner state workgroup

• Connecticut• Illinois• Minnesota• Texas

• Your feedback!

TWO CORE ASPECTS

Framework• Set of components and quality indicators• Provides the structure for the self-

assessment

Self-assessment • Scale that provides criteria for levels of

implementation within each quality indicator

• Ratings are assigned based on level of implementation within each indicator

PURPOSE OF THE FAMILY FRAMEWORK• Provides a common language and organizing

structure for discussing family outcomes and experiences

• Assist states in developing a measurement system that captures various aspects of family outcomes and experiences, such as• benefits that families receive from program participation• family satisfaction with services• perceived helpfulness of the services• family involvement with the service delivery system

FRAMEWORK

Components• 7 key areas of a measurement system

Quality indicators• 15 specific statements that further

describe each of the components• 1 to 5 quality indicators per component

Elements• Various examples of what constitute

quality at the indicator level

FRAMEWORK COMPONENTS

EXAMPLE COMPONENT: PURPOSEQuality Indicator: The state has articulated purpose(s) of the family experiences and outcomes measurement system

Elements: a. Stakeholders are involved in development of the purpose(s).b. Written statement addresses why data are being collected and how

data are being used. Statement specifies who will use the data and for what purposes.

c. Statement is easily accessible to local administrators, providers, families and general public.

d. Families receiving services are fully informed of purposes of collecting data on family experiences and outcomes.

e. Purpose includes meeting reporting requirements and providing ongoing information for data-based decision-making for program improvement.

f. Purpose includes examining multiple aspects of families’ experiences with the program and their outcomes (e.g., helpfulness of early intervention, family outcomes, family-centered services)

SELF-ASSESSMENT

• Allows states to evaluate the quality of their family experiences and outcomes measurement system

• Assists states in setting priorities for improving their measurement system

• Provides information to assist states in advocating for resources for systems development

11

SELF-ASSESSMENT PROCESS: QUESTIONS TO CONSIDER• Who will participate in the assessment process? • What constitutes your Family Experiences and

Outcomes Measurement?• What is the purpose of the self assessment for

your program? • Do you want to complete the entire measure or

focus on one or two areas you are concerned about?

• What will happen with the results of your assessment?

HOW DOES IT WORK?

• Each quality indicator has multiple elements• Elements provide evidence to determine the extent of

implementation • Each element is rated as

• NY = Not Yet• IP = In Process• IF = Fully Implemented• DK = Don’t know• NA = Not applicable (only where applicable)

• The quality indicator is given a rating based on the ratings of the elements

QUALITY INDICATOR RATINGS

Implementation of ElementsQuality

Indicator Rating

None of the elements are yet in process 1Some of the elements are in process 2All of the elements are in process 3At least one element is fully implemented and the rest are in process 4

Most of the elements are fully implemented and the rest are in process 5

Nearly all elements are fully implemented and the rest are in process 6

All elements are fully implemented 7

SAMPLE RATING PAGEComponent

Indicator

Element

HOW DID IT GO?

DISCUSSION QUESTIONS• What uses can you see for the tool in your state or program?

• How can families be involved in the self-assessment process?

• What additional ECO resources might be useful?