EmpowEring FamiliEs A Three-Year Evaluation

Re

ad

y t

o L

ea

Rn

an

d t

he P

Ro

vid

en

ce S

ch

oo

LS

The following is a synopsis of an analysis conducted by the program’s external evaluators – the Wellesley Centers forWomen out of Wellesley College in Wellesley, Massachusetts. That report was released in January 2018.

In the spring of 2014 Ready to Learn Providence, in part-nership with the Providence Public School Department, launched an ambitious four-year program, Empowering Families. Funded with a $3 million grant from the U.S. Department of Education’s Investing in Innovation Fund, and coupled with matching funds from foundations and businesses,1 the program seeks to increase the role of families in the education of their children. It also seeks to increase their understanding – as well as that of educators – of the so-cial, emotional and cognitive development of young children.

A 16-hour training series, Mind in the Making: The Seven Essential Life Skills Every Child Needs, is the cornerstone of Empowering Families. MITM, developed by Ellen Galinsky when she was with the Families and Work Institute, is now a program of the Bezos Family Foundation. It covers the ex-ecutive function skills – the skills that regulate our thoughts, feelings and actions – that successful students possess, and offers strategies for developing them at home and in school.

The four-year goal of Empowering Families is to bring MITM to 1,880 Providence parents, grandparents and other guardians of children aged prekindergarten to Grade 3. Si-multaneously, we are training 400 Providence teachers and auxiliary staff in the school system who work or interact with these young children. By doing this, Empowering Families is giving families and educators a common understanding of a young child’s social, emotional and cognitive development.

Ready to Learn Providence (R2LP) has targeted seven or eight elementary schools in each year of the program. By the conclusion of the fourth year in December 2018, R2LP

will have reached all 22 elementary schools in Providence.In addition to recruiting for MITM, bilingual R2LP staff

assigned to the targeted schools have worked with families to encourage school engagement and to help them feel more comfortable coming to the schools. They also partner with the school registration office during the kindergarten enrollment period to welcome parents to Providence Schools and to offer them the opportunity to participate in Mind in the Making.

Conducting the external evaluation of Empowering Fami-lies is the Wellesley Centers for Women out of Wellesley Col-lege in Wellesley, Massachusetts. In January 2018 it released its report on the program’s first three years (2014-2016).

As of the end of the period covered in this evaluation, 1,276 family members and 278 educators had participated in the program.

Overall, the evaluators found that the program had exhibited a high fidelity in terms of training, ac-tivities and attendance of participants. Addition-ally, they found statistically significant differences between pre- and post- intervention surveys in attitudes toward parent engagement by both families and educators. Specifically, par-ents expressed more confidence in their own ability to pro-mote the academic skills of their children and in the impor-tance of their involvement in the education of their children.

Educator surveys have indicated significant shifts in their attitudes toward parent involvement, parental inclusion, and developmentally appropriate beliefs regarding young children.

Thus far, researchers say Empowering Families ap-

An overview of all three years

February 2018

The vast majority (86%) of all participants strongly agreed that they were able to under-stand the training. This is important given that the vast majority of the training for families was done in Spanish and the level of educa-tion is low. The vast majority also strongly agreed that the training helped improve their skills and that they learned new information.

PreK Family SurveyA significant difference was found pre- and

post-intervention in the belief that a parent should be actively involved in his or her child’s education. A significant difference was also found in attitudes toward their own ability to influence their children’s schooling (e.g. I know how to help my child do well in school).

This is important because research has indicated that kindergarten is an important entrance point for families regarding involve-m e n t i n c h i l -dren’s schooling.

Pre- and post-surveys also found that these prekin-dergarten families

had adopted less authoritarian parenting views (e.g. It’s all right for a child to disagree with his or her parents; A child’s ideas should be seriously considered in making family

“The information shared is essential for teaching children at young ages. Too many times we are focused on curriculum without focusing on the development of the child.” – Teacher comment on post-

intervention survey

Cohort 2Fall 2014 – Summer 2015

For the purpose of data col-lection, pre- and post-inter-vention surveys were collected for parents and educators who participated in the training. The pre-post design includes the completion of a survey at the beginning of the Mind in the Making training protocol and the immediate completion of the same survey at the end of the training sessions. The survey was designed to assess attitudes and behaviors related to paren-tal engagement in schools. The parent survey also included a parent modernity scale and an assessment of their children’s social and emotional develop-ment pre- and post-intervention. This allowed researchers to as-sess the potential impact of the intervention on attitudes toward parent involvement, parenting, and perceptions of children’s development. The full report is available as a pdf for those who want to see the specific mea-sures used by the evaluators. Go to http://www.r2lp.org/research/

Data Collectionpears to be positively promoting parental engagement and developmentally appropri-ate interactions across multiple participant types. At the conclusion of the four-year program, Wellesley will examine in far more depth the overall impact across all schools. In the report released in January 2018, it looked at each cohort – or year – individually.

Cohort 1Spring 2014

In the spring of 2014 (immediately after receiving the grant), Empowering Families was offered only to prekindergarten families. The program was fully launched in the elementary schools at the start of the school year that fall. The primary function of Cohort 1 was to provide preliminary data regarding the efficacy of the intervention as well as to collect infor-mation that would inform and guide future training. The vast majority of the participants perceived the training, the facilitators and the overall learning experience very positively.

PreK Family SurveyData indicated the intervention had a

significant and positive impact on “Parent Effi-cacy.” The Parent Efficacy Scale includes items such as: I know how to help my child do well in school and I make a significant difference in my child’s school performance. Evaluators also found a statistically significant impact on “Parental Instruc-tion.” Items for the Parental Instruction scale include: parents teaching children to follow teachers’ direc-tions, to work hard, and to ask questions when he/she doesn’t understand some-thing. No significant differences between the pre- and post-surveys were found in the parental report of children’s social and emotional development.

decisions). Research has found that less au-thoritarian and controlling parenting leads to stronger executive function skills in children.

K-3 Family SurveyThese families all had children already en-

rolled in the public schools, and they already had some engagement with those schools. Their results differed from PreK families in ratings of how important certain academic and social activities are to children (e.g. How important is it that your child does well in school; How important it is to children that parents talk with teachers). For K-3 families,

2

activities (e.g. we show our child that we like it when he or she wants to learn new things).

PreK families also exhibited less authoritarian views of parenting post- and pre-intervention.

K-3 Family SurveySignificant differences were also noted

among K-3 families. They showed signifi-cant differences pre-intervention and post-

intervention regard-ing the importance they placed on their school involvement (e.g. How important is it that you show in-terest in your child’s school activities; How important is it

that you keep close track of whether your child is behaving appropriate at school) and the aca-demics of their children. This pattern is similar to the finding for the K-3 survey for Cohort 2.

Like the PreK families, K-3 families ex-pressed significantly fewer authoritarian values post intervention.

Educator SurveyEducators placed greater importance on

parental involvement post-intervention ver-sus pre-intervention. Educators also scored significantly higher for developmentally appropriate beliefs. There appears to be a significant and posi-tive relationship be-tween participation in the intervention and their beliefs about child development and attitudes toward parent involvement.

Child OutcomesIn Cohort 3, PreK parents completed the

Devereux Early Childhood Assessment Pre-school Program and K-3 parents completed the Devereaux Student Strengths Assess-ment (DESSA). They assess the social and emotional development of young children.

Post-intervention versus pre-intervention, PreK parents rated their children significantly higher on the subscales of Self-Regulation

researchers did not see a shift in the impor-tance they placed on general involvement or in their perceptions of their own efficacy. Instead, the shift was in their perceptions of how important they believed specific parent engagement and academic activities were to their children’s success (e.g. How important is it that your child reads and looks at a book).

K-3 families re-ported significantly higher scores in progressive (less au-thoritarian) attitudes towards parenting post-intervention versus pre-interven-tion.

Educator SurveyResults of the educator survey indicated

some significant shifts in educator attitudes toward parent engagement. Educators ranked the importance of engagement activities sig-nificantly higher at post-assessment versus pre-assessment. Additionally, educators reported engaging in more engagement activities.

“Now I know much more about kids and how to talk with them at their level.” – Parent after completing Mind in

the Making

Cohort 3Fall 2015 – Summer 2016

PreK Family SurveySignificant shifts in families’ attitudes

regarding parental engagement and parenting were noted. There was a significant shift in the importance they place on their general involve-ment and engagement with their children’s school, and on the importance they placed on showing their children school-related

and Attachment/Relationships. Although this study cannot show causality, the find-ings indicate either a more positive shift in parental perceptions of their children’s development or potentially an actual change in children’s social/emotional behaviors.

K-3 families reported significant dif-ferences pre- and post-intervention in the developmental outcomes of children as measured by the DESSA. Significant differ-ences pre/post intervention were noted in parental perceptions of children’s empathy, problem solving, and overall protective factors.

In the third year the evaluators also as-sessed two groups of kindergarten children directly – children whose teachers had com-pleted Mind in the Making, and children whose teachers hadn’t. This analysis showed

no significant associa-tion between teach-ers’ enrollment in the intervention and gains in children’s socio-emotional function-ing, achievement and executive functioning.

The evaluators stressed that this single find-ing does not provide sufficient evidence to diminish the other evidence that the children had made gains in those areas as perceived by their parents and teachers.

“Mind in the Making has made me much more comfortable in the schools.” – Parent

“I love that parents will have the opportunity to experience this training and its valuable information.” – Teacher

3

Conclusions of Evaluators

To date, Empowering Families has been successfully implemented in the Providence Public Schools. The training protocol has been successfully completed in its entirety across trainings for parents and teachers, including Spanish trainings for non-English speaking families. Additionally, the attendance of participants has been consistently high, including the par-ticipation of primarily non-English speaking and low-income families. The vast majority of participants attended more than half of the available training hours. Importantly, partici-pants consistently responded positively to the training sessions and reported gaining knowl-edge. This is reflected, in part, in the pre-post differences evident in the survey findings.

Educators report more developmentally appropriate beliefs and attitudes post-inter-vention as well as stronger views regarding the importance of parental involvement. The greatest impact, however, appears to be on families. Families reported signifi-cantly more modern parental views and less traditional/authoritarian views post-inter-vention. This was found across all cohorts as well as for the overall sample of parents.

Parents reported stronger views regarding the importance they placed on general involve-ment and their children’s involvement in school, as well as engagement in significantly more parental-school activities post intervention.

For prekindergarten parents, significant differences were noted in parental views of efficacy related to their abilities to help and support their children in school. This is im-portant given the research that stresses the importance of parental involvement to chil-dren’s overall academic success. As such, the intervention appears to be having a significant and positive impact on parents' views toward parental involvement, their perceptions of their own efficacy related to involvement, and their behaviors related to school involvement.

Importantly, significant differences pre- and post-intervention were noted in parental reports of child outcomes as measured by the DESSA (children K-3) and DECA (PreK

children). Although the study cannot show causality, this at least highlights a significant

and positive change in parental perceptions of children’s development.

Highlights of the Evaluation

The training protocol has been successfully completed in its en-• tirety and with fidelity for families and teachers.

Attendance of participants has been consistently high, including • that of non-English and low-income families.

Participants consistently responded positively to the training, and • reported gaining knowledge.

Educators report more developmentally appropriate beliefs sur-• rounding early childhood development.

Families reported significantly less authoritarian views toward • parenting, which research shows leads to higher executive function skills in children.

Parents reported stronger views regarding the importance of their • involvement in the schools and in the education of their children.

Parents expressed more confidence in their ability to help and sup-• port their children in school.

Parents reported growth in the social, emotional and cognitive • development of their children.

Ready to LeaRn PRovidence 945 Westminster Street • Providence, RI 02903

(401) 490-9960www.r2lp.org

Ready to LeaRn PRovidence is a program of the

Roger Williams University School of Continuing Studies.

1The following organizations are mak-ing Empowering Families possible: the US Department of Education, Investing in Innovation Fund; The Rhode Island Foundation; United Way of Rhode Is-land; the Anne E. Casey Foundation; The Campaign for Grade Level Reading; The A.M. Fund; The Bezos Family Founda-tion; and the Hasbro Children's Fund.

February 7, 2018