maintaining a social-emotional intervention and its benefits for institutionalized children

Maintaining a Social-Emotional Intervention and Its Benefits forInstitutionalized Children

Robert B. McCall, Christina J. Groark,and Larry Fish

University of Pittsburgh

Rifkat J. Muhamedrahimov and Oleg I. PalmovSt. Petersburg State University

Natalia V. NikiforovaBaby Home 13, St. Petersburg, Russian Federation

This article reports the maintenance of one of the largest interventions conducted in St. Petersburg (RussianFederation) orphanages for children birth to 4 years using regular caregiving staff. One orphanage receivedtraining plus structural changes, another training only, and a third business as usual. The interventionproduced substantial differences between these institutions on the Home Observation for Measurement of theEnvironment (HOME) Inventory and on the Battelle Developmental Inventory scores for children. Theseinstitutional differences in HOME scores (N = 298) and Battelle scores for children (N = 357) departing theinstitutions for families in St. Petersburg and the United States were maintained for at least 6 years after theintervention project. This result may be associated with certain features of the intervention and activities con-ducted during the follow-up interval.

Sustainability refers to maintaining the effectivenessof interventions and transferring them to commu-nity agencies to be operated after the demonstrationfunding and researchers are no longer involved.The inability of researchers and practitioners to sus-tain successful interventions in the community hasbeen lamented for decades. For example, Sarason(1967) noted that “psychologists are as good as any-body else in initiating change and as bad as every-body else in sustaining it” (p. 232). This frequentlack of success is a long-standing issue in manydomains, including public health (e.g., Altman,1995; Goodman & Steckler, 1987, 1989a, 1989b), theprevention of adolescent problem behavior (e.g.,Bumbarger & Perkins, 2008; Gomez, Greenberg, &Feinberg, 2005; Tibbits, Bumbarger, Kyler, & Per-kins, 2010), and social-emotional and early care andeducation interventions for young children (e.g.,Domitrovich, Moore, & Greenberg, 2011). Ofcourse, some interventions have been sustained

(e.g., Eckenroade et al., 2010), but it is still often thecase that intervention effects commonly fade afterinitial funding terminates, the intervention is trans-ferred from researchers to practitioners, andattempts to implement the original program in newsites and new communities are made (e.g., Gomezet al., 2005; Hallfors, Cho, Livert, & Kadushin,2002; Hallfors & Godette, 2002; Mancini & Marek,2004).

Across various disciplines, “sustainability” refersto a set of functions, typically conducted in sequence,including (a) demonstrating that the original inter-vention was effective, (b) maintaining the interven-tion effects in the original setting after initial researchfunding ends, (c) transferring responsibility for theintervention to practitioners and community agen-cies, (d) getting the same or similar evidenced-basedinterventions used in new communities by non-researchers, and (e) demonstrating over long periodsof time the effectiveness of such evidence-basedinterventions in communities. Generally, theseliteratures converge on the proposition that betterlong-term outcomes are obtained if planning forsustainability occurs when the original intervention

This research funded by Grants HD039017 and HD050212 toauthors McCall and Groark from the Eunice Shriver KennedyNational Institute of Child Health and Human Development.The interpretations and opinions expressed in this article arethose of the authors, not their funders.

Correspondence concerning this article should be addressed toRobert B. McCall, Office of Child Development, 400 N. Lexing-ton Street, Pittsburgh, PA 15208. Electronic mail may be sent [email protected].

© 2013 The AuthorsChild Development © 2013 Society for Research in Child Development, Inc.All rights reserved. 0009-3920/2013/8405-0018DOI: 10.1111/cdev.12098

Child Development, September/October 2013, Volume 84, Number 5, Pages 1734–1749

is created, and if steps are taken along the way thatare deliberately aimed at facilitating the maintenanceof the intervention and its transference to new con-texts and practitioners (e.g., Altman, 1995; Berry,Senter, Cheadle, Greewald, & Peason, 2005; Mancini& Marek, 2004; Mittelmark, Hunt, Heath, & Schmid,1993; Scheirer, 2005).

This article focuses on the second of the abovesustainability components, which is called “mainte-nance” of effectiveness. Once funding for the initialdemonstration has ended and the interventionbecomes a routine part of an institution’s program-ming, the fidelity with which the intervention isimplemented often deteriorates, and the benefitsonce demonstrated diminish with time, sometimescompletely. Typically this occurs when no plans oractivities were made for intervention maintenance.

This has been the case for interventions designedto improve the development of institutionalized chil-dren. Most of the early interventions were conductedprimarily as basic research projects designed to showincreased stimulation of different kinds wouldimprove children’s development. These studies pro-vided infants and young children with specific extrastimulation implemented by the experimenter orresearch assistants for a short period of time eachday over several weeks. Generally, such interven-tions produced developmental improvements orthey prevented the developmental decline that wasobserved in untreated infants. These were “demon-strations” that benefits could be produced, therewere no attempts to continue the intervention, andthus the developmental benefits diminished or dis-appeared (Bakermans-Kranenburg, van IJzendoorn,& Juffer, 2008; Rosas & McCall, in press). Even morecomprehensive and longer term demonstration inter-ventions may produce fading benefits if nothing isdone to deliberately maintain them. For example,Sparling, Dragomir, Ramey, and Florescu (2005)provided young Romanian institutionalized childrenwith newly hired and trained caregivers, one for eachof four children, who conducted a variety of educa-tional activities over a 12-month intervention period.The children improved on the Denver Developmen-tal Screening Test, but Carlson and Earls (1997)reported that the benefits to these children and to theinstitution faded in the years following the end of theintervention.

More recently, some interventions have beenaimed at transforming an entire institution usingregular staff with a long-term goal of producingpermanent improvement in its operation and chil-dren’s development. An estimated 2 million(USAID, 2009) to 8 million (Human Rights Watch,

1999) children reside in institutions worldwide, andthe published literature indicates that conditions inmost institutions are not supportive of children’sdevelopment (e.g., McCall, van IJzendoorn, Juffer,Groark, & Groza, 2011; Rosas & McCall, in press).Although most international child welfare profes-sionals advocate for family alternatives to institu-tions, it is likely to take many years, if not decades,for most low-resource countries to place all childrenwithout permanent parents in families. Conse-quently, institutions potentially could be improvedfor the many children who are likely to reside therewhile alternatives are being developed (Groark &McCall, 2011; McCall, 2011).

This study is a follow-up of the most comprehen-sive intervention reported in an institution forinfants and young children using regular staff.It was conducted between 2000 and 2005, withfollow-up assessments extending 6 years after theend of the original intervention project. Three insti-tutions, called Baby Homes (BHs), in St. Petersburg(Russian Federation) were involved. In one BH,caregivers received training designed to promotemore warm, sensitive, contingently responsive, andchild-directed caregiver–child interactions, and avariety of structural changes were implemented tosupport the training and promote social-emotionalrelationships between caregivers and children in amore family-like context. A second BH receivedtraining but no structural changes, and a third con-tinued with business as usual. The intervention pro-duced the intended changes in caregiver behaviorand the BH environment as measured by the HomeObservation for Measurement of the Environment(HOME) Inventory, and children’s physical, cogni-tive, and social-emotional development improvedsubstantially in the double-intervention BH (The St.Petersburg-USA Orphanage Research Team, 2008).

In addition, the intervention was designed to beconsistent with the sustainability literature (reviewedabove) and its specific suggestions for maintaininginterventions (Groark & McCall, 2008; The St. Peters-burg-USA Orphanage Research Team, 2008). Forexample, the intervention was designed to be main-tained after the demonstration phase on the regulargovernment-supplied budget of the BH, and addi-tional steps were taken after the end of the interven-tion project to maintain its effectiveness (see below).

It was hypothesized that these maintenanceactivities would preserve the intervention in twoways, first by maintaining the improved caregiverbehavior and BH environment as reflected on theHOME Inventory, and second, as a result,maintaining the benefits to children’s behavioral

Maintaining a Social-Emotional Intervention 1735

development as measured by the Battelle Develop-mental Inventory.

A follow-up project was funded approximately2 years after the intervention project had termi-nated. The follow-up focused on a subset of childrenfrom the three intervention BHs, including childrenadopted to the United States through the Interna-tional Assistance Group, a Pittsburgh-based adop-tion agency specializing in the placement of Russianchildren in U.S. families, and children placed inSt. Petersburg adoptive, foster, and birth parent orkinship families. Children departing the three BHsfor these family destinations were given some of thesame measurements as during the intervention. Thisstudy uses this reduced set of measurements onboth the intervention and follow-up samples ofchildren transitioning to the above destinations.

To the best of the authors’ knowledge, this isthe only long-term follow-up of a comprehensiveintervention using regular institution staff. Follow-up assessments of children randomly assignedeither to remain in the institution or go to a speciallydesigned foster care program have been reported(e.g., Fox, Almas, Degnan, Nelson, & Zeanah, 2011;Johnson et al., 2010). Children who remained intheir randomly assigned conditions through 8 yearsof age have continued to demonstrate physical andcognitive benefits of foster care. Presumably, thequality of foster care remained high following place-ment, so the “treatment” condition persisted, butlimited assessments of its continuing quality havebeen published. Some children were transferred togovernment organized foster care, which tended tobe somewhat less effective (Fox et al., 2011).

Method

Participants

Baby Home. This study was conducted in threeBHs caring for children birth to 4 years of age in St.Petersburg (Russian Federation) which had beeninvolved in a major intervention project from 2000to 2005. Details of these institutions, the caregiversand children involved, the interventions that wereimplemented, and the results of those interventionscan be found in The St. Petersburg-USA OrphanageResearch Team (2005, 2008).

Briefly, caregivers in one BH received training ininfant-toddler development and in warm, sensitive,contingently responsive, child-directed caregiver-child interactions. In addition, numerous structuraland employment changes were made in this institu-tion that promoted better caregiver–child inter-

actions from fewer caregivers. This condition wascalled Training + Structural Changes (T + SC). Thestructural changes consisted of (a) smaller groupsizes, (b) two primary and four secondary caregiv-ers assigned to each subgroup in which one of theprimary caregivers was present for most of the chil-dren’s waking hours 7 days a week, (c) a reductionfrom approximately nine to six caregivers assignedto a ward but minimum change in total caregiverhours, (d) heterogeneous grouping of children byage and disability status within a group, (e) termi-nation of periodic graduations of children to newwards when they reached a certain age or develop-mental milestone, (f) the regular assignment of spe-cific caregivers to be substitutes for a particularward when needed, (g) the creation of an in-housemonitoring and supervisory system, and (h) familyhour in both the morning and afternoon in whichvisitors to the wards were not allowed and childrenwere to be with their caregivers. A second BHreceived Training Only (TO), and a third conductedbusiness as usual and had No Intervention (NoI).

Training + Structural Changes was implementedfirst, and it took approximately 1 year to completethe training and structural changes. Then trainingstarted in TO, and assessments were begun in NoIshortly thereafter. The T + SC intervention wascompletely implemented by September 3, 2000, theTO intervention was completed on January 13,2002, and assessments were begun in NoI on Octo-ber 18, 2002.

Maintenance activities. Attempts to maintain theintervention were of two types—creating an inter-vention that had features that would promotemaintenance after the original demonstration pro-ject ended and procedures implemented during theintervention and the time following the interventionthat might contribute to program maintenance.

The original intervention involved several com-ponents thought to promote maintenance. First, thedouble intervention was implemented in a BH inwhich the director was thoroughly committed toimplementing and maintaining the intervention andinsisted that her staff do so. This director remainedin her position throughout the intervention andfollow-up periods.

Second, although the intervention project pro-vided substantial resources to this BH includingsome funds to hire additional caregivers, the inter-vention was designed to reduce the number ofdifferent caregivers in children’s lives but not toincrease the number of caregiver hours available tochildren. Thus, the total number of caregiver hoursavailable in the BH did not change much with the

1736 McCall et al.

intervention and could be sustained financially inlarge part after intervention funds terminated.

Third, a train-the-trainer strategy was adopted sothat training personnel were continually availableto prepare new caregivers who replaced those wholeft during and after the intervention.

Fourth, an in-house monitoring and supervisionsystem was established in which specialists (e.g.,staff professionals in children with disabilities,early education) were responsible for monitoringthe caregivers and encouraging them to imple-ment the training on the wards on a continuingbasis.

Once the intervention project ended, all financialsupport stopped, and the interventions had to bemaintained on the BHs’ government-provided bud-get. In the TO institution, separate funds wereobtained after the intervention period to assist thisBH to implement structural changes, especiallyreduced group size. This was only partly successful,because not all groups were reduced in size, inte-gration by age and disability status was only partlyaccomplished, and family hour was not imple-mented.

Furthermore, following the end of the interven-tion project, additional coaching and technical assis-tance were provided by author Palmov and twoother professionals to both T + SC and TO, whichconsisted of weekly or bimonthly visitations, super-vision, and periodic refresher training. Also, authorGroark visited caregivers in T + SC and TO two orthree times per year to encourage appropriate care-giver–child interactions and troubleshoot. Thesesupports were given during and between both pro-jects. When several new caregivers were hired inT + SC and TO, a brief training course consisting of25 hr over 4–5 weeks was given during and afterthe intervention period by those specialists origi-nally trained as “trainers.” However, the in-housemonitoring and supervisory system encouragingappropriate caregiver–child interactions that hadbeen implemented in the T + SC was not created inTO.

Caregivers. The primary and secondary caregiv-ers in T + SC and the medical nurses and assistantteachers (i.e., main regular caregivers) up toapproximately 50 in each BH were individuallyassessed periodically with the HOME Inventory(24-month group version; Bradley & Caldwell,1995; Caldwell & Bradley, 1984).

The number of caregivers holding these positionsvaried from one assessment occasion to the nextand between BHs. At each time point, 46–59 care-givers in T + SC, 45–53 caregivers in TO, and 30–52

caregivers in NoI were assessed. It should be notedthat NoI was reduced substantially in size by thelocal government by the time the follow-up projectassessments were conducted, which explains itslower number of caregivers during the two follow-up assessments.

Departing children. The sample for the currentanalyses was dictated by the sampling criteria forthe follow-up project that focused on assessing thedevelopment of children who transitioned fromthese three BHs to families, either in the St. Peters-burg region or to the United States. This sample(see Table 4 for Ns) consisted of all children whowere being adopted into the United States (USAadoption) under the auspices of the InternationalAssistance Group, a Pittsburgh adoption agencyspecializing in placing children from St. Petersburgand other cities in the Russian Federation. In addi-tion, all children who were being returned to birthparents or taken by relatives of the birth parent(i.e., birth family), entering foster care (i.e., fostercare; foster parents are paid monthly by the gov-ernment), and adopted or being taken by “non-relative kin” (a classification of parents whoreceive some financial assistance and may or maynot adopt the child if and when parental rights arelegally terminated) in the St. Petersburg area (col-lectively St. Petersburg adoption). Only adopted chil-dren who spent at least 3 months in residence inone of the three BHs described above after theinterventions had been completely implementedwere included, a residency requirement deemedthe minimum time necessary for the BH interven-tions to have an effect on children. Their averagelength of residency was 12.8 months (SD = 8.9).Therefore, this sample does not include childrenwho “passed through” a BH in less than 3 months,did not transition to families, were adopted toRussian families living outside St. Petersburg,adopted to the United States through other agen-cies, adopted to other countries, or aged out of theinstitution. Consequently, this sample includedfewer children with disabilities or other develop-mental problems than the total BH population. Nochildren were sampled between the interventionand follow-up project periods because of lack offunding.

Departing children fell into three time periods:Those who departed (a) during the interventionproject, (b) during the first 2 years of the follow-upproject (Follow-up Time 1), and (c) during the thirdand fourth years of the follow-up project (Follow-upTime 2). Ns varied with time period, BH, andassessment and are reported with the results.


Assessments

HOME Inventory. The 24-month institutionalversion of the HOME Inventory (Bradley & Cald-well, 1995; Caldwell & Bradley, 1984) was selectedbecause this age was approximately in the middleof the age range of children residing in theseBHs. It consists of 5–11 items in each of six sub-scales (responsivity, acceptance, organization,learning materials, involvement, and variety) thatare scored yes or no. Subscales and total scoresrepresent the number of items scored yes. Noticethat a caregiver needs to display a given itemonly once to at least one child to be given a yesfor that item. Thus, the HOME reflects the pres-ence of certain characteristics, but generally nottheir prevalence within a ward. In addition, a spe-cial sociability subscale was created by the authorsconsisting of the number of yes answers to allitems on the HOME that pertained to the socialbehavior of caregivers and children. Specifically,the sociability scale included 21 items (e.g., “Care-giver continuously vocalizes to children at leasttwice,” “Caregiver responds verbally to children’svocalizations or verbalizations,” “Caregiver con-verses freely and easily,” “Caregiver investsmaturing toys with value via personal attention,”“Caregiver structures children’s play periods,”etc.; see The St. Petersburg-USA OrphanageResearch Team, 2008).

The HOME Inventory was conducted focusing onan individual caregiver attending to 5–14 children,typically with other caregivers present. An assess-ment consisted of 60 min of observation, includingat least 45 min in which the children were notasleep and not being fed, changed, or bathed (i.e.,“free time”), plus 10–15 min in which they wereengaged in feeding, changing, or bathing.

The HOME is one of the most widely usedassessments of the environment and caregiverbehavior in home and nonresidential group set-tings, and the total score and subscale scores havebeen found to correlate with motor, social, andmental competence in young children (Bradley,Corwyn, Burchinal, McAdoo, & Coll, 2001), to pre-dict mental performance in adolescents (Pettit,Bates, & Dodge, 1997), and to be sensitive to pre-ventive early interventions (Bakermans-Kranenburg,van IJzendoorn, & Bradley, 2005).

Battelle Developmental Inventory. The BattelleDevelopmental Inventory (LINC Associates, 1988)was used to assess children’s general behavioraldevelopment. It was selected because the itemswere more relevant (“authentic”) to the BH con-

text and it was better suited to children withmild disabilities than many other tests of generalbehavioral development. It is appropriate for chil-dren birth to 95 months, and provides a totalscore plus subscales for adaptive behavior (i.e.,self-maintenance skills), gross motor, fine motor,communication, cognition, and personal-socialdevelopment (see details in The St. Petersburg-USA Orphanage Research Team, 2008). However,while the full scale was administered during theintervention, only the communication and per-sonal-social subscales were given at departureduring the follow-up project and reported here,because the intervention produced the maximumdifferences between BHs on these two subscales.Scores are reported as developmental quotients(DQs), which consisted of mental age divided bychronological age.

Procedure and Reliability

HOME. The HOME was administered in anentire BH before any training or structural changeswere begun (i.e., preintervention), approximately ayear after the interventions were completely imple-mented in a BH, and every year thereafter until theend of the intervention project (2005). Approxi-mately 2 years later, the follow-up project was initi-ated, and the HOME was administered again inapproximately 2008 and 2011.

Only the preintervention HOME assessment, thelast two assessments (i.e., Intervention Time 1 andTime 2) during the intervention period thatoccurred in approximately 2003 or 2004 and 2005,and the two follow-up assessments Follow-upTime 3 and Follow-up Time 4 (in 2008 and 2011)are reported here. The actual time between assess-ments varied somewhat between BHs, so thelength of time between assessments is approxi-mate. Also, HOME assessments given to replace-ment caregivers between these all-BH assessmentsduring the intervention project are not utilized inthis report, which means that some data presentedhere are not identical to similar data presented inThe St. Petersburg-USA Orphanage Research Team(2008).

A team of four assessors was trained by twoexperts in the HOME. After practicing, formal reli-ability was determined before the intervention pro-ject and approximately 2 years later. Correlationsbetween assessors were .90+ for the six subscalesand .98 for the total score. Agreement was slightlylower 2 years later in part because of lower vari-ability in the reliability sample (for details, see The

1738 McCall et al.

St. Petersburg-USA Orphanage Research Team,2008). The same team of assessors (minus one wholeft after the intervention project) conducted all ofthe HOME assessments across more than 10 yearsof this project.

Battelle Developmental Inventory. Battelles wereadministered within the BH approximately 1 monthbefore children’s departure (M = 4.6 weeks, SD =7.1).

During the intervention project, children werecomprehensively assessed periodically, but onlydata from the Battelle personal-social and commu-nication subscales from their departure assessmentare reported here (i.e., Intervention). Duringthe follow-up project, departure scores obtainedduring the first 2 (i.e., Follow-up Time 1) andnext 2 years (i.e., Follow-up Time 2) are reportedhere.

Four Battelle assessors were trained in theadministration of the Battelle, and 2 years into theintervention project two additional assessors weretrained. Formal reliabilities were obtained for allassessors. For the total score, 87% of the pairs ofscores during formal reliability were within 2points, suggesting that unreliability was less than2% of the average subscale score and approximately1% of the average total score (The St. Petersburg-USA Orphanage Research Team, 2008).

Results

Estimates of Staff Turnover

The HOME data provided the opportunity toestimate staff turnover. Turnover of staff representsa potential impediment to maintaining the interven-tion, it contributes to inconsistencies of caregiversin children’s lives and thus could influence chil-dren’s developmental scores, and it could be a con-founding factor if the three BHs were markedlyand persistently different in turnover rates.

Table 1 presents estimates of staff turnover inthree time periods covering 8 years in the threeBHs in this study. The first time period is approxi-mately 2003–2005, which represents the last18 months of the intervention project. The secondtime period is approximately 2005–2008 represent-ing the 3 years between the final HOME assessmentof the intervention project and the first assessmentof the follow-up project. The third time period isapproximately 2008–2011 representing the 3 yearsbetween the two assessments conducted during thefollow-up project.

Two different estimates of staff turnover werecalculated. The first represented the number andpercentage of new caregivers assessed at the secondassessment during a time period relative to the totalnumber of caregivers assessed at the first assessment

Table 1Two Estimates of Staff Turnover Over 8 Years in Three Baby Homes

BH

2003–2005 =

18 months 2005–2008 = 3 years 2008–2011 = 3 yearsWt.’d ave.annual ratecT1–T2a T2b T2–T3a T3b T3–T4a T4b

T + SC 10/46 10/56 29/56 29/50 17/50 17/4922% 18% 52% 58% 34% 35%

Annuald 11% 9% 17% 19% 11% 12% 13% 14%TO 8/45 8/52 30/52 30/49 23/49 23/50

18% 15% 58% 61% 47% 46%Annuald 9% 8% 19% 20% 16% 15% 15% 15%NoI 8/43 8/51 2/51 2/30 12/30 12/39

19% 16% 4% 7% 40% 31%Annuald 9% 8% 1% 2% 13% 10% 8% 7%Total 26/134 26/159 61/159 61/129 52/129 52/138

19% 16% 38% 47% 40% 37%Annuald 10% 8% 13% 16% 13% 13% 12% 13%

Note. T + SC = Training + Structural Changes in Baby Home; TO = Training Only in Baby Home; NoI = No Intervention in BabyHome; T1, T2, T3, T4 = time periods; Wt.’d ave. = averaged weighted by Ns in each group.aThe number and percentage of new caregivers assessed at the second assessment relative to the total number of caregivers assessed atthe first assessment. bThe number of new caregivers assessed at the second assessment relative to the total number of caregiversassessed at the second assessment. cThe average annual rate weighted by the number of years in each of the three time periods.dApproximate annual turnover rate for each period of time; 18 months is used as the length of T1–T2. Annual rate may not equalperiod rate divided by years because of rounding.


during that time period. This constituted the num-ber and percentage of new caregivers hired over thetime period relative to the number at the beginningof the time period. The second estimate of staff turn-over represented the number of new caregiversassessed at the second assessment during the desig-nated time period relative to the total number ofcaregivers assessed at that second assessment. Thiswas equal to the percentage of new caregivers atthe second assessment. Both estimates are presentedin Table 1, and the percentage turnover as well asthe annualized percentage for that time period(boldface) are also presented, the latter being themost comparable figure across time periods andbetween BHs.

First, the marginal annualized turnover rates forthe two estimates were very similar within BHsand within time periods. Second, the weightedaverage annual turnover rates for T + SC and TOwere very similar (14%–16%), but the rate for NoIwas somewhat lower (7%–8%).

Third, turnover rates averaged 13% per yearand generally were not persistently different forthe three BHs, minimizing the potential confound-ing role of turnover. However, rates were some-what variable across time period and for BHswithin time periods, especially 2005–2008 and2008–2011. Intervention and administrative circum-stances and changes in society may havecontributed to this variability. The T + SC BH hadsomewhat higher turnover rates between the endof the intervention and the beginning of follow-up(2005–2008), because the intervention project hadsupported a few additional staff, some of whomneeded to be let go afterward. The TO BH hadhigher turnover rates during the last two timeperiods, because they let go part-time staff infavor of keeping staff who would work moredays per week. NoI was threatened by the localgovernment with being closed entirely during themiddle time period, but ultimately remained openwith approximately half the number of childrenand caregivers. The slight increase in turnoverrates after 2005 may reflect an improving Russianeconomy that offered more alternative employ-ment opportunities for caregivers.

Maintenance of BH Differences in HOME InventoryScores

Two types of analyses over the 8-year period ofthis study were performed to deal with the fact thatcaregivers turn over at approximately 13% a yearand only 32 of the original 134 (24%) caregivers

across the three BHs had all four HOMEassessments. The first type of analysis was a quasi-cross-sectional analysis in which the sample of allcaregivers available during a given assessment per-iod was used even though some caregivers wererepresented in more than one time period. These“quasi-independent group analyses” violated theassumption of independence, but that violationwould reduce the sensitivity and power of the anal-ysis relative to a longitudinal approach. This analy-sis provided a “snapshot” of the environment andcaregiver behavior of these BHs at each time point,and included the maximum number of caregivers. Asecond approach was to use overlapping longitudinalsegments (e.g., Time 1–2, Time 2–3, Time 3–4), whichprovided information on true intraindividualchange or stability in HOME scores for caregiverswho were present at both adjacent assessments,albeit with smaller sample sizes and over shorterspans of time.

A major question was whether the differencesbetween BHs in HOME Inventory scores producedby the intervention conditions would be maintainedafter the intervention project ended and through thefollow-up project. Table 2 presents the means, stan-dard deviations, and Ns for the three interventionconditions (T + SC, TO, NoI) at five time points(PreIntervention, Intervention Times 1 and 2, andFollow-up Times 3 and 4), and Figure 1 presents theresults graphically. Of primary interest in this studyare the data from the four time periods followingthe intervention; the preintervention means indicatethe status of the three BHs prior to the interventionto illustrate the amount of change subsequently pro-duced by the intervention. During intervention Time1 and Time 2, only the scores for T + SC were sig-nificantly higher than their preintervention scores(Table 2). An analysis of variance of InterventionCondition (T + SC, TO, NoI) 9 Time (1–4) pro-duced a significant difference among the three inter-vention conditions, F(2, 548) = 84.39, p < .001,g2 = .24, and no BH 9 Time interaction, F < 1. Thisindicates that the interventions produced differencesbetween the BHs, especially T + SC versus the othertwo BHs, and these differences between BHsremained over a period of at least 8 years, 6 yearsafter the intervention project ended.

This analysis also produced a significant timeeffect, F(3, 548) = 36.34, p < .001, g2 = .17 (Figure 1),consisting of an increase in total HOME scoresbetween the end of the intervention (Time 2) and thebeginning of the follow-up period (Time 3). Impor-tantly, however, this occurred for all three BHs andthe differences between the three BHs remained.

1740 McCall et al.

Scheff�e tests comparing each pair of time points indi-cated that Intervention Times 1 and 2 and Follow-upTimes 3 and 4 were not significantly different(p > .62) but each of the two intervention times weresignificantly different from each of the two follow-uptimes (p < .001).

Results were similar when the Sociability scoreand the six conventional subscales of the HOMEwere analyzed. For the Sociability score, there wasa BH effect, F(2, 548) = 69.44, p < .001, g2 = .20, butno BH 9 Time interaction, F(6, 548) = 1.47. A timeeffect, F(3,548) = 36.99, p < .001, g2 = .17, wasaccompanied by the same pairwise results as fortotal score (i.e., Intervention < Follow-up). Multi-variate analysis of variance (MANOVA) results forthe six HOME subscales were similar with a BHeffect, F(12, 1,086) = 26.46, p < .001, and a timeeffect, F(36, 2,387) = 3.04, p < .001. Although the

multivariate interaction was significant in this case,univariate tests indicated a nonsystematic patternof scores for individual BHs over time.

Overlapping longitudinal analyses. The means,standard deviations, Ns, and statistical results forthree overlapping longitudinal analyses comparingTime 1–2, Time 2–3, and Time 3–4 are presented atthe top of Table 3. The means for these longitudinalsamples are very close to the quasi-cross sectionalsamples (Table 2). The analyses of variance con-ducted within each of these three time periods werevery uniform in their main results, specifically thatthere was an intervention condition effect in eachtime period accounting for partial g2 of .21–.37, andat the same time in each case the InterventionCondition 9 Time interaction had an F � 1. A sig-nificant time effect occurred between the end ofintervention and the beginning of follow-up (2005–2008), consistent with the quasi-cross-sectional anal-ysis reported above.

Simple effects tests between each pair of inter-vention conditions were conducted within the con-text of each of these three longitudinal analyses,and the three BHs were also compared separatelyat Time 2, Time 3, and Time 4. In each of thesecomparisons, a significant difference was producedbetween each pair of intervention conditions, exceptin the longitudinal analysis across Times 2 and 3in which both T + SC and TO were significantlydifferent from NoI but not from each other. Itshould be noted that by Time 3 and Time 4 allthree BHs were significantly different from oneanother indicating that the Time 3 score for TO wasan unusual deviation within that period.

Therefore, both quasi-cross-sectional and over-lapping longitudinal analyses show that the inter-vention-produced differences in HOME scoresamong the three BHs were maintained throughout

Table 2Mean (Standard Deviations) Home Observation for Measurement of the Environment (HOME) Inventory Total Scores From PreinterventionThrough Follow-Up for Three Orphanage Intervention Conditions

T + SC TO NoI

Preintervention 35.1 (4.3) (N = 59) 34.5 (3.2) (N = 53) 33.3 (3.4) (N = 52)InterventionTime 1 37.2 (2.4)*** (N = 46) 34.7 (3.1) (N = 45) 33.4 (3.0) (N = 43)Time 2 36.6 (3.0)** (N = 56) 34.7 (3.1) (N = 52) 32.6 (4.1) (N = 51)

Follow-upTime 3 39.4 (2.2)*** (N = 50) 38.0 (2.8)*** (N = 49) 34.7 (3.1)† (N = 30)Time 4 39.6 (2.2)*** (N = 49) 37.8 (2.6)*** (N = 50) 35.5 (4.2)*** (N = 39)

Note. T + SC = Training + Structural Changes in Baby Home; TO = Training Only in Baby Home; NoI = No Intervention in Baby Home.†p = .06. **p � .01. ***p � .001 for simple effects tests of HOME Inventory total score versus preintervention level for that BabyHome.

T+SC

TO

NoI

32

33

34

35

36

37

38

39

40

Preintervention Time 1 Time 2 Time 3 Time 4

Mea

n H

OM

E To

tal S

core

s

Intervention Follow-up

Figure 1. Mean Home Observation for Measurement of theEnvironment (HOME) total score as a function of time for thethree intervention conditions. T + SC = Training + StructuralChanges in Baby Home; TO = Training Only in Baby Home; NoI= No Intervention in Baby Home.


the intervention project as well as for 6 years after-ward during the follow-up project.

Old versus new caregivers. It was of some interestto note whether replacement caregivers (“new”),who received a mini training course, quicklyadapted to the caregiving environment of their par-ticular intervention condition relative to caregiverswho had been in that BH for at least 2–3 years. Con-sequently, separately at Times 2, 3, and 4, caregiverswere divided into those who had been present forthe previous HOME assessment (“old” caregivers)versus those for whom the current HOME assess-ment was their first (“new”).

The means, standard deviations, Ns, and statisti-cal results for these comparisons are presented in thebottom of Table 3. The statistical analysis consistedof Intervention Condition 9 Old versus New Care-givers for the HOME assessment at a particular timepoint. Again, the results were quite uniform inshowing that BH differences were highly significantat each of the three time points, and simple effectstests showed that in each case all three BHs werepairwise significantly different from one another.Furthermore, there was no significant InterventionCondition 9 Old versus New Caregivers interaction,indicating that new caregivers adjusted to the level

of caregiving in their respective BHs. At one timepoint (2005–2008), there was a significant differencebetween old and new caregivers, but ironically thiswas a case in which new caregivers scored higher onthe HOME than the older caregivers, but again with-out interaction with intervention condition.

Children’s Battelle Subscale Departure Scores

Table 4 presents the departure Battelle DQmeans (standard deviations) for the personal-socialand communication subscales as a function of thethree intervention conditions and the three timepoints. These data were analyzed with an Interven-tion Condition (T + SC, TO, NoI) 9 Time (Interven-tion, Follow-up Time 1, Follow-up Time 2)independent groups MANOVA. Preliminary testsindicated no effects for age at placement, nor didage at placement meet the requirements for it to beused as a covariate in these analyses. The mainresults of interest were a significant multivariateintervention condition effect, F(4, 694) = 7.58,p � .001, g2 = .04, and no interaction with time,F(8, 694) = 1.57, p = .13. The intervention conditioneffect was significant in univariate tests for per-sonal-social, F(2, 348) = 5.50, p = .004, g2 = .03, and

Table 3Overlapping Longitudinal Analyses of Intervention Conditions Over Years (Top) and Comparison of Old and New Caregivers at Three Time Pointsfor Home Observation for Measurement of the Environment Inventory Total Scores

Longitudinal

2003 (T1)–2005 (T2) 2005 (T2)–2008 (T3) 2008 (T3)–2011 (T4)

Intervention Intervention Follow-up Follow-up

T1 T2 T2 T3 T3 T4

T + SC(N = 46, 21, 24)

37.2 (2.4) 36.7 (3.0) 37.0 (2.2) 38.8 (2.4) 39.3 (2.6) 40.2 (1.9)

TO(N = 44, 19, 17)

34.8 (3.1) 34.6 (3.2) 35.2 (3.2) 38.0 (2.9) 37.4 (3.5) 38.1 (2.5)

NoI(N = 43, 28, 19)

33.4 (3.0) 32.6 (4.1) 32.9 (3.5) 34.4 (3.0) 34.3 (3.0) 35.1 (3.6)

BH F(2, 260) = 34.47***, g2 = .21 F(2, 130) = 26.41***, g2 = .29 F(2, 114) = 33.91***, g2 = .37Time F(1, 260) = 1.80 F(1, 130) = 15.57*** F(1, 114) = 2.34BH & time F(2, 260) � 1 F(2, 130) � 1 F(2, 114) � 1Cross-sectional T2 T3 T4

Old New Old New Old NewT + SC 36.7 (3.0) (N = 46) 36.6 (2.9) (N = 10) 38.8 (2.4) (N = 21) 39.9(1.9) (N = 29) 40.2 (1.9) (N = 24) 39.0 (2.3) (N = 25)TO 34.6 (3.2) (N = 44) 34.9 (2.9) (N = 8) 38.0 (2.9) (N = 19) 38.0(2.7) (N = 30) 38.1 (2.5) (N = 17) 37.6 (2.6) (N = 33)NoI 32.6 (4.1) (N = 43) 32.4 (4.1) (N = 8) 34.4 (3.0) (N = 28) 38.5(0.7) (N = 2) 35.1 (3.6) (N = 19) 36.0 (4.8) (N = 20)BH F(2, 153) = 10.81***, g2 = .12 F(2, 123) = 5.65**, g2 = .08 F(2, 132) = 20.15***, g2 = .23Old versus new F(1, 153) � 1 F(1, 123) = 5.76*, g2 = .05 F(1, 132) � 1BH 9 Old/New F(2, 153) � 1 F(2, 123) = 2.15 F(2, 132) = 1.40

Note. T + SC = Training + Structural Changes in Baby Home; TO = Training Only in Baby Home; NoI = No Intervention in BabyHome; BH = Baby Home; T1, T2, T3, T4 = time periods.*p < .05. **p < .01. ***p < .001.

1742 McCall et al.

for communication, F(2, 348) = 15.45, p � .001,g2 = .08.

Consistent with the HOME results, there was alsoa time effect, F(4, 694) = 3.30, p = .011, g2 = .02, andthis was significant in univariate tests for both per-sonal-social, F(2, 348) = 5.74, p = .004, g2 = .03, andcommunication, F(2, 348) = 3.78, p = .024, g2 = .02.Although there was no significant multivariate Inter-vention Condition 9 Time interaction as reportedabove, the univariate tests of this interactionapproached significance for personal-social, F(4,348) = 2.15, p = .074, and were significant for com-munication, F(4, 348) = 2.88, p = .023, g2 = .03.

These effects can be seen in Figure 2. The overallmultivariate results indicated that the three inter-vention conditions maintained their differences

across these three time periods. Scores tended toincrease between the intervention and Follow-upTime 1. The multivariate interaction was not signifi-cant. While T + SC maintained much higher scoresthan NoI throughout the follow-up period, TO’sscores during the follow-up period were as high asthose from T + SC.

Discussion

Main Results

This article demonstrates that a comprehensiveintervention in a St. Petersburg BH was maintainedand continued to be associated with better develop-mental scores for children through the intervention

Table 4Departure Battelle Subscale Developmental Quotient Means (Standard Deviations) as a Function of Intervention Conditions and Time

Intervention condition BDI subscale Intervention Follow-up Time 1 Follow-up Time 2 Total

T + SC (N = 46, 59, 42; 147)Personal-social 77.8 (19.7) 75.9 (25.9) 79.7 (20.1) 77.6 (22.4)Communication 81.4 (24.0) 75.2 (23.2) 83.9 (20.2) 79.6 (22.8)

TO (N = 61, 31, 19; 111)Personal-social 65.9 (17.5) 81.3 (23.9) 79.8 (24.9) 72.6 (21.9)Communication 67.0 (15.1) 80.8 (22.1) 80.3 (23.1) 73.1 (19.8)

NoI (N = 43, 43, 13; 99)Personal-social 61.4 (17.2) 71.5 (21.5) 70.4 (21.4) 67.0 (20.1)Communication 58.9 (17.0) 65.3 (18.9) 67.4 (24.1) 62.8 (19.0)

Total (N = 147, 111, 99; 357)Personal-social 68.3 (19.2) 75.7 (24.2) 78.1 (21.7) 73.1 (22.0)Communication 69.1 (20.6) 73.3 (22.3) 80.1 (22.2) 73.0 (21.9)

Note. BDI = Battelle Developmental Inventory; T + SC = Training + Structural Changes in Baby Home; TO = Training Only in BabyHome; NoI = No Intervention in Baby Home.

55

60

65

70

75

80

85

90

95

100

Interven on Time 1 Time 2 Total

Bael

leSu

bsca

leD

Q's

Follow-up

NoI

TO

T+SC

NoI

TO

T+SC

Solid=Personal-SocialOpen=Communca on

Figure 2. Personal-social (solid) and communication (open) Battelle subscale scores for the three intervention conditions as a function oftime. T + SC = Training + Structural Changes in Baby Home; TO = Training Only in Baby Home; NoI = No Intervention in BabyHome.


project period as well as for approximately 6 yearsafter the intervention project ended compared totwo comparison BHs. The interventions consisted ofT + SC, which included training caregivers inwarm, sensitive, contingently responsive, and child-directed caregiver–child interactions plus a varietyof structural changes that reduced group sizes,assigned primary and secondary caregivers, elimi-nated periodic graduations to wards, integratedgroups by age and disability status, and promotedmore consistent relationships with fewer caregivers.Another BH received TO without the structuralchanges, and a third had NoI.

The results were quite uniform for the HOMEInventory total scores, a sociability index madeup of 21 items reflecting caregiver–child socialinteractions, and the HOME subscales. They dem-onstrated that the intervention produced anincrease in scores in T + SC, but not TO and NoI,from preintervention levels to Intervention Times1 and 2 (approximately the last 1–2 years of theintervention project). Moreover, across four timepoints spanning approximately 8 years (Interven-tion Times 1 and 2, Follow-up Times 3 and 4),the differences between intervention conditions(BHs) remained significant and did not interactwith time.

In addition, HOME scores remained steady overthe two time points during the intervention and thetwo time points during follow-up, but they roseunexpectedly over the 3 years between the end ofthe intervention project and the beginning of thefollow-up (Times 2 and 3, 2005–2008). This was truefor HOME total score, the sociability index, and theset of six HOME subscales, and it was also true forall three intervention conditions. This increase fol-lowing the end of the intervention project contrastswith the usual expectation that intervention effectstend to fade over time.

The Battelle Developmental Inventory’s personal-social and communication subscales essentially mir-rored the trend in HOME scores. Specifically, themultivariate differences in children’s behavioraldevelopment scores produced by intervention con-ditions were maintained during the last years of theintervention project and for 6 years thereafter withno multivariate interaction with time. Similar to theHOME results, Battelle scores increased after theintervention period, and although the multivariateinteraction was not significant, univariate resultsshowed this change over time to increase for TOand NoI but not for T + SC, and TO’s scoresapproached or exceeded those of T + SC during thefollow-up period.

Maintenance of Intervention Effects

Several factors might have contributed to themaintenance of intervention effects on caregivers,the environment, and children’s development.

First, the nature of the intervention may be afactor. In contrast to interventions consisting of sup-plementary stimulation often provided by researchassistants or special supplementary staff in whicheffects typically diminish after the intervention isterminated, the current intervention was aimed atchanging the behavior of all regular staff membersin an entire institution. Moreover, the interventiondid not focus on having caregivers perform specificactions with children, but rather emphasized thegeneral principles of warm, sensitive, contingentlyresponsive, child-directed caregiver–child interac-tions that caregivers could then implement on thewards in a variety of different ways to fit individualchildren. More speculatively, it is possible that thechildren themselves helped to maintain the caregiver–child interactions. During the intervention, children’sdevelopment in T + SC improved in noticeableways; children were more active and responsive tocaregivers, and they expected caregivers to talk andinteract with them. Although this was more visiblein older children, age integration was part of theT + SC intervention so that older children resided inevery ward. Thus, it is possible that the reciprocalnature of caregiver–child interactions helps to main-tain those sensitive and responsive engagements.

A second contributor was the deliberate attemptsduring the planning of the intervention to help it per-sist (Altman, 1995). A train-the-trainer strategy hadbeen employed so that trainers would be available totrain replacement (“new”) caregivers. Such a strategymay have contributed to the finding that new care-givers scored the same on the HOME as old caregiv-ers at each time point within each BH. Also, the factthat BH differences were the same for old and newcaregivers suggests that caregivers may also adapt tothe caregiving environment in which they work. Fur-thermore, an intervention that reduced the numbersof caregivers in children’s lives, reduced the numberof children per caregiver, but did not increase thetotal number of caregiver hours could be maintainedafter the initial grant period using only the govern-ment-supplied budget for the BH.

Third, the importance of having a director com-mitted to the intervention should not be minimized.In the authors’ experience in this and studies ofinterventions in other institutions, a committeddirector is imperative; without it, staff remainunmotivated to change. It is important to note in

1744 McCall et al.

this study that the directors of the other two BHsalso wanted their respective treatments, includingthe director of NoI who believed in the effective-ness of the existing institutional procedures.

Fourth, certain activities during the follow-upperiod probably helped. For example, the monitor-ing and supervision system that was established inT + SC as part of the intervention and continuedthereafter was designed to continuously encouragecaregivers to implement on the wards the trainingthey had received. Furthermore, two of the authorsand two other staff visited T + SC and TO periodi-cally to remind caregivers of their training and pro-vide additional training sessions in the yearsfollowing the end of the intervention project.

Staff Turnover Rates

Staff turnover has the potential to corrode themaintenance of the intervention. Turnover rates inthese BHs averaged approximately 13% per year,which was relatively consistent across the threeBHs and across time, with a few exceptions. TheNoI BH was threatened with a total shut down, butultimately was allowed to remain open withapproximately half the child population and fewercaregivers. This resulted in extremely low turnoverrates between 2005 and 2008. Higher rates werereported in the other two BHs, because the numberof children being sent to orphanages declined dur-ing this period of time; BHs tried to maintain stafflevels, but some caregivers left to take other jobs.

A turnover rate of 13% per year could mean thatone in four caregivers is replaced in 2 years andmore than one in three in 3 years—a span of timethat characterizes the residential term of someinfants and young children. Indeed, turnover rateplus other factors contributing to many and chang-ing caregivers meant that in these BHs during theintervention years (2003–2005), children experienced60–100 different caregivers in TO and NoI by19 months of age (the intervention in T + SCreduced this figure by approximately half; The St.Petersburg-USA Orphanage Research Team, 2008).

At a rate of 13% per year, one might estimate aturnover rate of (8 years 9 13%) > 100% over the8-year period. In fact, however, of the original 134caregivers assessed at Time 1, 32 (24%) remained atTime 4, which suggests a cumulative 8-year turnoverof 76%, not 100%. This implies that a core group ofcaregivers remain in the BHs over a prolonged per-iod of time, while some of the remaining caregiverpositions may turn over several times over the8 years.

The generalizability of these St. Petersburg (RF)staff-turnover rates to other institutions around theworld is not certain. On the one hand, the threeBHs were relatively consistent in their turnoverrates across the 8-year period. Caregiving inSt. Petersburg is a relatively low-status, low-payingjob, although contributions are made by the govern-ment toward caregiver pensions. This may not bethe case in every country. For example, in someCentral American countries caregiving in an institu-tion is a government job that brings with it anacceptable (rather than low) salary and fairly per-manent government employment that is threatenedif a caregiver leaves for a nongovernment job. Inthis environment, caregivers may be much morereluctant to leave their institutional employment.This speculation prompts a possible suggestion ofincreasing the salaries, pension contribution, or jobsecurity of caregivers as a policy change that mightpromote less staff turnover and more consistentcaregivers in the lives of institutionalized children.

Societal Improvements

An unexpected finding was that not only were care-giver behaviors and children’s development main-tained after the intervention, but they actuallyimproved. It is unlikely that the factors designed tomaintain the intervention produced this subsequentincrease in HOME and Battelle scores, because theimprovement occurred in each BH and only betweenthe end of the intervention and the beginning of thefollow-up; maintenance factors existed within the inter-vention period and within the follow-up period, butno such improvement occurred during these intervals.

We can only speculate about events that occurredin St. Petersburg predominantly between 2005 and2008 that might have influenced caregiver behaviorand the caregiving environment and presumably itseffects on children’s development (Rosas et al.,2011). Mainly, the government began special effortsto keep children in families and reduce the numberof children being sent to BHs. For example, SocialServices began to remove fewer infants and toddlersfrom at-risk families and provided at-risk families,single mothers, and mothers with special needs withmore financial help as well as food and clothing.Also, a media campaign encouraged family carealternatives, and the Russian economy improved; asa result there were fewer “social risk” children,some of whom previously had been relinquished toBHs, and more families could afford to adopt.

While the number of children being sent to BHsin St. Petersburg declined substantially (e.g., BH


populations declined approximately 10%–15%), BHdirectors attempted to maintain their staffs. The netresult was smaller groups of children cared for bynearly the same number of caregivers, which pro-duced lower children–caregiver ratios. Thus, thenumber of different caregivers experienced by achild likely did not change greatly during the inter-vention in T + SC, but this factor was not found tomediate intervention effects (Rosas et al., 2011);however, the number of children per caregiver diddecrease. It is possible that when confronted withfewer children, caregivers are less rushed and havemore time to interact with children, and this envi-ronment draws more typical adult–child behaviorfrom the caregivers. Thus, smaller children–care-giver ratios may be a major contributor to institu-tionalized children’s development, a propositionconsistent with differences between institutions inthis regard and children’s development (Groarket al., 2012) and the literature on nonresidentialchild care (e.g., Kontos & Fiene, 1987; NICHD EarlyChild Care Research Network, 2000).

Even if this explanation were correct, it is sur-prising that smaller groups and fewer children percaregiver alone could apparently produce as muchor more improvement in HOME scores as the origi-nal comprehensive intervention. But dramaticimprovement over time in HOME scales may beassociated with the HOME scale itself and its appli-cation to group settings. Items on the HOME arescored yes or no as a function of whether the care-giver performed the indicated action at least oncewith at least one child during the 1-hr observationperiod. With fewer children and more time avail-able for interactions with them, it may not be sur-prising that caregivers had more opportunity toperform the behaviors on the HOME at least onceduring the observation period.

Intervention differences in children’s personal-social and communication development were alsomaintained during and after the intervention, andthese scores also tended to improve between theend of intervention and follow-up but not as dra-matically as the HOME scale. The intervention inT + SC produced marked changes in total BattelleInventory scores, improving children exposed tothe T + SC intervention for 9 months or morefrom an average of DQ = 57 to 92 = 35 DQpoints (The St. Petersburg-USA OrphanageResearch Team, 2008). Thus, we believe thatreducing group size and the number of childrenper caregiver likely improved caregiver behaviorwith children and children’s development inotherwise deficient environments, but its effects

were not as strong as those produced by a com-prehensive intervention.

The most unexpected finding was the substantialapparent improvement in Battelle scores for TO chil-dren (Figure 2) between the intervention and follow-up periods to levels equal to or exceeding those forT + SC children (although the interaction was notstatistically significant). One possible explanation isthat TO was encouraged to implement the samestructural changes as T + SC during this time period;although this was only partly accomplished, it mayhave produced some or all of this improvement.

A second explanation pertains to changes inplacement practices, partly in response to thedecline in the number of children being sent to BHsgenerally. The TO BH tried to keep the number ofchildren in residence as high as possible so theycould justify retaining staff. This BH is located nextto a maternity hospital, which enabled the BH totake many of their children whose legal and devel-opmental status were such that they could be morequickly transitioned to domestic families. Theseinfants had significantly higher birth weights,lengths, and head circumferences than infants dis-tributed by the health administration to the otherBHs. These children tended to be transferred toadoptive and foster families, but the total numberof children transitioned nevertheless declined inpart because some of the transitions occurred beforethe 3-month residency requirement of this study.The NoI BH was reduced by the administration toapproximately half its original size. In contrast,T + SC had always been willing to take childrenwith more severe health and developmental prob-lems and disabilities; such children became a higherpercentage of those being sent to BHs, and theT + SC director was vigorous in trying to reunifymany of these children with their biological families(i.e., 40% of the T + SC vs. 15% of the TO children).These recent historical changes meant that the num-ber of children transitioning to families declined inTO and NoI but not in T + SC, and the develop-mental level of such children (Figure 2) appears tohave increased somewhat in TO and declined a bitin T + SC (although this interaction was not statisti-cally significant).

Assets and Limitations

The St. Petersburg study is the most comprehen-sive intervention in institutions for infants andtoddlers using regular institutional staff caregivers(Bakermans-Kranenburg et al., 2008), and it isthe most comprehensively evaluated institutional

1746 McCall et al.

intervention. It is also the only one that made delib-erate attempts in the planning of the intervention aswell as after the intervention project terminated tosupport the maintenance of the intervention and itsbenefits for children. Furthermore, this is the onlyreport of a successful attempt to maintain an inter-vention in an institution for infants and toddlersover the years of the intervention project itself aswell as for 6 years after its termination.

This report has several limitations. Unfortu-nately, funding was not available for the 2 yearsbetween the end of the intervention and the start ofthe follow-up project, and thus no children depart-ing these institutions during this 2-year intervalwere given departure assessments. Furthermore,only two Battelle subscales were administered tochildren at departure during the follow-up com-pared to the full-scale Battelle and other behavioralmeasures administered during the intervention pro-ject. Moreover, several events beyond our controloccurred after the intervention terminated. Forexample, the NoI BH was reduced to approximatelyhalf of its original size. Furthermore, changes inSt. Petersburg and the Russian Federation reducedthe number of children being sent to BHs but thenumber of staff declined less, producing smallergroup sizes and fewer children per caregiver, whichwe interpret to have produced higher HOME andBattelle scores, especially in TO and NoI. Further-more, the local government changed how theyplaced children in these institutions and that mayhave influenced children’s departure scores.

Is it possible for the societal changes to explain themain finding of the maintenance of interventioneffects? Although such factors likely contributed tothe increase in HOME and Battelle scores over time,the general result was that the differences between theintervention conditions were maintained across time,which could not easily be explained by societalchanges that applied to all three BHs. Moreover,although TO appeared to catch up with T + SC withrespect to children’s development but not HOMEscores, this appears tied to the government sendingTO more developmentally advanced children,because the difference between T + SC and NoIremained substantial.

A major limitation would appear to be the lackin this report of preintervention data to demon-strate that the differences between BHs that weremaintained during and following the interventionwere not associated with preexisting developmentaldifferences in the children. The report of the origi-nal intervention (The St. Petersburg-USA Orphan-age Research Team, 2008) plots and analyzes

baseline-to-departure changes or covaries initialstatus, making it quite clear that the interventioneffects—that were maintained over 6 years in thisstudy—were not due to selective sampling differ-ences between BHs. The same is true for the main-tenance of the HOME results.

A related concern is the possibility that BHswere assigned different types of children over theyears and that children destined for different typesof families were developmentally different. Arecently completed study of the same children rep-resented in this report shows that children whowere transitioned to U.S. adoptive families and St.Petersburg adoptive, foster, and biological familiesfrom these three BHs were not different in birthweight, length, and head circumference; departureheight, weight, and Battelle scores; and length ofresidence in the BH. Thus, although some shiftingin allocations and departure destinations betweenBHs did occur during follow-up, the lack of overalldevelopmental differences associated with thesefactors minimizes the likelihood that they played amajor role in the maintenance effect.

Another limitation is that this study used onlythose BH children who transitioned to U.S. familiesthrough one adoption agency and to Russian fami-lies. It seems unlikely that the reported results areunique to this subsample. The children studied areindeed selected (by parents), but by far the largestgroup in the BHs who were not studied are alsoselected for adoption to the United States andWestern Europe but through other agencies. Agreed,children who remain in the BHs are more likely tohave disabilities and would score lower, but T + SCwould have a disproportionate number of these.Even so, the T + SC intervention also improved thedevelopment of children with disabilities (The St.Petersburg-USA Orphanage Research Team, 2008),and there is no reason to believe that the maintainedenvironment in T + SC would not continue toimprove children with disabilities and others whoremained in the BH.

Finally, the literature reviewed above emphasizesthat intervention effects are more likely to be main-tained if the intervention itself is created in a wayto support maintenance and if various activitiesaimed at maintenance are conducted after the inter-vention project is terminated, and we believe thesteps taken in these regards contributed to themaintenance we observed. Nevertheless, this studydoes not provide direct support for this speculation,because no control condition was conducted thatomitted those activities. Furthermore, this studyonly reports the “maintenance” of an intervention


after the intervention project terminated in the sameinstitutions and with some of the original research-ers still involved with these institutions. Thus, thisis only the first step in the broader process of “sus-tainability,” which involves transferring the projectto new institutions and new staff.

References

Altman, D. G. (1995). Strategies for community healthintervention: Promises, paradoxes, pitfalls. Psychoso-matic Medicine, 57, 226–233. doi:10.1093/her/16.2.215

Bakermans-Kranenburg, M. J., van IJzendoorn, M. H., &Bradley, R. H. (2005). Those who have, receive: TheMatthew Effect in early childhood intervention in thehome environment. Review of Educational Research, 75,1–26. doi:10.3102/00346543075001001

Bakermans-Kranenburg, M. J., van IJzendoorn, M. H., &Juffer, F. (2008). Earlier is better: A meta-analysis of70 years of interventions improving cognitive develop-ment in institutionalized children. Monographs of theSociety for Research in Child Development, 73(Serial No.291), 279–293. doi:10.1111/j.1750-8606.2011.00231.x

Berry, W. L., Senter, S., Cheadle, A., Greewald, H. P., &Peason, D. (2005). Evaluating the legacy of communityhealth initiatives. American Journal of Evaluation, 26, 150–165. doi:10.1177/1524839906287066

Bradley, R. H., & Caldwell, B. M. (1995). Caregiving andthe regulation of child growth and development:Describing proximal aspects of caregiving systems.Developmental Review, 15, 38–85.

Bradley, R. H., Corwyn, R. F., Burchinal, M., McAdoo, H. P.,& Coll, C. G. (2001). The home environments of childrenin the United States. Part II: Relations with behavioraldevelopment through age thirteen. Child Development, 72,1868–1886. doi:10.1111/1467-8624.t01-1-00383

Bumbarger, B. K., & Perkins, D. F. (2008). After random-ized trials: Issues related to the dissemination of evi-dence-based interventions. Journal of Children’s Services,3, 55–64. doi:10.1108/17466660200800012

Caldwell, B. M., & Bradley, R. H. (1984). Home Observa-tion for Measurement of the Environment. Little Rock:University of Arkansas at Little Rock.

Carlson, M., & Earls, F. (1997). Psychological andneuroendocrinological consequences of early socialdeprivation in institutionalized children in Romania.Integrative neurobiology of affiliation. Annals of theNew York Academy of Sciences, 807, 419–428. doi:10.1111/j.1749-6632.1997.tb51936.x

Domitrovich, C. E., Moore, J., & Greenberg, M. T. (2011).Maximizing the effectiveness of social-emotional interventionsfor young children through high quality implementation ofevidence-based interventions. Unpublished manuscript,Pennsylvania State University.

Eckenroade, J., Campa, M., Luckey, D. W., Henderson, C.R. Jr., Cole, R., Kitzman, H., et al. (2010). Long-termeffects of prenatal and infancy nurse home visitation on

the life course of youths: A 19-year follow-up of a ran-domized trial. Archives of Pediatric and Adolescent Medi-cine, 164, 9–15. doi:10.1001/archpediatrics.2009.240

Fox, N. A., Almas, A. N., Degnan, K. A., Nelson, C. A., &Zeanah, C. H. (2011). The effects of severe psychosocialdeprivation and foster care intervention on cognitivedevelopment at 8 years of age: Findings from theBucharest Early Intervention Project. Journal of ChildPsychology and Psychiatry, 52, 919–928.

Gomez, B., Greenberg, M., & Feinberg, M. (2005). Sustain-ability of community coalitions: An evaluation of Com-munities That Care. Prevention Science, 6, 199–202.

Goodman, R. M., & Steckler, A. (1987). The life and deathof a health promotion program: An institutionalizationcase study. International Quarterly of Community HealthEducation, 8, 5–21.

Goodman, R. M., & Steckler, A. (1989a). A framework forassessing program institutionalization. Knowledge in Soci-ety: The International Journal of Knowledge Transfer, 2, 57–71.

Goodman, R. M., & Steckler, A. (1989b). A model for theinstitutionalization of health promotion programs. Fam-ily and Community Health, 11, 63–78.

Groark, C. J., & McCall, R. B. (2008). Community-basedinterventions and services. In M. Rutter, D. V. M.Bishop, D. S. Pine, S. Scott, J. Stevenson, & A. Thapar(Eds.), Rutter’s child and adolescent psychiatry (5th ed.,pp. 971–978). London: Blackwell.

Groark, C. J., & McCall, R. B. (2011). Improving institu-tions: Can we? Should we? How? International Journalof Child and Family Welfare, 14, 64–72.

Groark, C. J., McCall, R. B., Fish, L., McCarthy, S. K.,Eichner, J. C., & Gee, A. D. (2012). Structure, caregiver-child interactions, and children’s general physical and behav-ioral development in three Central American institutions.Unpublished manuscript, University of PittsburghOffice of Child Development.

Hallfors, D., Cho, H., Livert, D., & Kadushin, C. (2002).Fighting back against substance abuse: Are communitycoalitions winning? American Journal of Preventive Medi-cine, 23, 237–245.

Hallfors, D., & Godette, D. (2002). Will the “principles ofeffectiveness” improve prevention practice? Early find-ings from a diffusion study. Health Education Research,17, 461–470.

Human Rights Watch. (1999). Human Rights Watch worldreport. New York: Author.

Johnson, D. E., Guthrie, D., Smyke, A. T., Koga, S. F.,Fox, N. A., & Nelson, C. A. III (2010). Growth andassociations between auxology, caregiving environ-ment, and cognition in socially deprived Romanianchildren randomized to foster versus ongoing institu-tional care. Archives of Pediatric and Adolescent Medicine,164, E1–E10.

Kontos, S., & Fiene, R. (1987). Childcare quality, compliancewith regulations, and children’s development: The Penn-sylvania study. In D. A. Phillips (Ed.), Quality in childcare:What does research tell us? (pp. 57–80). Washington, DC:National Association for the Education of Young Children.

1748 McCall et al.

LINC Associates. (1988). The Battelle Developmental Inven-tory scoring booklet. Itasca, IL: Riverside.

Mancini, J. A., & Marek, L. I. (2004). Sustaining commu-nity-based programs for families. Family Relations, 53,339–347.

McCall, R. B. (2011). Research, practice, and policy per-spectives on issues of children without permanentparental care. In R. B. McCall, M. H. van IJzendoorn, F.Juffer, C. J. Groark, & V. K. Groza (Eds.). Childrenwithout permanent parents: Research, practice, andpolicy. Monographs of the Society for Child Development,76(Serial No. 301), 223–272.

McCall, R. B., van IJzendoorn, M. H., Juffer, F., Groark,C. J., & Groza, V. K. (Eds.). (2011). Children withoutpermanent parents: Research, practice, and policy.Monographs of the Society for Research in Child Develop-ment, 76(Serial No. 901).

Mittelmark, M. B., Hunt, M. K., Heath, G. W., & Schmid,T. L. (1993). Realistic outcomes: Lessons fromcommunity-based research and demonstration pro-grams for the prevention of cardiovascular disease.Journal of Public Health Policy, 14, 437–462.

NICHD Early Child Care Research Network. (2000).Characteristics and quality of childcare for toddlers andpreschoolers. Applied Developmental Science, 4, 116–135.

Pettit, G. S., Bates, J. E., & Dodge, K. E. (1997). Support ofparenting, ecological context, and children’s adjust-ment: A seven-year longitudinal study. Child Develop-ment, 68, 908–923.

Rosas, J. M., & McCall, R. B. (in press). Characteristics ofinstitutions, interventions, and resident children’sdevelopment. International Journal of Behavioral Develop-ment.

Rosas, J. M., McCall, R. B., Groark, C. J., Muhamedrahi-mov, R. J., Palmov, O. I., & Nikiforova, N. V. (2011).

Environmental quality as mediator between an institutionalintervention and children’s developmental outcomes.Unpublished manuscript, University of Pittsburgh,Office of Child Development.

Sarason, D. (1967). Generalized interpolation in H1.Transactions of the American Mathematical Society, 127,179–203.

Scheirer, M. (2005). Is sustainability possible? A reviewand commentary on empirical studies of program sus-tainability. American Journal of Evaluation, 26, 320–347.

Sparling, J., Dragomir, C., Ramey, S. L., & Florescu,L. (2005). An educational intervention improvesdevelopmental progress of young children in aRomanian orphanage. Infant Mental Health Journal,26, 127–142.

The St. Petersburg-USA Orphanage Research Team.(2005). Characteristics of children, caregivers, andorphanages for young children in St. Petersburg, Rus-sian Federation. Journal of Applied Developmental Psy-chology, 26, 477–506.

The St. Petersburg-USA Orphanage Research Team. (2008).The effects of early social-emotional and relationshipexperience on the development of young orphanage chil-dren. Monographs of the Society for Research in Child Devel-opment, 73(Serial No. 291).

Tibbits, M., Bumbarger, B., Kyler, S., & Perkins, D. (2010).Sustaining evidence-based interventions under real-world conditions: Results from a large-scale diffusionproject. Prevention Science, 11, 252–262.

USAID. (2009). U.S. government and partners: Workingtogether on a comprehensive coordinated and effectiveresponse to highly vulnerable children. Washington, DC:Author.


maintaining a social-emotional intervention and its benefits for institutionalized children

Documents