impact of a statewide home visiting program on parenting and on child health and development

Child Abuse & Neglect 31 (2007) 829–852

Impact of a statewide home visiting program on parentingand on child health and development�

Debra Caldera a, Lori Burrell b, Kira Rodriguez b, Sarah Shea Crowne b,Charles Rohde c, Anne Duggan c,∗

a Alaska State Department of Health and Social Services, Anchorage, AK, USAb Johns Hopkins University School of Medicine, Baltimore, MD, USA

c Johns Hopkins University, Bloomberg School of Public Health, Baltimore, MD, USA

Received 6 April 2005; received in revised form 15 February 2007; accepted 17 February 2007

Abstract

Objectives: To assess the impact of a voluntary, paraprofessional home visiting program on promoting child healthand development and maternal parenting knowledge, attitudes, and behaviors.Methods: This collaborative, experimental study of 6 Healthy Families Alaska (HFAK) programs enrolled 325families from 1/00 to 7/01, randomly assigned them to HFAK and control groups, interviewed mothers at base-line, and followed families until children were 2 years old (85% follow-up). Child outcomes included health careuse, development and behavior. Parent outcomes included knowledge of infant development, parenting attitudes,quality of the home environment, and parent-child interaction. HFAK records were reviewed to measure home vis-iting services. Home visitors were surveyed to measure knowledge, perceived effectiveness and perceived trainingadequacy.Results: There was no overall impact on child health, but HFAK group children had more favorable developmentaland behavioral outcomes. HFAK and control mothers had similar parenting outcomes except that HFAK mothershad greater parenting self-efficacy (35.1 vs. 34.6 based on the Teti Self-Efficacy Scale, p < .05). Fewer HFAKfamilies had a poor home environment for learning (20% vs. 31%, p < .001). HFAK families were more likely touse center-based parenting services (48% vs. 39%, p < .05). The impact was greater for families with lower baselinerisk (Family Stress Checklist scores < 45). There was little evidence of efficacy for families with a higher dose ofservice.

� Support was provided by the Alaska Mental Health Trust Authority and the Alaska State Department of Health and SocialServices.

∗ Corresponding author address: General Pediatrics Research Center, Johns Hopkins University School of Medicine, 1620McElderry Street, Baltimore, MD 21205-1903, USA.

0145-2134/$ – see front matter © 2007 Elsevier Ltd. All rights reserved.doi:10.1016/j.chiabu.2007.02.008

dx.doi.org/10.1016/j.chiabu.2007.02.008

830 D. Caldera et al. / Child Abuse & Neglect 31 (2007) 829–852

Conclusions: The program promoted child development and reduced problem behaviors at 2 years. Impact could bestrengthened by improving home visitor effectiveness in promoting effective parenting. Future research is neededto determine whether short-term benefits are sustained.© 2007 Elsevier Ltd. All rights reserved.

Keywords: Home visiting; Child development; Child health; Medical home

Introduction

Home visiting is a widely used strategy that has been targeted to a broad range of populations toachieve an equally broad range of parent and child outcomes. The American Academy of Pediatrics hasrecommended experimental evaluation of home visiting and the use of results from carefully conductedevaluative research in advocating for home visiting (American Academy of Pediatrics Council on Childand Adolescent Health, 1998). Gomby, Culross, and Behrman (1999) have made similar recommenda-tions. However, research on home visiting impact has yielded mixed results, raising questions of howbest to design, target and implement home-based services.

Two recent meta-analyses conclude that home visiting can promote effective parenting behavior andchild developmental outcomes (Centers for Disease Control & Prevention, 2003; Sweet & Appelbaum,2004). However, they differed in their conclusions about how effect sizes are influenced by broad programdesign features, targeting of specific populations, and articulation of program goals. An example relatesto program staffing. The first meta-analysis concluded that the impact on preventing child maltreatmentindicators was more consistent for programs staffed by professionals (Centers for Disease Control &Prevention, 2003). In contrast, the second review determined that the impact on preventing indicatorsof abuse was greater for programs staffed by paraprofessionals (Sweet & Appelbaum, 2004). However,it found that the impact on promoting child cognitive development was greater for programs staffed byprofessionals and that impact in promoting effective parenting was not associated with program staffing.Sweet and Appelbaum conclude that research is needed on other factors that can contribute to programefficacy, such as home visitors’ perceptions of program goals, fidelity of implementation, and moderationof program impact by family attributes.

Hebbeler and Gerlach-Downie examined home visitors’ perceptions of program goals and the mech-anisms for achieving them—the program’s “theory of change”—in a qualitative study to explain theresults of a concurrent randomized trial of home visiting (Hebbeler & Gerlach-Downie, 2002). The ran-domized trial found limited program success in promoting child development, its primary goal (Wagner& Clayton, 1999). Hebbeler and Gerlach-Downie determined that the program’s limited impact wasconsistent with the home visitors’ actions during visits; the home visitors’ actions, in turn, were con-sistent with their understanding of the program’s theory of change. Based on the program’s underlyingassumptions—for example, that if the mother feels good the child feels good and that parents have withinthemselves the knowledge to be good parents—the home visitors emphasized family support, placed lessemphasis on information-sharing around effective parenting practices, and de-emphasized expectationsfor parent-child interaction.

Guterman has noted that the actual duration and intensity of home visiting services is key to achievingintended outcomes (Guterman, 2001). Our own review of randomized trials of home visiting programs

D. Caldera et al. / Child Abuse & Neglect 31 (2007) 829–852 831

found, however, that few describe delivery of actual services (Duggan et al., 2000). Thus, there is needfor research comparing actual services to program models to aid in interpreting outcomes.

One widely replicated model to prevent child maltreatment and promote child health and developmentis Healthy Families America (HFA), which targets families with multiple risks for child maltreatmentof their newborns. The HFA model was inspired by Hawaii’s Healthy Start Program and is defined bycritical elements of training, staffing and service provision (Frankel, Friedman, Johnson, Thies-Huber, &Zuiderveen, 2000). HFA recommends using standardized protocols to identify and target at-risk familieson the basis of malleable psychosocial risks. Currently, there are HFA programs in 35 states; in somestates, this represents sites in a few communities, while in other states it represents statewide systems ofcare in nearly every community (L. Schreiber, personal communication, November 4, 2006).

Findings from our experimental study of Hawaii’s Healthy Start Program (HSP) have been widelypublished (Duggan et al., 2000, 1999, 2004a, 2004b, 2004c; El-Kamary et al., 2004; King et al., 2005;Nelson et al., 2005; Windham et al., 2004). This study of Alaska’s HFA program replicates our Hawaiistudy, though with a different population and with a national program whose implementation systemwas more mature than Hawaii’s HSP. The HSP and Healthy Families Alaska (HFAK) randomized trialsaddress issues raised in the home visiting research described earlier. The studies assess impact on achievingintended outcomes, relate impact to service delivery, assess parent outcomes as mediators of impact onchild outcomes, explain service delivery in terms of the program model and implementation system, andexamine family attributes as moderators of program impact.

This paper focuses on HFAK’s impact on promoting child health and development as mediated by itsimpact on parenting attitudes, knowledge, and behaviors. A companion paper reports impact on preventingchild maltreatment and parent risks for maltreatment (Duggan et al., 2007).

Methods

HFAK program model

The Alaska Legislature established the state’s first HFAK program in 1995. HFAK was designed usingHFA site development guides, training and technical assistance. Like HFA, it aims to promote positiveparenting, child health and child development (Frankel et al., 2000; Healthy Families America, 2001). Acompanion article in this volume includes a complete description of the HFAK program model, includinghome visitor and supervisor training; the role of the home visitor; screening and assessment; and programeligibility (Duggan et al., 2007).

HFAK has explicit performance indicators around child health and development. These include that100% of children will be screened for developmental delay at specific time points, that all who fail tomeet milestones will be referred for follow-up, and that ≥90% of those referred will follow through withthe referral. In addition, HFAK sites are to assure that ≥95% of children have a Medical Home and that≥90% are fully immunized by 2 years.

Setting and sample

The study, which began in July 1999, focused on six of the state’s seven HFAK programs. The otherprogram was not included because the state-level staff felt it was not mature enough to be effective in


achieving the intended goals. The study was approved by the institutional review boards of The JohnsHopkins University School of Medicine and the hospitals where study families were recruited.

HFAK staff identified at-risk families using their usual protocol from January 2000 through July 2001.Of 388 at-risk families who met study inclusion criteria, 364 (94%) signed informed consent statementsagreeing to take part in the study and were randomized. Assignments to the HFAK group (n = 179)and control group (n = 185) were predetermined using a table of random numbers, equal allocation, andrandomization within site in blocks of 10.

Research staff completed the study’s baseline interview with the mother within a month of studyenrollment. Overall, 325 (90%) of those randomized were interviewed at baseline (HFAK group, n = 162;control group, n = 163). The other families either explicitly declined the baseline interview (n = 12) orcould not be located (n = 27).

Data collection and measurement

Baseline data were collected via structured maternal interviews and review of HFAK program records.Follow-up data were collected from four sources when the children were 2 years old: (1) pediatricmedical records; (2) interviews with the child’s primary caregiver and/or birth mother; (3) observationof the home environment and of primary caregiver/child interaction; and (4) child developmental testing.Trained research staff blinded to family group assignment and independent of HFAK collected baselineand follow-up data.

Baseline family attributes

A maternal interview was used to measure baseline demographic and risk variables. Risks includedmaternal mental health, maternal substance use, and partner violence. Maternal depressive symptomswere measured by the Center for Epidemiological Studies Depression Scale (CES-D) (Husaini, Neff,Harrington, Hughes, & Stone, 1980; Radloff, 1977). General mental health was measured by the 5-itemversion of the Mental Health Index (MHI-5), an overall measure of anxiety and depressive symptoms(McHorney & Ware, 1995).

Maternal substance use was measured as illicit drug use or problem alcohol use. Maternal illicit druguse was defined as self-report of any drug use in the past 2 years. Maternal problem alcohol use wasdefined as self-report of alcohol use in the year prior to pregnancy with a history of alcohol problems,as indicated by a CAGE Questionnaire score = 2 positive (Fiellin, Reid, & O’Connor, 2004; Mayfield,McLeod, & Hall, 1974). Paternal substance use was defined as illicit drug use or problem alcohol use inthe past year per maternal report. Illicit drug use was defined as any drug use in the past year. Problemalcohol use was defined as ≥6 drinks at one time.

The Revised Conflict Tactics Scale (CTS2) was used to measure partner violence (Straus, 1995; Straus,Hamby, Boney-McCoy, & Sugarman, 1996). Partner violence was defined as any incident of physicalassault by the mother or her partner directed toward the other in the preceding year, per maternal report.

We measured maternal psychological resources at baseline as a potential moderator of program impact.The variable is derived from measures of intelligence (Nelson, 1982), mental health (Berwick et al.,1991), and sense of mastery (Pearlin & Schooler, 1978). These instruments have been shown to haveadequate reliability, with split-half reliability coefficient of .93 for the National Adult Reading Test(NART) (Willshire, Kinsella, & Prior, 1991), Cronbach’s alpha >.85 for the MHI-5 (Berwick et al., 1991;


Stewart, Hays, & Ware, 1988), and reliabilities >.70 for the sense of mastery scale (Pearlin & Schooler,1978). Per the method of Kitzman et al. (1997), we standardized the intelligence, mental health and senseof mastery scores to a mean of 100 and standard deviation of 10, then summed, standardized again to amean of 100 and standard deviation of 10, then dichotomized, with values <100 categorized as indicatingpoor psychological resources.

Program staff used the Kempe Family Stress Checklist (FSC) to assess families for risk (Korfmacher,2000). We used FSC scores as a potential moderator of program impact. We defined high baseline riskas FSC scores ≥45. Scores <45 were defined as moderate risk.

Child outcomes

Injuries requiring medical care, hospitalizations, and ED visits. Injuries requiring medical care wereidentified through maternal interview. Hospitalizations and ED visits were identified through medicalrecord review.

Child development. The Bayley Scales of Infant Development (BSID) were used to assess cognitive andpsychomotor development (Bayley, 1993). The BSID Mental Development Index (MDI) and PsychomotorDevelopment Index (PDI) scores were used as continuous measures and as binary variables dichotomizedas <85 and 85, the recommended cutoff for mild delay (Bayley, 1993).

Child behavior. The Child Behavior Check List (CBCL) for ages 11/2 to 5 years was used to measureproblem behavior per maternal report (Achenbach, Edelbrock, & Howell, 1987). It was analyzed as acontinuous variable using T-scores for internalizing and externalizing behavior and as a binary variableusing recommended cutpoints for normal (<60) versus borderline (60–63) and clinical scores (>63)(Achenbach & Rescorla, 2000). The Nursing Child Assessment Satellite Teaching (NCAST) scale wasused to assess the target child’s response to a caregiver who is teaching him/her a new skill (Barnard,1994). Total child response scores ranged from 0 (lack of focus and response) to 23 (highly focused andresponsive).

Parent outcomes

Parent knowledge, attitudes, and self-ratings. The Knowledge of Infant Development Inventory (KIDI)was used to assess maternal knowledge of child development (Larsen & Juhasz, 1986). Higherscores indicate greater knowledge. We used the Adult-Adolescent Parenting Index (AAPI) to mea-sure mothers’ parenting and child-rearing attitudes (Bavolek, 2001). The total score is derived fromits four subscales and high scores indicate more favorable parenting attitudes. The Infant Caregiv-ing Inventory was used to evaluate parental perceptions and knowledge about the influences of infantcaregiving practices on infant and maternal well-being (Parks & Smeriglio, 2001). A high score indi-cates greater appreciation of the influence of caregiving practices. The Teti Maternal Self-efficacyScale was used to assess maternal self-ratings of competence and effectiveness in the parentingrole (Teti & Gelfand, 2001). Higher scores are more favorable. The Guidubaldi Parent SatisfactionScale evaluates mothers’ satisfaction with their relationship with the child, their parent performance,and their general satisfaction (Guidubaldi & Cleminshaw, 1985). Higher scores indicate greatersatisfaction.


Observational measures of parenting. The infant-toddler version of the Home Observation for Measure-ment of the Environment (HOME) Inventory was used to measure the quality of the home environment(Bradley, 1993; Elardo & Bradley, 1981). We used the total score and each subscale score as continuousvariables: emotional and verbal responsivity, acceptance of child’s behavior, organization of the environ-ment, provision of play materials, parental involvement with child, and opportunities for variety. Higherscores are more favorable. The total score was also used as a binary variable, with scores ≥1 standarddeviations below the mean (≤33) defined as a poor environment, a cutpoint used in previous research(Bradley et al., 1989).

The NCAST was used to observe and measure the quality of the primary caregiver’s behaviorin teaching the child a new skill (Barnard, 1994). The caregiver total score incorporates sensitiv-ity to cues from the child, response to the child’s distress, and fostering of social-emotional andcognitive development. We used the total caregiver score and these subscale scores as continuousmeasures. For the total score, we also used a binary measure. The 10th percentile is recommendedas the lower limit of the normal range (Sumner & Spietz, 1994). We used ≤35, the sample’s 20thpercentile score, to define poor interaction because our sample is comprised of high-risk families.This score is lower than the reported 10th percentile scores for Caucasians (39) and slightly higherthan the 10th percentile scores for African-Americans (34) and Hispanics (33) (Sumner & Spietz,1994).

Disciplinary strategies. We used Straus’ parent-child Conflict Tactics Scale (PC-CTS) to elicit maternalself-report of parenting behaviors in the past year (Straus, 1995). For this report, we used the nonviolentdiscipline subscale as a continuous measure. The PC-CTS also asks mothers how often they used specifictactics in the past week: shouting, yelling or screaming; slapping on the hand, arm or leg; and spankingon bottom with bare hand. We used the combined frequency of these behaviors as a continuous measure.

Maternal recognition of child developmental delay. At follow-up, the mother was asked to rate her child’sdevelopment compared to that of most other children. The response choices were: faster than others,about the same, slower than others, much slower than others, and don’t know. For children assessed asdevelopmentally delayed, we considered the mother to recognize the delay if she responded that her childwas developing slower than other children.

Family linkage with a medical home and other community services. Achievement of a Medical Home wasdefined by access to primary care, adherence to AAP guidelines for well child care visits and immunizationstatus (American Academy of Pediatrics, 2001; American Academy of Pediatrics Ad Hoc Task Force onDefinition of the Medical Home, 1992). By maternal interview, we determined whether the child had aspecific primary care provider at follow-up. Dates of well child visits and receipt of immunizations weredetermined through medical record review.

Linkage to material support (e.g., WIC, food stamps) was measured through maternal interview. Themother was first asked if she had received each service in the past year; those who had not were thenasked whether they had wanted or needed the service. For public health nursing, and for center- andhome-based parenting and child development services, we prepared community-specific lists of sourcesfor all such services. In the follow-up interview, we used a life history chart approach to elicit whetherand when the family had used each service since the child was born.


Actual home visiting services

Family engagement, individualized family support plan development, and visit content. Length of HFAKenrollment, reasons for dropout, visit frequency, and home visit content were drawn from the HFAKmanagement information system (MIS). The Individualized Family Support Plan (IFSP) is a written planbetween the family and the home visitor to assist in setting achievable goals to alleviate family stress andto enhance aspects of parental and family functioning. IFSP completion and review dates were drawnfrom the MIS. As part of regular HFAK operations, home visitors completed a “home visit log” form foreach visit, marking a structured checklist of content areas to indicate those that had been included in thevisit. As part of regular HFAK operations, these data were entered into the MIS.

Child developmental screening. The MIS provided data on the date, result and action taken for each devel-opmental screening by HFAK home visitors. Screening rates were calculated for families still active inHFAK at each screening point. The denominator excluded children with known pervasive developmentaldelay, as there was no need to re-screen children for whom developmental delay was already recognized.

Home visitor response to concerns about child development. HFAK staff periodically screened childrenusing the Ages and Stages Questionnaire (ASQ). For positive screens, we calculated the proportion withevidence of a referral and described the outcome of the referral.

Adequate services. Four binary measures of service adequacy were developed. Two were based only onlength of enrollment: enrollment≥12 months and enrollment≥24 months. The third measure was based ondata in the HFAK MIS and reflected visit frequency and content—receipt of ≥20 visits with “parenting”content. Parenting content was defined by two MIS categories: child development role modeling anddevelopment activities, and education on or modeling of interaction. The fourth service adequacy measurewas the “high dose of service” used in earlier home visiting research—enrollment ≥24 months and receiptof ≥75% of expected visits and ≤3 months on Level X, which is a program level assigned to familieswho are hard to engage (Duggan et al., 2004c).

Implementation system

The implementation system includes staff recruitment and training, policies, protocols, and mech-anisms to integrate HFAK services with other community services. We assessed these through homevisitor surveys, review of training curricula, observation of selected training sessions, review of policyand procedure manuals, and discussion with program leaders. Home visitors completed self-administeredquestionnaires in 2001 and 2003. The first survey included the KIDI to measure home visitor knowledgeof infant development. Steering committee input was used to modify the instrument slightly to assureface validity. Internal consistency of the modified instrument was >.60; this is comparable to internal con-sistency of the original measure (Larsen & Juhasz, 1986). The second survey included the High/ScopeKnowledge Scale which measures knowledge of developmental milestones (Epstein, 2001). This instru-ment elicits the age range at which most children achieve each of 73 milestones. In both surveys, homevisitors rated their competence in developing a trusting relationship with parents and in helping parentsacquire specific knowledge and skills. There were 16 items pertaining to competence in working withmothers, and 16 parallel items pertaining to working with fathers. There were five response choices for


each item, ranging from strongly agree to strongly disagree. In both surveys, the home visitors rated theadequacy of their training in parallel content areas using the same response choices.

Analysis

For most outcomes, analysis was limited to families where the biological mother had not relinquishedher role as caregiver by the 2-year follow-up. Student’s t test and χ2 were used to assess the representa-tiveness of the original sample, the baseline comparability of the study groups, and the representativenessof those followed at 2 years relative to the original sample. We developed multiple logistic regressionmodels with study group by baseline attribute interaction terms to test for differential attrition by studygroup.

HFAK effectiveness

We used intention to treat analysis. Multiple linear and logistic regression models were developed toassess program impact. The models incorporated program site as a cluster variable and covariates forbaseline attributes on which study groups differed significantly.

Impact moderators. For outcomes with a significant overall group effect, we expanded the regressionmodels using the approach of Baron and Kenny (1986) to test whether group effects were limited tospecific subgroups. The hypothesized moderators were: parity (first vs. higher order birth), psychologicalresources (adequate vs. low), FSC score (moderate vs. high), and physical partner violence at baseline(positive vs. negative, excluding women without a partner). We also used this approach to explore whetherthere were significant HFAK effects within sample subgroups for outcomes with a modest overall groupdifference that failed to reach statistical significance. A modest group difference was defined as an adjustedodds ratio (AOR) <.75 or >1.33 for binary outcomes and an effect size (ES) >.20 for continuous outcomes.

Impact mediators. For child outcomes with a significant overall group effect, we expanded the regressionmodels using the approach of Baron and Kenny (1986) to assess whether the impact on the child’soutcomes was mediated by the impact on parenting. Their criteria for mediation are that the independentvariable (HFAK) be related to the outcome (child’s outcome) and that the hypothesized mediator (parent’soutcome) be significantly related to both the independent variable and the outcome. We defined statisticalsignificance as p < .05 and evidence of a trend as p = .05 and <.10. We felt it was important to exploretrends because there are few published experimental studies of program impact on parent outcomes asmediators of program effects. We did not apply corrections to significance levels to account for multipletests. This increases the possibility of a Type I error, but we felt it was justified because it allowed us toreport findings that might be worthy of further research.

HFAK efficacy

Lack of fidelity would weaken program effectiveness, but HFAK might still be efficacious for familieswhere actual services adhered to the model. We tested for efficacy using the service adequacy measuresdefined earlier. We used an “adherers only” approach, limiting the HFAK sample to families with anadequate dose and controlling for baseline variables on which they differed from the control group


(Piantadosi, 1997). We did not apply corrections to significance levels to account for multiple tests butreport only statistically significant results, not trends.

Results

Study families were representative of HFAK families overall. Families with versus those without abaseline interview were comparable on FSC items (all p > .20). Participants’ mean total FSC score (42.9)was similar to that of families assessed positive in Alaska in the 18 months following study recruitment(44.1).

Poor maternal mental health, substance use and partner violence were common among both groups atbaseline because HFAK targets at-risk families. At baseline, HFAK and control group mothers were similaron most characteristics except HFAK mothers were significantly less likely to have poor psychologicalresources, were less likely to have enrolled prenatally and showed a trend toward being less likely tobe depressed (Table 1). Prenatal enrollment and poor psychological resources were positively associated(p < .01). We controlled for psychosocial resources in all analyses.

We obtained a follow-up interview for 85% of the HFAK group and 86% of the control group. In familieswho were followed versus not, mothers were more likely to have worked prior to study enrollment (76%vs. 57%, p = .01), more likely to be married to or living with the child’s father (56% vs. 34%, p < .01), andless likely to have enrolled prenatally (44% vs. 66%, p < .01). The association of parents’ relationshipwith follow-up was stronger for control than HFAK group families (p < .05). We controlled for this in theanalyses of program impact.

Program effectiveness

Impact on child outcomes (Table 2). The groups had similar child health outcomes. About two-thirds ofchildren had sustained no injury requiring medical care and about three-fifths had not been hospitalized,but only a fifth in each group had not used the ED.

HFAK children had better developmental and behavioral outcomes than control children. They weremore likely to score in the normal range on the Bayley MDI (58% vs. 48%, p < .05) and the CBCLinternalizing scale (87% vs. 79%, p < .01) and had more favorable mean scores for these measures.HFAK children were significantly more likely to score in the normal range on the CBCL externalizingscale (82% vs. 77%, p < .01) and showed a trend toward lower mean scores on this scale.

Impact on parenting outcomes (Table 3). There was no difference between groups for most parentingoutcomes. HFAK mothers had significantly higher ratings on the Maternal Self-efficacy Scale (35.1 vs.34.6, p < .05). Fewer HFAK than control families had extremely poor total HOME scores (20% vs. 31%,p < .001). Group scores did not differ significantly on any HOME subscale (not shown in Table 3). Therewas a trend toward greater sensitivity to the child’s cues for HFAK versus control mothers (9.4 vs. 9.2,ES = .19, p = .08).

Impact on family linkage to a medical home and other community resources (Table 4). HFAK familieswere more likely to have health care coverage for the target child (95% vs. 90%, p < .05). They did notdiffer from control families in receipt of well child care and immunizations. The HFAK goal was for 90%


Table 1Baseline characteristics of HFAK and control groups

HFAK (n = 162) Control (n = 163) p

Maternal age in years (SD) 23.4 (5.7) 23.7 (5.7) .62Mother graduated from high school 59% 57% .69Mother worked in year prior to enrollment 75% 71% .34Below poverty level 57% 58% .79Index child is/will be first birth 48% 53% .35Mother speaks language other than English at home 15% 14% .74

Mother’s primary ethnicity .57Alaska Native 23% 20%Caucasian 54% 56%Multiracial 10% 7%Other 13% 17%

Parents’ relationship .23None 21% 20%Friends or going together 29% 24%Living together 26% 37%Married 24% 20%

Partner violencea 45% 52% .21Poor psychological resourcesb 37% 50% .02Depressive symptomsc 52% 61% .09

Heavy alcohol useMotherd 35% 37% .69Fathere 35% 28% .26

Illicit drug useMotherf 48% 48% .87Fathere,g 42% 48% .40

Maternal substance useh 55% 57% .77Enrolled prenatally 41% 53% .03

a Any incident of physical assault by mother or her partner. Excludes mothers without a partner.b Composite measure of poor mental health, poor sense of mastery, and poor intellectual functioning.c Center for Epidemiological Studies Depression Scale score ≥16.d Ever had ≥6 drinks at any one time prior to last year.e In the last year.f In the last 2 years.g Limited to cases where mother could respond to questions about father’s drug use (HFAK, n = 109; control, n = 100).h A CAGE Questionnaire score ≥2 and drank in past year or any illicit drug use in 2 years prior to baseline interview.

of children to have a Medical Home, which implies a specific primary care provider and achievement ofAAP guidelines for well child care. In fact, 74% of HFAK group children had a specific primary careprovider, and only 4% had all recommended AAP well child visits by the strict definition. Even whenusing a less restrictive definition, most HFAK families failed to receive required AAP well child visitsand there was no difference between HFAK and control groups. In the child’s first year of life, 45% ofHFAK children versus 54% of control children had ≥5 well child visits (AOR .72, p = .15). In the secondyear of life, 14% of HFAK children and 13% of control children had ≥3 such visits (AOR 1.04, p = .94).


Table 2Child health, developmental and behavioral outcomes, by study group

HFAK Control AORa 95% CI p

No injuries requiring medical careb 71% 68% 1.10 .48, 2.49 .83No hospitalizationsb 63% 58% 1.20 .58, 2.48 .63No emergency department visitsb 19% 22% .81 .49, 1.34 .42

Bayley Scales MDI ≥ 85c 58% 48% 1.55 1.01, 2.37 <.05Bayley Scales PDI ≥ 85c 85% 80% 1.36 .72, 2.58 .35CBCL internalizing score normald 87% 79% 2.06 1.31, 3.25 <.01CBCL externalizing score normald 82% 77% 1.48 1.13, 1.94 <.01

HFAK Control B 95% CI ESe p

Bayley Scales MDIc 88.0 84.8 3.2 1.2, 5.2 .29 <.05Bayley Scales PDIc 98.1 96.0 2.1 −1.2, 5.4 .19 .16CBCL total internalizing scored 48.2 51.0 −2.8 −4.2, −1.5 .36 <.01CBCL total externalizing scored 50.8 53.0 −2.2 −5.0, .5 .28 .09Child NCAST scoref 18.4 18.5 −.09 −1.1, .9 .02 .83

a Adjusted odds ratio.b Per medical record; sample is families with complete medical record data (HFAK, n = 131; control, n = 137).c Per developmental testing; sample is children of biological mothers with custody of index child at follow-up (HFAK, n = 126;

control, n = 123).d Per maternal report; sample is biological mothers with custody of index child at follow-up (HFAK, n = 126; control, n = 123).e Effect size.f Per observation of mother-child interaction; sample is families where biological mother had custody of index child at follow-up

(HFAK, n = 126; control, n = 123).

The HFAK goal was for 90% of children to be fully immunized at 2 years but only 27% of HFAK childrenwere fully immunized.

HFAK and control groups did not differ in receipt of material support. Most mothers in both groupsaccessed services they needed or wanted. Nearly half the families in each group had received publichealth nursing services (48% HFAK and 50% control, p = .84). HFAK families were more likely thancontrol families to be linked with Infant Learning Services (AOR 1.98, p = .06) and center-based parentingservices (AOR 1.45, p < .05).

Baseline attributes as moderators of HFAK impact. For outcomes with a significant overall group effect,the regression models were expanded to test whether group effects were limited to specific subgroups.Some overall HFAK impacts were limited to families with lower baseline vulnerability. The decrease inpoor HOME scores was limited to families with moderate baseline FSC scores (AOR = .32; 95% CI .17,.62; p < .01) and families who were not violent at baseline (AOR = .55; 95% CI .36, .84; p = .01). Thepromotion of normal externalizing problem behavior was limited to nonviolent families (AOR = 1.80;95% CI 1.08, 3.00; p < .05). Improved access to center-based parenting services was limited to nonviolentfamilies (AOR = 2.47; 95% CI 1.65, 3.69; p < .001).

Of the outcomes that met our definition for a modest, nonsignificant overall group effect, there wasonly one instance of a significant effect in a sample subgroup. Among families with extremely high-riskscores at baseline (FSC score ≥ 45), HFAK families were less likely to access child support services(AOR = .22; 95% CI .11, .45; p < .001).


Table 3Parenting knowledge, attitudes, and behaviors, by study groupa

HFAK Control B 95% CI ES p

Parenting knowledgeKIDI score 73.5 70.7 2.8 −1.9, 7.6 .32 .18

Parenting attitudesAAPI score 130.0 125.6 4.5 −3.2, 12.1 .25 .20Infant caregiving score 112.1 109.5 2.6 −3.1, 8.3 .15 .29

Self-ratings of parentingMaternal self-efficacy score 35.1 34.6 .5 .2, .8 .19 <.05

Parental satisfactionWith child-parent relationship 34.6 34.3 .3 −.9, 1.6 .10 .54With parent performance 29.4 29.2 .2 −.6, 1.1 .07 .52General 30.5 29.9 .6 −.2, 1.3 .18 .11

Home environmentTotal HOME score 36.7 35.9 .8 −.2, 1.9 .18 .10

Parent-child interactionTotal caregiver contingency score 15.4 15.0 .4 −.8, 1.6 .15 .40Sensitivity to cues 9.4 9.2 .2 −.1, .5 .19 .08Response to distress 9.2 8.9 .3 −.2, .8 .20 .14Social-emotional growth fostering 9.0 8.8 .2 −.2, .6 .18 .26Cognitive growth fostering 11.8 11.9 −.1 −.6, .4 .04 .57

Disciplinary strategies: frequency of useNonviolent strategies—past year 50.4 50.5 −.05 −6.1, 6.0 .01 .98Mild physical strategies—past week 4.0 4.6 −.5 −2.7, 1.6 .05 .54

HFAK Control AOR 95% CI p

Poor total HOME scoreb 20% 31% .51 .36, .72 <.001Poor NCAST caregiver interaction scorec 17% 21% .79 .50, 1.25 .31Recognition of child developmental delay 20% 24% .82 .21, 3.17 .77

a Sample is biological mothers with custody of index child at follow-up interview; HFAK, n = 126, control, n = 123.b Score ≤ 33.c Total caregiver score ≤ 35.

Mediators of impact on child outcomes. Four parenting measures were potential mediators of HFAK’simpact on child development and behavior—fewer poor HOME scores, increased maternal self-efficacy,increased use of center-based parenting services, and increased access to health care coverage. PoorHOME scores met the criteria for mediation for two child outcomes: MDI scores and internalizingbehavior. However, it did not significantly attenuate the impact of HFAK on these outcomes when addedto regression models as a covariate. Maternal self-efficacy met the criteria for mediation of HFAK onthe impact on children’s internalizing and externalizing behaviors, but did not significantly attenuate theimpact of HFAK on these outcomes when added to regression models as a covariate. Neither increaseduse of center-based services nor increased access to health care coverage met Baron and Kenny’s criteriafor mediation for any child outcome.


Table 4Linkage to a medical home and other community services, by study group

Service HFAK Control AOR 95% CI p

Medical homea

Has health care coverage 95% 90% 2.05 1.06, 4.00 <.05Has specific primary care provider 74% 78% .76 .44, 1.30 .31Adequate well child visits 4% 8% .96 .12, 1.73 .25Immunizations up to date 27% 27% 1.01 .61, 1.68 .96

Material supportb

WIC 90% 90% 1.15 .46, 2.86 .76Emergency food 83% 76% 1.53 .79, 2.96 .20Food stamps 80% 80% 1.00 .70, 1.43 .99Income assistance (TANF) 76% 76% 1.00 .51, 1.94 .99Child support enforcement 67% 77% .62 .27, 1.43 .26

Parenting servicesa

Public health nursing 48% 50% .93 .48, 1.83 .84Infant learning program 19% 10% 1.98 .97, 4.01 .06Home-based parenting servicesc 18% 25% .61 .25, 1.52 .29Center-based parenting services 48% 39% 1.45 1.05, 2.02 <.05

a Denominator is families where biological mothers had custody and completed the year 2 follow-up interview. HFAK, n = 126;control, n = 123.

b Denominator is families where biological mother had custody and reported at follow-up that she wanted or needed the servicein the preceding year.

c Excluding HFAK, public health nursing, and infant learning program.

Adherence to the HFAK model

Family engagement, IFSP development, and home visit content (Table 5). Actual services often adheredpoorly to the model and varied among HFAK program sites. About half of families were still enrolledwhen the child turned 12 months old and about a third were still enrolled at 24 months. Refusal was themost common reason for dropout. Visit frequency was lower than intended. Among active families, visitsoccurred every 2–3 weeks on average. Only 4% of families had a high dose of service. Of families activeat 1 year, 38% had no IFSPs, and only 2% had all four expected IFSPs. Most visits included contenton child development and parent-child interaction. Only half the families had any visits that includeddiscussion of health services.

Developmental screening and referral (Table 6). Developmental screens were completed for about halfof children whose families were active in HFAK. Between 4% and 21% of children screened positive ateach time point. Overall, 21 children had a total of 27 positive screens. For 11 of the 21 children, therewas evidence that referral was considered. Two families refused referral, three were already receivingfollow-up services, four followed through on referral, and two lacked data on referral status. There were14 children who scored <85 on the Bayley MDI or PDI at 2 years and whose families remained active inHFAK for 2 years. All 14 children had at least one developmental screen, 50% had only negative screens,and 50% had at least one positive screen.


Table 5Actual HFAK services provided per MIS, all sites combined and range among sites

Process measure All sites Range

Family retention (percent of families active)At 6 months 75% 70–84%At 12 months 54% 40–62%At 24 months 32% 14–46%

Reason for dropoutRefusal 35% 19–50%In work/school 8% 0–27%

Number of visits by active families (mean)Year 1 22.1 15.4–26.4Year 2 20.0 11.7–27.3

High dose of servicesa 4% 0–9%

Use of IFSP, families active ≥ 1 yearNo IFSPs in first year 38% 20–60%1 IFSP in first year 34% 25–50%2–3 IFSPs in first year 26% 0–35%All 4 IFSPs in first year 2% 0–12%

Percent of visits with selected contentAny child development activityb 74% 63–84%

Role modeling and development activities 43% 32–49%Education on child development 60% 47–77%

Parent-child interaction 84% 75–96%Education on/modeling of interaction 51% 36–69%Education on child management/discipline 23% 8–40%Support on stresses of parenting 52% 36–72%Discussion of feelings about child 50% 30–84%

Parenting curriculum used 57% 50–68%

Percent of families with selected visit content≥20 visits with parenting content 26% 6–46%≥20 visits with child development content 18% 6–34%

Percent of families with ≥1 visit includingDiscussion of health services/providers 49% 31–71%Discussion of/plan for immunizations 41% 24–54%Support/accompaniment to health services 30% 19–40%

a Enrolled >24 months and received >75% of expected visits and spent <3 months on Level X, which is a nonvisit programlevel for families who are hard to engage.

b Includes those listed plus administration of developmental screens.

Program efficacy

Child outcomes. While the HFAK and control groups did not differ significantly on seven child outcomes(Table 2), HFAK might still be efficacious for families where actual services adhered to the model. Wefound evidence of HFAK efficacy for the PDI for families with ≥20 visits with parenting content. This


Table 6Percent of HFAK children screened and percent of screens positive, by age

Age at screen (months) Active families Children screeneda Screen positiveb

n n % n %

4 127 67 53 8 128 99 50 50 2 4

12 83 45 54 4 916 70 29 41 6 2120 58 30 52 5 1724 49 19 39 2 10

a Families active ≥1 month past screening due date.b Score outside normal range on ≥1 domains.

HFAK subgroup had higher mean scores than controls (B = 5.3; 95% CI .60, 10.1; p < .05) and were morelikely to score in the normal range (AOR 7.6; 95% CI 1.7, 34.7; p < .05).

Parent outcomes. HFAK and control groups did not differ significantly on 27 parenting outcomes(Tables 3 and 4). We found evidence of HFAK efficacy for three of these outcomes. KIDI scores weremore favorable for families enrolled 2 years (B = 5.4; 95% CI 1.4, 9.5; p < .05) and for families with ≥20visits with parenting content (B = 4.4; 95% CI 1.4, 7.5; p < .05). Mothers used mild physical discipline lessoften (B = −1.8; 95% CI −3.3, −.30; p < .05) in families enrolled 2 years compared to controls. HFAKfamilies were more likely than controls to be referred to infant learning programs if they were enrolled≥12 months (AOR 2.8; 95% CI 1.1, 7.4; p < .05) or ≥24 months (AOR 3.1; 95% CI 1.1, 8.4; p < .05).

On the other hand, two outcomes were significantly worse for families meeting our definitions foradequate services. Families with ≥20 visits with parenting content were less likely to have a pediatricprimary care provider (AOR .60; 95% CI .40, .92; p < .05) and less likely to obtain needed child support(AOR .34; 95% CI .12, .98; p < .05).

Implementation system

Home visitors’ actions were consistent with self-ratings of effectiveness (Table 7). Nearly all homevisitors felt effective in establishing a trusting relationship with the mother and in working with her topromote positive parenting and personal growth. Fewer felt effective working in these areas with fathers.

Table 7Percent of items on which home visitors rated themselves as effective and their training as adequate, by content area (n = 24)

Content areas Self-rating as effective in working with Training rated as adequate

Mothers Fathers

Establishing a trusting relationship 94% 81% 91%Promoting positive parenting 92% 79% 89%Promoting personal growth 86% 72% 78%Promoting natural support networks 59% 54% 70%Addressing risks for child maltreatment 59% 49% 58%


Fewer home visitors felt competent in helping parents establish support networks and address risks ofchild maltreatment.

Home visitors had higher KIDI scores than the mothers (home visitor 83.5 + 6.1 (mean + SD) vs. parents74.3 + 9.1, p < .001). Home visitor KIDI scores ranged widely, from 71 to 93. On the High/Scope, homevisitors often failed to identify the correct age range at which developmental milestones are achieved.On average, they gave the correct age category for 49% of milestones; the percent of correct responsesranged from 37% to 62%.

Home visitors’ ratings of training were concordant with its attributes. Training focused on parentgoal setting, parent-child interaction, and child health and development. It encouraged trainees to avoidbecoming crisis oriented (Healthy Families America, 2001). The implementation system enabled andreinforced home visitors’ emphasis on parenting in home visits. All program sites used child developmentcurricula. All sites encouraged home visitors to look for and praise positive parenting during home visits.

In the late 1990s, HFAK shifted the purpose of the IFSP. Originally, it was a case management toolto guide services in relation to the risks that defined family eligibility for HFAK; by 2000, home visitorswere trained to use the IFSP as a tool to help parents develop problem-solving skills. Training emphasizedthat goals should be those of the parent, not the worker (Great Kids Inc., 2000; Healthy Families America,2001).

Discussion

This study assessed the impact of a statewide HFA program in promoting positive parenting, childhealth and child development. HFAK had a positive impact on some parenting outcomes. The pro-gram promoted child development and reduced problem behaviors but did not improve measuresof child health. HFAK’s impact on child and parent outcomes was consistent with actual servicedelivery.

Parenting

HFAK significantly improved three parenting outcomes—maternal self-efficacy, the quality of thehome environment for learning, and the use of center-based parenting services. It showed trends inimproving mothers’ sensitivity to the children’s cues and use of infant learning program services. Groupdifferences for most other parenting outcomes were in the desired direction but were too small to be ofclinical or statistical significance.

Child development

Bayley Mental Development Index (MDI) scores were substantially lower than national norms inboth the HFAK and control groups. Group differences in child cognitive development (ES = .29) weremodest by Cohen’s standard (Cohen, 1988), but were larger than the mean effect size reported in Sweetand Appelbaum’s meta-analysis of home visiting programs (ES = .18) (Sweet & Appelbaum, 2004).HFAK’s impact on the MDI was explained in part by its positive impact in preventing a poor-qualitylearning environment as measured by the HOME. Several other parenting measures were associ-ated with child mental development, but did not differ by study group. It is possible that HFAK’s


impact on the MDI would have been greater if the program had improved these other parentingmeasures.

Child behavior

Per maternal report, HFAK reduced problem behaviors in children. This outcome was partiallymediated by the impact in preventing a poor-quality learning environment and in promoting maternalself-efficacy. Maternal depressive symptoms and partner violence also were associated with children’sproblem behaviors, but were not influenced by HFAK (Duggan et al., 2007). It is possible that HFAK’sprevention of problem behaviors would have been greater yet if it had reduced these risks.

Impact of implementation system on outcomes

HFAK’s impact on child and parent outcomes was consistent with actual service delivery. Home visitorswere more likely to address child development and positive parenting than child health. Their actions tohelp parents cope and to reinforce positive parenting behaviors are consistent with improved maternalparenting efficacy.

HFAK’s model and implementation system explained how services were delivered and their impact onintended outcomes. The program targeted families with multiple malleable risks for child maltreatment.It was a voluntary program, but the targeted families were identified through screening and assessment,not self-identification and request for services. In this light, a high attrition rate is not surprising.

Most home visitors felt they were adequately trained to promote positive parenting through reinforce-ment of positive behaviors. Far fewer felt adequately trained and effective in addressing parenting risks.Content analysis of training materials confirmed an emphasis on promoting positive parenting. The pro-gram’s philosophy, supervision practices, management information system and parenting curricula alldirected home visitors to observe parenting practices and to reinforce positive behaviors.

Impact for family subgroups

Some researchers have questioned whether programs that lack specialized intervention approachescan be successful in working with families with psychosocial risks for maltreatment (Chaffin, Bonner, &Hill, 2001; Eckenrode et al., 2000; Guterman, 1997, 2001). We tested parity and level of family risk asmoderators because some home-based interventions are limited to first-time mothers (Olds, Henderson,Chamberlin, & Tatelbaum, 1986) or to families with mid-level as opposed to severe parenting risk(Bugental et al., 2002). We tested partner violence and maternal psychological resources as moderatorsbecause these attributes were found to moderate the impact of the nurse home visiting model (Eckenrode etal., 2000; Olds & Korfmacher, 1998). For several outcomes, we found that partner violence and high base-line FSC scores attenuated the program’s impact. These outcomes include child externalizing behavior,extremely poor HOME scores, access to center-based parenting services, and sensitivity to the child’s cues.

The attenuating effect of family violence is consistent with findings in Eckenrode et al.’s follow-upof families in the Elmira nurse home visitation study (Eckenrode et al., 2000). Attenuated effects forfamilies at greater baseline risk supports Chaffin’s argument that the HFA model might not be adequatefor high-risk families (Chaffin, 2004).

High rates of attrition from HFAK and lack of fidelity to the program model raises the question ofwhether the program was efficacious for families with an adequate dose of service. We found little evidence


of this when measuring adequacy by duration of enrollment or frequency of visits with parenting content.In over 100 tests for group differences, 7 showed a significant positive effect, 2 showed a significantnegative effect, and the remainder showed no significant group difference.

Comparison with randomized trial of Hawaii’s Healthy Start Program

How do results in this study compare with the results of our randomized trial of Hawaii’s Healthy StartProgram (HSP), the inspiration for HFA? Overall, HSP and HFAK had similar family retention rates, visitfrequency rates and parent ratings of home visitors; within each state, program sites varied considerablyon process measures (Duggan et al., 2000). In both states, developmental screening often missed childrenwho were delayed, and IFSPs were used less often than called for in the program model (Duggan et al.,2000; King et al., 2005).

For the most part, program impact on child health care and outcomes was similar in Hawaii and Alaska(Duggan et al., 1999). In both states, home visiting promoted access to health care coverage. In Hawaii,but not Alaska, home visited families were more likely to be linked to a specific primary care provider.Preventive health care coverage and primary care availability were the same in the two states, and soare unlikely to be the reason for this difference. Home visiting did not increase receipt of preventivehealth care or decrease emergency department use and hospitalizations in either state. These outcomesare consistent with service delivery. In Hawaii, home visitors made certain that families had a specificprimary care provider but did not emphasize preventive care beyond this. In Alaska, home visitors failedto discuss health care at all for half the families.

The Hawaii and Alaska programs differed in their impact on child development and behavior.For example, in Hawaii at 2 years, the HSP group did not differ from controls on the MDI (102.3vs. 102.6, p = .92) (Duggan et al., 1999). Additionally, the HSP study found no impact on thehome environment or parent-child interaction when children were 2 years old (Duggan et al., 1999).In contrast, HFAK improved child cognitive development and behavior as well as some parentingoutcomes.

What explains these differences in child and parent outcomes? A range of early childhood serviceswas available in both Hawaii and Alaska. Thus, the differences in outcomes are more likely explainedby differences in HSP and HFAK home visiting than by differences in the availability of other earlychildhood services. The Hawaii study was conducted during a transition period arising from the program’sdesignation as an early intervention (EI) program for children at risk for special health care needs. HSPleaders and staff were in the midst of reconciling two goals—the original goal of preventing child abuseby addressing the risks that had made families eligible for service and the more recent goal of promotingchild development using EI’s family-centered strengths-based approach (Duggan et al., 2004c). Fiveyears later, Alaska had fully implemented the HFAK model using a strengths-based approach to promotechild development and positive parenting. HFAK made greater use of child development curricula andreinforced positive parenting more consistently than did HSP; neither HFAK nor HSP focused stronglyon preventing child abuse by addressing parent risks.

Methodological considerations

The study confirmed the feasibility of carrying out comprehensive experimental research in the contextof a scaled up program (Chen, 1998). It used a broad range of measures and data sources. Sample size


assured adequate power to detect clinically meaningful group differences. Data were collected by researchstaff unaware of family group assignment.

The strength of our findings is enhanced by their consistency on several levels. Study findings wereconsistent across measures and data sources. Impact on outcomes was consistent with the services actuallyprovided; services provided were consistent with the program model and implementation system. Theseconsistencies are concordant with results of other HSP/HFA randomized trials (Bugental et al., 2002;Duggan et al., 2004a, 2004c; Landsverk et al., 2002; Windham et al., 2004).

Study participants were representative of families who usually enroll in HFAK and other home visitingprograms that target families using the FSC. HFAK participation rates for study families were similar toHFAK participation rates before and after the study. Families could accept the program while refusingthe study; few chose this option. There was a high follow-up rate for both study groups.

A limitation of this study is that we were hampered in assessing efficacy because few families metour definition of adequacy incorporating both duration of enrollment and visit frequency. We tested forefficacy with great caution because available techniques are subject to methodological error (Piantadosi,1997). Service adequacy is an outcome, not a baseline attribute, and so comparison groups are likely todiffer on unmeasured variables that could influence service adequacy but could not be controlled for inanalysis. Results must be interpreted carefully.

A second limitation is that families knew the overall study purpose and their group assignment whichmight have biased outcome measures based on self-report, including maternal ratings of child behaviorand self-efficacy. The prevalence of internalizing problem behaviors in the HFAK group was lower thannational norms. Further research is needed to determine whether the program had substantial success inreducing problem behaviors or if reporting bias increased apparent program effects.

Lastly, several parent and child outcomes were measured at the same time, preventing us from estab-lishing temporal sequence. We cannot discern whether maternal self-efficacy is a factor or consequenceof reduced problem behaviors in the child. Although we measured a broad range of parenting outcomes,these explained little of the program’s impact on child outcomes. It is possible that some unmeasuredaspects of parent behavior, attitudes or knowledge impacted by the program accounted for the groupdifferences in child outcomes.

Implications

HFAK policies and procedures are concordant with HFA Critical Elements, training and credentialing.Thus, study findings have implications for HFA and similar programs nationally. HFAK’s positive impacton child development and behavior and its reduction of a poor home environment for learning confirms thepotential effectiveness of a scaled up model of paraprofessional home visiting targeting at-risk families.The findings support a strengths-based approach, consistent use of parenting curricula, and reinforcementof positive parenting.

Three other key findings have implications for program and policy development to increase programeffectiveness. First, impact was attenuated in families at greatest baseline risk. This suggests that theprogram might not be appropriate for families with more severe levels of risks or specific types of risk.Second, actual services departed from the model and varied widely among HFAK sites. This suggeststhe need to strengthen both network-wide and program-specific aspects of the implementation systemsuch as staff training and monitoring. Third, program efficacy tended to be no greater than effectivenesswhen measuring dose with broad indicators such as length of enrollment and frequency of parenting


content in visits. This suggests the need for precise specification of a model’s theory of change andboth quantitative and qualitative measures of dose that are directly tied to each of aspect of the model’stheory.

The HSP/HFA model and implementation system are relatively weak in addressing the ineffectiveor negative parenting behaviors observed by home visitors. In HFAK and similar programs, home vis-itors are advised to build trust and wait for the parent to raise an issue rather than do so themselves.Home visitors were discouraged from focusing on problematic parenting behaviors unless asked bythe parent. It is likely that this contributed to our finding of minimal program impact on parent-childinteraction.

Need for continuing research and development

Continued follow-up of the study cohort is needed to determine longer-term benefits. It is possi-ble that early improvements in child development and behavior will be sustained and will ultimatelydecrease parenting stress and improve family functioning. On the other hand, HFAK’s short-termlack of impact on parent risks for maltreatment might have longer-term adverse consequences forchildren. Continued follow-up with multiple reporters of children’s behavior is needed to rule outmaternal reporting bias. Experimental study in other settings is needed to test the replicability of ourfindings.

There is a great need for experimental testing of theory-based modifications to this and other homevisiting models. As Bugental et al. have noted, prevention programs focused on families at-risk of childmaltreatment are largely empirically based (Bugental et al., 2002). Theory has taken a secondary role inprogram development. Theory-based program development is especially important for the HSP/HFAKmodel because it is uniquely ambitious in its target population and goals. It targets families with mul-tiple, malleable risks for child maltreatment. One of its challenges, therefore, is to engage families ina prevention/early intervention agenda while embracing a family-centered, strengths-based philosophy.Another challenge is to establish service priorities and protocols on the theory-based causal pathwaysfrom services to these goals.

Because the model uses paraprofessionals, its implementation system shoulders the burden of assuringthat staff acquire, maintain and use core skills. Training programs that fail to prepare home visitors needto be retooled. Where individual home visitors fail to achieve or maintain skills, they need to be retrained.Supervisors need better tools for monitoring home visitor competence, home visit content and servicequality. In short, evaluation of staff skills and service quality need to be integral to training and programoperations.

Conclusion

This experimental study found that a home visiting program targeted to families at-risk of childmaltreatment improved some aspects of parenting, child development and child behavior, but not childhealth. Outcomes were consistent with the program model and implementation system. The program wasless effective in families at greatest risk, suggesting that it might not be appropriate for such families.Unintended variations in service content and quality underscore the need for stronger implementationsystems when taking a model to scale.


Acknowledgments

The authors thank the Healthy Families Alaska program leadership and staff and the study SteeringCommittee members (listed below) for their contributions to study methods, execution, and interpre-tation of findings. The collaborating groups were highly motivated to conduct the study faithfully andto share lessons learned. Their contributions assured the relevance, timeliness and validity of studyfindings.

DHSS staff members on the Steering Committee included the following, in alphabetical order: JaneAtuk; Stephanie Birch; Chera Boom; Linda Borghols; Kathryn Cohen; Penny Cordes; Nancy Cornwell;Sandra Csaszar; Diane DeMay; Brad Gessner, MD; Joanne Gibbens; Jill Holdren; Diane Ingle; Mar-cia Kennai; John Levering; Karen Martinek; Phillip Mitchell; Pam Muth; Jonathan Nelson; Jeri Powers;Janine Schoellhorn; Claudia Shanley; and Dale Williams. Steering Committee members from communityorganizations included: Nancy Burke (Alaska Mental Health Trust Authority); Glenda Felts (Cook InletTribal Council); Peter Holck (Alaska Native Health Board); Susan LaBelle (Alaska Mental Health TrustAuthority); Sally Mead (Prevention Associates); Shirley Pittz (RurAL CAP); and Karen Ward (Univer-sity of Alaska). Steering Committee members from the HFAK network of service providers include thefollowing: Beth Corven (Healthy Families Fairbanks); JoAnn Hagen (Kenai Family Support Program);Stephanie Hill (Cook Inlet Tribal Council/New Beginnings Program); Wes Hill (New Beginnings Pro-gram); Delores Martinez (Cook Inlet Tribal Council/New Beginnings Program); Viann Nations (CookInlet Tribal Council/New Beginnings Program); Sue Olson (Kenai Public Health Center); Edy Rodewald(Healthy Families Juneau); Donna Shock (Healthy Families Fairbanks); Jackie Snyder (Healthy FamiliesAnchorage); and Kristen Vernola (Healthy Families Mat-Su).

References

Achenbach, T. M., Edelbrock, C., & Howell, C. T. (1987). Empirically based assessment of the behavioral/emotional problemsof 2- and 3-year-old children. Journal of Abnormal Child Psychology, 15, 629–650.

Achenbach, T. M., & Rescorla, L. A. (2000). Manual for the ASEBA preschool forms & profiles. Burlington, VT: University ofVermont, Research Center for Children, Youth, & Families.

American Academy of Pediatrics. (2001). Medical Home Initiatives. AAP Web site.American Academy of Pediatrics Ad Hoc Task Force on Definition of the Medical Home. (1992). The medical home. Pediatrics,

90, 774.American Academy of Pediatrics Council on Child and Adolescent Health. (1998). The role of home-visitation programs in

improving health outcomes for children and families. Pediatrics, 101, 486–489.Barnard, K. (1994). The Nursing Child Assessment Satellite Training (NCAST) Teaching Scale. Seattle, WA: University of

Washington, School of Nursing.Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual,

strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.Bavolek, S. J. (2001). Adult-Adolescent Parenting Inventory (AAPI). In J. Touliatos, B. F. Perlmutter, & M. A. Straus (Eds.),

Handbook of family measurement techniques (pp. 156–157). Thousand Oaks, CA: Sage Publications, Inc.Bayley, N. (1993). Bayley Scales of Infant Development (2nd ed.). San Antonio, TX: The Psychological Corporation, Harcourt

Brace & Company.Berwick, D. M., Murphy, J. M., Goldman, P. A., Ware, J. E., Jr., Barsky, A. J., & Weinstein, M. C. (1991). Performance of a

five-item mental health screening test. Medical Care, 29, 169–176.Bradley, R. H. (1993). Children’s home environments, health, behavior, and intervention efforts: A review using the HOME

Inventory as a marker measure. Genetic, Social, and General Psychology Monographs, 119, 437–490.


Bradley, R. H., Caldwell, B. M., Rock, S. L., Ramey, C. T., Barnard, K. E., Gray, A., Hammond, M., Gottfried, A., Siegel, L.,& Johnson, D. L. (1989). Home environment and cognitive development in the first three years of life: A collaborative studyinvolving six sites and three ethnic groups in North America. Developmental Psychology, 25, 217–235.

Bugental, D. B., Ellerson, P. C., Lin, E. K., Rainey, B., Kokotovic, A., & O’Hara, N. (2002). A cognitive approach to child abuseprevention. Journal of Family Psychology, 16, 243–258.

Centers for Disease Control and Prevention. (2003). First reports evaluating the effectiveness of strategies for preventing violence:Early child home visitation and firearm laws. Findings from the Task Force on Community Preventive Services. Morbidityand Mortality Weekly Report, 52, No. RR-14, pp. 1–20.

Chaffin, M. (2004). Invited commentary: Is it time to rethink Healthy Start/Healthy Families? Child Abuse & Neglect, 28,589–595.

Chaffin, M., Bonner, B. L., & Hill, R. F. (2001). Family preservation and family support programs: Child maltreatment outcomesacross client risk levels and program types. Child Abuse & Neglect, 25, 1269–1289.

Chen, H. (1998). Theory-driven evaluations. In A. J. Reynolds & H. J. Walberg (Eds.), Evaluation research for educationalproductivity (1st ed., pp. 15–34). Greenwich, CT: Jai Press Inc.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Duggan, A. K., Caldera, D., Rodriguez, K., Burrell, L., Rohde, C., & Crowne, S. S. (2007). Impact of a statewide home visiting

program to prevent child abuse. Child Abuse & Neglect, 31, 801–827.Duggan, A. K., Fuddy, L., Burrell, L., Higman, S. M., McFarlane, E., Windham, A., & Sia, C. (2004). Randomized trial of a

statewide home visiting program to prevent child abuse: Impact in reducing parental risk factors. Child Abuse & Neglect, 28,623–643.

Duggan, A. K., Fuddy, L., McFarlane, E., Burrell, L., Windham, A., Higman, S. M., & Sia, C. (2004). Evaluating a statewidehome visiting program to prevent child abuse in at-risk families of newborns: Fathers’ participation and outcomes. ChildMaltreatment, 9, 3–17.

Duggan, A. K., McFarlane, E., Fuddy, L., Burrell, L., Higman, S. M., Windham, A., & Sia, C. (2004). Randomized trial of astatewide home visiting program to prevent child abuse: Impact in preventing child abuse and neglect. Child Abuse & Neglect,28, 597–622.

Duggan, A. K., McFarlane, E. C., Windham, A. M., Rohde, C. A., Salkever, D. S., Fuddy, L., Rosenberg, L. A., Buch-binder, S. B., & Sia, C. C. J. (1999). Evaluation of Hawaii’s Healthy Start Program. The Future of Children, 9, 66–90.

Duggan, A., Windham, A., McFarlane, E., Fuddy, L., Rohde, C., Buchbinder, S., & Sia, C. (2000). Hawaii’s Healthy StartProgram of Home Visiting for At-Risk Families: Evaluation of family identification, family engagement and service delivery.Pediatrics, 105, 250–259.

Eckenrode, J. J., Ganzel, B., Henderson, C. R., Jr., Smith, E., Olds, D. L., Powers, J., Cole, R. E., Kitzman, H. J., & Sidora,K. (2000). Preventing child abuse and neglect with a program of nurse home visitation: The limiting effects of domesticviolence. Journal of the American Medical Association, 284, 1385–1391.

Elardo, R., & Bradley, R. H. (1981). The Home Observation for Measurement of the Environment (HOME) Scale: A review ofthe research. Developmental Review, 1, 113–145.

El-Kamary, S. S., Higman, S. M., Fuddy, L., McFarlane, E., Sia, C., & Duggan, A. K. (2004). Hawaii’s Healthy Start homevisiting program: Determinants and impact of rapid repeat birth. Pediatrics, 114, e317–e326.

Epstein, A. S. (2001). The High/Scope Knowledge Scale. In J. Touliatos, B. F. Perlmutter, & M. A. Straus (Eds.), Handbook offamily measurement techniques (p. 169). Thousand Oaks, CA: Sage Publications, Inc.

Fiellin, D. A., Reid, M. C., & O’Connor, P. G. (2004). Screening for alcohol problems in primary care: A systematic review.Archives of Internal Medicine, 160, 1977–1989.

Frankel, S., Friedman, L., Johnson, A., Thies-Huber, A., & Zuiderveen, S. (2000). Healthy Families America site developmentguide. Chicago, IL: Prevent Child Abuse America.

Gomby, D. S., Culross, P. L., & Behrman, R. E. (1999). Home visiting: Recent program evaluations—analysis and recommen-dations. The Future of Children, 9, 4–26.

Great Kids Inc. (2000). Family support worker training materials. San Angelo, TX: Great Kids Inc..Guidubaldi, J., & Cleminshaw, H. K. (1985). The development of the Cleminshaw-Guidubaldi Parent Satisfaction Scale. Journal

of Clinical Child Psychology, 14, 293–298.Guterman, N. B. (1997). Early prevention of physical child abuse and neglect: Existing evidence and future directions. Child

Maltreatment, 2, 12–34.


Guterman, N. B. (2001). Stopping child maltreatment before it starts: Emerging horizons in early home visitation services.Thousand Oaks, CA: Sage Publications.

Healthy Families America. (2001). Healthy Families America training participant’s manual—Program manager, supervisor,family support worker and assessment worker. Chicago, IL: Prevent Child Abuse America.

Hebbeler, K. M., & Gerlach-Downie, S. (2002). Inside the black box of home visiting: A qualitative analysis of why intendedoutcomes were not achieved. Early Childhood Research Quarterly, 17, 28–51.

Husaini, B. A., Neff, J. A., Harrington, J. B., Hughes, M. D., & Stone, R. H. (1980). Depression in rural communities: Validatingthe CES-D Scale. Journal of Community Psychology, 8, 20–27.

King, T. M., Rosenberg, L. A., Fuddy, L., McFarlane, E., Sia, C., & Duggan, A. K. (2005). Prevalence and early identifi-cation of language delays among at-risk three year olds. Journal of Developmental and Behavioral Pediatrics, 26, 293–303.

Kitzman, H., Olds, D. L., Henderson, C. R., Jr., Hanks, C., Cole, R., Tatelbaum, R., McConnochie, K. M., Sidora, K., Luckey,D. W., Shaver, D., Engelhardt, K., James, D., & Barnard, K. (1997). Effect of prenatal and infancy home visitation by nurseson pregnancy outcomes, childhood injuries, and repeated childbearing. Journal of the American Medical Association, 278,644–652.

Korfmacher, J. (2000). The Kempe Family Stress Inventory: A review. Child Abuse & Neglect, 24, 129–140.Landsverk, J., Carrilio, T., Connelly, C. D., Ganger, W. C., Slymen, D. J., Newton, R. R., Leslie, L., & Jones, C. (2002). Healthy

Families San Diego clinical trial: Technical report. San Diego, CA: Child and Adolescent Services Research Center. SanDiego Children’s Hospital and Health Center.

Larsen, J. J., & Juhasz, A. M. (1986). The Knowledge of Child Development Inventory. Adolescence, 21, 39–54.Mayfield, D., McLeod, G., & Hall, P. (1974). The CAGE Questionnaire: Validation of a new alcoholism screening instrument.

American Journal of Psychiatry, 131, 1121–1123.McHorney, C., & Ware, J. (1995). Construction and validation of an alternate form general mental health scale for the Medical

Outcomes Study Short-Form 36-Item Health Survey. Medical Care, 33, 15–28.Nelson, H. E. (1982). National Adult Reading Test (NART): Test manual. Windsor, UK: NFER-NELSON.Nelson, C. S., Higman, S. M., Sia, C., McFarlane, E., Fuddy, L., & Duggan, A. K. (2005). Medical homes for at-risk

children: Parental reports of clinician-parent relationships, anticipatory guidance, and behavior changes. Pediatrics, 115,48–56.

Olds, D. L., Henderson, C. R., Chamberlin, R., & Tatelbaum, R. (1986). Preventing child abuse and neglect: A randomized trialof nurse home visitation. Pediatrics, 78, 65–78.

Olds, D. L., & Korfmacher, J. (1998). Maternal psychological characteristics as influences on home visitation contact. Journalof Community Psychology, 26, 23–36.

Parks, P. L., & Smeriglio, V. L. (2001). Infant Caregiving Inventory (ICI)—Revised. In J. Touliatos, B. F. Perlmutter, &M. A. Straus (Eds.), Handbook of family measurement techniques (p. 186). Thousand Oaks, CA: Sage Publications,Inc.

Pearlin, L. I., & Schooler, C. (1978). The structure of coping. Journal of Health and Social Behavior, 19, 2–21.Piantadosi, S. (1997). Clinical trials: A methodologic perspective. New York: Wiley-Interscience.Radloff, L. S. (1977). The CES-D Scale: A self-report depression scale for research in the general population. Applied Psycho-

logical Measurement, 1, 385–401.Stewart, A. L., Hays, R. D., & Ware, J. E., Jr. (1988). The MOS short-form general health survey: Reliability and validity in a

patient population. Medical Care, 26, 724–735.Straus, M. A. (1995). Manual for the Conflict Tactics Scales (CTS) and test forms for the Revised Conflict Tactics Scale (CTS2).

Durham, NH: University of New Hampshire Family Research Laboratory.Straus, M. A., Hamby, S. L., Boney-McCoy, S., & Sugarman, D. B. (1996). The Revised Conflict Tactics Scales (CTS2):

Development and preliminary psychometric data. Journal of Family Issues, 17, 283–316.Sumner, G., & Spietz, A. (1994). What the teaching scale measures. In G. Sumner & A. Spietz (Eds.), NCAST caregiver/parent-

child interaction teaching manual (pp. 102–132). Seattle, WA: NCAST Publications, University of Washington, School ofNursing.

Sweet, M. A., & Appelbaum, M. I. (2004). Is home visiting an effective strategy? A meta-analytic review of home visitingprograms for families with young children. Child Development, 75, 1435–1456.

Teti, D. M., & Gelfand, D. M. (2001). Maternal Self-efficacy Scale. In J. Touliatos, B. F. Perlmutter, & M. A. Straus (Eds.),Handbook of family measurement techniques (pp. 196–197). Thousand Oaks, CA: Sage Publications, Inc.


Wagner, M. M., & Clayton, S. L. (1999). The Parents as Teachers program: Results from two demonstrations. The Future ofChildren, 9, 91–115.

Willshire, D., Kinsella, G., & Prior, M. (1991). Estimating WAIS-R IQ from the National Adult Reading Test: A cross-validation.Journal of Clinical and Experimental Neuropsychology, 13, 204–216.

Windham, A., Rosenberg, L. A., Fuddy, L., McFarlane, E., Sia, C., & Duggan, A. K. (2004). Risk of mother-reported childphysical and psychological abuse in the first 3 years of life. Child Abuse & Neglect, 28, 645–667.

impact of a statewide home visiting program on parenting and on child health and development

Documents