A Telehealth-Enhanced ReferralProcess in Pediatric Primary Care: ACluster Randomized TrialTumaini R. Coker, MD, MBA,a Lorena Porras-Javier, MPH,b,c Lily Zhang, MS,d Neelkamal Soares, MD,e Christine Park, MD, MPH,f

Alpa Patel, MD,g Lingqi Tang, MS, PhD,d Paul J. Chung, MD, MS,b,c,h Bonnie T. Zima, MD, MPHd,i

abstractOBJECTIVES: To improve the mental health (MH) referral process for children referred fromprimary care to community mental health clinics (CMHCs) by using a community-partneredapproach.

METHODS: Our partners were a multisite federally qualified health center and 2 CMHCs in LosAngeles County. We randomly assigned 6 federally qualified health center clinics to theintervention or as a control and implemented a newly developed telehealth-enhanced referralprocess (video orientation to the CMHC and a live videoconference CMHC screening visit) forall MH referrals from the intervention clinics. Our primary outcome was CMHC access definedby completion of the initial access point for referral (CMHC screening visit). We usedmultivariate logistic and linear regression to examine intervention impact on our primaryoutcome. To accommodate the cluster design, we used mixed-effect regression models.

RESULTS: A total of 342 children ages 5 to 12 were enrolled; 86.5% were Latino, 61.7% wereboys, and the mean age at enrollment was 8.6 years. Children using the telehealth-enabledreferral process had 3 times the odds of completing the initial CMHC screening visit comparedwith children who were referred by using usual care procedures (80.49% vs 64.04%; adjustedodds ratio 3.02 [95% confidence interval 1.47 to 6.22]). Among children who completed theCMHC screening visit, intervention participants took 6.6 days longer to achieve it but alsoreported greater satisfaction with the referral system compared with controls. Once this initialaccess point in referral was completed, .80% of eligible intervention and control participants(174 of 213) went on to an MH visit.

CONCLUSIONS: A novel telehealth-enhanced referral process developed by using a community-partnered approach improved initial access to CMHCs for children referred from primary care.

WHAT’S KNOWN ON THIS SUBJECT: Children who areMedicaid insured and require specialty mental healthcare are often referred to community mental healthclinics; however, nearly 80% of children who needmental health services do not receive them.

WHAT THIS STUDY ADDS: A novel telehealth-enhancedreferral process developed by using a community-partnered approach to the intervention designimproved initial access to community mental healthclinics for children referred from primary care.

To cite: Coker TR, Porras-Javier L, Zhang L, et al. ATelehealth-Enhanced Referral Process in PediatricPrimary Care: A Cluster Randomized Trial. Pediatrics.2019;143(3):e20182738

aDepartment of Pediatrics, University of Washington School of Medicine, and Seattle Children’s Research Institute,Seattle, Washington; bUniversity of California, Los Angeles Mattel Children’s Hospital, Los Angeles, California;iDepartments of Psychiatry and Biobehavioral Sciences and cPediatrics, David Geffen School of Medicine at UCLA,Los Angeles, California; dCenter for Health Services and Society, Semel Institute for Neuroscience and HumanBehavior, University of California, Los Angeles, Los Angeles, California; eHomer Stryker M.D. School of Medicine,Western Michigan University, Kalamazoo, Michigan; fNortheast Valley Health Corporation, San Fernando,California; gChild and Family Guidance Center, Northridge, California; and hKaiser Permanente School of Medicine,Pasadena, California

Dr Coker conceptualized and designed the study, coordinated and supervised data collection,analysis, and interpretation, drafted the initial manuscript, and reviewed and revised themanuscript; Drs Park and Patel conceptualized and designed the study and reviewed and revisedthe manuscript; Drs Zima and Soares and Ms Porras-Javier made substantial contributions toconception and design and acquisition and interpretation of data and reviewed and revised themanuscript; Drs Zhang and Chung and Ms Tang made substantial contributions to data analysis andinterpretation and reviewed and revised the manuscript; (Continued)

PEDIATRICS Volume 143, number 3, March 2019:e20182738 ARTICLE by guest on October 2, 2020www.aappublications.org/newsDownloaded from

An estimated 15% to 20% of USchildren suffer from a mental health(MH) disorder, but nearly 80% ofthose who need MH services donot receive them.1–4 MH needs thatgo unaddressed adversely impactchild health and well-being, familyfunctioning, and eventual adulthealth and productivity.5,6 AlthoughMH disorders affect a significantproportion of the pediatricpopulation, African American andLatino children living in poverty areoften affected at higher rates and areconsistently less likely to receivespecialty MH services.2,7–13

Explanations for poor access tospecialty MH services are multifactorial,particularly for children in low-income and minority populations.Parents may be unaware of insurancecoverage and benefits for MHservices and how or where to findappropriate clinicians to providethese services.14,15 They may notrecognize their child’s behavioralproblems as a concern for which toseek medical care, and when they do,they may face barriers related to thestigma of MH disorders and specialtycare clinics and clinicians.7–9,15–19

Children who are publicly insuredface additional access barriersbecause of the requirement ofspecialty MH care referrals tocommunity mental health clinics(CMHCs) for diagnostic andtherapeutic MH services. TheseCMHCs may be more difficult toaccess because of various factors,including unfamiliarity with theclinic’s screening and enrollmentprocess,20 stigma of attendinga CMHC,21–23 and clinic location.24

Providing MH services using primarycare or specialty clinicians infragmented systems of care oftenresults in suboptimal care forchildren receiving care in eithersetting.25 Collaborative care modelslinking primary care with specialtyMH care can improve use of MHservices and outcomes for childrenand adolescents.26–33

There are, however, multiple barriersto implementing these types ofmodels for collaborative care,colocation, and integrated care toimprove MH access for families.Telehealth34 provides a promisingsolution that allows primary careproviders (PCPs) and specialty careproviders to engage in systems forcare coordination, communication,and collaboration, particularly whentrue, in-person integration of separatesystems is not possible.35–46

Under the current referral structureand process, parents must navigatea complex multistep referral and careprocess once the referral to a CMHChas been initiated by a PCP. Toimprove access to specialty MHcare, we partnered with a multisitefederally qualified health center(FQHC) and 2 CMHCs to design andtest an innovative telehealth-basedstructure and process for the MHreferral process, with a goal ofenhancing access to subspecialtyMH care. Using the Donabedianmodel47,48 to guide interventiondevelopment, we examined whetherand how the structural and processelements of the current referralsystem of primary care to CMHCcould be improved to increase thelikelihood of a completed referral,leading to improved access tospecialty MH care services forchildren in low-income communities.

Our study objectives were to developand test an intervention to improveinitial access to CMHCs. Our primaryoutcome was the completion of theinitial access to the CMHC, namely, aneligibility screening visit. Secondaryoutcomes were parent satisfactionwith the referral process and with careoverall, family-centeredness of care,and child health-related quality of life.

METHODS

Study Setting

Our partners were a multisite FQHC(with 6 clinics) and 2 CMHCs that

serve a large population of childrenwho are publicly insured. The 2CMHCs are contracted by the LosAngeles County Department of MentalHealth (DMH) to provide MH servicesto publicly insured children nearthe geographical areas served by6 clinical sites of the FQHC.

Intervention Development

We used a community-partneredapproach for interventiondevelopment that has been used inprevious studies to partner withclinic stakeholders in clinical deliverydesign projects aimed at improvingcare for children who are publiclyinsured.34 We systematically engagedthe major stakeholders in a processthat developed a new referral systemto enhance FQHC patients’ accessto and successful enrollment inthese CMHCs. The project workinggroup (PWG) was made up of 26individuals (14 FQHC clinic providersand/or staff, 8 MH care providersand/or staff, and 4 parents) whoreviewed qualitative data fromkey stakeholders (interviews of 7parents and 13 providers and/orstaff), identified key transition pointsin which access to and coordinationof care were likely compromised,and developed solutions.20,49

The PWG outlined a workflowto support the newly developedreferral system (called Telehealth-Coordinated Referral). The researchteam worked with the PWG toimplement, refine, and pilot thenew referral structure and process(henceforth, “referral system”) among19 families. Additional adjustmentswere made to the intervention onthe basis of the pilot data.

Usual Care Referral Process

Parents receive an MH care referralfrom their primary care clinician atthe FQHC. The referral is faxed fromthe primary care clinic to the CMHC.The screening department at theCMHC initiates the first contact withthe family for a phone eligibility

2 COKER et al by guest on October 2, 2020www.aappublications.org/newsDownloaded from

screening, which can occur on anyweekday. A case manager from theCMHC screening department asksthe parent a series of questionsregarding insurance coverage,their child’s MH concerns, and otherissues to determine eligibility. Thisinformation is provided to a CMHCtherapist, and the patient is thenscheduled for a 2-hour in-personintake visit with a CMHC therapist.

Intervention Referral Process

The intervention process wasdesigned to enhance patient access tothe CHMC eligibility screening visit,the first step in the initiation ofspecialty MH care at the CMHC.

The intervention process is asfollows: Parents receive a CMHCreferral from their primary careclinician at the FQHC and watch a 5-minute video introduction to thereferred CMHC or receive a textmessage link to watch the video ata later time. The parents schedulea return visit to the clinic fora telehealth eligibility screening visitwith the FQHC’s telehealth carecoordinator. These visits are availableto be scheduled on 1 selected day perweek at each clinical site. Uponreturn, the parents meet with theFQHC telehealth care coordinator,who connects via videoconference tothe screening department at theCMHC. A case manager from theCMHC screening departmentconducts the eligibility screeningprocess via a live videoconferencevisit with the parents and FQHCtelehealth coordinator (located atthe FQHC site). The parents answera series of questions regardinginsurance coverage, their child’sMH concerns, and other issues.The CMHC case manager makesan initial determination of eligibilityfor the family and provides thisinformation to a CMHC therapist,and the parents are then scheduledfor a 2-hour in-person intake visitwith the therapist.

Study Design and Procedures

The 6 clinics were randomly assignedby the study statistician in blocksby their location and size (3 inthe intervention and 3 controls)using computer-generated randomallocation. The 3 clinics randomlyassigned to the interventionimplemented the new referralprocess for all MH referrals, andthe 3 clinics randomly assigned tothe control used the usual referralprocess. The study was approvedby the University of California, LosAngeles Institutional Review Board.

Adult parents or legal guardians ofa child age 5 to 12 years at the FQHCwho received a referral to 1 of the 2participating CMHCs in the past30 days were invited to enroll in thestudy from April 2015 to December2016. A trained bilingual and/orbicultural (English and Spanish)research associate (RA) called theparents within 30 days of the referralto invite them to enroll, and if theparents agreed, the RA consented theparent and collected baseline data,including demographics, at that time.Parents were asked to participate ina 6-month postenrollment phonesurvey and received a cash incentivefor survey completion; data collectionoccurred through the end of the 6-month follow-up period (June 2017).Because of the cluster-randomizeddesign, neither the participants northe RAs were blinded to groupassignment.

Sample

Of 542 parent-child dyads receivinga referral during the time of studyenrollment, 483 were assessed forstudy eligibility, and 342 of thesewere enrolled (Fig 1).

Study Variables

Our primary outcome was completionof the CMHC eligibility screening visit.Data from CMHC visit logs (anelectronic record of all visits) wereused to determine completion of theinitial screening visit within 6 months

of referral. The CMHC screeningvisit logs are part of DMH-requireddocumentation of services provided,which is required for CMHC billingto the DMH. The CMHC screenersfollow a strict protocol with requireddocumentation (electronically datestamped by encounter day in theelectronic medical record) for everyscreening encounter; this did notdiffer by format of the screening visit.For all participants (interventionand control), all 3 data points of datesfor referral at the FQHC, screeningat the CMHC, and, if eligible, theCMHC intake visit were consistentlyavailable and chronologically sound.We also collected data from thesevisit logs on 2 additional outcomesclosely related to this primaryoutcome, including the number ofdays that elapsed from referralto the day of the CMHC eligibilityscreening visit, and for those whowere deemed eligible, completion ofthe subsequent in-person intake visit.

Secondary outcome measures atthe 6-month follow-up includedparent-reported measures of childhealth–related quality of life by usingthe previously validated PediatricQuality of Life Inventory 4.0,50

family-centeredness of care (by usingthe 6-item family-centered carescale developed by the Maternal andChild Health Bureau in collaborationwith the National Center for HealthStatistics51 and used in the NationalSurvey of Children with SpecialHealth Care Needs and the NationalSurvey of Children’s Health), andoverall satisfaction with the referralprocess and the care received(by using 2 adapted items fromthe Consumer Assessment ofHealthcare Providers and Systems[CAHPS] Health Plan Survey).52

Family-centeredness of care itemswere only asked of parents who hadcompleted at least 1 CMHC therapyvisit by the 6-month follow-up.

We also collected data on parentalfactors that may impact whetherparents could successfully complete

PEDIATRICS Volume 143, number 3, March 2019 3 by guest on October 2, 2020www.aappublications.org/newsDownloaded from

a referral for their child. These factorsincluded family demographics, thechild’s need for an MH assessment (asdefined as scoring above the cut pointfor psychological impairment onthe Pediatric Symptom Checklist[PSC]53,54), and parental risk fordepression (score $10) by using thePatient Health Questionnaire.55 Thetrial protocol is available on request.

Analysis

All analyses were performed by usingan intention-to-treat analysis.Differences were examined betweenthe intervention and control groupson baseline characteristics and onoutcome measures. Main outcomemeasures were examined inregression models. Interventionstatus was the main independentvariable; we used linear regressionfor the continuous outcome variable

(number of days from referral date toscreening date) and logisticregression for binary variables(completion of screening visit andintake visit). We first present modelsadjusted only for variables that wereunbalanced at baseline. Next, wepresent these regression modelsfurther adjusted for child’s age andsex, household annual income,parents’ highest educationalattainment, parental employment,PSC score, parent and child overallhealth, and parental depression. Toaccommodate the cluster design, weused a mixed-effect regression model(with random effects for the clinic) byusing SAS proc (SAS Institute, Inc,Cary, NC) mixed for a continuouslyscaled outcome and GLIMMIX fora binary outcome. Results ofregression models are presented asbetween-group differences for linear

regression and odds ratios (ORs) forlogistic regression with 95%confidence intervals (CIs). We used 2-sided tests with P , .05 for statisticalsignificance. All analyses wereperformed using SAS version 9.4 (SASInstitute, Inc).

Our a priori power analysis wasbased on the primary outcome ofinitial access to a screening visit andrequired an analytic N of 320 fora minimum detectable effect size of0.518 with 80% power, a = .05(2-sided) with 6 clusters anda 1:1 randomization, and intraclasscorrelation coefficient of 0.01.

RESULTS

Overall, 342 parents of children whowere referred to the CMHCs wereenrolled and completed a baselinesurvey (intervention: n = 164;control: n = 178); completion datafor the CMHC screening visit wereavailable for all participants. Forsecondary outcomes, 289 parents(85%) completed the 6-month survey(Fig 1).

Baseline characteristics were similaracross intervention- and control-group participants, with the exceptionof parents’ current employmentstatus and annual household income(Table 1). More control parents(61.8%) than intervention parents(42.7%) were not employed, andmore intervention parents (19.2%)than control parents (10.7%)reported an annual income of$$35000. The mean child age atenrollment was 8.6 years (SD: 2.3).Of children, 87% were Latino, 7%were non-Latino white, and 2%were African American; 40.4%lived in households in which Englishwas the primary language; 64.6%reported the highest householdeducation level as high school or less.Approximately 27% of children hadpsychological impairment by parentreport on the PSC (Table 1).

A greater proportion of children inthe intervention (80.49%) completed

FIGURE 1Participant flow diagram.

4 COKER et al by guest on October 2, 2020www.aappublications.org/newsDownloaded from

the initial-access screening visitcompared with control children(64.04%; Table 2). The numberof days between referral and the

initial-access screening visit wasgreater for intervention families(mean: 23.6 days) than controlfamilies (mean: 17.1 days). Among

those families who were scheduledfor an intake visit after the initialscreening visit (n = 213), .80%completed the in-person intake visit,

TABLE 1 Sample Characteristics

Variables Analytic N Overall Control (N = 178) Intervention (N = 164) P

Child demographicsChild race/ethnicity, n (%) 342 .63Latino — 296 (86.5) 153 (86.0) 143 (87.2) —

White, non-Latino — 24 (7.0) 11 (6.2) 13 (7.9) —

African American, non-Latino — 7 (2.0) 4 (2.2) 3 (1.8) —

Other, non-Latino — 15 (4.4) 10 (5.6) 5 (3.0) —

Male sex, n (%) 342 211 (61.7) 110 (61.8) 101 (61.6) .97Child age at enrollment, mean 6 SD, y 342 8.6 6 2.3 8.5 6 2.3 8.7 6 2.3 .46Health insurance, n (%) 342 .65Medicaid — 336 (98.2) 176 (98.9) 160 (97.6) —

Private insurance — 3 (0.9) 1 (0.6) 2 (1.2) —

Uninsured — 3 (0.9) 1 (0.6) 2 (1.2) —

Child overall health rating, n (%) 342Excellent — 79 (23.1) 41 (23.0) 38 (23.2) .45Very good — 107 (31.3) 52 (29.2) 55 (33.5) —

Good — 108 (31.6) 55 (30.9) 53 (32.3) —

Fair or poor — 48 (14.0) 30 (16.9) 18 (11.0) —

Pediatric quality-of-life rating, mean 6 SDTotal scale score 342 74.7 6 16.8 74.4 6 16.6 74.9 6 17.1 .77Physical health summary score 342 85.2 6 18.7 85.8 6 17.4 84.6 6 20.0 .54Psychosocial health summary score 342 69.0 6 18.6 68.3 6 18.7 69.8 6 18.4 .47

PSC rating, n (%)Psychological impairment 342 91 (26.6) 46 (25.8) 45 (27.4) .74

Parent demographicsParent race and/or ethnicity, n (%) 338 .47Hispanic — 291 (86.1) 150 (85.7) 141 (86.5) —

White, non-Latino — 33 (9.8) 16 (9.1) 17 (10.4) —

African American, non-Latino — 8 (2.4) 4 (2.3) 4 (2.5) —

Other, non-Latino — 6 (1.8) 5 (2.9) 1 (0.6) —

Female sex, n (%) 342 328 (95.9) 173 (97.2) 155 (94.5) .21English language proficiency, n (%) 337 .74Very well or well — 179 (53.1) 95 (54.0) 84 (52.2) —

Not well or not at all — 158 (46.9) 81 (46.0) 77 (47.8) —

Marital status, n (%) 342 .99Married or living with partner — 198 (57.9) 103 (57.9) 95 (57.9) —

Single, separated, or divorced — 144 (42.1) 75 (42.1) 69 (42.1) —

Currently employment, n (%) 342 .001Working full-time or part-time — 162 (47.4) 68 (38.2) 94 (57.3) —

Not working — 180 (52.6) 110 (61.8) 70 (42.7) —

Highest household educational attainment, n (%) 342 .89Less than high school — 127 (37.1) 64 (36.0) 63 (38.4) —

High school or GED — 94 (27.5) 52 (29.2) 42 (25.6) —

Some college or 2-y degree — 83 (24.3) 43 (24.2) 40 (24.4) —

4-y college degree or greater — 38 (11.1) 19 (10.7) 19 (11.6) —

Annual household income, $, n (%) 325 .04,10 000 — 65 (20.0) 42 (24.9) 23 (14.7) —

10 001–19 999 — 113 (34.8) 59 (34.9) 54 (34.6) —

20 000–34 999 — 99 (30.5) 50 (29.6) 49 (31.4) —

$35 000 — 48 (14.8) 18 (10.7) 30 (19.2) —

Household primary language is English, n (%) 342 138 (40.4) 72 (40.4) 66 (40.2) .97Parental probable depression (score $10) , n (%) 341 45 (13.2) 26 (14.6) 19 (11.7) .421Parent overall health rating, n (%) 341 .93Excellent or very good — 95 (27.9) 48 (27.0) 47 (28.8) —

Good — 135 (39.6) 71 (39.9) 64 (39.3) —

Fair or poor — 111 (32.6) 59 (33.1) 52 (31.9) —

GED, general equivalency diploma; —, not applicable.

PEDIATRICS Volume 143, number 3, March 2019 5 by guest on October 2, 2020www.aappublications.org/newsDownloaded from

which did not vary by interventionstatus (80.2% vs 83.5%).

In the adjusted analysis, children inthe intervention remained more likelyto complete the initial-accessscreening visit compared with controlchildren (adjusted OR 3.17 [95% CI1.46 to 6.91]). The difference in timebetween referral and initial-accessvisit was not statistically significantafter adjustment for covariates(Table 3).

Among parents who had completed atleast 1 CMHC therapy visit by the 6-month follow-up, 86.3% of parents inthe intervention and 75.3% controlparents reported receiving family-centered care, but this difference wasnot statistically significant (P = .08).Parents in the intervention groupreported higher satisfaction with thereferral system and with care overall(Table 4). Quality of life did not varyby intervention status at the 6-monthfollow-up.

DISCUSSION

Using a community-partneredapproach, our academic researchteam worked with a multisite FQHC,its 2 local CMHCs, and its families to

develop a new referral system forchildren referred from primary careto the CMHC using new structuralelements (ie, a telehealth carecoordinator) and processes (ie,telehealth-enhanced screening byCMHCs) for referrals. Families in theintervention were more likely tocomplete the initial access point forenrollment in the CMHCs. For thosewho completed this initial accesspoint (the CMHC eligibility screeningvisit), the families in the interventiontook ∼6 more days to achieve thisaccess compared with controlfamilies.

Our findings highlight the importanceof this initial access point fora successful referral to the CHMC. Just64% of control families successfullycompleted the CHMC screening visitcompared with 80% of interventionfamilies; however, once this step wascompleted, at least 80% of familiesfrom both the control andintervention groups were able tocontinue on to the intake visit andCHMC services.

The increased time to the initialaccess point was anticipated for theintervention clinics because thetelehealth care coordinator and CMHC

staff held all the videoconferencescreening visits on a singlepreselected day each week. Thislimited the availability of slots for thescreening visits but allowed parentsto have a coordinator at the FQHCprovide personalized assistance inconnecting with the CMHCs.

The PWG selected completion of thescreening visit as the indicator ofaccess because some families may bedetermined to be ineligible for MHservices after the screening visit fora number of reasons (eg, income, MHcondition, and zip code). Because ourintervention was focused on thisinitial access point and was poweredfor it as a primary outcome, it is notsurprising that we did not see anysignificant differences in health (ie,psychological impairment and qualityof life) among those who successfullyaccessed care regardless of the studygroup assignment. However, witha longer study follow-up period, it ispossible that variation in healthoutcomes could be studied amonga sample of all who were initiallyreferred, particularly if the higherrates of access for children in theintervention translate into a greaterproportion of children receivingservices.

Because the intervention referralsystem had multiple elements,including the videoconference visit,the FQHC telehealth care coordinator,and the CMHC orientation video, itis not clear which elements wereresponsible for the findings. Thecontrol families did not have access to

TABLE 2 Access to MH Clinic After Referral

Unadjusted Analysis Adjusted Analysisa

Control Intervention P Intervention Versus Control

Difference or OR (95% CI) P

Days to initial-access completion (N = 246) 17.10 6 20.36 23.69 6 20.51 .01 4.55 (210.83 to 19.94) .56Weeks to initial-access completion (N = 246) 2.93 6 2.95 3.84 6 2.90 .02 0.61 (21.55 to 2.76) .56

Initial access completed (N = 342) 114 (64.04%) 132 (80.49%) ,.001 3.02 (1.47 to 6.22) .003Completed an intake visitb (N = 213) 81 (83.51%) 93 (80.17%) .53 0.81 (0.43 to 1.52) .51

a Adjusted for employment status (working versus not working) and income ($$20 000 vs ,$20 000), which were selected because they were unbalanced between 2 arms (refer to thedescriptive table) at baseline.b Of 342 children, 213 were deemed eligible to receive CMHC services after the CMHC initial-access screening. Reasons for ineligibility for services included a zip code outside of theCMHC’s catchment area, presence of a developmental disability, lack of an MH need, private health insurance coverage, and not meeting income requirements.

TABLE 3 Access to MH Clinic After Referral (Full Adjusted Analysis)

Access to MH Clinic (Analytic N) OR (95% CI)a P

Initial access completed (N = 342) 3.17 (1.46 to 6.91) .004Time to initial-access completion (N = 246), d 4.04 (212.06 to 20.13) .62Time to initial-access completion (N = 246), wk 0.52 (21.75 to 2.79) .65Completed an intake visit (N = 213) 0.82 (0.41 to 1.63) .57

a Adjusted for employment status, income, child age and sex, household income, highest parental educational attainment,PSC score, child overall health, parental depression, and parent overall health.

6 COKER et al by guest on October 2, 2020www.aappublications.org/newsDownloaded from

any elements of the interventionbut did have the opportunity toconnect with the CMHC for theinitial eligibility screening visitvia phone, which could have beenmore convenient than returning tothe FQHC for the videoconferenceeligibility screening visit with thetelehealth care coordinator. Despitethis, the intervention familieswere more likely to have completedan eligibility screening visit; wecan hypothesize that the assistancefrom the telehealth care coordinatormay have played an important rolein access for families.

In other interventions, researchershave aimed to improve access toMH among low-income school-agedchildren by focusing on colocationof behavioral health within primarycare,31,33 collaborative care modelsof care that may include midlevelMH providers working with thePCPs,56,57 and immediate access topsychiatry consultation for PCPs.58

Each of these models has beenstudied, with varying levels ofevidence of effectiveness inincreasing MH care access.31,33,56–58

These models, however, may requirea restructuring of services, staff, orfinancing.

In the new referral systemimplemented by the FQHCs andCMHCs, no changes were made to theactual services that parents andchildren received, to organizational

structure and staffing (except for thetelehealth care coordinator), or tobilling arrangements. Thus, withoutdrastic organizational level changesand with a focus on a key accesspoint, a new referral structure andprocess can still lead to significantimprovements in patients’ accessto care.

Our findings are potentiallygeneralizable to primary careand/or CMHC partners. Manycounties and states use a similar“carved-out” system of MH referralfor children who are Medicaidinsured, and many CMHCs usea similar multistep process forenrollment for specialty care.Telehealth technology is increasinglyused as an element of delivery ofMH care, and thus, practices mayhave the technological tools andknowledge in place to use thisintervention.59

There are some key limitations to ourfindings. First, the referral systemwas created to address the specificneeds of the community partners and,thus, may need adaptation to begeneralizable to other locations orsettings. Second, our primaryoutcome was improved access to careafter initial referral, but we do notaddress in our findings the quality ofservices that patients receive oncethey do gain access to care at theCMHC. Therefore, we cannot concludethat the telehealth intervention for

improved referral is associated withimproved clinical outcomes. Finally,the CMHCs did not involve the payersof MH care for this population,limiting our capacity to identifybarriers and system solutions thatmay improve the intervention’ssustainability.

This study reveals that a noveltelehealth-coordinated referralprocess developed by usinga community-partnered approach tointervention design significantlyimproved initial access to CMHCs forchildren referred from primary care.Future research is needed to examinethe effectiveness of the interventionby using a larger sample size ofcommunity-based MH clinics anda longer follow-up period to accessclinical outcomes.

ABBREVIATIONS

CI: confidence intervalCMHC: community mental health

clinicDMH: Department of Mental

HealthFQHC: federally qualified health

centerMH: mental healthOR: odds ratioPCP: primary care providerPSC: Pediatric Symptom ChecklistPWG: project working groupRA: research associate

TABLE 4 Family-Centered Care, Parent Satisfaction, and Health-Related Quality of Life at 6-Month Follow-up

Variables Analytic N Overall Control (N = 155) Intervention (N = 134) P

Family-centered care, n (%)a 157 .08Yes — 127 (80.9) 58 (75.3) 69 (86.3) —

No — 30 (19.1) 19 (24.7) 11 (13.8) —

Parent satisfaction, mean 6 SDSatisfaction with referral system 286 8.3 6 2.3 7.9 6 2.7 8.8 6 1.7 .003Satisfaction with care 277 8.9 6 1.6 8.6 6 1.8 9.1 6 1.4 .01

Pediatric quality of life, mean 6 SDTotal scale score 289 80.8 6 15.3 80.6 6 15.2 81.0 6 15.6 .82Physical health summary score 288 88.3 6 16.7 87.7 6 17.0 89.0 6 16.3 .50Psychosocial health summary score 289 76.7 6 17.7 76.8 6 17.3 76.7 6 18.1 .98

a For family-centered care items, we only asked of participants who had started CMHC therapy visits by the time of the 6-mo follow-up.

PEDIATRICS Volume 143, number 3, March 2019 7 by guest on October 2, 2020www.aappublications.org/newsDownloaded from

and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

This trial has been registered at www.clinicaltrials.gov (identifier NCT02396576).

DOI: https://doi.org/10.1542/peds.2018-2738

Accepted for publication Dec 14, 2018

Address correspondence to Tumaini R. Coker, MD, MBA, Seattle Children’s Research Institute, 2001 8th Ave, Room 650, Seattle, WA 98121. E-mail: tumaini.coker@

seattlechildrens.org

PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).

Copyright © 2019 by the American Academy of Pediatrics

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

FUNDING: Supported through grants from Patient-Centered Outcomes Research Institute (IH-12-11-4168) and the California Community Foundation (BAPP-14-

107111). All statements in this report, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of the

Patient-Centered Outcomes Research Institute or its Board of Governors of the Methodology Committee.

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2018-3765.

REFERENCES

1. Briggs-Gowan MJ, Horwitz SM, Schwab-Stone ME, Leventhal JM, Leaf PJ. Mentalhealth in pediatric settings: distributionof disorders and factors related toservice use. J Am Acad Child AdolescPsychiatry. 2000;39(7):841–849

2. Kataoka SH, Zhang L, Wells KB. Unmetneed for mental health care among U.S.children: variation by ethnicity andinsurance status. Am J Psychiatry. 2002;159(9):1548–1555

3. US Department of Health and HumanServices. Mental Health: A Report of theSurgeon General. Rockville, MD: USDepartment of Health and HumanServices, Substance Abuse and MentalHealth Services Administration, Centerfor Mental Health Services, NationalInstitutes of Health, National Institute ofMental Health; 1999

4. Leaf PJ, Alegria M, Cohen P, et al. Mentalhealth service use in the communityand schools: results from the four-community MECA Study. Methods forthe Epidemiology of Child andAdolescent Mental Disorders Study.J Am Acad Child Adolesc Psychiatry.1996;35(7):889–897

5. Fergusson DM, Woodward LJ. Mentalhealth, educational, and social roleoutcomes of adolescents withdepression. Arch Gen Psychiatry. 2002;59(3):225–231

6. Shaw M, Hodgkins P, Caci H, et al. Asystematic review and analysis of long-term outcomes in attention deficit

hyperactivity disorder: effects oftreatment and non-treatment. BMCMed. 2012;10:99

7. Coker TR, Elliott MN, Kataoka S, et al.Racial/ethnic disparities in the mentalhealth care utilization of fifth gradechildren. Acad Pediatr. 2009;9(2):89–96

8. Zimmerman FJ. Social and economicdeterminants of disparities inprofessional help-seeking for childmental health problems: evidence froma national sample. Health Serv Res.2005;40(5, pt 1):1514–1533

9. McMiller WP, Weisz JR. Help-seekingpreceding mental health clinic intakeamong African-American, Latino, andCaucasian youths. J Am Acad ChildAdolesc Psychiatry. 1996;35(8):1086–1094

10. Sturm R, Ringel JS, Andreyeva T.Geographic disparities in children’smental health care. Pediatrics. 2003;112(4). Available at: www.pediatrics.org/cgi/content/full/112/4/e308

11. Hough RL, Hazen AL, Soriano FI, Wood P,McCabe K, Yeh M. Mental healthservices for Latino adolescents withpsychiatric disorders. Psychiatr Serv.2002;53(12):1556–1562

12. Cunningham PJ, Freiman MP.Determinants of ambulatory mentalhealth services use for school-agechildren and adolescents. Health ServRes. 1996;31(4):409–427

13. Howell E. Access to Children’s MentalHealth Services Under Medicaid andSCHIP. Washington, DC: The UrbanInstitute; 2004

14. Huang L, Stroul B, Friedman R, et al.Transforming mental health care forchildren and their families. Am Psychol.2005;60(6):615–627

15. Children’s Defense Fund. The barriers:why is it so difficult for children to getmental health screens and assessments?2009. Available at: www.childrensdefense.org/child-research-data-publications/data/barriers-children-mental-health-screens-assesments.html. AccessedDecember 7, 2012

16. Cuffe SP, Waller JL, Cuccaro ML,Pumariega AJ, Garrison CZ. Race andgender differences in the treatment ofpsychiatric disorders in youngadolescents. J Am Acad Child AdolescPsychiatry. 1995;34(11):1536–1543

17. Lambert MC, Weisz JR, Knight F. Over-and undercontrolled clinic referralproblems of Jamaican and Americanchildren and adolescents: the culturegeneral and the culture specific.J Consult Clin Psychol. 1989;57(4):467–472

18. Weisz JR, Suwanlert S, Chaiyasit W,Walter BR. Over- and undercontrolledreferral problems among children andadolescents from Thailand and theUnited States: the wat and wai ofcultural differences. J Consult ClinPsychol. 1987;55(5):719–726

8 COKER et al by guest on October 2, 2020www.aappublications.org/newsDownloaded from

19. González MJ. Access to mental healthservices: the struggle of povertyaffected urban children of color. ChildAdolesc Social Work J. 2005;22(3–4):245–256

20. Porras-Javier L, Bromley E, Lopez M,Coker TR. Challenges to effectiveprimary care-specialty communicationand coordination in the mental healthreferral and care process for publiclyinsured children. J Behav Health ServRes. 2018;45(4):668–677

21. Golberstein E, Eisenberg D, Gollust SE.Perceived stigma and mental healthcare seeking. Psychiatr Serv. 2008;59(4):392–399

22. Owens PL, Hoagwood K, Horwitz SM,et al. Barriers to children’s mentalhealth services. J Am Acad ChildAdolesc Psychiatry. 2002;41(6):731–738

23. Nadeem E, Lange JM, Edge D, Fongwa M,Belin T, Miranda J. Does stigma keeppoor young immigrant and U.S.-bornBlack and Latina women from seekingmental health care? Psychiatr Serv.2007;58(12):1547–1554

24. Syed ST, Gerber BS, Sharp LK. Travelingtowards disease: transportationbarriers to health care access.J Community Health. 2013;38(5):976–993

25. Zima BT, Bussing R, Tang L, et al. Qualityof care for childhood attention-deficit/hyperactivity disorder in a managedcare medicaid program. J Am AcadChild Adolesc Psychiatry. 2010;49(12):1225–1237, 1237.e1–1237.e11

26. Kolko DJ, Perrin E. The integration ofbehavioral health interventions inchildren’s health care: services,science, and suggestions. J Clin ChildAdolesc Psychol. 2014;43(2):216–228

27. Asarnow JR, Rozenman M, Wiblin J,Zeltzer L. Integrated medical-behavioralcare compared with usual primarycare for child and adolescentbehavioral health: a meta-analysis.JAMA Pediatr. 2015;169(10):929–937

28. Olfson M. Evaluating the quality ofcommunity care for attention-deficit/hyperactivity disorder. J Am Acad ChildAdolesc Psychiatry. 2010;49(12):1183–1185

29. Sarvet B, Gold J, Bostic JQ, et al.Improving access to mental health carefor children: the Massachusetts Child

Psychiatry Access Project. Pediatrics.2010;126(6):1191–1200

30. Gruttadaro D, Markey D. A Family Guide:Integrating Mental Health and PediatricPrimary Care. 1st ed. Arlington, VA:National Alliance on Mental Illness; 2011

31. Williams J, Shore SE, Foy JM. Co-location of mental health professionalsin primary care settings: three NorthCarolina models. Clin Pediatr (Phila).2006;45(6):537–543

32. Ford J, Steinberg K, Pidano A, HonigfeldL, Meyers J. Behavioral Health Servicesin Pediatric Primary Care: Meeting theNeeds in Connecticut. Farmington, CT:Child Health and Development Instituteof Connecticut; 2006. Available at: file:///C:/Users/lporras/Downloads/behavioral_health_services_in_pediatric_primary_care__executive_brief.pdf. Accessed April 28, 2016

33. Briggs RD, Racine AD, Chinitz S.Preventive pediatric mental health care:a co-location model. Infant Ment HealthJ. 2007;28(5):481–495

34. US Department of Health and HumanServices, Health Resources andServices Administration. Telehealth.2012. Available at: https://www.hrsa.gov/rural-health/telehealth/index.html.Accessed December 9, 2012

35. Hjelm NM. Benefits and drawbacks oftelemedicine. J Telemed Telecare. 2005;11(2):60–70

36. Goldstein F, Myers K. Telemental health:a new collaboration for pediatriciansand child psychiatrists. Pediatr Ann.2014;43(2):79–84

37. Hilt RJ, Barclay RP, Bush J, Stout B,Anderson N, Wignall JR. A statewidechild telepsychiatry consult systemyields desired health system changesand savings. Telemed J E Health. 2015;21(7):533–537

38. Nelson EL, Barnard M, Cain S. Treatingchildhood depression overvideoconferencing. Telemed J E Health.2003;9(1):49–55

39. Soares NS, Langkamp DL. Telehealth indevelopmental-behavioral pediatrics.J Dev Behav Pediatr. 2012;33(8):656–665

40. Yellowlees PM, Hilty DM, Marks SL,Neufeld J, Bourgeois JA. A retrospectiveanalysis of a child and adolescenteMental Health program. J Am Acad

Child Adolesc Psychiatry. 2008;47(1):103–107

41. Myers KM, Valentine JM, Melzer SM.Child and adolescent telepsychiatry:utilization and satisfaction. Telemed J EHealth. 2008;14(2):131–137

42. Diamond JM, Bloch RM. Telepsychiatryassessments of child or adolescentbehavior disorders: a review ofevidence and issues. Telemed J EHealth. 2010;16(6):712–716

43. Ekeland AG, Bowes A, Flottorp S.Effectiveness of telemedicine:a systematic review of reviews. IntJ Med Inform. 2010;79(11):736–771

44. Grady B, Myers KM, Nelson EL, et al;American Telemedicine AssociationTelemental Health Standards andGuidelines Working Group. Evidence-based practice for telemental health.Telemed J E Health. 2011;17(2):131–148

45. Thomas CR, Holzer CE III. Nationaldistribution of child and adolescentpsychiatrists. J Am Acad Child AdolescPsychiatry. 1999;38(1):9–15; discussion15–16

46. Thomas CR, Holzer CE III. The continuingshortage of child and adolescentpsychiatrists. J Am Acad Child AdolescPsychiatry. 2006;45(9):1023–1031

47. Donabedian A. The quality of care. Howcan it be assessed? JAMA. 1988;260(12):1743–1748

48. Donabedian A. Evaluating the quality ofmedical care. Milbank Mem Fund Q.1966;44(suppl 3):166–206

49. Contreras S, Porras-Javier L, Zima BT,et al. Development of a telehealth-coordinated intervention to improveaccess to community-based mentalhealth. Ethn Dis. 2018;28(suppl 2):457–466

50. Varni JW, Seid M, Kurtin PS. PedsQL 4.0:reliability and validity of the PediatricQuality of Life Inventory version 4.0generic core scales in healthy andpatient populations. Med Care. 2001;39(8):800–812

51. Child and Adolescent HealthMeasurement Initiative. 2005-2006National Survey of Children withspecial health care needs indicatordata set, data resource centerfor child and adolescent health.Available at: http://childhealthdata.

PEDIATRICS Volume 143, number 3, March 2019 9 by guest on October 2, 2020www.aappublications.org/newsDownloaded from

org/learn-about-the-nsch/archive-prior-year-data-documents-and-resources/2005-06-ns-cshcn. AccessedMay 12, 2009

52. Agency for Healthcare Researchand Quality. Topics and relateditems in the CAHPS Health PlanSurvey 4.0 Core Questionnaires.Available at: https://www.cahps.ahrq.gov/content/products/PDF/PROD_HP4_Core.htm. AccessedNovember 20, 2015

53. Little M, Murphy JM, Jellinek MS, BishopSJ, Arnett HL. Screening 4- and5-year-old children for psychosocialdysfunction: a preliminary studywith the Pediatric Symptom Checklist.J Dev Behav Pediatr. 1994;15(3):191–197

54. Pagano M, Murphy JM, Pedersen M,et al. Screening for psychosocialproblems in 4-5-year-olds duringroutine EPSDT examinations: validityand reliability in a Mexican-Americansample. Clin Pediatr (Phila). 1996;35(3):139–146

55. Kroenke K, Spitzer RL, Williams JB. ThePHQ-9: validity of a brief depressionseverity measure. J Gen Intern Med.2001;16(9):606–613

56. Curry SJ, Hollis J, Bush T, et al. Arandomized trial of a family-basedsmoking prevention intervention inmanaged care. Prev Med. 2003;37(6, pt1):617–626

57. Pbert L, Flint AJ, Fletcher KE, Young MH,Druker S, DiFranza JR. Effect of

a pediatric practice-based smokingprevention and cessation interventionfor adolescents: a randomized,controlled trial. Pediatrics. 2008;121(4).Available at: www.pediatrics.org/cgi/content/full/121/4/e738

58. Connor DF, McLaughlin TJ, Jeffers-TerryM, et al. Targeted child psychiatricservices: a new model of pediatricprimary clinician–child psychiatrycollaborative care. Clin Pediatr (Phila).2006;45(5):423–434

59. Myers K, Comer JS. The case fortelemental health for improvingthe accessibility and quality ofchildren’s mental health services.J Child Adolesc Psychopharmacol. 2016;26(3):186–191

10 COKER et al by guest on October 2, 2020www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2018-2738 originally published online February 15, 2019; 2019;143;Pediatrics 

Park, Alpa Patel, Lingqi Tang, Paul J. Chung and Bonnie T. ZimaTumaini R. Coker, Lorena Porras-Javier, Lily Zhang, Neelkamal Soares, Christine

Randomized TrialA Telehealth-Enhanced Referral Process in Pediatric Primary Care: A Cluster

ServicesUpdated Information &

http://pediatrics.aappublications.org/content/143/3/e20182738including high resolution figures, can be found at:

Referenceshttp://pediatrics.aappublications.org/content/143/3/e20182738#BIBLThis article cites 51 articles, 3 of which you can access for free at:

Subspecialty Collections

http://www.aappublications.org/cgi/collection/telehealth_care_subTelehealth Carechnology_subhttp://www.aappublications.org/cgi/collection/health_information_teHealth Information Technology_subhttp://www.aappublications.org/cgi/collection/system-based_practiceSystem-Based Practicee_management_subhttp://www.aappublications.org/cgi/collection/administration:practicAdministration/Practice Managementfollowing collection(s): This article, along with others on similar topics, appears in the

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtmlin its entirety can be found online at: Information about reproducing this article in parts (figures, tables) or

Reprintshttp://www.aappublications.org/site/misc/reprints.xhtmlInformation about ordering reprints can be found online:

by guest on October 2, 2020www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2018-2738 originally published online February 15, 2019; 2019;143;Pediatrics 

Park, Alpa Patel, Lingqi Tang, Paul J. Chung and Bonnie T. ZimaTumaini R. Coker, Lorena Porras-Javier, Lily Zhang, Neelkamal Soares, Christine

Randomized TrialA Telehealth-Enhanced Referral Process in Pediatric Primary Care: A Cluster

http://pediatrics.aappublications.org/content/143/3/e20182738located on the World Wide Web at:

The online version of this article, along with updated information and services, is

by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397. the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, 60143. Copyright © 2019has been published continuously since 1948. Pediatrics is owned, published, and trademarked by Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it

by guest on October 2, 2020www.aappublications.org/newsDownloaded from