systemic alliance in individual therapy: factor analysis of the itas-sf and the relationship with...

SYSTEMIC ALLIANCE IN INDIVIDUAL THERAPY:FACTOR ANALYSIS OF THE ITAS-SF AND THE

RELATIONSHIP WITH THERAPY OUTCOMES ANDTERMINATION STATUS

Jesse OwenUniversity of Louisville

Clients providing systematic feedback to therapists via self-report measures of psychologi-cal distress and working alliance have been shown to increase therapy outcomes. How-ever, there are few systemic-based measures that are feasible for therapists to use.Recently, Pinsof et al. (Family Process, 2008, 47, 281) developed a brief systemic alli-ance measure (ITAS-SF) for individual therapy. The current study tested the factorstructure of this measure and examined whether the subscales related to clients’ therapyoutcomes and termination status (N = 570). The results demonstrated supported a 3-factormodel for the ITAS-SF (as compared to the seven factors proposed by Pinsof et al.). Inthe first factor, content combined the goals for therapy, the tasks or methods to reachthose goals and bond between the client and therapist. The second factor reflected howclients perceive the relationship with the therapist (i.e., interpersonal dimension–self ⁄ ther-apist), and the third factor reflected how clients perceive the alliance between their socialnetwork and the therapist (i.e., interpersonal dimension others ⁄ therapist). The two inter-personal factors were related to therapy outcome and termination status.

There is growing empirical evidence to suggest that when individual clients provide systematicfeedback to their therapists, via ratings scales of psychological distress and therapeutic alliance,they have better therapy outcomes (Lambert & Shimokawa, 2011). For instance, Anker, Duncan,and Sparks (2009) conducted a randomized clinical trial of 205 couples and found that when clientratings of the alliance and psychological distress were integrated throughout therapy sessions,outcomes for couples were significantly better than couples who did not provide this feedback.This finding along with a series of studies of individual therapy (see Lambert & Shimokawa, 2011for a review) suggests that therapists’ ability to attend to the treatment progress and allianceduring therapy could enhance therapy outcomes. Indeed, therapeutic alliance is one of the most con-sistent and meaningful predictors of therapy outcome across treatment modalities and approaches(Horvath, Del Re, Fluckiger, & Symonds, 2011; Orlinsky, Ronnestad, &Willutzki, 2004).

In individual therapy, therapeutic alliance, typically, has been defined as the agreementbetween the client and therapist on the goals for therapy, the methods or tasks to reach thosegoals, and the emotional or relational connection (Bordin, 1979). While this definition has pre-vailed and is useful, it does not account for the interpersonal influences on clients and the ther-apy process (see Heatherington, Friedlander, & Greenberg, 2005). That is, from aninterpersonal standpoint, traditional conceptions of the therapeutic alliance only reflect the rela-tionship between the client and the therapist. Thus, as many marriage and family therapiststreat individual clients while retaining a systemic perspective in their conceptualizations andinterventions, they may benefit from using systemic alliance measures as a form of feedback.

The Integrative Psychotherapy Alliance-revised (IPAr) model incorporates clients’ rela-tional networks into the process of therapy (Pinsof, 1994, 1995). Specifically, the IPAr modelseparates the therapeutic alliance into two main dimensions: content and Interpersonal (Pinsof,1995). Consistent with traditional definitions of alliance, the Content dimension includes goals,tasks, and bonds. The Interpersonal dimension is defined by four relational subsystems, which

Jesse Owen, PhD, Assistant Professor, Department of Education and Counseling Psychology, College of

Education, University of Louisville.

Address correspondence to Jesse Owen, Department of Education and Counseling Psychology, College of

Education, University of Louisville, Louisville, Kentucky 40292; E-mail: [email protected]

Journal of Marital and Family Therapydoi: 10.1111/j.1752-0606.2011.00268.x

JOURNAL OF MARITAL AND FAMILY THERAPY 1

captures the investment and collaboration between the therapist, client, and the client’s rela-tional network for therapeutic progress (Fig. 1). The first subsystem, client ⁄ therapist (i.e., Self),describes clients view of the alliance with their therapist, which is most similar with the tradi-tional conception of alliance. The second subsystem encapsulates clients’ perception of the alli-ance between their social network (e.g., peers, family, and significant others) and the therapist(i.e., Other). For example, a client and therapist may agree about the goals for therapy (e.g.,high Self-alliance), but the client’s partner may not be supportive of the therapist’s interven-tions (e.g., low Other alliance). Note in individual therapy, the client’s partner does not directlyparticipate in the sessions, but can still be affected by and be influential to the process of ther-apy. The third subsystem does not directly involve the therapist, rather it describes clients’ per-ception of alliance, in regard to the goals, tasks, and relational bond of treatment, with theirsocial network (i.e., Within). For instance, a client may come to therapy contemplating chang-ing careers; however, the client’s partner may not approve of such exploration. Thus, theWithin subsystem does not merely measure the quality of the relationship between clients andtheir social network; rather, it assesses the degree to which clients and their social network arealigned for the purpose of therapeutic gain. The last subsystem reflects the alliance between allmembers—the client’s social network, the client, and the therapist (i.e., Group). Simply, theGroup alliance assumes the whole of the social network can be influential above and beyondthe subsystems that create it (Pinsof, 1995; Pinsof, Zinbarg, and Knobloch-Fedders, 2008).

The interpersonal alliance dimensions have traditionally been used in couples and familytherapy, where the partner or family members are typically present in the therapy room(Heatherington & Friedlander, 1990; Johnson & Talitman, 1997; Knobloch-Fedders, Pinsof, &Mann, 2004). However, clients’ interpersonal networks are also present (albeit not physically)in individual therapy. Clients’ friends and family have an investment in clients’ well-being, suchas a wife caring about her husband’s depression. Therefore, as clients attempt to make changesin their life, their social network will also be affected. In fact, clients’ social networks can be animportant factor in making therapeutic gains, even when they do not participate in the therapysessions (Corrigan & Phelan, 2004; Laudet, Cleland, Magura, Vogel, & Knight, 2004; Mallinckrodt,1996). For instance, Corrigan and Phelan (2004) found that the clients’ satisfaction with theirsocial network was related to increased hope and fewer depressive symptoms. Moreover, clients’social support is associated with positive gains in therapy after controlling for the client ⁄ thera-pist therapeutic alliance (Mallinckrodt, 1996). However, most studies have not directly assessedthe systemic nature of alliance (i.e., Other, Within, and Group alliances).

Currently, there are only two known studies that have examined the systemic alliance inindividual therapy (Pinsof & Catherall, 1986; Pinsof et al., 2008). Results from these studiesaccentuate that additional information can be gained from examining interpersonal alliance.For instance, in a study of clients treated in an outpatient community clinic, Pinsof et al. (2008)found that higher Within alliance was related to better therapy outcomes. In other words, cli-ents and their significant others’ collective ‘‘buy-in’’ to the treatment was related to better

Client Therapist

Social Network

Within Other

Self

Group

Figure 1. Theoretical configuration of the interpersonal subsystems for systemic alliance.

2 JOURNAL OF MARITAL AND FAMILY THERAPY

outcomes. However, lower levels on the Other subscale of alliance related to therapy drop out,suggesting that clients who perceived that their social networks were not allied with thetherapist were more likely to end treatment prematurely. Moreover, these studies are consistentwith growing research that suggests systemically oriented interventions even when conductedwith individuals can be beneficial (e.g., Owen & Rhoades, 2010; Szapocnik, Kurtines, Foote,Perez-Vidal, & Hervis, 1986). Collectively, these results support the importance of includingsystemic conceptualizations of the therapy process and the therapeutic alliance in particular.

EXTENDING CURRENT RESEARCH ON SYSTEMIC THERAPEUTIC ALLIANCE

Although research has found that systemic alliance is beneficial to consider, there are otherareas that warrant further consideration. First, Pinsof et al. (2008) proposed a measure of sys-temic alliance, Individual Treatment Alliance Scale Revised-Short Form (ITASr-SF). This scalehas 15 items that reflect the 3 · 4 factor structure of systemic alliance. Consistent with the IPArmodel, it has three Content subscales (e.g., goals, tasks, and bonds) and four Interpersonal sub-scales (e.g., Self, Other, Within, and Group). The ITASr-SF was developed through a confirma-tory factor analysis based on the original 36-item version of the scale. The reduction of itemswas based on the modification fit indices as the original factor structure did not adequately fitthe data (Pinsof et al., 2008). Given the recent development of this scale, further psychometricevidence is needed to determine whether the factor structure would be replicated with a largersample when they were administered the 15-item version of the ITASr-SF.

Second, the examination of systemic alliance in individual therapy is still in its infancy. Inthis study, we will examine therapy outcome, specifically gains in psychological well-being—acommon indicator of therapy effectiveness. Further, we will test the relationship between sys-temic alliance and client-reported termination. Most studies have examined therapists’ perspec-tive of termination (Wierzbicki & Pekarik, 1993). However, therapists’ view of termination isbest understood as one side of the coin, with the other being the clients’ perspective. Forinstance, Owen, Smith, and Rodolfa (2009) examined clients’ report of the termination processand found the ways in which clients decided to end therapy was notably diverse (e.g., mutualdecision between the therapist and client to end therapy, client initiated by ‘‘no-show’’, clientinitiated after discussing termination with their therapist, and therapist initiated). Moreover,the ways clients end therapy has varied as a function of alliance (Owen, Smith et al., 2009).Simply, clients who ended therapy by no-show reported lower alliance with their therapist.However, there are no known studies examining how Interpersonal alliance relates to clients’report of termination. It is reasonable to posit that strong alliances with social networks (e.g.,Other and Within alliances) would strengthen the commitment to therapy, as the clients’ socialnetwork are aligned with the goals for therapy. Thus, replicating and extending this literaturecan provide a more nuance understanding to the change process, which ultimately can assisttherapists to optimize the therapy experience.

CURRENT STUDY

This study combined two samples from previous studies (see Owen, Imel, Tao et al., 2010;Owen, Smith et al., 2009) to analyze the ITASr-SF. These studies were not associated with theoriginal Pinsof et al. (2008) study. In these prior studies, only the total ITASr-SF score was used.First, we posited that the theoretical 3 · 4 factor structure of ITASr-SF would be supported(hypothesis 1). Second, we used the ITASr-SF to predict therapy outcome and client reportedtermination. We hypothesized that higher Interpersonal alliance would be related to bettertherapy outcomes (hypothesis 2) and less client-initiated termination-‘‘no-show’’ (hypothesis 3).

METHOD

ParticipantsClients from a large university counseling center were recruited to participate in a study of

their counseling experiences. The final sample included 570 clients. One hundred and forty-two


clients were excluded because they did not complete the entire survey or they did not fit thetreatment parameters of this study (e.g., group or couples therapy, came for an off-campusreferral, or did not attend at least two sessions). The response rate was calculated based on thenumber of respondents (n = 712) divided by the total number of emails sent (n = 2,373) minusthe returned emails (n = 133; total 32% response rate).

Clients included 352 women and 190 men (28 clients did not indicate their sex) with a med-ian age of 22 years old (range = 18–44). Thirty-two percent of the clients were graduate stu-dents, 22.5% were seniors, 18% were juniors, 12.5% were sophomores, 8.4% were freshman,1.6% were nonstudents, and 4.4% did not indicate an educational level. Over half, 61.1% ofthe sample was White, 1.4% of clients identified as African American, 19% identified as AsianAmerican, 7.5% identified as Hispanic, and 12.5% identified as multiethnic. Four clients(0.7%) did not indicate their ethnicity.

Clients’ reasons for seeking help were assessed through an open-ended question: ‘‘Pleasedescribe what concerns you wanted to address in your counseling?’’ The responses ranged fromeating disorders, stress-management, anxiety, academic difficulties, depression, and adjustmentto relationship difficulties. Given the variability and lack of clear diagnostic determinants, thespecific concerns were not analyzed. The median number of sessions reported in the currentstudy was six (SD = 5.4), which is slightly higher than the average number of sessions at thiscounseling center (4.7 sessions).

MeasuresIndividual Treatment Alliance Scale Revised-Short Form. The ITASr-SF is a 15-item client-

rated measure of therapeutic alliance, which is rated on a seven-point scale ranging from 7(Completely Agree) to 1 (Completely Disagree). The measure is intended to represent the 3 · 4theoretical structure of the IPAr model (Pinsof et al., 2008). Similar to most alliance scales isthe assessment of goals, tasks, and bonds (e.g., Content dimension). Next, the ITASr-SFassesses alliance within four interpersonal subsystems. The first interpersonal subsystem, theSelf subscale, describes the alliance between the client and therapist, an example item is: ‘‘Thetherapist does not understand me’’ (reverse coded). The second subsystem operationalized bythe ITASr-SF is the Other subscale, which reflects clients’ perceptions of the alliance betweentheir social network and the therapist. An example item is: ‘‘The people who are important tome would feel accepted by the therapist.’’ The third subsystem, the Within subscale, describesthe alliance between clients and their social network (the therapist is not directly assessed in thissubsystem). An example item is ‘‘Some of the people who are important to me would not beaccepting of my involvement in this therapy’’ (reverse coded). Lastly, the Group subscale cap-tures the gestalt of clients’ social networks and the therapist in regard to the alliance. An exam-ple item is: ‘‘The therapist is helping me with my important relationships.’’ Psychometrics forthe ITASr-SF are presented in the results section.

Schwartz Outcome Scale-10 (SOS-10). The SOS-10 was the primary outcome measure inthe current study and is a 10-item scale designed to assess clients’ psychological well-being overthe past week on a seven-point scale ranging from 1 (Never) to 7 (All the time or nearly all thetime). The reference sample (N > 9,000) for the SOS-10 was drawn from various clinical popu-lations (Blais et. al., 1999) (e.g., inpatient, outpatient, and college counseling centers) and non-clinical populations (e.g., adults from the community, college students) (see Owen & Imel, 2009for a review). Across studies, the SOS-10 has exhibited adequate reliability (e.g., test ⁄ re-test,r = .88; Cronbach’s alpha (a) = .91; Owen & Imel, 2009). Further, the SOS-10, the convergentand divergent validity, has been supported in previous studies as it correlates in the predicteddirection with a variety of clinical and psychological well-being scales (e.g., Beck’s HopelessnessScale, Beck, Kovacs, & Weissman, 1975; OQ-45, Lambert et al., 1996; SF-12 Mental healthand Physical health, Ware, Kosinski, & Keller, 1995) and reliably discriminates between clinicaland nonclinical samples (Blais et al., 1999; Hilsenroth, Ackerman, & Blagys, 2001). In thisstudy, the Cronbach’s alpha (a) of the SOS-was .93.

Initial emotional state. Clients retrospectively rated their emotional state at intake ‘‘Howwere you feeling when you started counseling?’’ on a five-point scale ranging from 1 (Very good[Life was much the way I liked it to be]) to 5 (Very poor [I barely managed to deal with things])


(Consumer Reports, 1994; Seligman, 1995). Similar to previous retrospective studies (e.g., Nielsenet al., 2004; Owen, Wong, & Rodolfa, 2009; Seligman, 1995), we used the initial emotion stateas a proxy for pretherapy functioning. Some support exists for the use of retrospective assess-ments of psychological functioning. For instance, Nielsen et al. (2004) compared clients’ recallof emotional state at intake with their actual intake score on the OQ-45, which was filled outbefore every session. The results showed that clients scores at intake on the OQ-45 werestrongly correlated with clients’ recall of their initial emotional state after 55 weeks, (r = .57;Nielsen et al., 2004). Nielsen et al. (2004) concluded that the relationship was ‘‘of sufficientmagnitude to fall within the range of validity indexes generally accepted for measures of psy-chotherapy outcome’’ (p. 33). Further, clients’ recall of emotional state was not affected by thelength of time between when they started therapy and when they completed the retrospectiveassessment (Nielsen et al., 2004).

How Therapy Ended. Clients were asked how therapy ended through a multiple choice for-mat (Owen, Smith et al., 2009). Responses were as follows: (a) client initiated the end of ther-apy without talking with the therapist (Client-End: No-Show; n = 100), (b) client initiated theend of therapy after talking with the therapist (Client-End: Discuss; n = 55), (c) therapist initi-ated the end of therapy (Therapist-End; n = 18), (d) the end of therapy was mutually decided(Mutual-End; n = 128), and (e) currently in therapy (Currently in Therapy; n = 187). Eighty-three participants did not respond or indicated the ‘‘other’’ response. This measure of termina-tion status has successfully differentiated clients’ alliance scores and treatment outcomes (Owen,Smith et al., 2009).

ProceduresDuring intake, clients were asked whether they would be willing to receive a survey about

their therapy experience on their intake card. Clients who agreed were sent an email at the endof the academic quarter and were able to access the anonymous survey online. Clients whoagreed to participate initially completed informed consent, the therapy process and outcomemeasures, and other questions related to the functioning of the counseling center (not analyzedhere). All measures were completed anonymously and online. We did not collect unique identi-fying information that could reveal clients’ identity, and the clients were assured the resultswould be presented to therapists in aggregate. To reduce the amount of time between the endof therapy and completing the measures, we emailed clients who attended therapy services dur-ing the first 6 months at mid-year; clients who came for services after the first 6 months wereemailed at the end of the year. Clients were only emailed at one time (e.g., at mid-year or endof the year, not both). Prior studies have shown that the length of time between the end oftherapy and completion of the measures did not significantly impact the ratings of therapy pro-cess and outcome (e.g., Nielsen et al., 2004; Owen, Tao, & Rodolfa, 2009).

Data Analytic Strategy for Confirmatory Factor AnalysesTo test the factor structure of the ITASr-SF, confirmatory factor analyses were conducted

utilizing AMOS 16 (Arbuckle, 2007). Confirmatory factor analysis is typically indicated forlater stages of scale development, particularly after a clear theory guiding the scale’s purpose isdelineated and the scale has demonstrated some initial validation (Kline, 1998). Accordingly,the systemic alliance theory has been clearly espoused and the subscales of the ITASr-SF havesome initial validation (Pinsof, 1995; Pinsof et al., 2008).

Statistically speaking, confirmatory factor analysis models only the relationships betweenthe specified observed variables and the related factor(s), whereas in exploratory factor analysis,all variables are free to relate to all factors. In other words, confirmatory factor analysis allowsfor a direct test of the proposed factor structure rather than searching for the existence of a fac-tor structure. As with exploratory factor analysis, testing theoretically meaningful alternativemodels lends credence to the conclusions one can draw from the results (MacCallum, Wegener,Uchino, & Fabrigar, 1993). Thus, we also included a test of the one-factor model and orthogo-nal and oblique factor structures.

Proposed models and model fit indices. To test the first hypothesis, we modeled the rela-tionship between the seven factors and the items that represent these subscales. Specifically, the


ITASr-SF theoretically assumes that each item simultaneously loads on a content and interpersonalfactor. Additionally, we tested the difference between the orthogonal model (e.g., uncorrelatedfactors) and the oblique model (e.g., correlated factors). Similar to Pinsof et al. (2008), this testallows for an examination of a larger second-order factor structure for the ITASr-SF.

As there is little consensus about the most appropriate fit statistics for confirmatory factoranalyses (Heubeck & Neill, 2000), we used a variety of indices and reported the findings fromPinsof et al. (2008) for comparison. First, the root mean square error approximation (RMSEA)was one measure of model fit used for this study. RMSEA scores that are closer to 0 are pre-ferred (Kenny & McCoach, 2003). Models with estimates ranging from 0.05 to 0.08 are deemedadequate, and in Pinsof et al. (2008), the RMSEA was 0.08. Second, the standardized rootmean residual (SRMR) was used and lower values are preferred; SRMR of 0.08 or lower sug-gest an adequate fitting model (Kline, 1998), and in Pinsof et al. (2008), the SRMR was 0.06.Third, the Comparative Fit Index (CFI) estimates should be around 0.90 or higher for ade-quate model fit (Kline, 1998), and the CFI was 0.93 in Pinsof et al. (2008). Lastly, we comparedthe chi-square estimates for the various models to determine which models were a better repre-sentation of the data.

RESULTS

Confirmatory Factor Analysis for the ITASr-SFInitially, we tested the theoretical seven-factor structure for the ITASr-SF wherein each

item loaded on a content and interpersonal factor. We also modeled the correlation betweenthe content latent factors (e.g., tasks, goals, and bonds with each other) and the interpersonallatent factors (e.g., Self, Other, Group, and Within subscales with each other). Similar to Pinsofet al. (2008), the model did not produce an admissible solution. Sources for this error appearedto rest in the high correlations between the latent subscales. The reported latent scale correla-tions, which should be interpreted with caution given the lack of convergence, were high forthe Self and Group subscales (r = .88) and the Other and Within subscales (r = .91). More-over, the content subscales all demonstrated correlations over 0.90. In contrast, the Self sub-scale was moderately correlated with Other and Within subscales (rs = .68, .60), and theGroup subscale was also moderately correlated with Other and Within subscales (rs = .70,.59). Correlations above .75 are evidence of multicollinearity and suggest that the subscales areproducing redundant information (Tabachnick & Fidell, 2001). Accordingly, we attempted toreplicate the empirical factor structure found in Pinsof et al. (2008) where six items did notcross load on two factors. However, this factor structure also returned an inadmissible solution.Collectively, these results did not support hypothesis 1.

Given the lack of replication and high correlations between the subscales, we tested amodel wherein the content factors loaded on one factor and the Self ⁄Group subscales loadedon a second factor, and Other ⁄Within subscales loaded on a third factor. The decision to col-lapse these subscales was based on their high correlations (e.g., multicollinearity) that suggestedthese subscales are measuring similar concepts (statistically speaking). Additionally, this factorstructure also has merit theoretically (see Discussion section). The results for the three-factorstructure were supported by the analyses without any additional changes based on the modifi-cation indices (Table 1). Next, we tested whether the oblique or orthogonal models would be abetter representation of the data. The oblique (or correlated) solution was a better fit to thedata than the orthogonal model. Lastly, we compared the fit of the model with a one-factormodel (i.e., global alliance). The three-factor structure showed a better fit to the data than aone-factor model. These results suggest that the three-factor model is a good representation ofthe data, and the subscales are related, yet distinct elements of the therapeutic alliance.

The internal consistency estimates for the three factors were good: content, a = .89, Self ⁄ -Group alliance a = .78 and Other ⁄Within alliance a = .84. For illustration, we calculated thepercentage of clients who reported a ‘‘strong alliance’’ defined by, on average, rating the alli-ance as ‘‘Completely Agree’’ or ‘‘Strongly Agree’’ (the top two response options on the seven-point scale) for the three subscales. For the Self ⁄Group subscale, 40.9% of clients were codedas having a strong alliance (M = 5.54, SD = 0.76). Forty-three percent (43.3%) of clients


reported having a strong Other ⁄Within alliance (M = 5.50, SD = 1.14). Lastly, 46.3% of cli-ents reported having a strong Content alliance (M = 5.72, SD = 0.99).

Prediction of Therapy Outcome and Termination StatusPredictive validity for the ITASr-SF was examined by testing the relationship between the

Interpersonal subscales (Self ⁄Group, Other ⁄Within) and therapy outcome and termination sta-tus. Given the overlap of Content and Interpersonal items, we decided to not examine the pre-dictive ability of the Content factor. As a general analytic plan, we utilized hierarchalregression models to better illuminate the relationships between the Interpersonal subscales andtherapy outcome and termination status. Accordingly, we entered control variables (e.g., pre-therapy functioning) in the first step, the Other ⁄Within subscale in the second step, and theSelf ⁄Group subscale in the last step. This approach will allow for an examination of the changein the variance for the Other ⁄Within subscale when the Self ⁄Group subscale is introduced inthe final step.

To predict therapy outcome, we conducted a hierarchal linear regression with SOS-10 (cur-rent psychological well-being) as the dependent variable, and initial emotional state (entered inthe first step), Other ⁄Within (second step) and Self ⁄Group (third step) subscales as the indepen-dent variables. The results from all three steps were statistically significant. Table 2 reports the

Table 1Summary of Confirmatory Factor Analyses for ITASr-SF

v2 dfCFI(>0.90)

RMSEA(<0.08)

SRMR(<0.06)

One-factor 1991.05 90 0.67 0.19 0.13Three-factor oblique 441.87 80 0.94 0.08 0.04Three-factor orthogonal 981.23 83 0.84 0.14 0.27

Notes. One factor = all 15 items loading on one factor. Three factors = (a) Content (goals,tasks, bonds), (b) Self + Group subscales, and (c) Within + Other subscales. Chi-squaredifference tests: one-factor versus three-factor oblique = v2 difference 1549.18, p < .001;three-factor orthogonal versus three-factor oblique = v2 difference 539.36, p < .001.CFI = Comparative Fit Index; ITASr-SF = Individual Treatment Alliance ScaleRevised-Short Form; RMSEA = root mean square error approximation; SRMR = stan-dardized root mean residual.

Table 2Summary of Hierarchal Linear Regression Predicting Treatment Outcome by ITASr-SFSubscales after Controlling for Initial Emotional State

B (SE) b r sr

Initial emotional state )0.31 (0.05) ).25** ).25** ).26**Other ⁄Within 0.15 (0.04) .16** .31** .15**Self ⁄Group 0.37 (0.07) .26** .32** .24**

Notes. Reported statistics are from the final model. Step 1 F(1, 565) = 35.95, R2 = .06; Step2 F = 47.28, R2 = .14; Step 3 F = 44.25, R2 = .19, r = zero-order correlation, sr = par-tial correlation; ITASr-SF = Individual Treatment Alliance Scale Revised-Short Form.**p < .001.


model statistics for the three steps and the regression coefficients, bivariate and partialcorrelations for variables in the final model. Of note, the Other ⁄Within subscale demonstrateda partial correlation with therapy outcome of .30 in the second step (after controlling for initialemotional state). However, this effect decreased to .15 when the Self ⁄Group subscale wasentered in the final step. Nonetheless, both alliance subscales contributed to the prediction oftherapy outcome in the final model.

Lastly, we predicted termination status by clients’ initial emotional state, current psycho-logical well-being, Self ⁄Group, and Other ⁄Within alliance through a hierarchal multinomiallogistic regression. Termination status has five categories (Client-End: Discuss; Client-End: No-Show; Therapist-End; Mutual-End; and Currently in Therapy). We used Client-End: No-Showas the reference category, as this category theoretically represents the most incongruent connec-tion between the client and therapist.

In the first step, we entered the control variables clients’ initial emotional state and currentpsychological well-being. The results of this model were significant, v2(8) = 17.28, p = .03,with current psychological well-being as the only significant predictor of termination status,v2(4) = 15.64, p = .004. In the second step, we entered the Other ⁄Within alliance score. TheOther ⁄Within alliance did not significantly predict termination status, v2(4) = 2.06, p = .73.Finally, we entered the Self ⁄Group alliance score. The overall model, with all predictors, wasstatistically significant, v2(16) = 48.15, p < .001. The results in Table 3 are presented as a con-trast to Clients-End: No-Show for the final model. The results demonstrated that clients whoinitiated the end of therapy by discussing the end of therapy, mutually deciding to end therapy,or were currently in therapy had higher Self ⁄Group scores as compared to clients who ended

Table 3Summary of Hierarchal Multinomial Logistic Regression Predicting Termination Status

B (SE) exp(B)95% CIfor exp(B)

Client-end: discussa

Self ⁄Group alliance 0.56* (0.23) 1.75 1.12–2.74Other ⁄Within alliance )0.22 (0.16) 0.81 0.59–1.11Initial emotional state 0.25 (0.20) 1.28 0.86–1.89SOS-10 0.36 (0.19) 1.43 0.99–2.06

Therapist-enda

Self ⁄Group alliance 0.15 (0.32) 1.17 0.62–2.20Other ⁄Within alliance 0.03 (0.26) 1.03 0.62–1.73Initial emotional state 0.22 (0.31) 1.25 0.68–2.28SOS-10 )0.08 (0.26) 0.93 0.56–1.53

Mutual-enda

Self ⁄Group alliance 0.47** (0.17) 1.60 1.14–2.24Other ⁄Within alliance )0.30* (0.13) 0.74 0.57–0.96Initial emotional state 0.14 (0.16) 1.15 0.84–1.56SOS-10 0.19 (0.14) 1.21 0.92–1.59

Current in therapya

Self ⁄Group alliance 0.87** (0.17) 2.38 1.72–3.31Other ⁄Within alliance )0.38** (0.13) 0.68 0.53–0.87Initial emotional state 0.07 (0.15) 1.08 0.81–1.45SOS-10 )0.11 (0.13) 0.90 0.90–1.15

Notes. Reported statistics are from the final model. SOS-10 = Schwartz Outcome Scale-10.aReference group is Client-end: no-show. *p < .05; **p < .01.


therapy by no-show. In contrast to the previous step, clients who mutually ended therapy withtherapist or were currently in therapy had lower Other ⁄Within alliance scores than clients whodecided to end therapy by no-show. The emergence of the Other ⁄Within alliance score as statis-tically significant predictor in the presence of Self ⁄Group alliance score is a type of suppressioneffect (Henard, 1998; Holling, 1983). Simply, the results suggest that the Other ⁄Within alliancescore has a relationship with termination status, only after accounting for the varianceexplained by the Self ⁄Group alliance. Clients’ initial emotional state and SOS-10 scores wereno longer related to how therapy ended after controlling for alliance scores. These results pro-vide new empirical evidence about the processes that may be operating as clients end therapy.

DISCUSSION

The present study highlights the importance of understanding the systemic nature of thetherapeutic alliance in individual therapy. In particular, the current study demonstrated thatthe systemic nature of the alliance can be adequately assessed in a brief measure and systemicalliance is a meaningful predictor of therapy outcome and termination status. These findingsunderscore the belief that therapists never do therapy with one person, and the therapists maybe wise to consider the influences of clients’ social network in the process of individual therapy.

The current study is the first study to administer the 15-item version of the ITASr-SF andthen test the factor structure. The results from the factorial analyses painted a slightly differ-ent picture than the original theoretical IPAr model or empirically derived factor structure(Pinsof, 1995; Pinsof et al., 2008). We found three subscales (in contrast to the original 12subscales): (a) the Content subscale, which reflects the goals, tasks, and bonds, (b) the Selfand Group subscales were combined for the second subscale, and (c) the Within and Othersubscales were collapsed into the third subscale. The use of a global Content alliance scale isconsistent with the use and development of previous alliance measures (Blais, 2004). The join-ing of the Self and Group subscales is consistent with prior research on systemic alliance incouples and family systemic alliance measures (e.g., Mamodhoussen, Wright, Tremblay, &Poitras-Wright, 2005; Pinsof et al., 2008). However, the combining of the Within and Othersubscales is notably different than previous research (e.g., Pinsof et al., 2008). One potentialreason for the dichotomous interpersonal factor structure of the ITASr-SF could be the per-spective that clients are encouraged to reflect on when responding to the items. For instance,the Self and Group items prompt clients to reflect on their experiences with the therapist,whereas the Other and Within items encourages clients to reflect on what their social networkwould think about the therapist or the therapy process. While there are still notable differ-ences in the wording of the items for the various subscales, clients may not discern the nuanceof the items. Alternatively, the retrospective methodology used to assess alliance may haveobscured clients’ ability to discern the finer points of the items. Future verification of the fac-tor structure with other samples is needed before drawing firm conclusions. Nonetheless, theresults of this study still supports that the ITASr-SF captures the interpersonal nature of thetherapeutic alliance.

The Self ⁄Group and the Other ⁄Within subscales of the ITASr-SF significantly predictedtherapy outcome (e.g., psychological well-being). The magnitudes of the univariate alliance-outcomecorrelations for both subscales were consistent to prior studies utilizing prospective designs(Horvath et al., 2011). As to be expected, the two subscales share some common variance andthe Other ⁄Within subscale demonstrated a lower relationship with outcome than the Self ⁄ -Group subscale when analyzed together. Nonetheless, clients’ therapeutic gains were related tohow their social networks were supportive and aligned with the process of therapy—a likelycorrelate with social support (e.g., Corrigan & Phelan, 2004; Mallinckrodt, 1996). However,conceptualizing the relationship of social networks directly into individual therapy illuminates aspecial case of how clients’ significant others can influence the change process.

The ITASr-SF also provided unique information for how clients end therapy. Clients whoended therapy by ‘‘no-show’’ reported lower Self ⁄Group alliance than clients who initiated theend of therapy after discussing it with their therapist, mutually decided to end therapy, or werestill in therapy. Simply, clients who are less connected with their therapist are less likely to


continue services. This adds to growing literature that suggests client–therapist alliance is acentral component to continuation in therapy (e.g., Miller, Duncan, Sorrell, & Brown, 2005;Owen, Smith et al., 2009).

While we expected that lower Other ⁄Within alliance would be related to ending therapy by‘‘no-show’’ (since that would indicate clients’ social network were not supportive of the therapyprocess), we found no such relationship. However, clients who ended therapy by ‘‘no-show’’had higher Other ⁄Within alliance scores than clients who ended therapy through a mutual deci-sion with their therapist or who were still in therapy only when their Self ⁄Group alliance waslow. In other words, clients who do not feel allied with their therapist but believe that theirsocial network is supportive of and invested in the therapy process are more likely to end ther-apy by no-show. Potentially, these clients feel that going to therapy is not worth while, giventhe low Self ⁄Group alliance with their therapist, and simultaneously feel that their social net-work is allied in the therapeutic process and the changes they are attempting to make in theirlife. This discrepancy is intriguing as it may illuminate a difficult decision that clients have tomake about continuing their treatment. However, these post hoc explanations are currentlyspeculative and need additionally empirical vetting.

Implications for Clinical Practice and ResearchThe findings in the current study, coupled with prior studies (e.g., Pinsof & Catherall,

1986; Pinsof et al., 2008), suggest that the systemic model of alliance for individual therapyindeed has merit and warrants further investigations to understand the apparent complexitiesinvolved in the relationships between systemic alliance and therapy outcome and terminationstatus. Therapists may benefit from conceptualizing the alliance beyond the walls of the therapyoffice to include significant social networks of the client. Failure to do so may result in missingessential clinical information in the process of therapy. While many therapists are attuned tothe social networks of their clients, the ways in which these important people view the therapyprocess may increase therapists’ ability to maximize these relationships for therapeutic gains.Simply, the clients spend more time with their social networks than with their therapist. Thus,therapist can empower themselves and their clients by understanding how these relationshipsinfluence the process of therapy.

Clients’ perception of their social networks about therapy can be easily assessed by periodi-cally administering the ITASr-SF. This measure can be completed within 5 min, which makes itfeasible for therapists to use (Pinsof et al., 2008). The use of the ITASr-SF or therapists’ invita-tions to discuss these relational aspects with clients can assist the therapy process and ulti-mately increase therapeutic progress. In prior studies, examining the alliance with clients hasshown to be an effective way to decrease therapy dropout and increase therapy outcomes (e.g.,Anker et al., 2009; Miller et al., 2005).

Given the resounding support for alliance, researchers have called for more complex con-ceptualizations of therapeutic alliance (e.g., DeRubeis, Brotman, & Gibbons, 2005; Orlinskyet al., 2004). The IPAr answers this call. Future research should examine the specific ways inwhich therapists align with clients’ social network (e.g., Other ⁄Within alliance) as well as theways that clients’ include their social networks in the therapy process. Additionally, to increasetherapists’ ability to connect with clients’ social network, research should explore how clientsexpress their needs with their social network and the ways they recapitulate this process in ther-apy. Lastly, there could be meaningful moderators of alliance, especially Other ⁄Within alliance.In particular, it may be enlightening to see how therapists work with clients who have socialnetworks that are not supportive of the therapy process.

LimitationsThe results from this study should be interpreted within the scope of its methodological

strengths and limitations. The retrospective nature of this study coupled with the electronic sur-vey methodology raises several concerns. First, all the measures were completed at the sametime, which makes clients’ pretherapy functioning score a process of memory recall. There issome support for the method of retrospective assessment (see Nielsen et al., 2004; Owen, Smithet al., 2009; Owen, Wong et al., 2009; Seligman, 1995), and the alliance-outcome correlation in


the current study revealed similar magnitudes as longitudinal studies. However, thismethodology should not be considered a proxy for prospective designs. For instance, the finerpoints of process and outcome were unable to be captured (e.g., trajectories of therapeutic alli-ance and outcome).

Second, the response rate in the current study was low (32%); however, it is slightly higherto other electronic surveys (Northey, 2005). This limitation may have influenced the results(e.g., positive response bias, memory recall). However, the range of scores and outcomes werestill consistent of prospective therapy outcome designs. Third, this study carries the typicalstrengths and limitations of data sets obtained in naturalistic settings. As such, this studytraded more rigorous controls (e.g., prescreening, training therapists, monitoring interventions)for a larger, more diverse sample and naturalistic treatments. Therefore, these results should bereplicated in other treatment settings. Lastly, the ITASr-SF does not specify the person or per-sons that constitute the clients’ social network. While this limitation might obscure the compar-ison between clients, it also increases the flexibility of the measure.

REFERENCES

Anker, M. G., Duncan, B. L., & Sparks, J. A. (2009). Using client feedback to improve couple therapy outcomes:

A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77, 693–

704.

Arbuckle, J. L. (2007). Amos 16.0 user’s guide. Spring House, PA: Amos Development Corporation.

Beck, A., Kovacs, M., & Weissman, A. (1975). Hopelessness and suicidal behavior: An overview. Journal of the

American Medical Association, 234, 1146–1149.

Blais, M. A. (2004). Development of an inpatient treatment alliance scale. Journal of Nervous and Mental Disease,

192(7), 487–493.

Blais, M. A., Lenderking, W. R., Baer, L., deLorell, A., Peets, K., Leahy, L., et al. (1999). Development and ini-

tial validation of a brief mental health outcome measure. Journal of Personality Assessment, 73, 359–373.

Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy:

Theory, Research, and Practice, 16, 252–260.

Corrigan, P. W., & Phelan, S. M. (2004). Social support and recovery in people with serious mental illnesses.

Community Mental Health Journal, 40(6), 513–523.

DeRubeis, R. J., Brotman, M. A., & Gibbons, C. J. (2005). A conceptual and methodological analysis of the

nonspecifics argument. Clinical Psychology: Science and Practice, 12, 174–183.

Heatherington, L., & Friedlander, M. L. (1990). Couple and family therapy alliance scales: Empirical consider-

ations. Journal of Marital and Family Therapy, 16(3), 299–306.

Heatherington, L., Friedlander, M. L., & Greenberg, L. (2005). Change process research in couple and family

therapy: Methodological challenges and opportunities. Journal of Family Psychology, 19, 18–27.

Henard, D. H. (1998). Suppressor variable effects: Toward understanding an elusive data dynamic. Paper presented

at the Annual Meeting of the Southwest Educational Research Association, Houston, TX.

Heubeck, B. G., & Neill, J. T. (2000). Confirmatory factor analysis and reliability of the mental health inventory

for Australian adolescents. Psychological Reports, 87(2), 431–440.

Hilsenroth, M., Ackerman, S., & Blagys, M. (2001). Evaluating the phase model of change during short-term

psychodynamic psychotherapy. Psychotherapy Research, 11, 29–47.

Holling, H. (1983). Suppressor structures in the general linear model. Educational and Psychological Measurement,

43, 1–9.

Horvath, A. O., Del Re, A. C., Fluckiger, C., & Symonds, D. (2011). Alliance in individual psychotherapy. Psy-

chotherapy, 48, 9–16.

Johnson, S. M., & Talitman, E. (1997). Predictors of success in emotionally focused marital therapy. Journal of

Marital and Family Therapy, 23, 135–152.

Kenny, D. A., & McCoach, D. B. (2003). Effect of the number of variables on measures of fit in structural equa-

tion modeling. Structural Equation Modeling, 10(3), 333–351.

Kline, R. B. (1998). Principles and practice of structural equation modeling. New York: Guilford Press.

Knobloch-Fedders, L. M., Pinsof, W. M., & Mann, B. J. (2004). The formation of the therapeutic alliance in

couple therapy. Family Process, 43, 425–442.

Lambert, M. J., Burlingame, G. M., Umphress, V. J., Hansen, N. B., Vermeersch, D., Clouse, G., et al. (1996).

The reliability and validity of the Outcome Questionnaire. Clinical Psychology and Psychotherapy, 3, 106–

116.

Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72–79.


Laudet, A. B., Cleland, C. M., Magura, S., Vogel, H. S., & Knight, E. L. (2004). Social support mediates the

effects of dual-focus mutual aid groups on abstinence from substance use. American Journal of Community

Psychology, 34(3 ⁄ 4), 175–185.MacCallum, R. C., Wegener, D. T., Uchino, B. N., & Fabrigar, L. R. (1993). The problem of equivalent models

in applications of covariance structure analysis. Psychological Bulletin, 114, 185–199.

Mallinckrodt, B. (1996). Change in social support, working alliance, and psychological symptoms in brief ther-

apy. Journal of Counseling Psychology, 43, 448–455.

Mamodhoussen, S., Wright, J., Tremblay, N., & Poitras-Wright, H. (2005). Impact of marital and psychological

distress on therapeutic alliance in couples undergoing couple therapy. Journal of Marital and Family Ther-

apy, 31(2), 159–169.

Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The partners for change outcome management

system. Journal of Clinical Psychology: In Session, 61, 199–208.

Nielsen, S. L., Smart, D. W., Isakson, R. L., Worthen, V. E., Gregersen, A. T., & Lambert, M. J. (2004). The

consumer reports effectiveness score: What did consumers report? Journal of Counseling Psychology, 51(1),

25–37.

Northey, W. F. (2005). Studying marriage and family therapists in the 21st century: Methodological and techno-

logical issues. Journal of Marital and Family Therapy, 31, 99–106.

Orlinsky, D. E., Ronnestad, M. H., & Willutzki, U. (2004). Fifty years of process-outcome research: Continuity

and change. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy (pp. 307–365). New

York: John Wiley & Sons.

Owen, J. J., & Imel, Z. (2009). Utilizing rating scales in psychotherapy practice: Rationale and practical applica-

tions. In L. Baer & M. Blais (Eds.), Handbook of clinical rating scales and assessment in psychiatry and men-

tal health (pp. 257–270). New York, NY: Humana Press.

Owen, J. J., Imel, Z., Tao, K., Smith, A., Wampold, B., & Rodolfa, E. (2010). Cultural ruptures in psychother-

apy: Working alliance as a mediator between clients’ perceptions of microaggressions and therapy outcomes.

Counselling and Psychotherapy Research. Prepublication on-line. doi: 10.1080/14733145.2010.491551.

Owen, J. J., & Rhoades, G. K. (2010). Reducing interparental conflict among parents in contentious child

custody disputes: An initial investigation of the working together program. Journal of Marital and Family

Therapy. doi: 10:1111/j.1752-0606.2010.00215.x

Owen, J. J., Smith, A. A., & Rodolfa, E. (2009). Clients’ expected number of sessions, treatment effectiveness,

and termination status: Using empirical evidence to inform session limit policies. Journal of College Student

Psychotherapy, 23, 118–134.

Owen, J., Tao, K. T., & Rodolfa, E. R. (2010). Microaggressions against women: Initial evidence. The Counseling

Psychologist, 38, 923–946.

Owen, J. J., Wong, J. Y., & Rodolfa, E. R. (2009). Empirical search for gender competence. Psychotherapy, 46,

448–458.

Pinsof, W. M. (1994). An integrative systems perspective on the therapeutic alliance: Theoretical, clinical and

research implications. In A. Horvath & L. Greenberg (Eds.), The working alliance: Theory, research and

practice (pp. 173–195). New York: John Wiley & Sons.

Pinsof, W. M. (1995). Integrative problem centered therapy: A synthesis of biological, individual and family thera-

pies. New York: Basic Books.

Pinsof, W. M., & Catherall, D. R. (1986). The integrative psychotherapy alliance: Family, couple, and individual

therapy scales. Journal of Marital and Family Therapy, 12(2), 137–151.

Pinsof, W. B., Zinbarg, R., & Knobloch-Fedders, L. M. (2008). Factorial and construct validity of the revised

short form integrative psychotherapy alliance scales for family, couple, and individual therapy. Family Pro-

cess, 47, 281–301.

Seligman, M. E. P. (1995). The effectiveness of psychotherapy: The consumer reports study. American Psycholo-

gist, 50, 965–974.

Szapocnik, J., Kurtines, W. M., Foote, F., Perez-Vidal, A., & Hervis, O. E. (1986). Conjoint versus one person

family therapy: Further evidence for the effectiveness of conducting family therapy through one person.

Journal of Consulting and Clinical Psychology, 54, 395–397.

Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Needham Heights, MA: Allyn

and Bacon.

Ware, J. E., Kosinski, M., & Keller, S. D. (1995). SF-12: How to score the SF-12 physical and mental health sum-

mary scales (2nd ed.). Boston, MA: The Health Institute, New England Medical Center.

Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional Psychology:

Research and Practice, 24(2), 190–195.


systemic alliance in individual therapy: factor analysis of the itas-sf and the relationship with...

Documents