interparental conflict, adolescent behavioral problems...

10.1177/0013164403258462ARTICLEEDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT

EPSTEIN ET AL.

INTERPARENTAL CONFLICT, ADOLESCENT BEHAVIORALPROBLEMS, AND ADOLESCENT COMPETENCE:CONVERGENT AND DISCRIMINANT VALIDITY

MONICA K. EPSTEINUniversity of South Florida

KIMBERLY RENKUniversity of Central Florida

AMY M. DUHIGYale University

GEORGETTA L. BOSCOCounseling Center of Tampa Bay, Inc.

VICKY PHARESUniversity of South Florida

To address the lack of studies examining the convergent and discriminant validity ofcross-informant ratings, several statistical approaches were used in this study to evaluatethe convergent and discriminant validity for ratings of interparental conflict, adolescentbehavioral problems, and adolescent competence. A total of 272 adolescent-mother-father triads completed measures assessing interparental conflict, adolescent behavioralproblems, and adolescent competence. The validity of these measures appearedquestionnable when relationships were examined using Campbell and Fiske’s criteriaand confirmatory factor analysis, respectively. In contrast, analyses demonstrated ade-quate fit of cross-informant ratings in a correlated uniqueness model, supporting conver-gent and discriminant validity. Overall, results suggested that informant effects should beanalyzed when examining interparental conflict and adolescent adjustment. These find-ings emphasize the importance of considering the measurement issues of cross-infor-mant ratings when examining interparental conflict, adolescent behavioral problems,and adolescent competence.

Keywords: cross-informant ratings; conflict; behavior problems; competence

Educational and Psychological Measurement, Vol. 64 No. 3, June 2004 475-495DOI: 10.1177/0013164403258462© 2004 Sage Publications

475

The relationships among interparental conflict, adolescent behavioralproblems, and adolescent competence have been well documented in theresearch literature (as reviewed by Cummings & Davies, 1994). In general,findings have suggested that interparental conflict is distressing for childrenand adolescents. Furthermore, interparental conflict has been linked closelyto increased emotional and behavioral problems as well as to decreased com-petence in children and adolescents. Although a number of studies have ex-amined the interrelationships among these variables, few researchers haveexamined the validity of these constructs directly when using multiple infor-mants. Issues of the convergent and discriminant validity of constructs mea-sured with the ratings of multiple informants as well as issues pertaining topotential informant bias should be addressed if research in this area is to beadvanced. The purpose of this study was to evaluate these validity issuesusing a variety of statistical approaches and to examine the impact of theseissues on the assessment of interparental conflict, adolescent behavioralproblems, and adolescent competence.

Research has suggested that multiple informants should be used whenassessing conflict and violence in families (e.g., Sternberg, Lamb, & Dawud-Noursi, 1997). Thus far, most studies use parental reports only to assess thelevel of conflict in families rather than employing information from multipleinformants. Parents may underestimate or overestimate the amount of con-flict their children have witnessed (Grych, Seid, & Fincham, 1992) relative tothe reports of the children themselves, however. Parents may provide under-estimates when they are unaware of the amount of conflict their children havewitnessed, whereas they may provide overestimates when they define inter-actions that are not salient to children in the family as conflictual (Grychet al., 1992). As a result, Grych and colleagues (1992) suggested that moreaccurate estimates of children’s exposure to conflict would be provided byincluding measures that assess directly such exposure from children’sperspectives.

Similarly, many researchers have advocated for the use of multiple infor-mants in the rating of emotional and behavioral functioning in children andadolescents (e.g., Achenbach, McConaughy, & Howell, 1987; Duhig, Renk,Epstein, & Phares, 2000). Mothers and fathers are called on often to provideratings of such functioning. Such ratings, however, have been criticized asdisplaying bias and a lack of construct validity when multiple informantsfrom the same family do not demonstrate a substantial degree of congruencein their ratings (e.g., Noller & Guthrie, 1991). These findings have suggestedthat there is no “gold standard” in the ratings of emotional and behavioral

476 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT

This research was supported by the National Institute of Mental Health, Grant No. R2949601-03. Correspondence concerning this article should be addressed to Kimberly Renk,Ph.D., University of Central Florida, Department of Psychology, P.O. Box 161390, Orlando, FL32816; e-mail: [email protected].

problems in children and adolescents and that it is very important that thevalidity of cross-informant ratings of such behavior be examined.

Because there is no “gold standard” when examining cross-informant rat-ings of interparental conflict, adolescents’ emotional and behavioral prob-lems, and adolescents’ competence, the examination of the validity of theseconstructs should be continued. Research concerning construct validity typi-cally has emphasized the degree to which any measure or method determinesthe establishment of convergence and discrimination. Agreement amongmultiple informants regarding measured theoretical constructs or traits dem-onstrates convergent validity (e.g., Bryant, 2000). The ability of a constructor trait to be distinguished from other underlying constructs or traits, espe-cially when measured by the same informants, is evidence of discriminantvalidity (e.g., Bryant, 2000). Historically, Campbell and Fiske (1959) advo-cated using convergent and discriminant validity to establish construct valid-ity through the use of a multitrait-multimethod (MTMM) matrix. They fur-ther stated the need for multiple informants (or methods) and multiple traitsin determining convergent and discriminant validity. Although researchershave documented good convergent validity in parental ratings of internaliz-ing and externalizing behavioral problems in children and adolescents, therealso has been evidence of weaker discriminant validity and large informanteffects (Stanger & Lewis, 1993).

A major difficulty with the establishment of construct validity has beenthe degree to which ratings are influenced by informant (or method) vari-ance. Informant (or method) variance may distort the observed correlationsbetween measures (Campbell & Fiske, 1959). Because each informant con-tributes a considerable amount of variance not accounted for by other infor-mants (i.e., unique variance), the optimal approach to assessing the relation-ships among interparental conflict, adolescent behavioral problems, andadolescent competence involves the use of multiple sources of informationfor each construct (e.g., Grych & Fincham, 1990). For example, moderatedegrees of consistency between maternal and paternal reports of emotionaland behavioral problems in children and adolescents have been found(Achenbach et al., 1987; Duhig et al., 2000). The agreement of children andadolescents with their parents, however, has been found to be considerablylower than parent-parent agreement, especially with regard to internalizingbehavioral problems in children and adolescents (Achenbach et al., 1987).

In an effort to address such validity issues when using multiple infor-mants, three different methods for examining the convergent and discrim-inant validity of interparental conflict, adolescent behavioral problems, andadolescent competence were examined in this study. Campbell and Fiske’s(1959) original work with MTMM matrices assessed construct validationthrough the visual inspection of the correlations between each of the traitsand methods or informants. Campbell and Fiske (1959) proposed that con-

EPSTEIN ET AL. 477

struct validation could be determined by following several “rules.” First,Campbell and Fiske (1959) proposed that the monotrait-heteromethod corre-lations (i.e., the same traits rated by different informants) would differ signif-icantly from zero and would be sufficiently large if convergent validity wasdemonstrated. In other words, measures of the same construct should be cor-related highly if they assess a common construct (Bagozzi, 1993), even whenrated by multiple informants.

Next, Campbell and Fiske (1959) proposed that discriminant validity alsoshould be assessed. For discriminant validity to be demonstrated, themonotrait-heteromethod coefficients (i.e., same traits rated by differentinformants) should be higher than their corresponding heterotrait-heteromethod coefficients (i.e., different traits rated by different informants;Campbell & Fiske, 1959). Such a finding would demonstrate that efforts tomeasure the same concept by using different informants should yield highercorrelations than would efforts to measure different concepts by differentinformants (Bagozzi, 1993). In addition, the monotrait-heteromethod coeffi-cients should be higher than their corresponding heterotrait-monomethodcoefficients (i.e., different traits rated by the same informant; Campbell &Fiske, 1959). Such a finding would demonstrate that efforts to measure thesame concept with ratings by different informants produce higher correla-tions than do efforts to measure different concepts rated by the same infor-mant (Bagozzi, 1993). Finally, the pattern of correlations should be the sameamong the heterotrait-monomethod and heterotrait-heteromethodcorrelations (Campbell & Fiske, 1959).

Because Campbell and Fiske’s (1959; Fiske & Campbell, 1992) proce-dure does not provide precise standards for determining how well these crite-ria should be met and does not consider the importance of the magnitudes ofdifferences between pairs of coefficients (Bagozzi, 1993), a confirmatoryfactor analysis model has been proposed as an alternative for assessing con-vergent and discriminant validity (e.g., Wothke, 1996). This model hypothe-sizes that the total variation in the measurement of specific constructs stemsfrom the combination of trait, method or informant, and error effects (e.g.,Jöreskog, 1974). Such a model has several advantages such as allowing forthe estimate of the reliability of each measure, yielding measures of good-ness-of-fit for the overall model, and allowing for the comparison of alternateexplanatory models (Bryant, 2000). This method, however, requires that atleast three traits and three methods, four traits and two methods, or two traitsand four methods be incorporated into the model (Bagozzi, 1993). Further-more, the use of confirmatory factor analysis often results in technical prob-lems such as improper solutions, with parameter estimates falling outside ofthe permissible range (e.g., Marsh, 1989). Due to these problems, a variantof the confirmatory factor analysis approach known as the correlated


uniqueness model analyzed with a latent variable approach has beenproposed.

Kenny and Berman (1980) suggested that such a latent variable approachcould be used to examine the ratings of multiple informants for a variety ofconstructs (also discussed as multiperspective-multidyad confirmatory fac-tor analysis by Bray, Maxwell, & Cole, 1995). In such a model, the ratings ofmultiple informants are specified as imperfect indicators of the specific con-structs of interest. The relationship of the ratings of specific constructs pro-vided by multiple informants is measured using factor pattern coefficients,with high loadings of ratings on their respective construct indicating conver-gent validity. Because it is expected that each rating provided by informantsalso will include a certain amount of variability beyond the “true” relation-ship between the construct and the rating provided by the informants, residu-als are accounted for in the model (Cook & Goldstein, 1993). These residualsallow for the examination of informant effects. Finally, the constructs areallowed to correlate in this model, providing a measure of the relationshipsamong these constructs after removing the influence of measurement errorand systematic informant effects (Cook & Goldstein, 1993). These correla-tions should reflect the discriminant validity of the constructs of interest(Cook & Goldstein, 1993). Thus, this model is important because it allowsresearchers to examine the unique and common perspectives of multipleinformants while also examining additional variability, such as informant (ormethod) effects. This latent variable approach can be examined by using acorrelated uniqueness model in covariance structural modeling.

Applying each of the statistical models discussed here, this study evalu-ated the relationships among the observed measures of the hypothesizedlatent constructs (interparental conflict, adolescent internalizing and ex-ternalizing behavioral problems, and adolescent competence) provided bydifferent informants (adolescents, mothers, and fathers). The specified mod-els proposed that ratings of different informants that tap the same constructswould demonstrate convergent validity. Thus, it was expected that correla-tions among ratings of the same constructs by different informants would behigh in the Campbell and Fiske (1959) model. For the confirmatory factormodel and the correlated uniqueness model, it was expected that the ratingsof interparental conflict, adolescent internalizing and externalizing behav-ioral problems, and adolescent competence, whether rated by adolescents,mothers, or fathers, would load highly on their respective constructs. It alsowas expected that residuals across the ratings provided by the same infor-mant would be correlated in the correlated uniqueness model. This correla-tion should occur due to systematic informant effects across the ratings ofdifferent constructs. Finally, it was expected that the constructs examinedhere would demonstrate discriminant validity. In other words, the correla-

EPSTEIN ET AL. 479

tions between the constructs would be low to moderate, demonstrating thatthese constructs were related but different.

Method

Participants

A total of 272 biological adolescent-mother-father triads were recruitedfrom a southeastern suburban area to participate in the study. Of the families,142 had a male adolescent, whereas 130 families had a female adolescent.Adolescents’ages ranged from 11 to 18 years, with a mean age of 13.52 years(SD = 1.84 years). Mothers ranged in age from 27 to 55 years (M = 40.13years, SD = 5.06 years), and fathers ranged in age from 25 to 70 years (M =42.77 years, SD = 6.43 years). Regarding ethnicity, the sample was primarilyCaucasian (74.6%), with some families of African American (12.9%),Latino/Latina (10.7%), Asian American (0.4%), Native American (1.1%),and other (0.4%) ethnicities. A majority of the parents reported being mar-ried to their adolescent’s other parent (83.1%). The remainder were divorcedfrom their adolescent’s other parent and remarried (5.9%), were divorced butnot remarried (8.1%), had never married their adolescent’s other parent(0.7%), were widowed (0.4%), or had some other family configuration(0.4%). Information regarding marital status was unavailable for four (1.5%)of the families.

Although parents were recruited from the community, several mothersand fathers were experiencing varying degrees of psychological symptoms.Based on information obtained through structured clinical interviews usingthe Mini Structured Clinical Interview for the Diagnostic and StatisticalManual (SCID) and complete modules of the SCID (First, Gibbon, Williams,& Spitzer, 1995) conducted by trained researchers, 108 mothers met diag-nostic criteria for at least one current or previous diagnosis during their ado-lescent’s lifetime (22.4% met criteria for major depressive disorder, 31.3%met criteria for an anxiety disorder, and 16.9% met criteria for alcohol, sub-stance abuse, or dependence). A total of 106 fathers met diagnostic criteriaduring their adolescent’s lifetime (11.0% met criteria for major depressivedisorder, 10.7% met criteria for an anxiety disorder, and 40.5% met criteriafor alcohol or substance abuse or dependence). A total of 50 families con-sisted of mothers with a psychiatric diagnosis and nonclinical fathers, 52families had fathers with a psychiatric diagnosis and nonclinical mothers,and 49 families had mothers and fathers who both met clinical diagnostic cri-teria. In 98 families, neither parent met criteria for a psychiatric diagnosis. Atotal of 70 (25.7%) mothers were receiving individual psychological or psy-chiatric services, whereas 49 (18.0%) fathers were receiving individual psy-chological or psychiatric services. In all, 11 (4.0%) mothers and 11 (4.0%)


fathers were receiving marital or family treatment. Thus, the population offamilies examined here were diverse in their experience of psychologicalsymptoms.

Procedure

As part of a larger, prospective study, families were recruited via adver-tisements throughout the community. To meet eligibility for participation,mothers and fathers had to be the biological parents of adolescents who werebetween the ages of 11 and 18 years. Parents with more than one adolescentwere asked to select one of their adolescents for participation. Thus, parentshad complete control of which adolescent was brought to the research ses-sion. If both parents were not living currently in the same house, adolescentswere required to have at least monthly face-to-face contact with noncustodialparents. Families who contacted the researchers were invited to attend a 3-hour session at a southeastern university and subsequently were given $60for their participation.

After obtaining informed consent and permission from both parents andassent from adolescents at the beginning of the session, each participant wasadministered individually the research protocol by trained graduate-levelresearch assistants in clinical psychology. Each parent completed the struc-tured clinical interview mentioned previously (Mini-SCID and SCID mod-ules; First et al., 1995) and questionnaires designed to assess interparentalconflict, adolescent behavioral problems, and adolescent competence alongwith additional questionnaires not included in this study. Adolescents com-pleted parallel measures of interparental conflict and their own behavioralproblems and competence as well as other measures not included in thisstudy. Participants were provided with a brief explanation of the study andwere given the opportunity to have any questions answered following thecompletion of their participation.

Measures

Interparental conflict. Adolescents’ perceptions of interparental conflictwere measured using the six-item Marital Discord subscale of the Child’sPerception Questionnaire (CPQ; Emery & O’Leary, 1982). Mothers andfathers completed the O’Leary-Porter Scale–Revised (OPS-R; Porter &O’Leary, 1980), which was developed to assess the frequency of overt mari-tal hostility observed by adolescents. The scores for both these measureshave had adequate reliability and validity (e.g., Emery & O’Leary, 1982;Porter & O’Leary, 1980). In this study, the scores for both measures demon-strated adequate internal consistency (.90 for the CPQ and .83 for the OPSwhen completed by mothers; .81 for the OPS when completed by fathers).

EPSTEIN ET AL. 481

Adolescent behavioral problems and competence. To assess the adoles-cents’ behavioral problems and competence, adolescents completed theYouth Self-Report (YSR; Achenbach, 1991b). The YSR is a measuredesigned to provide standardized descriptions of emotional and behavioralproblems and competence in children and adolescents. A total of 118 itemsmeasure self-reported internalizing, externalizing, and total behavioral prob-lems. Self-reported involvement in sports, activities, organizations, andfamily and social interactions as well as the quality of these activities andinteractions were used to assess adolescents’ competence.

In addition, mothers and fathers completed the Child Behavior Checklist(CBCL; Achenbach, 1991a) independently. The CBCL is a parent-completed measure similar to the YSR. This measure consists of 120 prob-lem behavior and 20 competence items. The CBCL has been shown to dis-criminate between clinic-referred and nonreferred children (Achenbach,1991a).

T-scores from both the YSR and the CBCL for adolescent internalizingbehavioral problems, adolescent externalizing behavioral problems, andadolescent competence were used in this study. Although T-scores may resultin a truncated distribution for some measures, Achenbach (1991a) reportedthat the T-scores for the Internalizing Behavior Problems, ExternalizingBehavior Problems, and Competence scales were not truncated and shouldyield results similar to those obtained with raw scores. The Total BehaviorProblems scores from the YSR and CBCL were not used in this study due tothe item overlap between this scale and those for the Internalizing and Ex-ternalizing Behavior Problems scales. Adequate reliability and validity forscores on both the YSR and the CBCL have been documented (Achenbach,1991a, 1991b).

Results

Differences Among Ratings Provided byAdolescents, Mothers, and Fathers

Descriptive statistics for the measures used in this study are provided inTable 1. To examine the potential differences in the ratings provided by ado-lescents, mothers, and fathers, a series of paired-samples t tests were con-ducted. With regard to interparental conflict, adolescents differed signifi-cantly from their mothers, t (df = 269) = 29.84, p < .000, d = 3.64, and theirfathers, t (df = 260) = 30.21, p < .000, d = 3.75, in their ratings. In contrast,maternal and paternal ratings of interparental conflict did not differ signifi-cantly, t (df = 261) = .03, not significant, d = .00. With regard to adolescentinternalizing behavioral problems, adolescents differed significantly fromtheir mothers, t (df = 267) = 2.03, p < .04, d = .25, but not from their fathers, t


(df = 264) = .20, not significant, d = .02, in their ratings. Mothers and fathersalso differed significantly in their ratings of their adolescents’ internalizingbehavioral problems, t (df = 261) = –2.12, p < .04, d = –.26. With regard toexternalizing behavioral problems, adolescents did not differ significantlyfrom their mothers, t (df = 267) = –.43, not significant, d = –.05, or from theirfathers, t (df = 264) = –.07, not significant, d = .00, in their ratings. Similarly,mothers and fathers did not differ significantly in their ratings, t (df = 261) =.37, not significant, d = .05. Finally, with regard to adolescent competence,adolescents differed significantly from their mothers, t (df = 246) = –2.30, p <.02, d = –.29, and from their fathers, t (df = 244) = –2.89, p < .004, d= –.37,in their ratings. Mothers and fathers, however, did not differ in their ratings, t(df = 240) = –1.71, not significant, d = –.22.

Application of the Campbell and Fiske Criteria

Campbell and Fiske’s (1959) criteria were applied to the correlationmatrix in Table 2. To assess convergent validity, the validity diagonals wereexamined. Although each of the correlations in the validity diagonals weresignificantly different from zero, the diagonal values were not all large. Thecorrelations ranged from .15 to .58, with those correlations measuring theinternalizing behavioral problems of adolescents being the smallest in mag-nitude (.25, .15, and .38). These correlation magnitudes were consistent withprevious findings, which state that cross-informants tend to show a smalleramount of agreement on measures of internalizing behavioral problems inchildren and adolescents (e.g., Achenbach et al., 1987; Duhig et al., 2000).

EPSTEIN ET AL. 483

Table 1Descriptive Statistics

M SD Skew Kurtosis

Interparental conflictAdolescent report 1.34 0.20 1.33 0.98Maternal report 12.24 6.16 0.65 0.73Paternal report 12.12 5.89 0.61 0.11

Internalizing problemsAdolescent report 48.75 9.90 0.49 0.37Maternal report 50.31 10.40 0.24 –0.25Paternal report 48.82 9.92 0.23 0.03

Externalizing problemsAdolescent report 50.59 10.31 0.02 –0.39Maternal report 50.28 10.05 0.39 0.16Paternal report 50.47 9.80 0.18 –0.28

CompetenceAdolescent report 48.86 10.67 –0.30 1.00Maternal report 47.62 9.41 0.16 –0.14Paternal report 46.75 9.19 0.11 –0.04

484

Tabl

e 2

Mul

titr

ait-

Mul

tim

etho

d C

orre

lati

on C

oeffi

cien

t Mat

rix

for

Stud

y M

easu

res

12

34

56

78

910

1112

Ado

lesc

ent r

epor

t

1.In

terp

aren

tal c

onfl

ict

—2.

Inte

rnal

izin

g pr

oble

ms

.39a **

—3.

Ext

erna

lizin

g pr

oble

ms

.36a **

.54a **

—4.

Com

pete

nce

–.12

a–.

15a *

–.14

a *—

Mat

erna

l rep

ort

5.In

terp

aren

tal c

onfl

ict

.43b **

.16*

*.1

9**

–.06

—6.

Inte

rnal

izin

g pr

oble

ms

.17*

*.2

5b **.1

5*–.

17**

.23a **

—7.

Ext

erna

lizin

g pr

oble

ms

.20*

*.2

0**

.43b **

–.13

*.3

0a **.5

6a **—

8.C

ompe

tenc

e–.

23**

–.08

–.19

**.4

3b **–.

20a **

–.31

a **–.

40a **

—

Pate

rnal

rep

ort

9.In

terp

aren

tal c

onfl

ict

.36b **

.15*

.15*

–.10

.42b **

.13*

.19*

*–.

15**

—10

.In

tern

aliz

ing

prob

lem

s.1

0.1

5b *.0

8–.

29**

.03

.38b **

.22*

*–.

36**

.23a **

—11

.E

xter

naliz

ing

prob

lem

s.0

8.0

7.3

3b **–.

14**

.07

.23*

*.5

3b **–.

35**

.27a **

.61a **

—12

.C

ompe

tenc

e–.

12–.

20**

–.16

**.3

9b **–.

11–.

24**

–.24

**.5

8b **–.

12a

–.29

a **–.

28a **

—

Not

e:N

umbe

rs r

epre

sent

het

erot

rait-

hete

rom

etho

d di

scri

min

ant v

alid

ity c

oeff

icie

nts

unle

ss o

ther

wis

e in

dica

ted.

a. H

eter

otra

it-m

onom

etho

d di

scri

min

ant v

alid

ity c

oeff

icie

nts.

b. C

onve

rgen

t val

idity

coe

ffic

ient

s.*p

< .0

5. *

*p<

.01.

To examine discriminant validity, the validity diagonals were first com-pared with the heterotrait-heteromethod coefficients. This assessmentinvolved the comparison of each of the 12 correlations in the validity diago-nal with each of the 36 heterotrait-heteromethod coefficients. This compari-son revealed several heterotrait-heteromethod coefficients that were largerthan those in the validity diagonal. In particular, the correlations betweenadolescent-mother reports of internalizing behavioral problems, adolescent-father reports of interparental conflict, adolescent-father reports of internal-izing behavioral problems, and adolescent-father reports of externalizingbehavioral problems were larger than were those found in the validity diago-nal. As the proportion of failures of this first criterion was p = 24 / 432 = .055,which is at the same level as would be expected by chance (p = .05), itappeared that this criterion was met overall.

Next, the validity diagonals were compared with the heterotrait-monomethod coefficients. This assessment involved the comparison of eachof the 12 correlations in the validity diagonal with each of the 18 heterotrait-monomethod coefficients. Again, several violations were noted. In this case,the proportion of failures was p = 61 / 216 = .28, which is greater than wouldbe expected by chance. As a result, the second criterion for discriminantvalidity, in which coefficients in the validity diagonal should be larger thanheterotrait-monomethod coefficients, was not supported.

Finally, the pattern of correlations across the heterotrait-heteromethodand the heterotrait-monomethod correlations was examined using the rankorder of coefficients in each triangle. The pattern of correlations in theheterotrait-heteromethod and the heterotrait-monomethod triangles variedgreatly. As a result, the third criterion for discriminant validity, in which thepattern of correlation across the heterotrait-heteromethod and the heterotrait-monomethod triangles should be the same, was not supported.

Application of the Confirmatory Factor Analysis Model

Due to the subjectivity of Campbell and Fiske’s (1959) criteria forMTMM matrices, a confirmatory factor analysis model was used to deter-mine the relationships among the measured variables (assessed by the CPQ,OPS-R, YSR, and CBCL) in reference to the latent constructs (interparentalconflict, internalizing behavioral problems, externalizing behavioral prob-lems, and competence as well as adolescent ratings, maternal ratings, andpaternal ratings). In the confirmatory factor analysis model, both trait factorsand method factors are specified and are allowed to correlate. Thus, thismodel consisted of four intercorrelated trait factors (Interparental Conflict,Internalizing Behavioral Problems, Externalizing Behavioral Problems, andSocial Competence) and three intercorrelated method factors (AdolescentRatings, Maternal Ratings, and Paternal Ratings), each predicting three mea-

EPSTEIN ET AL. 485

sured variables. More specifically, the measured variables completed by ado-lescents, mothers, and fathers were allowed to load on their respective latentconstructs (e.g., the adolescents’ report on the CPQ and maternal and pater-nal reports on the OPS-R were allowed to load on the interparental conflictconstruct). If the measures completed by the different informants were to dis-play convergent validity, the factor pattern coefficients of these measuresshould be higher on each of their respective constructs than for other con-structs. In addition, it would be expected that ratings completed by eachinformant regardless of the construct measured also should load highly oneach respective method factor. Finally, each of the latent variables shoulddemonstrate low to moderate correlations rather than large correlations ifthey exhibit discriminant validity.

Model results. Based on the results for the proposed model, it was deter-mined that the solution was not proper. Several of the parameter estimateswere out of range, consistent with problems noted in previous research (e.g.,Marsh, 1989). As a result, the fit indices of this model were of questionnablevalue and are not reported here.

Application of the Correlated Uniqueness Model

Due to the subjectivity in identifying relationships in MTMM matricesusing the criteria provided by Campbell and Fiske (1959) and the statisticalproblems demonstrated with confirmatory factor analyses, a correlateduniqueness model also was used to determine the relationships of the mea-sured variables (assessed by the CPQ, OPS-R, YSR, and CBCL) to the latentconstructs (interparental conflict, internalizing behavioral problems,externalizing behavioral problems, and competence). As mentioned previ-ously, a correlated uniqueness model is consistent with the measuring of thelatent variable approach proposed by Kenny and Berman (1980). In corre-lated uniqueness models, only trait factors rather than trait and method fac-tors are specified in the model (Kenny & Kashy, 1992). The trait factors orconstructs of interest are allowed to correlate. To accommodate possibleinformant (or method) variance, the disturbance terms for each measurecompleted by the same informant (adolescent, mother, or father) are allowedto covary (Kenny & Kashy, 1992). The advantage of using this type ofapproach is that it allows for multidimensional effects of method (Kenny &Kashy, 1992) and has the ability to produce an identified solution. In otherapproaches, separate method and trait factors are specified for MTMM mod-els, sometimes resulting in empirical underidentification of the model. Thus,the unique and common contribution of multiple informants to the ratings ofa construct of interest as well as possible variance accounted for by using thesame informants across constructs can be measured.


The model tested consisted of four intercorrelated latent variables(interparental conflict, adolescent internalizing behavioral problems, adoles-cent externalizing behavioral problems, and adolescent competence), eachpredicting three measured variables (adolescent report, maternal report, andpaternal report). More specifically, the measured variables completed byadolescents, mothers, and fathers were allowed to load on their respectivelatent constructs (e.g., the adolescents’ report on the CPQ and maternal andpaternal reports on the OPS-R were allowed to load on the interparental con-flict construct; e.g., Bray et al., 1995). Furthermore, all measured variablescompleted by the same informant were allowed to covary in an effort toaccount for informant (or method) variance (e.g., the scores from the CPQand the YSR scales completed by adolescents were allowed to covary).Finally, interparental conflict, adolescent internalizing behavioral problems,adolescent externalizing behavioral problems, and adolescent competence(the latent constructs or variables) were allowed to intercorrelate. Theselatent variables should demonstrate low to moderate correlations rather thanlarge correlations if they exhibit discriminant validity. A correlated unique-ness model was then tested using LISREL 8.

Model results. Based on the fit measures for the proposed model, the fit ofthe data to the model was good. Although the χ2 of exact fit was significant(χ2 = 52.56, df = 30, p < .007), indicating that the proposed model was not aplausible model, this χ2 index tends to be sensitive to sample size and todepartures from multivariate normality of variables (Schumacker & Lomax,1996). As a result, other indices of fit were examined to derive a better esti-mate of the fit of the model. The χ2 of not a close fit (χ2 = .05, df = 30) indi-cated that the proposed model was a good estimate. In addition, the root meansquare error of approximation was small (.05), the normed fit index was large(.95), and the nonnormed fit index was large (.95). These measures indicatedthat the model demonstrated good fit to the data. Thus, the model appeared toaccount adequately for the relationships among the measured and latent vari-ables. The residuals examining informant effects, factor pattern coefficients,factor structure coefficients, and intercorrelations of the latent constructs areprovided in Tables 3 and 4.

If the ratings provided by each of the informants in the family are reliable,these ratings should be related to their respective latent variables. In thismodel, such a relationship was demonstrated through the significant factorpattern coefficients of the measured ratings on each of the respective latentvariables. Each of the factor pattern coefficients in this model was greaterthan .33. Although the factor pattern coefficients indicate the presence ofconvergent validity, none of these factor pattern coefficients were perfectindicators of their latent variables. This finding suggested that each infor-mant also was providing a unique component in his or her ratings. In addi-

EPSTEIN ET AL. 487

488

Tabl

e 3

Cor

rela

tion

s A

mon

g th

e R

esid

uals

for

Exa

min

ing

Info

rman

t Effe

cts

in th

e C

orre

late

d U

niqu

enes

s M

odel

CPQ

OPS

MO

PSD

YSR

INT

CB

INT

MC

BIN

TD

YSR

EX

TC

BE

XT

MC

BE

XT

DY

SRC

OM

PC

BC

OM

PMC

BC

OM

PD

CPQ

.—O

PSM

—.—

.—O

PSD

..—

.—.—

YSR

INT

.27*

*.—

.—.—

CB

INT

M.—

.12*

.—.—

.—C

BIN

TD

.—.—

.13*

.—.—

.—Y

SRE

XT

.23*

*.—

.—.4

6**

.—.—

.—C

BE

XT

M.—

.13*

*.—

.—.3

5**

.—.—

—C

BE

XT

D.—

.—.1

5**

.—.—

.39*

*.—

——

YSR

CO

MP

–.01

.—.—

–.04

.—.—

–.05

——

—C

BC

OM

PM.—

–.04

.—.—

–.05

.—.—

–.08

——

—C

BC

OM

PD.—

.—–.

01.—

.—.0

4.—

—–.

03—

——

CPQ

=ad

oles

cent

repo

rtof

conf

lict;

OPS

M=

mat

erna

lrep

orto

fcon

flic

t;O

PSD

=pa

tern

alre

port

ofco

nflic

t;Y

SRIN

T=

adol

esce

ntre

port

ofin

tern

aliz

ing;

CB

INT

M=

mat

erna

lrep

orto

fin-

tern

aliz

ing;

CB

INT

D=

pate

rnal

repo

rtof

inte

rnal

izin

g;Y

SRE

XT

=ad

oles

cent

repo

rtof

exte

rnal

izin

g;C

BE

XT

M=

mat

erna

lre

port

ofex

tern

aliz

ing;

CB

EX

TD

=pa

tern

alre

port

ofex

tern

aliz

ing;

YSR

CO

MP

= a

dole

scen

t rep

ort o

f co

mpe

tenc

e; C

BC

OM

PM =

mat

erna

l rep

ort o

f co

mpe

tenc

e; C

BC

OM

PD =

pat

erna

l rep

ort o

f co

mpe

tenc

e.*p

< .0

5. *

*p<

.01.

tion, with regard to ratings of interparental conflict, adolescent internalizingbehavioral problems, and adolescent externalizing behavioral problems, itappears that the residuals across measures completed by the same informantare related moderately to each other and are significant in a statistical sense,indicating a systematic bias of informant ratings.

EPSTEIN ET AL. 489

Table 4Factor Pattern Coefficients and Intercorrelation of Constructs in the Correlated UniquenessModel

Conflict Internalizing Externalizing Competence

Factor pattern coefficientsCPQ .57OPSM .66OPSD .64YSRINT .33CBINTM .55CBINTD .66YSREXT .52CBEXTM .73CBEXTD .71YSRCOMP .55CBCOMPM .81CBCOMPD .72

Factor structure coefficientsCPQ .57 .18 .19 –.18OPSM .66 .20 .21 –.20OPSD .64 .20 .21 –.20YSRINT .10 .33 .16 –.21CBINTM .17 .55 .26 –.35CBINTD .20 .66 .31 –.42YSREXT .17 .25 .52 –.26CBEXTM .24 .35 .73 –.37CBEXTD .23 .34 .71 –.36YSRCOMP –.17 –.35 –.27 .55CBCOMPM –.25 –.52 –.41 .81CBCOMPD –.22 –.46 –.36 .72

Intercorrelations of constructsConflict —Internalizing .31 —Externalizing .32 .47 —Competence –.31 –.64 –.50 —

Note. All factor pattern coefficients and interfactor correlations are statistically significant at .01 level. CPQ =adolescent report of conflict; OPSM = maternal report of conflict; OPSD = paternal report of conflict;YSRINT = adolescent report of internalizing; CBINTM = maternal report of internalizing; CBINTD = pater-nal report of internalizing; YSREXT = adolescent report of externalizing; CBEXTM = maternal report ofexternalizing; CBEXTD = paternal report of externalizing; YSRCOMP = adolescent report of competence;CBCOMPM = maternal report of competence; CBCOMPD = paternal report of competence.

This model also demonstrated the discriminant validity of the constructsof interest through the moderate but not large correlations among the latentvariables. Furthermore, the correlations among adolescent competence andthe other latent variables were in the expected direction. Overall, these rela-tionships demonstrated that these constructs were different but related.

Discussion

This study examined the validity of the cross-informant ratings ofinterparental conflict, adolescent behavioral problems, and adolescent com-petence using several statistical procedures. The findings of this study sup-ported the convergent validity of interparental conflict, adolescent internaliz-ing behavioral problems, adolescent externalizing behavioral problems, andadolescent competence as assessed by the ratings of adolescents, mothers,and fathers in the same family. Although the convergent validity of these con-structs was supported, the discriminant validity of these constructs was notsupported strongly. The presence of common as well as unique componentsin the cross-informant ratings of family members supported the use of multi-ple informants in determining the levels of interparental conflict, adolescentbehavioral problems, and adolescent competence.

With regard to the use of Campbell and Fiske’s (1959) criteria in the deter-mination of construct validity, convergent validity was supported, whereasdivergent validity was more questionable. Adolescents and their mothers andfathers tended to demonstrate significant agreement in their ratings of inter-parental conflict, adolescent behavioral problems, and adolescent compe-tence. As demonstrated in other studies (e.g., Achenbach et al., 1987; Duhiget al., 2000), these cross-informants tended to show a smaller amount ofagreement on measures of internalizing behavioral problems in children andadolescents. Because internalizing behavioral problems are less observableand more internal in experience, it is not uncommon for cross-informants todemonstrate lower concordance in such instances (Achenbach et al., 1987;Kenny, 1991; Kenrick & Stringfield, 1980).

When using Campbell and Fiske’s (1959) criteria, several violations werenoted with regard to the discriminant validity of the MTMM matrix exam-ined in this study. These findings suggested that each of the informants in thisstudy were contributing unique variance across their ratings of the constructsexamined. For example, in some cases, ratings of different constructs pro-vided by different informants and ratings of different constructs provided bythe same informant were larger than were ratings of the same construct bydifferent informants. As demonstrated by other researchers (e.g., Stanger &Lewis, 1993), there has been evidence of weak discriminant validity andinformant effects in the ratings of such variables. Overall, these results sug-gested that it is vitally important to continue an examination of these prob-lems in the cross-informant ratings of family and adolescent variables.


Unfortunately, the use of confirmatory factor analysis did not assist in theelucidation of the relationships among the constructs measured using cross-informant ratings in this study. Consistent with the findings of other research-ers (Bagozzi, 1993; Marsh, 1989; Marsh & Bailey, 1991), who have noted thenumber of problems that seem to plague confirmatory factor analyses whenused to interpret MTMM data, the solution for the confirmatory factor analy-sis model examined in this study exhibited out-of-range values for severalof the parameter estimates. As a result, the solution for this model could notbe used adequately to draw conclusions about the validity of the cross-informant ratings examined in this study.

With regard to the correlated uniqueness model used in this study, the con-struct validity of the ratings of interparental conflict, adolescent behavioralproblems, and adolescent competence was supported. With regard to conver-gent validity, factor pattern coefficients loaded significantly on their respec-tive latent variables. Although these coefficients were not equal across allinformants, they were consistently moderate to high for each latent variable.Consistent with previous research (e.g., Achenbach et al., 1987; Duhig et al.,2000), loadings for mothers and fathers were relatively higher and more con-sistent than were those provided by the adolescents themselves. Overall, thismodel also supported the divergent validity of the ratings of interparentalconflict, adolescent behavioral problems, and adolescent competence. Thelatent variables examined in this study were related but different. In mostcases, interparental conflict, internalizing behavioral problems,externalizing behavioral problems, and competence were correlated moder-ately, as suggested by several researchers (e.g., Cummings & Davies, 1994;Grych & Fincham, 1990).

Although the ratings of cross-informants using the correlated uniquenessmodel in this study provided support for the construct validity of interpar-ental conflict, adolescent behavioral problems, and adolescent competence,these ratings were not perfect. The residuals in the correlated uniquenessmodel were related moderately and significantly in the cases of interparentalconflict, adolescent internalizing behavioral problems, and externalizingbehavioral problems. Because adolescents and their mothers and fatherswere administered parallel forms of measures assessing these constructs, itmay be concluded that the ratings provided by these cross-informants alsowere affected by measurement effects and by systematic informant bias. Incontrast, significant relationships were not noted in the ratings of adolescentcompetence, possibly resulting from the multifaceted way in which adoles-cent competence was measured in this study. The competence scores sup-plied by the YSR and the CBCL are composed of a variety of components,including activities in which children participate as well as family and peerinteractions (Achenbach, 1991a, 1991b). These findings suggested thatcross-informant ratings of adolescent competence, in particular, deservefurther study.

EPSTEIN ET AL. 491

Ratings of interparental conflict, adolescent behavioral problems, andadolescent competence may be related to informant characteristics for sev-eral reasons. First, all family members provide their own unique individualperspectives of events that occur in their families (e.g., Achenbach et al.,1987). Second, all of the family informants may provide slightly differentinformation based on different experiences in their own families and on dif-ferential knowledge about family members. For example, adolescents maynot have witnessed all of the instances of interparental conflict in their fami-lies, which may result in differing reports across informants in the family(Grych et al., 1992). Finally, family members can witness the behaviors ofthemselves and other individuals in the family but are only privy to their owninternal states and perceptions. The added information from such internalstates may result in differences in cross-informant ratings of an individual’sown behavior versus that of other family members, especially in the case ofadolescent internalizing behavioral problems. As a result, using different sta-tistical procedures to demonstrate construct validity, the findings of thisstudy supported the continued use of multiple informants in the assessmentof family functioning. Only by considering the perceptions of multiplefamily members can clinicians and researchers reach a “gold standard” inassessment of families.

The results of this study also supported the continued use of new and inno-vative statistical methods to decipher the components of construct validity. Itshould be noted that each of the statistical methods used in this study has itsown weaknesses. For example, the analysis of MTMM matrices using Camp-bell and Fiske’s (1959) proposed criteria is somewhat subjective in nature(Bagozzi, 1993). Furthermore, as demonstrated in this study, the use of con-firmatory factor analysis for such data also can be problematic (Bagozzi,1993; Marsh, 1989; Marsh & Bailey, 1991). The correlated uniquenessmodel was used in this study to explore a latent variable approach because itrepresents the data more accurately when the effects of a particular methodare not unidimensional and because it results often in well-defined andproper solutions.

The correlated uniqueness model has its own limitations, however. Forexample, Byrne and Goffin (1993) noted that solutions of correlated unique-ness models typically yield trait loadings and correlations that are higher thanare those for a general factor analysis model. This characteristic of the corre-lated uniqueness model may limit our findings by providing stronger evi-dence of convergent validity and weaker evidence of discriminant validity.Another limitation to the correlated uniqueness model is that it requires thetheoretical factor pattern coefficients to be equal in order to eliminate a bias-ing effect. Although the theoretical factor pattern coefficients may be equalin magnitude for each of the latent constructs, the factor pattern coefficients


of the measured variables on their respective constructs were consistent butwere not equal in the model used in this study. This finding may have been aresult of the documented differences in cross-informant ratings (e.g.,Achenbach et al., 1987; Duhig et al., 2000). As a result, trait variance andtrait-trait covariance may have been overestimated, causing a distortion ofestimates of convergent and discriminant validity. Although the correlateduniqueness model has limitations, it has promise for encouraging researchersto refine and evaluate the understanding of these complex constructs (Brayet al., 1995).

The findings of this study also should be viewed within the context of itslimitations. Although a large sample of adolescent-mother-father triads wereused for this study, these families volunteered actively to participate in thisstudy. All families contacted the researchers in response to advertisementsdisplayed in the community and then attended a research session in a univer-sity setting. As a result, these families may have been particularly motivatedor willing to share information about themselves. Furthermore, the charac-teristics of this sample of families may reduce the generalizability of theresults of this study to other types of families. For example, the majority ofthese families reported that they were Caucasian and consisted of an intactmarital dyad. In addition, many of these mothers and fathers were experienc-ing varying degrees of psychological symptoms, perhaps making them dif-ferent from other mothers and fathers in the community. Finally, mothers andfathers were allowed to select which of their adolescents, if there were morethan one in the family, would participate. Consequently, the results describedin this study may not be representative of those of families with othercharacteristics.

In summary, this study highlighted the importance of using multiple infor-mants to evaluate interparental conflict, adolescent behavioral problems, andadolescent competence. Several statistical methods were used to examine theconstruct validity of the ratings of these variables. Although the convergentvalidity of the ratings of these constructs appeared adequate when relation-ships were examined using Campbell and Fiske’s (1959) criteria, discrim-inant validity was not supported strongly. The use of confirmatory factoranalysis was not helpful in the elucidation of the construct validity of the con-structs examined as a result of problems that occur commonly with the use ofthis procedure. Finally, analyses demonstrated adequate fit of the cross-informant ratings in a correlated uniqueness model in this sample of families,supporting convergent and discriminant validity. Overall, results suggestedthat measurement issues and informant effects should be examined in the rat-ings of interparental conflict, adolescent behavioral problems, and ado-lescent competence.

EPSTEIN ET AL. 493

References

Achenbach, T. M. (1991a). Manual for the Child Behavior Checklist/4-18 and 1991 profile.Burlington: University of Vermont Department of Psychiatry.

Achenbach, T. M. (1991b). Manual for the Youth Self-Report and 1991 profile. Burlington: Uni-versity of Vermont Department of Psychiatry.

Achenbach, T. M., McConaughy, S. T., & Howell, C. T. (1987). Child/adolescent behavioral andemotional problems. Implications of cross-informant correlations for situational specificity.Psychological Bulletin, 101, 213-232.

Bagozzi, R. P. (1993). Assessing construct validity in personality research: Applications to mea-sures of self-esteem. Journal of Research in Personality, 27, 49-87.

Bray, J. H., Maxwell, S. E., & Cole, D. (1995). Multivariate statistics for family psychologyresearch. Journal of Family Psychology, 9, 144-160.

Bryant, F. B. (2000). Assessing the validity of measurement. In L. G. Grimm & P. R. Yarnold(Eds.), Reading and understanding more multivariate statistics (pp. 99-146). Washington,DC: American Psychological Association.

Byrne, B. M., & Goffin, R. D. (1993). Modeling MTMM data from additive and multiplicativecovariance structures: An audit of construct validity concordance. Multivariate BehavioralResearch, 29, 67-96.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by themultitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.

Cook, W. L., & Goldstein, M. J. (1993). Multiple perspectives on family relationships: A latentvariables model. Child Development, 64, 1377-1388.

Cummings, E. M., & Davies, P. (1994). Children and marital conflict: The impact of family dis-pute and resolution. New York: Guilford.

Duhig, A. M., Renk, K., Epstein, M. K., & Phares, V. (2000). Interparental agreement on inter-nalizing, externalizing, and total behavior problems: A meta-analysis. Clinical Psychology:Science and Practice, 7, 435-453.

Emery, R. E., & O’Leary, K. D. (1982). Children’s perceptions of marital discord and behaviorproblems of boys and girls. Journal of Abnormal Child Psychology, 10, 11-24.

First, M. B., Gibbon, M., Williams, J. B. W., & Spitzer, R. L. (1995). Instruction manual for theMini-Structured Clinical Interview for DSM-IV (SCID) Version 2. Washington, DC: Ameri-can Psychiatric Press.

Fiske, D. W., & Campbell, D. T. (1992). Citations do not solve problems. Psychological Bulletin,112, 393-395.

Grych, J. H., & Fincham, F. D. (1990). Marital conflict and children’s adjustment: A cognitive-contextual framework. Psychological Bulletin, 108, 267-290.

Grych, J. H., Seid, M., & Fincham, F. D. (1992). Assessing marital conflict from the child’s per-spective: The Children’s Perception of Interparental Conflict Scale. Child Development, 63,558-572.

Jöreskog, K. G. (1974). Analyzing psychological data by structural analysis of covariance matri-ces. In R. C. Atkinson, D.H. Krantz, R. D. Luce, & P. Suppes (Eds.), Contemporary develop-ments in mathematical psychology (Volume 2, pp. 1-56). New York: Freeman.

Kenny, D. A. (1991). A general model of consensus and accuracy in interpersonal perception.Psychological Review, 98, 155-163.

Kenny, D. A., & Berman, J. A. (1980). Statistical approaches to the correction of correlation bias.Psychological Bulletin, 88, 288-295.

Kenny, D. A., & Kashy, D. A. (1992). Analysis of the multitrait-multimethod matrix by confir-matory factor analysis. Psychological Bulletin, 112, 165-172.

Kenrick, D. T., & Stringfield, D. O. (1980). Personality traits and the eye of the beholder: Cross-ing some traditional philosophical boundaries in the search for consistency in all of the peo-ple. Psychological Review, 87, 88-104.


Marsh, H. W. (1989). Confirmatory factor analyses of multitrait-multimethod data: Many prob-lems and a few solutions. Applied Psychological Measurement, 13, 335-361.

Marsh, H. W., & Bailey, M. (1991). Confirmatory factor analyses of multitrait-multimethoddata:A comparison of alternative models. Applied Psychological Measurement, 15, 47-70.

Noller, P., & Guthrie, D. (1991). Studying communication in marriage: An integration and criti-cal evaluation. Advances in Personal Relationships, 3, 37-73.

Porter, B., & O’Leary, K. D. (1980). Marital discord and childhood behavior problems. Journalof Abnormal Child Psychology, 8, 287-295.

Schumacker, R. E., & Lomax, R. G. (1996). A beginner’s guide to structural equation modeling.Hillsdale, NJ: Lawrence Erlbaum.

Stanger, C., & Lewis, M. (1993). Agreement among parents, teachers, and children on internaliz-ing and externalizing behavior problems. Journal of Clinical Child Psychology, 22, 107-115.

Sternberg, K. J., Lamb, M. E., & Dawud-Noursi, S. (1997). Using multiple informants and cross-cultural research to study the effects of domestic violence on developmentalpsychopathology: Illustrations from research in Israel. In S. S. Luthar, J. A. Burack,D. Cicchetti, & J. R. Weisz (Eds.), Developmental psychopathology: Perspectives on adjust-ment, risk, and disorder (pp. 417-436). New York: Cambridge University Press.

Wothke, W. (1996). Models for multitrait-multimethod matrix analysis. In G. A. Marcoulides &R. E. Schumaker (Eds.), Advanced structural equation modeling: Issues and applications(pp. 7-56). Hillsdale, NJ: Lawrence Erlbaum.

EPSTEIN ET AL. 495

interparental conflict, adolescent behavioral problems...

Documents