empirically based assessment of the behavioral/emotional problems...

Journal of Abnormal Child Psychology, Vol. 15, No.4, 1987, pp. 629-650

Empirically Based Assessment of the Behavioral/Emotional Problems of 2- and

3- Year-Old Children

Thomas M. Achenbach, 1'3 Craig Edelbrock, 2 and Catherine T. HowelP

The aim was to determine whether ratings of 2- and 3-year-olds couM yieM more differentiation among their behavioral~emotional problems than the internalizing-externalizing dichotomy found in previous studies. The 99-item Child Behavior Checklist for Ages 2-3 (CBCL/2-3) was designed to extend previously developed empirically based assessment procedures to 2-and 3-year-olds. Factor analyses of the CBCL/2-3 completed by parents of 398 2- and 3-year-olds yielded six syndromes having at least eight items loading >_ .30 and designated as Social Withdrawal, Depressed, Sleep Prob- lems, Somatic Problems, Aggressive, and Destructive. Second-order analyses showed that the first two were related to a broad-band internalizing grouping, whereas the last two were related to a broad-band externalizing grouping. Scales for the six syndromes, two broad-band groupings, and total problem score were constructed from scores obtained by 273 children in a general population sample. Mean test-retest reliability r was .87, 1-year stability r was .69, 1-year predictive r with CBCL/4-16 scales at age 4 was .63, 2-year predictive r was .55, and 3-year predictive r was .49. Children referred for mental health services scored significantly higher than nonreferred children on all scales. A lack of significant r's with the Min- nesota Child Development Inventory, Bayley, and McCarthy indicate that

Manuscript received in final form May 13, 1987. This research was supported by March of Dimes Birth Defects Foundation Grants 12-88 and 12-186, a Faculty Scholars Award from the W.T. Grant Foundation, and a Biomedical Re- search Support Grant from the University of Vermont College of Medicine.

1Department of Psychiatry, University of Vermont, Burlington, Vermont 05401. 2Department of Psychiatry, University of Massachusetts Medical School, Worcester, Massa- chusetts 01605.

3Address all correspondence, including requests for information on the Child Behavior Check- list, to T.M. Achenbach, Department of Psychiatry, University of Vermont, 1 South Pros- pect Street, Burlington, Vermont 05401,

629

0091-0627/87/1200-0629505.00/0 �9 1987 Plenum Publishing Corporation

630 Achenbach, Edelbrock, and Howell

the CBCL/2-3 taps behavioral~emotional problems independently of the developmental variance tapped by these measures.

In contrast to extensive efforts to standardize the assessment of behavioral/ emotional problems among older children (e.g., Achenbach & Edelbrock, 1983, 1986; Goyette, Conners, & Ulrich, 1978; Miller, 1981; Quay & Peter- son, 1983), few such efforts have been devoted to 2- and 3-year-olds. This may reflect not only a general paucity of research on psychopathology in the 2- to 3-year age period but also difficulties in defining criteria for de- viance and a lack of differentiated concepts of disorders for these ages.

One of the few efforts to standardize assessment of problems in this age range is the Behavior Screening Questionnaire (BSQ) developed in England by Richman and Graham (1971) for screening 3-year-olds. It consists of questions that are administered by trained interviewers to mothers whose reports about their children are scored on 3-step scales for 12 problem areas, such as eating, sleeping, activity, concentration, and fears. The BSQ has been used in a study of 705 London 3-year-olds who were reassess- ed at the age of 8 (Richman, Stevenson, & Graham, 1975, 1982), as well as in American (Cornely & Bromet, 1986; Earls, 1980) and Canadian studies (Minde & Minde, 1977).

A 19-item checklist version of the BSQ, called the Behavior Checklist (BCL), has been developed for mothers to complete independently (Rich- man, 1977). There is also a 22-item version-the Preschool Behavior Check- list (PBCL)-for completion by preschool teachers (McGuire & Richman, 1986).

Richman and Graham (1971) reported a test-retest r of .77 between two interviewers repeating the BSQ over a 1-week interval, while Richman (1977) reported a test-retest r of .81 for the BCL completed twice by mothers over a 4-week interval. Test-retest reliability has evidently not been published for the PBCL, but McGuire and Richman (1986) reported an r of .68 between ratings by a pair of nursery school staff members. Cutoff points on the three instruments were found to discriminate significantly between children who on other grounds were considered to be normal and those considered deviant. The instruments thus appear to fulfill their intended function of screening to identify children who have enough problems to warrant further investigation. Cluster analysis and factor analyses have also indicated some degree of differentiation among the patterns of problems tapped by these instruments, although the main distinctions were between different types of toilet problems in the cluster analyses and "con- duct/restless" versus "emotional/miserable" syndromes in the factor analyses (McGuire & Richman, 1986; Richman et al., 1982).

Empirically Based Assessment of 2- and 3-Year-Old Children 631

Other efforts have concentrated on ratings by preschool and day care staff. Behar and String field (1974) and Kohn and Rosman (1972), for exam- ple, developed checklists for rating 3- to 6-year-olds, while Crowther, Bond, and Rolf (1981) developed one for 2- to 5-year-olds. Data obtained with these checklists were factor-analyzed to yield two broad-band dimensions concerning aggressive, "externalizing" problems versus anxious, "internalizing" problems. Behar and Stringfield retained a third factor comprising four items with high loadings that they labeled as Hyperactive-Distractible. Test-retest correlations of the Behar and Kohn scales over periods of 3 to 6 months have ranged from .60 to .94, and they have shown significant discrimination between various clinical and nonclinical criterion groups (Behar & Stringfield, 1974; Kohn & Rosman, 1972, 1973). The Kohn and Crowther et al. scales have also shown significant correlations with other behavior rating scales completed several years later, when the subjects were of school age (Fischer, Rolf, Hasazi, & Cummings, 1984; Kohn, 1977).

Although not targeted specifically on the 2- to 3-year age range, the foregoing studies have yielded reliable and valid ratings of broad-band dimensions of behavioral/emotional problems among preschoolers. The relatively undifferentiated item pools and analytic methods, however, leave open the question of whether more differentiated syndromes of problems could be found among 2- and 3-year-olds. If so, how do the more differentiated syndromes compare with the numerous syndromes found in multivariate analyses of problems reported for older children? (For reviews, see Achenbach, 1985; Achenbach & Edelbrock, 1978; Quay, 1986.) How stable are syndromes during the early preschool period and how are they related to measures of development during this period?

To answer these questions, we extended empirically based procedures previously developed for assessing older children to 2- and 3-year-olds. Em- pirically based assessment of behavioral/emotional problems obtains reports of children's problems in a standardized format from informants such as parents, teachers, trained observers, interviewers, and the subjects themselves. As detailed by Achenbach and McConaughy (1987), empirically based assessment of children's behavioral/emotional problems embodies the following psychometric principles: (1) Assessment should employ standardized procedures. (2) Items should be aggregated to provide quantitative scores for each aspect of functioning. (3) Scores should be normed to indicate how an individual compares with representative samples of agemates. (4) To be considered psychometrically sound, assessment procedures must evince reliability and validity, although the types of reliability and validity vary with the type of procedure.

The present study was designed to develop a more differentiated empirically based procedure for assessing the behavioral/emotional problems

632 Achenbaeh, Edelbroek, and Howell

of 2- and 3-year-olds than done previously and to test its relations to other standardized measures during the preschool period. We employed parents' reports, because parents are the most intimately involved and universally available informants concerning the problems of 2- and 3-year-olds.

METHOD

Instrument Development

The assessment instrument, called the Child Behavior Checklist for Ages 2-3 (CBCL/2-3), was modeled on instruments for rating older children by parents, teachers, trained observers, interviewers, and the children themselves (summarized by Achenbach & McConaughy, 1987). It consists of 99 items describing behavioral/emotional problems that parents and parent-surrogates can report with a minimum of inference. Space is also provided for writing in additional problems that are not specifically listed. Examples of items include Aches or pains (without medical cause), Avoids looking others in the eye, Refuses to eat, and Unhappy, sad, or depressed. Respondents are instructed to rate the items as follows: "For each item that describes the child now or within thepast 2 months, please circle the 2 if the item is very true or often true of the child. Circle the 1 if the item is somewhat or sometimes true of the child. If the item is not true of the child, circle the 0."

The CBCL/2-3 requires only fifth-grade reading skills. If a respondent's reading skills are in doubt, an interviewer can hand the CBCL/2-3 to the respondent, give oral instructions, read the items aloud, and record the responses. Respondents who can read will generally begin answering the items spontaneously. Most respondents can complete the form and demographic data in less than 10 minutes.

The items were assembled as follows: 1. The Child Behavior Checklist for Ages 4-16 (CBCL/4-16; Achen-

bach & Edelbrock, 1983) was examined to find items appropriate for 2- and 3-year-olds, and the prevalence of these items in normative and clinical samples of 4- and 5-year-olds was determined (Achenbach & Edelbrock, 1981). This yielded 57 CBCL/4-16 items for inclusion in the CBCL/2-3, with slight changes of wording to adapt some of them for the younger ages.

2. Interviews with parents of toddlers were used to glean concerns that could be formulated as ratable items.

3. Previous studies were reviewed to identify additional problems reported for this age range (Behar & Stringfield, 1974; Crowther et al., 1981; Heinstein, 1969; Kohn & Rosman, 1972; Richman et al., 1975).

Empirically Based Assessment of 2- and 3-Year-Old Children

Samples

633

The CBCL/2-3 was used in three types of samples. Longitudinal Sample. Fifty-five low-birthweight and 32 full-term

children living in Vermont and northern New York participated in a longitudinal study that began when the children were born (Rauh, Achen- bach, Nurcombe, Howell, & Teti, in press). Parents completed the CBCL/2-3 within about 2 weeks of their child's second and third birthdays and the CBCL/4-16 within about 2 weeks of their child's fourth and fifth birthdays (all ages corrected for gestational age). They also completed the Minnesota Child Development Inventory (MCDI; Ireton & Thwing, 1974) and were in- terviewed at home at the age 2-4 assessments. Their children were tested with the Bayley (1969) scales at age 2 and the McCarthy (1972) scales at ages 3 and 4. Because some measures were unobtainable for some children, N's varied as reported in the Results.

Representative General Population Sample. A general population survey sample of 273 was obtained by sending interviewers to 218 randomly selected clusters comprising 30 households each in 34 residential census tracts of eight communities in the Worcester, Massachusetts, metropolitan area. The census tracts were stratified by income to yield a sample that was equally distributed across lower-, middle-, and upper-SES groups. Parents of 2- and 3-year-olds not receiving any clinical services were handed the CBCL and asked to respond to each item as the interviewer read the items and recorded the responses. If a household had more than one 2- or 3-year-old, one was selected by using a random number table. Of the households identified as having a 2- or 3-year-old, 3.8~ declined to complete the CBCL. An additional 5~ of the households either refused to provide any information on family composition or could not be reached in five call-backs. If we assume that the same 5.6~ of these households had toddlers as was found for the households successfully screened, this would add 5% • 5.6~ = .3~ to the target subjects on whom we failed to obtain completed CBCLs. Added to the 3.8~ of toddlers for whom parents declined to complete the CBCL, this produces a total of 4.1~ who may have been missed, for an estimated completion rate of 95.9~

Clinical Sample. A clinical sample consisted of 96 children referred to mental health services in California, Missouri, Utah, and Vermont. The main reasons for referral were behavioral/emotional problems and developmental delays of unknown origin.

Factor-Analytic Procedure

Following the procedures employed for the CBCL/4-16 (Achenbach & Edelbrock, 1983), we used principal components analysis with varimax


rotations to identify "syndromes," i.e., groups of cooccurring problems. To form a sample in which to identify syndromes, we randomly drew 198 CBCLs from the 273 randomly selected general population subjects. To these, we added all 96 children seen in mental health services, the 55 low- birthweight children, and the 32 full-term children in the longitudinal study described above. To increase the number of CBCLs for 2-year-olds considered to be at risk for behavior problems, we included 2-year CBCLs for 17 low-birthweight children whose 3-year CBCLS were also used. No other subjects were included more than once. We did not include the entire Massachusetts general population sample because we wished to maintain as much geographic diversity and as large a proportion of at-risk children as possible.

The factor-analytic sample of 398 had the following demographic characteristics: age- 38% 2-year-olds, 62% 3-year-olds; s e x - 50% of each sex; SES-Hollingshead (1975) 9-step rating of parental occupation mean = 4.9, SD = 2.5; ethnici ty-89% Anglo, 5% black, and 6% mixed or other. Mothers filled out 91% of the CBCLs, fathers 4%, and other informants (such as guardians) 4%.

The following five items were excluded from our factor analyses because they were reported for less than 5~ of the sample of 398: 39. Headaches (without medical cause); 51. Overweight; 57. Problems with eyes (without medical cause); 89. Underactive, slow moving, or lacks energy; 93. Vomiting, throwing up (without medical cause).

The remaining 94 items were subjected to a principal components analysis with varimax rotations of from 6 to 12 components. The rotations were compared to identify groups of items that consistently occurred together with loadings ~ .30 on a particular factor. Six syndromes were identified in this way. The first six factors in the seven-factor rotation were deemed to provide the most representative versions of these syndromes, in that they included the largest proportions of items found to cooccur on factors that appeared in multiple rotations. The sum of squared loadings (eigenvalues) ranged from 2.1 to 15.4 for the 6 unrotated components, while they ranged from 3.0 to 10.6 after rotation.

RESULTS

As found in our analyses of the CBCL/4-16 and its teacher and direct observational versions, the first factor consisted mainly of aggressive behaviors and had many more items with high loadings than the other factors. Because many of these items also loaded on other factors, we retained only those loading _> .40 on the first factor for the syndrome scale based on that factor. For the syndrome scales based on the remaining


five factors, we retained items loading _ .30. Items that loaded above the cutoff on more than one factor (___ .40 for the Aggressive factor, _> .30 for the other five) were retained on each of the syndrome scales for which they met this criterion. Although retention of these items on more than one scale raises the correlations between scales, it is intended to reflect the tendency of certain problems to covary with more than one syndrome, just as fever covaries with more than one type of physical illness. Table I lists ab- breviated statements of the items of each syndrome, with their loadings, sum of squared leadings, and descriptive titles for the syndromes.

Four of the six syndromes had rather clear counterparts in those identified for several age groups of each sex in parents' ratings on the CBCL/4-16 and were therefore given similar names. These included the Social Withdrawal, Depressed, Somatic Problems, and Aggressive syndromes. Although small groups of sleep problems constituted separate factors or grouped together with other types of problems at other ages, sleep problems formed a clearer syndrome at ages 2-3 than at the older ages. The Destructive syndrome found here lacked any clear counterpart in syndromes found at older ages.

Comparisons of Boys and Girls

To determine whether there were sex differences in the scale scores, we used t tests to compare the 133 boys and 140 girls from the general population sample on each of the six scales. Girls scored slightly higher on the Depressed scale, while boys score slightly higher on the other five scales. Because none of the differences was significant, however, we combined the sexes for assignment of standard scores, as described next.

Standard Scores for Syndrome Scales

To provide a standardized normative basis for scoring the syndrome scales, we summed the item scores for each syndrome on the 273 CBCLs obtained in the general population survey. The normative sample had the following demographic characteristics: age-54% 2-year-olds; sex-49%o male; SES-Hollingshead 9-step occupational rating mean = 4.9, SD = 2.7; ethnicity-87%o Anglo, 7% black, and 6% mixed or other. Mothers filled out 89% of the CBCLs, fathers 9%, and other informants 2%. Cumulative frequency distributions of the raw scale scores were formed for each of the six scales. The percentiles of the cumulative frequency distributions were used to assign normalized T scores using the following procedure, as done for the CBCL/4-16, Teacher's Report Form, and Youth Self-Report (Achenbach & Edelbrock, 1983, 1986, 1987):

Table I. Factor Loadings of Items on Syndrome Scales a

L Social withdrawal .55 73. Shy, timid .47 2. Acts too young .46 20. Clings to adults .46 4. Avoids eye contact .44 98. Withdrawn .42 76. Speech problem .39 3. Afraid to try new things .36 62. Refuses active games .35 68. Self-conscious .34 92. Upset by new people or situations .33 12. Constantly seeks help .32 70. Little affection .30 24. Doesn't eat well .30 37. Upset by separation

Sum of squared loadings = 3.6

II. Depressed .62 90. Sad .58 43. Looks unhappy .50 99. Worrying .49 62. Refuses active games .43 67. Unresponsive .42 98. Withdrawn .42 28. Doesn't want to go out .39 83. Sulks .38 82. Moody .36 70. Little affection .35 26. No fun .33 80. Strange behavior .32 25. Doesn't get along with other kids .31 77. Stares blankly .30 22. Disturbed by change


III. Sleep problems .68 94. Wakes often .66 38. Can ' t sleep .62 64. Resists bed .59 22. Doesn't want to sleep alone .58 84. Talks, cries in sleep .56 48. Nightmares .56 74. Sleeps little .32 50. Overtired


IV. Somatic Problems .48 45. Nausea .47 1. Aches .47 78. Stomachaches .39 19. Diarrhea .39 62. Refuses to eat .36 60. Rashes .35 24. Doesn't eat well .35 52. Painful b.m. .32 54. Picking .32 77. Stares blankly

Table I. Cont inued

.30 12. Constipated

.30 63. Rocks Sum of squared loadings = 3.0

V. Aggressive .64 85. Temper .63 15. Defiant .63 81. Stubborn .62 58. Punishment doesn't change behavior .61 16. Demands must be met .61 20. Disobedient .59 35. Fights .59 88. Uncooperative .58 40. Hits .57 13. Cries much .57 29. Easily frustrated .57 44. Angry moods .56 97. Whining .55 69. Selfish .54 8. Can' t wait .54 66. Screams .52 27. Lacks guilt .52 36. Gets into everything .51 91. Loud .50 59. Quickly shifts .50 82. Moody .47 5. Can' t concentrate .46 18. Destroys others ' things .45 6. Can' t sit still .45 17. Destroys own things .45 30. Jealous .45 53. Attacks people .45 83. Sulks .45 96. Wants attention .42 23. Doesn't answer .41 25. Doesn' t get along with other kids .40 34. Accident-prone


VI. Destructive .50 18. Destroys others ' things .47 17. Destroys own things .45 71. Little interest .44 14. Cruel to animals .44 31. Eats nonfood .44 75. Smears b .m. .37 41. Holds breath .37 42. Hurts accidentally .36 72. Little fear .35 70. Little affection .33 34. Accident-prone .33 67. Unresponsive to affection .32 5. Can' t concentrate .30 36. Gets into everything


aItems are designated with the numbers they bear on the CBCL/2-3 and summaries of their content.


1. To give all the scales a common T score starting point and to pre- vent overinterpretation of differences between low scores that are well within the normal range, we assigned a Tscore of 55 to all raw scores up to the 69th percentile. Tscores were assigned to raw scores from the 69thto the 98th percentile (T score = 70) according to the standard procedure, using the table provided by Abramowitz and Stegun (1968). Even though we trun- cated the lower ends of the scales for purposes of assigning Tscores, resear- chers desiring to preserve the full range of variation in scores can use raw scale scores for statistical analysis.

2. Because the distribution of raw scores became discontinuous beyond the 98th percentile (Tscore =- 70) on most scales and because percentiles above the 98th are so tenuous, we derived scores from 71 to 100 by di- viding the raw scores above the 98th percentile into 30 intervals to which T scores were then assigned (Achenbach & Edelbrock, 1983, provide detailed illustrations).

Second-Order Factor Analyses of Syndrome Scales

To determine whether the six syndrome scales formed broad-band groupings like those found in previous analyses of preschoolers and older children, we performed principal components analyses of the correlations between the T scores on the six scales for the 398 subjects used in the first- order factor analysis. To eliminate artifactual correlations between scales, items that loaded on more than one scale were counted only on the scale on which they loaded highest. The correlations between scales ranged f r o m . 16 for Sleep Problems with both Destructive and Social Withdrawal to .46 for Depressed with Social Withdrawal and .47 for Depressed with Aggressive.

A varimax rotation yielded loadings of .92 for the Social Withdrawal scale and .65 for the Depressed scale on one factor, and loadings of .84 for the Destructive and .74 for the Aggressive scale on the second factor. The remaining two scales had no loadings > .21 on either factor. This indicated two broad-band groupings like those derived from the CBCL/4-16 that we designated as Internalizing and Externalizing. They also resembled groupings labeled elsewhere as Overcontrolled versus Undercontrolled, Apathy- Withdrawal versus Anger-Defiance, and Anxious-Fearful versus Hostile- Aggressive (Achenbach & Edelbrock, 1978; Behar & Stringfield, 1974; Kohn & Rosman, 1972).

Standard Scores for Broad-Band Groupings and Total Score

To provide standard scores for the two broad-band groupings, we summed the items of the two Internalizing scales for each of the 273 sub-


jects in the normative sample to obtain a distribution of Internalizing scores, and likewise for the two Externalizing scales. Items that loaded on both Internalizing scales or both Externalizing scales were counted only once to obtain the total Internalizing or Externalizing score. However, the five items that loaded on at least one Internalizing scale and one Externaliz- ing scale were counted toward both the Internalizing and Externalizing scores.

Because t tests showed no significant sex differences, the percentiles of the cumulative frequency distributions for all subjects were used to assign T scores to the raw scores in the same way as done for the narrow-band scales, except that T scores were based directly on percentiles f rom the bo t tom of the distribution instead of assigning all scores up to the 69th percentile a T score of 55. This was done because the larger number of items on the Inter- nalizing and Externalizing scales produced less skewed distributions and less bunching up at a few very low scores. Furthermore, because of the large number of possible raw scores above the 98th percentile, we assigned a T score of 89 to the highest raw score actually found in our samples. The raw scores f rom the 98th percentile to the highest raw score were assigned T scores in equal intervals f rom 71 to 89. The raw scores above the highest actually found in our samples were then assigned T scores in equal intervals f rom 90 through 100. The same procedure was followed in assigning T scores to the total problem score, for which there were also no significant sex differences. (These procedures for assigning T scores to Internalizing, Externalizing, and total problems were the same as used for the C B C L / 4 - 16.)

The Child Behavior Profile for Ages 2-3

To enable users to see a child's overall pattern of scores and compare the child with normative samples of age-mates, we constructed the Child Behavior Profile for Ages 2-3, which is modeled on the profiles previously constructed for other versions of the CBCL. The narrow-band scales loading highest on the second-order Internalizing factor are listed on the left, while those loading highest on the second-order Externalizing factor are listed on the right. The two scales that did not load highly on either second-order factor are separated f rom the Internalizing and Externalizing scales in the middle of the profile.

Scores for each item are entered beside the items and are summed to obtain the total raw score for each scale. By marking the raw scale scores in the columns of the graphic display, the user can form a profile depicting the child's overall pattern of problems. Percentiles of the normative sample can be read f rom the left of the profile, while normalized T scores can be read


from the right. Following the convention adopted for the CBCL/4-16 (Achenbach & Edelbrock, 1983), a broken line across the graphic display marks the 98th percentile of the normative sample, which is used as a border between the normal and clinical range. Microcomputer programs are available to compute all scores and print them out in a profile format.

Test-Retest Refiability

Mothers of 20 children from the longitudinal sample described in the Method section and 41 from the general population sample completed the CBCL/2-3 at intervals averaging 7.7 days. As Table II shows, the test-retest r's ranged from .79 for the Destructive scale to .92 for the Sleep Problems scale, with a mean of .87, all significant at p < .001. (All mean r's reported in this paper were computed by z transformation.) Because r primarily reflects stability in rank ordering, we also tested the differences in mean scores from Time 1 to Time 2, using t tests. All except the Depressed scale showed small declines, with 7 being significant, as shown in Table II. The tendency for reported problems to decline over brief test-retest intervals has also been found in many other rating scales, as well as in interviews administered by trained interviewers (e.g., Achenbach & Edelbrock, 1983, 1986, 1987; Edelbrock, Costello, Dulcan, Kalas, & Conover, 1985; Evans, 1975; Richman & Graham, 1971). The mean decline in scores was 1.4 points across all scales, with the largest being 4.7 for total problems.

One- Year Stability

To assess longer-term stability, we computed r's and t tests between 2- and 3-year scores obtained by 73 children in the longitudinal sample described in the Method section. As Table II shows, the 1-year test-retest r's ranged from .56 for the Depressed scale to .76 for the Internalizing and total problems scales, with a mean of .69, all significant at p < .001. This is very close to the mean 1-year test-retest r of .72 found for the same children from age 4 to 5 on the corresponding scales of the CBCL/4-16.

Unlike shorter test-retest intervals, the 1-year test-retest comparisons did not show a consistent tendency for scores to decline. Instead, six scales showed increases, while three showed decreases. Only the increase on the Sleep Problems scale and the decrease on the Destructive scale were significant, however, both at p < .001, with the changes being 1.1 and 1.4 points, respectively. The mean change was .7 across all scales.

For comparison with a standardized developmental measure, we also computed test-retest r's for the scales of the MCDI completed by parents of


Table II. Test-Retest Reliability and 1-Year Stability

1 -week 1 -year reliability a stability b

Scale r r

Social withdrawal .87" .70 Depressed .85 .56 Sleep problems .92 .61 a Somatic problems ,86 c'e .59 Aggressive .87c .71 Destructive .79 c .7U 'e Internalizing ,88 c .76 Externalizing .88c .70 Total problems .91 c .76 Mean r .87 .69

~N = 61; all r's are significant at p < .001. bN = 73; all r's are significant at p < .001. CTime 1 > Time 2 by t test, p < .01. aTime 2 > Time 1 by t test, p < .01. eWhen correcting each column for the number ofp < .01 differences expected by chance in 9 analyses, these Time 1 versus Time 2 differences are the most likely to be chance, because they had the smallest t values.

62 children in the longi tud ina l sample over the same 1-year per iod f rom age

2 to 3. The r's ranged f rom .38 for the Personal-Social scale to .73 for the Comprehens ion -Concep tua l scale, with a mean of .59, somewhat lower than the 1-year mean r of .69 ob ta ined for the C B C L / 2 - 3 .

Because the M C D I is scored only in terms of raw scores that no rma l ly

increase with age, there was no poin t in testing the differences between

mean scores f rom year to year. However , the McCar thy Scales offered an oppor tun i ty to compare the 1-year test-retest stabili ty of the age-s tandard- ized scores of a directly adminis tered cognitive measure f rom age 3 to 4

with that of the CBCL from age 2 to 3. For 83 children in the longi tud ina l s tudy who received the McCar thy at ages 3 and 4, the r's for the s tandard scale scores ranged f rom .32 for the Motor scale to .83 for the General Cog- nitive Index, with a mean of .67, slightly lower than the 1-year mean of .69 for the C B C L / 2 - 3 . The McCar thy Moto r Scale also showed a s ignif icant increase in s tandard scores f rom age 3 to 4, t = 4.61, p < .001.

Discriminative Validity

Because few 2- and 3-year-olds receive menta l health services for behav io ra l / emo t iona l problems and so little research has been done on such problems at these ages, it is hard to make definit ive tests of discr iminat ive


validity. The present study was designed mainly to determine whether more differentiated patterns of problems could be found for this age group than had previously been reported. Our findings indicate that at least six syndromes warrant further study in 2- and 3-year-olds. Few children of this age have been independently distinguished according to these syndromes, however. Our main method for testing discriminative validity was therefore to compare scores for nonreferred children with those for children who on other grounds were deemed to need mental health services.

We tested the discriminative validity of the CBCL/2-3 scales by draw- ing from our general population normative sample 96 CBCLs that could be precisely matched for age and sex to CBCLs for our 96 children referred for mental health services. The two groups (total N = 192) were also matched as closely as possible for ethnicity and SES. Eleven children in each group were nonwhite, while 15 in the clinical sample lacked data on ethnicity. For purposes of analysis, they were coded as white. The mean Hollingshead SES score was 4.8 for the 96 nonreferred children and was also 4.8 for the 75 referred children who had sufficient parental occupational data for scoring SES. For purposes of analysis, the children lacking SES were scored at the overall mean of 4.8.

To control for possible associations with age, sex, ethnicity, and SES, we regressed the raw scores of each profile scale on referral status (nonreferred coded 0, referred coded 1), with the four demographic variables partial- led out. The results showed higher scores for referred children than for nonreferred children on all scales at p < .001. Table III displays these findings in terms of the percent of variance accounted for by referral status after the four demographic variables were partialed out. These effects ranged from 5~ for the Social Withdrawal scale to 32070 for the total problem score, which is a large effect according to Cohen's (1977) criteria.

Neither age nor ethnicity were significantly associated with any scale scores, although the effect of ethnicity may have been limited by the small number of nonwhites. Boys had higher scores than girls on Sleep Problems (3070 of variance) at p < .05. (Note that half the subjects in these analyses were clinically referred, whereas no significant sex differences were found for nonreferred children.) Lower SES children had significantly higher scores on all except the Sleep Problems and Aggressive scales, with p values ranging from .05 (2070 of variance) on the Somatic scale to < .01 (4070 of variance) on the Destructive and total problems scales. The largest demographic effects (4070) were thus smaller than the smallest effect of referral status (5070).

Although quantitative scoring of the scales is likely to maximize their informational value, we also tested the discriminative validity of categories based on cutoff points established at the 90th percentile of our normative


Table III. Percent of Variance Accounted for by Significant Effects of Referral Status, Sex,

and SES a

Referral Scale status Sex SES

Social withdrawal 5 - 3 Depressed 16 - 3 Sleep problems 10 3 b - Somatic problems 6 - 2 b Aggressive 17 -- - Destructive 9 -- 4 Internalizing 13 - 3 Externalizing 16 - 2 b Total problems 32 - 4

"N = 192. Each significant effect is what remained after partialling out the other listed variables, plus age and ethnicity, which had no significant effects.

bWhen correcting each column for the number of p < .05 findings expected by chance in analyses, these are the most likely to be chance effects, because they accounted for the least variance. All effects of referral status were significant at p < .001.

sample for total problems, In ternal iz ing, and External iz ing, and at the 98th

percentile for the six n a r r o w - b a n d scales. We did this by categorizing chi ldren as "normal" if they scored at or below the cu tof f and "clinical" if they scored above it. Regressions again showed significant effects (p < .01) of referral status with the four demographic variables part ialed out for all

scale scores. The effects of referral status in re la t ion to this d icho tomous categorizat ion were, however, somewhat smaller than in re la t ion to quan-

ti tative scoring, with effect sizes ranging f rom 4 to 24% for the categories versus 5 to 32% for quant i ta t ive scoring. Chi squares also showed that

s ignif icantly more referred than nonrefer red chi ldren scored above the cut-

offs (p < .01 for all scales).

Predictive Correlations with the CBCL/4-16 at Ages 4 and 5

To assess predictive power, we computed r's between the C B C L / 2 - 3 and the C B C L / 4 - 1 6 for the long i tud ina l sample described in the Method section. Table IV shows r's of the 2- and 3-year scores with the 4- and 5-year scores on the scales having counterpar ts on bo th ins t ruments , computed

separately for boys and girls because of the sex differences in the composi- t ion of scales on the age 4-5 CBCL profile. The mean of all the r's f rom age 2 to 4 and 3 to 4 was .65 for boys and .52 for girls. Across bo th sexes, the

Tab

le I

V.

Cor

rela

tion

s of

2-

and

3-Y

ear

Sco

res

wit

h C

orre

spon

ding

4-

and

5-Y

ear

Chi

ld B

ehav

ior

Pro

file

Sca

les ~

Ag

e 2-

3 sc

ales

Age

S

ocia

l D

e-

Som

a-

Agg

res-

In

tern

al-

Ext

erna

l-

4-5

wit

hdra

wal

pr

esse

d ti

c si

ve

izin

g iz

ing

Tot

al

scal

e 2

3 2

3 2

3 2

3 2

3 2

3 2

3

ix

Boy

s S

ocia

l 4

.38

.63

wit

hdra

wal

5

(.21

) .5

3 D

epre

ssed

4 5

Som

atic

4 5

Agg

ress

ive

4 5 In

tern

aliz

ing

4 5 E

xter

nali

zing

4 5

Tot

al

4 5 G

irls

S

ocia

l 4

(.32

) .3

7 w

ithd

raw

al

5 .3

4 .4

3 D

epre

ssed

4 5

Som

atic

4 5

Agg

ress

ive

4 5 In

tern

aliz

ing

4 5 E

xter

nali

zing

4 5

Tot

al

4 5

.45

.69

.48

.47

.36

.51

(.16

) .4

0

(.15

) (.

28)

.31

.44

.54

(.22

) .5

6 .4

3

.75

.86

.62

.66

.64

.60

.57

.53

.49

.75

.46

.61

.49

.53

.46

.60

.76

.84

.65

.67

.59

.58

.62

.59

.64

.84

.59

.70

.70

.66

.62

.65

g~

O

aSam

ple

size

s ra

nge

from

32

to 3

8 fo

r bo

ys a

nd f

rom

36

to 3

7 fo

r gi

rls.

All

r's

are

sig

nifi

cant

at

p <

.05

exce

pt t

hose

in

pare

nthe

ses.


mean predictive r's were as follows: 1-year r f rom age 3 to 4 = .63; 2-year r from age 2 to 4 = .52; 2-year r from age 3 to 5 = .56; 3-year r from age 2 to 5 = .49. For comparison across a 2-year period from one cognitive measure to another, the r between the Bayley Mental Scale and the McCarthy General Cognitive Index was .63 for the 86 children of both sexes having both measures, similar to the r of .65 between the 2-year and 4-year CBCL total scores for both sexes combined.

Interparent Agreement

Pearson f s and t tests were computed between scale scores obtained from CBCL ratings by mothers and fathers in the longitudinal sample at ages 2 (N = 61), 3 (N = 54), and 4 (N = 57). The mean interparent r across all scales was .47 at age 2, .57 at age 3, and .58 at age 4. This compares with an interparent r o f .57 for ratings of general population 3-year-old boys and .22 for girls on Richman and Graham's BSQ (Earls, 1980) and .73 for CBCL ratings of 4- and 5-year-olds referred for mental health services (Achenbach & Edelbrock, 1983). It is possible that agreement is generally higher for children with more problems than those of our longitudinal sample, whose mean total problem scores of 37.2 at age 2, 38.7 at age 3, and 26.5 at age 4 were close to the normative sample means of 40.7 at ages 2-3 and 24.7 at ages 4-5 (Achenbach & Edelbrock, 1983).

The interparent r's of .57 and .58 at ages 3 and 4 were comparable to the mean interparent r of .59 found in metaanalyses of 31 samples (Achen- bach, McConaughy, & Howell, 1987), but the r of .47 at age 2 suggests that interparent agreement may generally be poorer at this young age. Mothers tended to score all scales higher than fathers at age 2 (differences significant by t test at p < .05 on Aggressive, Externalizing, and total problem scales), probably reflecting mothers' greater involvement in their children's problems at age 2. The directions of the differences between parents' ratings were not consistent at ages 3 and 4. No differences were significant at age 3, and only 2 out of the 22 comparisons done separately for each sex on all the age 4 scales showed significant (p < .05) differences. This is less than the three .05 level findings that would be expected by chance in 22 comparisons, using a .01 protection level (Field & Armenakis, 1974; Sakoda, Cohen, & Beall, 1954).

Relations to Developmental Measures

The various forms of the CBCL are designed to obtain reports of behavioral/emotional problems by informants seeing a child at a particular

646 Aehenbach, Edelbroek, and Howell

age. Different forms are used for different age ranges and for different types of informants, such as parents, teachers, and observers. Age differences in the patterning and prevalence of problems are reflected in the factor-analytically derived scales and norms for each age group. The rating and scoring instruments are thus designed to take account of developmental differences in problems, but they are not intended as measures of development per se.

Whether children's scores or patterns of problems tend to change or remain stable as they develop is a separate issue from the empirically based assessment of their problems at a particular point in their development. Our longitudinal analyses indicated that the stability of parents' ratings of problems from age 2 to 4 was on a par with parents' MCDI ratings of developmental achievements and with Bayley and McCarthy cognitive test scores. But do the ratings of problems tap aspects of functioning other than those tapped by developmental measures such as the MCDI, Bayley, and McCarthy? To find out, we computed r's between these measures and the CBCL scores obtained at the same time in the longitudinal sample.

No concurrent r's between CBCL total problem scores and MCDI, Bayley, or McCarthy scores were significant at any of the three ages. This indicates that the CBCL as a whole is not merely reflecting the same variance as the developmental measures. However, the one CBCL problem scale that implies a developmental a s p e c t - t h e CBCL/4-16 Immature scale for boys aged 4 to 5 - d i d show significant (negative) r's with the MCDI General Development scale (r = - .40, p = .01) and Fine Motor Scale (r = - . 3 5 , p = .03). There were also 12 significant (positive) r's, ranging from .33 to .46, among the 24 r's between the 8 MCDI scales and the 3 CBCL/4-16 competence scales for 4-year-old boys, suggesting modest overlap between these measures of competence for boys of this age.

DISCUSSION

Our factor analyses of CBCLs for 398 2- and 3-year-olds showed that at least six syndromes of problems could be identified in parents' ratings. The Social Withdrawal, Depressed, Somatic Problems, and Aggressive syndromes had counterparts in syndromes found previously for older children, although the overall item pool and the precise composition of the syndromes differed somewhat from those for older children. Weak counterparts of the Sleep Problems syndrome have been found at older ages (Achenbach & Edelbrock, 1978), but the syndrome that we designated as Destructive did not appear to have clear analogues in other samples. Both these syndromes involve problems that are likely to be more common, salient, and closely intertwined at ages 2 to 3 than later. Our second-order


factor analyses yielded broad-band internalizing and externalizing groupings like those found in most other factor analyses of children's behavioral/ emotional problems.

Test-retest reliability for the narrow-band scales, broad-band groupings, and total score averaged .87, while 1-year stability averaged .69, which was on a par with the MCDI and McCarthy scales. Across a 2-year period, the predictive r was .65 between total problem scores from the CBCL/2-3 at age 2 to the CBCL/4-16 at age 4, similar to the r of .63 between the Bayley Mental Scale at age 2 and the McCarthy General Cognitive Index at age 4. Across a 3-year period, the predictive r was .61 between total problem scores from the CBCL/2-3 at age 2 to the CBCL/4-16 at age 5. Clinically referred children scored higher than nonreferred children on all scales, with minimal demographic effects. Test-retest reliablility, stability, and discriminative validity were thus satisfactory.

The independence of the CBCL/2-3 from aspects of development measured by the MCDI, Bayley, and McCarthy was shown by the lack of any significant correlations between the CBCL/2-3 scales and these measures. The Immature scale and competence scales of the CBCL/4-16 did show some significant r's with the MCDI for 4-year-old boys, however, indicating modest overlap with developmental ratings for boys after ages 2 to 3.

CONCLUSIONS

The instruments reviewed in the introduction were designed primarily for screening to identify preschoolers deviant enough to need further assessment. Previous cluster- and factor-analytic efforts to obtain greater differentiation have employed small item sets that yielded clusters of children distinguished largely by differences in toilet problems (Richman et al., 1982) and factors having few high loading items beyond the internalizing-externalizing dichotomy (McGuire & Richman, 1986).

Like other studies, our analyses yielded broad-band internalizing and externalizing groupings, but also showed that considerably more differentiation is possible for 2- and 3-year-olds in terms of six narrow-band syndromes. Two of these- the Social Withdrawal and Depressed syndromes- are related to the internalizing grouping, whereas two others- the Aggres- sive and Destructive syndromes--are related to the externalizing grouping. The remaining syndromes-Sleep Problems and Somatic Problems-are not clearly aligned with either broad-band grouping for 2- and 3-year-olds,

�9 although analogues of the Somatic Problems syndrome have been found to join the internalizing grouping at older ages (Achenbach & Edelbrock, 1983). Our findings do not necessarily exhaust the possibilities since inclu-


sion of very rare problems and extremely deviant children might have re- vealed additional syndromes.

The identification of six syndromes, provision of standard scores based on normative samples, and demonstration of reliability, stability, and discriminative validity advance the empirically based assessment of 2- and 3-year-olds' problems beyond screening. The similarity between the empirically based procedures for 2- and 3-year-olds and those for older children, as well as the significant correlations between the 2- and 3-year and 4- and 5-year syndrome scales, offers opportunities for more precise longitudinal studies of behavioral/emotional problems originating in the early years. The speed and economy of the procedures also make them practical for almost any study where parents or parent surrogates are available.

REFERENCES

Abramowitz, M., & Stegun, I. A. (1968). Handbook of mathematical functions. Washington, D.C.: National Bureau of Standards.

Achenbach, T. M. (1985). Assessment and taxonomy of child and adolescent psychopathology. Beverly Hills: Sage.

Achenbach, T. M., & Edelbrock, C. S. (1978). The classification of child psychopathology: A review and analysis of empirical efforts. Psychological Bulletin, 85, 1275-1301.

Achenbach, T. M., & Edelbrock, C. S. (1981). Behavioral problems and competencies reported by parents of normal and disturbed children aged four through sixteen. Monographs of the Society for Research in Child Development, 46, (Serial No. 188).

Achenbach, T. M., & Edelbrock, C. (1983)~ Manual for the Child Behavior Checklist and Re- vised Child Behavior Profile. Burlington: University of Vermont, Department of Psychiatry.

Achenbach, T. M., & Edelbrock, C. (1986). Manual for the Teacher's Report Form and Teacher Version of the Child Behavior Profile. Burlington: University of Vermont, Department of Psychiatry.

Achenbach, T. M., & Edelbrock, C. (1987). Manual for the Youth Self-Report and Profile. Burlington: University of Vermont, Department of Psychiatry.

Achenbach, T. M., & McConaughy, S. H. (1987). Empirically-based assessment of child and adolascent psychopathology: Practical applications. Newbury Park, CA: Sage.

Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situa- tional specificity. Psychological Bulletin, 101, 213-232.

Bayley, N. (1969). Barley Scales oflnfant Development. New York: Psychological Corpora- tion.

Behar, L. B., & Stringfield, S. (1974). A behavior rating scale for the preschool child. Deve- lopmental Psychology, 10, 601-610.

Cohen, J. (1977). Statisticalpower analysis for the behavioral sciences (rev. ed.). New York: Academic Press.

Cornely, P., & Bromet, E. (1986). Prevalence of behavior problems in three-year-old children living near Three Mile Island: A comparative analysis. Journal of Child Psychology and Psychiatry, 27, 489-498.

Crowther, J. H., Bond, L. A., & Rolf, J. E. (1981). The incidence, prevalence, and severity of behavior disorders among preschool-age children in day care. Journal of Abnormal Child Psychology, 9, 23-42.


Earls, F. (1980). The prevaience of behavior problems in 3-year-old children. Comparison of the reports of mothers and fathers. Journal of the American Academy of Child Psy- chiatry, 19, 439-452.

Edelbrock, C., Costello, A. J., Dulcan, M. K., Kalas, R., & Conover, N. C. (1985). Age differences in the reliability of the psychiatric interview of the child. ChiM Development, 56, 265-275.

Evans, W. R. (1975). The Behavior Problem Checklist. Data from an inner city population. Psychology in the Schools, 12, 301-303.

Fetid, H. S., & Armenakis, A. A. (1974). On use of multiple tests of significance in psychological research. Psychological Reports, 35, 427-431.

Fischer, M., Rolf, J. E., Hasazi, J. E., & Cummings, L. (1984). Follow-up of a preschool epidemiological sample: Cross-age continuities and predictions of later adjustment with internalizing and externalizing dimensions of behavior. Child Development, 55, 137- 150.

Goyette, C. H., Conners, C. K., & Ulrich, R. F. (1978). Normative data on revised Conners Parent and Teacher Rating Scales. Journal of Abnormal Child Psychology, 6, 221-236.

Heinstein, M. (1969). Behavior problems of young children in California. Berkeley: California Department of Public Health.

HoUingshead, A. B. (1975). Four Factor Index of Social Status, Unpublished manuscript, Yale University, Department of Sociology.

Ireton, H., & Thwing, E. J. (1974). Minnesota ChiM Development Inventory. Minneapolis: Behavior Science Systems.

Kohn, M. (1977). Social competence, symptoms, and underachievement in childhood: A longi- tudinalperspeetive. New York: Wiley.

Kohn, M., & Rosman, B. L. (1972). A social competence scale and symptom checklist for the preschool child: Factor dimensions, their cross-instrument generality, and longitudinal persistence. Developmental Psychology, 6, 430-444.

Kohn, M., & Rosman, B. L. (1973). A two-factor model of emotional disturbance in the young child: Validity and screening efficiency. Journal of ChiM Psychology and Psychiatry, 14, 31-56.

McCarthy, D. (1972). McCarthy Scales of Children's Abilities. New York: Psychological Cor- poration.

McGuire, J., & Richman, N. (1986). Screening for Behavior problems in nurseries: The reliability and validity of the Preschool Behavior Checklist. Journal of Child Psychology and Psychiatry, 27, 7-32.

Miller, L. C. (1981). Louisville Behavior Checklist Manual Los Angeles: Western Psycholo- gical Services.

Minde, R., & Minde, K. (1977). Behavioural screening of pre-school children: A new approach to mental health? In P. J. Graham (Ed.), Epidemiological approaches to child psychiatry (pp. 139-164). London: Academic Press.

Quay, H. C. (1986). Classification. In H. C. Quay & J. S. Werry (Eds.), Psychopathological disorders of childhood (3rd ed.). New York: Wiley.

Quay, H. C., & Peterson, D. R. (1983). lnterim Manual for the Revised Behavior Problem Checklist. Coral Gables, Florida: University of Miami, Applied Social Sciences.

Rauh, V., Achenbach, T. M., Nurcombe, B., Howell, C. T., & Teti, D. M. (in press). Minimi- zing adverse effects of low birthweight: Four-year results of an early intervention pro- gram. ChiM Development.

Richman, N. (1977). Is a behavior checklist for preschool children useful? In P. J. Graham (Ed.), Epidemiological approaches to child psychiatry (pp. 125-136). London: Acade- mic Press.

Richman, N., & Graham, P. (1971). A behavioral screening questionnaire for use with three year old children: Preliminary findings. Journal of Child Psychology and Psychiatry, 12, 5-33.

Richman, N., Stevenson, J., & Graham, P. J. (1975). Prevalence of behavior problems in three-year-old children: An epidemiological study in a London borough. Journal of Child Psychology and Psychiatry, 16, 222-287.


Richman, N., Stevenson, J., & Graham, P. J. (1982). Pre-school to schoo# A behavioral study. London and New York: Academic Press.

Sakoda, J. M., Cohen, B. H., & Beall, G. (1954). Test of significance for a series of statistical tests. Psychological Bulletin, 51, 172-175.

empirically based assessment of the behavioral/emotional problems...

Documents