J. ChildPsychol. Psychiat. Vol. 32, No. 3, pp. 501-513, 1991 0021-9630/91 $3.00 + 0.00Printed in Great Britain. Pergamon Press pic

© 1991 Association for Child Psychology and Psychiatry

Conditional Probabilities of Child InterviewSymptoms in the Diagnosis of Attention

Deficit Disorder

Steven Landau,* Richard Milich"^ and Thomas A. Widiger"^

Abstract—Based upon the standardized interviews of 76 clinic-referred boys, children's reportsof behaviors suggestive of attention deficit disorder (ADD) were examined to determinewhich symptoms were most efficient predictors of an ADD diagnosis. Unlike previousresearch that has focused exlusively upon the use of sensitivity and specificity rates, thisinvestigation £ilso employed the conditional probability rates of positive and negativepredictive power to establish the diagnostic utility of symptoms. As such, this study attemptedto establish the diagnostic efficiency of some ofthe child-reported symptoms and describesa new method to examine any criteria set that is to be used for classification purposes.Results indicated that the optimal child-reported predictors of an ADD diagnosis were notthose symptoms typically considered hallmark indicators ofthe disorder. In addition, nosymptom was found to be an efficient exclusion criterion for diagnosis. These findings arediscussed in terms ofthe role ofthe child as informant for the diagnosis of children's behaviordisorders and the need for multisource assessment.

Keywords: Child diagnosis, psychiatric interview, diagnostic efficiency

With the publication of DSM-III (American Psychiatric Association, 1980) and itsrecent updating, DSM-III-R (American Psychiatric Association, 1987), there hasbeen a flurry of research interest in the assessment and classification of child psychiatricdisorders, as many investigators are attempting to establish the reliability and validityof the various diagnostic criteria. This may be an especially important undertakingwhen working with children who present with symptoms suggestive of attention deficitdisorder (ADD), now known as attention-deficit hyperactivity disorder (ADHD) forseveral reasons. First, the perspective towards this disorder has dramatically changedduring this decade since the 1980 publication of DSM. Second, the ADHD symptomsare now, for the first time, considered in descending order of discriminatory power(i.e. the most useful item first), although the validity of this particular ranking is yetto be established. Lastly, there is much to suggest that the selection of treatment

Accepted manuscript received 29 May 1990

* Illinois State University, Illinois, U.S.A.^University of Kentucky, Kentucky, U.S.A.Requests for reprints to: Dr Steven Landau, Department of Psychology, Illinois State University, Normal,IL 61761, U.S.A.

501

502 Steven Landau et al.

(Sleator, 1982), response to treatment (Pelham & Bender, 1982), and long-termoutcome (August, Stewart & Holmes, 1983) are inextricably linked to the validityof this diagnosis. As such, it is important to establish the nature ofthe relationshipbetween the child's particular symptomatic behavior and an ADD designation.

Because the successful outcome of behavior disorder evaluation seems to be afunction of a multisource-multimethod perspective (Gresham, 1985), a criticalcomponent in the assessment process involves the appropriate selection of instrumentsand informants. Typically, clinicians have relied upon rating scale, psychometric,and direct observation data, plus interview reports from teachers and parents whenevaluating the ADD child (Barkley, 1985). One source that has not been seriouslyconsidered is the child as informant. In this case, a standardized psychiatric interview,designed to elicit information directly relevant to the ADD diagnostic criteria, couldbe administered directly to the child. In fact, there has been a recent growth of interestin the utility of structured child interviews as a means to establish a valid symptompicture (Hill, 1985). Much of this research compares the responses ofthe child withsome other significant informant to determine degree of correspondence in the reportof symptoms (Edelbrock, Costello, Dulcan, Conover & Kala, 1986; Herjanic & Reich,1982) or in terms of diagnosis (Bninshaw & Szatmari, 1988; Reich, Herjanic, Weiner& Gandhy, 1982). However, most of this research reports agreement data that areequivocal at best, thereby indicating that much disparity exists among informants.In addition, and possibly more important, this type of investigation lacks a 'goldstandard' (Robins, 1985), or veritable criterion diagnosis, as these studies are designedto consider each informant as an equal. As a result, they provide information thatactually addresses the reliability of interview schedules and, therefore, do not haveimplications for the validity or clinical utility of these interviews. Thus, the first purposeof this investigation was to determine the diagnostic utility of ADD symptoms derivedfrom a standardized child interview. This was accomplished by examining specificchOd-reported symptoms, phrased and organized consistent with the DSM-III (1980)diagnostic criteria for attention deficit disorder, and then determining which symptomsare most efficient in the prediction of the ADD diagnosis.

Most investigations of psychiatric classification have involved procedures that permitconvergent and discriminant comparisons among diagnosed groups regarding amultitude of non-diagnostic correlates. By so doing, these studies reveal the waysby which groups tend to differ once formed. Unfortunately, this type of research designdoes not address the predictive validity of the specific diagnostic criteria nor the mannerby which the symptoms relate to diagnosis. In addition, when these studies haveaddressed this question, the approach typically has been limited to a considerationof which symptoms were sensitive to the presence of the disorder and which werespecific to the disorder. Sensitivity is the proportion of persons with the disorder whohave the symptom, and specificity is the proportion of persons without the disorderwho do not have the symptom. As such, sensitivity represents the probability of asymptom given a disorder, or true-positive, whereas specificity indicates the probabilityof not having a symptom given the absence of a disorder, or true-negative (cf. Kline,1988). Even though these probability rates reveal important features about diagnosticgroup membership, they are not directly relevant to the type of diagnostic decisionrequired of the psychologist (Dawes, 1986; Widiger, Hurt, Frances, Clarkin &

Child interview symptoms 503

Gilmore, 1984). For example, sensitivity indicates the probability that a child witha particular diagnosis will present with an identified symptom, but the psychologistactually needs to know the probability that a child warrants a diagnosis given thathe/she presents with a particular symptom.

The statistics of greater relevance to the diagnostic decision are positive predictivepower (PPP) and negative predictive power (NPP). Sensitivity, specificity, PPP, andNPP are each derived from a two-by-two contingency table of hit rates, and eachcan be calculated from one another with the additional knowledge of the base ratesof the disorder and symptoms (see Fig. 1). However, each provides differentinformation regarding the relationship of a symptom to a disorder (Widiger et al, 1984).

Meets Does notdiagnostic meet

criteria criteria

Symptom present A E

Symptom absent C F

B D

Specificity =C/D = true negative rate Negative predictive power {NPP)=C/FSensitivity =A/B=true positive rate Positive predictive power ( PPP ) =A/E

Fig. 1. The relationship among symptoms, diagnosis, and the itemstatistics.

PPP indicates the utility of the symptom (S) as an inclusion criterion, as it is theconditional probability ofthe disorder (D) given the presence of a symptom [P(D|S)].NPP indicates the value ofthe symptom as an exclusion criterion, as it is the conditionalprobability of not having the disorder given the absence of a symptom [P(D|S)].Sensitivity does reveal some information regarding the symptom's efficiency as anexclusion criterion, as the symptom that is always present when the disorder is presentmay rule out the disorder by its absence (Meehl & Golden, 1982). Likewise, specificityprovides some mformation regarding the symptom's efficiency as an inclusion criterion,for the symptom that rarely is present when the disorder is absent may indicate thepresence of the disorder when that symptom is present (Meehl & Golden, 1982).However, as has recently been demonstrated by Milich, Widiger and Landau (1987),sensitivity and specificity are fallible as exclusion and inclusion statistics, respectively,and PPP and NPP are the statistics most relevant to the diagnostic decision requiredof the psychologist who must discover the important symptoms and then generatethe appropriate diagnosis. This is especially significant given that DSM-III-R nowranks the diagnostic criteria in terms of descending discriminatory power.

Specifically, Milich et al. (1987) examined the utility of these different probabilitystatistics, generated from maternal psychiatric interviews, in the differential diagnosisof attention deficit and conduct disorders for a heterogeneous sample of 76 clinic-referred boys. As part ofthe clinic evaluation, the mother of each boy was interviewedusing the parent version of the Diagnostic Interview for Children and Adolescents(DICA-P; Herjanic & Reich, 1982). Following these interviews, each maternal report

504 Steven Landau et al.

of her son's symptomatic behavior at home was examined to establish the relativeutility of each symptom in the prediction of either DSM-III diagnosis, with thesediagnoses derived from the maternal interview.

Milich et al. (1987) found that some ADD symptoms were more useful as inclusioncriteria while others were more useful as exclusion criteria. For example, the ADDsymptoms "can't sit still", "restless sleeper", "leaving games unfinished", and "runsaround" were the most efficient symptoms for identifying the presence of ADD.However, these symptoms occurred with relatively low frequency (base rates of 0.25,0.17, 0.28, and 0.26, respectively) and therefore did not identify a large proportionof ADD boys (sensitivity rates of 0.38, 0.25, 0.40, and 0.38, respectively). Nevertheless,they were quite specific to the disorder (specificity rates of 0.89, 0.92, 0.86, and 0.86,respectively) and represented the optimal inclusion criteria (PPP rates of 0.79, 0.77,0.76, and 0.75, respectively). Of particular interest was the fmding that the symptom"easily distracted", a hallmark characteristic of attention deficit disorder, served asthe best exclusion criterion for an ADD diagnosis. It occurred rather frequently (baserate of 0.80), was common to ADD boys (sensitivity rate of 0.95), and its absencestrongly suggested that attention deficit disorder was not present (NPP rate of 0.87).In other words, if the mother denied distractibility as being characteristic of her son,it is highly likely that he would not deserve an ADD diagnosis. Results also indicatedthat, according to maternal reports, none of the ADD symptoms was two-waypathognomic, that is, serving as both efficient inclusion and exclusion criteria. Thus,being free ofthe symptoms such as unable to sit still, restless sleeper, leaving gamesunfinished, and running around did not strongly suggest that ADD was absent asNPP rates ranged from 0.52 to 0.56. In addition, being described as "easily distracted"did not necessarily suggest that an ADD diagnosis was likely since the PPP rate wasonly 0.62.

The Bayesian perspective provided by these analyses is comparable in some respectsto the recent interest in applying receiver operating characteristics (ROC) analysesto psychiatric diagnosis (Hsiao, Bartko & Potter, 1989; Kraemer, 1988). ROC analysisalso involves an examination of the hit rate statistics in determining the optimalthreshold, indicator, or test for diagnosis. However, ROC analyses have involvedsensitivity and specificity rates and, as such, suffer from the same limitations previouslynoted for these statistics. In addition, ROC analysis is most relevant to an effort todetermine which test among many provides the optimal balance of sensitivity andspecificity, not which symptoms are optimal for inclusion tests and which are optimalas exclusion tests. However, an adaptation of ROC analysis that was based on positiveand negative predictive power statistics would be comparable to the Bayesian analysesin this paper.

The Milich et al. (1987) study is noteworthy in that it represents the first knownattempt to apply a Bayesian perspective to the child clinical classification process.By so doing, results demonstrated that symptoms that are most descriptive of attentiondeficit disorder (e.g. "easily distracted") may not be the most efficient in the diagnosisof that disorder. The present investigation is an extension of the earlier one, andexamines the utility of the child as informant. Thus, the first purpose of this studywas to examine the contribution of a standardized child psychiatric interview in thediagnosis of ADD. A second objective was to further describe an alternative method

Child interview symptoms 505

of assessing the diagnostic efficiency of specific criteria based, in this case, on thereferred child's self-reports. As such, this is the first known study to actually applythe Bayesian item statistics of PPP and NPP, in combination with sensitivity andspecificity rates, to symptoms from a child interview for the purpose of determiningwhich diagnostic criteria are most predictive of diagnosis.

MethodSubjects

Subjects were 76 boys who had been referred to an outpatient child psychiatry clinic and were seenagain at follow-up approximately 2 years later. The initial referral sample (A'̂ = 100) consisted ofconsecutively referred 6-12-year old boys who were not psychotic or retarded and included a varietyof academic and behavioral/psychiatric disorders. At referral, each child received a DSM-III diagnosisbased upon the findings of a multidisciplinary staff conference. Among the 100 boys, 24% receiveda diagnosis of attention deficit disorder alone; 20% a diagnosis of conduct disorder alone; 10% bothdiagnoses; and 46% deferred or another diagnosis such as depressed, learning disabled,oppositional/defiant. As such, this sample represented a heterogeneous distribution of behavior disorders(see Milich, Loney & Landau, 1982, for a description ofthe sample characteristics at referral).

Ofthe original 100 boys seen at referral, 17 families refused to participate in this follow-up investigationand another six families had left the clinic catchment area, resulting in a follow-up sample of 77. Oneboy was in the custody of the state at the time of follow-up, thereby making it impossible to obtaina maternal interview. The remaining 76 boys, who had a mean follow-up age of 11.9 years (S.D. = 1.7),had a mean Wechsler Intelligence Scale for Children—Revised (Wechsler, 1974) Full Scale IQof 98.8(S.D. = 11.4) and a mean Hollingshead and Redlich (1958) socioeconomic status (SES) score of 3.5(S.D. = 1.2).

ProcedureAs part of the follow-up evaluation, each boy was given the standardized Diagnostic Interview for

Children and Adolescents (DICA; see Herjanic & Reich, 1982, for a review of psychometric properties).This interview, which is a highly structured questionnaire, examines a wide variety of adaptive andmaladaptive functioning about the boy's relationships at home and school plus a survey of somatic andpsychiatric symptoms. Most ofthe questions provide for a "yes" or "no" response, with "yes" indicatingendorsement of a particular difficulty or symptom (see Herjanic & Reich, 1982, for a completedescription). For the purpose of this study, each boy was asked 19 items that relate directly to behaviorssuggestive of ADD (e.g. "Do you often start on your school work and not finish it, even if you knowhow to do it?") (see Table 1).

In addition, mothers of these 76 boys were interviewed with the parent version ofthe same questionnaire(DICA-P; Herjanic & Reich, 1982) as part of the clinic follow-up evaluation. These interview datawere then used to derive an independent diagnosis for each of the subjects and it was these diagnosesfrom the DICA-P that were used as the criterion measure in this study. The mothers were asked 31questions consisting of 15 pairs of items about home and school functioning plus one item about restlesssleep. However, because parents are not known to provide reliable reports of school-based problems(Hinshaw, 1987), an ADD diagnosis was determined by the 16 items representing the child's functioningat home. These items covered the 16 diagnostic criteria for a DSM-III diagnosis of ADD (i.e. five itemseach dealing with attention problems and overactivity and six items dealing with impulsivity).

Mothers' responses to the interview questions were initially coded into one of four categories: noproblem, minor problem, major problem in the past, and current major problem. However, becauseanalyses required dichotomous data, the first two categories were coded as symptom-free and the lattertwo as symptom-present. A boy received a diagnosis of ADD, with or without hyperactivity, in a mannerconsistent with the DSM-III criteria (e.g. at least three "yes" responses to the impulsivity items). Then,base rates, PPP, NPP, sensitivity, and specificity rates were determined for each ofthe boy's self-reportedsymptoms to determine which diagnostic criteria are most efficient in the prediction of an ADD diagnosis.

Steven Landau et al.

Results

Consistent with the procedures used by Costello, Edelbrock and Costello (1985)and Milich et al. (1987), this study did not make a distinction between the presenceversus absence of hyperactivity in the diagnosis of ADD. Ofthe 76 boys who servedas referred subjects, 40 boys earned a diagnosis of ADD, with or without otherdiagnoses. The resulting base rate of 0.53 may initially seem elevated but is consistentwith previously established rates from this regional tertiary outpatient psychiatricservice affiliated with a medical center (see Milich et al., 1982).

Table 1. Questions from DICA addressing attention deficit disorder

1. Are you the kind of person who tends to leave things unfinished, like not finishing a game or project?2. Do you often start on your school work and then not finish it, even if you know how to do it?3. Do your parents usually have to tell you something three or four times before you answer them

or do what they say?4. Has your teacher ever complained that you don't listen at school?5. Do your parents or teachers ever tell you that you don't pay close enough attention to the jobs

or tasks they ask you to do?6. Do you have trouble keeping your mind on things which you enjoy, such as reading a good story

or watching a TV program?7. Do you have trouble keeping your mind on an assignment at school? Even an interesting one?8. Do you usually get up sind leave the table before you are through eating, or get up and walk around

during a TV show (not just commercials)?9. Does your teacher complain that you don't stay in your seat when you are supposed to?

10. Are you the kind of person who tends to get into trouble, or maybe even hurt, because you rushinto doing things without thinking about what might happen?

11. Do you tend to rush from one activity to another, even if you haven't finished the first one?12. At school do you rush through assignments without stopping to see if you have done them correctly?13. Do you have trouble finding your clothes or books or equipment when it is time to go to school

or to an activity?14. At school do you frequently lose your paper or work books and pencils?15. Does your mother (or dad) often yell at you about something, and you don't know what it's about?16. At school does your teacher often have to tell you what you're supposed to do, after the rest of

the class has already started doing it?17. Do you get into trouble at school because you often speak out when you're supposed to be quiet?18. When playing games or lining up to go to class, do you often try to get in before your turn, or

push ahead in the line?19. Are you the kind of person who is always on the move, who can't sit still very well?

Table 2 presents the point-biserial correlations, base rates, and the sensitivity,specificity, PPP and NPP conditional probability rates for each ofthe child-reportedsymptoms in the prediction of the ADD diagnosis. The base rate was derived bydividing the number of boys with the specific symptom by the total number ofboys. As suggested by Fig. 1, sensitivity (SEN) was determined by the number ofcases with an ADD diagnosis and the specific symptom divided by the number ofcases who received an ADD diagnosis; specificity (SPE) was determined by the numberof boys without an ADD diagnosis who did not have the symptom divided by thenumber of boys who did not receive an ADD diagnosis; positive predictive power(PPP) was determined by the number of boys with the symptom who received anADD diagnosis divided by the number of boys with the symptom; and negative

Ghild interview symptoms 507

predictive power (NPP) was determined by the number of boys without the symptomwho did not receive an ADD diagnosis divided by the number of boys without thesymptom.

Table 2. Conditional probabilities and base rates for tte ADD symptoms (N = 76)

Symptom

Leaves games unfinishedDoesn't finish schoolworkParents must repeat commandsDoesn't listen in schoolDoesn't pay attentionCan't attend to fun activitiesTrouble attending at schoolLeaves meal/TV before finishedTeacher complains out-of-seatRush about without thinkingChange activities without finishingRush through school assignmentsTrouble finding thingsLose books and papersParents yell—don't know whyTeacher must repeat instructionsTalk at school when shouldn'tPush ahead in line/can't wait turnCan't sit still

MS.D.

BR

0.160.250.370.240.300.280.300.070.160.280.220.280.180.200.360.280.280.090.470.250.10

SEN

0.150.250.380.300.350.350.350.100.230.350.250.300.200.230.480.350.300.150.530.290.11

SPE

0.830.750.640.830.750.810.750.970.920.810.810.750.830.830.780.810.750.970.580.800.10

PPP

0.500.530.540.670.610.670.610.800.750.670.590.570.570.600.700.670.570.860.580.630.09

NPP

0.470.470.480.520.510.530.510.490.520.530.490.490.480.490.570.530.490.510.530.510.03

PB

0.050.130.27**0.27**0.44***0.28**0.33**0.34***0.40***0.55***0.28**0.46***0.40***0.42***0.39***0.57***0.29**0.21*0.39***

Note: ADD = attention deficit disorder; BR = base rate; SEN = sensitivity; SPE = specificity;PPP = positive predictive power; NPP = negative predictive power; PB = correlation of symptom withtotal symptom score after removal of that symptom.

*p<0.05; **p<O.Ol; ***/>< 0.001.

The point-biserial correlation represents an item's correlation with the total itemscore after that item has been removed from the symptom list. As such, it is a measureof each item's cohesiveness with the child's total symptom picture. Two items (i.e."leaves games unfinished" and "doesn't finish schoolwork") were found to havenonsignificant correlations, thereby indicating that boys were endorsing these in amanner inconsistent with the other ADD symptoms. These two items also had thelowest PPP rates (0.50 and 0.53, respectively), indicating that their utility in theidentification of ADD would be questionable for this sample of boys.

Examination of the base rates for all symptoms reveals a mean base rate of 0.25(S.D. = 0.10) with a range of 0.09 ("pushes ahead in line") to 0.36 ("parents ye l l -don't know why"). When contrasted with the mean base rate of 0.50 for mother-reported ADD symptoms for this sample described by Milich et al., (1987), it is evidentthat, on average, these boys were admitting to only one-half the symptoms thattheir mothers were willing to attribute to them. However, this discrepancy is consistentwith earlier research using the DICA (Brunshaw & Szatmari, 1988; Herjanic & Reich,1982).

Steven Landau et al.

To establish the specific rates of diagnostic efficiency in terms of inclusion andexclusion criteria, PPP and NPP rates were examined. The ADD symptoms obtaineda mean PPP rate of 0.63 (S.D. =0.09) in identifying an ADD diagnosis, whereasthe mean NPP rate was 0.51 (S.D. = 0.03). Those items that clearly seemed the mostefficient inclusion indicators of an ADD diagnosis were "leaves meal/TV beforefmished", "teacher complains about out-of-seat", "pushes ahead in line", and"parents yell—don't know why" (PPP rates of 0.80, 0.75, 0.86, and 0.70,respectively). However, the first three symptoms had very low sensitivity rates andwere able to identify only 10%, 23%, and 15% ofthe ADD cases, respectively. Assuch, their usefulness in the classification of ADD may be quite limited. In contrast,the "parents yell" item may be the best symptom because it had a high PPP rateand would identify 48% ofthe cases. Results also demonstrated that the boy's failureto admit to these symptoms would seem to have no implication for excluding thedisorder, as NPP rates for these items (ranging from 0.49 to 0.52) reveal an equallikelihood of an ADD or no diagnosis and a probability of no diagnosis that is nogreater than predicting on the basis ofthe base rate alone (0.47). In fact, the NPPrates for all ofthe symptoms indicate that, for this sample of boys, no symptom appearsto be an efficient exclusion criterion for an ADD diagnosis.

Examination of the other ADD symptoms also reveals interesting informationregarding the diagnostic utility of symptoms. For example, the item "can't sit still",which was admitted by 36 ofthe 76 boys (BR = 0.47), was by far the most frequentsymptom endorsed by this sample. It had the largest sensitivity rate (0.53), indicatingthat it was characteristic of 21 of the 40 diagnosed ADD children. However, theprobability that a boy would be identified as ADD given that he admitted to thisproblem was only 0.58. In contrast, when mother reports of "can't sit still" wereexamined by Milich et ai (1987) for this same sample of boys, this was found to bethe most efficient inclusion item of all in the symptom list (PPP of 0.79).

Discussion

This investigation had two objectives. The first was to examine the utility of astandardized child interview in the diagnosis of attention deficit disorder. As such,an attempt was made to determine if the child client is, indeed, a reliable informantregarding those symptoms that serve as diagnostic criteria for ADD classification.The second purpose was to describe an alternative method of examining the diagnosticefficiency of symptoms used for classification, and this may serve as a model for futureresearch to assess any criteria set generated to classify children. By the use of positiveand negative predictive power item statistics, and comparing these with the morefrequently employed sensitivity and specificity rates, it would be possible to determinewhich symptoms are best used as inclusion criteria, and which are best as exclusioncriteria, for the diagnosis. In this context, the present study represents the first knownattempt to apply these probability statistics to symptoms derived directly from clinic-referred children's interview reports of their own symptomatic behavior. Previousresearch in this area has addressed the behavioral correlates of ADD group membershipby establishing the prevalence of relevant symptoms once the diagnosis has been

Child interview symptoms 509

derived. Even though this is important as a means to establish the descriptive validityof childhood disorders, this particular use of sensitivity and specificity rates has littleutility for the practitioner who must first identify a symptom and then derive a diagnosis(Dawes, 1986). In addition, the fact that DSM-III-R presents criteria in rank orderof discriminatory power indicates that researchers should employ a method to quantifythe relative efficiency of each symptom in a criteria set.

Results of this study indicated that all but two of the ADD symptoms had significantpoint-biserial correlations with the total symptom score. Even though these correlationsdo not have direct implication for the diagnostic utility of symptoms, many of thesymptoms that showed the greatest covariation with the disorder (e.g. "teacher mustrepeat instructions") were clearly not the most useful in the prediction ofthe ADDdiagnosis (i.e. PPP = 0.67). In addition, it was surprising to note that two hallmarkbehavioral characteristics of inattention (i.e. "leaves games unfinished" and "doesn'tfinish schoolwork") were actually found to be unrelated to the child's total ADDsymptom score. Apparently, boys responded to these two items in a mannerinconsistent with their endorsement of other ADD symptoms. As such, the specificcontribution that these symptoms can make to the diagnostic process may be ofquestionable utility when interviewing children.

Results also indicated that three of the four symptoms having the highest PPP rateswere qualified by low sensitivity rates. Even though these items may be highlypredictive of an ADD diagnosis when endorsed, they are not descriptive of manyADD boys. Specifically, the fact that the boy admits to problems regarding leavingmeals or TV prematurely, teacher complaints about being out-of-seat, and pushingahead in line would respectively identify only 10%, 23% and 15% of the ADD subjects.In contrast, the boy's admission that his parents yell at him was both predictive ofdiagnosis and would characterize 48% of those so diagnosed. As such, the item"parents yell—don't know why" may be the most efficient inclusion symptom, atleast for this local clinic setting.

The finding regarding this particular item is interesting especially given the factthat the item does not directly represent one of the established symptoms of attentiondeficit disorder. Instead, the efficiency of this item simply indicates that the motherand son agree that the parent is upset about the boy's behavior. Both referral statusand diagnosis are evidence of parental concern, and the son's endorsement of thisitem reveals his perception of negative, albeit inexplicable, pcirental feedback. However,when one examines the specific ADD symptoms, the present findings do not suggestthat mothers and sons show correspondence to the same degree.

When addressing the issue of efficient exclusion criteria, one should identify thosesymptoms that occur in almost all cases of the disorder so that their absence is highlysuggestive ofthe absence ofthe disorder (Meehl & Golden, 1982). Unfortunately,all NPP values in this study suggest that the probability ofthe absence ofthe diagnosis,given the absence ofthe symptom, is approximately equal for all ofthe ADD diagnosticcriteria. As a result, the fact that these boys denied the existence of a symptom mayhave no implication for ruling out the presence of the disorder.

It is important to acknowledge that both PPP and NPP rates are partially dependenton local setting base rates for symptoms and diagnoses. It has been suggested thatthis phenomenon introduces unreliability to diagnostic efficiency fmdings, and that

510 Steven Landau et al.

sensitivity and specificity are the statistics of choice because they should be independentof base rates. One would expect that the proportion of diagnosed ADD children who,for example, have trouble staying seated (i.e. sensitivity) would be the same irrespectiveof clinic location or setting, although the probability of an ADD diagnosis given thatout-of-seat symptom (i.e. PPP) may differ as a function of local diagnostic practices(Milich et ai, 1987). In fact, there is evidence to indicate that sensitivity and specificityrates fail to remain stable across settings (Robins, 1985). Therefore, differences inlocal base rates should be considered a desirable feature of this type of research asthey encourage the practitioner to recognize that diagnostic decisions are made inthe context of local setting practices (Meehl & Rosen, 1955). Indeed, it is an acceptedand implicit practice in medicine to consider the local prevalence and populationswhen making diagnoses. A fever will be suggestive of different disorders dependingon the prevalence ofthe disorders at that particular time or setting (e.g. during anepidemic, at an elementary school versus a drug rehabilitation center). Similarly,a disorder that is common in a particular setting (ADD at our regional tertiary clinic)is not readily ruled out by the denial of these symptoms by children (i.e. NPP ratesare low). But, if the setting was one in which the disorder was in fact rare, then themore difficult task would be to identify its presence than its absence. Specifically,the failure to assess the effect of local base rates would obscure accurate diagnosis(Dawes, 1986).

One finding from the current study that was characteristically different for the dataobtained from the parents by Milich et ai (1987) was the relatively infrequentendorsement by the children of ADD symptoms. Indeed, source of information (parentversus teacher versus child) is another factor that will greatly influence the base rateof symptoms. Recognition of this fact strongly suggests that the clinician must notapply the same diagnostic decision rules to data from different sources. As indicatedin Table 2, the average likelihood that these boys would admit to each of the ADDdiagnostic criteria was 0.25, even though the probability for an ADD diagnosis was0.53. This stands in contrast to the average symptom base rate of 0.50 from motherswho described this same sample of boys (Milich et ai, 1987). Thus, these referredchildren were unwilling or unable to admit to as many symptoms as their motherswere willing to attribute to them. It is this low base rate for symptoms that accountsfor the low NPP values. As a result, one is unable to specify which symptoms, ifdenied, should serve to exclude consideration of an ADD diagnosis. The finding thatADD children appear to deny their symptomatology suggests that data obtained fromparent interviews may be more useful than data from child interviews in ruling outthe presence of ADD. The one exception to this trend was "can't sit still", whichwas acknowledged by 47 % ofthe boys. However, this still occurred in only half (53 %)ofthe ADD children and was not specific to them. The item then fails to serve eitheras a useful inclusion or exclusion criterion. Consistent with what is known aboutchildren's self-reports (see Landau & Milich, 1990), the admission of difficulties orproblems may or may not have implications for establishing the presence of a disorder;the denial of problems by the child seems to have no implications for ruling out thedisorder.

This phenomenon is not unexpected and should be considered whenever dealingwith the data that result from child interviews. For example, it has been established

Child interview symptoms 511

that parents and their children tend to show better correspondence when describingexternalizing, undercontroUed symptoms as opposed to internalizing, overcontrolledbehaviors (Herjanic & Reich, 1982). However, when examining the nature ofdiscrepancies, Edelbrock et ai (1986) found that parent interviews yielded significantlyhigher symptom scores for conduct and behavior problems whereas child interviewsyielded significantly higher symptom scores for affective, internalizing problems. Insummary, even though there may be better agreement between parent and childregarding overt and observable behavior problems, such as those examined by thisinvestigation, absolute level of agreement is not a sufficient consideration. Instead,the psychologist should also consider the direction of disagreement as a function ofpresenting problems. Consistent with other research, this study found that, in contrastwith the child's self-report, parents overstated the presence of ADD behaviors.

A second explanation for the reported low level of agreement between mother andson may be due, in part, to the fact that the mother was describing home-basedsymptoms whereas the son's report could pertain to problems at home or school. Infact, this phenomenon poses a serious cheJlenge for any diagnostician who mustreconcile different values and experiences of different informants who provide interviewdata for classification. Even though this problem serves as a limitation ofthe presentinvestigation, it points to the need to compare the utility or hit rate of symptomsin different settings, as these rates are likely to vary as a function of different situationalfactors and different respondent perspectives. As such, it may be necessary to employdifferent diagnostic criteria in different settings with different informants to generatethe most valid classification. An examination of PPP and NPP hit rates, as a functionof source or setting, will permit such a determination.

A second purpose of this investigation was to offer a heuristic description of howPPP and NPP procedures can be employed when attempting to address theclassification of childhood disorders. As such, these techniques may be applicablewhenever there is a need to validate diagnostic or placement criteria. Regarding specificchild psychiatric disorders, the PPP and NPP procedures are particularly suitableto examine the diagnostic efficiency of DSM-III-R criteria for the diagnosis of ADHD(American Psychiatric Association, 1987). Unlike its predecessor, this version ofthetaxonomy specifies a set of 14 ADHD criteria, eight or more of which are necessaryfor diagnosis. As stated previously, this polythetic format places these criteria indescending order of discriminatory power even though each symptom is equallyweighted for diagnosis. However, it is yet to be demonstrated which symptoms shouldbe given greater weight when considering diagnostic decisions. The proceduresdescribed by the present investigation are ideally suited for that type of validationstudy, and the findings suggest that the symptoms acknowledged by referred boysthat are most useful in identifying the presence of ADD are, for example, "leavesmeal/TV before finished" and "parents yell—don't know why". In addition, theDSM-III-R criteria only address inclusion criteria and make no provision for exclusioncriteria. This study failed to identify efficient exclusion symptoms due to the generaltendency of boys to deny the symptomatology. Milich et ai (1987), however, did findthat some ADD symptoms were actually more useful for ruling out other disordersthan ruling in ADD.

The results of this investigation are considered preliminary for several reasons and.

512 Steven Landau et al.

thus, require replication. For one, the value of these findings may be more heuristicthan practical, as they are based on the DSM-III symptom list instead of thosediagnostic criteria reflected by DSM-III-R. Second, the obtained results weredependent upon diagnoses derived from maternal interview data. If diagnoses weredetermined from a different source and/or method, one could expect that differentsymptoms might emerge as more efficient inclusion or exclusion criteria for diagnosis.Third, since this study involved an examination of child-reported symptoms in theprediction of mother-derived diagnosis, this investigation may tell us as much aboutmother-son agreement as it does about diagnostic efficiency of symptoms. Futureresearch that examines the PPP and NPP statistics will permit further considerationof these questions.

Finally, subsequent work with the PPP and NPP statistics should also examinethe diagnostic efficiency of several symptoms in combination, as combinations ofsymptoms tend to operate differently as inclusion and exclusion criteria than singlesymptoms (Clarkin, Widiger, Frances, Hurt & Gilmore, 1983). However, becausethe number of subjects who present with specific symptom combinations is markedlyless than the total number of subjects in the sample, a larger sample than was availablefor the present investigation would be indicated for this type of analysis.

References

American Psychiatric Association (1980). Diagnostic and statistical manual of mental disorders (3rd edn).Washington, DC: American Psychiatric Association.

American Psychiatric Association (1987). Diagnostic and statistical manual of mental disorders (3rd edn, revised).Washington, DC: American Psychiatric Association.

August, G., Stewart, M. & Holmes, C. (1983). A four-year follow-up of hyperactive boys with andwithout conduct disorder. British Journal of Psychiatry, 143, 192-198.

Barkley, R. A. (1985). Attention deficit disorders. In P. H. Bornstein & A. E. Kazdin (Eds), Handbookof clinical behavior therapy with children (pp. 158-217). Homewood, IL: Dorsey Press.

Brunshaw, J. M. & Szatmari, P. (1988). The agreement between behaviour checklists and structuredpsychiatric interviews for children. Canadian Journal of Psychiatry, 33, 474-481.

Clarkin, J . E., Widiger, T. A., Erances, A., Hurt, S. W. & Gilmore, M. (1983). Prototypic typologyand the borderline personality disorder. Journal of Abnormal Psychology, 92, 263-275.

Costello, E. J., Edelbrock, C. S. & Costello, A. J. (1985). Validity ofthe NIMH diagnostic interviewschedule for children: a comparison between psychiatric and pediatric referrals, yowrna/ of AbnormalChild Psychology, 13, 579-595.

Dawes, R. M. (1986). Representative thinking in clinical judgment. Clinical Psychology Review, 6, 425-441.Edelbrock, C , Costello, A. J. , Dulcan, M. K., Conover, N. C. & Kala, R. (1986). Parent-child

agreement on child psychiatric symptoms assessed via structured interview, yourraa/ of Child Psychologyand Psychiatry, 27, 181-190.

Gresham, E. M. (1985). Behavior disorder assessment: conceptual, definitional, and practicalconsiderations. School Psychology Review, 14, 495-509.

Herjanic, B. & Reich, W. (1982). Development of a structured psychiatric interview for children:agreement between child and parent on individual symptoms. Journal of Abnormal Child Psychology,10, 307-324.

Hill, P. (1985). The diagnostic interview with the individual child. In M. Rutter & L. Hersov (Eds),Child and adolescent psychiatry: modem approaches (pp. 249-263). London: Blackwell.

Hinshaw, S. P. (1987). On the distinction between attention deficits/hyperactivity and conductproblems/aggression in child psychopathology. Psychological Bulletin, 101, 443-463.

Hollingshead, A. & Redlich, E. (1958). Social class and mental illness. New York: Wiley.

Child interview symptoms 513

Hsiao, J. K., Bartko, J . J. & Potter, W. Z. (1989). Diagnosing diagnoses. Receiver operatingcharacteristic methods and psychiatry. Archives of General Psychiatry, 46, 664-667.

Kline, R. B. (1988). Methodological considerations in the evaluation ofthe convergence of psychiatricdiagnoses and parent-informant checklists, youraa/ of Abnormal Child Psychology, 16, 289-298.

Kraemer, H. (1988). Assessment of 2 x 2 associations: generalization of signal-detection methodology.American Statistician, 42, 37-49.

Landau, S. & Milich, R. (1990). Assessment of social status and social competence. In A. M. La Greca(Ed.), Through the eyes of a child: obtaining self reports from children and adolescents (pp. 259-291). NewYork: Allyn & Bacon.

Meehl, P. E. & Golden, R. R. (1982). Taxometric methods. In P. C. Kendall & J. N. Butcher (Eds),Handbook of research methods in clinical psychology (pp. 127-181). New York: Wiley.

Meehl, P. E. & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs,patterns, or cutting scores. Psychological Bulletin, 52, 194-216.

Milich, R., Loney, J. & Landau, S. (1982). Independent dimensions of hyperactivity and aggression:validation with playroom observation data. Journal of Abnormal Psychology, 91, 183-198.

Milich, R., Widiger, T. A. & Landau, S. (1987). Differential diagnosis of attention deficit and conductdisorders using conditional probabilities, yourna/ of Consulting and Clinical Psychology, 55, 762-767.

Pelham, W. E. & Bender, M. E. (1982). Peer relations in hyperactive children: description and treatment.In K. Gadow & I. Bialer (Eds), Advances in leaming arid behavioral disabilities (Vol. 1, pp. 365-436).Greenwich, CT: JAI Press.

Reich, W., Herjanic, B., Weiner, Z. & Gandhy, P. R. (1982). Development of a structured psychiatricinterview for children: agreement on diagnosis comparing child and parent interview. Journal ofAbnormal Child Psychology, 10, 325-336.

Robins, L. N. (1985). Epidemiology: refiections on testing the validity of psychiatric interviews. Archivesof General Psychiatry, 42, 918-924.

Sleator, E. (1982). Office diagnosis of hyperactivity by the physician. In K. D. Gadow & I. Bialer(Eds), Advances in learning and behavior disabilities (Vol. 1, pp. 341-364). Greenwich, CT: JAI Press.

Wechsler D. (1974). Wechsler intelligence scale for children—revised. New York: Psychological Corporation.Widiger, T. A., Hurt, S. W., Erances, A., Clarkin, J. E. & Gilmore, M. (1984). Diagnostic efficiency

and DSM-III . Archives of General Psychiatry, 4 1 , 1005-1012.