Cafl(u/i(1II Jrmrlwl of Scll rJol 1'5),(110108)' V"lumt' 15, Numbu I, 1999, 1/ -20

Diagnostic Utility of WIse-III Subtest Variability Among Students with Learning Disabilities

Marley W Watki1ls Th e Pennsylvania State University

PS,w·/wlugisl.f IIlII,t! long r:o'ljeclrlrcd Ilwl \Verh,flu .whIt'S' variability may bt' an I'mlicator of iWrlling di.mfJilit)'. Rl'sMrch on ;mnsubte.rt $cotlu (rtUlRe) with l"'t'l'iOlIS Wecll sler scales indica/I'd 111m scO/trr t'rJuld nOI rt'//nbly discriminatt! bl'l",un disaMt!tJ lUlllnomlisabled children. Ti,e ProJil1' VariabililY Indu (PV1) also Jailc(1 to demonSIrCItt' adequate diagnostic ulililY w;,h till! !VISC·R, TIlt'l"ut!nt .f/lld)' arSt'ssl'd IIII' diagnos lic IIti/it)' 0/ slIhtul ,'ariability imlicl'S on Ille !VISC· III hl' comlmring 684 .fllldl!nt wilh learning disabi"lll's 10 Iht! 2,200 rill/dun in IIII' standanlizmion sampll' Rt's"lls 0/ Rt!cl'il'u Opnat;"g CUrI'1! (ROC) analySts ind,catl'd Ihal WISC·1I1 sllbt('5t mriob,lit-" (u qll(.nli/it'd b v n.nRI' and I'arianct' t!lh'/JIIt!tJ no dlOllnostlt' IHility in distinglllsiling hl'lll't!t'n children "'illl It'llrn;ng dlSObihllt!S and ch i/drt'n from IIII' SlOndartl;zal ion sl.mpll', /I "'lIS conc/Ifllt'd 'hot 1II,,.rprl'l(l/ w n a/WISe.1I1 SlIbll'sl \'Uriahi/il)' 11U_' no pltICt! in 'he sril'n'ific practict' 0/ scllool pS\'clloloR I'

Lt!s IlSyrllologll eJ onl lang't'mp.r S/lpposi que la vtniabmt' des SOIIS·ustS dt' I\ loc/I.riU 'I ~ltl i lre lin indicalellr d '/flle diffieultl d 'apprtmlisso8~. La rt'chuche .1111'1" l'at'ilJl/OtI elllr~ It's SO/U· I~sts iI raide dt's i cl"tlles anllritwrt!S ot' Wt' chslt!r illdiqlWil que It. "arialt'on ne pOllI'ail pas illlblir des distinClions ciairt!S tn lrt! It!s en/allts qlH ()n t lll'S diffiCIIllb d 'apprenlissag~ et Ct! llX qlli n 't'n iprolll'eni pas, Lt! "PmJile VlIrillhililY /I/dt!x" (PVI) a igolemenl Ichoul ~ dimolllru 1'llIilitl d'llII diaglloslic adeqlwl (/VI1C Ie WISC· R. La 'Jr~se nle ell/dt! eVil/lwit {'uli!ile dll diognoslic de /0 "arimioli des illdices des SOI/S · lt .fts sur It' !VISC· III en comlJarant 684 /tlldilllllS ayu/II des di/Jicl/ltis d'apprt'nli.rsage allX 2200 en/ants /IIi/isis pOllr I' I clwtll il/oll dt' standardisation, Les analyses des risl/flats 1111 "Rt'Ct'ivt'r O,lera/ing CllrI'e " (ROC) indiqlloient qlu !t,mr;abili,1 des SOllS ,ttsts WISC· III, Il'lIt qut' qlwntiJile par I'lcor' t'l 10 mriance, n 'a dtmo"trl OIlClme /fIilill dll

diagnostic dans 10 lliSlinction en lre les en/ants a),ont des prob/~mes d 'opprentissage n celu' pra.'enOlu de I 'lchantil/on dt' slllndardisotion. /I en a /ti conelll que l'inluprltarioll dt' la I'oriabilitl dt'S sOlls " ~sls WIse· III n '0 pas so plact' dans 10 pratiqllt! SC'Ientifiqllt! dt' la psych% git' tn mitit'lf seo/aire.

Psychologists have long conjectured that the variability of an individual 's scaled

scores across the subtes ts of an intelligence lest may be an indicator of

neurological dysfunction (Drebing. el al.. 1994; Ivnik. et al.. 1994) and learning

disabi lity (McLean. Reynolds, & Kaufman, 1990). Specific sublest variations

COJ'Te spondcnce concerning this article should be addressed to MArley W, Watkins. Department o f Educat io nal Dnd School Psychology and Special Education. The Pennsy lvan ia State University. 227 CE DAR Building, Universi ty Park. PA 16802, Electronic mail may be sent via InternellO mwwIO@psu ,edu .

12 M. n~ U~lIki".f

have been postulated to be related to learning di sabilit ies (Kaufman . 1994:

Rou rke, 1998) and both the Wechsler Adult Inte ll igence Sca le - Third Ed ition

(WAIS -III : Wechsler. 1997) and the Wechsler Inte ll igence Scale for Chi ld ren ­

Third Edition (WISe-HI ; Wechsler. 199 1) manuals report Ihal inlcrsubtcst scatter

is "frequently considered as diagnostica lly significant " (Wechsler. 199 1. p. 177).

The most popular index of subtest variabilit y has been the ra nge. That is.

the di ffe rence between the highest and lowes t subtest sca led scores. Tables of

this index, called intcrsubtcst scatter, are provided for the WAIS-I11 and wise· III standardization samples in their respec ti ve manuals. However. research on

intersubtest scatter with previous Wechslertesls has been unproductive (Sail \cr.

1992). When students with learning di sabilities we re co mpared to normal

children on the Wechsler In te lligence Sca le for Children - Revi sed (W ISC-R:

Wechsler, 1974). they sometimes ex hibited greater scalier but mean differences

were too small for adequate diagnostic utilit y (Ande rson . Kaufman . &

Kaufman , 1976; Gutkin , 1979). Likewise. Wechsle r Adu lt In telligence Sca le

- Revised (WAIS-R: Wechsler, 19BI) scatter could not di stingui sh between

pati e nt s with a nd without b rain damag e (Ryan, Paolo. & Smit h .

1992). Intersubtesl scatter may not, however, be the most powerfu l measure of

subtes! variab il ity. Plake, Reynolds, and Gutk in ( 198 1) sugges ted that subtest

variance is a more useful index of variability because it utili zes information

from all subtests, rathert hanjust the highest and lowest subtests . Their Profile

Variability Index (PV£) is computed by applying the sample vari ance formu la

to the subtest scores of an indi vidual exam inee . Just as variance is a beller

measure of dispersion than range (Wilcox, 1996), the PVI may be a more

adequate measure of sublest va ri abili ty (McLean , Reynolds. & Kaufman.

1990) . For example. for two sets of scores (6. 10. 10. 10. 14 ve rsus 6. 8. 10.

12, 14), the range is insensitive (8 for both sets) whereas the PV I is sensiti ve

(8 and 10, respectively) to score differences.

Research on the PVI with previous Wechsler sca les has been equ ivoca l.

Kline , Snyder, Guilmette, and Caste ll anos ( 1993) computed the PVI from the

WISC-R scores of 146 Canadian students referred for poor achieveme nt , but

found weak ex tern al va lidity in term s of its re lat ionship with academic

achievement. They did not, however, evalu ate the PVI's d iagnos tic ut ility.

McLean . Reynolds. and Kaufman (1990) calcu lated the PVI for the WAIS-R

standardization sample and provided normati ve base rates, but did not ana lyze

its d iagnostic utility. Ivnik et al. ( 1994) fou nd that WAIS -R PVI scores were

higher among neuro logically suspect o lder persons, but had little c lin ical utility.

Although the d iagnosti c utility of sublest va ri abi lit y indices has not been

supported with previous Wechsler scales. it is possible that rece ntly revised

intelligence tes ts such as the WISC-1l1 may yie ld more useful information.

IVISe-1I1 S .. blt!.f l VlIrillbilil)' J3

The WI SC-III is a direc t descendant of the WISC- R. but only around 73 % of

the WISC-R ite ms were reta ined in the WI SC-III (Edwards & Edwards. 1993).

T he WISC-III also con tains a new sub tes t as well as numerous revis ions of

material s and adm inistration procedures. These changes make it difficult to

know whether results of previou s profil e research can be appli ed to the WISC­

III (K line. et a l. . 1993).

An initial ana lys is ofWISC-llI scatter was not favo rab le. Da ley and Nagle

( 1996) found tha t WiSe- III intersubtest scatter did not have adequate

di agnos ti c utilit y fo r 308 South Carolina students with learning di sabilities.

They did not. however. inves tigate the diagnos tic utility of the PVI with these

student :-o . Kline et a l. ( 1993) sugges tcd that the d iagnos tic ut ility of the WISC­

III PV I should bc assessed with child ren class ified as learni ng disabled. Given

that WISe-JlI subtest variabi lity has not been adequate ly investigated, lhe

present st udy was conducted to test the diagnostic utility of WISC- lIl subtest

variability among a large sample of children with learning di sabilities .

Method

Instrument The WIS C- III is an indi vidu ally admi ni stered test of intellec tual ab ility for

children aged 6-0 to 16- 1 I years. It consists of 10 mandatory subtests (M::;

10: SD = 3) which combine to yield Verbal (V IO). Performance (PIO) and

Ful l Scale (FS IO) 10·s (M = 100: SD = 15). Full detai ls of the instrument are

avai lab le in Wechs le r ( 199 I ).

Pa rticipallts Learning Di sabi lities Sa mple . All students who received co mprehe nsi ve

psychoeducational evalu at ions in three southwestern United States suburban

schoo l di stri cts during one school year were init ially eligible . Participants

were sc lected from special educat ion records based upon two c riteria: (a)

cognitive assessment included lhe 10 manda tory subtests of the WISC-IlI :

and (b) placement in a learning disability (LD) program .

Students' spec ial educati on eli gibi lity and placement was determined by

multidi sciplinary evaluation teams following assessment by a stale certified

school psychologist. Teams fo llowed state spec ial educat ion regulations which

were si milar to U.S. federal guidelines in that a learning disabi lity was defined

as a s ignifi cant abi lity-achievement discrepancy in the absence of sensory

deficits, educat ional disadvantage. and mental retardation.

These selecti on criteria identified 684 students who were enrolled in grades

kindergarten through eleven. Medi an grade placement was 5th grade with 68%

of the participants placed in g rades kindergarten through 6. Mean age was

/4 /11. \ \~ Walkins

10.8 yea rs (sd = 2.7). Ma les co mpri sed 73% and fem a les 27% o f t he

parti cipants. Ethni c identity, as reported by parcnt ~ on schoo l records. was

60% White, 18% Native American , 14 % Hi s pan ic. and 8% Black. Si nce

anonymo us archi val records were used. it was no t possi bl e to describe students

by othe r c rit eria (i.e .• como rbid conditi o ns. SES status. ctc.).

Academ ic achievement leve ls in reading, math . and wri tt en express ion for

94% of the parti c ipants were measured wit h the Woodcock Jo hnson Tests of

Achievement - Re vised ( Woodcock & Math e r. 1989). A ca demi c

achiev e ment of th e rema ining s tud e nt s was assessed with s ix o th e r

achi evement tes ts (e.g . . W IAT, KTEA . WRAT. c tc .). Mean reading and

math ac hi eve m e nt sco res did not diffe r bet wccn WJ - R a nd othe r

achieveme nt measures (reading t= (641 ) = .28. p= .78: maO, t (64 1) = -.90. P = .37).

Table I presents summary intellectual and academic achievement ~cores for

partic ipating students. Although lower than average. cogn iti ve and acade mic

achievement level s a re consistent with othe r compilations or da ta fro l11 c hild re n

enro lled in special education programs ( Kavale & Nyc. 1985). Additionall y.

ability-ac hievement disc repanc ies calcul ated by subtrac ting achievement

scores from FSIQ reflect underachievemen t in a ll academic areas.

Table 1 Sum mary of WISe- III and Academic A c hi evement S ta nd ard Score s.

Discrepancy Between Ability and Achievement. Scatlc r. and Profi le Va ri abil ity

Ind ex ( PVI ) Sco re s for Lea rnin g Disabl e d , Reading Di sa b le d . a nd

Standard ization Samples Across WISC-III Subtests.

Learning Disabled· Reading Disabled·

Measure Mean SD Mean SD

VtQ 90.7 11.8 96.8 8.9

P ' Q 94.3 13.7 102.6 ' 0 .2

FS'Q 9 1.6 12.0 99.3 8.4

Reading Achievement 81.3 13.8 75.2 8.9

Math Achievement 84.9 t4.2 96.4 10.4

Writing Achievement 76.4 11.4 78.6 10.4

Reading Discrepancy 10.2 15.2 24.1 7 .6

Math Di screpancy 6.7 J 1.7 2.9 8. 1

Scatter (Total) 7.6 2.3 7.5 2.4

Scalter (Verbal) 4 .8 2. 1 4.7 2.0

.sc:lItcr (Perrtlrm:mct') 5.Q 204 5.9 '" PVljTI1!:l1) 6.2 .1.5 6 . 1 .1.6

!lVI (Ve rbal) 4.5 .1 .8 4.3 .l .5

PVI (Performance) 6.5 5 .0 6 .7 5 .2

• N = 684 b N = 131 'N= 2200

Standardization<

Mean

100

'00 100

7 .5

4 .6

6.0

6 .0

4 . 1

6.8

SD

15

15

15

2 .3

2 .0

V

..1 • .1

3.3

5 . 1

Wise-III S"h ,eJt Variabili,y /5

A subsal11p lc o r pa rti c ipants was identified 10 al low specia li zed ana lyses

ro r s tude nt s wi th unambiguou s. specific rcading di sabilit ies. Selection crit e ria

in c lud ed: (a) id en tifi cati o n as learn in g di sa bled in reading by a

multidi sc ip linary eva luatio n leam ; (b) WiSe-III Full Scale IQ (FSIQ) greater

than 84 ; (c) FS IQ-rcading ach ievement discrepancy greater than 15 po ints:

(d) FS IQ- math ac hi eveme nt di screpancy less th an 15 points; and (e) not

ident ified as learni ng di sabled in math by a multidiscip linary evaluation team .

These c riteria se lected 13 1 students from the larger sample of children with

learn ing disabilities. T heir mean cognitive and achievement scores are provided

in Tab le J. Whereas the genera l lea rning di sabilities group was marked by

FSIQ -reading and FS IQ - math di sc re pan c ies o f 10 .2 an d 6.7 poin. s.

respective ly, the s pccific reading di sabilities subsample had di screpancies in

reading and math or 24.1 and 2.9 poi nts, respective ly. Di stributio ns of age,

g rade placement . gender. and e thnic identit y were simi lar to the tota l sampl e

o r students with lea rning d isabiliti es (M age = 9.4; Md grade = 3: 75% male;

72.5% Whi.e. 12.2% Native American. 11 .5% Hi spani c. 3.8 % Black) .

Nondi sabled Sampl e. The WISC-III United States standardizat ion sample

incl uded a representa tive samp le of 2,200 children aged 6-0 to 16- 11 years.

See Wechs ler ( 199 1) ror a complete desc ripti on or the standardization sample.

Procedure WI SC-III s ubtest scores fo r each sample of participan ts were used to compute

six indiccs of inlersublest variab il ity : (a) intc rsubtes t sca tter of ten manda to ry

subtes ts . (b) intersubtes t sca tter o r five ve rba l s ubtests. (c) inte rsuht cs t scatte r

o r fi ve perrormance subtes ts, (d) PV I o f ten manda tory subtests. (e) PVI of

five verba l subtests. and (f) PVI of five performance subtes ts. Inte rsubtest

scalier indices were computed by taking the difre rence between Ihe highes t

and lowest subtest standard scores of each student as per Wechs le r ( 199 1, p .

266). The PVI was ca lcu lated as the vari ance of subtcst standard scores as per

Pl ake. Reyno lds. and Gu .kin ( 198 1).

A lla/ysis The51! 5i.t.. indice~ of subtest va.riabi lity were then u<;Jed tf) e'!.timale diagnoQ,tic

utilit y via Receiver Operating C urve (ROC) ana lysis (Dawson ~ Saunders &

Trapp. 1990). ROC s tati sti cs are rccommended as hi g hly appropri ate for

assessing the di agnosti c accuracy of psychological tes ts (McFall & Treat. 1999;

Rey, Mo rri s- Yates. & Stanislaw. 1992; Sechrest. Stick le. & Stewart . 1998)

because they do not depend on the prevalence of disabilities in the populat ion

and consequ e ntl y prov ide a d escript io n of diagnos ti c accuracy that is

independent of both base rate and decis ion thres ho ld effects (Metz. 1978).

16 M . Ill. U'mkin.J

A ROC ana lys is g raphically represents a tcst' s di agnosti c accunlcy across

its full range of cut scores. In the current case. the ROC curve graphs the true

positive ratio to fa lse-positive ratio as the variabili ty cul score for diagnosing

a learning disability ranges from zero to it s maxi mulll va lue . In contrast.

traditiona l analyses of WiSe-Ill variab ilit y use a !' ing lc prevalence ra lc (i .e ..

Table 8 .5 in the WiSe-III manual) to establi sh the cu t score.

As illustrated in Figure 1. the ROC cu rve ofa lest with zero di scriminating

power is a dashed diagonal line called the " random ROC : ' The morc accura tely

a test is able to discriminate between indiv idu als with and wit ho ut the target

1.0

0.8

0.6

0.4

0.2

0.0

0.0

Scatter ROC

0.2

Random

ROC

0.4 0.6 0.8 1.0

False Positive Rate

Figure I. ROC Analysis of Intersubtest Scatter (Range) for Students with

Learning Disabilit ies and Children from the WISe-III Standardization Sample.

disorder. the more its ROC curve will deviate toward the upper left corner of

the graph. The area under the ROC curve quant ifies its diagnostic util ity (AUe:

Hanley & McNeil, 1982). A perfect test would produce an AUe of 1.0 whereas

the random ROC always accounts for .50 of the area under the curve.

Results

WISC /II SlIlnu( Vllrjohilil)' 17

Table 2

/\rca , Under the ROC Curve (A UC) fo r Range (Scatter) and Variance (PVI)

Indi ce, for Learning Di sabled (LD) and Reading Disabled (RD) Groups.

LD RD

Hal1t!,cSc31Ier l ndcJ(

10 Sublesu .51 .50

5 Verbal Suhles\ s .53 .52

5 Performance Subtesls .50 .50

Prorilc Variahili ty Index

I () SuhtcSlS .52 .52

5 Verba l Sublesls .53 .52

5 Performance Sublests .50 .50

Note : Verbal subtes ts were In formation. Similarities . Arithmetic. Vocabulary,

and Co mprehe ns ion. Performance subtests included Pic ture Comp letio n,

Codi ng. Picture Arrangement. Block Design. and Object Assembl y.

.. -" CI:: .. .:: -.~ c. .. :: ,:

1.0

0.8

0.6

0.4

0.2

0.0

0.0

PVI

ROC

L / 0.2

/ /

/ /

/

0.4

/ /

/

/ /

/

...... //Random

/ ROC /

/ /

0.6 0.8 1.0

Fa lse Positive Rate

/ /

Fi gure 2. ROC Analysis fo r the PVI Indc;'ll (Variance) for S tudents with

Learning Disabi lit ies and Children from the WISC-III Standardization Sample.

/8 M. \Y. Walk in.!"

Mean intcrsubtcst scalter and PVI indices for eac h sa mple o f participants are

presented in Table I . Ins pec tion of the ROC curve of Figure I . whi ch is based

on the intcrsubtest scatter of the ten mandatory WiSe-III subtcs ts from the

genera l learn ing di sabled sample compared to the st.t ndardiz3tion sa mple .

reveals that it does not diverge from the rando m ROC . Likewi se. Figure 2

indicates that the ROC curve for the PVI calculated from the same len WISC­

III subtests of these parti c ipants is congruou s with the random ROC. ROC

curves for the ot her suhlcst va ri abi lity compari sons were a lmost identi ca l to

those pictured in Figures I and 2.

Based upon non parametric formulae provided by Hs iao. Bartko. and Potter

(1989). the AUe or Figure I was .51 and the AUe or Figure 2 was .52 . Areas

under the curve for a ll six va ri abi lity indices for each group of children with

disabilities are presented in Table 2 . AVC's of the magnitude found in Tab le 2

represent low diagnostic accuracy (Swcts , 1988). Thai is. the probabili ty that

a randomly chosen c hild w ith learn ing disabilities will have greate r subtcst

variability than a rando mly selected chi ld from the WISe-III standardizat io n

sample is only .5 1 - .53 . Thi s represent s discrimi nation at a c hance level.

Discussion WIse-III suhtcst va ri abil ity as quantified by range and varia nce ex hibited no

diagnos ti c utility in distinguishing between chi ldren wi th learning disab ilities

and chi ldren from the WISe-Ill s tandardizat ion sa mpl e. These findings are

consistent with res ult s from the WISC· R (Anderson. Kaufman , & Kaufman .

1976; Gutkin. 1979; K li ne. et a l. . 1993). Likewise. they arc congrue nt wi th

the negative discriminabi lity reported by Daley and Nagle ( 1996) for WISC·

III intersubtest scatter.

ROC ana lyses indicated that subtest variability is a useless indicator of

learning disabi lity status across the fu ll range of possib le cut scores (i.e ..

prevalence levels). The WISC-Ill developmental index , ACID profile . and

SCAD profile have also prove n to have littl e or no diagnostic utilit y in

identifying except iona l children (Watkins. 1996: Watkins, Kus h. & Glutting,

J 997a, J 997b) . Whe n considered within the broader, and generall y negative,

comext of sub test profile research (Kavalc & Fo rness. 1984: Kramer. Henning ­

S tout , Ullman. & Schnel lenberg. 1987; McDermott. Fantuzzo. & G lutt ing.

1990; McDermott , Fantuzzo. Glutting, Watkins. & Baggaley. 1992; Mueller.

Dennis. & S h o rt , 1986; Watk in s & Ku s h, 1994), s ubtest va riability is

unsupported as a tool in the diagnosis of learn ing di sabilities. Within the

interpretati ve framework presented by Kamphau s ( 1998). us ing s ubtest

variabi lity as an indicator o f learning disabilities wo uld cons titute a case o f

ac ting in opposition to sc ientific evidence.

W ISC-1I1 S lIbtt!$t I'llrillbillty /9

Rf!fUf!nCf!S

Ande:r~on . M .• Kaufman. A. S .. &. Kaufman. N. L. (1976). Usc of the: WlSC· R .... ith a 1earnin, disable:d

popul:mnn: SOllie: diagnostic implications. Psyr:h% tO' in ,hf! Schools. IJ. 381 ·386.

Dale)'. C. E_. &: Nagle:. R. J. (1996). Relevance of WISC· III indie:a to,s for asseumenl or learning

disabilil,e:\ . Jmlrnal of I'SI'r:ho,dlle:a r;01lal Asu,ssmt"'. 14, 320·333 .

1J:' .... son ·Sau nders. R .. &: Trapp. R. G. ( 1990). Basic and clin ical biol/a/;uiu. Norwalk. CT: Appleton &:

lange:

Dre:blng. C .. Sal1 .. P .. Van Gorp. W .• Chcr ... insky. A .. &. Uchiyama. C. ( 1994). WAIS-R intersubtest se:auer

in patie:nt~ with deme:nti a of Ah;heimer ' ~ t)'pe. Jr/llrflal ofClifl ical Ps .... ch%g)'. 50. 753 ·7.58 .

Ed .... ards. R .. &: Ed .... ard~. 1. L. (1993). The WIS C· IlI: A practi tioner perspecti ve. Journal (If

Ps.\·('hot'dIlCflftflm,1 ASSf'S.fml'''' mo" ograph suiu. AdwJncts in pS~'cholo,ical aSJI/!ssmf!nl: Wf!chsl,r

/ n ' I'/IIIlf!n ('(' S('(, II' f m Cltlldrf!n ·Third £di,iall. 144· 150.

Gotkln. T B ( 1979, WISC· R scalier indices: Useful in formation for dirrerential diagnosis? JOllrnal of

Schtwl Ps.,·chlJll'R.'·' 17. 368· ]71.

lI anley. l A .. &. l\IcNe:il. B J ( 1982). The mean ing and use: of the area under a reedver operal ing

characte:ri stic (ROCI cu rve . RadiO /OR)'. /4J. 29-36.

Ilsi80. J. K .• Bart ko. J 1 .. &: Poue: r. W. "I •. {1989} Diagnosing diagnoses. Arr'hivf!S o/Ge" ual PS),chi(llr},.

46.664 ·667 .

Ivnik. R. J .. Smi th. G. E .• Male:e:. J. F .• Kok me:n. E .. &: T.ngalo~. E. G. ( 1994). Mayo cognitive fae:tor

sea In : DiStingui shing normal and cl inical samples b)' profile variability. Nf!u rops»,chaIQR)'. 8. 203· 209.

Kamphaus. R. W. t 1998). Inte:lligence lest inte:rpreta tion: Acting in the absence of evidence. In A. Printera

&: D. 11 . Saklofske (Eds.). !VISe·11I c lin ical Ulf! and inlupreuu {on (pp. 39· 57). Ne .... York: Academie:

I'r ess

Kaurman. A S. ( 1994). Inullif(f!nl us,ing ... i,h Ih, WIS e ·lIl. Ne .... York : Wiley.

Ka vale . K. A .. &: Forne:ss. S. R ( 1984). A metl. anal ysis or the validit)' of Wechsler scale profile s and

lec:ate:goritat i ()n ~ : Patte: rns or parodies' uarflin& DisabWliu Quor/f!r/y. 7. 136· 156.

Ka val e: . K A .. &: Nyc . C. f 1911.5 1. Parameters of kaming disabilities in achievement. linguistic.

neUfop'ycholo8ical. and SQCiallbchlvioral domains . Thf! Jo"r"al oj S~cial £dUCOlion. 19.443-4.58.

Kline:. R. IJ .• Snyder. J .. Guilmette:. S .. &: Castellanos. M. ( 1993). Eltte:rnal valid ity of the profile

"ariability inde:J. for the K· ABC. Stanford· Binet. and the WISC· R: Another e:ul ·dc·sae:. Journa l of

l..f!arJ1 i"R Disablli,iu, 26 . .5 .57 ·567.

Kra me: r. 1. J.. Henning-Stout. M .. Ullnlan. D. P .. &. Se:hnellenbc:rg. R. P. ( 1987). The vi abilil )' of scatter

analysis on the WISC· R and thc SBlS: Eumining a vestige. jaurnol of Psychof!ducalional

ASSf!$Smtn,. 5. 31·47 .

Me:De:rmott . P A .• FantuZl.o. J. W .• &. Glullin, . J . J. ( 1990). Ju~ t say no to subtest analysis: A cri tique on

We:chsle:r theory and practice . Journal of Ps),chof!dru:al ionol ASSf!SSmf!lII. 8. 290· 302 .

Me: Dermott . P A., FanrU7.lO. J W .. &: Glutt ing. J . J .. Watkins. M. W .. &. 03 ggaley. A. R. ( 1992). lIlusions

of mcaning in the IpSltivc assess ment o f children's abili ty. Journol of Spf! cia l EdllcaliQn. 25, .504 ·.526.

McFall. R. M .. &. Treat. T. A. t 1999). Quanti fying the in formation value of clinical assessments .... ith

.il!nll! detee:tion theory Annual Rf!\';f!M' of PSI·cha/oIU. 50. 2 14·241 .

Me:Lean. J E .. Reynolds. C. R . &. Kaufman. A. S. ( 1990). WAIS· R l ubtest scaUer using the profile:

vari abilit)· indCll I' .f)'chol/'gkal A.fJusmf!lII: A Jo"rna/ of ConJll/l ing ond Clinical Psycholog),. 1. 289·

292.

Met'l.. C. E. (1978). Dasie: principles of ROC analysis. Sf!minars in Nuc/ttJI' Mf!dicinf!. 8. 283·298.

Mueller. II. II .. Dennis. S. S .. &: Short. R. H. ( 1986). fI. metl ·uploration of WISC· R he:tor score: profiles

a\ a fune:tion of dr.gnosis I nd inte:llectual kvel. Canmlian JQurrial Qf School PJye: hology. 2. 2 1-43.

Plake:. 8 . S .. Reynolds. C. R .. &: Gutk in. T. B. ( 198 1). A technique for the comparison of profile vuiability

bet .... e:en ",dependent groups. Jiwrnal ofClinir;a/ Ps)·('hoIQRY. 17. 142· 1<16.

Re:schly. D. 1 .• &: Wilson. M. S. ( 199.5). School psychology practitioners and fae:ulty : 1986 to 1991 ·92

tle:nds in de:mosraph ie5. roles. 51ti Sr'Clion. and sylte:m reform. $chaol PsycholoRY Revif!w, 14. 62 ·80. Resehly. D. 1 .. &: Wilson. M. S. ( 1997). Chaue:tcris tic:s of se:hool ps)'ehology sradu~te education :

Implications ror the: e:ntr ), . level discunion Ind doc torl l· levc:l 5pc:cililly definition. School Ps~'Chnlog)'

Rf! vil'It·. 16. 74 ·92.

20 M. IV, Watkin.t

Rey. J . M .. Mo"is· Vales. A • &: Sunisllw, n . ( 1992). Mca.surinllhe accutlcy of dil!-nOMic (C5I~ using

recei"er operaun, charlctcri5lics (ROC) anal),sis. Inltrfla/ion(ll Journa/IJ/ MelhodJ in PJychiolric

Rt'ua,d" 2, 39 50

Roberts. A II , de Rust . J O. (1994), Role Ind rUDClion of school psycbololl\U . 1992·93: A comparative

Slud)'. Pncho/n,l' in Ihe ScllQob , J/ , 113- 119.

Roulke.8 P (1998). S'lnifinnce of Ycrbal-pcrfonnancc dlscrcp.nCIC~ fOf 5ubl)'PCS of children with

Icunlnl diubilllic:s: Opponunilics for the WISe· 1I1. In A. Prililcr." 0 II. Saldohkc (&b). Wlse­III c/inka/ flU Qrtd i",trprt,otion (pp. 39-57) New York : Academic Preu.

R)'ln. J . J .• Paolo. A. M .• &: Smith. A. J. ( 1992). Wechsler Adull InlcHiICnte SCllc· Revlsed (WA IS -R)

inlcrsuhlcsi SCltler in brain-damaJed patienls: A companson with the standlUdu.allon ~mple .

Psycholo&icol Anenmc:nl. 4. 63-66

Satekr, J . M. ( 1992) ASJe'S.flnt", ofchildrtn: RtVISc:d and updaltlllhmi tllmun. San 0":80; krome M

Sauler.

Sechrest, t .. Stickle. T. R .. &: Stewart. M. ( 1998) . In A. S. Bellack I.e M . lIer\en (Ed~ . ) . Comprc:hc:nsi-'e

clinicIJI pSyChO/DIY: Voillm e: 4. Osstumt'" (pp. 1.32). NY: Elsevier.

Smith. D. K. (1984). Practici ng school psychologists : Thei r charactc r ls t ic~ . ac tivit iu. and popUlations

scrvcd. Professi,,,.o/ Ps)'Chulogy: Reseor('h and Pr(JClk r . IJ. 798 ·810

Swell. J . A. (1988). Measuring the aecurlCY or dia8no~lic s),slem5. Scien('f'. 240. 128.5 1293.

Walkins. M. W. ( 199ti). Di88noslic utility of the WISe · 1I1 developmental indu a\ a predictor of learning

diubi lit ies Journ(J1 of Le(Jrnin~ Disabilities. 29. 305-3 12

Watkins. M . W .. &: Kush. J . C. ( 1994). WISC-R subtest analysis: Thc ri,ht way. the wronl ... ay. or no

way? Schaul PsycholalY RC:I'it,,\ 21. 640-651

Watkins. M. W .. Kush. J C .• &. Gluuinll. J. J . ( 1997a). Prevalence and dialnostic utility of the WISC· 1I1

SCAD pronte arnonl children wi th disabilities. School PSl'chala~.v QU(Jrlrrh-. 12. 235-241.

Watlr.ins. M . W • Kush. J . Co. &: Glullinl. I . J. (1997b). Discriminant and prcdictivc validity of Ihc WISC·

III ACID profilc amonl chlldrcn with karninl disabilities. PJI·rhalal ... • In ,h,. Schools. 14, 309-319

Wechsler. D. ( 1974). MClfluol for Ihe Wu hslrr b"c:lliSf'ncr Scalc: fa' Child"" • Rt"'Jed. San Antonio. TX '

Psycholo.iea l Corporation

Wceh~lcr. D. ( 19811. MOn lltd for Ihe l4'uhJ/f"r Adlllll"'tlll~f'"cr S('olt • Rf'I'lud San Antonio. l'X:

Psychololical Corporation.

Wechsler. D. ( 1991). Ma"ual for Iht Wuhslr, fn'elll~tnu Scalf" fOt Child"n 7'h"d l:.'dillO" . San Anlonlo.

TX : P~)'eholo!iClI Corpora tIon.

Wi leo ... . R. R. ( 1996). SllI/is/ieJ of the sorinl Jri,."us. NY: AcadcllllC P, ess .

Wil50n. M. S .. &. Reschl )'. D. J (1 996). Assessment in school ,,~)'eho lolY Itainin,and prlctice School

PJ),chIJ luIO' R il vitw. 25. 9-23.

Woodcock. R. W .. &: Mllher. N. (1989 ). WJ -R Tests of Achievcment: Eurniner ' , MlnU l l. ln R. W.

Woodcock &: M . 8 . John50n. lIIatldcad-Jnh" son PJ\·('ho ·£dIlCIII/onal Ba"",)'·Rtl'/ud. Allen. TX :

DLM .