document resume ed 253 126 he 017 927ed 253 126 author title pub date note pub mg edrs price...
TRANSCRIPT
ED 253 126
AUTHORTITLE
PUB DATENOTE
PUB mg
EDRS PRICEDESCRIPTORS
DOCUMENT RESUME
HE 017 927
Kroc, Richard J.Comparing Citation Rates with Other MeasureS ofScholarly Productivity.Oct 84
' .28p.; Pgper presented at the Joint Meeting of theAmerican Educational. Research Association Division Jand the Associationipor the Study of Higher Education(San Francisco, CA, October*28-30, 1984).Reports Research/Technical (143)Speeches/Conference Papers (150)
MF01/PCO2 Plu0Posta0.*College FaJUlty; Conferences; *Evaluation Criteria;Financial Support; -Higher Education; InstitutionalCharacteristicS; *Productivity; Reputation;*Scholarship; *Schools of Education *Wrip iting forPublication
IDENTIFIERS *Faculty Publishing
'ABSTRACTThe construct validity of scholarly productivity was
investigated, with attention to definifions and measurementapproaches. A nonrepresentative sample of 51.schools of education wasselected to include very produgtive schools. Measures were made ofthe following variables: citations, rankings froth previous studies,publications, conference participation, funding, and generalinstitutional characteristics. Publication counts were estimatedlromdata in the Educational Resources I4lformation Center (ERIC) system.Each school of educption's participation in the American EducationalResearch Association conference. was assessed for 1981 and 1982.Information on grants awarded to schools of education during1978-Y982 was obtained from the Smiths9nian Science InformationExchange. Finally, the Higher Education General Information Surveyprovided data on _five variables: percentage of aoctoraes granted,total research expenditures, total overall expenditures, governmentgrants and contracts per full-time equivaldncy, and, averagesalary. Conclusions include the following: program size did nosignificantlydetermine prestige; surveys on reputation did notappear to be reliable; and. citation counts seemed to b the bestmeasure of scholarly work. (SW)
.,
1
***************************************************1**********Ao*****Reproductions Supplied by EDRS are the best that can be made
from ihe original document.******************.************,*****t***********************v**********
0.1
to-
-
Comparing Citation Rates with.Other Measures
"PERMISSION TO REPRODUCE TINS
MATERIAL.HAS BEEN GRANTED. BY
TO THE EDUCATIONAL RESOURCES
.INFORMATION CENTER (ERIC)"
of Scholarly Productivity *
. U.S. DEPARTMENT OF EDUCATIONNATIONAL INSI IIUT( ,I.DOCAT ION
t OUCAT IONAL Itl SOIATC.IS 4NrpliNIA I IONCLNITIt (Intel .
(..411,441.'etonlimt hat, tUtUo toptOthtflUti aA
Richard J. K r o c mcmvtul hum the, tmus011 et oftiointe*on
ofiti,nallmodMIt(11 Imbues kive boon made to 11114110VO
telpot 1)4:1) itmlity.
P us .11 owmoos *Anted it 11:*do(*.mom do not ectrwtin, roptoent offIc141NIE
UniversAy of Colorado
pn511n/n of putt V.
In.assessments of excellence in higher education, uestions
of scholarly productivity are often pivotal.. Research and
publication are the driving forces in the modern university,
evaluators of university programs are attuned ;() this fact.
From i6divAual.promotion and tenure decisions to overall...
institution-al prestige, scholarly research is a fUndament.I1141 ,
issue.
A variety' of procedures hive been used 'to measure
productivity. Some researchers have surveyed faculty or
and
\
administrators to obtain ranjdngs of universities and
departments. ,Others have-used the, quantity or quality of
publications to rate programs, sometimes including presenttAans
of papers at conferences. Another measure is the type and .
amount of funding obtained by a ,department, and, at -thy
university level, characteristics such as faculty salaries and
research expenditures have been ] -inked to faculty productivity.-.
Fln14h aelly, citation rates, a measure of peer recognition, have
been used to- assess scholarly accomplishment. In some way,
of these measures has' been associated.wi.th productt sty.
.This study ..addressed tke construct validitx oft cholarly
productivity: Mow this concept has :been defined, and how
.:S
Paper p sented at the Joint Meeting of the American Educational ResearchAssoc.]. ion DivIston J and the Association for the Study of) Higher Education,San Fr n4pcd, C.41ifornia, October-2830, 1984.
Sr
-.)111yr
At*
ti
*2
various ways of measuring it compared. In particular, data
pertaining to schools of ed.ucation were examined. The
. perspective was that of an administrator or evaluator who must
,collect and assess information regarding productivity,
discovering in the 'process what data are available for making
.valid, useful comparisons among 'schools of educatico, whatti
problems occur in gathering this data, what relationships 'exist
amoog measures, and what interpretations can be made. These
issues are of considerable practical significance, reflecting
processes that often have a direct impact 611 the future of a
program, a school or a career'.
Interrelationships among variousfmeasures of productivity
have been explored indifferent ways by several researchers.
Hagstrom (1971) determined that a single, unrotave'd factor was
sufficient to account for most of the variance among 188
university.science and mathematics departMents on ten variables
related to institutional quality. Astin and Solmon discovered
two factors,'"scholarly excellence of faculty" and. "commitment
to teaching, ". in their factor analysis of a questionnaire on
undergraduate excellence (Astin &.Solmon, 1981; Solmon & Astin,
19.81).' They' also found evidence thatoyerall institutional -
V.-
prestige is an important aspect.of'individual departmental
ratings. Another study ,(Andersen, Narjn, & McAllister, 1978)
used regression techniques to predict prestige rankings from
various publication and citation data, finding that variables
related. to .size, as well as to. quality; are predictive of
preWge. Endler, Rushton and R.oediger. (1975) correlated both
.
4
,
-)t
4
publicition counts and 'citation data with other measu.res,
concluding that, citation rates.may be Wetter than publication'
counts as a\ea'sure of quality. :Sash (1983) produced composite
rankings of schools of education using conference presentations
and publication counts. 1r
ThiS Atudy..examined. relationships among a series of
N.
measuc.es of productivity for a single sample of schools'usincl
coAputerized data bases easily accessible5to any researcher. In
particular, the results of citation analysis, a method not often
used in many fields, including education 4Kroc, 1984), was co.-.
pared with 'other
Sample
V .
Method
A non-representative sample of 51 s.chb9ls of education was
chosen from rankings in.other studies tO include as many of the
most productive schools. as possiblei (see Table 1). For each-.
school, faculty lists were obtainetfrom 1981 course catalogs-or
-by writing directly to the university. Measures were made of
the following variables:' citations, rankings from previous
studies, publications, conference participation, funding, and-.
general institutionbl characteristics. Data or the 4600 ..
, V i afaculty included in the study were- agg'regated at the school
r
..,
,
level... .
.,
Citations41.
The number of citations for each faculty member in the 1981
Social Science Citation Index (SSCI) b4as counted. The SSCI
fists, by author, citations of that author's, work during a given
. .
.11
-*
4
year. k.set of rules was developed .to standardize the counting.
procedure, and two Judges were trained, enabling an assessment,.
v
to b made of the relipility of citation counts? 'Me obtaineda
inte rater-4prrelation coefficient of .r *.90 ipdicated that.it
cttations can be re.l.iably, counted.4.
Th'e,tC1 proved to b'ie a:r1ch s:oirce data,-fully.
ref rencing 160 education *journals, as yell, assmany other
journals in related fields, on an 'annual basis. A more thorough, ,
discu.ssi'on of SSCI citation analysis occurs elsewhere (Kroc,
k
4
198/1).
Rankings from Previous Studies
Rankings of schools of education from nine studies were
recorded. Five of these researA 'efforts (Blau & Margulies,
1975; Cartter, 977; Ladd.& 1.ipset, 1979; Sieber, 1'966;
Walberg,'1972) were surveys of, either education faculty, AERA
memb.ers, or deans of. schools of education. Theotherjour
studies prodcued ranks based on publication counts°(West, 1978),
AERA -presentations (Dole, 1981; Schubert, 1979) or both (Eash,
1983).
Publications,
Two data bases, Research in Education (RIE) and the Current.
Index to Journals in Education (CIJE) werl.e searched using the
ERIC system. The high cost of obtaining.this information 'for
each facultymember ensured that there were limitations on the
amount of data, that could be 'collected and on the. number of
fculty surveyed, resulting in a decisionto sample 25% of. the
4600 faculty. Based, on previously obtained citation data,
N.! ':;,
5
faculty from each university were, split iQto four strata, and an
optimal allocation 'formula (tokiran,11953, p. 93) was used to
determine the propo'rtion of ot2servations within each strata
which wbuld minimize the standard errorof the mean publi6tion. ,
'counts.
Conferenceayarticipatison ,
Conference participation was measured by counting the
number of presentations listed in the Annual meeting programs1
at two confere7,s,..-the American Educational Research kssocia- .
tion (AERA) conferencv in 1981 and 1982. Two consecutive 'AERA. . t
conferences were surveyed to control for bias due o the meeting..- A .
ocation: The 1981 conference was htld in 1.0*Angeles, while
the 1982 location was Boston.\ All types of AERA presentations
were given equal weights, except discussants, who were not
counted.
Eundi n1
Only one accessible source of funding data was found, the
Smithsonian Sciene InfOrmation Exchange (SSIE), an agency that
compiled and stored for computer access information on.grants
issu'ed between 1978 and 1982. . SSIE received project
descriptions from over 1;300 organizations: federal, state and.
local government agencies; nonprofit assviations.and founda-.
tions,; colleges and iniversities; alid,tto a liMi/ed extent,
private industry and foreign research organizations. Approxi-'
mately,90% of the information was provided by agencies of the
federal government. Unfortunately, the SSIE was phased out late
in 1981, a victim of federal budget cuts. Nowever4, the National
4
14
6
-0
Technical ijnfOrmation Servise'(NTIS) has assumed 'atRortion of
SSIP's former. services, but does not yet gather djta from the
agencies which are mOstI11kely to furid educational research.,1
The disSolution of the SSIE was not a problem in this study,
'since funding data was. available through 198o
Thee organization of the SSIE data-base madlg it possible to
ts
enter the program' and university name at a computer terminal and
receive a printout containing abstracts of the iovernmenti grants
obtainwed by that education program between 1978 and 1982. Each
.
abstract listed the geant's,nvestigators, title, sponsoring ,
organization, performing organization, funding period, and
dollar amount., 'Unfortunately, this' information was not always
complete; in fact, 30% of the abstracts did not list a dollar
amount for the grant. Still, it was possible to count the
number -of grants obtained by each program, As well as to get
some indication of the money involved.
Institutional Characteristics
All universities that 1eceive federal funding must 'report
certain information annually to the government.' This data is
documented in the Higher Education General Information Survey
(HEGIS) and is ;available 011 magnetic tape.' Variables which were
accessible and of interest for this study were dollar amount of -
government grants and contracts per FTE (1978-79), total
research expenditures (1979-80), total expenditures (1979-80),
percentage of doctorates granted (1979-80), ,and average faculty
salaries (1980-81). All of these measures were at the univer-
sityilevel, not at the school or,department level.
..4
'Analysis r. ,
7
Two sets .of rankings of schools of education were produced
fir each variable, one based on total.counts for each school'apd
the other on .total ,.counts. divided by the number of faculty in
each school. Cpmparisons of the six sets of variables, using
both unit's of analysis, were'mode by examining correlation
$ coefficients. The patterns of 'these coeffic) ts provided data
on how.the variables in this study were inter-re ated as well
as how they Compared with measures of productivity fron' other
studies.
RESULTS
Relationship Between.Cqation Counts
and Rankings from Other Studies
As discussed, a number of studies of education schools, have
been done. The ranking' resulting from these studies were
correlated with oni another as well as with citation (Jaya,
producing the colificients 4a Table On the diagonals are the
number of schools which were ranked in the study that is listed
at the. top of the column. For example., the ranks for 23
programs which' were available from tht Dole (1981.) study,
correlated'.26 with rankings from the Ladd and Lipset (1979)
study.. Five sets of rankings (Cartter, 1977; Ladd & Lipset,
1979; Sieber, 1966; 8.1au,and Margulies, 1975; an6:Walberg, 1962)
were based on surveys which asked resp6ndents to list the top
schbols. of education. The correlatAons among these: five studies
showed considerable variation, indicating how surv6 riesponses
vr,
8 . I I.
it
. can differ. For examp bath, artter. (1977) and Ladd and.
lipset (1379) surVeyed e cationjaculty memberS.in.,Ottempts to
discover the most di-Stingu shed sCifools of education. Yet their-.4 .
. .
.
rankings correlated only Prestige'rank,ingsf then ..may not
be very consistent, even when, similar questionsA1 re asked of the
same grbup of respondents.
Studies that obtained less subjective measures of quality
fared somewhat better when compared with one-another. Schubert.,
(1979), Dole (1981)-and Eash (1983) used AERA presentations as
measures for ranking programs. Schubetrt cjounted total presen7
.tatons, Dole :os,ed preseniatiOns per faculty, and Eash i-eported
both-. The ,ranks of Dole and Eash show a correlation of ..70,
.v4ile those of Schubert, an(C'Eash correlate .83, an imdication
that AERA presentation counts are somewhat consjstent..4.
-Similarly, Eas.h (1983) and West,, (`197.8) ranked programs
according to the )otal number of articles appearingk
eading
'education journals. The correlation between these studies,
r = .49, woald probably have been highfrr had more of the same
journals been surveyed. Nonetheless, there' is evidence .of
moderate reliability in ranking by counts of journal articles.
ReSults of studies using total faculty counts showed low.
'correlations with those calculating rgsults on a per faculty
basis. for example, the AERA presentation ranks of.Dole and.
Schubert correlate only .19. In general, ratings using the
faculty member as the unit of analysis correlate more highly
with-prestige rankings; as shown by comparing Walberg's 11972
rankings with those of Eash. When asked,to rank schoo,ls':of
r,
Ir.
9
education-, survey,respondentsseeff to think more in terms of. ,
research h-producOvity pr faculty memberthan they do of total
productivity.
Meah-citation'eanks correlated moderately to stronly \vith
all .survey rankings except Ladd and LiOset's and with '4.11 other
rarkkillgs which used individual faculty as the uni,t,of analysis.. .
The highest correlations were with the Carttvr survey it
.83) and with Eash's rankings of AERA preseneationsper FTE
a
0(r =.65). Mean citation rank, then, showed a relationship .
with the ranking,methods used sin other studies.
Although'mean citation rank correlated well with' other
types of ratings, total citation counts were not as strongly'
relatedto other measures. Percentage of faculty with 7,,ero
citations, on the other hand, followed a pattern identical to
that of the an citation rate, indicating that they both may be,.
tapping the same dimension of productivity.I
- Publ i cat ion vCounts
Publications were estimated from,data in the ERIC system.
Two other studies (West, 1978; Eash 1983) used publication
counts as a method for ranking programs. Table 3 shows inter--
correlations,-among the ranks produced from thest various methods
of measuring publications, while Table 4 displays correlations
between publication ranks., citation measures,:and rankings from.
other studies..
The consistency between .Sash's" coups of articles in the 14
journals and the publication measures fn this study was shown by
the strong relationship between Eash'.s total article's and the4
''' "77- 77.t."*.:::; "7. 7.!--7"7 . .
4
10
-*total .ERIC publication rank (r .75) as well as by the-7
correlation betwe'en'the, two FTE counts. (r = .63)1, West's
r- method of rankng (based on total publication counts in Li
journal's) did, not correlate as well. with either of the ,other
measures'. = .33 with total 1ERIC publication rank, and r
.49,with Eash's total counts) .
. The correlations between publication .rankings and other
p
ty()es of radVings showed a wide range of values. In general,
though, the five survey results had more in common with per-
faculty measure% of publications than with total counts: four
aut of /the five studies had higher correlation's with per faculty
publication rates.,
Citation ranks were not strongly related to publication
counts, 'as shown by the coefficients in the first three columqs
of Table 4. Mean citation rate,correlatedonly .34 with ash's
FTC publication 'ranks and .39 with per faculty ERIC publication
ranks, while total citation rank' showed a similar degree of
relationship with the three measures of total publications. A
stronger correlation, though, was found between the percentage
of faculty with no citations bnd per faculty publication ranks.
It may be that those faculty who are more often cited have qu'ite
variable publication pAtterns, th'us lowering any correlation
between. citation and publication rates, while tho4e who are not
cited tend homogenously not to publish, thereby increasing th.es.
relationship between publication rates and the percentage with
no citations.
It should be note that the publication,ocounts i .this
p.
.
a
So.:Se
4 11'4
,
study were basect on combining entries from both the CIJE andk
RIE. -These publicaion counts, tlieb, include bath journal
articles and "fugitive" literature such as technical reports and
evaluations.
Conference FlaitriciallV
Pa
Cacti school of "Oucation's participation in the American
Educational ReSearch'AssociOtion, conference was assessed for two
years, 1981 and 1982., Three other- studies used AERA participa-.
ti on as a means for ranking programs: Schubert (1979) used data
from 1975 to 1979 to'obtain counts of total presentations by '
each school of education., Dole (1981) used Schubert's figures,
divirigaeach total by the number of faculty to produce' rankings,
with faculty as the unit of analysis. 'Sash (1983) counted AERA
presentations over. a sAten-year period, 1975-1981, com Aling
both total and FTE rankings of schools. Table 5 shows e
intercorrelations among these three studies and with the ranks
from this, study. These coefficients Indicat-ed greater consis -.
tency among AERA studies than among studies of publication
rates. This seems reasonabl*, given the ease with which AERA
participation'can he judged. from the program meeting notes, and
thefact'that each publication study used counts from a differ-
ent set of journals.-,100
Table 6 indicates correlations between the _various "AERA
'measures and three groups of rankings, representing citation
rates, publiption counts. anal prestige surveys. Citation
'measures correlated better with AERA presentatlons than. with.
publication.rates. Perhaps authors who are cited-more often
,
At
t.
have a greater ii1cl n at ton to submj t: th 4" WQrk to, the -n atci 'Dna.]
exposure of fIr.ed by Ae/RA conferencs.,
COrrelations between. l'AEW data ',And, publication ranks showed.',
\ a pattern of moderate relationships, whil prestige su7rveys didF(.)
not correlate consistently with AERA results, as alight. be of
expected.
Fundi ng
A computer search .of the Smithsonian Science Information .;;...
Exch-arige (SSIE) eliciet lists of abstracts of grants awarded to
schools of edVcation between 1978 and 1982. From this intorma-
ti on, the number of grants "for each education' program .as well as
the doljar amount was extracted.
Correlations between funding data and other i n d i c e s a r e .;'
P,jndicatielPf In Table 7. Higher. corpse] apt i ons are apparent for the
,0:hu,mbe"r ,qpier4.s (first two rows of the table) as opposed to
j ASS
- -; ,
d.,011,af ianiatiqXs..:.(1 a st two rows). (s,- mentioned previously, there
was:di.ff$.6ultiy- obtainir0 complete information on the amount
f -the :award s.;if.,;..../`... .
Clearly4i,t-aj on rates correlate better than publication4.,-..
couhts or' XE.Wpres'entati ons with- funding data. If. the case ismade"Olat more spoducti ve fa cul ty ° aoe better ab.l e to obtain
grants, thien',.the'se riptrelations. enhance the validity of. .
citation eovnt5 'as an,
i nde,,R af,.Ischolarly work, particu.larly in. .
comParisOiri,;:'.w.it publication data and AERA. conference 'Partici-
patj any 1),p that more cited scholars write stronger grant 41'
\\1
proposal-5, or<<thast'thel`r reputations influence the decisions
of the grantinCagdncies.'
\
at 4
.
13
Institutiohal Characteristics. .
The Higher1
Education General Information Survey (HEGIS) was
,the :source for data on five variables: percentage of doctorates
k,
'granted, total research expendFtures1 total overall'expendi-
tures, gownment grants, and contracts per FTE? and average,
faculty. salary. ;Alf_of these, variables were measured fort the
entire univerSitY,pot at the school of education level, and
reflect'either 1978 -1979 or the 1979-1980 academic year. fe
Table 8 shOws-correlations between institutional charac-
teristics and four sets of variables consideredin this study:
citation rates,, publication data, AERA presentation ranks and
funding information. These. coefficients show' that data reflec-
tive.of the university As a. whole was related 'to department-
level indicators of prodyctivity. This was.tHhe for the schools
of education in this Study across all measures. Perhaps
universities which command greater resources., and hence can
afford larger salaries and expenditures, are able to attract
more productive faculty.
4s with the SSLE funding data, institutional variables,
especially 1,..xpendi.tures, grants and contracts; and salarieS,
tended td correlate best with'citation rates. Although the halo
effect majf, have been an issue with these data, this does not
explain why citation rates are the strongest correlates. A more
likely explanation maybe that these institutional variables
a. direct or indirect influence_on departmental productivity, and
1'
the numper of citations were a better reflection of this.00
O
4
D.
s
N
r.,
14
Conclusion
a
Few would doubt the importance of scholarly roductivitY
university, life; yet obtaining measures suitable :r making
comparisons amongwunivers,ities iliay be difficult pnd.controver-
sial. This, study. provided comprehensive data on several
relevant variables for A selecte4-d sample of schools of educa-
tison. Aelationsdlips among these measures .suggest'several points.
related to construct validity.
First, the tendency for prestige survey results to
correlate more highly with meansorathec than with totals on
other measures was an indication that the program size was not a
silnificant.determinant of pre-Se: quality was more. impo'rtant
than quantity.4
Second, surveys had an-annoying Cendency to s how inconsis-
ient relationships with one another. Although the small number
of,zschools ranked in some studies, as well as differences in
when the assessments were made, may have contributed to Ulis
Troblem, the reliability of surveys on reputation was4ot
evident in_these data.
Third, 'the moderate Correlations found among the various
measures indicated.considerable shared variance and, perhaps,AL \
justification for. thinking of schonanly productd vity vi,n schools
of education as a''unitary concept.
Finally, citation analysis, a reliable process with a1
compelling and Logical basis, produced data whilich .showed a
stronger re1ationship than did any other measures with funding
4.
4.
15
and' institutional variables. The evidence in this. study implied
that *citation counts are most central to the concept of.
fprodu.aivity: Although all measures are somewhat flawed,
particularly when used to evaluate indiNidua) careers, the
citation rate may* be the best single measure 'of scholarly work.
e
I
rr
d.
ti
14.
:
v
REFERENCES .
. fa
Anderson, R. C. Narin, & McAllister, P.7- Publitation,ratings at Journal of.t.he American Societz.for,WorMatio Science, 1178, M-1)-10:101.
Astin A. W , & Solmon, L. C. Are reputational ratings neededto measure quality. Change, 1981, 13, 14-19.
Blau; P. M.J, & Margulies, R. 7. The reputations of America.pprofessional schools. Change, 1975, 6(10),'42 -47.
.The Cartter Report on the leading schools of education, raw, and.
business. Change, 1`577, 9(2,), 44-48..
Cochran, W. G. 'Samplingrtechniques. New York: John Wiley and' Sons, 1953.. .
Faculty size in institutional proa'uctivity. Schoolof Edueition,.bn-FieT'ilIy ofPennsylvaTra,1§81.
Eash, M. .Educational research productivity of institutions ofhigher education. American Educational Pesearch Journal,1983, 2Q(1),
Endler, N. S., Rushton, J. P., & Roediger, H. L. Productivity. and scholarly. impact (citations) of Briitish, Canadiah, andU.S. departments of'-psychology (1975). American
tsno9.91q, 197a,.33, '1,064 - 1,082,
Hagstrom, W, 0. Inputs; outputs, and the prestige of univer-sity science departments. Sociology of Education, 1971,,44, 375-397.
Kroc, R. J. Using citation analysts to assess scholarlyproductivity. Educational Researcher, 1984, 13(6),17-22.
Ladd, E. C. & Upset, S. M. The 1977 survey of the Americanprofessoriate. Chronicle oftilhfs Education, 17(18),7-8.
93Schubert, W.'H. Contributions to AERA annual pro.9rams as an
(
indicator of i'ns itutjonal productivity. EAucationalResearcher, 1979 8(7), 13-17.
,Sieber,, S. D. The organization of educational researcW. NewYork: Columbia 'University, BureaTWTApplied SocialResearch, 1966. .
am.
Solmon, L. C., & Astin, A. W. Departments %Jithout distin-guished graduate programs. Chan.91, 1981, 23-28.
..
4
NR
r
.Walberg, H. J. University distinction in educational research:An exploratory turvey. educational Researcher, 1972,1(1), 15-16.
West, C. K.. Productivity ratings of instituti,QA based onpublication in the journals of the American EducationResearch. Association: 1970-1976. Educational Researcher1978, 7(2), 13-14.
ti
'4."?sil.
If
4
4
w.
.
4
..1
Ar
Table
List .of Universities
University of ArizonaArizona State bniveFsityBoston UniversityUniversity of California at BerkeleyUniversity of talifornia at Los AngelesUniversity of California at Santa BarbaraCat lic)Universityn varsity of Chicago
City University of NewyorkUniversity of ColoradoColumbia UniversityUniversity of ConnecticutCornep. UniversityUniversity of FloridaGeorgia State UniversityHarvard University
,,
University of HoustonUniversity of Illinois: Champaign ,UrbanaUniversity of Illinois: Chicago-CircleIndiana University,University of IowaUniversity of KansasUniversity of MichiganMichigan State UniversityUniversity,of MinnesotaUniversity of MissouriUniversity of Montana
. University of NebraskaNew York UniversityUniversity of North Carolina: chip*i HillNorthwestern Un,,ivetsityOhio' State University:University of Oregontniversity of PennsylvaniaPennsylvania State UniversityUniversity of PittsburghPurdue University..UEINWEsity of RochesterRutgers University
. University of Southern CaliforniaStanford UniveriitySyracuse UniveriityTemple University(University of TexasPeabody-Vanderbilt' University 4University of Virginia.Virginia Polytechnic InstituteWashington University
; Unfvertity. of WashingtonUniversity of Wisconsin
Table 2
Correlations Among Other Studies Producng Rankings of Education Program* and Citation Path
a
4. 'V iiII Ili t
^ ...I4 :s. .-.S.
*V
'0 ....l 0
w 43 ..C a .1.. o--
t- a .04-1 .. 44 .4 a '.0. .. - .a.a 0 ..I
41 - i ' i P:4
O.0
0,.1% i i , ..1. I a
Mean Citation*Mean Citation RankTotal Citation Plank
ercent'o1 raculty 141tillo CitationCarttar 119771Slam and Margulies 119751Ladd and Lipsett (19791:Siabor-11966)Skilful'', 119721
Schubert 119791pole 119011
M*Al 11979)t ask (190)1 AEA Totalg ash 119m1) ADA Per r?EEash419011 Total Solicit,*Kash 419011 Article* Per rTE
1.
. . .
4
-.7S n,S8noS1 .69
nS1
. i -,.
-.77* .79'
.47
rp*S1
BEST COPY AVAILABLE
-.79 -.40.8) .11
.54 .17
.06 .19n.14 .51
n.12
2 0
M1ek
M
2
1 I
a-.22 .J7 -.45 .04 -.11 -.10 '.01 , .09.07 :19 .47 -.06 .,40 .2$ .21 .65 -,;34*. 4 1 - 14 .17 .41 .7111 '.65 .04 .31 .42.11 .75 4 4 .11 .SS
*.S9 .09 .43.
.14 ..61 ' 11.01 .51, ...05 ..71 Ofq .41 :411'
.47 .06 .05 -1.21 .01 *.. ***06 Ao.n.1S .49 .26 q.0.24 .17 .7
' n110 . 9 6 $4 ,:27 .52°1 :65 .t 0,2 .21 .'.'
.19 Ob,))1!. osaA; V 70 .03
44e 1' 'al) 1.20
.427
os24At2*.
r
a
Table 3
Intarcocrolatidls at Publication Measure, or
Alt
EI
2
2
C
2
a
2A
a.7.; «,0
r4 4. .4
.0
I
Total ERIC Publication Rank .33 .76
Par Eacplty ERIC Publication Rank .47 .39 .63
Mast (1978) Total Publication Rank. ..49 .30 ,
'Bash (19931 Total Publication Rank .19
Lash (19931 FTE Publication Rank
x ,
. I
Table 4
Correlations &Mouton Publication 14saauta and (ahoy ..11enkin9a S.
J;
*-k
J
et A 4-4v-1 00
I.- .4
etr1. ......
,
'.......
S 11 a ta ..-...
.......
Iri$7-r7 ,
Crb 1 00at0-4
V 1..... p4..01.11 14 1 I 01
/ 417.1 M
4$ 0 Ni
C;t
i C.4 1 I N -.4
USill
-0 3g
Total ERIC Publication Rank -IlPer Faculty ERIC Publicationul
!lank .34
Moat (1978) Total Publication fsank .25
caontash 103) Total PublitiRank
19..06
-84tilt (1983) FTC Publicationsank .39
.09.
.45
.24
.08
.54
.31
-.12
.20
...49
.21
.11
.11
.05
-.40
.51
..-.06
.10
-.111
-.48
-.22
51
.47..
.24 .
17A .
.38
.00...
.31
'.21"
.04
.37
`.27\---....
.. 55-.
.66 .-.111,..
.48 .60
4
.31 .68
.83 4 .36
..03
.60
.31
,126
.36
.
.000.000-*---
22
.
.42 -.27. Lz
-.84 7 .51
.46 .20
. .02
.. L 1: 7 . 4; ;i
-44
. 1
:
.411
:
v
1
, ..1.1
J
-4.
.$
4Table $
inturcorrelatione.Among ARM Presentation Measures
ARRA Rank Per Faculty
V.
s,
.ah-.,r;--.---.=s. , v
oY.
u. .... 1t
a : 1...
. a- .4 a
I e !4 1. g
Aig 1
ARM Total Rank
Sash AERA Rank Par FETEA,
Rash ARRA Total Rank
Dole AERA Rank Per Faculty
Schubert ARM Total Rank
' #..A
234
.66 '.014
. '.79
.09
.71 .26
.20 .83
.70 .09
.11 .83
.19
I
,
i
."1
p
. I
I
r--
BEST COPY AVAILABLF
Table 6
4
Correltinp. between IWRA.FropentoltionAleiimireio and Other Rankinga
3.4
.n.
wl
AERA Rank Per Faculty .56 .20 .64 :10 .60 -.20 '..35 .42 ..55 -.06 .26 .37
ARIA Total Rank 06 .57 .34 .63 .41 .39 *.19 .55 .01 ...29 .46 ,41 .
.Each AERA Rank Per PTE. ,.65 ,04 49 -.35 .51 -.27 .59 .20 .67 -.05 -,.04 .52
Each -AERA Total Rank .21 .65 .42, .S .01 .42 -.44 446 --.31.,-.26 .17 .27
-Dol *ERA flank Per Faculty : .40 -.10 -.SS 426' :36 .0) .60 .31 .53 #01 .26 .SE
Schubort AERA-Total Rank ,.06 -.41 -.11 .v...60 -.36 -.55 -.13 -.60, .03 .21 --..311 ...49
.66
.53
.72
.25
.65 _.,=
.57 -
et.
Table 7
Correlations Retween Funding Data and Other Measures of Productivityfor 51 Schools of Education
t0z
4.1
Id614
0
01 U)
4,4w P
'VP.O
.14
-N1 4
c a
1 .0
tl .4-I-0
0
OS
.C.0 .1-1
-.10
-.0Id
04U
4.)-.I Al -41
0-14 -el 4-I U . 44
P.
.0 :1:13
IXor-i
4u r-11 P.
- AI.44-4
C.)-1 H 4.1
0
, ........ .4, , 7.1
5Ncr.co CO
11 U 4-1 1,-.4iti.... 4.....
isS4.0
E-4 04ra
111
P.>40 r5a.4a. g1 r-.VI gg g A
g 114
4.1
Total Numberof Grants -.55 -.71 -.49 -.44 -.35
1
Number of GrantsPer Faculty -.65 -'.47 -.77 .01 -.45
Total Numberof Dollars -.38 -.18 -.22 -i,21
Number of DollarsPer Faculty -.55 -.25 -.45 .02 -.40
-.43 -.09 -.38 -.34 -.54 -.08 -:39 06 -.48
-.08 -.3 !-.18 -.53 ..30 .r..46 -.09 -.34
.4.09 ,31 .22 -.39 -..11 -.48 08 .0.21
.22 ...25 .23 ,-.57 -.39 -.56 .-.182.-.47
P.
:34
a.
Tata* 8 /
I.
Correlations between Institutional, characteristics and ileastarlts of Productivity for M Schools of Plue.ltion
2 .
13 ac
II al iI 1
..1
2 ../3 4. a
V / 8 §.. a0
g . a J3 3 .1 . I
r . . 1. . ....
1 1 3 ./ 4 2 4,
x., 2
2 Om *4
1 i ... II E 1.. e e5 ...3 &
. ...... Z' -:;-;
$-
(.10 3 t u 4:14
.-4....
0.4,0....
.i140
1-6
44. 6
La I44WI
. .
Percentage ofDoctora t41111 7030
Total Nesearch .11Egpenditi.lio -.SS -.30v1- .06 -.21 -.OS -.32
3
1
-.14 -.44 -.0.! -.4, -.1.1 7 *Ai
.. ' Total 'Expenditures -.60 '-21,. . ..(.,0 1 .1 - .20 0% :,,. t.,,,.''.,' f". Government Grants and,. .' - ..
Cant cacts..Por FTC ov-i-Slit -.2411 ' -63 ..411 ..16Avorage Faculty ,
Salary ' 6..41i . -.56 ...II -...10 ,-.43 .l0 ..-.S240
, e ..-
0 r-.4 -`10"4.4.4-4 - "
VPIP
t42§Ssi4t....
4.
. s
#;u44
, . , ,n.
, . . .
. .
;
I
1a
.23 241, ''. .i7
7. 4 -.33 wall ...34 ...III *. .36 .36-..4.I . .12 *,..14 as , .**.35 -.02 . .33 .44
*0,13 ,...39 .Al .7.30 .02 -.34 -...04 .lit) .41
-.44 ".3 . i . .40 -.4'1 .44 ......1111 ...At! ..42. .113,;
.43.
.
+3.
. ."
cf. . .
. .
: -."
'
, .