artigo crowley

8/6/2019 Artigo Crowley

1/44

Resampling Methods for Computation-Intensive Data Analysis in Ecology and Evolution

Author(s): Philip H. CrowleySource: Annual Review of Ecology and Systematics, Vol. 23 (1992), pp. 405-447Published by: Annual ReviewsStable URL: http://www.jstor.org/stable/2097295 .

Accessed: 11/07/2011 18:37

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless

you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you

may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=annrevs. .

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed

page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

Annual Reviews is collaborating with JSTOR to digitize, preserve and extend access toAnnual Review of

Ecology and Systematics.

http://www.jstor.org
http://www.jstor.org/action/showPublisher?publisherCode=annrevshttp://www.jstor.org/stable/2097295?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/action/showPublisher?publisherCode=annrevshttp://www.jstor.org/action/showPublisher?publisherCode=annrevshttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/2097295?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=annrevs


2/44

Annu.Rev. Ecol. Syst.1992.23:405-47Copyright 1992 byAnnualReviewsnc. Allrights eserved

RESAMPLING METHODS FORCOMPUTATION-INTENSIVEDATA ANALYSIS INECOLOGY AND EVOLUTIONPhilipH. CrowleyCenter orEvolutionarycology, . H. Morganchoolof Biological ciences,Universityf Kentucky,exington,entucky0506-0225,ndNERC CentreorPopulationiology,mperial ollege t Silwood ark,Ascot, erkshireL5 7PYUnitedingdomKEYWORDS: bootstrap,ackknife, onte arlomethods,ermutationests,andomizationests

INTRODUCTIONThe advent ffast, elativelynexpensivethus,widely vailable)microcom-putersstransforminghewayweanalyze ata necological ndevolutionaryresearch.Even more profound, owever, re the associated changes nquestionssked, mpirical ethodssed, tudiesonducted,nd nterpretationsoffered. ow that n array f computation-intensivetatisticalmethods snewly vailable or eneral se, t seemsparticularlymportanto assess theiradvantagesndlimitations,o notehowthey recurrentlyeingused,andthen o considermplicationsor hefuture.I focus n thisreview n four elated echniques nown n the statisticalandbiologicaliteratures randomizationorpermutation)ests,MonteCarlomethods, ootstrapping,nd the ackknife. refer o them ollectivelysresampling ethods, ecauseeach involves aking everal-to-manyamplesfrom he original ata set (randomization,ootstrap,ackknife) r fromstochasticrocessike he ne believed o havegeneratedhedata et MonteCarlo).Each of thesemethodss actuallyn extensive amilyftechniquesandspecific pplicationshat annot e thoroughlyxamined ere; nstead,brieflyharacterizehefocalmethods nd then urveyhe ecentiteraturenecology ndevolution o identifyhe ssuesmost requentlyssociatedwiththese echniques.temergeshat esampling ethods re wellrepresentedn

4050066-4162/92/1120-0405$02.00


3/44

406 CROWLEYdata analysesrelated o some of themost mportantssues and intensecontroversiesurrentlyn these ields f research.The specific bjectives f thispaper re:

1. to acquaint wider rray f ecologistsnd evolutionaryiologistswiththeseuseful echniques, hich, t leastuntil ery ecently,avebeenunderemphasizedr gnorednstatisticalraining;2. to documenthe ssociationetween ertainesearchuestionsndoneor more f these esampling ethods;3. to emphasize herole ofthefocalmethodsn expandingherange ffeasible xperimentalesigns nd nshiftinghe onceptualasisofdataanalysis;4. to compare and contrast esamplingmethodswith more standardapproaches,otingssumptionsndother ey eatureshat earon theirappropriatenessor articularpplications;nd5. tohighlightethodshat eed larificationnddevelopment,nthehopethat hesewill soon be addressedy statisticiansndbiometricians.

The reviews intentionallyiased oward cological tudies,n accordwithmyown research xperiencend nterests.impler, nivariatenalyses reemphasizedn the nterestf clarity ndalso becausea review fcomputa-tion-intensiveultivariateethodsnecologys nprogress165). To respectpage limits n contributionso thisvolume, have restrictedhenumberfexamples ited andemphasizedmorerecent apersmost ikely o containadditional itationsf relevant ork. assumehere hat eaders refamiliarwith udimentarytatisticalonceptsnd basicmethods.The presenteview roceeds s follows: irst, describe rieflyhefourresamplingechniques,ncludingrelativelytraightforwardxample f eachfrom he iteraturefecology nd evolution. ext, summarizeesults f asystematiciteratureearch or pplications,ncluding computerearch fbiologicalournals ndedited olumes ublisheduring 985-1990, ndmyown search y hand hroughll issuesof twoprominentcological ournalsfor he eriod 985-1991.Resamplingechniquesreusedtotest or emporaltrendsn theuse ofthesemethods nd fordifferencesnfrequencyf usebetween cological nd evolutionarytudies. ublicationsdentifiedn thesearch reclassified y topic ndsubtopic, romwhich re distilled evenmajorssues onsidered ith xample pplicationsn more etail.Focusthenshifts o the relation f resampling pplicationso classical and activelydevelopingtatistical ethodology.inally, discuss dvantages, isadvan-tages, nd mplicationsf thesemethods, ighlight ethodologicaluestionsthat eserve ttention,ndclose with ome pecific ecommendations.


4/44

STATISTICAL RESAMPLING METHODS 407BRIEF DESCRIPTIONS OF THE FOCAL METHODSLucid descriptionsnd examples f thesemethods nd computer rogramssuitable or mplementinghem re available n recent ooksby Edgington(64), Noreen 186), andManly 163).RandomizationReferringo theuse of randomizationests n analyzing ata, R. A. Fisheronce claimed hat tatisticalconclusions ave no justificationeyond hefactthat hey greewith hosewhich ould have been arrived t by thiselementary ethod"75). Ina randomizationest, he hance ftype errorunder henullhypothesisi.e. the -value) sdeterminedyrepeatedandomassignmentf he ata otreatmentevels.Thep-valuessimplyhe roportionof ll data rrangementsieldingest tatisticst east sextremenmagnitudeas thevalueresultingromhe rrangementctually bservedsee Figure1).When henullhypothesissthat he bservedmagnitudefthe est tatisticis say)not argerhanwould e expectedy hance a one-tailedypothesis),then he extremealuesto be countedn calculating hep-valueare thosegreater hanor equal to the observed est-statisticalue. When the nullhypothesiss that heobservedmagnitudef the est tatistics notdifferentfrom hance xpectationa two-tailed ypothesis),hen eparate ounts remadeof valuesgreaterhan requal tothe bservednd ofvalues ess thanorequalto the bserved; he ower fthese ountss doubled nddivided ythe otal umberfdata rrangementsoobtainhe wo-tailed-value subjectto the onstrainthat < 1).Data resamplingequires ooling ll data from hetreatmentevels i.e.experimentallystablished r "observed" roups) o be compared nd thenreassigningata randomlynd without eplacemento thetreatmentevels,keeping henumberfobservationsertreatmentevelthe sameas in theoriginal ata. In some cases, all possibleredistributionsf data amongtreatmentevels anbe readily btained,esultingnan "exact" andomizationtest. In othercases, generallywhen the potentialnumber f differentredistributionspproaches r exceeds104-10 , some of these often bout103)aresampledwith eplacementor he test,which s thenknown s a"sampled"andomizationest.Randomizationests reoften asedonstandardest tatisticse.g. t,F); itis thusthe method f resamplinghe data and of calculating that redefinitive,atherhan he tatisticsed.But he otentialouse special-purposeoradhocstatisticss a particularlymportantdvantagef the andomizationapproach andofresampling oregenerally),incethismay ncrease hestatisticalower oaccept herelevantlternativeypothesis64, 163).


5/44

408 CROWLEY

nput

ComputeStatistic

tatisticAs Extreme Or MoreNO So Than ObservedStatistic

YESAdd 1 To SampleCounter Data

F-ina N(*n) NOYES

P-Value * Counter TotalDivided ByNumber of Samples (n)

Output(P)Figure Flowdiagram fhypothesisestingia randomizationests,Monte arlomethods,ndsometypes fbootstrappingechniques. xcept or xactrandomizationhen ew amples reneeded, hisogic susuallymplementedsing computerrogramhat eneratesndprocessesa largenumberf amples typically 1000);only udimentaryrogrammingkills reneeded,unless he tatistics particularlyomplex r thedata tructureequires sophisticatedamplingalgorithm.ee Edgington64), Noreen 186), and Manly 163) for xampleprograms. hisdiagrams a slightmodificationf one on the over and Figure , p. 51) ofE.S. Edgington'sbook 64).

The basic rationale orrandomization ethods s thatunderthe nullhypothesisf,for xample, o differenceetweenreatment-leveleans, nyof he ossible istributionsfdata mong reatmentevels sequally robable.This equiprobabilitys assumed to follow from i) random ampling fpopulations eing omparedcontemporarypplicationsenerallyvoid thisassumption,utthosethat nvoke t areknown s permutationests), ii)random ssignmentf experimentalnitsto treatmentevel, or (iii) fornonexperimentaltudies, implyaking hedata obe "exchangeable"monglevels nthe bsence f treatmentffectssee 163, 250). However, andom-


6/44

STATISTICAL ESAMPLING ETHODS 409ization tests of differencesmong means are sensitive o differencesnvariances ndothermomentse.g. see 15, 219, 242; contraryo assertionsn64, p. vi, andothers), mplyinghat oneof the numeratedssumptionssstrictlyufficient.n lieu of defensiblelternatives,sufficientssumptionsthat bserved istributionsfdata re denticalxcept or he eaturesctuallycomparedn the est. ee Table 1.For example,consider he studyby Loreau (156) on temporal ichedifferentiationn carabidbeetles.Loreau was interestednwhetherpeciesshiftedheir eriods factivityeasonallyuch hat iche verlap r anotherindex ermedmean ompetitiveoad" sreduced, s might e expected romcompetitionheory. he dataconsisted fbiweekly ctivityevelsby speciesand habitatcorrelated erewith uccessionaltage)over four-yeareriod.Subject to some constraintsn the timing f peak activity nd on theboundaries f the active period hatwere ntended o preserve iologicalrealism, heobserved emporal istributionf activity ithin species wasseasonally hiftedt random, nd thetwo ndiceswerecalculated or achspecies-habitatombination.oreach species ndhabitat,his rocedure asrepeatedystematicallyi.e. by exactrandomization) hen he otalnumberofdistincteorderingsas less than 000, orrandomly ith eplacementfreorderingsi.e. by ampled andomization)000timeswhen he otal umberofdistincteorderingsxceeded4000. Thep-valueswere determineds inFigure .

Resultsdifferedmonghabitats, mong onstraintsmposed, ndamongspecies ubsets onsidered.nthe eechwood ndpinewood abitats,-valuesfor oth esponsendices ended oapproachrachieve tatisticalignificanceas increasinglyevere ndrealistic onstraintsere mposed ntherandom-izationprocess. n thesecases, niche overlap nd meancompetitiveoadcalculatedfrom heoriginaldata were lower than95% or more of thecorrespondingalues generatedrom herandomeasonalshiftsn activitypattern.t sdifficulto magine owthese ypothesesouldhavebeen estedwith these data by standard tatisticalmethods.This exampleand thecontinuingebate verthe nterpretationf suchcarabid ata 53, 266, andtheir eferences)ndicate omeofthe hallenges hat an arise nattemptingtooperationalizehenullhypothesisndspecifyhemost uitable andom-ization lgorithm.evertheless,heoverall atternfstatisticalignificancein thepresenttudy oes suggest ncreasinglyistinct ichedifferentiationfrom uccessional o "climax" beechwoodforest, s would be expectedaccordingocompetitionheorynd someother ossible nterpretations.MonteCarloIn MonteCarlomethods, particularandomrocesse.g. binomialoinflipsora complex tochasticimulation odel) sassumed ounderliehe bserveddata ndeterminingonfidencentervalsr the xpected esponse nder he


7/44

410 CROWLEY

E m sQ sQZ ZZZ Z

i :g .s >z z z z z z X

C~~~~ Z ZZ Z Z

2 v Q Q Zz z > 9X

R < 8 ' , w t g _ ~~~~~~~~~~~~s4) 3 Z 't E E Q 't 3 Q )2 O 't H c;


8/44

STATISTICAL RESAMPLING METHODS 41140. 010

0 .0 CZr, 00 CZ0 v0)0 .0 cd - wCZ o 030 C's03 j--Q CZ cn c- cnO O

W O 0= U CZ 0 C'sli w =0 0 053 C'S cn cnto 0 4. 'ei a'o 10 SZ(n 0) 0co > 0w 0 tnC's C,3 wetu 0 Q) an) 4 C'S St10 0 V) C's-=1 U C13 w w0 a) z WO 00 c cw,3 ) 0r. s oo to0) 0 C's 4.SZO wa) 0 = 0) 10& 0) z: .0 o CZC13 w w'6 100. 0 'nosot =1C,3U >.a) C,3 E C,3 0.0 0. r 000 W cd ww C's, Cd >(n 0 oi 03cn CZ a) O03 4;;4 0 W,3 J-- W 14 - - CZ wC c 6 - CZCZ 0) 0 0) 03 Iz . Ir. 0O) cn U C's 0 =Ei .- 0) oi w >0 - Q)

c,3 10 w o ,r.1-)> > C13 UCZ4 > 03 CIS CIS0

C'3 -c boZ .0 C,3CZ QL.O - E 1-1 . w = C'S 0 C,3r. " V) tn 0 0 0 W CWI 0CO 03 U m - o o -030 0) 00w Cd s tn > m X 7R 0)"nn w 00 Al -3 -n 1-cd U O., U CZcr = j 4. cl0U ct 42 10 CZ 2 04-4W I 0 s 0 0 cnW- 0 U.4m., O .2 gCZ C;E-4 0 = 0 w g w .- -5. CZr= CZ 6 0 g:1 0 w V Ec: r. Z c- x'V cr 4 cwlO 'c 'O'- QJCd >, 0. 0. a, - = w1O cl.- .- b >, =cn c CZ0 0 WW 0W U 0 CZ W0 'a cr W 0n 10 W a) cd0 0 W;G W U I!.,2 : .1.2 CZ :2


9/44

412 CROWLEYnullhypothesisorhypothesisesting. his random rocess s then ampledrepeatedlye.g. many oin flips re simulatedrthe imulation odel s runmany imes),with est tatisticsalculatedneachcase. Forhypothesisesting(e.g. is a particularoinunequallyikely oproduceheads" r tails"? or-do empirical bservationsifferignificantlyromhemodel'spredictions?),the -value s found rom his requencyistributionf test tatisticsxactlyas for randomizationsee Figure1). In fact,randomizations generallyconsidered o be the pecial ase of MonteCarlotests n which herelevantrandomprocess simply amplesthe distributionf test-statisticaluesassociatedwith quiprobableearrangementsf data among reatmentevels(163). Nevertheless,omedifferencesn assumptionsndrestrictionsrise ncomparing ore ypicalMonteCarlotests i.e. those n which heobserveddataare notusedto mplementherandom rocess) nd randomizationests(see Table 1).MonteCarlo methods re oftenused to generate onfidencentervals(whereas this is possible but uncommon nd usually cumbersomewithrandomization-e.g.ee 163,p. 18-20). Though ot articularlyifficult,hisis procedurally ore complex hanhypothesisesting ecause it requiresaccumulatingndmaintainingrderedrraysfextremealues fthe tatistic(correspondingo the tails of the distribution)s these re generatedseeFigure 2, which llustrateshe"percentilemethod";more"adventurous"methods, ith lear dvantagesn some cases, are described .g. in 67 and69; for recentworkon MonteCarlo methods ee 12, 82, 85, and theirreferences).Consider demographictudy f thecolonialgorgonian eptogorgiavirgulata sing projectionmatricesnd Monte Carlo methods o analyzetime-varyingopulation rowth95). Field measurementsf recruitment,colony rowth,nd urvivalor ive izeclasses ver 4months,upplementedbyother ecundityata,wereused toconstruct3 5 x 5 monthlyrojectionmatrices. ach entry epresentedheexpected umberf individualsntherow size class that rose by survival, rowth, r reproductionrom nindividualn the olumn ize class one monthefore.Multiplyinghismatrixby a column ector epresentinghenumbers f individualsn eachof thefive ize classes t the eginningfthemonthrojectedythematrix ieldedthenumbers resentneach size classa monthater.The less complex ftwoMonteCarloapplicationsnthepaper oncernsdeterminationsf elasticityi.e. proportionalontributionsy recruitment,growth, nd survival ates to population rowth ate) usingthe matrixtechniques.hequestion f nterestnthegorgoniantudywaswhetherheobserved atternsnthedatacouldbeattributableimplyo thegeneral ormofthematricesatherhan heir iologicaldetails. f so, then n arbitrarydistributionf nonzero ntriesn thematriceshouldgenerate lasticity


10/44

STATISTICAL RESAMPLINGMETHODS 413

(Bootstrapping) anput Begin (Monte Carlo)

Sample -NO

Compute ~~Orderew Final YESValue Within SamplStatistic Tail Array

Taitr m I ai>Arrayfull? NO Delete LeastExtreme Value

Statstic NOMore xtreme ThanLesExtreme n TailsConfidenceLimitsLeast Extreme ValuesIn Tail Arrays

Figure 2 Flow diagram for constructing confidence intervals via Monte Carlo and bootstrappingaccording to the percentile method. This requires a computer program that uses "tail arrays" tocollect and reorder the smallest and largest values of the statistics generated. If the confidencelevel of interest is 100(1 -a)%, where a is the corresponding significance level, and n values arecomputed to estimate the confidence interval, then each tail array for two-tail limits will contain1 + otn/2 values (ignoring any fraction). Thus if n = 10000 and a = 0.05, then the tail arraysultimately hold the 251 largest and the 251 smallest values of the statistic. Initially, the two arraysare filled by the first 502 values, such that the larger values are ordered in one tail and the smallervalues are ordered in the other. Then each subsequent value smaller than the largest in the lowertail or larger than the smallest in the upper tail is ordered within the appropriate array, and theleast extreme value is eliminated; intermediate values, insufficiently extreme for either tail, arenot stored. After all n values have been calculated and ordered appropriately, the interval definedby the least extreme value in each of the two tails is the confidence interval. One-tailed confidenceintervals are handled similarly. Suitable programs are provided by Noreen (186). With boot-strapping, substantial bias can result from this straightforward approach in some cases (see 67and 69 for some ways of dealing with this potential problem)

patternsndvital ates tatisticallyndistinguishablerom hose bserved. nethousand andom rojectionmatrices ereconstructed ith he same zeroelements s in thedatamatrices, utwith llnonzero lements rawn romuniformistributionanging rom ero to one. (Note that f the nonzeroelements adrepeatedlyeenrandomlycrambled, atherhan rawn rom


11/44

414 CROWLEYparticulartatisticalistribution,hen hiswouldhavebeen classified s arandomizationest.) Some of theobserved atternid indeed eem to bemimickedy the randommatrices; utMonteCarlotests howed hat hegorgonianecruitmentndpopulationrowthates sed as test tatistics eresignificantlytypical fvital ates erived rom herandommatrices. o thetestshelped eparate eneral eaturesf suchmatrices romhe ystem-spe-cific nformationontainedn the data. As in thecarabidrandomizationexample bove, t s difficulto seehow any pproach ther han esamplingcouldhave been, seful ere.TheJackknifeThe ackknife,ike ts ll-purposeamesake,wasintendedooffer rude uteffectivessistance hen more recise ool s unavailable252). It providessystematicmethods f resampling he actual data using relatively ewcalculationshat an often e doneefficientlyn a calculator. he directresults f these alculations re an "improved"i.e. less sampling-biased)estimate f some sampleparametere.g. mean,kurtosis,ntrinsic ate ofincrease)and often f theapproximateariance nd confidence ntervalassociatedwith heestimate. he confidencentervals sometimes sed inhypothesis ests occasionally he ackknifed ata themselves, nownaspseudovalues,reused;see 114and 180 for eviews n theackknife).

Though igher-orderersionsmay ccasionally e usefule.g. 194, 195),thefirst-orderackknifes byfar hemost ommonlysed andproceeds sfollows: uppose hat heparameterf nterest (e.g. the rue tandardrrorof themean for heunderlyingormal istributionfmeans) s estimatedappropriatelyver hewhole ample fmobservationss k.Pseudovaluesassociated ith achobservationarethen btaineds Ki = k- (m- 1)(ki- k),wherek.i s just the tandardarameteralculationwith he th bservationdeleted romhe ample. heexpressionnthe ight-handideof his quationis the ampleparameterstimateminus biasterm, eflectinghedeviationofthe -deletedstimate i fromhefull ample stimate .The mean fthepseudovalues is then he ackknifestimate fK. The difference - Kmeasures heoverall ampling ias of theoriginal stimate (bias can, forexample, distort stimates f population ensity, articularly hentheindividualsre tronglylumpednspace;see67 for erivationsfthe boverelationships).Ignoringhe correlationsecessarily resent mong thepseudovalues,calculatingheir ariance2in theusualway, nddividing ythenumber fobservationshengenerates hevariance 2 mof the ackknifestimate .Now theassumptionhat uch ackknifestimates re based on normallydistributedrror ields heparametriconfidencenterval or he stimate:


12/44

STATISTICAL ESAMPLING ETHODS 415? t*s/m1/2, heret* is the appropriatewo-tailed riticalvalue of thet-distributionithm-I degrees ffreedom.Note that hevariance stimate equires ssuming hat he correlationsamongpseudovaluesreunimportant,ut general onditionsn which hismight e valid havenotbeen established163). Thenormalityssumption,thoughustifiableytheCentral imitTheorem or arge sample sizes, isdifficulto evaluatewith he smaller-samplepplications here t is moredubious. ee Table 1.In a study f nterclutchntervalsndreproductiveuccessof feral igeonsnestingn a building t theUniversityf Kansas,Johnson Johnston131)used the ackknifeo test herelationetween hree election arametersndfourmorphologicaleatures. ver 00banded irdswere ncludedn he tudy;survival nd reproductivectivity ere bservedeveral imes erweekovera 17-montheriod ndassociatedwithmeasurementsfbodymass,tarsuslength, ill ength, ndbillwidth. he threeelection arametersf nterestherewerethe tandardizedirectionalelection ifferentiali), the slopeofrelative itness egressedn themorphologicalrait; he tandardizedtabiliz-ing selection ifferentialC), forwhich positive alue ndicates isruptiveselection nd a negative alue mplies tabilizingelection;nd the tandard-ized directionalelection radient,(), resultingrommultiple egressionfrelative itnessn the combinedmorphologicalariables.Though hiswasnot xplicitlytated, heparametersere resumablyonsideredignificantlynonzerowhen the two-tailed onfidence nterval ailedto includezero,followingheprocedureor onstructinghe ntervalhats outlinedbove.Results ndicated highly ignificantirectionalelection ifferentiali)andgradient,() for emale odymass, nd a significantirectionalelectiondifferentialor emale ill ength,nterpretedere s correlatedelection. husfecundityelection elated ointerclutchntervalpparentlytargets"emalebodymass. nthis xample, or arametersfunknowntatisticalistributioncalculated ver hewholedataset, nly heackknifendbootstrapnd theirclosekin ouldreadilystimatehe amplingariationequiredor ypothesistests.TheBootstrapBootstrappings a quite ecent echnique66) thats stilldeveloping apidlyand ttractinguch ttentionnthe tatisticaliteraturee.g. 57,68, 77). Likerandomizationnd the ackknife, ootstrappingocuseson resamplingheactualdatato reveal ome of the ubtler atternsheymply in fact, esultsobtained rom hebootstrapreoftenlosely pproximatedythose romhejackknife-66).Here, hebasicnotions that hedata hemselves,iewed sa frequencyistribution,epresenthebest vailable mageof thefrequency


13/44

416 CROWLEYdistributionrom hich heyweredrawn. hus hebootstrap etaphorefersto thesense nwhich hedata are used ntheir wn statisticalnalysis.Tobootstrapconfidencentervalor statistice.g. mean, kewness,r speciesdiversity)alculated rom singledata set of mobservations,or xample,simply equiresmrandom rawswith eplacementer ample romhe riginaldata, alculatinghe tatistic,ndrepeatinghe rocessmany imes ccordingto the scheme llustratedn Figure2. Here again, this simply pecifiesparticularandom rocess hat echnicallyepresents special case of theMonte arlomethod.Bootstrappinganalso be used nhypothesisestinge.g. 49, 76, 100);forexample,withdata from achtreatmentevel or dataset to be compared)sampledwith eplacementeparately,ests an be formulatedccording othe extent f overlapbetween onfidencentervals r by combining hebootstrappedamples ocalculate test tatisticas inFigure1; see 163, p.28, and186, p. 80). Thiscan still e considered special ase oftheMonteCarlomethod, ut ince eparate andom rocesses re used togenerateheseparateamples rom hich he omparisonsremade, he pproachs quitedifferentrom he sualMonteCarlo pproach. ootstrappingsdistinctromrandomization,hich edistributeshe riginal ata etover reatmentevels,andit contrasts ith heparametricnd ess computation-intenseackknifeapproach. ykeepinghesampling rocess eparate etween hecomparedtreatmentevels, bootstrappinghouldbe less dependent han mostotherstatistical ethods nsimilaritynunderlyingtatisticalistributionsmongtreatmentevels B. F. J.Manly, ersonalommunication;ee Table 1).In an extensivetudy fpredatorndparasitoidelection ressure ngallsize of thegoldenrod all fly urosta olidaginis, brahamsontal (1) usedbootstrappingo avoidproblems ithnon-normalitynd correlatedamplesthat rose nprevious nalyses. election ntensitiesngallsize attributabletonaturalnemy ttackwere alculated s thedifferenceetween hemeangalldiameterf he electedndividualsnd he opulation-meanalldiameter,divided ythepopulationtandardeviation.or each of 20 populationsndtwomortalityourcesi.e. insectsndbirds), he bserved umberf inkedobservationsgall size inmm, urvival rom herelevant aturalnemy s 0or 1) weresampledwith eplacementrom heoriginal ata,the selectionintensityascalculated,ndthis rocesswasrepeated000times ogeneratea two-tailedonfidencenterval,s inFigure . Anobservedelectionntensitywasconsideredignificantlyonzerof ts ower onfidenceimitwasgreaterthan ero. Twoselectionntensities ere onsideredignificantlyifferentftheir 5% confidencentervals id notoverlap; his atterwould be a veryconservativepproach o hypothesisesting,xcept hat he -valueswerenotadjusted or he argenumberfcomparisonsmplied.The many ignificantelectionntensitiesmposedby insectswere all


14/44

STATISTICAL ESAMPLING ETHODS 417positive, nd thefew ignificantelectionntensitiesmposed ybirdswerenegative. aken ogether,aturalnemieshereforeeem o generatetabiliz-ing election n gall size, thoughhe arasitoidffect redominated,esultingin someoverall irectionalelection or ncreasing all size as well.RECENT APPLICATIONSIN ECOLOGY ANDEVOLUTIONOverviewTo determineowrandomization,onteCarlo, ackknife,ndbootstrappingmethodsrecurrentlyeingused nthe iterature,conducted search falarge computer atabase BIOSIS Previews n-linedatabase, 2100 ArchStreet, hiladelphia,ennsylvania9103-1399 SA) for he ublication ears1985-1990 and directly xamined ll issues of the ournalsEcology andOecologiafor1985-1991.During heperiod f nterest,pproximately000 biologicalournalswerebeing bstractedyBIOSIS, apparentlyncludingll majorournals necologyand volution.earchingitles,bstracts,ndkeywords ielded 91 referencesfrom 54 ournals nd11chaptersrom ooks, nce he ew bviousmistakeswere eliminatedy a direct canof theabstracts. reliedon the BIOSISclassificationchemeto draw appropriateistinctionsetween eferencesclassified s "ecology," evolution," rboth. t is possible hat heMonteCarlocategorys somewhatnflatedelative o theothers,incethetermssometimessedfor wider ange f imulation ethodshanust he tatisticaltechniquesf interestere;butthedirect xaminationf ournals ndicatedthat ny uch ffect ouldbe minor.Some results f thecomputerearch representednTable 2. All fourresampling ethodsre well representedntherecentiterature,ithMonteCarlomethodsverall bout wice s frequents bootstrapping,hichnturnwas almost wice s common s eitherandomizationests rthe ackknife.Thehypothesishat esampling ethodsrebecomingmore ommonn theliterature as corroboratedtatistically,hough heevidence o supporthisfor nyparticular ethod asmore quivocalrandomization,onteCarlo,bootstrap)rclearly ontradictoryjackknife)tested yrandomization;eeTable 2 and Appendix ). Eachofthefourmethodsand all taken ogether)was used disproportionatelyn evolutionarytudies elative o ecologicalstudiesMonteCarlo tests; ee Table 2, Appendix ), as suggested ytheobservedroportionsofevolutionarytudies v = 0.144-0.360) relativeothe verall roportionnallpapers ublished0.098).I scannedhemethodsndresults ectionsnd all figuresndtables fthe1485 articlespublishedn Ecology and the 2128 articlespublished n


15/44

418 CROWLEY

1.0 n n s* N

o o o o o o o o o o o Ico o0

o-~~~~~~~~~W) o r- t mt

G ~ ~ ~ ~ ~ ~ ~ ~~~~I W

Z-'' r-n cn W) 0 a OWS Gi


16/44

STATISTICAL RESAMPLING METHODS 4191 'I', 00 10,-

00 > 0 -0O.- B :1 o 0- All0-so taE =wo 'tt :,Z,= :5 g - -00 0 C,0 z 2:..00 o,c=q =0 0 00ou cr O=O= cwOQ t Z 'O ro,A - ' Al0 0 00 E' Z - >0 .0 O, 00 .., O,, -O r 0O 0CZ orb0Cd0 0 A0 r-00 O 01- CZ Ir. r>., -"n 40, 0. m0 Q.CZ m 4..8) RrS= . .2 0 0 O O 'O0 \0 C14 C, 0) :a Z :'Tt CZ > -O cz,

-AlO O O clW) 'No CZ .0 0 A=O co40. 'O O 50r- O.,0toCa

0,_=0 cd O 0 w '= - 0,6 (3 .2 cul .00 .00 C)0 E 00-Ow Z CZ '01 ta V0 Z! u0 -0 , -00 r- 'No r- 0 r- 1:4kn 00 ON 00 O CZ01 01, 7s >, E-W) 00 u O 6C, O c: r.>,O > 0 0

>,.4 N E >00 00 W) .- V Q ,O >C)00 0 oo CS 0 r.crcl 0 u -m O.,4-iO -6 0 0cr0 E U.0 Ooo 13 >,-0 utn cr

o c Stcn B t - w.0 m m - " oloC) -t:; -,Jc7" 01z) 0m C\ ',z = 42, cul2 O.0 O., Qu in. 0ob en 0 W

0 0 ou oo U .- J:L, O0 0 0 0 .- _ O0) :1 = 0 = 0 = 00 ts U .- - -,=t - - O C's O0 0t =>0 O oito C'S O $.0 oo0 65 C'sQ O O00 0 00 >

""g : ;-,,O 63 "a 0-0 0 g w> oC's 0 00 00 wr. Z 079 2 0 o03wo, 2 > u - - ;a .- ou C-13 o-


17/44

420 CROWLEYOecologia from 985-1991. For 1985-1990only, herewere 1270 articlesin Ecology and 1841 in Oecologia.) Regardless f the authors' riginaldesignation,classifiedmethodss Monte arlowhen mathematicalandomprocesswas executed epeatedlyogeneraten estimatefbiological ariation(e.g. a confidencenterval) r to testa hypothesis, ollowing igure 1.Analyses pecificallylassified s binomial estswereexcluded except sexplicitly oted), hough ll were quivalento exact andomizationests seebelow).During 985-1990,87 or 6.9% of theEcologypapers nd 53 or 2.9% oftheOecologiapapersncluded neormore f he ocal esamplingechniques.(In Ecology, llowing or he 8 papers ach using2 of themethods, herewere31 randomization,3 MonteCarlo, 15 ackknife,nd16 bootstrap.nOecologia, with 3 papers each using2 of the methods, herewere 21randomization,2 MonteCarlo, 5 bootstrap,nd 8 jackknife.) he BIOSISsearchwas thus elativelynefficientverall 17/87= 19.5% forEcology,and 10/53= 18.9% forOecologia,)and the efficienciesrobably ifferedamongmethods.This snotparticularlyurprising,ince tatistical ethodsmaynotoftenwarrant entionnthetitle, bstract,rkey words, houghsome maybemore ikely o be mentionedhan thers.) heoverallnumberof ecology/evolutionrticles uring 985-1990thatused theseresamplingmethodsanbevery oughlystimateds the otal umberf rticlesdentifiedbytheBIOSIS search ivided ythemean f hese wo fficienciesxpressedas a decimalfraction, hich o thenearest nteger quals 2036. Clearly,thoroughndcomprehensiveeview f this ndmore ecentmaterial ouldbe overwhelming,oth or eviewersndreaders.Combininghefull1985-1991 direct-examinationatawiththe 1985-1990 BIOSIS resultsndclassifyinghepapersby content enerated able3. Notice hat omeparticularlyontroversialssues n ecology nd evolution(e.g. nullmodels, ize-ratioheory,etectingensity ependence,hylogeny)arewellrepresentedere, erhapsmainlyoexploithe onsiderablelexibilityofresampling ethodsnapplicationsnvolvingonstandardodels ndteststatistics.hisflexibilityan be a mixed lessing, owever,s I notebelow.Thepoor representationfbehavior nd behavioralcology n thetable sprobably rtifactual,eflectinghe eparationf behavior rom cology ndevolution ithinIOSIS.SomeActive reasofApplicationNULL MODELS, COMPETITION,AND COMMUNITYSTRUCTURE Contemporaryinterestncompetitions a mechanism nderlyingommunitytructureedin the 1970s to the formulationf null or neutral)models,withwhichstatisticalests fpredictedatternsouldbeconducted35, 223, 229). Since


18/44

STATISTICAL ESAMPLING ETHODS 421thenotion f probabilisticodel hat an generatehe tatisticalistributionconsistent ith null hypothesiss the essenceof MonteCarlo hypothesistesting, onteCarlotests e.g. 201, 223) andclosely elated xactrandom-ization estse.g. 50 and 247, which eferredo them sing hebroadererm"binomialests")were oonprominentnthese nalyses. he controversyhateruptedetweenhose ormulatingullmodels or his urpose 42-44, 229,247) and those onsiderablyess enthusiasticboutthis pproach 41, 56,91-93) provides cautionaryale: the potential ordifferingull models,misunderstandingsfmethods, roceduralrrors,nd alternativenterpreta-tions fsimilar esults an be highwith esampling ethodscf MonteCarlotests n43, 44, and93; see 262). Nevertheless,henullmodel pproacheemstohavetakenhold n therecentiterature,artly ia MonteCarlo methods(Table 3; see 202, 265).There s muchrelativelyntapped otentialo use resamplingestsfordetectingommunityrganizatione.g. guild tructure:24, 129, 265) andcommunityimilarity118, 263,264), andfor esting hethernvironmentalfactors an account or ommunitytructure29, 83). In assessing peciesdiversityndthevariationssociatedwith hese stimates,he ackknife asbeenusedmost ftene.g. 110,194, 195),but ootstrappplicationse.g. 23)maybecomemore ommon.Much f henullmodel ontroversyas addressedhe istributionf peciesabundance ndpresence/absencen islands 214, 244, and analogous itua-tionsconsideredn 265). A good overview f this ssue and problemsassociatedwithhoosing ppropriateonstraintsnrandomizationsprovidedbyManly 163, p. 233 fF).Numerous ecentttemptsotest ornichedifferences34, 109, 162) andto measure verlap 2, 107) have used resamplingmethods, articularlyrandomization125, 152, 265; see202). In other ases, temporaliche hiftshave been testedby resampling61, 206, and the binomialor exactrandomizationestn50). SimberloffBoecklen's orensicnalysis f SantaRosalia 230) stimulatedeveral esamplingests 96, 271, andthe quivalentof exactrandomizationn 19) of the constant-size-ratioypothesisromHutchinson'sriginal aper 120).Considerableecent orknplant cology as focusedncompetitionromimmediate eighbors27, 136,251) and related istortionfthepopulationsize distribution133, 143, 227). The geometryfaccesstoresourcesndthus fpotential esponse o competitionas also been characterized136,215). These plantneighborhood-competitionnd size-distributionitationsinvolve hegamut fresampling ethods onsiderednthis eview, oth orhypothesisests 27, 251) and to estimateonfidencentervalsor heGinicoefficientan ndicatorf size nequality; 33,227) orto calculate kewnessby ackknifing143).


19/44

Table 3 Classificationy researchopic nd statistical ethodftherelevantapers dentifiedn theBIOSIS survey 1985-1990) andby direct xamination f ournals Ecology and Oecologia, 1985-1991)'MonteTOPIC/Subtopic Randomization Carlo Bootstrap Jackknife

Competitiontotal) 31 36 20 17Null models 3 12 0 0Niche differentiation,verlap& breadth 14 9 4 4Size-ratio heory f nichedisplacement 1 3 0 1Niche-shiftynamics interactionntensities 3 1 1 0Plant ize hierarchies 1 0 7 3Plantneighborhoodompetition 4 5 0 1Communitytructuretotal) 14 16 7 7Detecting rganization 2 5 2 0Diversity 2 2 3 6Communityimilarity 7 2 2 1Temporal ariabilitynd stability 1 5 1 2Detecting ensity ependence 7 4 0 0Spatialpatternsnd processes total) 15 33 3 9Dispersion spatialpattern 6 17 1 3Dispersal& migration 5 7 1 4Scale effects 4 3 0 1Demographytotal) 5 47 10 8Population ize of density 2 19 6 7Vital rates 2 9 4 7Growth,ize & age relationships 1 7 0 0Stock-recruitmentelations 0 5 0 0Agricultural/fisheries 0 8 0 2Environmentalactorstotal) 4 58 10 3Absorption scatteringf light 0 16 1 0Air-quality odels& indicators 0 10 4 0Aquatic nvironmentaluality/toxicology 2 8 3 2Lake & stream cidification 1 4 1 0Surface, oil & groundwater 1 13 0 1Behavior/behavioralcology total) 10 16 2 2Social organization 3 2 0 0Foraging 6 7 2 1Evolution/evolutionarycology total) 61 44 49 19Selectionntensity response 0 10 3 4Genetic ifferentiationcorrelation 1 2 2 2Mutation ates 0 7 0 0Morphometricomparisons 3 2 2 3Phylogeny 5 1 33 9


20/44

STATISTICAL RESAMPLING METHODS 423

Statisticalndmodelingmethodstotal) 41 63 43 44Analysis f variance 3 2 0 1Regression correlation 5 8 0 5Mantel's est 7 0 0 0Discriminantunctionnalysis 2 2 0 5Nearest-neighbornalysis 1 6 0 0Sensitivity,rror uncertainty 0 14 1 3Power nalysis 0 6 0 1Confidencentervals variance 1 4 31 16Bias estimation reduction 0 10 7 8Grand otal 145 330 139 114

'Direct earch fthe wo cologicalournalsocated ll of he eferencesndetectingensity ependencend lmostall of he ompetitiontudies. he statisticalndmodeling ethodseferencesere erivedoth rom IOSIS andfromthe irect earch.Referencesor he emainingopicswere btained lmost ntirelyr ntirelyromheBIOSIS search.Data shown erewere ollected sing lengthierist f topics ndsubtopics,ut hose ategoriesccumulatingewerthan ive itationsre not hown; opic otalsnclude he dditionalitationsrom ubtopics ot hown, nd the randtotal ncludes itations rom opicsnot hown.Manyof thepapers re tallied n more han ne category.An underutilizedandomizationethodfverywidepotentialpplication,particularlyn communitynalyses, s Mantel'stest 119, 166; see thedescriptionndexamplesn 163and165). This flexible echniqueests orcorrelationetweenwo ormore) quare istancematrices.ypically,ntriesin one matrix xpressEuclidiandistances or some alternative easure)

betweensay)species nquantitativeultivariateeaturese.g. indiets), ndthe thermatrix ay epresentpostulatedatternmongpecies e.g. zeroesand ones indicatingmembershipr not n the sameguild).By randomlyreassigningows and columns f one matrix o species,recalculatinghecorrelationetween orrespondingff-diagonal atrix lementswhere hetest tatistics the umof themultiplicativeroductsfthese orrespondingmatrixlements),ndthen epeatinghis equencemany imes, he tatisticaltendencyn theoriginal atafor hepostulatedatterno match hedistancepatternanreadily e assessed e.g. see 198).DETECTING DENSITY DEPENDENCE Anotherontroversyflongstandingnthe cologicaliteratureoncernshe ole fdensity ependencenpopulationdynamics.See e.g. 11, 55, 267, andtheir eferencesor vidence hat hecontroversyontinuesnabated.)wo mportantilestones ere henitiationofexperimentalield ests fdensity ependence70) andtheformulationfstatistical ethodso detect ensity ependencen temporalequencesofdensityata 26). Resamplingmethods aveproven sefuln both f theseapproaches,articularlyhe atter48, 51, 54, 203, 204, 210, 258, 259; anapplicationoanalysis f a field est s inprogress-D. M. Johnson,. H.Martin,. B. Crowder,. H. Crowley,npreparation).Though oncerns avebeenexpressed boutthepotential ordetecting


21/44

424 CROWLEYdensity ependencendensityequences 84), recently evelopedmethods,particularlyhetwo testsbasedon randomizationethods y Pollard t al(204) and Reddingiusnd denBoer 210), appear ufficientlyowerfulo beuseful 48, 51, 259). Recent ariationsn this andomizationheme avebeenused to evaluatebias in k-factornalysis 258), to extend henotion fdetectingensity ependenceo the ommunityevel 48, 51), andto derivetestable redictionsbout hedirectionf densityhanges 48, 51).The Pollard t al (204) "randomizationest" nd theReddingiusnd denBoer (210) "permutationest"both nvolve crambling he orderof theobservedhangesn og-transformedensitya measure fpopulationrowthover he ime nterval)or omparison ith he bserved equence.Pollard tal (204) used the orrelationoefficientetween ensity t the tart f eachinterval nd the associated hange n density uring he nterval s theteststatisticdensity ependencemplies n nverse orrelation;he andomizationtest voids heproblemsnherentn the nalogous arametricpproach otedin 159 and 239). Reddingiusnd denBoer 210) used the og-range etweenthehighest nd owestdensitieseached n thedensity equence s theteststatisticdensity ependencemplies small og-range). ther est tatisticsmay be more ppropriaterpowerfulnparticularases (cf the"violationnumber"tatisticn48 and51). It may ften e helpfulouse several ifferenttests nd est tatisticsnthe amedata et, ince he est esultsre ometimescomplementary51, 259), thoughhismayraiseconcerns bout dequatelyprotectinghe hance ftype1 error ver ll tests.SPATIALPATTERNS AND PROCESSES Characterizingpatial atternsndpro-cessesis a major hallengencontemporarycologicalresearch.A diversearray fresamplingpproaches as been usedfor hispurpose.Descriptivemethodsnclude ssessinghe patial istributionfsparsely ampled ointsandthespatial reasmost losely ssociatedwith oint ocationse.g. treelocations-236 and 136,respectively),ndparticularlypatial utocorrelation(149, 232). Hypothesisestshave been usedtodetect onindependencefanimal ocations237, 248); variationsnterritoryize (249); differencesndispersionmong ize classes, species, ndquadrats105, an applicationfMantel's est);differencesnassociationfplant istributionnd abundancewith axonomicompositions vegetationtructure222, also via Mantel'stest); ndan association etween patial istributionndtemporal ynamics(228). MonteCarlomethods ave mprovednd extendedhe lassicnearestneighbornalysis f Clark& Evans 39; see 33; 151; and 163, p. 21-23 andchapter ).In other ases, geographicalimits f populations ave beenestablished(220), and hemplicationsf patial cale 4, 122,215) andofenvironmentalheterogeneity4, 215, 222) havebeen ddressed.


22/44

STATISTICAL ESAMPLING ETHODS 425In consideringnsect ispersal rocesses,MonteCarlo imulation as beenused to evaluate heneed for stochastic ormulationo predict ispersal(253). Error ssociatedwith stimatesf thediffusionoefficient160, 188)or of the adius fpatch etection103) has been ssessedprimarily ith hejackknife.

ESTIMATING POPULATION SIZE AND VITAL RATES Resampling methods,par-ticularlyMonte Carlo, are now in fairly ommon se to reducebias anddeterminerror ssociatedwith stimates f population ensity111, 173,181). The "smoothed ootstrap"226) and randomizationests 171, 269)havebeenused o detect ensityhanges,mainlynnon-experimentaltudies.Oneofthose astrandomizationxamples i.e. 269) invoked n approachknown s MRPP multiresponseermutationrocedures-176-178).MRPP,a special ase ofMantel's est 163, p. 209), is conceptuallyonsistent ithgraphical epresentationsf the data and readily xtends o multivariateproblems.With hismethod, redefinedroupse.g. sites, reatments)an betested or ifferencessing tandardtatisticalistancemeasures,rom hicha test statistics derived nd thenassessed by ordinary andomizationproceduresFigure 1). Interestingly,tandard andF tests nd commonnonparametricests re special cases of MRPP, though ractitionersrguethat onstandardormulationsre generally ore ppropriate269).Following he omparisonf ackknifendbootstrap ethods y Meyer tal (175), there as beenmuchrecent nterestnmeasuringnd testing ordifferencesndemographicostsofpredatorefense13, 212, 260) andofother nvironmentalactors88, 139), as measured ytheper-capitancreaserateof zooplankton.See 95 for n assessment ftemporal hanges n theper-capitancrease ate erivedromominantigenvaluesf matrix odel.)Resamplingmethods ave also beenapplied otests nderrorstimatesorothervitalrates birth ate:58; mortality:53; relative rowth ate:37;transmissionateof an insectvirus:63; manydifferentitalrates:254),reproductiveffort86), andextinctionate 199, 200).

ENVIRONMENTALODELING As theneedfor eliable nvironmentalredic-tionsndmonitoringas teadilyncreased,broad ange f elativelyealistic,quantitativeodelshas appearedn thebasicandespeciallyn theappliedecological iterature. focal ssue nmany fthese tudies s evaluatinghemodel'sfit odata;for robabilistic odels,MonteCarlomethods re oftenthebest ptionnd havecommonlyeenused.Withregard o aquaticenvironments,esampling as been appliedintoxicologicalmodels 20, 24, 221) and laboratoryests 205), time-seriesanalysis fBOD data 197 via the Bayesian ootstrap"),esting ensitivityof akes ophosphorusoadings28, 150),estimatingn ndex fwateruality


23/44

426 CROWLEY(97), and ssessing he mpact f ciddeposition62, 128, 132).Applicationsin soil andgroundwaterystemsnclude hoseon soil hydraulic roperties(117, 234), estimatingunoff10,60,99), andmonitoringroundwateruality(155, 185, 240). In theatmosphere,esamplingechniques ave been usedwithmodelsof carbondioxideuptake nd exchange121, 138, 268), forimpact ssessment f radionuclideallout 21, 211, 261), and especially nair-quality odels nd ndicatorse.g. 25, 101, 108).Animportantomponentf many limate nd plant-growthodels,bothaquatic ndterrestrial,s absorptionnd catteringf ncidentolar adiation.MonteCarlo applicationsreparticularlyommon n these tudies e.g. 5,38, 94).EVOLUTIONARY PROCESSES AND RATES Resamplingmethods igure romi-nentlynanalyses fnatural,exual, nd group election. xamples ncludethe ntroductoryase studies fthe ackknifendthebootstraparlynthisreview 1, 131), studies etermininghemagnitudefsexual election172,187) and groupselection 78, 102), and others oncernedwith variousresponses o selection142, 182, 257).Rates of evolution ave been assessed and contrastedia resamplingapplications78, 90, 140), as have mutationates 79, 106, 189, 190) andevolutionarymplicationsfgenetic rift189, 190, 217). In an analysis ftaxon xtinctionates,Raup& Sepkoski209 and referencesherein) sedrandomizationests o identifyignificanteriodicityf majorextinctionevents n thegeologicrecord also 116; see 116 and 193 on speciationperiodicity),utQuinn 207) argued hat ootstrappings more ppropriatefor his urpose see 16and theoverviewn 163, p. 192ff).All four esampling ethods avebeenused odetect enetic ifferentiationbetween opulations asedon immunologic225), electrophoretic45, 59),andnucleotide-difference216, 246) data.Discriminantunctionnalysis,particularlyiththehelp of Monte Carlo (225) or randomization238)methods,an woveusefulnsuch tudies.PHYLOGENY Phylogeneticnalysis asevolved apidly ince he1970swithwidespread se both fmolecularechniquesndofcomputerimulationnddata nalysis. ioneeringimulationtudies yRaupet al (208) demonstratedthe possible importancef stochastic rocessesand potential iases ininterpretinghylogenies.omeofthe arlywork nnullmodels mphasizedbiogeographicata e.g. thebinomialrexact andomizationest f247; alsosee 41 on the avoidable ndunavoidable iases in suchstudies), nd theusefulness f statisticallyontrastingroposed hylogeniesgainst nullpatterns becomingmorewidely ecognized73; see thebootstrappproachof89 andan exactMonteCarlomethodn233).


24/44

STATISTICAL ESAMPLING ETHODS 427An importantecent evelopment as the formulationf techniques orestablishingonfidencentervalsormonophyleticroups jackknifingvertaxa: 146; bootstrappingvercharacters: 2; see 196 for a comparativeevaluation f these and relatedmethods).Felsenstein's requentlysedapproachnalyses haracterata containedn a speciesx character atrix.Bootstrapamples f charactersor, strictly,f thecolumns f species-spe-cific values for particularharacters)re used to constructlternativephylogeneticrees; hepercentagef these ontainingmonophyleticrouppresentnthetreebasedon theoriginal ata then stimatesheconfidencethat hegroup s indeedmonophyletic.nherentssumptionshat haracterswere ampled andomlynd volvedndependentlyaise ome oncerns boutthemethod's alidity72, 73, 224) but pparentlyavenotdeterredpplica-tions of this and related ootstrappingpproaches e.g. 126, 127, 270).Bootstrappinglearly redominatesnrecentesamplingtudies fphylogeny(Table3), includingoth ladisticnalysese.g. 47, 145, 147) and he heneticstudiesmphasizedbove.Another ssue of currentnterestoncernswhether articular atternsderivedrom hylogeneticata an beconsidered onrandom.ere,random-ization estshavebeen used to scramble haracter aluesamong peciestodeterminehetherhe ree erived romhe riginal ata equiredignificantlyfewerevolutionarytep-changeshan the trees derivedfrom crambled

characteralues nonrandomnessas detectedn 6 butnot n7; also see ananalogous ladisticnalysisn 192).Statistical ethodologyRELEVANT YPES OFANALYSIS There s much tatisticalndbiometricalresearchn progress ontinuinghe developmentf resamplingmethods(particularlyhebootstrap).Moreover, o a greaterxtent hanwithotherstatisticalpproaches,ach newapplicationends o extend hemethodolog-icalpossibilitiesecauseofthe dhocnaturefresamplingnalysis.Here,note how resamplingmethods ave been used to supplementr improvestandard tatisticalmethods nd to stimulate r enhancenew researchinitiativesswell.Resamplingmethods void some of the more restrictivessumptionsinvolvednstandardegressionndcorrelationnalyses e.g. see 64, p. 197),andthere re nowmany ublishedpplicationse.g. regression:0, 81, 170;correlation:23,249, 256).Theuseful eneralizedorrelationethodsnownas Mantel's test and multiresponseermutationroceduresMRPP) havealready een describednd characterizednthe iteratureummarybove.Analysis f variance eserves pecial ttentionecauseof ts central ole nthedesignnd nalysis fexperimentsndbecause frestrictivessumptions


25/44

428 CROWLEYthat an proliferate ith omplexityfthedesign 191; 64, p. 58 ff; ecentapplications nclude 130, 162, and 168). Help needed from tatisticalresearchersn problems ssociatedwithANOVA is noted elow.Manymultivariateethodsre urrentlyeing evitalizedndextended iaresamplinge.g. cluster nalysis:148, 184, 255; discriminantunctionanalysis:14, 29, 80; principalomponentsnalysis: 41; indirect radientanalysis: 44). t sprimarilyhemethodseadily pplicable o single amples(MonteCarlo,bootstrap,ackknife)hat reof nterestnthis ontext,houghrandomizationan be useful ordiscriminantunctionnalysis 163, 238).See Manly 163 and especially165) forthorough eviewof multivariateresamplingpplications.Many ther tandardssues nd pproachesnexperimentalesign nddataanalysishave been addressedwith esampling ethods. ome of the moreimportantfthese re ssessing rrorsssociatedwith ampling2, 103, 256)or directmeasurement3), estimatinghepower f hypothesisests 32, 87,130), determiningnd reducing ias (2, 173, 183), and determiningheappropriateample ize 22, 154,164).In empirical tudies, he need formethods f analyzing he ecologicalresponse o large-scale erturbations31, 169) has led to some resamplingapplicationsssociatedwith nterventionnalysis Monte Carlo: 158) orrandomizednterventionnalysisrandomizationests: 2). Theapproach ereisusually asedon paired ystems,neexperimentalndone control; ach smonitoredxtensivelyeforeiid fterhe xperimentalystemsmanipulated,so that omeofthese bservationsan be assumed ssentiallyndependent(though utocorrelations directly ssessed). This general pproach r asuccessormayprove aluable, articularlyhere eplicatedxperimentsreinfeasible,ut dditionalare hould e takenoensurehat henullhypothesisis tested gainst n appropriatelternativee.g. by transformingo reduceheteroscedasticityrother istributionalifferenceshatmayconfound hetest; ee 71 and 242).Inmodeling tudies,here s much urrentnterestn ncorporatingge orsize structure36, 174) or explicitlyepresentingndividuals52) withinpopulationmodels. Moreover, ptimization odels now morecommonlyinclude tochasticlements e.g. see 161) or parameterncertaintieshatcomplicatenterpretation.nthese nd imilar ases, resampling ethods anprove articularlysefulncharacterizinghemodel's ehavior nd valuatingits consistency ith mpirical bservationse.g. age structure:37; individ-ual-basedmodel:157; optimization:13).CLOSE RELATIONSHIPS WITH MORE STANDARD METHODS The resamplingtechniquesf interest ere reall closelyrelated o the more tandardnd


26/44

STATISTICAL ESAMPLING ETHODS 429widely amiliartatistical ethods.y virtue fconceptualimplicityndthelargenumber fnonparametricests t has spawned, andomizationan beconsideredundamentalothe tandard ethods17, 46, 134).Transformingdata toranks s primarilydevice oreduce atasets oa general ormhatpermitsonstructionfnonparametricignificanceables,with ntries t owsample sizes determinedy randomizationnd at higher ample sizes bynormal rchi-squarepproximationsotherandomizationesults. he firstthree ommonly sed testsn Table 4 areexamples fthese; he ign est salso a kind frank um est.Fisher's xact est nd thebinomialgoodness-of-fit)est redirectlyalculatedases of xact andomization.he testsistedinthe able re ust a fewof themore ommon onparametricests ound nthe cology-Oecologiaample.MonteCarlomethodsregenerallysedtoderive tatisticalables or estsbasedondata ssumed o follow articularistributions,uch s t,F, andx2tests. ntheMonte arlo ests f nterestere, he ctual tatisticalistributionmaybeunknown,o long s the elevanttochasticrocess anbe simulatedaccordingothe chemenFigures and2. Insome asesamong he cologyandOecologiaarticles, smaller umberfsimulations as used to drawconclusionswithout formal est e.g. 104, 245) orwerecomparedwithobservationssing tandardategoricalests e.g. 55, 141)orparametricests(e.g. 74, 98). Suchhybrid pproachesmayoften roveusefulwhere heunderlyingssumptionsan bemet, ut n several fthese ases,the tandardMonteCarlotestmight avebeenmore efensiblend straightforward.Applicationsfthe ackknifenvolving ypothesisests rdeterminationofconfidenceimits ely nparametricritical alues ndsignificanceables(seeabove).Thoughhe ootstraps notnherentlyied oparametricethods,oneareaof ctive evelopments knowns the arametricootstrap,n whichthestandardrror fthemean s bootstrappedndthen sed inparametricanalysesas with the ackknifee.g. see 67, 186, 231). Of course,the

Table 4 Percentagesf papers publishedn Oecologia(1985-1991)featuringome common onparametricests,all of which re orareequivalento)randomizationests.Test PercentageMann-Whitney 11.3Spearman ank orrelation 7.1Wilcoxonmatched-pairsigned-ranks 4.7Fisher's xacttest 2.7Signtest 1.3Binomial est 1.2


27/44

430 CROWLEYasymptoticonvergence f statisticalampling istributionsn the normaldistributiont sufficientlyarge ample izes s implied y theCentral imitTheorem.DISCUSSIONAdvantages ndDisadvantages f TheseMethodsAn attempto sort ut theprosand cons of resampling ethods elative othemore tandardtatisticalechniquesaisesmany ssues fvaryingubtlelyandcomplexityTable 5). When heir tringentssumptionsre met,para-metric rocedures aximize ower i.e. the hanceof rejecting falsenullhypothesisn favor f a true lternative),or specified ype error ate i.e.chance f falsely ejectingtrue ull) 186). Butrarely t small r moderatesample izes can all of the assumptionse known r convincinglyemon-strated o apply. The conservativepproach s then o resort o standardnonparametricethodsrresampling.Nonparametricethodsregenerallylightlyoconsiderably eaker hanthe strongerf parametricnd resamplingmethods or several reasons.Essentially ll nonparametricechniquesn common se were necessarilydesignedorminimizingomputation.nsome ases, his esultedn nherentlylowpower e.g. the ign est-see 135). n others, lossofpower r nadequateprotectionfthe ype erroratemaybeattributedoreducing ata oranks,approximationselated o ties in rank ests, ontinuityorrectionst lowfrequenciesor ategoricalests, r thepossibilityf naccuratepproxima-tions nsome ables t ntermediateumbersf observations64). Often, he

Table 5 Key features f three ategoriesf statistical ethodsStandardNonparametricFeature Standard arametric ethods Methods ResamplingMethods

Statisticalower High when ssumptions et) Moderate HighKnownbyresearchers Verywidely Widely Sometimes increasingAcceptance Widespread Widespread Common& increasingStandardization Veryhigh High ModerateFlexibility Low Moderate HighAssumptions Moderate-strong Moderate Weak-moderate(see Table 1) (robustosomedepartures)Populationr sample Population Sample Population(except andomization)Time & effortost Moderate Somewhat Higher& decreasinglowerConceptual omplexity High Moderate Low-moderate


28/44

STATISTICAL ESAMPLING ETHODS 431available tables seriously onstrainhe analysesby incompletenesse.g.Friedman's NOVA-by-ranks),yprovidingnly ne-tailedr nly wo-tailedsignificancealues Fisher's xacttest nd chi-square, espectively),r byprovidingnly rough ndicationfthe -value'smagnitudemostnonpara-metric ests) see 64).In contrast,andomizationestsyield bout he ame significanceevel asparametricethods hen heparametricssumptionsre met 115, 218) butmayhavemore ower han arametric ethods hen ata re from on-nor-maldistributions64, p.94; 135). Less s known bout ircumstancesn whichMonteCarlo,bootstrap,nd ackknifemethodsmaybe morepowerfulhanstandard arametricnalysis but ee examples n 67).

Anobvious urrentdvantage fusing tandard arametricnd nonpara-metric echniquess that hey re widely nown nd accepted y editors ndother esearchers,hough esampling ethodsrenow learlyn common seas well. Yet bootstrapping ayhave been swept nto the mainstreamfecological ndparticularlyvolutionaryesearch omewhatheadofa full,balanced valuationf ts apabilitiesnd hortcomings.ootstraponfidencelimits nd hypothesisesting re not always reliable e.g. see 67); somefamiliarityith urrent ethodologicaldvances ndperhaps omead-hoccheckingouldprove mportant,nd use should e restrictedocaseswhererandomizationests ndparametricethodsre nappropriate163, 186).Standardizationndflexibilityf tatistical ethods ust rade ff, o someextent.The psychologicalhift ssociatedwith the conceptually impleresampling pproach,n whichthe data analystnecessarilyontrols ndunderstandsachstepfrom ypothesisormationodesigning sufficientlypowerfulest tatisticand perhapshe est tself)o calculatingn intuitivelymeaningful-value, an be "liberating"186). Otherwisenfeasiblexperi-mentaldesigns e.g. those based on nonrandom ampling r requiringnonstandardesponse ariables) ecome vailablewith esampling ethods.But his egree fversatilityarries he ost hat ther efensibleest tatisticsorproceduresor onductinghe est tselfmay ead to differentonclusions(e.g. the urveyf"nullmodels" bove)-or as a worst ase evenunderminetheobjectivityfthe data analysis see 9 and 113). It is thus mportantoconsider range falternativeest tatisticsndproceduresnd to ustifyhechoicesmade, deally efore hedata reanalyzed.t should e clear hat heappropriatelternativeypothesisould ndeed e supportedyrejectinghenull;an instructivease is the randomizationestof differencesetweenmeans, n which henullhypothesisan be rejected or denticalmeansbutdifferentariances242).The strikingifferencesn assumptions nderpinninghe classical andresampling ethodsTable 1; 186,p. 84-92)necessarilyonstrainhe ptionsto an extent hat s often verlooked r ignored y editors, eferees,nd


29/44

432 CROWLEYresearchers. aking ubious ssumptionshould bviously e avoidedwherepossible,butthis hould e balanced gainst he acit ssumptionsnvolvedin using ome of themore peculative esampling ethodsparticularlyhebootstrapnd ackknife).Randomizationests and by implicationheir erivative onparametrictests) pply nly othe ampleshemselves,ot o omeunderlyingopulationthatmayhavebeen ampled. hisobviates heneedfor andom amples ndfor ertain ssumptionsbout hepopulation f origin, ut t also lengthensthe nterpretivextrapolationromhe bserved esults othegeneralituationorpopulationf nterest.ome such xtrapolations almost lwaysnecessaryinanycase, and this roblems of greaterheoreticalhan pplied elevance(64).Finally, esampling ethodsurrentlyequire naveragemore xpenditureoftime ndefforter nalysis han o classicalmethods,argely ecauseofthenecessary omputerrogramming.n fact, herequired rogrammingsoftenuite traightforward,ndprogramsormany ommonpplicationsrewidely vailable n the iteraturee.g. 64, 163, 186) oras shareware. oon,commercial ainframendmicrocomputerackageswillbe available 186).Implicationsor nterpretingndCommunicatingesultsIn hypothesisesting,he nterpretationfresults ecessarily inges n thep-value, rat east n tsmagnitudeelativeo the ritical alue. An attractivefeaturefresampling ethodss that he direct alculation f thep-valueobviates the discrete ecision-theoryistinctionetween ignificantndnonsignificantesults,leftharplyf rbitrarilyta knife-edgeritical alue.Instead, hep-value can simply e understoodo measure hedegreeofconsistencyetween he data and the nullhypothesis,houghheclassicalsignificanceevels (0.05, 0.01, etc) retain heirutility s benchmarks.Moreover,hedirectlyalculated-valuemay emuch asier ocommunicatetonontechnicalecision-makers;s noted n the ntroduction,na random-ization estof a differenceetween womeans, hep-value s simply heproportionfrandomssignmentsfdata otreatmentshat ives differencebetween roup verages t least as largeas the differencebtainedn theexperiment64, p. 10).As withother tatisticalarameterstimates,n error stimate orthep-value s desirable, articularlyhere hiserror eflectsnlya moderatenumberfrepetitionse.g. 1000)for resampling ethodsampled andom-ization,MonteCarlo,orbootstrapping).n the atter ase, the100(1-a)%confidencentervals well approximatedy p + ti_,x(p(1-p)n)X2 with ninfinite umber f degrees ffreedom, here o s the ignificanceevel,tl-is the ritical alueofthe distributionor ignificanceevelcX,ndn is thenumberfrepetitionse.g. see 186, p. 34). Notethat hese rror ounds n


30/44

STATISTICAL ESAMPLING ETHODS 433thep-valuereflect ariationerived romhe ntensityf resampling. orespeculatively,t maybepossiblensome ases to obtain rror ounds n thep-valueassociatedwith mpirical ampling ariationsing hebootstrap rjackknife,ut am unawarehat hishas yetbeen ttempted.In research eports,rrorstimates ia resamplingenerally equire ittlefanfare-only he nameof the method nd the number f repetitionsifapplicable).Hypothesis esting equiresmore nformationn themethodssection, ncluding pecificationnd ustificationf thenull and alternativehypotheses. nless the approach s exotic or particularlyentral o thepresentation,eferencesosupport esampling ethodsreunnecessary.hemethod, umber f repetitionsif applicable),magnitudef the est tatistic,andp-value with onfidencentervalwhen appropriate)enerallyppearparentheticallyith esultsf a hypothesisest.MethodologicalssuesThatDeserveAttentionMuch tatisticalesearchemainsobe done odevelop,mprove,nd valuatetheseresamplingechniques. nder hegeneral heme frobustness,othparametricndresampling ethods eed obe furtherxaminednd omparedin their ensitivityo non-normality,on-equivalencef distributionse.g.unequalvariances), nd sample size. Also, what are the implicationsorrandomizationests fnonrandomssignmentotreatmentevels ndfor therresamplingndparametricests fnonrandomampling?deally, uch tudiesshould ocus nfeaturesypical fsmall-to-moderate-sizedamples nd thewayssuchdataare actually athered,atherhan xclusivelynthe harac-teristicsf large statisticalopulationsromwhich uch samplesmaybedrawn64).Until esampling ethods ecame enerallyeasible elativelyecently,heavailable tatistical ethods ere ufficientlyonstraininghat heformula-tion fnull ndalternativeypothesesas beenrelativelytraightforwardndunambiguous.Now that the horizonforthesehypotheses as widenedconsiderably,erhaps ew nduseful uidelinesanbe devised hatwillhelppractitionerso match ypotheses ore ffectivelyo tests nd test tatistics.

Manyof themore omplex escriptivendhypothesis-testingechniquestraditionallyasedonrelativelyssumption-boundarametricethodsanbeeffectivelyefitteds resamplingmethodssee above and 163, 165). Ofparticularmportancenecology nd volutionremethodshat elate irectlyto common xperimentalesigns, ike analysis f variance.ANOVA hastraditionallyeen onsideredobustodeparturesrom he tandardarametricassumptions,houghot llagree e.g. 18, 243),andmore omplex ariations(e.g. factorial NOVA, ANCOVA, MANOVA) can be morevulnerableoviolationsfparametricssumptionse.g. 191). nparallelwith urthertudiesofrobustness,he developmentfresampling NOVA and itsvariations


31/44

434 CROWLEYshould ontinuee.g. 8, jackknife NOVA; 130, randomizationNOVA;162, bootstrap NOVA), and controversiesike contrastingiews on inter-action ermsn factorial NOVA (see 64 vs 163) needto be resolved.Further tatistical esearch n Monte Carlo methods nd especially nbootstrapping ill continue o attract mmediate nterestnd additionalapplicationsnecology nd evolution.SomeSpecific ecommendations

1. Resamplingmethodshould be part of basic statistical raining necology nd evolution. t east ntil hesemethodsre ncorporatedntomainframetatisticsackages,hiswill equire ome omputer-program-ming kills s well. n exchange, erhaps ess emphasis an be placedonstandard onparametricethods.2. Parameterstimateshould e accompanied y stimatesfthe ssoci-ated variation. esampling ethodsmake tpossible or hisprincipletobe very roadlyifnotuniversally)pplied.3. With mall-to-moderateample sizes, maintain healthy kepticismabout he ppropriatenessf arametricnalysis. venfailuresorejectnormalitynd equal variancess nullhypothesesre rarelyonclusive,sincethepower ftests oevaluate hems lowat therelevantamplesizes. When herandom rocess hat eneratedhedata s statisticallyuncharacterized,he onservativepproachsto use defensible ethodsmaking hefewesttrongnd unverifiablessumptions.4. Transformationshould e usedto mprovehe quivalencefdistribu-tions nrandomizationestsnessentiallyhe amewaythat hese reused inparametricnalyses.This shouldhelpneutralize potentialproblemwith standard andomization ethods hathas oftenbeenunrecognized.5. Wherequivalence fdistributionss unlikelyo holdor to be achievedby transformation,ultisampleypothesisests an be conducted ybootstrapping. ith his pproach termed bootstrappedandomiza-tion" in 186), data fordifferentreatmentevels are bootstrappedindependentlyefore he est tatistics calculated.n ourpresenttateof gnorance, ootstrappinghouldnotordinarilye used where ara-metric rrandomizationethodspply.6. Forconfidencentervalsnd hypothesisesting,thermore heoreticallydefensiblemethodshouldgenerally e usedinstead ftheackknife.The jackknifes particularlyseful or liminatingias in parameterestimates67), as a check rextensionfthebootstrap68), or ncaseswhere he ther eavilyomputation-intensiveethods renotfeasible(67).7. Whereossible,researchershould ttemptoensure hat heir xper-


32/44

STATISTICAL RESAMPLINGMETHODS 435

imental nits onstituterandom ample f ome opulationf nterest.Though t mayoften e unclear ow to accomplish his, he object sto retain efensibleptions or he statisticalnalysis.When he casecan be made, tmaybe useful o explicitlydentifyhepopulationhathasbeen ampled andomly8. Inthe bsence frandom ampling, ypothesesomparingwo rmoresampleswith quivalentistributionshould e tested y andomization.Randomizationan also be used to constructonfidencentervals,thoughhese end obe relativelyonservative64).9. Userandomization,onteCarlo,orbootstrapping ethodsnstead fstandard onparametric ethods,articularly henmaximizingoweris essential.Any nonparametricest an be replaced y a potentiallymore lexible nd powerfulutotherwisequivalent esamplingest.10. Wheneverossible nd appropriate,se a large number frepetitionsinresamplingests?20000). This sparticularlymportanthentcaninfluenceheway hedata re nterpretede.g. when he -value s near0.05). For randomization, = 1000 and n = 5000 are generallyconsideredminimal or ests t the 5% and 1% significanceevels,respectively64, 65, 167).11. When sing esampling ethods,efine ull nd alternativeypotheseswith pecialcare. Justifyhese hoices n themethodsection ftheresearcheport.12. More attentionhould be paid to the applicability f assumptionsunderlyingtatistical nalysesby researchers,ditors, nd referees.With esampling ethodsecoming idely nownndcommonlysed,standards or cceptably horoughnd rigorous ata analysis houldcontinueo rise.A summaryf somecommon ituationsrisingn data analysis nd themost ppropriateethodsor ealingwith hemspresentednTable6.

ACKNOWLEDGMENTSIt is a pleasure o thankomepeoplewho haveaided thepreparationf thisreviewnddevelopmentf he deasherein: eter hesson, an Johnson,ndJerry aglefor ntroducinge to randomizationndbootstrapping;ugeneEdgingtonndBryanManly or ncouragementndfor reakinghegroundso effectively;en Hudson or elpingwith heBIOSIS and ournal earchesof the iterature;ryanManly,BrianMcArdle,Cidambi rinivasan, lareVeltman,nd participantsnthe 1992 course n experimentationnbenthicecologysponsored y theUniversityf Ume'a, Sweden,forstimulatingdiscussions; ickCrawley, anJohnson,ohn awton, ryanManly,BrianMcArdle, ndClareVeltmanor eadingndcommentingn themanuscript;MickCrawley or ettingmerummaget willthroughis ournals;Brenda


33/44

436 CROWLEYTable 6 Recommended ethods.

Empirical Distribution(s) PreferredPurpose Sampling sample ize' (orunderlying odel) methodConfidencentervals Random Large Any Parametricandsingle-sample Random Small-moderate Normal Parametrichypothesisests Random Small-moderate Known,but non-normal2MonteCarloRandom Small-moderate Unknown2 BootstrappingNon-random Any Any Randomization3Multisample Random Any Equivalent Parametrichypothesisests Random Any Non-equivalent2 BootstrappingNon-random Any Equivalent Randomization

Non-random Any Non-equivalent2 ---'In practice,arge ndmoderateampleizes regenerallyistinguishedubjectively.2Transformationsfthe ata an t east ometimeseshapehe istribution(s)dequatelyor arametricnalysiswithrandomampling)rrandomizationests with on-randomampling),houghhismay rove ifficulto demonstrateconvincingly.3Awkwardo mplement,nd endso yield onservativeonfidenceimits.

andChaunceyCurtzfornewperspectivesn nullhypotheses;roductionEditorNancyDonhamforgraceunder ressure;Marcel Dekker, nc., forpermissionouse a slightly odified ersion fE.S. Edgington'siagram sFigure1; JohnLawtonforhostingmysabbaticalyearat the Centre orPopulation iology; and Lillie, Sarah, and Martin orbeing patient ndunderstandinghenthe crunch ame. I acknowledge unding rom heNational cienceFoundationrant NT 9014938 and a VisitingResearchFellowship romheRoyalSociety f London.APPENDIX:HOWTHE DATAOF TABLE WEREANALYZED

I used randomizationnd a MonteCarlo method o test two a-priorihypothesesoncerninghedatasummarizedn Table 2. The rationale ndproceduresedineachcase are brieflyescribed ere s examples f howthis eneral pproachanbe implemented.Hypothesis : Thefocalmethodsonsiderednthisreview re becomingmore ommonlysedover he astseveralyears necology ndevolution.TheBIOSIS literatureearch ver he ixpublicationears1985-1990ontopics n ecology nd evolutionTable 2) provides hebasis for test.Toassure dequateample izesfor ach eparatemethod,eferencesittingitheror bothof thesecategories or particular ublication ear werepooled,yielding ublicationrequenciesor achof he ixyears obe tested or rend.Hypothesispredicts positiverendnpublicationrequencyveryears,generating one-tailed est gainst henullhypothesisfno trend. chosethe linearregressionoefficient or least-squares lope of publicationfrequencys a functionfpublicationear obe theresponsemeasure. It


34/44

STATISTICAL ESAMPLING ETHODS 437canbe demonstratedhat sing he orrelationoefficientfor hispurposenecessarilyroducesdentical esults;moreover, hemuch simplermetriclxiyi,where i andyiare the oordinatesf the thdata point, s equivalentto either fthese nd was themeasure ctually sedinmy tests-see 163,pp. 91-92.) Testing ypothesis then equired alculatinghe bserved =bo orthe implerquivalent)nddemonstratinghat value s large r argeris very nlikely ohave arisen ychance lone.Now f trend-freeull rocess eneratedhe bservedequence, hen hereshould e nothingpecial bout he rdernwhichhe ublicationrequencieswere observed; ny reorderinghould producea statisticallyquivalentsequence 163,p. 92). There re6! = 720differentrderingsfsixnumbers.Calculating regressionoefficient(orequivalent)or ach andcomparingit withbo indicatedheproportionf these s largeor larger hanbo. Thisproportionas taken o betheprobabilitythat he bservedequence ouldhavebeengeneratedy the amekind f trend-freerocess hat roduced heother19 sequences.When was smallerhan he elevantignificanceevel(generally.05), I rejectedhenullhypothesisnfavor fthe lternativei.e.Hypothesis); otherwise was unable oreject henull,and Hypothesiswasnot upported.This exemplifiesystematicandomization,hich eterminesxact -val-ues. My PascalcomputerrogramwrittennTurboPascal6.0) to calculateb, r, andp was ustover100 ines ong, hemajorityf whichwereneededtogeneratehe720 reorderingsf thedata.The solutionould nstead avebeen found y sampled andomizationsing much implerlgorithmsee163), half s many rogramines, ndsome dditionalun ime roughly0sec ratherhan fractionf a second n a typical 86/387microcomputer).Hypothesis: Thefocal methods onsideredn thisreview re used indifferingrequencyn ecologicalversus volutionarytudies.The dataof Table2 permithishypothesisobe tested or achmethod.Only hedata nthe ecology" nd"evolution"owsof the ablewereused.First,he ew aperswithinears lassifiedsboth cological nd volutionarywere emoved rom he bservedrequencies,liminatingsource fpositivecorrelation.heresiduals ere henummed long ows i.e. overyears), ndtheoverallproportiono of evolutionarytudies utof all ecologicalandevolutionarytudies or hegivenmethod as calculateds the volutionowsumdivided ythe umof the volutionow um ndthe cology ow sum.(Thisis of course quivalento using he overallproportionf ecologicalstudies,whichs simply -vo.)To determinehethero corroboratedypothesis, I useda MonteCarlomethod o generate nulldistributionf v-values or omparison ith o. fa givenmethod ere sed ust s frequentlyn both inds fstudies,hen heobserved requenciesn a particular earcouldhave been generateds asample rom binomial rocess asedon thefrequenciesf all ecology nd


35/44

438 CROWLEYall evolution tudiesnthatyear. thus andomly eneratedhe number fstudies bservedn eachyear,with he hance fanyparticulartudy's eing"evolutionary"qualling he evolutionary"roportionn thegivenyear, ndI tallied hedistributionetween volutionndecology.When hese istribu-tions adbeen imulatedor ll sixyears f he equence, he verall roportionv ofevolutionarytudieswas found s above fromhe volution ndecologyrowtotals. ach of 20,000suchv-valueswas determined,ndtheproportionas extreme rmore xtremehanvo becamethep-valueestimate. houghthis -valuewas derived rom random ample fv-values ndwas thereforeinexact, he argenumber f terationsssured dequate recision.See 163,pp. 32-36. The precision f this p-value could be assessed usingtheconfidence-intervalalculationescribedntheDiscussion ubsectionntitledImplicationsor nterpretingnd Communicatingesults.)Special provisionmustbe made fortwo-tailedestsof hypothesesikeHypothesis . I notedwhether o was larger r smaller hantheoverallexpected alueormean ofv, setting hecomputer rogram o calculatefrom-valuesqually ar rfartherrom hemean.As istypicalor wo-tailedtests, his equiredhat he ignificanceevelbe halved o 0.025. Thus f499orfewer f thev-valueswere s extreme r more xtremehan o (as inallcases in Table 2), then he nullhypothesis as rejected nd hypothesiscorroborated. y Pascalprogramo calculate oandp was ustover50 linesandwasstraightforwardowrite;he ubstantialumberf terationsequiredabout10-60 sec to runon a 386/387microcomputer,epending n theobserved ublicationrequenciesor hemethod f nterest.Literature ited

1. Abrahamson,W. G., Sattler,J. F.,McCrea, K. D., Weis, A. E. 1989.Variationn selection ressuresn thegoldenrod all fly nd thecompetitiveinteractionsf its natural enemies.Oecologia 79:15-222. Abrams, ., Nyblade, ., Sheldon, .1986. Resourcepartitioningnd com-petition or hells n a subtidal ermitcrab pecies ssemblage. ecologia69:429-453. Althawadi,A. M., Grace, J. 1986.Water use by the desert cucurbitCitrullus colocynthis L.) Schrad.Oecologia 70:475-804. Antonovics, ., Clay, K., Schmitt, .1987. The measurementfsmall-scaleenvironmentalheterogeneity singclonal transplants f Anthoxanthumodoratum and Danthonia spicata.Oecologia 71:601-75. Antyufeev,. S., Marshak, . L. 1990.

Inversion f Monte Carlo model forestimating egetation anopyparame-ters.Remote ens. Environ. 3:201-106. Archie,J.W. 1989. A randomizationtest for phylogeneticnformationnsystematicata. Syst.Zool. 38:239-527. Archie,J. W. 1989. Phylogenies fplant amilies: demonstrationf phy-logenetic andomnessn DNA sequencedata derived rom roteins. volution43:1796-8008. Arveson, .N., Schmitz, . H. 1970.Robust procedures or variance om-ponent roblems singthe ackknife.Biometrics 6:677-869. Basu, D. 1980. Randomizationnalysisof experimentalata: the Fisherran-domization est. J. Am. Stat. Assoc.75:575-8210. Benjamini, ., Harpaz, Y. 1986. Ob-servational ainfall unoffnalysisforestimatingffectsf cloudseeding n


36/44

STATISTICAL RESAMPLING METHODS 439waterresources n northernsrael. J.Hydrol. 83:299-30611. Berryman, . A. 1991. Stabilizationorregulation:hat t allmeans.Oecol-ogia 86:140-4312. Besag, J.,Clifford,. 1991. SequentialMonteCarlo p-values.Biometrika8:301-413. Black, A. R., Dodson, S. I. 1990.Demographic osts of Chaoborus-in-duced phenotypelasticityn Daphniapulex. Oecologia 83:117-2214. Block,W. M., Brennan,. A., Gutierrez,R. J. 1991. Ecomorphologicalelation-ships of a guild of ground-foragingbirds in northern alifornia,USA.Oecologia 87:449-5815. Boik, R. J. 1987. The Fisher-Pitmanpermutationest: non-robustlterna-tiveto the normal heory -testwhenvariances are heterogeneous. r. J.Math. Stat. Psychol. 40:26-4216. Boyajian,G. E. 1991. Taxon age andselectivityf extinction. aleobiology17:49-5717. Bradley, . V. 1968. Distribution-FreeStatistical ests.Englewood liffs, J:Prentice-Hall18. Bradley,J.V. 1978. Robustness? r.J. Math. Stat. Psychol. 31:144-5219. Brandl, ., Topp,W. 1985.Size struc-tureofPterostichuspp. (Carabidae):aspects fcompetition.ikos 44:234-3820. Breck,J. E., DeAngelis,D. L., VanWinkle,W., Christensen,. W. 1988.Potential mportance f spatial andtemporal eterogeneityn pH, alumi-num, ndcalciumnallowing urvivalof a fishpopulation: model demon-stration.col. Modell. 41:1-1621. Breshears,. D., Kirchner,. B., Otis,M. D., Whicker, . W. 1989. Uncer-taintyn predictionsf fallout adio-nuclides n foods and of subsequentingestion. ealthPhys. 57:943-5322. Briggs,J. M., Knapp,A. K. 1991.Estimating boveground iomass intallgrass prairie with the harvestmethod:eterminingroperample izeusing jackknifingnd Monte Carlosimulations.outhwest at. 36:1-623. Brokaw,N. V. L., Scheiner, . M.1989.Species compositionngapsandstructuref a tropical orest. cology70:538-4124. Brueggemann,., Trapp, ., Matthies,M. 1991. Behavior assessment f avolatile hemical n the Rhine River.Environ. oxicol.Chem. 10:1097-10425. Brusasca, ., Tinarelli, ., Anfossi, .1989. Comparisonetween heresultsof a Monte arlo tmosphericiffusion

model and tracer xperiments. tmos.Environ. 23:1263-8026. Bulmer,M. G. 1975. The statisticalanalysisof density ependence.Bio-metrics 1:901-1127. Cain, M. L. 1990. Patternsf Solidagoaltissima amet rowth nd mortality:the role of below-ground amet on-nections.Oecologia 82:201-928. Canale, R. P., Effler, . W. 1989.Stochastic phosphorus model forOnondaga Lake, New York, USA.WaterRes. 23:1009-1629. Capone, T. A., Kushlan,J. A. 1991.Fish ommunitytructuren dry-seasonstream ools. Ecology 72:983-9230. Carnes, . A., Slade, N. A. 1988. Theuse of regression or detecting om-petition ithmulticollinearata.Ecol-ogy 69:1266-7431. Carpenter,. R. 1990. Large-scale er-turbations:pportunitiesor nnovation.Ecology 71:2038-4332. Carpenter,. R., Frost, . M., Heisey,D., Kratz, T. K. 1989. Randomizedinterventionnalysis nd the nterpre-tation fwhole-ecosystemxperiments.Ecology 70:1142-5233. Casas, J. 1990. Multidimensionalostdistributionnd nonrandomarasitism:a case study nd a stochasticmodel.Ecology 71:1893-90334. Case,T. J.1990.Patternsf oexistencein sexual and asexual species ofCnemidophorusizards.Oecologia83:220-2735. Caswell,H. L. 1976. Communitytruc-ture:a neutralmodelanalysis.Ecol.Monogr. 46:327-5436. Caswell,H. 1989. MatrixPopulationModels. Sunderland, ass: Sinauer37. Chapin, . S. III, Groves, . H., Evans,L. T. 1989.Physiologicaleterminantsof growth ate in response o phos-phorus upply n wild and cultivatedHordeumspecies. Oecologia 79:96-10538. Chuh,H. T., Tan, H. S. 1990. A

multiconstituentndmultilayer icro-wavebackscatterodel or vegetativemedium.RemoteSens. Environ.31:137-5439. Clark, .J., vans, . C. 1954.Distanceto nearestneighbor s a measureofspatial relationshipsn populations.Ecology 35:445-5340. Cochran,W. G. 1954. Some methodsfor trengtheninghe ommon i2 ests.Biometrics 0:417-5141. Colwell,R. K., Winkler, . W. 1984.A null modelfornull models n bio-geography.nEcologicalCommunities:Conceptual ssues and theEvidence,


37/44

440 CROWLEYed. D. R. Strong, . Simberloff,. G.Abele, A. B. Thistle, pp. 344-59.Princeton, J: Princeton niv. Press42. Connor,E. F., Simberloff,. 1979.The assembly f speciescommunities:Chance or competition?cology 60:1132-4043. Connor,E. F., Simberloff,. 1984.Neutralmodels of species' co-occur-rence atterns.ee Ref.41, pp. 316-3144. Connor,E. F., Simberloff,. 1984.Rejoinder. ee Ref. 41, pp. 41-4345. Corbin,K. W., Livezey,B. C., Hum-phrey, . S. 1988. Genetic ifferenti-ation mong teamer-ducksAnatidae:Tachyeres):n electrophoreticnalysis.Condor 90:773-8146. Cotton,J. W. 1973. Even better hanbefore.Contemp. sychol. 18:168-6947. Cracraft, . 1991. Patternsf diversi-ficationwithin ontinental iotas: hi-erarchical ongruencemong he reasofendemismfAustralian ertebrates.Aust. Syst. Bot. 4:211-2748. Crowley, . H. 1992. Density epen-dence, oundedness,nd ttraction:e-tecting tabilityn stochastic ystems.Oecologia. 90:246-5449. Crowley, . H., Gillett, ., Lawton,J.H. 1988. Contests between larvaldamselflies: mpirical tepstowardbetter ESS model. Anim. Behav.36:1496-51150. Crowley, . H., Johnson,. M. 1982.Habitat nd seasonalitys niche axesin an odonate ommunity.cology 3:1064-7751. Crowley, . H., Johnson,. M. 1992.Variabilitynd stabilityf a dragonflyassemblage.Oecologia. 90:260-6952. DeAngelis,D. L., Gross,L. J., eds.0000. Individual-BasedApproachesand Models in Ecology. New York:Chapman-Hall53. denBoer,P.J. 1989. Comment n thearticle On testing emporal ichedif-ferentiationncarabid eetles," yM.Loreau. Oecologia 81:97-98

54. denBoer,P. J. 1990. On the stabili-zation fanimalnumbers. roblems ftesting. . Whatdo we conclude romsignificant ests?Oecologia 83:38-4655. denBoer,P. J. 1991. Seeingthe reesfor hewood:random alks rboundedfluctuationsof population size?Oecologia 86:484-9156. Diamond,J.M., Gilpin,M. E. 1982.Examination f the "null" model ofConnor nd Simberloffor pecies o-occurrencesn islands.Oecologia52:64-7457. DiCiccio,T. J., Romano,J.P. 1989.

A reviewof bootstrap onfidencen-tervals.J. R. Stat. Soc. B 50:338-5458. Dorazio, R. M. 1986. Estimatingop-ulation irth ates f zooplankton henrates of egg deposition nd hatchingare periodic.Oecologia 69:532-4159. Douwes, P., Stille,B. 1988. Selectiveversus tochastic rocesses n the ge-neticdifferentiationf populations fthe utterflyrebiaembla hnbg. Lep-idoptera: atyridae). ereditas 09:37-4460. Dressing, . A., Spooner, .,Kreglow,J.M., Beasley,E. O., Westerman,.W. 1987. Water nd sedimentamplerforplotand field tudies.J. Environ.Qual. 16:59-6461. DuBowy, P. J. 1988. Waterfowlom-munities nd seasonal environments:temporalvariabilityn interspecificcompetition.cology 69:1439-5362. Duff,K. E., Smol, J. P. 1991. Mor-phological descriptions and strati-graphic distributionsf the chryso-phycean tomatocystsrom recentlyacidifiedake,Adirondack ark,NewYork,USA. J. Paleolimnol. :73-11363. Dwyer,G. 1991. The rolesof density,stage, ndpatchinessn he ransmissionof an insect irus.Ecology 72:559- 464. Edgington,. S. 1987.RandomizationTests.New York:MarcelDekker.2nded.65. Edwards,D. 1985. Exact simulation-based inference: survey,with ddi-tions. J. Stat. Comput. Simul.22:307-2666. Efron,B. 1979. Bootstrapmethods:another ook at the jackknife.Ann.Stat. 7:1-2667. Efron,B. 1982. The Jackknife,heBootstrapndOther esamplinglans.Philadelphia:IAM68. Efron,B. 1992. Jackknife-after-boot-strap tandardrrorsnd nfluenceunc-tions. J. Roy. Stat. Soc. B 54:83-12769. Efron, ., Tibshirani, . 1986. Boot-strapmethodsor tandardrrors,on-fidencentervals,nd othermethodsfstatisticalccuracy. tat. Sci. 1:54-7770. Eisenberg, . M. 1966.Theregulationof densityn a natural opulation fthepond snail,Lymnaea lodes.Ecol-ogy 47:889-90671. Faust,K., Romney,A. K. 1985. Theeffectf kewed istributionsn matrixpermutationests.Br. J. Math. Stat.Psychol.38:152-6072. Felsenstein, . 1985.Confidenceimitsonphylogenies:napproach sing hebootstrap.volution 9:783-9173. Felsenstein,. 1988. Phylogeniesrom


38/44

STATISTICAL RESAMPLING METHODS 441molecular equences: nferencend re-liability. nnu.Rev. Genet. 2:521-6574. Firbank, . G., Watkinson,. R. 1987.On the analysis f competitiont thelevelof the ndividual lant.Oecologia71:308-1775. Fisher, . ., Hall,P. 1990.Onbootstraphypothesisesting.Aust. J. Stat. 32:177-9076. Fisher,N. I., Hall, P. 1991. Bootstrapalgorithmsor mall samples.J. Stat.Plan. Infer. 27:157-6977. Fisher,R. A. 1936. The coefficientfracial likeness nd the future f cra-niometry.. R. Anthropol.nst.66:57-6378. Fix,A. G. 1984. Kin groups nd traitgroups:population tructurend epi-demicdisease selection.Am.J. Phys.Anthropol. 5:201-1279. Foley,P. 1987. Molecular lock ratesas lociundertabilizingelection. roc.Natl. Acad. Sci. USA 84:7996-800080. Forina,M., Armanino, ., Leardi,R.,Drava, G. 1991. A class-modellingtechniqueased onpotential unctions.J. Chemometrics:435-5481. Gallagher,. D., Gardner,.B., Jumars,P. A. 1990. Competitionmong thepioneers n a seasonal oft-bottomen-thic succession: ield xperimentsndanalysis f the Gilpin-Ayala ompeti-tion model.Oecologia 83:427-4282. Garthwaite,. H., Buckland,S. T.1992. Generating onteCarlo confi-dence ntervals y theRobbins-Monroprocess.Appl. Stat. 41:159-7183. Gascon,C. 1991. Population-nd om-munity-levelnalyses of species oc-currences of central Amazonianraipforestadpoles.Ecology 72:173 -4684. Gaston,K. J., Lawton,J.H. 1987.Atest fstatisticalechniquesordetect-ing density ependencen sequentialcensuses of animal populations.Oecologia 74:404-1085. Gates, J. 1991. Exact Monte Carlotestsusing several statistics. . Stat.Comput. imul. 38:211-1886. Gebhardt,M., Ribi,G. 1987. Repro-ductive ffortnd growthn thepro-sobranch snail, Viviparus ater.Oecologia 74:209-1487. Gerrodette,. 1991.Modelsfor owerof detecting rends-a replyto Linkand Hatfield. cology 72:1889-9288. Gilbert, . J. 1990. Differentialffectsof Anabaena affinis n cladoceransandrotifers: echanismsndimplica-tions.Ecology71:1727-4089. Gilinsky, . L., Bamback, . K. 1986.Theevolutionaryootstrap: newap-

proach o the study f taxonomic i-versity. aleobiology12:251-6890. Gillespie, J. H. 1986. Variability fevolutionaryatesof DNA. Genetics113:1077-9291. Gilpin,M. E., Diamond,J. M. 1982.Factors ontributingo non-randomnessin species co-occurrencesn islands.Oecologia 52:75-8492. Gilpin,M. E., Diamond,J. M. 1984.Are species co-occurrencesn islandsnon-random,nd are null hypothesesusefulncommunitycology? ee Ref.41, pp. 297-31593. Gilpin,M. E., Diamond,J. M. 1984.Rejoinder. ee Ref. 41, pp. 332-4194. Gordon,H. R. 1991. Absorptionndscattering stimatesfrom irradiancemeasurements: onte Carlo simula-tions.Limnol.Oceanogr.36:769-7795. Gotelli,N. J.1991. Demographic od-els forLeptogorgia irgulata, shal-low-water gorgonian. Ecology 72:457-6796. Gotelli,N. J.,Lewis,F. G. III, Young,C. M. 1987. Body-sizedifferencesna colonizingmphipod-molluscssem-blage. Oecologia 72:104-897. G-Valdecasas,A., Baltanas,A. 1990.Jackknifend bootstrapstimationfbiologicalndex fwater uality.WaterRes. 24:1279-8498. Haefner, .W. 1988. Niche shiftsnGreater ntillean nolis communities:effectsf niche metric nd biologicalresolution on null model tests.Oecologia 77:107-1799. Haith,D. A. 1986. Simulated egionalvariationsnpesticide unoff. . Envi-ron. Qual. 15:5-8100. Hall, P., Wilson, S. R. 1991. Twoguidelinesor ootstrapypothesisest-ing. Biometrics 7:757-62101. Hanna, S. R. 1988.Airqualitymodelevaluation and uncertainty. . AirPollut. ControlAssoc. 38:406-12102. Harpending, ., Rogers,A. 1987. OnWright'smechanismor ntergroupe-lection.J. Theor.Biol. 127:51-62103. Harrison, . 1989. Long-distanceis-persal and colonization n the BayCheckerspot butterfly,Euphydryaseditha bayensis. Ecology 70:1236-43104. Harvey,B. C., Stewart,A. J. 1991.Fish ize andhabitat epth elationshipsin headwater treams.Oecologia 87:336-42105. Harvey, . E., Davis, F. W., Gale,N.1988. The analysis fclass dispersionpatternsusing matrixcomparisons.Ecology 69:537-42106. Hastings,A. 1987. Substitutionates


39/44

442 CROWLEYunder stabilizing election.

artigo crowley

Documents