This article was downloaded by: [Osaka University]On: 19 November 2014, At: 20:45Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

Communications in Statistics - Theory and MethodsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/lsta20

THE SEQUENTIAL BOOTSTRAP: A COMPARISON WITHREGULAR BOOTSTRAPO. J. Shoemaker a & P. K. Pathak ba U.S. Bureau of Labor Statistics , 2, Mass Avenue NE #3655, Washington, DC, 20212,U.S.A.b Michigan State University , Alto Wells Hall, East Larsing, MI, 48824, U.S.A.Published online: 20 Aug 2006.

To cite this article: O. J. Shoemaker & P. K. Pathak (2001) THE SEQUENTIAL BOOTSTRAP: A COMPARISON WITH REGULARBOOTSTRAP, Communications in Statistics - Theory and Methods, 30:8-9, 1661-1674, DOI: 10.1081/STA-100105691

To link to this article: http://dx.doi.org/10.1081/STA-100105691

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose ofthe Content. Any opinions and views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources of information. Taylor and Francis shallnot be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

THE SEQUENTIAL BOOTSTRAP: A

COMPARISON WITH REGULAR

BOOTSTRAP

O. J. Shoemaker1 and P. K. Pathak2

1U.S. Bureau of Labor Statistics, 2, Mass AvenueNE #3655, Washington, DC 20212, USA

2Michigan State University, Alto Wells Hall, EastLarsing, MI 48824, USA

ABSTRACT

Based on Bradley Efron’s observation that individual resam-ples in the regular bootstrap have support on approximately63% of the original observations, C. R. Rao, P. K. Pathakand V. I. Koltchinskii (1) have proposed a sequential resam-pling scheme. This sequential bootstrap stabilizes the infor-mation content of each resample by fixing the number ofunique observations and letting N, the number of observatonsin each resample, vary. The Rao-Pathak-Koltchinskii paperestablishes the asymptotic correctness (consistency) of thesequential bootstrap. The main object of our investigation isto study the empirical properties of the Rao-Pathak-Koltchinskii sequential bootstrap as compared to the regularbootstrap. In all our settings, sequential bootstrap performsas well or better than regular bootstrap. In the particular casewhere we estimate standard errors of sample medians, we findthat sequential bootstrap outperforms regular bootstrap byreducing variability in the final bootstrap estimates.

1661

Copyright & 2001 by Marcel Dekker, Inc. www.dekker.com

COMMUN. STATIST.—THEORY METH., 30(8&9), 1661–1674 (2001)

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

Key Words: Bootstrap resampling; Quantile estimation;

Sample median.

1. INTRODUCTION

The bootstrap is an old and simple idea, but its usage in modern

statistics is generally traced and attributed to Brad Efron’s (2) article in

the Annals of Statistics: ‘‘Bootstrap methods: Another Look at the

Jackknife’’. In the article Efron sets out the basic resampling principles

and procedures that constitute what we have come to call the bootstrap

method. He establishes the validity of this approach as an estimation meth-

odology for a general class of functionals and discusses several settings in

which this bootstrap procedure can be efficaciously applied. In 1981, in

back-to-back articles in The Annals of Statistics, Kesar Singh (3) and

Bickel & Friedman (4), working independently, formally established the

asymptotic first-order correctness of the regular bootstrap.

The bootstrap methodology is, in general, a three-part process: resam-

pling, evaluation and estimation. If we let X¼ (X1, X2, . . . ,Xn) be an i.i.d.

sample of size n from an unknown distribution F, then the regular (Efron

(5)) bootstrap algorithm for estimating the standard error of some func-

tional of interest (say, the sample mean) consists of the following three steps:

(1) Selection of B independent bootstrap samples X*1, X*2, . . . , X*B,

each consisting of n data values drawn with replacement

(SRSWR) from X.

(2) Evaluation of each bootstrap resample with some functional s.

���ðbÞ ¼ sðX�b

Þ b ¼ 1, 2, . . . ,B:

(3) Estimation of the standard error by the sample standard devia-

tion of the B replications.

seB ¼XB

b¼1½��

�ðbÞ���

�ð�Þ�

2=Bn o1=2

,

where ���ð�Þ ¼

PBb¼1 ��

�ðbÞ=B:

The resampling scheme described in (1) is ordinary resampling,

which we will refer to as regular bootstrap. Sequential resampling and the

sequential bootstrap will differ from regular bootstrap only in step (1).

1662 SHOEMAKER AND PATHAK

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

2. THE SEQUENTIAL BOOTSTRAP

In a 1997 paper, Bootstrap by Sequential Resampling, C. R. Rao, P. K.Pathak and V. I. Koltchinskii proposed a sequential resampling scheme asan alternative to the regular bootstrap resampling scheme. Regular resam-pling, as Efron observed, is supported on approximately 63% of the originalobservations. For regular bootstrap, the resample size is fixed at n, withthe number of distinct observations a random variable (Kn). This Kn isdistributed binomially, with success probability [1�(1�1/n)n]. Thus,

EðK nÞ ffi nð:63Þ

VarðKnÞ ffi n½1� ð1� 1=nÞn�ð1� 1=nÞn�

ffi nð:23Þ

so

�ðKnÞ ffi :48n1=2

With this �, we can then obtain a simplified 95% Confidence Interval:.63 n 2 (.48) n1/2, which is approximately .63 n n1/2. For a moderatesample size, such as n¼ 25, we would then get 16 5, or an interval rangeof (11, 21). Rao-Pathak-Koltchinskii (1) observe that ‘‘. . . [V]iewed from apurely information content of the bootstrap samples, this variability isneither necessary nor desirable.’’ By fixing the unique number of observa-tions at m¼ .63 n, the sequential resampling scheme stabilizes the informa-tion content of the bootstrap resample. The resample size is then no longerfixed at n, but becomes a random variable N, whose expectation is n, andðN=nÞ ! 1 in probability.

If we then let ��n ¼ffiffiffin

pð�ðFFnÞ � �ðFnÞÞ be the regular bootstrap pivot

and let ��N ¼ffiffiffiffiN

pð�ðFFNÞ � �ðFnÞÞ be the sequential bootstrap pivot (setting

�2¼ 1), it can be shown ( Mitra and Pathak (6)) that

Eð��N � ��nÞ2

Varð�nÞ¼ Oðn�1=2

Þ

Thus, the distance between the sequential bootstrap pivot and the regularbootstrap pivot converges to zero at an acceptable rate.

The more formal theoretical justifications that establish the asymptoticcorrectness of the sequential bootstrap (i.e., the conditional weak conver-gence of the pivot �N to the pivot �n), are presented in the main body ofthe Rao-Pathak-Koltchinskii paper. They show the asymptotic distancebetween the two pivots to be on the order of n–3/4. See also Babu, Pathakand Rao (7).

SEQUENTIAL BOOTSTRAP 1663

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

The implementation rules for the sequential bootstrap are relativelysimple. ‘‘To select a [sequential] bootstrap sample, draw observations fromS sequentially by SWSWR until there are ðmþ 1Þ � nð1� e�1

Þ þ 1 distinctobservations in the observed bootstrap sample, with the last observation inthe bootstrap sample discarded to ensure simplicity in technical details.’’(Rao-Pathak-Koltchinskii (1)). Thus, instead of XRegBoot¼ (X1,X2, . . . ,Xn), we now get XSeqBoot¼ (X1,X2, . . . ,XN), with the number of distinctobservations in XSeqBoot set at .63 n.

3. SEQUENTIAL BOOTSTRAP AND

ASYMPTOTIC NORMALITY

Enno Mammen, in his 1992 monograph, ‘‘When Does BootstrapWork? Asymptotic Results and Simulations,’’ establishes a framework,both theoretical and empirical, for investigating the relative merits of severalbootstrap procedures as compared with classical (normal) approximations.The empirical methodologies that Mammen has prescribed are the ones weshall use to compare the sequential bootstrap with regular bootstrap. So,along with Mammen, we choose the following three sampling distributionsto work from:

MODEL A. X1, . . . , X20 i.i.d with a w2 distribution having 4 dfs.MODEL B. X1, . . . , X20 i.i.d with a normal mixture distribution..5N(.5, (.5)2)þ.5N(�.5, (.7)2)MODEL C. X1, . . ., X20 are independent and Xi has a normal mixture

distribtuion.

i � 1

n� 1N

n� i

n� 1, ð:5Þ2

� �þn� i

n� 1N �

i � 1

n� 1, ð:7Þ2

� �

In the first two sampling models (A & B), the random variables arei.i.d. In Model C, the random variables are independent but not identicallydistributed. We will demonstrate empirically that both the regular bootstrapand the sequential bootstrap are consistent estimators in the particular set-ting in which the functional of interest is the sample mean, �XXn, with n¼ 20.We will estimate each sample mean distribution at selected quantile values,first, by a normal approximation, then by two unstudentized bootstrapdistributions (regular and sequential) at these selected quantile values, andfinally, by their two respective studentized versions at their normalizedquantile values. Our goal is to assess the accuracy of these five competingmethods in estimating the underlying probability distribution by assessingthe accuracy with which each method estimates a given probability at agiven quantile value.

1664 SHOEMAKER AND PATHAK

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

We first look at the problem of estimating the probabilitiesPðð �XXn � �Þ � upÞ, with each up being the pth quantile, p¼ .01, j/20 (forj¼ 1, . . . , 19) and .99. The 21 up’s for each of the three models are pre-determined from (16,000) Monte Carlo simulations of ð �XXn � �Þ per model.The distribution function for the unstudentized bootstrap estimate isP�

ðð �XX�� �XXnÞ � upÞ, with �XX�

¼ �XX�n or �XX�

¼ �XX�N (n for the regular bootstrap

sample mean estimate, N for the sequential bootstrap sample mean esti-mate). The normal approximation to Pðð �XXn � �Þ � upÞ is �ðup=SSnÞ, withSS2n ¼ n�2 Pn

i¼1 ðXi ��XXnÞ

2 [see Mammen (8), p. 10].For the studentized versions, we look at the problem of estimating

the probabilities PððSS�1n Þð �XXn � �Þ � vpÞ, with each vp being the pth quantile.

These 21 vp’s are pre-determined from (16,000) Monte Carlo simulations,with each simulated ð �XXn � �Þ now normalized by SSn. For regular boot-strap, the distribution function for the studentized estimate isP�

ððSS ��1n Þð �XX�

n ��XXnÞ � vpÞ, with SS

�2n ¼ n�2 Pn

i¼1 ðX�i �

�XX�n Þ

2. For sequentialbootstrap, the distribution function for its studentized estimate isP�

ððSS ��1N Þð �XX�

N � �XXnÞ � vpÞ, with SS�2N ¼ N�2 PN

i¼1 ðX�i �

�XX�NÞ

2.We then estimate and measure, at each appropriate probability of the

underlying distribution (Model A, B, or C), the five approximation meth-ods: normal, regular bootstrap, regular bootstrap studentized, sequentialbootstrap and sequential bootstrap studentized. The three error measure-ments that we use are expected squared error, standard deviation and bias.

ESQE ¼1

S

XSi¼1

ðP�i � pÞ

2, StDEV ¼1

S � 1

XSi¼1

ðP�i �

�PP�Þ2

" #1=2

,

BIAS ¼ �PP�� p, with �PP�

¼ S�1XS

i¼1P�i :

We generate S¼ 10,000 samples of 20 observations according to the threemodels above and, for every probability value and every sample, we calcu-late the regular bootstrap estimate, the studentized regular bootstrapestimate, the sequential bootstrap estimate, the studentized sequential boot-strap estimate, and the normal approximation of the probabilities ( p). Inthe four bootstrap resamplings we then use 1000 replications each time.The expected squared error, standard deviation and bias of these five pro-cedures are then plotted against the cdf probabilities ( p) and graphed inFigures 1a–1f.

In the standard deviation results the two resampling schemes areindistinguishable, even using Model C. In the ESQE results both sequentialand regular bootstrap consistently perform as well as normal approxima-tion, with sequential bootstrap displaying slightly higher ESQEs attimes than does regular bootstrap. These slight ESQE differences seem to

SEQUENTIAL BOOTSTRAP 1665

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

Figure 1a. N¼Normal Approximation; R¼Regular Bootstrap; S¼ Sequential

Bootstrap; R¼Regular (Studentized); S¼ Sequential (Studentized)

Figure 1b. N¼Normal Approximation; R¼Regular Bootstrap; S¼ Sequential

Bootstrap; R¼Regular (Studentized); S¼ Sequential (Studentized)

1666 SHOEMAKER AND PATHAK

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

Figure 1c. N¼Normal Approximation; R¼Regular Bootstrap; S¼ SequentialBootstrap; R¼Regular (Studentized); S¼ Sequential (Studentized)

Figure 1d. N¼Normal Approximation; R¼Regular Bootstrap; S¼ SequentialBootstrap; R¼Regular (Studentized); S¼ Sequential (Studentized)

SEQUENTIAL BOOTSTRAP 1667

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

Figure 1e. N¼Normal Approximation; R¼Regular Bootstrap; S¼ SequentialBootstrap; R¼Regular (Studentized); S¼ Sequential (Studentized)

Figure 1f. N¼Normal Approximation; R¼Regular Bootstrap; S¼ SequentialBootstrap; R¼Regular (Studentized); S¼ Sequential (Studentized)

1668 SHOEMAKER AND PATHAK

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

occur mostly at the ‘‘peaks’’ where non-scaling (i.e., unstudentizing) hasits greatest impact. The unstudentized regular and sequential bootstrapdistributions closely track the normal approximations. For the normalapproximations and the unstudentized estimates, the expected squarederrors and standard deviations maximize near the 0.2 and 0.8 quantilevalues, and are minimized at 0, 0.5 and 1. Since we are estimating pro-babilities along the (0, 1) interval in all our settings, the estimates arebehaving as expected from a binary distributution, with the two studentizedbootstraps correcting for the skewness, while the two unstudentizedbootstraps and the normal approximations do not correct for this skewness.The two studentized bootstrap distributions display a clear improvementover the normal approximations, as well as over the two unstudentizedbootstraps.

There is a noticeable and even interesting pattern of alternating dom-inance between the studentized sequential bootstrap bias results and theirregular studentized counterparts, with the sequential dominating the regularbelow the 0.5 mark and regular bootstrap dominating above 0.5. The pat-tern persists in all the models that we investigate but, if the biases aresquared, then roughly half the settings display a reversal of this pattern.Moreover, the squared biases here amount to less than 1% of the totalexpected squared error, so these interesting differences are almost tooslight to consider. The intuition would be that this pattern in the bias resultsis an artifact of the binary skewness at work at these probability points. Ingeneral, the two bootstrap schemes, whether studentized or unstudentized,are producing, all along the c.d.f., in all of these settings, nearly identicalprobability estimates.

4. BOOTSTRAP CONFIDENCE INTERVALS

In this section we move to quantile estimation and, more specifically,to the percentile estimation required for establishing confidence intervals forsample means. We look only at two-sided confidence intervals at confidencelevel .95. Using n¼ (10, 20, 30), and restricting ourselves to non-parametricbootstrapping, we investigate the comparative performances of our twobootstrap resampling schemes, working with the following distributionalsettings: Uniform (0, 1), Beta (9, 3), Exponential (1) and Chi-square (4).Several bootstrapping methodologies have been suggested for estimatingconfidence interval percentiles. We look at five of them: Percentile,Percentile-t, Hybrid, Bias-Corrected and Accelerated Bias-Corrected. (Theprogrammatic details for bootstrapping each of these five methodologiescan be found in Shao and Tu (9), pp. 131–141.) For each of the five

SEQUENTIAL BOOTSTRAP 1669

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

techniques we apply our two resampling schemes and then compare

performances, both with respect to each other and against exact confidence

intervals.

For the ten sets of bootstrap confidence limits, we have generated 1500

bootstraps resamples for each of 1000 sample simulations. Viewed horizon-

tally, the graph below shows the comparative performances of the two

resampling schemes, with the ten bootstrap confidence limits graphically

paired side by side. For all five methods in all twelve distributional settings

the pairs are nearly indistinguishable. We display only the one graph – for

Chisq (4), n¼ 20 (Figure 2). The comparative results are similar in each of

the other eleven settings we investigate. At each of the five pairings (regular

versus sequential), the coverage probabilities displayed in the middle of the

graph give further empirical evidence that sequential resampling provides

quantile and percentile estimates on a par with regular resampling.

Taking the average absolute difference for each of the five bootstrap

pairings (regular versus sequential), in the twelve different settings, we get

the following Table of differences.

Five of the twelve mean differences are negative, seven are positive.There is no discernible bias in any one particular direction.

5. BOOTSTRAP ESTIMATION OF STANDARD

ERRORS OF SAMPLE MEDIANS

An important setting in which to test whether the sequential boot-strap’s performance compares favorably or not with regular bootstrap isstandard error estimation when the parameter of interest is the samplemedian. If we let T(X) be the sample median, then T(X*) is the median ofthe bootstrap sample. The bootstrap estimate, whether using sequential orregular resampling, of standard error is the standard deviation of the

Table.

n¼ 10 N¼ 20 n¼ 30

Uniform 0.0108 0.0045 0.0030Beta 0.0044 0.0044 0.0008

Chi-square 0.0032 0.0028 0.0050Exponential 0.0102 0.0046 0.0024

1670 SHOEMAKER AND PATHAK

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

bootstrap replications:

seBOOT ¼XBb¼1

½TðX�b Þ � Tð�Þ�

2=ðB� 1Þ

( )1=2

,

where T(E)¼PB

b¼1 T(X*b)/B.

We sample from two well-known distributions, Uniform and StandardNormal, using n¼ 8 through n¼ 20. For both sequential and regular boot-strap, we run 2000 resamplings per sample simulation (sims¼ 150). We run10,000 Monte Carlo simulations to compute ‘‘gold standard’’ results. TheMonte Carlo results reveal a staircase effect (with each odd-even pair a step)in both distributional settings. The two bootstrap results appear to be diver-ging from each other at each odd and even point, with the sequential resultsgenerally doing a better job of estimating the standard errors. For n-even,regular bootstrap gives a lower estimate each time; for n-odd, regular

Figure 2.

SEQUENTIAL BOOTSTRAP 1671

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

bootstrap gives a higher estimate than does sequential bootstrap each(n-odd) time. Regular bootstrap appears to produce a more spiked effectand so would seem to be more sensitive to whether the sample size is odd oreven. In Figure 3a, in the normal settings, the sequential bootstrap estimates

Figure 3a.

Figure 3b.

1672 SHOEMAKER AND PATHAK

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

are, on average, closer to the Monte Carlo (or Exact) standard errors thanare the regular bootstrap estimates (a mean absolute value difference of0.017 for sequential bootstrap versus 0.019 for regular bootstrap acrossthese two sets of standard error estimates). The resampling nature of thesequential bootstrap seems to be producing smoother standard error esti-mates here, and thus showing where sequential bootstrapping does indeedreduce the variability about a set of final bootstrap results.

The results shown in Figure 3b are less compelling, but a similar‘‘spikier’’ pattern occurs with the regular bootstrap results, and sequentialbootstrap still appears to be producing smoother standard error results.Whether a general conclusion can be drawn as to the potential superiorityof the sequential bootstrap for quantile estimation remains a question forfurther investigation. But the sequential results do seem to track these(known) exact results more closely than do the regular bootstrap results.

6. SUMMARY AND CONCLUSIONS

In a wide range of classical settings, we have empirically tested anddemonstrated that the sequential bootstrap produces robust and accurateestimates in small to moderate sample size settings. In every instance wherewe have compared sequential bootstrap with regular bootstrap we find thesequential bootstrap performs equally well and, in the case of estimation ofstandard error of the sample median, generally better.

ACKNOWLEDGMENT

We wish to express our sincere thanks to the reviewer for his helpfuland constructive comments.

REFERENCES

1. Rao, C.R.; Pathak, P.K.; Koltchinskii, V.I. Bootstrap by SequentialResampling. Journal of Statistical Planning and Inference 1997, 64,257–281.

2. Efron, B. Bootstrap Methods: Another Look at the Jackknife. TheAnnals of Statistics 1979, 7, 1–26.

3. Singh, K. On the Asymptotic Accuracy of Efron’s Bootstrap. TheAnnals of Statistics 1981, 9, 1187–1195.

4. Bickel, P.J.; Freedman, D.A. Some Asymptotic Theory for theBootstrap. The Annals of Statistics 1981, 9, 1196–1217.

SEQUENTIAL BOOTSTRAP 1673

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

ORDER REPRINTS

5. Efron, B.; Tibshirani, R.J. An Introduction to the Bootstrap. New York:Chapman & Hall, 1993.

6. Mitra, S.K.; Pathak, P.K. The Nature of Simple Random Sampling.The Annals of Statistics 1984, 12, 1536–1542.

7. Babu, G. Jogesh; Pathak, P.K.; Rao, C.R. Second Order Correctness ofthe Poisson Bootstrap. The Annals of Statistics 1999, 27.

8. Mammen, E. When Does Bootstrap Work? New York: Springer-Verlag,1992.

9. Shao, J.; Tu, D. The Jackknife and Bootstrap. New York: Springer,1995.

1674 SHOEMAKER AND PATHAK

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014

Order now!

Reprints of this article can also be ordered at

http://www.dekker.com/servlet/product/DOI/101081STA100105691

Request Permission or Order Reprints Instantly!

Interested in copying and sharing this article? In most cases, U.S. Copyright Law requires that you get permission from the article’s rightsholder before using copyrighted content.

All information and materials found in this article, including but not limited to text, trademarks, patents, logos, graphics and images (the "Materials"), are the copyrighted works and other forms of intellectual property of Marcel Dekker, Inc., or its licensors. All rights not expressly granted are reserved.

Get permission to lawfully reproduce and distribute the Materials or order reprints quickly and painlessly. Simply click on the "Request Permission/Reprints Here" link below and follow the instructions. Visit the U.S. Copyright Office for information on Fair Use limitations of U.S. copyright law. Please refer to The Association of American Publishers’ (AAP) website for guidelines on Fair Use in the Classroom.

The Materials are for your personal use only and cannot be reformatted, reposted, resold or distributed by electronic means or otherwise without permission from Marcel Dekker, Inc. Marcel Dekker, Inc. grants you the limited right to display the Materials only on your personal computer or personal wireless device, and to copy and download single copies of such Materials provided that any copyright, trademark or other notice appearing on such Materials is also retained by, displayed, copied or downloaded as part of the Materials and is not removed or obscured, and provided you do not edit, modify, alter or enhance the Materials. Please refer to our Website User Agreement for more details.

Dow

nloa

ded

by [

Osa

ka U

nive

rsity

] at

20:

45 1

9 N

ovem

ber

2014