A Sequential Test for Two Binomial PopulationsAuthor(s): Herbert RobbinsSource: Proceedings of the National Academy of Sciences of the United States of America,Vol. 71, No. 11 (Nov., 1974), pp. 4435-4436Published by: National Academy of SciencesStable URL: http://www.jstor.org/stable/64212 .

Accessed: 05/05/2014 23:49

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

National Academy of Sciences is collaborating with JSTOR to digitize, preserve and extend access toProceedings of the National Academy of Sciences of the United States of America.

http://www.jstor.org

This content downloaded from 130.132.123.28 on Mon, 5 May 2014 23:49:54 PMAll use subject to JSTOR Terms and Conditions

Proc. Nat. Acad. Sci. USA Vol. 71, No. 11, pp. 4435-4436, November 1974

A Sequential Test for Two Binomial Popt (statistical/clinical trials)

HERBERT ROBBINS

Columbia University, New York, N.Y. 10027; and Brookhaven Nationa:

Contributed by Herbert Robbins, September 9, 1974

ABSTRACT A sequential procedure is proposed for deciding which of two sequences of Bernoulli trials has the greater success probability. The procedure covers cases in which the two sequences are being observed at different rates, whether by design or otherwise, as well as the case of pairwise observation. The error probability of the proce- dure depends approximately only on the odds ratio of the two probabilities, a desirable property in deciding which of two medical treatments has the greater probability of success.

Let xi, . . . be independent observations, with P(xi = 1) = pi, P(xi = 0) = ql = 1 - pi, and let yi, ... be independent and

independent of the x's, with P(yi = 1) = p2, P(yi = 0) = q2 =

1- p2. We wish at some stage to terminate our observation of the two sequences and assert either that pl > p2 or that p2 >

pi. (The possibility of stopping without a decision is excluded.) The procedure that we propose is the following. For any posi- tive integers m and n let

Um = X + .. . + Xmn Vn = yl + .. + y,n Zm,n

2(num - mVn)/(m + n).

Stop with (M,N) = first (m,n) such that IZm,nl > B, where B is some preassigned positive constant, and assert that pi > p2 or p2 > pi according as ZM,N > B or zM,N < -B.

If X = piq2/p2ql is the odds ratio, then when pi > p2, so that X > 1, we shall show that the error probability, P (assert p2 >

pl), is

P(error) < 1/(B + 1) [1]

while the terminal sample sizes MI,N are such that

E 21llN \_1_ B(XB-) _ ('

_ _ 11 R + N > P-p21 (XB + 1)

( -if pi = p2 ? [2] 2pq

(When p2 > pi, [1] and [2] hold with X replaced by 1/X.) In the case of pairwise observation, with m = n at each

stage, and hence 1I = N, the formulas [1] and [2] are exact when B is a positive integer; here Zn,n = u, - v. and 2MN/ (Ml -+ N) = N = number of pairs (xi,yi) observed before termination. This case was treated by A. Wald (1), and the

present procedure may be regarded as a generalization of his, although even in the pairwise case it is conceptually somewhat different.

In what follows we shall first prove that formulas [1] and [2] are exact in the pairwise case when B is a positive integer, and then give a heuristic argument for their approximate validity when m s n, but the sampling rule that determines

44

ilations

Laboratory, Upton, New York 11973

whether the next observation is to be an x or a y is symmetric in the two sequences.

The pairwise case

The probability of observing any particular sequence xi,..., Xn; yl,. .. ,Y of 2n O's and l's is

fpl,p2 (xi1, . .) Xn; Yl, .- ., Yn) = PlUnqln- UnP2 q2 n - v,;

the likelihood ratio obtained by interchanging pi and p2 is

fpl ,P2/fP2,P1 ?= U-V

Hence, when B is a positive integer and X > 1, the error prob- ability is

P(error) = Pp,,p2 (assert p2 > p) = Ppl,p (UN - N = -B) co

= E fpl,P2 n=l {N=n,un - Vn= -B}

00 c0

= E E XUi-Un nfP

n=l { N=n, u,n- n= -B}

= -B Pp2,p (assert p2 > pi)

= X- Pp,p,, (assert pi > p2)

= X-I[1 - P(error)],

so that [1] holds exactly. Now by Wald's lemma and [1]

E(UN - vN) = (pi - p2)E(N)

= B[(l -P(error)] - BP(error),

so that [2] holds exactly when pl > p2. The case pi = p2 follows from the fact that then E(uN - vN)2 = B2 = 2 pqE(N) by Wald's lemma for second moments; alternatively, we can let p2 -- pi and use L'Hospital's rule.

The non-pairwise case

The probability of observing any particular sequence x,. . . ,Xm; yl,. ..,Yn of m + n O's and 's is

fplp2(Xl,. . . ,Xm; yl,. . ,yn) = plUmqlm-- Ump^Viq2 - Vn;

the likelihood ratio obtained by interchanging pi and pi is

fP1,P2/fP2,P1 = u- A-n(q2/ql)'r-x [3]

Putting pm = p + e, pi = p - e, if e > 0 is small compared to p and q then

(1+) /( )(1--)

? 2e[(1/p) + (l/q) ] _ e2e/pq

35

This content downloaded from 130.132.123.28 on Mon, 5 May 2014 23:49:54 PMAll use subject to JSTOR Terms and Conditions

4436 Statistics: Robbins

q2/ql= (1+ -;)/(1- ) > e2e/q,

so that by Eq. [3]

fp ,p2/fp2l "

e(2c/pq) [um-vnf p(n- m)] , \um-vn+p(n-m). [4]

Now

E(Um + vn mpn -+- np2 (m -n)e

m-+n m+-n m+n

Vr Umn - V+ mplqi + _np2 1 n -+ n (m+ n)2 4(m + n)

so that with high probability when m + n is large

Um + Vn + (m - n)c m - n m- +

and hence

U Um + Vn + (n - m)e [5]

m + n

Substituting this estimate of p into the exponent of X into [4] gives

Um -Vn +- p(n - m)

(m + n) (u - Vn) + (n - m) [u + vn + (n - m)e] m + n

_2(num - mvn) (n -m)2e (n m)2 -

- + n m +

n m + n m+n m+n m-n

so that by [4] with e > 0 and X > 1

fpl,p2 \ XZm. + (n -m)2e/(m+n) > XZ..n

fP2, Pl

Hence when X > I the probability of correct decision is

co

Pp1,p2 (assert pl > p2) = E fpi.P m,n-= { (M,N) = (m,n); Z.,,n > B}

~~> E EV"'- fp P1 m,n = 1 (M,N) = (m,n),z ,, > B}

> XBp p,pi (assert pi > p2) = XBPpl,p2 (assert p2 > pl)

= XB[1 - Pp,,P (assert pi > p2)],

Proc. Nat. Acad. Sci. USA 71 (1974)

so that

XB PP1,p2 (assert pi > p2) > B '

XB -I 1

and hence

P(error) = i - -P,p, (assert pi > p2) < -? 1' AXB +- I

This provides a heuristic justification of [1] when P1,p2 and B are such that the approximations made above are valid. Since we have made no assumptions about the relation be- tween m and n as sampling proceeds, other than that both become infinite (to assure termination with probability 1), it would be difficult to be more precise about the accuracy of the approximation [1] in all cases. We may hope, at least, that when it is not a good approximation it is because P (error) is too small to be of any practical significance.

To justify [2], we have by neglecting overshoot and using [1]

B(XB- 1I) E(zM,N) -- B[1 - P(error)] -BP(error) > . [6]

AB + I

But since uM _ il pi and vN '- N p2 with high probability, we have by a nontrivial act of faith the approximation

2 (fNMpl -iINp\ ( 2MaN \ E (M, N) -

2E -~ +N )-( 2)E I + N

which together with [6] gives [2] for pi > p. In the nonpairwise case many questions remain to be

answered by Monte Carlo simulation for various pi,pa pairs and by purely analytic means, e.g., in the form of asymptotic results as X -- 1 or B > oo. Even more interesting are the allo- cation problems in the case of clinical trials where the objective is to minimize the expected value of, say, the number of pa- tients given the inferior treatment before a decision is reached. The method of estimating a nuisance parameter p as in [5] works also in other two-population decision problems, in-

cluding the normal case that was treated by more exact methods in ref. 2.

This research was supported by PHS Grant 5-RO1-GM-16895- 06. 1. Wald, A. (1947) in Sequential Analysis (John Wiley and

Sons, New York), pp. 106-116. 2. Robbins, H. & Siegmund, 1). 0. (1974) "Sequential tests in-

volving two populations," J. Amer. Stat. Ass. 69, 132-139.

This content downloaded from 130.132.123.28 on Mon, 5 May 2014 23:49:54 PMAll use subject to JSTOR Terms and Conditions