I LEVEL

DEPARTMENT OF STATISTICS

The Ohio State University

i-ii

COLUMBUS, OHIO

DISTRIBUTION A

Approved f!-r public reluuse

IA I

1OPTIMIZATION IN STATISTICS-AN INTRODUCTION

L by

Jagdish S. RustagiDepartment of StatisticsThe Ohio State University

Ij Technical Report No. 249

IA-c. 7 1 on For

1 :Tlr CRA&IDTIC TABUr~runounced []Justification-

Dstribut ion..

Availgtbillty Codes,Avail and/or

D st SPecial

14October, 18Supported in part by Contract No. N000-14-78-C-0543 (NR 042-403) by

* Office of Naval Research. Reproduction in whole or in part is permittedI for any purpose of the United States Governmnt.

1-

I DISTRIBUTION STATEMENT A

.I Approved for public release;Distxubution Unlimitedi : ---, - _ . . . 7. 7-I.

- E C U :.IT Y C LA SSI IC A TIO N O F TH IS P A G E ( 14e . D eis, E rt.,.d)R E D I S U C O N1REPORT DOCUMENTATION PAGE 13FR1OPEIGFRIREPORT NUMBER 2. GO~VT ACCESSION NO. 3. RCCIPIENT'S CATALOG NUMBER

A& TITLE (end Sobrlrls) S YEO EOT&PRO OEE

' IOptimization in Statistics-An Introduction Technical Repor-t6 PERFORMING ORG. REPORT NUMBER

2497 .AUHRs ,rON'RACT OR GRANT NUMBE~r()

£ J. S. Rustagi NOOO-14-7 8-C-05*3

9. PIERFORiNG ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PRCJECT, TASK

Deparbtnnt of Statistics AREA &0WRKUNT NM1R

Ohio State University . I1 1958 Neil Ave., Columnbus, Ohio 43210III. CONTROLLING OFFICE NAME AND ADDRESS 1Q. REPOR- DATE

Office of Naval Research October, 1981Departirent of Navy 13. NUMBER OF PAGES

Arlington, Virginia 22207 1814. MONITORING AGENCY NAME 6 AODRESS(If different1 from, Controlling OIffice) '5. SECURITY CLASS. (of this report)

'So. DECL ASSI FIC ATION/fDOWN GtIADINGSCHEDULE

16. DISTRIBUTION STATEMENT (of thla Rep.rt)

DISTRIBUTION______________________STATEMENT_________________A_

Approved for public releaselDistribution Unlimited

17. DISTRIBUTION STATEMENT (of the abstrt enrtored i. Block 20. lifdifferprit from Report)

IS SUPPLEMENTARY NOTES

19. AE STRDT (Coninueo on reverse side it rece.wy and Identify by block nuoebr)

of experime~nts, sample surveys, mnultivariate statistical inference,information theory and regression analysis are provided. v

*DD I I,7 1473 EION OI NOV 6S IS OBSOLETED D DAN 73 1 4SEC UJRITY C L A SSI IC A TIO N O F THIS PA E ( ^ oC n b et. N nf te- d)

1.1 Introduction

fn many areas of science, business, industry and government, optimizing

I techniques are commonly applied to solve routine problems. Topics on

optimization have become an important area of study in disciplines such

tas Operations Research, Chemical Engineering, Electrical Engineering andEconomics. Mathet-tical techniques related to optimization have been

Ideveloped over the past several hundred years and with the applicationsof modern computers, these techniques are making an impact in many other

areas of science and engineering.

Statistical procedures often require optimization and in a sense

one may regard s-atistics as a subarea of optimization. There are many

applications of optimizing methods in the major branches of statistics

that the study of optimization becomes an important area for the statistician.

The variety and universality of the use of the optimization techniques

can be gauged by a cursory perusal of the contents of the two volumes

on Optimizing Methods in Statistics, Rustagi (1971, 1979). The purpose

here is to develop a logical introduction to important areas of optimization

as they are applied to statistical probleun. Several examples are given

from statistical areas where optimizing techniques play a major role in

their solution. We consider examples from Estimation, Nonparaimetric

Statistics, Design of Regression Experiments, Sample Surveys, Multivariate

Statistics, Inference, Information Theory and Regression Analysis to

motivate the study of optimization.

The scope of optimizing techniques is fairly extensive. However,

we shall put emphasis on those areas of optimization which find

frequent applications in statistical problems. The classical techniques

2

of optimization will be discussed first and numerical methods of optimization

will be discussed next. Linear and nonlinear prograning methods will

be described and variational techniques having connections with dynamic

progrmaming and Pontrryagin Principle will be discussed later. Applications

of other optimizing techniques such as those of Stochastic Approximation

will also be included.

1.2 Statistical examples using classical optimizing techniques.

Exmle: Let XIX 2 ,...,Xn be a random sample from a population2

having a normal distribution with unkown mean. p and uninown variance o 2

The estimation of W and 2 by the Method of Maximum Likelihood requires

maximizing L(,a 2 ) where1 Z(xi_1) 2

2 n 2 2

L(p,a) -(--) e (1.2.1)v2Tro

In this problem, the solution can be obtained by simply equating the

partial derivatives of log L to zero and solving the rtsulting equations

to obtain the necessary conditions for an optimum.

Suppose that there is a constraint imposed on the parameter 1i, say that

pi is always positive. In this case, further attention is to be paid to

the process of optimization to obtain the estimate for j.k

Example: Let pP 2,... 'Pk Pi > 0, and Z pi = 1 be the probabilities

of a trial ending in k possibilities. A sample of n trials, leads to

XlX 2, ... ,x k , occurrences of various possibilities. The maximm likelihood

estimates of plP 2,... 'pk are obtained by maximizing L(pl,... ,p ) such that

X1 Xx2 \kL(p 1 ,P 2,'"'Pk = Pl p 2 " k (1.2.2)

P29__ .. 9P) i 2 P_

A ~ - -- .V ,n m

-i I with constraints p+p 2 +.. + = i. Usual nrthQd of Lgrange's multiplier

rule is used to obtain the solution.

Suppose that there are inequality restrictions such as

I Pji f P2 ..< (12.3)

on the pi's. In that case the estimates have to use the modern methods

I of programming.

Example: Consider a nornad p-variate population with mean andematrix . Lt ,"' be a random sample from the distribution.

The logarithm of the likelihood is a constant multiple of L(Q,2) where

K (4)=-log tr -1..4

with W ? (V- ) C'- .

Again differential calculus provides the maximum likelihood estimates of u

and Z. Suppose an additional sample of size M is given on the first

k(k<p), of the components. Then the problem becomes more complicated.

A recent discussion is due to Anderson and Olkin (1979) for finding

?tximun Likelihood Estimates of k and k. Similar problems arise when

some of the components have missing observations.

SExample: Bounds of serial correlation coefficient for the tine

series xl,X2 ,... are needed. The serial correlation coefficient of 3zh order is

S I defined by

n-Sjtxt+

n t=l (1.2.5)n 2

trl

t!I

I

I Consider the upper bound of I for a 1. This is the problem of

l maximizing -, which is equivalent to

nmax Z xtxt+6

t~l

subject to the constraint

I 2. xt constant.

t=l

Using Lagrange's method, the solution of the above problem can be obtained

I Chanda (1962). Contrary to usual belief that correlation coefficient as

defined is between -1 and +1, the serial correlation coefficient is

I defined in (1.2.5) does have higher bounds than 1.

E:__a!e : (Constrained regression)

The multiple regression model generally assumes that

where y is nxl vector, is a nxp matrix of known constants, 5 is a

pxl vector of parameters and k is a nyl random vectors of errors, with

imeans .and covariance a 2I. The least squares estimates of are

obtained by

However, when 8 is constrained so as to be in a specified set e.g.

> 0' we have a constrained optimization problem and in most cases, such

an optimization problem. reqires the use of modern programming methods.

Example: (Optimal allocation in survey sampling)

A large number of problems in survey sampling require optimum

allocation of resources since the surveys are constrained by total cost,

Ljn or the sampling units. Consider for example the simple case in

L ., .. ... .

Ai

cluster, sampling where one is interested in determining an optimal size

I of a cluster which produces the minimum variance of the sample nean for

a given cost.

I Suppose M be the number of total units to be divided among N clusters

of size H0 each. Let 2w be within-cluster variance and SB be the between-

I cluster variance. Let the sample size selected be of size n. Let the

I corresponding cost C1 associated with a cluster regardless of its size

and Cw be the cost associated with each element regardless of cluster

size. Then for a fixed cost C, we have

C = nCB + n1M4 0 W

and we want to minimize the variance of the overall average, y,

2 2V(y) (i - -) SW O B

-' N (1.2.7)N n 0

The optimal solution turns out to be

opt S (12.8)

For further details and other problems in sampling using optimization

methods, the reader nmay refer to Jessen (1978) and Cochran (19,3).

1.3 Statistical examples using numerical techniques.

Example: Consider the problem of estimation of parameters of the

Gamma distribution

f(x) - e XBx - x > 0 (i,3.1)F (a) K

i =0 elsewhere.

The maximm likelihood estimates are given by equating the partial

derivatives of log L with respect to a and B, where

I _____________

I

A -Fx.

- + (-)E log x.

I - n log f(a) - na log f. (1.3.2)

IThe equations are

x n - '36 (1.3.3)

Elog xi;- I -n log = (1.3.4)

These equations can be solved only through numerical methods. Tables of

I the diaganma function F'C()/F() are available, Pearson and Hartley (1954),

to facilitate the solution.

I Example: (Survival Analysis)

Suppose the times to death of a, individual follow an exponential

Idistribution with parameter A. The number of A deaths observed at

times tl,t 2 ,... t are known out of n individuals under study. The study

is to be analyzed at tine a. The estimate of the parameter A is obrainedn

I by considering the followi.ng likelihood

-X E t.A P1 XaL iAa (l-- ) (1.3.5)

or log 1 : &log A-A Z t- + The likelihood equationi=l

for A is

-a- {+ l Ae-a = l 013.6)

I The above equation admits only a numerical solution for A. The roots of

ithe likelihood equations lead to the maximuIm likelihood estimates.

Example: (Reliability)

jRealistic nodels in reliability theory and survival aralysis rqui-re

nmTerical evaluation frequently. Consider the three parameter WeibullI

1 |~

7

distribution wmdel for the tiJre to failure, for a given individual. The

prohability density function is given by

~4~k) exp[- Ck

f_(t) =t > (- .)

Here 6, 6 > 0 and f > 0.

Suppose the experiment is conducted over the period (0,t O) and the times

of failures of L0 individuals out of t on test are given by t 1 ,t ,

Themn the likelihood of the sanple is given by

r 0 B-1

- ex [-Z

The maximun likelihood estimates of i, 8 and 6 can only be obtained

nunerically. Several such procedures are available in the literature.

Mann, Schafer and Singpurwalia (1974) provide many other mdels in reliability

and survival analysis leading to numerical procedures.

Example: (Curve fitting probie )

Suppose Teasureuwnts of neutron flux (y) are made in a nuclear

r. reactor at various points (x) and the curve to be fitted is

y(x) = A Cos (Bx+E) + C Cosh (Dx+E). (1.3.9)

At points, x., il,2,...,n, adjusted, y, were made and using the

asaumption that variance of the counts is proportional to its mean, It is of

interest to find A, B, C, D and E such that we minimize S with

1

1[ I H l ,

n[y. - A C& co (2-+.) C ,%(,+TM1 2S ( .= 7 . -n)

a-(A Cos (Bx-E) - C Cosh (77-E) (

or to simplify the problem by minimizing S" with

S* = E (a [Yi)y-A Cos (Bx,+E) - C Gosh (Dxi+E)] 2 . (1.3.11)i :

11

I The nonlinear form of the "unction does not allow us to obtain estiriates

of A, B, C, ) and E in olosed form. Hooke and Jeeves (1961) Dnrvide a

Snumerical method by "Direct Search" technique for this optLmization

Iproblem.Exar.le: (Response surface designs)

Consider the following relationship between the mean 1 of a response

variable y and the independent variables k with unknown paraeters k;

I where (xi,... Vx)t is a p-dimensional "design" variable and

(, ... is a a-dimensionai parameter. Since the function is

generally ,unknow.m, it is estimated by polynomial of certain degree and

then the estiirates of Q are obtained by some method of estiition.

l Let y({) estimated response at

The problem Ln response surface designs is to find that design, that is,

,suoh that we minimize

j = I (( .)- 11 ))(1.3.12)

over the class of all

Several approaches are available in the literature. Initial ixrpetus

was provided by Box and Wilson (1951).

'l e_I t k. .... -

IiMatharaticai optimi[zation teclmiques wit,, ineqiaality oonstnaints

Mnst often lead to problems of matheratical proxerrnrng. Therm is a

I corisiderable literature L-i the application of rrahematical programing In

statistics, for a reent survey, see Arthwnari and bdge (1981).

I Exaple: (Linear Fegression)

A comnn mdcel of ninear regression is the following

where y is a vector of n dLensions, , is a nm m-trix of rnown constants,

is a p-dirensional vector of xrlown parameters an(d k is an n-vector

Iof residuals. Suppose it is assured that

> 0 (1.4.1)

where is some gxs known matrix. It is proposed to estimate 3, in general

Iby least squarhes method so as to miniLize

subject to C1.4.1). The above problem then reduces to the problem of

quadratic programrting and tier are weii-bown algorithms to deal with

such problems. Several authors have cont-I;bed to this area, see o r

example Davis (1978), Judge and Takayana (1966).

Example: (Sampling)

IOne of the common problems in sanple surv¢ey is the estimation of the

total y, of a finite population given by

y N "Y (1.4.3)

where yl.,y2 ... ,y.* are the possible individual values of items in the

population; N. is the number of units in the population having values yj.

AN .

wiMMAI ~-

I Let n. be the number of units in the sample with values yi* so that under

simple nandom sampling, of size n, we have n = En., the maximuia likelihood

estimate of y is obtained by maximizing the likelihood

Tn

with n . , being the total sample size. The optimization in this case

reduces to an integer programming problem, Hartley and Rao (1969) in their

paper on A new estimation theory of Sample Surveys. For other problems

in survey sampling using mathematical pronnming, see Rao (1979).

Example: (Design of experiments)

An important class of designs is concerned with factorial experiments.

When the number of factors is large, all treatment combinations cannot

be used in a block of ordinary size and hence fractional factorial designs

1have been developed. A recent introduction in the study of fraction

factorial is the concept of cost optimality. The problems of finding

I cost optimal fraction factorials naturally lead to programming problems,

Neuhardt and Mount-Campbell (1978).

Exaple: (Least absolute value estimate in two-wa classification).

Consider a tram-way classification model

Yijk + i + 3* + e (1.'4,5)

iV 2.. ., Ii,,.,, ~ ,.,

with a i . can be regarded as kth observation at the ith

level of first factor and jth level of second factor.

We obtain the least absolute value estimates of , i' j by minimizing

k -k- y - jl" (1,4.6)ijk ij1

]. ii

This problem is equivalent to the following linear prog'amng problem.

Minimize E E X (d. + d7) (1.4.L7)ijk ijk ijk

subject to

a.i + , + - d'7 Y. (1.4.8)

d +ijk >0

1- -I dijk >01.

A large number of other application of programming methods to Least Absolute

Value Estimation is found in Gentle (1977).

V IExample: (Estimation of Markov chain probabilities)

Consider the problem of estimating the transition probabilities

pij(t) of the Markov chain x t t=1,2...,T, and ijzl,2,..... Here

Pij(t) = Pr{xtzt x t (1.4.9)

where .i, i=1,2,...)r are the finite number of the st-tes of the chain.

Here E Z pij(t) = 1 (1,4.10)

and 0 < p.. < t. (1.4.11)

Suppose the chain is observed for N(t) independent trials. Let w.(t) be7]the proportion of events which fall in jth category. The likelihood

I of the sample, then, can be obtained as follows

T N(t): ( .. 12)L : T(N(t)w (t))!(N(t) - ZN(t)wkt))( .

trl mnm k

I N(t)w. t)!I(wt-l)Pij(),)

IN(t) - E N(t)w(t)k k

(i - S E wi(t-l)pj.(t))ki

£j ,_ _ _. ..

l - ' I .. ... .. " ... . ...... ... i .. ....... . . ... I" .. ... .. II- - -i

12

The problem of raximizing the likelihood in (1.4.12) subject to (1.4.0) a 4d (.-.l)

4 I is a nonlinear Prof~annirg problem. This problem with other rr-anifestat ions;.! is studied by Lee, Judge and Zellner (1968).

-i I1.5 Variational methods in Statistics.

f Classical mthods based on calculus of variations have been used

extensively in applications, especially for studying physical and mechanical

system. Their use in statistics and economics has resulted in new

developments of variational methodolog,. Variational methods are concerned

with optimizations of functionals over a class of functions such as

minimizing or maximizing integrals of functions over a class of functions

subject to certain constraints. In statistics there are irany applications

which depend very heavily on variational techniques. A recent book on the

topic is by Rustagi (1976). We provide a few examples from statistics

using classical and wmdern variational techniques.

Example: (Order statistics)

Suppose x1 < x2 <... < xn is an ordered random sample from a continuous

distribution function F(x). The expectation of the largest order statistic

n is given by

L(F) f x d(Fn(x)). (1.5.1)

An important problem in utilizing order statistics is to find upper and

* lower bounds of L(F) when the mean and variance (say) of the random

* . variable X are given.

Similarly one nay want to find the bounds of the expectation of the

range, X - X, of the sample. That is,

min(max) f x d(l-Fn(x) - (l-F(x))n} (1.5.2)

subJectLO Ceai cost-raints.

a .2 .- 4

13

Such probleis occur in nonparam-tric statistical inference and varicus

generalizations have been discussed by Rustagi (1957).

Example: (Mann-Whitney-Wilcoxon statistic).

Suppose we are interested in the bounds of the variance of Mann-

Whitney-Wilcoxon statistic for various applications such as finding

confidence intervals for p = ?r(X < Y). The integral we minimize (maximize)

reduces to

2(Ef yI) =Lx: - kx)dx (1.5.3)

subject to the condition

SF(xdx (1.5.4)

This is a variational problem and has been treated in detail by Rustagi (1961).

Example: (Efficiency of tests).

Co-nsider a randomsa~l frm a p--ouation havnga....nou

distribution function F(x). Suppose we are interested in testing the

hypothesis

Ho: F(x) G(x)

HI : Gx) = F(x-6)

with e as some location parameter. The relative asymptotic efficiency

of Wilcoxon test with respect to t-test (which will be used if F and G were

normal distributions) is given by

I(f) = ff2 (x)dx (145,5)

where f(x) is the corre.sponding probability densivy function of X.

A problem of internst in nonparanetric in'-feCrence is to find bounds

of I(f) subject to side conditions sucL as,

. . .L-

I 14

[ Ii f(x)dx I

I f xf(x)dx 0.

For details, the reader is referred to Hodges and Lehm -nr (1956).

IExample: (Regression designs)

Consider a simple linear regression model,

I y =- + ax + C

where a, $ are the unknown parmeters, x is the independent variable

2 1and c is the error wi_h mean 0 and variance a ., In regression design

of experiments, the investigator is interested in allocating n observations

at X,...,xn so as to optimize certain function of the estirated parameters.

A cormon criterion of optimality is the D-optinality whiere the deteiranant

of the covariance ma rix of the estimet& factor of parameters is ptimized.

For example, given a sample of size n the covariance of the (^, is -1:b, where

" x. 2 2Zx.nE( (x -x)" nE(xi± )2

M -: (1,5,6)

2 .

Assu,.ming that o is kncvwn, the problem of optimal regression design is

to find xlX2,... ,xn such tl.tt the determinant of M is maximized. There

are many other criteria of optimiity, a detailed account is available in

Federov (1971).

I Example: (Robustness)

M-estimates of a location parameter, 0, for d probabilit density

Sfunction f(x-e), with cuii'u!tive distribution function, F(x-e), are defLned

by Huber (1972). A Statistic 7n based on the random sanmle, X1 X2,. ..,X,21 .. nT n

from Z(x-8) is an M-estimate if it maximizes Z D(xi-T ), for some metric P.

i is given by tne equation (1.5.7).

i , -i, -. _, . ... ¢ --.Z .'= -4 -,, A:- " --

!4

X *(x.-T ) -0 (1.5.7)iEb a n

with 4 2 p' Note that if p(x) -x', we getL lest suases stimtesand f ;(x) f(x)'leas s:arsetmte n fp - .we get m~aximum likelihood

estimates. It has been show-n by Huber under fairly general conditions

that the asymptotic variance of T is V(,F), where

V(p,F) [,(x-T) (dx)J(x-T) ( ]F(dx) ,(,s)

with Tn + T almost surely as n -* . The problem is to find an F over the

class of functions F which minimizes V(G,F). This reduces to a variational

problem. Unicueness and existence of the solutions have been discussed

by Huber (1972). Many other problems related to robustness studies

leading to the applications of variational rethods have beer. recently

discussed by Bickel (1965), Portnov (1977), and Collins and Fortnoy (1979).

Example: (Admissibility)

Let Q-, ) be the m-dimnsional multivariate norral density of a

random vector . Let 4(k) be an estinator of , and let the loss function be

L , ) - , -

with as a known matrix.

Suppose GQ) is the prior distribution function on R, REe ) L(,6()},

and the Bayes risk is denoted by

EGG,%) 2 R(,4)G(dQ). (1.5.9)

Then the Bayes estimator with prior G(k), is given by

or 6%Q) x + g , (1.5.11)I !

i 11

III when g*(x) !p I(x)C(d ) and Ag()denotes the gr'adient vector ofr *()

The sufficient condition for an estimator 6 F;) to be admissible is

the following:

I "TLhere exists non-negative firte Borel measures, G. il,2,...

G. having compact support with Gi({0}) = 1, such that

i B(Gi,6) B(Gi,%G.) 0 o (1.5.12)

as 1 - .

j The above condition (1.5.12) reduces to

1/2

~1 Arri112I f*(x)dx

I Minimizing I(g*,f*) answers the problem of admissibility of the estimator

{F(K) using techniques of calculus of variations. An elaborate account

is in Brown (1971).

Example: (Penalized mximum likelihood estimation)

For various reasons, the estimation of the probability density function

- f(x) based on a sample X,...,X is ,mde using a known penalty function

Let the likelihood (penalized) be

L(f) = x. . (1.5.14)iTl

The problem of finding penalized maximun likelihood estimates is to find

max L(f) subject to constraints,

ff(x)dx = 1

and f(x) > 0.

tis "Zation p-,oblem reduces to a problem in variational methods.

*Detailed discussion of this and related problems is given by De Montricher,

Tapia and Thompson (1975).

17

Refexrnces

Arthaari, T. S. and Dodge, Yadokah. Mathematical Pirograrui4 in Statistics,John Wiley and Sons, New York, 1981.

Bickel, P. (1965). On some robust estimates of location, Ann,. Math. Statist.36, 847-858.

Box, G.E.P. and Wilson, K. B. (1951). On the experimental attainm:-t ofoptimum conditions, J. R. Statist. Soc. 13, 1-45.

Brown, L. D. (1971). Admissible esti-c tors, recur-rent diffusions, andinsoluble boundary value pzroblems, Anrn. Math. Statist. 42, 885-9C3.

SCochran, W. 0. (1963). Sapling Techniques (2nd Edition), John Wiley andSons, New York.

Collins, J. R. and Portnoy, S. 1. (1981). Maximizing the variance ofM-estinatorns using the generalized method of moment spaces, Ann.Statist. 9, 567-577.

Davis, William W. (1978). Bayesian analysis of the linear model subjectto linear inequality constraints, J. Amer. Statist. Assoc. 73, 575-579.

De bntricher, G. F., Tapia, R. A., and Thompson, J. F. (1975). Nonparametricmaximum likelihood estimation of probability densities by penaltyfunction methods, Ann. Statist. 3, 1329-1348.

Federov, V. V. (1971). Theory of Optijaal Experiments, Academic Press,New York.

Fletcher, .. Practical Methods of 1-timization, Vol. 1. UconstrainedOptimizat-ion, John Wiley and Sons, New York, 1980.

Gentle, J. E. (1977) (Editor). Special issue of Coinunications in Statistl. -Simulation and Computations B 6, 313-438.

Hartley, H. 0. and Rao, J.N.K. (1969). A new estimation theory of samplesurveys, I!, New Developments in Survey Sampling, (Editors, N. L.Johnson and Harry Smith), Wiley-Interscience, New York, 147-169.

Hodges, J. L. and Lehmann, E. L. (1956). The efficiency of some non-parametric competitors of the t-test, Ann. Math. Statist. 27, 324-355.

Hooke, R. and Jeenes, T. A. (1961). Direct search solutions of numericaland statistical problems, J. Assoc. Comvut. Mach. 8, 212-219.

Huber, P. (1972). Robust statistics-A review, Ann. Math. Statist. 43,1042-1067.

Jessen, Raymond, J. (197S). Statis r -c Survey Techniques, John Wileyand Sons, New York.

\ .-J 7 7-- - -

Judge , G. G. and Takaya, T. (1966). Inequality restrintions L er :~o

anaysis, J. Amr.-Statist. Assoc. 61, 166-181.

nKupper, Lawrence L. and MeydSech, Edward F. (1973). A new approach tomtan squared error estimation of response surfaces, Biometrika 6S0,573-S79.lelhd

Lee, T. C., judge, G. G., and Zellner. A. (1958). eecximum of oehoi andBayesia estimation of transiti n probailities, J. Amer. Statist.Assoc. 693, 1162-1179.

Mann , . andR. E. , and Singp0 1alla N. D. (1974). Meths forStatistical Ana lysis of Reliability n ieaa o~ ie and Sons,New York.yan

ieaaJonWly -

Neu~iardt, J. au-d 11bunt-Campbell, C. A. (1978). Selection of cos-t-op'.tirm.a12k-p fractional factorials, Commm. Statist. -Sinrala.Coi ua B 7

369-383.

Pearson, K. and Hruztley, H. 0. (1954). Biometrika Tables for Stati-stici,-ans,

Vol. I and I!, Cambridge University Press, Cambridge.

Portnoy, S. L. (1977). Robust estisration in dependent situations, Ann.Statist. 5, 22-43.

Rao, J.N.K. (1979). Optinization in the design of sample surveys,Optimizing Methods in Statistics, (Editor J. S. Rustagi), AcademicPress, New York, 419-434.

Rustagi, J. S. (1957). On minimizing and maximizing a certain integralwith statistical applications, Ann. Math. Statist. 28, 309-328.

Rustagi, J. S. (1961). On the bounds of variance of .Mann-Mitney statistic,Amn.Inst. Statist. Math., 119-126.

Rustagi, J. S. (1976). Variational Methods in Statistics, Acadmlic Pness,

New York.

i_I"

II