Statistical Papers 35, 127-138 (1994) Statistical Papers �9 Springer-Verlag 1994

fl-expectation tolerance region for the heteroscedastic multiple regression model with multivariate Student-t error S. Khan

Received: April 27, 1992; revised version: March 4, 1994

A ~q-expectation tolerance region has been constructed for the multivariate regression model with heteroscedastic errors which follow a multivariate Student-t distribution with an unknown number of degrees of freedom. The /3-expectaion tolerance region obtained in this pal~er is optimal in the sense of having minimum enclosure among all such tolerance regions that guarantees that it would cover any preassigned proportions, namely, fl x 100 percent of the future responses from the model.

A M S 1991 Sub j ec t Class i f ica t ion: Primary 62F25, Secondary 62J05

K e y w o r d s and Phrases: 13-ezpec~ation ~olerance region, he~eroscedastic variance, multiple regression model, iuvarian~ differentials, prediction distribution, structural re- lation, multivariate Siudent-t dLs~ribution, F-distribution.

1. I N T R O D U C T I O N

Statistical tolerance regions are an important branch of inference. Guttman (1970)

summarized the theory of tolerance region and discussed the constructions of different

kinds of tolerance regions from classical and Bayesian viewpoints. He also pointed out a

wide range of applications of such regions in statistical theory as well as for the quality

control process in the industries. The particular type of tolerance region that we are

interested in here is the /3-expectation tolerance region. It is a region defined on the

sample space that on the average encloses a preassigned proportion, /3 of the responses

from a specific model. In this paper a /3-expectation tolerance region is constructed

from the distribution of the future responses. However, in this case the region would

enclose future responses from the model such that the expected value of its coverage

equals /3. It is to be noted that it is an inference about the future responses itself, that

includes the parameters of the model, rather than about any particular parameters of

the model. Aitchison and Du.n_~aore (!975) presented an extensive study on tolerance

regions and discussed many applications of such a tolerance region with real world

examples.

128

Wilson (1967) proposed tolerance regions for the parameters of the regression

model. Haq and Pdnco (1976) obtained fl-expectation tolerance region for the gen-

eralized multivariate model with independent normal errors by using a structural dis-

tribution. Haq (1982) used the structural relation to find the prediction distribution

for the multinormal model. Bishop (1976) proved that the fl-expectation tolerance

region based on a prediction distribution is optimal in the sense of having minimum

enclosure among all such regions. However, those works are based on the assumption

of independent errors only.

In this paper, we consider a heteroscedastic multiple regression model as given in

(2.2) and (2.3) whose subsequent realizations are not independent and follow a mul-

tivariate Student-t distribution. First, the methodology for the construction of a fl-

expectaion tolerance region for dependent responses is developed by using the prediction

distribution of the future responses. The region thus constructed is indeed an optimal

tolerance region (see Bishop (1976) for instance) in the sense of having minimum en-

closure. The joint density of the estimated regression coefficients, scale factor and the

future errors is derived from the conditional structural model by using the properties of

invariant differential (see Eaton, 1983, p.207-217 or Fraser, 1968, p.59-63). The predic-

tion distribution is then obtained by using the structural relation of the model. Unlike

Bayesian and structural distribution approaches this method avoids multidimensional

integration over the product space of the parameters. Finally, from the prediction dis-

tribution, the fl-expectation tolerance region, R(Y) has been derived. Such a region

would, on the average, enclose fl x 100 percent of the future responses generated from

the model in (2.2). Here, we assume that the sample size would be large enough to

satisfy n _> m + p , where n is the observed sample size, and m and p are the dimen-

sions of the regression matrix. Two different cases of the model under investigation are

considered: one with a single future response, and the other with an arbitrary number

of future responses, to construct fl-expectation tolerance regions.

2. T H E H E T E R O S C E D A S T I C M U L T I P L E R E G R E S S I O N M O D E L

W I T H M U L T I V A R I A T E S T U D E N T - t E R R O R

Often in time series data, the responses in the subsequent period of time are not

independent and the distribution has heavier tails than those of the normal distribution.

In such cases, Zellner (1976) suggested the multivariate Student-t distribution for the

error terms of the multiple regression model. He cited a number of previous studies with

similar assumption on the responses. The usual assumption of homoscedastic error term

is not always satisfactory either. When the components of the error vector, and hence

that of the response vector~ have unequal variances the assumption of heteroscedastic

error variance is essential. Furthermore, if the components of the error vector are not

129

independent, the joint density can not be written as the product of the marginal densities

of the components. Both, dependence and inequality of variances as well as the non-

normality can be taken care of by considering a heteroscedastic model with Student-t

distribution. Moreover, the Student-t distribution is a more general distribution in the

sense that it converges to the normal distribution when the number of degrees of freedom

grows large, and it becomes the Cauchy distribution when the degrees of freedom equMs

l .

A heteroscedastic multiple linear regression model can be written in the following

form:

y# = ~ x j + r e # for j = 1,2, . . . , n (2.1)

where the response vector, yj is the m-dimensional column vector representing the

j th response variables; X j is the j th p-dimensional column vector of the design

matrix; /3 is the regression matrix of order m • p , and F > 0 is the rn • rn order

diagonal matrix of the unequal scale parameters; emd r is the column vector of the

stochastic error variables associated with the responses. The error vector ej is assumed

to have a standardized multivariate Student-t probability distribution with r degrees

of freedom. Therefore, E(y~) = flXj and Coy(y/) = ~-~ ,~ , in which 2[7 = F F ' .

The responses in (2.1) can be expressed in the following matrix notation:

Y = f i X + r E (2.2)

where Y, X, and E are the matrices of order m x n , p x n and m x n respectively.

Here the expected value of E is O, an m • n matrix of zeros, but the covariance

matrix of the elements of E is In | Ira, and that of r E is ~ = In | ~ , where | is

the Kronecker product between two matrices.

Let us assume that the error variable E of the model specified in (2.2) follows a

standardized Student- t probability distribution with an unknown number, r, of degrees

of freedom. Thus, the probability density function of E can be written as

p ( E ) - ( r 7 r ) ~ r ( ~ ) (2.3)

where tr is the /~race operator and F(a') represents the usual gamma function with

argument a .

3. f l - E X P E C T A T I O N T O L E R A N C E R E G I O N : G E N E R A L

For the model under discussion, we derive fl-expectation tolerance regions for a

single future response as well as for a set of future responses. Rinco (1973), and Haq

and Rinco (1976), among others, showed that a /3-expectation tolerance region can be

130

based on the prediction distribution. They assumed normal and independent errors to

derive such a tolerance region for the generalized multivariate model with homoscedastic

responses. Khan (1992) has shown that for a given set of dependent responses y, a

3-expectation tolerance region, R(Y) defined as

E[C(R)[y] = 3 (3.1)

where C(R) is the coverage probability of R(Y) and the expectation is taken with

respect to the distribution of the parameter O, p(Sly ). In Bayesian approach the

expectation is taken with respect to the posterior density and it is replaced by the

structural density in structural method. In either of the cases, this would also be based

on the prediction distribution. This could be established by considering a set of future

responses, Yl from the model as follows:

y f = ~2r -~ Fef

where ~f is the vector of future predetermined variables. Then denoting the predictive

density of the future responses by p(yf [Y) it is evident that

/ p(yf , y)dyf = fR Jfnp(y,,O , y)dOdy , (3.2)

where P(YI:O ] y) is the joint density function of Y! and O, for given y . Since

the realized error vector, e and the future error vector, ef are not independent, so

YI and G are assumed to be not independent as well, and hence the density can't be

factored. However, by applying the rule of conditional probability and assuming that

the conditions of Fublni's theorem hold, it is observed that

/RP(Yf l y)dyI = /R ~ P(O I y)P(yf I O, y)dOdy,

= s I o, y)p(Oj )eyfeo

J[o P[Y! e R(Y) [ O, y] p(O [ Y) dO

=Ee [C(R) [ Y] = 3 (3.3)

where p(O ! y) is the density of the parameter | for given y . Haq and Rinco (1976)

obtained a /3-expectation tolerance region for homoscedastic independent normal errors

by using the structural distribution, instead of Bayes' posterior distribution. The result

in (3.3) thus allows us to construct a /3-expectation tolerance region with an expected

coverage probability /3 by using the predictive density of the dependent responses. In

the next section we outline the derivation of the prediction distribution by using this

structural relation.

131

4. T H E P R E D I C T I O N D I S T R I B U T I O N S

-expectation tolerance regions are defined on the sample space of the observed

responses that has an expected probability value /3 which is computed with respect to

the predictive density of the future responses. Thus to construct such a tolerance region

we require to obtain the prediction distribution first.

Towards the prediction density, we define the following statistics based on the

realized but unobserved errors:

B(E) = E X ' ( X X ' ) -1, the regression matrix of E on X;

s~(ei) = [ei - bi(ei)X][ei - bi(ei)X]', squared ith residual length;

C(E) = d i ag .{ sa (e i ) , s2 (e2 ) , " ' , sm(em)} , so tha tS(E) = C(E)C'(E); and

D(E) = C-a(E){E - B(E)X} , the standardized residual matrix.

Note that ei and bi(ei) are the i t h rows of E and B(E) respectively, and C(E)

is the square root factorisation of S(E). It can be easily shown that for a given set of

responses, Y from the model D(E) = D(Y) . Then the error matr ix E can be expressed as

E = B(E)X -4- C(E)D(E) (4.1)

and the structural relation can be written as

B(Y) = ~ +/ 'B(E), S(Y) = PC(E) (4.2)

where B(Y) is the regression matr ix of Y on X, and S(Y) is defined in the same

way as S(E) when E is replaced by Y.

Now, from the distribution of the error matrix E, the conditional probabili ty

element of the unobserved B(E) and C(E), given D(E) , can be obtained by using

the invariant differentials as follows: m

p(B(E) ,C(E)[D(E)) = r sn-p-l(ei) i=1 (4.3)

• + r

where r is the normalizing constant, and s~(ei) is the i th diagonal element of

S(E) = C(E)C'(E) for i = 1 , 2 , . . . , m .

The joint density function in (4.3) can also be expressed as

m

i=, (4.4) r - ~ m n

X 1 -4- - bi(ei)XX'b'i(ei ) -t- s~(ei r

i - - 1

132

where b/(e,) = { h , ( e j , b~,(e ,) , - . - , bp/(e,)} is the regression vector of e, on X . Ap- plying the same arguments as above, the prediction distribution can be obtained from

the joint density of the observed as well as the future error terms by exploitlng the

structural relations of the model. The details appear in the following two sub-sections.

4.1 T h e Case o f a S ing le F u t u r e R e s p o n s e

Let YI be a future response from the model (2.2) and (2.3), so that

Y! = ,3XI + F e i (4.5)

where X I is a p x 1 matr ix of the future predetermined variable, and e$ is the future

error vector associated with YI �9 Note that the location and and the scale parameters,

/3 and F in (4.5), have the same specifications as given for those in (2.1). Using the

structural relation of the model we can express (4.5) as follows:

YI = B ( Y ) X I - C ( Y ) C - I ( E ) B ( E ) X f + C(Y) c -1 (E)e l (4.6)

This yields the structural relation as follows:

C-1(Y){YI - B ( Y ) X I } = C - I ( E ) { e l - B (E)XI} . (4.7)

Since the future response is not independent of the previous responses but they are

uncorrelated, the joint density of E and e I can be given by

p(Z,e:)- ~ r 1 + { W = , e i e ~ + e : e ) } (4.8) ( ~ ) = r(~)

From (4.8) the joint density of bi(ei)'s, si(ei)'s, and el, conditional on D(E), is

obtained as r n

i=1 (4.9) [ " ]_,%.+~

~',{b,(e,)XX'bS(e/) + ,~(~,) + d~} x l + - g

i = 1

where K(D) is the normalizing constant and eil is the i th component of e I .

Note tha t the rows of X are orthogonal to D and DD' = I m �9 The normalizing

constant can be evaluated by using the Student- t integral and the Beta integral of the

second kind.

Thus integrating with respect to e I and hi(el) 's by using the multivariate-t in-

tegral and then by using the multivariate Beta integral of the second kind to integrate

out the .si(ei) 's, we obtain,

g ( D ) = 2m [XX'[ ~ F( ~ =2--+-m~) (4.10)

133

It is important and interesting to note that the normalizing constant does not depend

on D(E) and hence the conditional distribution is the same as the unconditional dis-

tribution.

We will first find the distribution of s'~l(el){ely - b i (e i )X f} from (4.9) and then

using the relation between Y! and e l , as given in (4.7), we shall obtain the required

prediction distribution of Yy. Let us make the following transformations:

wi -'- s[l(ei){ei . f - b i (e i )X f }

ui = bi(ei) (4.11)

vi = ~ , (e , )

where elf is the i th element of e I and wi is the i th element of w , each being a scalar

quantity. The Jacobian of the transformation is vi �9 Therefore, the joint probability

density function of ui, vi, and wi ' s becomes

m

/ : 1

x [1+ u i ( X X ' ' ' § X f X f ) u i § 2 v i u i w i X f § V2(1 + W i=1

(4.12)

with IQ (r, m, n,p) as the normalizing constant.

Collecting the terms involving u i ' s in (4.12) we have,

uiAu~ + 2viuiwiXy =(u i + viwiX~lA-1)A(ui + v ,w,X ' iA-1) ' (4.13)

2 2 t --I - v iw i X I A X f

where A = X X ~ § XIX t l is a p x p symmetric matrix. Setting this quadratic form,

(4.13) in to (4.12) and then integrating out u i ' s over ~P, and v~ 's over the positive

half of the real line, ~+, we get the marginal density for w as follows:

p(w, : i = 1 , 2 , . . . , m ) = IXX' l~[r("+'~- )]m (Tr) ~-IA]~" IF( ~ 2--~)] m

(4.14) m

• [1+ i=l

Now the prediction density of Yf is obtained from (4.14) by using the relation (4.7) as

follows:

(,,i) liftS] p Y = K2(r, rn, n ,p i=1 (4.15)

134

where QI = (1 - X ; A - 1 X I ) . The prediction distribution of each of the rn components

of the future response, Y/ has a univariate Student- t distribution (not independent)

with n - p degrees of freedom. We would use this predictive density to obtain a /9 -

expectation tolerance region. Notice that the prediction density does not depend on the

degrees of freedom, r of the original or the future sample responses.

4.2 T h e C a s e o f a Se t of F u t u r e R e s p o n s e s

In this section we would deal with a more general case by taking an arbi t rary

number of future responses from the model. Let us consider a set of n' future responses

( n ' _> 2 ) from the model (2.2) and (2.3), so that

Yl = ;~xl + t e l (4.16)

where fl and F are regression and scale parameters as before. The design matr ix

X l is an m x n' matr ix of full rank, and E l is an m x n' matr ix of future errors

corresponding to the future responses YI" Here n' may or may not be equal to n.

Again we assume that every response is dependent on the previous response but they

are uncorrelated. Therefore, following the same logic as we have used in the previous

section, the joint probability density function of E and E l can be expressed as

i m n m n t

where ~,(,, m, n, n') = r ( ' - ' G +"o ) / ( , ~ ) ~ r ( f ) . The joint probability density function of B(E), S(E), and E / , for given D(E), is obtained by following the same

argument as in (4.9), as follows:

r n

(4.1s)

r i=1

where e~ I is the i th row of E / and qdI(r, rn, n, n I) is the normalizing constant.

The normalizing constant kl'2(r , rn, n, n') can be evaluated by using the multivariate-

t integral and the Beta integral of the second kind. The integrations result in,

r = 2" IXX' I �9 r( "~"'~+""' ) n - - r n 7" ~ (,-7,) , [r("=-y)] r(~)

Now for the set of future responses described in (4.16) the structural relation can

be expressed in the following form:

C - ' ( Y ) { Y l - B ( Y ) X / } = C - ' ( E ) { E / - B (E)X / } .

135

In search of the prediction distribution of YI, first we derive the density function of the

statistic on the right hand side of the above structural relation from the distribution in

(4.18). Thus, substi tuting

Wi = s-~l(ei){eif -- bi(ei)Xf }

"u.i = hi(el)

in (4.18) and the 3acobian factor v i , we obtain the joint probabil i ty density function

of ui, vi, and wi 's, for all i 's, as follows:

m = +o ' - , - ' ] i = 1

m 1 - " + " ~ + = " '

• [1 + _1 E { u i ( X X , + XsXs)u ' + 2viuiXfw; + v~(1 + wiw;~ ] r

i - - 1

(4.19) The terms involving u l ' s in (4.19) can be expressed as

t t -- : l uiAu] + 2viuiXlw ~ =(ui + v iwiXlA )A(ui + viwiX~A -1)' 2 t - - 1 t

- v i wiXsA Xfwi

for i = 1 : 2 , . . . , m ; where A = XX~+.XyXCf is a p• symmetric matrix. Integrating

out ui's and v i ' s from (4.19) the marginal density of w i ' s becomes

p(wi : i = l , 2 , ' " , m ) = ~ t iAI�89 J (4.20)

_ t - 1 t -

i = 1

Thus each of the w i ' s is distributed (not independently) as an n ~ -dimensional multi-

variate Student-t variable with mean vector O, scale factor [In, - X}A-~XI] -�89 and

(n - p) degrees of freedom.

The prediction distr ibution of Yy is now obtained from (4.20) by using the struc-

tural relation as follows:

p(Y11Y)= k~s(n,n',p) f i [s~n'(yi)] i=1 (4.21)

where Q2 = [I,, - X~yA-'XI]. Clearly, each of the m rows of Yy, conditionM on the

corresponding row of Y, is distr ibuted as a multivariate-t variable.

136

Thus the prediction distribution of a set of future responses Y! has an m n ~-

dimensional multivariate Student-t distribution with ( n - p ) d.f.. Note that the degrees

of freedom of the prediction distribution does not depend on the degrees of freedom of

the original or the future sample responses, r.

The prediction distribution of the i th row of Yy, has the location parameter

b i ( y i ) X l and the scale parameter si(yi)[In, - X } A - 1 X I ] -�89 �9 It is interesting to note

that the prediction distribution for the same model with normal error distribution also

has a multivariate-t distribution.

5. ~ - E X P E C T A T I O N T O L E R A N C E R E G I O N : M U L T I L I N E A R M O D E L

In this section, we construct ]~-expectation tolerance regions for a single future

response as well as for a set of future responses from the model by using the prediction

distribution obtained in the previous section.

5.1 T h e To le rance Reg ion For a Single F u t u r e R e s p o n s e

From (4.16), it is evident that

p)] , �89 [(n - { Y f - B ( Y ) X I } S - ~ ( Y ) Q 1 ] (5.1)

has an m-dimensional multivariate Student-t distribution with (n - p ) degrees of

freedom. Using the property of the multivariate Student-t distribution the statistic

has an F-distribution with m and n - p degrees of freedom. Thus~ a E-expectation

tolerance region that will enclose 100 • fl percent of the future responses from the

model is the region:

(5.3)

where Fm,.-p,1-;~ is the (1 - 8) • 100 percent point of a central F-distribution with

rn and (n - p ) degrees of freedom. In the particular case the F-value can be replaced

by the univariate t-value when m = 1 as F(1, . -p , 1-;~) = t2 (~-p, ~-~)"

5.2 T h e Case o f a Set o f F u t u r e Responses

The prediction distribution of a set of future responses Y$ as given in (4.21) reveals

that the statistic

137

a central F distribution with rnn I and ( n - p) d.f.. Therefore, a region that will

enclose /3 x 100 percent of the future responses, Y! from the model is given by

(5.5) where Q2 = [ I , , - X ' I A - ~ X I ] and Fm, ' , , -m~-Z is the ( 1 - / 3 ) x 100 percent point of

a central F-distribution with rnn' and ( n - p ) degrees of freedom.

The predictive regions obtained in (5.5) do not depend on the parameters of the

model (2.2) and are valid for all possible values of /3 and F . Therefore, the afore-

discussed /3-expectation tolerance regions are indeed similar predictive regions (see

Aitchison asld Dunsmore, 1975, p.88-107) with expected coverage of size /3.

A c k n o w l e d g e m e n t : The author is very grateful to an unknown rcfcrcc for some very

useful comments and suggestions, and Professor M.S. Haq for his valuable advice that

improved the final version of the paper significm~tly.

R E F E R E N C E S

Aitchison, J. and Dunsmore, I.R. (1975). Statistical Prediction Analysis. Cambridge Unlver~ty Press, Cambridge.

Bishop, J. (1976). Parametric tolerance regions. Ph.D. Thesis, Dept. of Statistics, University of Toronto, Canada.

Eaton, M.L. (1983). Multivariate Statistics - A Vector Space Approach. Wiley, New York.

Fraser, D.A.S. (1968). The Structure of Inference. Wiley, New York.

Guttman, I. (1970). Statistical Tolerance Regions : Classical and Bayesian. Griffin, London.

Haq, M.S. (1982). Structural relation and prediction for multivariate models. Sr162 che Hef~e, 23, 218-228.

Haq, M.S. and Rinco, S. (1976). /3-expectation tolerance regions for a generalized multivariate model with normal error variables. J. of Multivariate Analysis 6 ,414- 432.

Khan, S. (1992). Predictive Inference for the Multilinear Models with Multivariate Student- t errors. Unpublished Ph.D. Thesis, Dept. of Statistics, Univ. of Western Ontario, Canada.

Rinco, S. (1973). /3-expectation tolerance regions based on the structural models. Ph.D. Thesis, Dept. of Statistics, Unv. of Western Ontario, Canada.

Wilson: A.L. (1967). An approach to simultaneous tolerance intervals in regression. Ann. Maf.h. Statis~., 38, 1536-40.

138

Zellner, A. (1976). Bayesian and non-Bayesian analysis of the regression model with multivariate Student-t error term. JASA, 60, 608-616.

Shahjahan Khan Department of Mathematics & Computing

University of Southern Queensland

Toowoomba, Qld. 4350

Australia

and

Institute of Statistical Research & Training

University of Dhaka, Dhaka-1000

Bangladesh