Exercise 2Exercise 4

Exercise 3Exercise 5

Exercise 2

(a)

Exercise 4Exercise 6

Exercise 2

b. We begin with SSE = e0e = (Y �X�)0(Y �X�). This can be written as:

SSE = e0e = (Y �X�)0(Y �X�) = (Y0��0X0

)(Y �X�) =

Y0Y �Y0X� � �0X0Y + �0X0X�.

Note: In the previous expression the second and third terms in the right-hand side are equal because

they are both scalars. The transpose of a scalar equal to itself. Also, if we substitute � = (X0X)�1X0Y

in the last term of the right-hand side this will become:

�0X0X� = �0X0X(X0X)�1X0Y = �0X0Y.

And using Y = X� we can show that

SSE = Y0Y � �0X0X� = Y0Y � �0X0Y.

c. Consider the multiple regression model

y = X� + ✏, with E(✏)=0 and cov(✏)=�2I.

Show that

(Y �X�)0(Y �X�) = (Y �X�)0(Y �X�) + (� � �)0X0X(� � �).

SOLUTION:

In the left hand side add and subtract X�:

(Y �X�)0(Y �X�) = (Y �X� +X� �X�)0(Y �X� +X� �X�)

=

⇣(Y �X�) +X(� � �)

⌘0 ⇣(Y �X�) +X(� � �)

⌘

= (Y �X�)0(Y �X�) + (� � �)0X0X(� � �)

+ (Y �X�)0X(� � �) + (� � �)0X0(Y �X�).

The last two terms are zero because e0X = 0. Therefore,

(Y �X�)0(Y �X�) = (Y �X�)0(Y �X�) + (� � �)0X0X(� � �).