matrix algebra of linear models - 國立中興大學benz.nchu.edu.tw/~kucst/matrix algebra.pdf ·...
TRANSCRIPT
1
March 1, 2016
The Basic Matrix Algebra in Linear Models
Chapter 1:Deal with generalized inverse matrices allied topics
Chapter 2:Extending to sections on the distribution of quadratic and
bilinear forms and the singular multinomial distribution
Chapter 3:Full Rank models
A sample explanation of regression →multiple regression
A unified treatment for testing a general linear hypothesis
Chapter 4:Models not of full rank
Dummy (0, 1) variables
Estimable functions
Non- estimable functions
Chapter 5:Non -full-rank model
Testing any testable linear hypothesis
Chapter 6 - Chapter 8:Give many details for the analysis of unbalanced
data (Unequal-subclass-numbers data).
Chapter 9 - Chapter 11:Data with variance components
2
Occupation
Education
High School
Incomplete High School College Graduate
Laborer 14 8 7
Artisan 10 - -
Professional - 17 22
Self-employed 3 9 10
Unequal numbers of observations in the subclasses including perhaps
some that contain no observations at all ⟹ unequal-numbers data,
unbalanced data, “messy” data.
“A” generalized inverse of a matrix A is defined, as any matrix G that
satisfies the equation.
AGA = A
Another name such as Conditional inverse
Pseudo inverse
g - inverse
G for a given matrix A is not unique.
To illustrate the existence of G & its non-uniqueness
If A has order 𝑝 × 𝑞
( )
( ) ( ) ( )
r q rr r
p p p q q q p q
p r r p r q r
ODP A Q
O O
More simply,
rD OPAQ
OO
P & Q are products of elementary operations.
r is the rank of A & Dr is a diagonal matrix of order r.
3
Matrix Algebra
1
m
ij j
i
a a
The sum of the diagonal element of a square matrix is called the trace of
the matrix, written tr (A).
i.e., for A = {𝑎𝑖𝑗} for 𝑖, 𝑗 = 1, … , 𝑛
tr (A) = 𝑎11 + 𝑎22 + ⋯ + 𝑎𝑛𝑛 = ∑ 𝑎𝑖𝑖𝑛𝑖=1
Example:
1 7 6
8 3 9 1 3 8 4
4 2 8
tr
When A is not square, the trace is not defined. That is, it does not exist.
tr(𝐴′) = tr(𝐴)
tr(𝐴 + 𝐵) = tr(𝐴) + tr(𝐵)
(𝐴 + 𝐵)′ = 𝐴′ + 𝐵′
| ( , )
ith row
r c
c s r s
ith i j th
elementcolumn
2x3
3 4
2 4
:
0 6 1 51 0 2
1 1 0 71 4 3
3 4 4 3
6 14 9 11
13 10 11 32
x
x
eg
A B
AB
Please refer to Chapter 1- Chapter 4
of the book “Matrix Algebra useful
for Statistics”.
.
4
AB is described as A post multiplied by B, or as A multiplied on the
right by B.
scalar
vector
Matrix (Matrices)
Identity matrices
(Unit matrix)
When A is of order p×q
I p A p×q = A p×q I q = A p×q
(The transpose of a product)
(AB) = B A
:eg
𝐴𝐵 = [1 0 −12 −1 3
] [1 1 10 2 43 0 7
] = [−2 1 −611 0 19
]
𝐵′𝐴′ = [1 0 31 2 01 4 7
] [10
−1
2−13
] = [−21
−6
110
19] = (𝐴𝐵)′
(The trace of a product)
tr (AB) = tr (BA)
Note that tr(AB) exists only if AB is square, which occurs only when A is
r×c and B is c×r. Then if AB = P = {pij} and BA = T = {tij}
tr(AB) = ∑ p𝑖𝑗𝑟𝑖=1 = ∑ (∑ a𝑖𝑗
𝑐𝑗=1 b𝑗𝑖)𝑟
𝑖=1 = ∑ (∑ b𝑗𝑖a𝑖𝑗𝑐𝑗=1 )𝑟
𝑖=1
= ∑ (∑ b𝑗𝑖a𝑖𝑗𝑟𝑖=1 )𝑐
𝑗=1 = ∑ (t𝑖𝑗) = tr(BA)𝑐𝑗=1
2 3
1 0 01 0
I and I 0 1 00 1
0 0 1
5
Partitioned matrices
11 12 11
21 22 21
A A BA= and B=
A A B
Then
11 12 11 11 11 12 21
21 22 21 21 11 22 21
A A B A B +A BAB= =
A A B A B +A B
(The laws of Algebra)
a. Associative Laws
(A+B)+C = A+ (B+C)
(AB)C = A (BC) = ABC
b. The distributive Laws
A (B+C) = AB+AC
c. Commutative Laws
A+B = B+A
But AB = BA
(?)
When AB
BA
both exist and are of the same order, they are not in general
equal.
1 2 0 -1 2 -3 0 -1 1 2 -3 -4= =
3 4 1 -1 4 -7 1 -1 3 4 -2 -2
IA=AI=A
0A=A0=0 for A being square
(Contrasts with scalar algebra)
AX+BX = (A+B) X
XA+XB = X (A+B)
6
But XP+QX generally does not have X as a factor.
(Notice) Even though
AB = 0, neither A nor B is 0
Further, BA = 2B
BA-2B = 0
B (A-2I) = 0
We cannot conclude either that A-2I = 0 or B = 0
(NO!)
Similarly, x2 = 0 ⟹ x = 0
e.g.
1 2 5
x= 2 4 10
-1 -2 -5
we have x2 = 0
Likewise, Y2 = I implies neither Y = I nor Y = -I
e.g. 21 0 1 0
Y= but Y =I=4 -1 0 1
Similarly, we can have M2 = M with both M≠I and M≠0
M = [3 −23 −2
] = M2
A square matrix is defined as symmetric when it equals its transpose;
i.e.,
A is symmetric when A = A , with aij = aji for i, j = 1,…, r for A r×r
(AB) = B A =BA
A A=0 implies A=0
tr(A A)=0 implies A=0
Recall that if a sum of square of real numbers is zero, then each of the
number is zero.
i.e. for real numbers x1,x2,…,xn ,2
i
i=1
x =0n
implies
1 2 ...... 0nx x x
c
2
j=1 =1 =1
tr(A A)= ( th diagonal element of A A)= ac r
kj
j k
j
Pxx =Qxx implies Px = Qx
(proof:)
(Pxx -Qxx )(P -Q )=(Px-Qx)x (P -Q )=(Px-Qx)(Px-Qx) =0
Px-Qx=0 i.e., Px=Qx
(Sums of outer products)
7
aj is the jth column of A & B j is the j th row of B
1
c
21 2 c j j
j=1
c
AB= a a a = a
Thus AB is the sum of outer products of columns of A with corresponding
row in B.
1 4 43 48 1 4 7 8 36 407 8
AB= 2 5 = 59 66 = 2 7 8 + 5 9 10 = 14 16 + 45 509 10
3 6 75 84 3 6 21 24 54 60
(Elementary Vectors) For ei being the ith column of In, namely a vector with unity for its ith
element and zero elsewhere.
ei is called an elementary vector.
1 2 12 1 2
1 0 0 1 0
e = 0 and e = 1 , E =e e = 0 0 0
0 0 0 0 0
ij i jE =e e is null except for element (i, j) being unity n
n i i
1
I = e ei
n
3 2 3 2
2
n n
2
n n n n
1 1, ,1
1 1 =n
1 1 1
1 1 1 1 1 = 1 1 =J
1 1 1
1 1 =J having all elements unity
J 1 1 with J =nJ
1and J = J with J J
n
n
n
n
r s r s
n n n
;
8
and for statistics
n n n
1C =I-J I- J
n
Known as a centering matrix
First observe that
2, 1 0 and CJ=JC=0C C C C
(Here are the vectors.)
Define
1 2
1 2 11 1( )
1 1
111 1
n
n
i
n i
i
x x x x
xx x x
x x xn n n n
x C x x J x x x x x xn
Is the data vector with each observation expressed as a deviation from x ?
(This is the origin of the same centering matrix for C)
being a data vector
1 is the mean
is the vector of deviations from the mean
is the sum of squares about the mean
x
xn
x C
x Cx
e.g.
2
1 12 2
1 12 2
C
A special case of the form X AX is known as a quadratic form, which can be used for
sums of squares.
2
22 2 2
1 1
( 1 ) (1 )
( )
n n
i i
i i
x Cx x x x x x x x x x nx
x x x nx x x nx x Cx
9
Idempotent Matrices:(From “same” ”power” [Latin])
When k is such that
k2 = k, we say k is idempotent.
kr = k for r being a positive integer
k2 = k implies (I-k)2 = I-k
(I-k) (I-k) = I-k-k+k2 = I-k
But (k-I) is not idempotent.
e.g.
2
I-J is idempotent
1 -12 2
C ex. C =-1 1
2 2
GA is idempotent whenever G is such that AGA = A
(A matrix G of the nature is called a generalized inverse of A)
Orthogonal Matrix:
Another useful class of Matrices
AA =I=A A Such matrices are called orthogonal.
The norm of a real vector 1 2x = x nx x is defined as norm of n
12 2i
i=1
X= X X ( x )
A vector is said to be either normal or a unit vector when its norm is
unity i.e., when X X=1
Any non-null vector x can be changed into a unit vector by multiplying it
by the scalar (1 X X );i.e., 1
u=( )XX X
is the normalized form of X
(because u u=1 ).
Non-null vectors X and Y are described as being orthogonal when X Y=0
e.g.
X = 1 2 2 4 X Y=0
Y = 6 3 -2 -2
Two vectors are defined as orthonormal vectors when they are
orthogonal and normal.
(e.g.)
u and v are orthonormal
When
u u=1=v v & u v=0
1u = 1 1 3 3 4
are orthonormal vectors6
v = -0.1 -0.7 -0.1 -0.4 0.4
10
The vectors of an orthonormal set are all normal, and pairwise orthogonal.
A matrix Pr×c whose rows constitute an orthonormal set of vectors is said
to have orthonormal rows, where rPP =I .
But P P is not necessarily an identity matrix Ic.
(e.g.)
2 3
1 0 01 0 0
P= PP =I but P P= 0 1 0 I0 1 0
0 0 0
Conversely , when Pr×c have orthonormal columns cP P=I but PP may
not be an identity matrix.
2PP =P P=I (P called orthogonal matrix)
Any two of the conditions implies the third.
(i) P square
(ii) P P=I (P has orthonormal columns)
(iii) PP =I (P has orthonormal rows)
(e.g.)
2 2 21
A= 3 - 3 06
2 1 -2
is an orthogonal matrix easily verify!
Quadratic Forms
n
2
i
i=1
(x -x) x cx
General form x Ax Any sum of squares can be represented as x Ax
1
2 2 2
1 2 3 2 1 2 1 3 1 1 2 2 3 2 1 3 2 3 3
3
2 2 2
1 1 2 1 3 2 2 3 3
2
i ii i j ij ji
i j>i
1 2 3 x
x Ax= x x x 4 7 6 x x +4x x +2x x +2x x +7x -2x x +3x x +6x x +5x
2 2 5 x
=x +x x (4+2)+x x (2+3)+7x +x x (-2+6)+5x
So, x Ax= x a + x x (a +a )
i ix x =1
for all i
i jx x =0
for =1,2, ,ni j
11
2 2 2
1 2 3 1 2 1 3 2 3x Ax=x +7x +5x +6x x +5x x +4x x
1 1 1 1 2 3
=x 5 7 0 x or x 4 7 6 x or
4 4 5 2 -2 5
IF A is symmetric then
11 3 2
2
x 3 7 2 x
12 2 5
2
for any particular quadratic
form there is a unique symmetric matrix A for which the quadratic form
can be expressed as x Ax .
When A is not symmetric then 1
(A+A )2
is symmetric. Hereafter,
whenever we deal with a quadratic form x Ax , we assume A=A .
(Positive definite Matrices) 1. When x Ax > 0 for all x other than x = 0 then x Ax is a positive
definite quadratic form and A=A is correspondingly a positive definite
(p.d.) matrix.
(e.g.)
1
2 2 2
1 2 3 2 1 2 3 1 2 1 3 2 3
3
2 2 2
1 2 1 3 2 3
2 2 1 x
x Ax= x x x 2 5 1 x 2x +5x +2x +4x x +2x x +2x x
1 1 2 x
=(x +2x ) +(x +x ) +(x +x )
2. When x Ax 0 for all x and x Ax=0 for some x 0 then x Ax is a
positive semi definite quadratic form and hence A=A is a positive semi
definite (p.s.d.) matrix.
p.d.
non-negative definite (n.n.d.).
p.s.d.
(e.g.)
1
2 2 2
1 2 3 2 1 2 1 3 2 3
3
37 -2 -24 x
x Ax= x x x -2 13 -3 x =(x -2x ) +(6x -4x ) +(3x -x )
-24 -3 17 x
This is zero for x = 2 1 3
(e.g.)
n2
i
i=1
(x -x) x cx is p.s.d.
C=I-J is idempotent
p.s.d.
12
(Determinants) 7 3 7 3
A= A 7(6) 3(4) 304 6 4 6
1 2 35 6 4 6 4 5
A 4 5 6 1( 1) 2( 1) 3( 1) 38 10 7 10 7 8
7 8 10
N-order determinants n
i+j
ij ij
j=1
A = a (-1) M for any i
When expanding by elements of a row.
(1) A = A
(2) If two rows of A are two same, A = 0
1 4 35 2 7 2 7 5
7 5 2 = -4 +3 =05 2 7 2 7 5
7 5 2
(3)Cofactors
C𝑖𝑗 = (−1)𝑖+𝑗|M𝑖𝑗| Where Mij is A with its ith row and jth column
deleted.
(4)Add multiple of a row (column) to a row (column).
DO NOT affect the value of the determinant.
1 3 2
A = 8 17 21 =1(17-147)-3(8-42)+2(56-34)=16
2 7 1
1 3 2 1 3 2
A = 8+4 17+12 21+8 = 12 29 29 =1(29-203)-3(12-58)+2(84-58)=16
2 7 1 2 7 1
(5) AB = A B When A and B are square and of the same order n.
(6)P 0
= P QX Q
For R and S square and of the same order n. o R
= R-I S
I A A 0 0 AB=
0 I -I B -I B
I A A 0 0 AB= A B = AB
0 I -I B -I B
(Corollaries)
(1) AB = BA (because A B = B A )
13
(2)22A = A (each equals A A )
(3)For orthogonal A, 2
A = 1 (because AA =I implies A =1)
(4)For idempotent A, 22A =0 or 1 (because A =A implies A = A )
(Elementary row operations)
21elementary operator matrix P (4)adding 4 times row1 to row2
ij
1 3 2 1 0 0 1 3 2
12 29 29 = 4 1 0 8 17 21
2 7 1 0 0 1 2 7 1
1 3 2 1 0 0 1 3 2 1 3 2
12 29 29 = 4 1 0 8 17 21 = 8 17 21
2 7 1 0 0 1 2 7 1 2 7 1
E = I with ith and jth row
ii
s interchanged.
R (λ) = I with ith diagonal element replaced by .
(e.g.)
12
33
ij ii ij
0 1 0 1 3 2 8 17 21
E A= 1 0 0 8 17 21 = 1 3 2
0 0 1 2 7 1 2 7 1
1 0 0 1 3 2 1 3 2
R (5)A= 0 1 0 8 17 21 = 8 17 21
0 0 5 2 7 1 10 35 5
P (λ) =1, R (λ) =λ and E =-1
4 6 2 3=2
1 7 1 7
A + B A+B
(Chapter 5, 6 and 7) of matrix algebra…..為前述介紹之內容
Chapter 8 generalized inverses
Addition, subtraction and multiplication have already been dealt.
Division does not exist in matrix algebra.
The concept of “dividing” by A is replaced by the concept of multiplying
by a matrix called the inverse of A.
The inverse of a square matrix A is a matrix whose product with A is the
identity matrix.
A-1 the inverse of A(A-inverse;A to the (power of) minus one.)
Ax = b
As x = A-1b
14
Where A-1A = I
A-1A = A A-1 = I and A-1 unique for given A.
( Derivation of the inverse ) tedious!
1 2 3
A= 4 5 6
7 8 10
Derive the cofactors of each column.
First column 1+1 2+1 3+1
5 6 2 3 2 3(-1) =2 (-1) =4 (-1) =-3
8 10 8 10 5 6
A =2(1)+4(4)-3(7)=-3
Second column:2, -11 & 6
Third column:-3, 6 & -3
Now consider the matrix obtained by replacing the elements of A by their
cofactors.
i.e.,
[1 2 34 5 67 8 10
]obtaining[2 2 −34 −11 6
−3 6 −3]
Transpose & multiply it by 1A
, it’s inverse
2 4 -31
2 -11 6-3
-3 6 -3
How to get the inverse function?
11 12 13
21 22 23
31 32 33
a a a
A= a a a
a a a
Formed a new matrix by replacing each element of A by its cofactor.
11 12 13
21 22 23
31 32 33
C C C
C C C
C C C
This was transposed, giving 11 21 31
12 22 32
13 23 33
C C C
C C C adjugate adjoint
C C C
Multiplied by the scalar 1A
1 1 2 1 3 1
-1
12 22 32
13 23 33
C C C
1 C C C =AA
C C C
If 10A A does not exist.
15
So
transposed
-1A with every element1
A =replaced by its cofactorA
(1)A-1 can exist only when A is square
(2)A-1 does exist only if A is nonzero when its determinant is zero, a
square matrix is said to be singular.
Properties of the inverse:
IF A is a square, nonsingular matrix
A-1 was the following properties.
(1) -1 -1A A=AA =I
(2)The inverse of A is unique
-1
-1 1 1 -1
because if S is another inverse different from A then SA=I,
SAA ,so S=A contradict!IA A
-1 -1-1
-1
A A = AA = I =11(3) A =
A 1A =
(4)The inverse matrix is nonsingular A
(5) -1 -1(A ) =A -1 -1 -1 -1 -1 -1 because I=A A, (A ) =(A ) A A=IA=A -1 -1 -1 -1 -1 -1(6)(A ) =(A ) because I=AA , I=I =(AA ) =(A ) A =(A ) A
(7)If A =A then -1 -1 -1 -1 -1(A ) =A because (A ) =(A ) =A
(8) -1 -1 -1 -1 -1 -1 -1 -1 -1 -1(AB) =B A because B A AB=B (A A)B=B IB=B B=I=(AB) AB
-1 -1 -1(AB) =B A
(Some simple special cases)
-1a X 1
A= has A for ab-XY 0Y b
b X
Y aab XY
1 12
14
13
200 00
0 40 0 0
0 03 00
[ A | I ] → → [ I | A-1 ] providing A-1 exists
16
(Chapter 6 Rank)
Ax = b, x = A-1b only if A-1 exists
And A-1 does exists only if |A|≠0
→permit us to ascertain whether or not |A| is zero, without having to
tediously expand |A| in full.
(Linear combinations of vectors)
(Refer to Page 7 outer products)
X= [x1 x2 ……xn] and a= [a1 a2……an]
1 1 2 2
1
......n
n n i i
i
a x a x a x a x
𝑋𝑎
𝑋𝑎 is a column vector, a linear combination of the columns of x
Similarly, b x is a row vector, it is a linear combination of the rows of x.
AB is a matrix:
Its rows are linear combinations of the rows of B ,and its columns are
linear combinations of the columns of A.
(Linear transformations)
𝑋𝑎 is called the linear transformation of the vector a to the vector xa,
with x being the matrix of the transformation.
y =Ax represents the linear transformation of x to y.
(Linear dependence & independence)
○1 Definitions
The product Xa is a vector, and it is a linear combination of the
column vector in X
Xa = a1x1+a2x2+……+anxn
Linearly dependent vectors:
If there exists a vector a 0, such that a1x1+a2x2+……+anxn = 0, then
provided none of x1, x2, …, xn is null.
Alternative :If Xa =0 for some non-null a
17
Then the column of x are linearly dependent vectors;
provided none is null.
Linearly independent vectors:
If a = 0 is the only vector for which a1x1+a2x2+……+anxn=0, then
provided none of x1, x2, …, xn is null, those vectors are said to be linearly
independent vectors.
(Alternative)
If Xa = 0 only for a =0, then the columns of x are linearly
independent vectors.
To sum up , Xa = 0 being true for some a≠0 means the columns of x
are linearly dependent, whereas it being true only for a = 0 means they
are linearly independent.
(The properties of linearly dependent vectors)
(a) At least two a s are nonzero
Because, x1, x2, …, xp are linearly dependent. When
a1x1+a2x2+……+apxp = 0, for not all the a s being zero
Suppose only one a is nonzero called 2a , there 2a 2x = 0
2a = 0 because 2x is not null.
Contradict
Therefore, more than one a is nonzero.
(b) Vectors are linear combination of others.
Suppose that a1 & a2 are nonzero
2
1 11 2( ) ...... ( ) 0paa
pa ax x x
32
1 1 11 2 3( ) ( ) ...... ( )paaa
pa a ax x x x
i.e., 𝑥1 can be expressed as a linear combination of the other x s .
(c) Partitioning Matrices
(d) Zero determinants
18
Suppose p linearly dependent vectors of order p are used as columns of
a matrix.
→linear dependence of vectors implies that one vector can always be
expressed as a linear combination of the others.
→ determinant = 0
(e.g.) 1 2 3
3 0 2
6 5 1
9 5 1
x x x
Subtracting ( 3 32 32 2
x x ) from 𝑥1
1 2 3
0 0 2
0 5 1 0
0 5 1
x x x
(e) Inverse Matrices
When the column (rows) of a square matrix are linearly dependent, that
matrix has not inverse. → Singular
because |A| = 0
(f) Testing for dependence
A simple test for linear dependence among p vector of order p is to
evaluate the determinant of the matrix formed from using the vectors as
columns.
That is, Zero determinant linearly dependent.
Otherwise LIN
Given a set of vectors, their dependence
independence
can be ascertained by
attempting to solve Xa = 0. If a solution can be found other than a = 0
→ it will be a non-null solution
→ the vector dependent.
Otherwise → LIN
19
Furthermore, for square x that has no null columns.
(i) Columns of x are linearly dependent.
equivalence
(ii) Xa = 0 can be satisfied for a non-null a.
(iii) x is singular, i.e., x-1 does not exist.
(LIN vectors)
a. Nonzero determinants and inverse matrices
(i)Columns of x are LIN
(ii) Xa = 0 only for a = 0
(iii) x is nonsingular, i.e., x-1 exists
b. A max. number of LIN vectors.
Theorem:A set of LIN vectors of order n cannot contain more than n
such vectors.
Corollary:When p vectors of order n are LIN then p≦n.
(pf):Let u1, u2, …, un be n LIN vectors of order n.
Let un+1 be any other non-null vector of order n.
We show that it and u1, u2, …, un linearly dependent.
Since U = [ u1, u2, …, un] has LIN columns, |U|≠0 & U-1 exist.
Let q = -U-1 un+1≠0 because un+1≠0, i.e., not all elements of q is zero.
Then Uq+Un+1 = 0, which can be rewritten as
q1u1+q2u2+…+qnun+un+1 = 0
With not all the q s being zeros.
→u1, u2, …,un+1 are linearly dependent.
𝑒𝑞𝑢𝑖𝑣𝑎𝑙𝑒𝑛𝑐𝑒 {
20
Theorem:The number of LIN rows in a matrix is the same as the
number of LIN columns.
(The Rank of a Matrix)
Definition: The rank of a matrix is the number of linearly independent
row (and columns) in the matrix.
Notation The rank of A → rA or r(A)
If rA ≡ r(A) = k then A has k LIN rows and columns.
(Some properties of rank)
(i) rA is a positive integer.
(ii) ( )p qr A p and q
(iii) ( )n nr A n
(iv) when Ar =r 0 there is at least one square sub matrix of A having
order r that is nonsingular.
( )
( ) ( ) ( )
r r r q r
p q
p r r p r q r
X YA
Z W
And X r×r is nonsingular.
All square sub matrices of order greater than r are singular.
(v) When ( )n nr A n then A is nonsingular & A-1 exist.
(vi) When ( )n nr A n then A is singular & A-1 does not exist.
(vii) When ( )p qr A p q , A is said to have full row rank, or to be of full
row rank. Its rank equals its number of rows.
(viii) When ( )p qr A q p , A is said to have full column rank.
(ix) When ( )n nr A n , A is said to have full rank, or to be of full rank. Its
21
rank equals its order, it is nonsingular, and its inverse exists. It is said to
be invertible.
Equivalent Statement of the existence of A-1 of order n “Inverse existing”:
1. A-1 exists
2. A is nonsingular
3. |A|≠0
4. A has full rank
5. rA = n
6. A has n LIN rows
7. A has n LIN columns
8. Ax = 0 has sole solution, x = 0
Permutation Matrices
For example,
1 1 3
1 1 3
4 4 12
2 2 5
M
24
1 0 0 0
0 0 0 1
0 0 1 0
0 1 0 0
E
24
1 0 0 0 1 1 3 1 1 3
0 0 0 1 1 1 3 2 2 5
0 0 1 0 4 4 12 4 4 12
0 1 0 0 2 2 5 1 1 3
E M
E24 is an identity matrix with its second and fourth rows
interchanged, and E24M is M with those same two rows interchanged.
Ers →symmetric orthogonal ( rsrs rs rsE E E E I )
In the same way that premultiplication of M by Ers interchanges rows
r and s of M, so does post multiplication interchange columns.
24 23
1 1 3 1 3 11 0 0
2 2 5 2 5 20 0 1
4 4 12 4 12 40 1 0
1 1 3 1 3 1
E ME
Consider
22
1 1 3 2
1 1 3 2
3 3 9 6
2 2 5 4
1 1 7 8
A
25 34 25
1 1 3 2 1 1 3 2
1 1 3 2 1 1 7 8
2 2 5 4 2 2 5 4
3 3 9 6 3 3 9 6
1 1 7 8 1 1 3 2
PA E E A E
Where
25 34 25
1 0 0 0 0 1 0 0 0 0
0 1 0 0 0 0 0 0 0 1
0 0 0 1 0 0 0 0 1 0
0 0 1 0 0 0 0 1 0 0
0 0 0 0 1 0 1 0 0 0
P E E E
For Q = E24
1 2 3 1
1 8 7 1
2 4 5 2
3 6 9 3
1 2 3 1
PAQ
P is a product of elementary permutation matrices (the E-matrices)
P is not necessarily symmetric, but it is always orthogonal. (Because it is
a product of orthogonal E-matrices)
So 1P P is also a permutation matrix.
(P is defined as an identity matrix with its rows resequenced, it is also an
identity matrix with its columns resequenced.)
Canonical Forms
3 elementary operators matrices
(Row operations)
12
0 1 0 1 1 1 2 2 2
1 0 0 2 2 2 1 1 1
0 0 1 3 3 3 3 3 3
E A
23
Rii ( ) multiplies the ith row of A by
22
1 0 0 1 1 1 1 1 1
(4) 0 4 0 2 2 2 8 8 8
0 0 1 3 3 3 3 3 3
R A
Pij ( ) A adds times the jth row of A to its ith row
12
1 0 1 1 1 1 2 1 2 1 2
( ) 0 1 0 2 2 2 2 2 2
0 0 1 3 3 3 3 3 3
P A
(Transposes)
ij ijE E
( ) ( )ii iiR R
And ( ) ( )ij jiP P
(Column operations)
Post multiplication by elementary operators performs similar
manipulations on the columns of A.
(e.g.)
12
1 1 1 1 0 0 1 1 1
( ) 2 2 2 1 0 2 2 2 2
3 3 3 0 0 1 3 3 3 3
A P
Inverses:
( ) 1ijP 1
ij ijE E
( )ijR 1
1( ) ( )ii iiR R
1ijE 1
( ) ( )ij ijP P
(Rank and the elementary operators)
The rank of a matrix is unaffected when it is multiplied by an
24
elementary operator.
r (EA) = r (A)
R-type
P-typethe same
The independence of rows is unaffected and the same number will be
linearly independent after making the product.
So,
2 3 2( ) ( , ) ( , ) ( , )r A r E A r E E A r E E E A
= r [PA]
It is done by using the operators of elementary operators to change A
until its rank is obvious.
(Equivalence)
When A is multiplied by elementary operator matrices, the product is said
to be equivalent to A
e.g. B = PAQ → B A
P and Q is the product of elementary operators
A = P-1BQ-1 A B Thus rA = rB
(Calculating Rank)
(e.g.)
𝐴 = [1 23 −15 −4
4 32 −20 −7
]
𝐴 = [1 20 −70 −14
4 3
−10 −11−20 −22
]
𝐴 ≅ [1 20 −70 0
4 3
−10 −110 0
] = 𝐵 rank = 2
r (B) = r (A)
(Row operations)
B = PA
= (PI) A
(−3)
(−5)
(−2)
25
= (E3E2E1I) A
P = E3E2E1I can be derived by carrying out on I the same row
operations as have been made on A to derive B
(Continued)
3
1 0 0 1 0 0 1 0 0
0 1 0 3 1 0 3 1 0
0 0 1 5 0 1 1 2 1
I p
1 0 0 1 2 4 3 1 2 4 3
3 1 0 3 1 2 2 0 7 10 11
1 2 1 5 4 0 7 0 0 0 0
PA
The same as before
(Column operations)
1 2 4 3 1 0 0 0 1 0 0 0 1 0 0 0
0 7 10 11 0 7 10 11 0 7 0 0 0 1 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
PA B c
Q was obtained by carrying out on an identity matrix the column
operations contained.
1 2 4 3
0 1 0 0
0 0 1 0
0 0 0 1
and then
8 17 7
10 117 7
1 2
0 1
0 0 1 0
0 0 0 1
Finally,
82 17 7 7
101 117 7 7
1
0
0 0 1 0
0 0 0 1
Q
26
82 17 7 7
101 117 7 7
11 0 0 1 2 4 3
03 1 0 3 1 2 2
0 0 1 01 2 1 5 4 0 7
0 0 0 1
1 0 0 0
0 1 0 0
0 0 0 0
PAQ
C
(The equivalent canonical form)
Theorem: (Its importance is that it always exists)
Any non-null matrix A of rank r is equivalent to
0
0 0
rIPAQ C
Where Ir is the identity matrix of order r, and the null sub matrices
are of approximate order to make C the same order as A. For A of
order m n , P and Q are nonsingular matrices of order m and n,
respectively, being products of elementary operators.