general vector spaces
TRANSCRIPT
Lecture 16
General Vector Spaces
Basis of Row Space and Column Space
Rank and Nullity
Covered Range : 4.8 +οΌ.9
2021/11/17Lec16_LA21_ Singling Lee 1
Terminologies
Row Space ;
Column Space ;
Null Space ;
2021/11/17Lec16_LA21_ Singling Lee 2
Recap Row(A), Col(A)and Null(A)
Definition 2 : A is an mΓnmatrix A=[aij]
Row(A) β Rn: Row space of A β span{r1,r2,β¦,rm}
Col(A) β Rm: Column space of A β span{c1,c2,β¦,cn}
Null(A) β Rn: Null space of A β Solution of Ax=0
Note : Ax=0 always has a solution x=0 (Orthogonal to each row vector)
The solution set is a vector subspace of Rn
2021/11/17Lec16_LA21_ Singling Lee 3
Recap Col (A) and the Solution of Ax=b
Theorem4.7.1 Ax = b is consistent if and only if b is in the column space of A.
Means b β Col(A) = span {c1, c2, β¦ ,cn}.
Ax : Linear combinations of column vectors of A .
2021/11/17Lec16_LA21_ Singling Lee 4
Recap Orthogonality of Row(A) and Null(A)
Theorem 3.4.3 : If π΄ is an π Γ π matrix, then the solution set of the homogeneous linear system π΄π± = π consists of all vectors in π π that are orthogonal to every row vector of π΄.
2021/11/17Lec16_LA21_ Singling Lee 5
Recap Relation Between Ax = 0 and Ax = b
Theorem 4.8.2 : If x0 is any solution of a consistent linear system of Ax = b and ifS={v1 ,v2 , β¦ , vk} is a basis for the null pace of A, then every solution of Ax = b can be expressed in the form
x = x0 + c1v1 + c2v2 + β¦ + ckvk
2021/11/17Lec16_LA21_ Singling Lee 6
Recap Subspace of Ax = 0 by Spanning (II)
Example 6 :
x2 = r, x4 = s, x5 = t
Solution : π± = (β3π β 4π β 2π‘, π, β2π , π , π‘, 0)
parametric equation : (x1, x2, x3, x4, x5 , x6) =
r (-3, 1, 0, 0, 0, 0) + s (-4, 0, -2, 1, 0, 0) + t (-2, 0, 0, 0, 1, 0)
β π―π = β3,1,0,0,0,0 , π―2 = β4,0, β2,1,0,0 , π―3 = (β2,0,0,0,1,0)
linearly ind. span the solution space and dimension = 3
2021/11/17Lec16_LA21_ Singling Lee 7
1 3 0 4 2 0 00 0 1 2 0 0 00 0 0 0 0 1 00 0 0 0 0 0 0
Recap Row Operations : Row(A) ,Null(A)
Theorem 4.8.3 :
a) Row equivalent matrices have same row space
b) Row equivalent matrices have same null space
Elementary row operations do not change the row space and null space of a matrix
2021/11/17Lec16_LA21_ Singling Lee 8
Recap Bases for Row(A) and Col(A) on ref Matrix
Theorem 4.8.4 :
If a matrix R is in row echelon form, then the rowvectors with the leading 1's (the nonzero rowvectors) form a basis for the row space of R, and thecolumn vectors with the leading 1's of the rowvectors from a basis for the column space of R.
2021/11/17Lec16_LA21_ Singling Lee 9
Recap Bases for Row(A) and Col(A) on ref Matrix
Example 3 pp 268 : (Row-Echelon Matrix)
π =
1 β2 5 0 30 1 3 0 00 0 0 1 00 0 0 0 0 Theorem 4.8.4
π«π = 1 β2 5 0 3π«π = 0 1 3 0 0π«π = 0 0 0 1 0
ππ =
1000
, ππ =
β2100
, ππ =
0010
β
2021/11/17Lec16_LA21_ Singling Lee 10
Recap Find the Basis of Row(A)
Find the basis of Row(A):
1. Use Gaussian Elimination to construct R : R : ref matrix of A,
2. Basis of Row Space of A are formed by βrows with leading 1'sβ in R.
2021/11/17Lec16_LA21_ Singling Lee 11
Recap Basis for Column Space Theorem 4.8.5 : If A and B are row equivalent matrices,
1) A given set of column vectors of A is linearly independent iff the corresponding column vectors of Bis linearly independent
2) A given set of column vectors of A forms a basis of the column space of A iff the corresponding column vectors of B forms a basis for the column space of B
How to use this theorem?
2021/11/17Lec16_LA21_ Singling Lee 12
Recap Examples for Basis of Col(A) Example 5 pp.270 Basis for Col(A)
π΄ =
1 β3 4 β2 5 42 β6 9 β1 8 22 β6 9 β1 9 7
β1 3 β4 2 β5 β4
β
R =
1 β3 4 β2 5 40 0 1 3 β2 β60 0 0 0 1 50 0 0 0 0 0
The correspond vectors in A :
c1, c2, c5 is a basis of Col(A)
2021/11/17Lec16_LA21_ Singling Lee 13
Reduce A to a row-echelon matrix (Gaussian Elimination) R
π1β² =
1000
π3β² =
4100
π5β² =
5β210
β
c1β, cοΌβ, c5β,
basis of Col(R)
ππ =
122
β1
ππ =
499
β4
π5 =
589
β5
Lecture 16
General Vector Spaces
Basis of Row Space and Column Space
Rank and Nullity
Covered Range : 4.8 +οΌ.9
2021/11/17Lec16_LA21_ Singling Lee 14
Basis from the Rows of AExample 6 pp. 270: Find a basis for row(A) spanned by r1, r2, r3, r4 of A
Consisting entirely of row vectors from A {δ½Ώη¨εδΎηvectors r1, r2, r3, r4 }
Use AT:
β column 1, 2, 4 for AT
2021/11/17Lec16_LA21_ Singling Lee 15
1 2 0 0 3
2 5 3 2 6
0 5 15 10 0
2 16 18 8 6
A
1 2 0 2 1 2 0 2
2 5 5 6 0 1 5 10
0 3 15 18 0 0 0 1
0 2 10 8 0 0 0 0
3 6 0 6 0 0 0 0
TA
1 2 31, 2,0,0,3 2, 5, 3, 2,6 2,6,18,8,6 r r r
β consists entirely of row vectors from A
Find Basis Vectors and Linear Comb. of Nonbasis Vectors
Example 8 pp. 272 :
Find a subset of vectors v1=(1, -2, 0, 3) v2=(2, -5,-3, 6) v3=(0, 1, 3, 0) v4=(2, -1, 4, -7) v5=(5, -8, 1, 2) that forms a basis
1) Use original vector as column vectors Column Space
2) Express each vector by the basis
2021/11/17Lec16_LA21_ Singling Lee 16
1 2 0 2 5β2 β5 1 β1 β80 β3 3 4 13 6 0 β7 2
β
1 0 2 0 10 1 β1 0 10 0 0 1 10 0 0 0 0
π―π π―π π―π π―π π―π π°π π°π π°π π°π π° π
{π―π , π―π , π―π } β {π°π , π°π , π°π }
Find Basis Vectors and Linear Comb. of Nonbasis Vectors
Example 8 pp. 272 :
2) Express each vector by the basis
(b) w3 = 2w1 - w2 v3 = 2v1 - v2
w5 = w1 + w2 + w4 v5 = v1 + v2 + v4
2021/11/17Lec16_LA21_ Singling Lee 17
0130
= 2
1β203
β
2β5β36
π―3 π―1 π―2
5β812
=
1β230
+
2β5β36
+
2β147
π―5 π―1 π―2 π―4
1 2 0 2 5β2 β5 1 β1 β80 β3 3 4 13 6 0 β7 2
β
1 0 2 0 10 1 β1 0 10 0 0 1 10 0 0 0 0
π―π π―π π―π π―π π―π π°π π°π π°π π°π π° π
{π―π , π―π , π―π } β {π°π , π°π , π°π }
Since πΈπ―π = π1πΈπ―π + β― + πππΈπ―π β π―π = πππ―π + β― + πππ―π!
Steps to Find a Basis of Vectors and Linear Comb. Of Nonbasis Vectors
Given S = {v1, v2, β¦, vk } β Rn
Find a basis β S for span(S)
1) Matrix A has v1, v2, β¦, vk as it column vectors
2) Reduce A to reduced row-echelon form R.
3) Columns that contain leading 1βs in Rβ The corresponding vectors of A are the basis
4) Express each nonbasis column vector of R as a linear combination of the leading 1 columns in R!
5) Replace the linear combination of nonbasis column vectors in R by the corresponding column vectors in A
2021/11/17Lec16_LA21_ Singling Lee 18
Recap General Questions
Question 1 : What are the relationships between Ax = b and Row Space, Column Space and Null Space of the coefficient matrix A ?
Question 2: What are the relationships among Row(A), Col(A) and Null(A) of a given matrix A ?
2021/11/17Lec16_LA21_ Singling Lee 19
Conclusion
Ax = b is consistent if and only if b β Col(A),
Use row echelon form (ref) to find basis of Row(A) and Col(A).
Use reduced row echelon form (rref) to find basis of Null(A).
General solutions of Ax = b : Special solution + Span (Basis of Null(A))
Orthogonality of Row(A) and Null(A).
2021/11/17Lec16_LA21_ Singling Lee 20
Lecture 16
General Vector Spaces
Basis of Row Space and Column Space
Rank and Nullity
Covered Range : 4.8 +οΌ.9
2021/11/17Lec16_LA21_ Singling Lee 21
dim(Row(A)) = dim(Col(A))Theorem 4.9.1 : The row space and the column space
have the same dimension !
Basis of Row(A) : The leading one rows in R (row-echelon matrix)!
Basis of Col(A) : Column vectors in A corresponding to the leading one columns in R (row-echelon matrix)!
2021/11/17Lec16_LA21_ Singling Lee 22
Rank and Nullity
Definition 1 : Given a matrix A : mΓn
Rank(A) = dim(row space of A) = dim(column space of A)
Nullity(A) = dim(null space of A)
Rank(A) = Rank(AT)
Rank(A) β¦min(m , n)
2021/11/17Lec16_LA21_ Singling Lee 23
Examples for Rank and Nullity (I) Example 1 pp 277 : Rank and Nullity
The solution space of Ax=0
2021/11/17Lec16_LA21_ Singling Lee 24
A=
β1 2 0 4 5 β332
β7β5
22
04
16
41
4 β9 2 β4 β4 7
->
1 0 β4 β28 β37 1300
10
β20
β120
β160
50
0 0 0 0 0 0
From Row1: π₯1β4π₯3 β28 π₯4 β37 π₯5 + 13π₯6 = 0
Row2: π₯2 β2π₯3 β 12π₯4 β16 π₯4 + 5π₯6 = 0
π₯1 =4π₯3 + 28π₯4 + 37π₯5 β 13π₯6 ; π₯2 = 2π₯3 +12π₯4 + 16π₯5 β 5π₯6
π₯3= r οΌ π₯4 = π οΌ π₯5 = π‘οΌ π₯6 = π’π₯1= 4π + 28π + 37π‘ β 13π’π₯2= 2π + 12π + 16π‘ β 5π’
Examples for Rank and Nullity (II)
The Solution of Ax= 0 with 4 parameters
Rank =2 , dim(Null(A)) = 4; Nullity = 4
2021/11/17Lec16_LA21_ Singling Lee 25
π₯3= r οΌ π₯4 = π οΌ π₯5 = π‘οΌ π₯6 = π’
A=
β1 2 0 4 5 β332
β7β5
22
04
16
41
4 β9 2 β4 β4 7
->
1 0 β4 β28 β37 1300
10
β20
β120
β160
50
0 0 0 0 0 0
π₯1= 4π + 28π + 37π‘ β 13π’π₯2= 2π + 12π + 16π‘ β 5π’
π₯1
π₯2
π₯3
π₯4
π₯5
π₯6
= r
421000
+ s
28120100
+ t
37160010
+ u
β13β50001
Find the basis of Null(A) (I)
Find the basis of Null(A):
1. Use Gaussian Elimination to construct RR : RR : Reduced row-echelon matrix of A,
2. Basis of Null(A) (Rn) are formed by :
Finding Free variables (parameters) xk : without leading 1's columns in R
2021/11/17Lec16_LA21_ Singling Lee 26
Find the basis of Null(A) (II)
Find the basis of Null(A) (βRn) :
Finding Free variables (parameters) xk : without leading 1's columns in R
Basis of Null (A) in Rn :
(a) having 1 at the position k for the corresponding vector,
(b) reverse the sign in column k of R at the appropriate positions in the vector,
(c) zeros at the positions for the other free variables!
2021/11/17Lec16_LA21_ Singling Lee 27
Basis of Null(A) (II)
The Solution of Ax= 0 with 4 parameters
Rank =2 , dim(Null(A)) = 4; Nullity = 4
2021/11/17Lec16_LA21_ Singling Lee 28
π₯3= r οΌ π₯4 = π οΌ π₯5 = π‘οΌ π₯6 = π’
A=
β1 2 0 4 5 β332
β7β5
22
04
16
41
4 β9 2 β4 β4 7
->
1 0 β4 β28 β37 1300
10
β20
β120
β160
50
0 0 0 0 0 0
π₯1= 4π + 28π + 37π‘ β 13π’π₯2= 2π + 12π + 16π‘ β 5π’
π₯1
π₯2
π₯3
π₯4
π₯5
π₯6
= r
421000
+ s
28120100
+ t
37160010
+ u
β13β50001
reduced row echelon matrix
β Linearly Ind.
Dimension Theorem for MatrixTheorem 4.9.2: If A is a mΓn matrix then Rank(A) + Nullity(A) = n
From Reduced Row-Echelon form.
Rank(A) + = n
Rank(A) + Nullity(A) = n
Similarly : Rank(AT) + Nullity(AT) = m
Rank(A) + Nullity(AT) = m
β If Rank(A) = r then , Nullity(A) = n-r ; Nullity(AT) = m - r
number of free
variables
2021/11/17Lec16_LA21_ Singling Lee 29
Nullity and Number of Parameters
Theorem 4.9.3: If A is an mΓn matrix then
(a) Rank(A) = number of leading one variables in rref of matrix A;
(b) Nullity(A) = number of parameters (free variables)
in general solution of Ax=0
2021/11/17Lec16_LA21_ Singling Lee 30
Rank, Nullity, and Linear Systems
Example 4 pp. 279
a) Find the number of parameters in general solution of Ax = 0 if A is a 5 x 7 matrix of rank 3!
b) Find the rank of a 5 x 7 matrix for which Ax=0 has a two-dimensional solution space.
Sol:
a) Nullity(A) = n- rank(A) = 7-3 = 4
b) Nullity(A) = 2. rank(A) = 7-2 = 5
2021/11/17Lec16_LA21_ Singling Lee 31
Equivalent Statements
Theorem 4.9.4 : If π΄π± = π is a consistent linear system of π equations in n unknown, and if π΄ has rank π, then the general solution of the system contains π β π parameters.
2021/11/17Lec16_LA21_ Singling Lee 32
Fundamental Spaces of a Matrix
Theorem 4.9.5: If A is any matrix, rank(A)=rank(AT)
If A : m Γ n matrix
rank(A) + nullity(A) = n
rank(A) + nullity(AT) = m
dim[row(A)] = dim[col(A)]= r = rank(A)
dim[null(A)]=n-r dim[null(AT)]=m-r
2021/11/17Lec16_LA21_ Singling Lee 33
The Consistence Theorem
If Ax = b and A is mΓn , then the following are equivalent.
a) Ax = b is consistent.
b) b β Col (A)
c) Rank(A) = Rank( [A | b] )
2021/11/17Lec16_LA21_ Singling Lee 34
Condition for Ax = b has β¦1 Solution
A is mΓn, the following are equivalent.
1) Ax = 0 has only the trivial solution.
2) The column vectors of A are linearly independent
3) Ax = b has at most one solution (none or one)for every mΓ1 vector b
2021/11/17Lec16_LA21_ Singling Lee 35
Orthogonal Complement
Definition 2: W is a subspace of Rn, the set of all vectors in Rn that are orthogonal to every vector in W is called the orthogonal complement(ζ£δΊ€θ£
ι€) of W, denote Wβ₯
2021/11/17Lec16_LA21_ Singling Lee 36
Properties of Orthogonal Complements (I)
Theorem 4.9.6: W: subspace of Rn
a) Wβ₯ is a subspace of Rn
b) The only vector common to W and Wβ₯ is 0
c) The orthogonal complement of Wβ₯ is W
2021/11/17Lec16_LA21_ Singling Lee 37
Examples for Orthogonal Complements
1. In R2 , what is the orthogonal complement Wβ₯ of a line W through the origin?
2. In R3 , what is the orthogonal complement Wβ₯ of a plane W through the origin ?
3. In R3 , what is the orthogonal complement of zerovector ?
4. In R3 , what is the orthogonal complement Wβ₯ of subspace W spanned by vectors (1,5,1) and (2,2,2)?
2021/11/17Lec16_LA21_ Singling Lee 38
Orthogonal Comp. for row(A) and null(A)
Theorem 4.9.7: A: m Γ n matrix
a) Null(A) and Row(A) are orthogonal comp. in Rn
from Ax=0 : each element in row space is orthogonal to any solution x;
b) Null(AT) and Col(A) are orthogonal comp. in Rm
from ATy=0
2021/11/17Lec16_LA21_ Singling Lee 39
Fundamental Theorem of Linear Algebra
2021/11/17Lec16_LA21_ Singling Lee 40
Solving Ax = b for an π Γ π matrix A with rank(A) = r
Equivalent StatementsTheorem 4.9.8: If A is an nΓn matrix, no duplicate
rows and columns, then the following are equivalent.
a) A is invertible.
b) Ax=0 has only the trivial solution.
c) The reduced row-echelon form of A is In.
d) A is expressible as a product of elementary matrices.
e) Ax=b is consistent for every nΓ1 matrix b.
f) Ax=b has exactly one solution for every nΓ1 matrix b.
2021/11/17Lec16_LA21_ Singling Lee 41
Equivalent Statements
g) det(A) β 0.
h) The column vectors of A are linearly ind.
i) The row vectors of A are linearly ind.
j) The column vectors of A span Rn. (col(A) = Rn )
k) The row vectors of A span Rn. (row(A) = Rn )
l) The column vectors of A form a basis for Rn.
m) The row vectors of A form a basis for Rn.
2021/11/17Lec16_LA21_ Singling Lee 42
Equivalent Statements [cont.]
n) A has rank n.
o) A has nullity 0
p) The orthogonal complement of the null space of A is Rn.
q) The orthogonal complement of the row space of A is {0}.
2021/11/17Lec16_LA21_ Singling Lee 43