lecture 10 dimensions, independence, basis and complete solution of linear systems
DESCRIPTION
Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems. Shang-Hua Teng. Linear Independence. Linear Combination. Linear Independence. is linearly independent if only if none of them can be expressed as a linear combination of the others. Examples. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/1.jpg)
Lecture 10Dimensions, Independence, Basis and Complete Solution of Linear Systems
Shang-Hua Teng
![Page 2: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/2.jpg)
Linear Independence
Linear Combination
in
v
,...,v,vv
n
1i i
21
is vectorsofset a of
Linear Independence
vectorsofset A 21 n,...,v,vv
is linearly independent if only if none of them can be expressed as a linear combination of the others
![Page 3: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/3.jpg)
Examples
1
1
0
,
4
2
2
,
1
0
1
1
1
2
2
1
1
0
1,,
![Page 4: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/4.jpg)
Linear Independence and Null Space
Theorem/Definition
vectorsofset A 21 n,...,v,vvis linearly independent if and only1v1+2v2+…+nvn=0 only happens when all ’s are zero
The columns of a matrix A are linearly independent when only solution to Ax=0 is x = 0
![Page 5: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/5.jpg)
2D and 3Dv
w
u
v
![Page 6: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/6.jpg)
How do we determine a set of vectors are independent?
Make them the columns of a matrix
Elimination
Computing their null space
![Page 7: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/7.jpg)
Permute Rows and Continuing Elimination (permute columns)
011121
131111
021111
110011
A
![Page 8: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/8.jpg)
Theorem
If Ax = 0 has more have more unknown than equations (m > n: more columns than rows), then it has nonzero solutions.
There must be free variables.
![Page 9: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/9.jpg)
Echelon Matrices
*
*
*
*
000000
**0000
*****0
******
A
Free variables
![Page 10: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/10.jpg)
Reduced Row Echelon Matrix R
1
0
0
0
000000
*10000
*0**10
*0**01
A
Free variables
![Page 11: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/11.jpg)
Computing the Reduced Row Echelon Matrix
• Elimination to Echelon Matrix
E1PA = U
• Divide the row of pivots by the pivots
• Upward Elimination
E2E1PA = R
![Page 12: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/12.jpg)
Example: Gauss-Jordan Method for Matrix Inverse
• [A I]
• E1[A I] = [U, E1]
• In its reduced Echelon Matrix
• A-1 [A I] = [I A-1]
![Page 13: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/13.jpg)
A Close Look at Reduced Echelon Matrix
• The last equation of R x = 0 is redundant
0 = 0
• Rank of A is the number of pivots rank(A).
00000
34100
12031
R
![Page 14: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/14.jpg)
What is the Rank of Outer Product
n
n
vv
u
u
1
1
![Page 15: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/15.jpg)
Rank and Reduced Row Echelon Matrix
0000000
1
0
0
0
000000
*10000
*0**10
*0**01
A
Free variables• Theorem/Definition
• Rank(A) = number of independent rows
• Rank(A) = number of independent columns
![Page 16: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/16.jpg)
Dimension of the Column Space and Null Space
• The dimensions of the column space of A is equal to Rank(A).
• The dimension of the null space of A is equal to the number of free variables which is n – Rank(A)
• A is an m by n matrix
![Page 17: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/17.jpg)
Rank and Reduced Row Echelon Matrix
0000000
1
0
0
0
000000
*10000
*0**10
*0**01
A
Free variablesFree Columns
Pivot columns
The Pivot columns are not combinations of earlier columns
![Page 18: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/18.jpg)
Reduced Echelon and Null Space Matrix
• Nullspace Matrix• Special Solutions
0
0
0
00000
34100
12031
5
4
3
2
1
x
x
x
x
x
Rx
100
010
340
001
123
N
100
010
001
100
010
340
001
123
![Page 19: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/19.jpg)
Null Space Matrix
• Ax=0 has n-Rank(A) free variables and special solutions
• The Nullspace matrix has n-Rank(A) columns• The columns of the nullspace matrix are
independent• The dimension of the Null space is n – rank(A)
![Page 20: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/20.jpg)
Complete Solution of Ax = 0• After column permutation, we can write
rows zeros
rowspivot
00 m-r
rFIR
r pivot columns n-r free columns
• Nullspace matrix
m-r
r
I
FN
Pivot variables
Free variables
• Moreover: RN = [0]
![Page 21: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/21.jpg)
Complete Solution to Ax = b• A is an m by n matrix, and b is an n-place vector
– Unique solution– Infinitely many solution– No solution
Suppose Ax = b has more then one solution, say x1, x2 then A x1 = bA x2 = b
So A (x1 - x2 ) = 0
(x1 - x2 ) is in nullspace(A)
![Page 22: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/22.jpg)
Complete Solution to Ax = b
Suppose we found a particular solution xp to Ax = b i.e, A xp = b
Let F be the indexes of free variables of Ax = 0 Let xF be the column vector of free variablesLet N be the nullspace matrix of A
Then
defines the complete set of solutions to Ax = b
FNx xp
![Page 23: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/23.jpg)
Example: Complete Solution to Ax = b
7
6
1
6131
4100
2031
4
3
2
1
x
x
x
x
![Page 24: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/24.jpg)
7
6
1
6131
4100
2031
Augmented matrix [A b]
Elimination to obtain [R d]
0
6
1
0000
4100
2031
![Page 25: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/25.jpg)
Set free variables to 0 to find a particular solution(1,0,6,0)T
Compute the nullspace matrix
10
40
01
23
Complete solution is
1
4
0
2
0
0
1
3
0
6
0
1
10
40
01
23
0
6
0
1
424
2 xxx
xx
![Page 26: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/26.jpg)
Full Rank Matrix
• Suppose A is an m by n matrix. Then
• A is full column if rank(A) = n– columns of A are independent
• A is full row rank if rank(A) = m– Rows of A are independent
),min()(rank nmA
![Page 27: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/27.jpg)
Full Column Rank Matrix
• Columns are independent
• All columns of A are pivot columns
• There are non free variables or special solutions
• The nullspace N(A) contains only the zero vector
• If Ax=b has a solution (it might not) then it has only one solution
0
IR
n by n
m-n rows of zeros
![Page 28: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/28.jpg)
Full Row Rank Matrix
• Rows are independent
• All rows of A have pivots, R has no zero rows
• Ax=b has a solution for every right hand side b
• The column space is the whole space Rm
• There are n-m special solutions in the null space of A
FIR
![Page 29: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/29.jpg)
The Whole Picture• Rank(A) = m = n Ax=b has unique solution IR
FIR
0
IR
00
FIR
• Rank(A) = m < n Ax=b has n-m dimensional solution
• Rank(A) = n < m Ax=b has 0 or 1 solution
• Rank(A) < n, Rank(A) < m Ax=b has 0 or n-rank(A) dimensions
![Page 30: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/30.jpg)
Basis and Dimension of a Vector Space
• A basis for a vector space is a sequence of vectors that – The vectors are linearly independent– The vectors span the space: every vector in the
vector can be expressed as a linear combination of these vectors
![Page 31: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/31.jpg)
Basis for 2D and n-D
• (1,0), (0,1)
• (1 1), (-1 –2)
• The vectors v1,v2,…vn are basis for Rn if and only if they are columns of an n by n invertible matrix
![Page 32: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/32.jpg)
Column and Row Subspace
• C(A): the space spanned by columns of A– Subspace in m dimensions
– The pivot columns of A are a basis for its column space
• Row space: the space spanned by rows of A– Subspace in n dimensions
– The row space of A is the same as the column space of AT, C(AT)
– The pivot rows of A are a basis for its row space
– The pivot rows of its Echolon matrix R are a basis for its row space
![Page 33: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/33.jpg)
Important Property I: Uniqueness of Combination
• The vectors v1,v2,…vn are basis for a vector space V, then for every vector v in V, there is a unique way to write v as a combination of v1,v2,…vn .
v = a1 v1+ a2 v2+…+ an vn
v = b1 v1+ b2 v2+…+ bn vn
• So: 0=(a1 - b1) v1 + (a2 -b2 )v2+…+ (an -bn )vn
![Page 34: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/34.jpg)
Important Property II: Dimension and Size of Basis
• If a vector space V has two set of bases– v1,v2,…vm . V = [v1,v2,…vm ]– w1,w2,…wn . W= [w1,w2,…wn ].
• then m = n– Proof: assume n > m, write W = VA– A is m by n, so Ax = 0 has a non-zero solution– So VAx = 0 and Wx = 0
• The dimension of a vector space is the number of vectors in every basis– Dimension of a vector space is well defined
![Page 35: Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems](https://reader035.vdocuments.us/reader035/viewer/2022062423/5681481a550346895db5459a/html5/thumbnails/35.jpg)
Dimensions of the Four SubspacesFundamental Theorem of Linear
Algebra, Part I• Row space: C(AT) – dimension = rank(A)
• Column space: C(A)– dimension = rank(A)
• Nullspace: N(A) – dimension = n-rank(A)
• Left Nullspace: N(AT) – dimension = m –rank(A)
•