lecture 18 linear algebra, ii - brown universitycs.brown.edu/courses/cs004/lectures/lec18.pdf ·...
TRANSCRIPT
![Page 1: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/1.jpg)
Lecture 18Linear Algebra, II
1
![Page 2: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/2.jpg)
Lecture 18 Goals
• Matrix inversion, singularity, rank, and determinants
• Solving systems of linear equations
2
![Page 3: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/3.jpg)
Problem of inversion
3
Solve for unknown vector x:
Ax = y
where A is mxn, x is nx1, and y is mx1.
![Page 4: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/4.jpg)
Problem of inversion
4
Solve for unknown vector x:
Ax = y
where A is mxn, x is nx1, and y is mx1.
That’s basic algebra!
x = y/A
![Page 5: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/5.jpg)
Problem of inversion
5
Solve for unknown vector x:
Ax = y
where A is mxn, x is nx1, and y is mx1.
Or is it?
x = y/A
Recall that matrix multiplication is not commutative, so we cannot divide B=C/A since AB/A is not equal to B.
What is Matrix “division” anyway?
![Page 6: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/6.jpg)
Problem of inversion
6
Solve for unknown vector x:
Ax = y
where A is mxn, x is nx1, and y is mx1.
Correct solution (use the Matrix inverse):
A\Ax = A\y => x = A\y
Order matters, i.e.
AB = C => A\AB=A\C => B = A\C
![Page 7: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/7.jpg)
Problem of inversion
7
Solve for unknown vector x:
Ax = y
where A is mxn, x is nx1, and b is mx1.
Correct solution (use the Matrix inverse):
A\Ax = A\y => x = A\y
For matrices, the definition of the “inverse”, or “one over” the matrix, has to be defined properly.
Key question: When does the inverse exist?
![Page 8: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/8.jpg)
Answer: The Determinant
If the determinant is non-zero, the matrix can be inverted and unique solution exists for Ax=y.
If the determinant is zero, the matrix cannot be inverted, there can be either 0 or an infinite number of solutions to Ax=y.
8
![Page 9: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/9.jpg)
Wait. What’s the Determinant?Geometric interpretation: The determinant describes how the area (2D), volume (3D), or hypervolume (4D or higher) defined by set of points X changes when matrix A is applied to X.
9Interactive Demonstration: http://demonstrations.wolfram.com/DeterminantsSeenGeometrically/
![Page 10: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/10.jpg)
Determinants (square matrices)
10
Consider a 2 x 2 matrix
The determinant of a 2 x 2 matrix is
![Page 11: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/11.jpg)
Determinants (square matrices)
11
Finding the determinant of a general n x n square matrix requires evaluation of a complicated polynomial of the coefficients of the matrix, but there is a simple recursive approach.
![Page 12: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/12.jpg)
Solving Ax=y
•When A is non-singular (has non-zero determinant) A inverse exists, and one can find x via
x = inv(A)*y
•However, depending on A, this is can be computationally inefficient and or less precise then using x = A\y
•The MATLAB \ operation (called mldivide) takes the form of A into account while trying to solve A\y
•doc mldivide
12
![Page 13: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/13.jpg)
Operation of A\y in MATLAB
13
![Page 14: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/14.jpg)
14
![Page 15: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/15.jpg)
15
![Page 16: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/16.jpg)
Linear Equations
•y = mx+b is a linear function, y(x)•Setting mx + b = c is a linear equation•Systems of equations
Have multiple Equations
Solving a system of equations involves finding a set (or sets) of values that allow all the equations to hold.
These are not always solvable!
16
![Page 17: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/17.jpg)
Linear equations - Example
17
Linear equations
can be written in matrix form
Or more symbolically as
where
![Page 18: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/18.jpg)
Linear equations – General Form
18
m linear equations with n variables:
Can be written in matrix as where
![Page 19: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/19.jpg)
Linear equationsFor the equation Ax=y, there are 3 distinct cases
19
=
==
Square, equal number of unknowns and equations
Underdetermined: more unknowns than equations
Overdetermined: fewer unknowns than equations
![Page 20: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/20.jpg)
Types of solutions with “random” data“Generally” the following observations would hold
20
=
==
One solution (eg., 2 lines intersect at one point)
Infinite solutions (eg., 2 planes intersect at many points)
No solutions (eg., 3 lines don’t intersect at a point)
![Page 21: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/21.jpg)
Linear equationsBut other things can happen. For example:
21
=
==
No solution (2 parallel lines) Many solutions (2 parallel lines)
No solutions (2 parallel planes)Solutions (3 lines that do intersect at a point)
![Page 22: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/22.jpg)
What is a linear function?
22
Let be a function. It is said to be linear if
Superposition can be applied to linear systems!
![Page 23: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/23.jpg)
Matrix representation of a linear function
23
![Page 24: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/24.jpg)
What is a linear function?
24
Let be a function. It is said to be linear if
This is sometimes called superposition
When f is represented with matrix A, it is clear it satisfies the properties above, i.e.,
A(x+y) = Ax+Ay
A(cx) = cAx
![Page 25: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/25.jpg)
Solving Ax=b
Two key questions1) How can we tell when no solutions exist?2) When solutions exist how can we find and represent them?
25
![Page 26: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/26.jpg)
Range of a matrix
26
![Page 27: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/27.jpg)
Ax=b has a solution when b is in R(A)
27
![Page 28: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/28.jpg)
Linear independence
28
![Page 29: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/29.jpg)
Rank of a matrix
29
The rank of a matrix is equal to the maximal number of linearly independent vectors in the range of the matrix.
A square matrix A of dimension n x n is said to be full rank if the rank of the matrix is n, i.e. rank(A) = n
If a matrix is full rank, it can be inverted.
The rank of a matrix can be computed using the command rank
![Page 30: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/30.jpg)
Key Question #1: Existence
How can we tell when no solution exists to Ax=b?•When rank([A b]) > rank(A) there isn’t a linear combination of the columns of A that can be used to represent y, i.e. Ax = b has no solution.
30
![Page 31: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/31.jpg)
Key Question #2: Uniqueness?
How can we tell the solution to Ax=b is unique?When multiple solutions exists how can we find and represent them?
31
![Page 32: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/32.jpg)
Nullspace of a matrix
32
![Page 33: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/33.jpg)
Nullspace != {0} => Infinite Solutions
33
![Page 34: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/34.jpg)
Existence and Uniqueness
•Ax=b•Existence: Having a solution means
b in R(A)•Having a unique solution means Az=0 iff z is an appropriately sized zero vector
•Otherwise x+z != x, and A(x+z)=Ax+Az = Ax = b•Uniqueness: Having a unique solution requires N(A)={0}
34
![Page 35: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/35.jpg)
When do we need to compute N(A)?
For dim(A) m by n, dim(Null(A)) is m by n-rank(A).
So when A is full rank (i.e. rank(A) = n), N(A) = {0}
35
![Page 36: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/36.jpg)
Finding the nullspace in MATLAB
36
null(A) returns an orthonormal basis for the null space of Anull(A,’r’) returns a “rational” basis for the null space of A
For illustrative purposes, use the script initializeMatrices.m, which puts 4 matrices of size 4 x 4 into the workspace:
A is of rank 4B is of rank 3C is of rank 2D is or rank 1
![Page 37: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/37.jpg)
Finding the null space in MATLAB
37
Since A is of rank 4, the null space of A is reduced to zero. The null space of the matrix B (of rank 3) is of dimension 1
The command null(A,’r’) will take the matrix A and return the set of vectors which belong to the null space of A. The ‘r’ is added so Matlab returns fractional vectors when possible.
Null space of A is empty, because the matrix is invertible
Null space of B is of dimension 1
![Page 38: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/38.jpg)
Sanity checks
38
![Page 39: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/39.jpg)
Sanity checks
39
![Page 40: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/40.jpg)
Sanity checks
40
![Page 41: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/41.jpg)
Procedure to solve a linear system
41
rank(A)=rank([A,y])
number of unknowns is equal to
rank(A)
yes
yes
no
no
Infinite number of sols
Use rref and null to compute solutions
Single solutionUse A\y
to compute solution
There is no solution
![Page 42: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/42.jpg)
Procedure to solve a linear system
42
rank(A)=rank([A,y])
number of unknowns is equal to
rank(A)
yes
yes
no
no
Infinite number of sols
Use rref and null to compute solutions
Single solutionUse A\y
to compute solution
There is no solution
![Page 43: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/43.jpg)
Step: check if rank(A) = rank([A,y]) → no
43
If rank(A) is not equal to rank([A,y]), y is not in the range of A
i.e. There is no x such that Ax = y
System has no solution
![Page 44: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/44.jpg)
Procedure to solve a linear system
44
rank(A)=rank([A,y])
number of unknowns is equal to
rank(A)
yes
yes
no
no
Infinite number of sols
Use rref and null to compute solutions
Single solutionUse A\y
to compute solution
There is no solution
![Page 45: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/45.jpg)
Procedure to solve a linear system
45
rank(A)=rank([A,y])
number of unknowns is equal to
rank(A)
yes
yes
no
no
Infinite number of sols
Use rref and null to compute solutions
Single solutionUse A\y
to compute solution
There is no solution
![Page 46: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/46.jpg)
Step: rank(A) = number of unknowns? → yes
46
If the rank of A is the same as the number of unknowns, the system can be inverted, and the system has a unique solution, which can be computed by A\y
![Page 47: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/47.jpg)
Procedure to solve a linear system
47
rank(A)=rank([A,y])
number of unknowns is equal to
rank(A)
yes
yes
no
no
Infinite number of sols
Use rref and null to compute solutions
Single solutionUse A\y
to compute solution
There is no solution
![Page 48: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/48.jpg)
Procedure to solve a linear system
48
rank(A)=rank([A,y])
number of unknowns is equal to
rank(A)
yes
yes
no
no
Infinite number of sols
Use rref and null to compute solutions
Single solutionUse A\y
to compute solution
There is no solution
![Page 49: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/49.jpg)
Step: rank(A) = number of unknowns? → no
49
In this case rank(A) < number of unknowns (i.e. it is smaller). An infinite number of solutions exist.
Imagine you can find a particular solution to this problem
Then if you take any vector in the null space, it is also a solution to this problem:
Because
![Page 50: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/50.jpg)
Step: rank(A) = number of unknowns? → no
50
The rank of A is not equal to the number of unknowns (i.e. it is smaller).
An infinite number of solutions exist. Now you might have to construct this infinite amount of solutions:
![Page 51: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/51.jpg)
Step: rank(A) = number of unknowns? → no
51
The rank of A is not equal to the number of unknowns (i.e. it is smaller).
An infinite number of solutions exist. Now you might have to construct this infinite amount of solutions:
![Page 52: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/52.jpg)
Step: rank(A) = number of unknowns? → no
52
The rank of A is not equal to the number of unknowns (i.e. it is smaller).
An infinite number of solutions exist. Now you might have to construct this infinite amount of solutions:
![Page 53: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/53.jpg)
Step: rank(A) = number of unknowns? → no
53
The rank of A is not equal to the number of unknowns (i.e. it is smaller).
An infinite number of solutions exist. Now you might have to construct this infinite amount of solutions:
How far can you go?
You can go as far as the null space permits, i.e. pick v’s from columns of null spaces. Their linear combinations span the null space.
![Page 54: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/54.jpg)
Step: rank(A) = number of unknowns? → no
54
How to do this with matlab?
Use the command rref
y is in the range of A. There is an infinity of solutions
How to compute them
For this, you need to use
B = rref([A,y])
![Page 55: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/55.jpg)
Step: rank(A) = number of unknowns? → no
55
Find a solution of this system (Gauss elimination has already been done for you)
Gauss form of A Particular solution
![Page 56: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/56.jpg)
Step: rank(A) = num. of unk.? → no
56
Find a solution of this system (Gauss elimination has already been done for you)
Obvious solution: [-1;1;0]
![Page 57: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/57.jpg)
Step: rank(A) = number of unknowns? → no
57
How do I find all solutions?
Add all the vectors from the null space
![Page 58: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/58.jpg)
Step: rank(A) = number of unknowns? → no
58
How do I find all solutions?
Add all the vectors from the null space
![Page 59: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/59.jpg)
Gauss Elimination (non-singular A)
Want to solve Ax = b• Forward elimination
• Starting with the first row, add or subtract multiples of that row to eliminate the first coefficient from the second row and beyond.
• Continue this process with the second row to remove the second coefficient from the third row and beyond.
• Stop when an upper triangular matrix remains.
• Back substitution• Starting with the last row, solve for the
unknown, then substitute that value into the next highest row.
• Because of the upper-triangular nature of the matrix, each row will contain only one more unknown.
1/22/18 59
![Page 60: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/60.jpg)
Order of Elimination
![Page 61: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/61.jpg)
Gaussian Elimination in 3D
•Using the first equation to eliminate x from the next two equations
![Page 62: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/62.jpg)
Gaussian Elimination in 3D
•Using the second equation to eliminate y from the third equation
![Page 63: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/63.jpg)
Gaussian Elimination in 3D
•Using the second equation to eliminate y from the third equation
![Page 64: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/64.jpg)
Solving Triangular Systems
•We now have a triangular system which is easily solved using a technique called Backward-Substitution.
![Page 65: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/65.jpg)
Solving Triangular Systems
• If A is upper triangular, we can solve Ax = b by:
![Page 66: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/66.jpg)
Backward Substitution
•From the previous work, we have
•And substitute z in the first two equations
![Page 67: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/67.jpg)
Backward Substitution
•We can solve y
![Page 68: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/68.jpg)
Backward Substitution
•Substitute to the first equation
![Page 69: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/69.jpg)
Gauss-Jordan EliminationKeep going until augmented matrix is reduced row echelon form (rref):
The rows (if any) consisting entirely of zeros are grouped together at the bottom of the matrix.In each row that does not consist entirely of zeros, the leftmost nonzero element is a 1 (called a leading 1 or a pivot).Each column that contains a leading 1 has zeros in all other entries.The leading 1 in any row is to the left of any leading 1’s in the rows below it.
Stop process in step 2 if you obtain a row whose elements are all zeros except the last one on the right. In that case, the system is inconsistent and has no solutions. Otherwise, finish step 2 and read the solutions of the system from the final matrix
1/22/18 69
![Page 70: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/70.jpg)
Overdetermined systems
70
![Page 71: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/71.jpg)
Overdetermined systems
71
![Page 72: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/72.jpg)
The “Least Squares” Problem
If A is an n-by-m array, and b is an n-by-1 vector, let c* be the smallest possible (over all choices of m-by-1 vectors x) mismatch between Ax and b (ie., pick x to make Ax as much like b as possible).
72
“is defined as”“the minimum, over all m-by-1 vectors x”
“the length (ie., norm) of the difference/mismatch between Ax and b.”
![Page 73: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/73.jpg)
Four cases for Least Squares
Recall least squares formulation
There are 4 scenariosc* = 0: the equation Ax=b has at least one solution
• only one x vector achieves this minimum• many different x vectors achieves the minimum
c* > 0: the equation Ax=b has no solutions• only one x vector achieves this minimum• many different x vectors achieves the minimum
73
![Page 74: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/74.jpg)
Four cases: x=A\b as solution ofNo Mismatch:
c* = 0, and only one x vector achieves this minimumChoose this x
c* = 0, and many different x vectors achieves the minimumFrom all these minimizers, choose smallest x (ie., norm)
Mismatch:c* > 0, and only one x vector achieves this minimum
Choose this x
c* > 0, and many different x vectors achieves the minimumFrom all minimizers, choose an x with the smallest norm
74
![Page 75: Lecture 18 Linear Algebra, II - Brown Universitycs.brown.edu/courses/cs004/lectures/lec18.pdf · Linear Equations •y = mx+b is a linear function, y(x) •Setting mx + b = c is a](https://reader036.vdocuments.us/reader036/viewer/2022071006/5fc398886c1be50b413ad2ad/html5/thumbnails/75.jpg)
The backslash operatorIf A is an n-by-m array, and b is an n-by-1 vector, then
>> x = A\bsolves the “least squares” problem. Namely
• If there is an x which solves Ax=b, then this x is computed
• If there is no x which solves Ax=b, then an x which minimizes the mismatch between Ax and b is computed.
In the case where many x satisfy one of the criterion above, then a smallest (in terms of vector norm) such x is computed.
So, mismatch is handled first. Among all equally suitable x vectors that minimize the mismatch, choose a smallest one.
75