linear algebra. session 11 - texas a&m universityroquesol/math_304_fall_2018... ·...
TRANSCRIPT
Abstract Linear Algebra II
Linear Algebra. Session 11
Dr. Marco A Roque Sol
11 / 06 / 2018
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider
the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system
of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:
x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now,
assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that
a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0)
does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact
but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem
is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate,
namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely,
there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors
inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides
(rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find
a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation
of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Least Squares Problems
Let’s consider the Overdetermined system of linear equations:x + 2y = 3
3x + 2y = 5x + y = 2.09
⇒
x + 2y = 3−4y = −4−y = −0.09
Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).
Problem
Find a good approximation of (x0, y0)
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach
is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit.
Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely,
we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for
a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y)
that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize
the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:
a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any
x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R
define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual
r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum
(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2
Least squares solution
System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...am1x1 + am2x2 + · · ·+ amnxn = bm
For any x ∈ R define a residual r(x) = b− Ax
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution
x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x
to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system
is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one
thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)||
(or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently,
||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A
be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an
m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and
let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let
b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector
x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂
is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution
of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system
Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax
if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and only
if it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution
of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated
normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system
ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).
||r(x)||2 = (m∑i=1
(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2
Let A be an m × n matrix and let b ∈ Rn
Theorem
A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax
is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector
in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A),
the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space
of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hence
the length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length
of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax
is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal
if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax
is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the
orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection
of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b
onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A)
that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is,
if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x)
is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal
to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).
We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know
that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ =
Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace
for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix.
Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A)
the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace
of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix
ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA.
Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus,
x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is
a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution
if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒
AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒
ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
proof
Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if
AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system
ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb
is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find
the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution to
x + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation,
the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system
can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Corollary
The normal system ATAx = ATb is always consistent.
Example 11.15
Find the least squares solution tox + 2y = 3
3x + 2y = 5x + y = 2.09
SolutionIn matrix notation, the system can be written as
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and
the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒
(11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)
⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 23 21 1
(xy
)=
35
2.09
and the normal system is
(1 3 12 2 1
) 1 23 21 1
(xy
)=
(1 3 12 2 1
) 35
2.09
⇒(
11 99 9
)(xy
)=
(20.0918.09
)⇒{
x = 1y = 1.01
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16
Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find
the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function
that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is
the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit
to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.16Find the constant function that is the least squares fit to thefollowing data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c ⇒
c = 1c = 0c = 1c = 2
⇒
1012
c ⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then,
the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
)
1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =
(1 1 1 1
) 1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 14(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1
(mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus,
the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the normal system is
(1 1 1 1
) 1111
c =(
1 1 1 1)
1012
c = 1
4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)
Thus, the constant function is
f (x) = 1
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find
the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function
that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is
the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit
tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.17
Find the linear polynomial function that is the least squares fit tothe following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1+c2x ⇒
c1 = 1
c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2
⇒
1 01 11 21 3
(c1c2
)=
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then,
the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
)
1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
)
1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
)
(c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)
⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus,
the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Then, the nomal system is
(1 1 1 10 1 2 3
) 1 01 11 21 3
(c1c2
)=
(1 1 1 10 1 2 3
) 1012
(
4 66 14
) (c1c2
)=
(48
)⇒{
c1 = 0.4c2 = 0.4
Thus, the linear function is
f (x) = 0.4 + 0.4x
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find
the quadratic polynomial function that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function
that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is
the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is the least squares fit
to the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
Example 11.18
Find the quadratic polynomial function that is the least squares fitto the following data
x 0 1 2 3
f(x) 1 0 1 2
Solution
f (x) = c1 + c2x + c3x2 ⇒
c1 = 1
c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2
⇒
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒
Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then,
the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1012
⇒Then, the nomal system is
1 1 1 10 1 2 30 1 4 9
1 0 01 1 11 2 41 3 9
c1
c2c3
=
1 1 1 10 1 2 30 1 4 9
1012
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
4 6 146 14 36
14 36 98
c1c2c3
=
48
22
⇒
c1 = 0.9c2 = −1.1c3 = 0.5
Thus, the quadratic function is
f (x) = 0.9− 1.1x + 0.5x2
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
4 6 146 14 36
14 36 98
c1c2c3
=
48
22
⇒
c1 = 0.9c2 = −1.1c3 = 0.5
Thus, the quadratic function is
f (x) = 0.9− 1.1x + 0.5x2
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
4 6 146 14 36
14 36 98
c1c2c3
=
48
22
⇒
c1 = 0.9c2 = −1.1c3 = 0.5
Thus, the quadratic function is
f (x) = 0.9− 1.1x + 0.5x2
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
4 6 146 14 36
14 36 98
c1c2c3
=
48
22
⇒
c1 = 0.9c2 = −1.1c3 = 0.5
Thus, the quadratic function is
f (x) = 0.9− 1.1x + 0.5x2
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
4 6 146 14 36
14 36 98
c1c2c3
=
48
22
⇒
c1 = 0.9c2 = −1.1c3 = 0.5
Thus, the quadratic function is
f (x) = 0.9− 1.1x + 0.5x2
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
4 6 146 14 36
14 36 98
c1c2c3
=
48
22
⇒
c1 = 0.9c2 = −1.1c3 = 0.5
Thus,
the quadratic function is
f (x) = 0.9− 1.1x + 0.5x2
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
4 6 146 14 36
14 36 98
c1c2c3
=
48
22
⇒
c1 = 0.9c2 = −1.1c3 = 0.5
Thus, the quadratic function is
f (x) = 0.9− 1.1x + 0.5x2
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Least Squares Problems.
4 6 146 14 36
14 36 98
c1c2c3
=
48
22
⇒
c1 = 0.9c2 = −1.1c3 = 0.5
Thus, the quadratic function is
f (x) = 0.9− 1.1x + 0.5x2
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let
< ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote
the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product
in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors
v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn
form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set
ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal
to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other:
< vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0
for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If,
in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition,
all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are
of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length,
vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk
iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled
an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance,
The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basis
e1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1).
Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthogonal sets
Let < ·, · > denote the scalar product in Rn
Definition
Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .
If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.
For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose
v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn
is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis
for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn
(i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and
an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let
x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and
y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvn
where xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere
xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=
∑ni=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =
√∑ni=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Orthonormal bases
Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).
Theorem
Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R
i) < x, y >=∑n
i=i xiyi
i) ||x|| =√∑n
i=i xiyi
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =
n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows
from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i)
when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
proof
i)
< x, y >=
⟨n∑i=i
xivi ,n∑j=i
yjvj
⟩=
n∑i=i
xi
⟨vi ,
n∑j=i
vj
⟩=
n∑i=i
xi
n∑j=i
yj 〈vi , vj〉 =n∑i=i
xiyi
ii) follows from i) when y = x �
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V
is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace
of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn.
Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be
the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projection
of a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn
onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V
is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace
spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, then
p = <x,v><v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp =
<x,v><v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V
admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an
orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis
v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed,
< p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=
∑kj=i
<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >=
<x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒
< x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒
(x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒
(x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.
If V is a one-dimensional subspace spanned by a v, thenp = <x,v>
<v,v>v
If V admits an orthogonal basis v1, v2, · · · , vk , then
p =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vk >
< vk , vk >vk
Indeed, < p, vi >=∑k
j=i<x,vj><vj ,vj>
< vj , vi >= <x,vi><vi ,vi>
< vi , vi >=
< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If
v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn
is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis
for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn,
then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector
x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If
v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn
is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis
for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn,
then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector
x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11
Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets
Orthogonal Sets.
Coordinates relative to an orthogonal basis
Theorem
If v1, v2, · · · , vn is an orthogonal basis for Rn, then
x =< x, v1 >
< v1, v1 >v1 +
< x, v2 >
< v2, v2 >v2 + ... +
< x, vn >
< vn, vn >vn
for any vector x ∈ Rn
Corollary
If v1, v2, · · · , vn is an orthonormal basis for Rn, then
z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn
for any vector x ∈ Rn.
Dr. Marco A Roque Sol Linear Algebra. Session 11