lecture 24: quick review from previous lecture · lecture 24: quick review from previous lecture...
TRANSCRIPT
Lecture 24: Quick review from previous lecture
• A square matrix A is called an orthogonal matrix if it satisfies
ATA = AAT= I.
• If A is an orthogonal matrix, then its columns (rows) form an orthonormal
basis on Rnwith respect to the Euclidean dot product.
—————————————————————————————————
Today we will discuss the QR factorization and orthogonal projections.
- Lecture will be recorded -
—————————————————————————————————
Lecture video can be found in Canvas ”Media Gallery”
MATH 4242-Week 10-3 1 Spring 2020
-
-
§ The QR Factorization
Revisit the Gram-Schmidt process:
Let A = [a1 · · · an] be n⇥ n nonsingular matrix, where aj is the ith column vector
of A. Thus, a1, · · · , an are linearly independent.
Turn a1, · · · , an to orthogonal vectors v1, · · · ,vn:
v1 = a1
v2 = a2 �ha2,v1ikv1k2
v1
v3 = a3 �ha3,v1ikv1k2
v1 �ha3,v2ikv2k2
v2
...
vn = an �han,v1ikv1k2
v1 � · · ·� han,vn�1ikvn�1k2
vn�1
Normalize v1, · · · ,vn to get orthonormal vectors q1, · · · ,qn:
That is,
qj =vj
kvjk)|{z}
rjj=kvjkrjjqj = vj.
We can get
r11q1 = a1
r22q2 = a2 � ha2,q1iq1
r33q3 = a3 � ha3,q1iq1 � ha3,q2iq2...
rnnqn = an � han,q1iq1 � · · ·� han,qn�1iqn�1
MATH 4242-Week 10-3 2 Spring 2020
"114119 , CAZ
, 9178 ' 9g- = yjll- a-97nF.ae z. + " i. case. > a
.
Vj=HVjH9j .HzT
= rjj 9J .
I3 723- -
Vin rn-I, h .
Let rij = haj,qii, then we further have
r11q1 = a1
r22q2 = a2 � r12q1
r33q3 = a3 � r13q1 � r23q2...
rnnqn = an � r1nq1 � · · ·� rn�1,nqn�1
We reach
a1 = r11q1
a2 = r12q1 + r22q2
a3 = r13q1 + r23q2 + r33q3...
an = r1nq1 + +r2nq2 + · · · + rn�1,nqn�1 + rnnqn
The Gram-Schmidt equations can then be recast into an equivalent matrix form:
0
@ a1 a2 · · · an
1
A
| {z }A
=
0
@ q1 q2 · · · qn
1
A
| {z }Q
0
BBB@
r11 r12 · · · r1nr22 · · · r2n
. . ....
rnn
1
CCCA
| {z }R
where rjj = kvjk and rij = haj,qii. This is called the QR factorization.
MATH 4242-Week 10-3 3 Spring 2020
{ai, . . ., an } l . independent EY. .. .
, rn } orthogonal9i= VillNill
-
⇒ ( Gi , - - y Gn ) ON
-
=
← Aft<"
aj.gs
"
toa orthogonal
Fact: Every nonsingular matrix can be factored into, A = QR, the product of
an orthogonal matrix Q and an upper triangular matrix R.
The factorization is unique if R is positive upper triangular(meaning that all
its diagonal entries are positive (rjj > 0).
Example. Let A = [a1 a2 a3] where
a1 =
0
@1
1
0
1
A , a2 =
0
@0
1
1
1
A , a3 =
0
@1
0
1
1
A .
Find the QR factorization of A.[Ans:] We have saw in Lecture 22 how to apply Gram-Schmidt to the vectors ai:
We found the resulting orthogonal basis to be
v1 =
0
@1
1
0
1
A ,v2 =
0
@�1/21/21
1
A ,v3 =
0
@2/3
�2/32/3
1
A .
Thus, letting qk =vkkvkk
, we have the orthonormal basis is
q1 =1p2
0
@1
1
0
1
A ,q2 =
r2
3
0
@�1/21/21
1
A ,q3 =
0
@1/p3
�1/p3
1/p3
1
A
It is remaining to find R.
MATH 4242-Week 10-3 4 Spring 2020
-
CD-
= = =
=
= =
→⇐We have known D= ( 8 , Ez E, 7 .
To find R : R -
- ( "g' "
I. Try,
].Hii = 11 Vill
. K ,= 11411=52
,r. >= 11411 = FI{ Vij = ( Aj , Gi > 833=11411 = ¥
,
[Example Continue:]
§ Use QR factorization of A to solve linear system Ax = b.
If A = QR, we want to use it to solve the system Ax = b:
From the linear system, we get
QRx = b
Since Q�1= QT
(Q is orthogonal), we have
Rx = QTb.
Therefore, we just need to solve
Rx = QTb
by applying back substitution since R is upper triangular matrix.
Example. Apply this to solve the system
0
@1 0 1
1 1 0
0 1 1
1
A
0
@xyz
1
A =
0
@1
0
0
1
A = b
MATH 4242-Week 10-3 5 Spring 2020
riz = Laz,8 , > = ( ( Y)
,
¥ ( I ) ) = t.
% = C as , 9 , > = ( ( lol , rt ( Il ) = Is.
ru -
- ca .. as = "
. ÷Eo¥÷ ).
#
--
-
T1-- O
-A ~
From previous example . A = Q R.
OR LEI -- l ! ).
R ( El = ay !)
[Example Continue:]
MATH 4242-Week 10-3 6 Spring 2020
E. ¥: ÷÷÷iHt②
using bad. as.mn.
① F z =
.
⇒ 2- = I.
⑦ Ey t fro Z = tf ⇒ y = -K.
③x -- "
'
any . (#
4.4 Orthogonal Projections and Subspaces
§ Orthogonal projections
Definition: We call a vector x in an inner product space V is orthogonal to
the subspace W of V if it is orthogonal to every vector in W , that is,
hx,wi = 0 for all w 2 W.
Suppose w1, · · · ,wn is the basis of W . Thus,
x is orthogonal to W , hx,wii = 0, i = 1, · · · , n.
[To see this:]
Definition: The orthogonal projection of v onto the subspace W of V is
the element w 2 W such that the di↵erence z = v �w orthogonal to W .
We use the notation
z ? W.
The orthogonal projection is “unique”.
Note that such w is the unique vector in W that is “closet to” v.
MATH 4242-Week 10-3 7 Spring 2020
Recall :-
gu- fYj w , orthogonal to w .
Et:EYEw , orthogonal projection of x onto vector w .
=
ax= Hw
⇒ ) since wi e-W,
C X, Wi 7=0 . IE ien.
(E) Any y E W can be written as
y = C, W, t - - - t Cn Wn.
( x, y > = (x ,
t> go
= c, cx.tv?j-.--cuLx/wu?
= O. #
" " IEEE:
Fact: Suppose u1, · · · ,un is an orthonormal basis of subspace W of V . If
w 2 W is the orthogonal projection of v 2 V onto W , then
w = c1u1 + · · · cnun,
where
cj = hv,uji, j = 1, · · · , n.
[To see this:]
Remark: Thus, v �Pn
k=1hv,ukiuk is orthogonal to W , that is,
⇣v �
nX
k=1
hv,ukiuk
⌘? W.
MATH 4242-Week 10-3 8 Spring 2020
-
= =
IT-
(il WEW can be written as linear combination
of E U ... . .
,Un } , so
w = I, U , t . - - t CI Un .
j = Cc , u , t . - -t Chun , Uj > = Cjlujsuj?= Cj llujll '
(2) Since W is orthogonal projection of uooEw(V- w) t W * e
-w
⇐
So, Lu-W , Uj > = o ⇒ C = CHIP-• It ( nil B orthogonal , then Cj = NY
" 4*112' #
"-
w
§ Orthogonal Subspaces
Definition: Two subspaces W,Z of V are called orthogonal if every vector
in W is orthogonal to every vector in Z, that is,
hw, zi = 0 for all w 2 W, z 2 Z.
Immediately, we also have
Fact: If {w1, · · · ,wn} span W and {z1, · · · , zk} span Z, then
W,Z are orthogonal , hwi, zji = 0
for all 1 i n, 1 j k.
Definition: If W is a subspace of V , its orthogonal complement W?
(pronounced “W perp”) is the set of all vectors orthogonal to W , that is,
W?= {v 2 V : hv,wi = 0 for all w 2 W}.
• It can be checked that W?is also a subspace of V .
• If W = span{w}, we will also denote W?by w
?.
• Note that the only vector contained in both W and W?is 0.
MATH 4242-Week 10-3 9 Spring 2020
-
--
O=O pv=5%0=
Example. Find W?, the orthogonal complement to W = span{w} in R3
, where
w =
0
@1
�2
1
1
A
Example. Suppose W = span{w1,w2}, where
w1 =
0
@1
2
1
1
A , w2 =
0
@0
�1
1
1
A .
To find W?.
MATH 4242-Week 10-3 10 Spring 2020
f'll,-2,11TI -_ ix. 4. as in wt HI
dWTI = 0
. 6-din?CLT -2 1) ( ¥ ) -- O .
put Otree variables.
x-
- ay - Z . w¥{ ( 21¥) I y.zc.IR).
*
WTI -- o
wie -
- o.
( IE ) 5=0 .
1- continue on Monday)