x'b - oregon state university
TRANSCRIPT
![Page 1: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/1.jpg)
LECTURE34 THECONJUGATEGRADIENT METHOD
Applies to SPD matrices
symmetric positive definite
Assume A ElRm mXE IR b c ID
Wewish to find X I suck that
AE b FA b givenAnoptimizationapproachlet f x'Bx x'b
fixA
x
![Page 2: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/2.jpg)
So Qf Ax bDirection of maximumAscent
Notethat If so Ani s bI i s the solution sought
8 2 f A since A is SPD e values arereal and positive is f is CONVEXconcaveup
In optimization framework find
win fGXcIRM
Rmd We can recenterproblemby a translation
let f fix A Ix E let r s x I
so I I ran'Arlx so rttOf
![Page 3: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/3.jpg)
Introduce a ITERATIVELINESEARCH
yk t Xk t 2kt Pkti k0,1
L c IR p E IRMP changesdirection
x itsmagnitude
To generate a sequence Xo74 so
that lemme I starting withK as
some initial guess Xo
Find me put so that thecorrection
is 1 to the currentsearch directions
If r ut s AXun b
We will require that
r Iti Pkt O
![Page 4: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/4.jpg)
In LD thepath looks likeXsHieitxzei
HE
B
N XoContours off A
Now we presenthow 2k and putare designedlet's assume we know Pati focus onfind 2kt Then we'll figureouthow to
find Pui
![Page 5: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/5.jpg)
Take flxktd ffxktdkt.pkdifferentiatewrthet andsetto 0
O ddg.flx.nu sQfYxktJdadg.kutaut.pu
0sCAxkttbTf xntdmpnn s ripSo if we design pm so that
rniipn.is 0
Since Reti s b DXu b Alxnthetipunrn 2kt APutt
then pathensospiciirn dump Apu
I Met PEE theRFiAp
k O I
![Page 6: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/6.jpg)
In fact we can designpi's so that
f rut Piso isiskti
let Pos RoPkn rut PlettPk
solet's find putPropose that Pi Apu Sig R
Isisjsmsijsfoii.gs
Wesaythat thevectors pjareA Conjugate
if f holds
Take multiplybyPj'djs0,1 k
Pippin spjdrntpkt.PEtpnBut PEARL
Pn A Pk
![Page 7: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/7.jpg)
Wenotice that with pos.ro
creates a sequence po p that areA conjugate
Rusk we can specifyI tipi since pi
spans 1cmHence AI Aldp t dmpmMultiplybothsidesby pie
P I BI s 2kmPj APai shenSjkyALGORITHM
Wewill use me hirnPutAPkt
Bht hireru ru i
![Page 8: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/8.jpg)
Instead ofpug rn
p PEATdktl.ph e jkttpEBpnWe'llshow afterpresentingthealgorithmthat these are equivalent
ALGORITHM
ro b Xo 0 Posts
for k s l ITMDX
Bet hireru.ir
Theti s th tBatiPk
ir aXk ti Xu t Lkti Pktrhet b Amexif Hrm It s Toland HXia MelleTokbp
![Page 9: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/9.jpg)
There are a couple of technical details
that need to be addressed so that
all of the quantities are explained
Arne Ktrkda s PkiiPui Apa PuiAPu
Weknow thathe b Axnb Amu Adupic
run ru da Apu
Multiply by rn
retreat ru ru da ru Apun0 since they are 1
![Page 10: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/10.jpg)
reversingorder using2n rnTApu hire a at
our dietputArne rim H
since As A
So followsfromjPkiiAru.rirITheotlre.rstatement is PuiApna PuiApuM
put PEATPEAR
Kirk
WeshowthisnextRetire
Weknow thatPutt Det Pkt Pk8kt ru dutApkrErw shirk out.ru Apu0 transposing2ndterm
anti PuAke shirk
pitsrusts nineThenumerator of putt
![Page 11: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/11.jpg)
Now the denominator
passion ru.ir
Since the ru i 2kApk
and perk i n O2kt PEApie thenPEAru itdknputiPEA.pe
4ru's
so the ratio of
Piedra
pgshirk since He aw.isRetry Cancel
I
![Page 12: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/12.jpg)
Another Useful fact THEOREM38lyif Xk Met dietputt
Pkti s ru t PuiPurhett he t2kt Apk
Then fortheproblem Axsb as long
as the iterationhasnot conveyed r.es othe algorithm GG proceed w o divisins
by zero andthe following subspacesare identical
x Xz Xan PoR The
Cro ri ru CbAb Akb
The residuals are 1
run rj o j skit
and pie Apj so j ski I
Use induction Set
![Page 13: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/13.jpg)
Let Xoso posb and ro b
AHoftlefobbwigc.beshownby applying
Ren by we seethat Xkt cCpop pay
by we see that poet cCroh she
by and he b Axn with 1 0 0roots
it follows thatCro r rn Kb Bb Nb
we already established that
ratio rj rurj antipidrjthat heir 0 solong as jcktl.TWjSince we can expand rj Eaipiisand we assure pfdpjfijfmellysmastarhywithpo.ro byconstruction
PuiApj O jsktl
![Page 14: x'b - Oregon State University](https://reader031.vdocuments.us/reader031/viewer/2022012016/615a9373b7108a797a260213/html5/thumbnails/14.jpg)
Krylov Subspace KThe subspace defined as
b Ab Bb A b Rwwhere A c Em
m be Clm is a
KH dimensional Krylov Space
Rmb Sonce A is SPD in CG then
Im Lb Ab Amb
spans Em