![Page 1: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/1.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 1
Discrete IllPosed and RankDeficient Problems
![Page 2: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/2.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 2
Overview● Definitions
– Inversion, SVD, Picard Condition,– Rank Deficient, Ill-Posed
● “Classical” Regularization– Tikhonov and the Discrete Smoothing
Norms
● Discretization– Quadrature and Galerkin
![Page 3: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/3.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 3
Overview 2● Discretization
– Discrete Picard Condition– Quadrature– Galerkin
● Other Techniques– TSVD– CG Interation
![Page 4: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/4.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 4
Inversion
● Solve for x, when A and b are known– (or for A, when x and b are known)– via Inverse, Psuedo-Inverse
Ax=b
x=At bx=A−1b
![Page 5: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/5.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 5
SVD● SVD(A)
– or GSVD(A,L)
● Singular Values
● Singular Vectors
– Left & Right
A=U V T=∑i=1
n
ui i v iT
U TU=VV T=I n
σ in non-increasing order
AT A=V 2V T AAT=U 2U T
(2)
![Page 6: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/6.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 6
Rank Deficient● Gap in the
Singular Values
● Approx. Zero – Noise!
![Page 7: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/7.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 7
IllPosed● No gap in values
● Lots of small values
![Page 8: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/8.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 8
(Discrete) Picard Condition
∫0
1
K s ,t f t =g s Ax=b
A=U V T=∑i=1
n
ui i viT
DiscreteContinuous
K s , t =∑i=1
∞
iui sv i t
SVE SVD
f t =∑i=1
∞ ui , g
iv i t x=Atb=∑
i=1
n uiT b
iv i
∑i=1
∞
ui , g i 2
∞...
denominator must decrease faster than numerator or we get an unbounded solution
![Page 9: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/9.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 9
Regularization
● “First regularize, then discretize.”
● Minimize “residual norm”– with Constrained Values,– with Constrained Size,
● Minimize Size, with Constrained Residual
● Minimize Residual and Size– Tikhonov!
r=∫0
1
K s ,t f t −g s
min∥r∥22 , x∈S
min∥r∥22 , w
min∥w ∥22 , ∥r∥2
2
min{∥r∥222∥w ∥2
2}
![Page 10: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/10.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 10
Tikhonov Regularization
● Solved by Least Squares using SVD
min {∥Ax−b∥22 2∥L x−x0∥2
2 }
![Page 11: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/11.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 11
Discrete Smoothing Norms
● Choices for “L”:
– Identity Matrix– Weighted Diagonal– Discrete Derivative Approximations
![Page 12: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/12.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 12
L = I● The identity matrix...
Additional constraint:
– Minimize the absolute value of the solution
min {∥Ax−b∥22 2∥I x−x0∥2
2 }
![Page 13: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/13.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 13
L = diag(w)● A diagonal matrix of weights on x...
Additional constraint:
– Minimize a weighted selection of values of the solution
min {∥Ax−b∥22 2∥W x−x0∥2
2 }
W=diag w
![Page 14: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/14.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 14
L = L1 or L2● A banded matrix approximating a
derivative operator...
Additional constraint:– Minimize “total variation” in values of
the solution (but still allow steep gradients)
min {∥Ax−b∥22 2∥L1 x−x0∥2
2 }
![Page 15: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/15.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 15
Priors● Allow the addition of “a priori”
information about the result
● Using no prior is the same as a zero prior
● Weight solution towards expectation
min {∥Ax−b∥22 2∥L x−x0∥2
2 }
![Page 16: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/16.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 16
Lcurve
(fake plot)
over-smoothingunder-smoothing
![Page 17: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/17.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 17
Discretization of Integral Equations
● Quadrature
● Galerkin
– Raliegh-Ritz
∫0
1
t dt≃∑j=1
n
w jt j aij=w j K s , t bi=g s i
aij=∫0
1
K s , t i s j t ds dt bi=∫0
1
g si sds
choose w j
choose ,
= If , K is symetric, and nodes are co-located
![Page 18: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/18.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 18
Solution
(fake plots)
![Page 19: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/19.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 19
Discussion
[http://www.clipartguide.com/_pages/0808-0712-3117-5830.html]
![Page 20: Discrete IllPosed and RankDeficient Problems · J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005. Created Date: 2/1/2009 9:32:01 PM](https://reader035.vdocuments.us/reader035/viewer/2022071019/5fd33c1711e0241b732ac1bc/html5/thumbnails/20.jpg)
Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 20
References● P. C. Hansen, Rank-Deficient and
Discrete Ill-Posed Problems, SIAM, 1998
● Wikipedia: Inverse Problems, http://en.wikipedia.org/wiki/Inverse_problem, visited Jan 28, 2009
● J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005