a rank-revealing method for low rank matrices with updating, downdating, and applications
DESCRIPTION
A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications. Tsung-Lin Lee (Michigan State University). joint work with Tien-Yien Li and Zhonggang Zeng. 2007 AMS Session Meeting, Chicago. Rank determination problems appear in 1. Image Processing - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/1.jpg)
A Rank-Revealing Method for Low Rank Matrices withUpdating, Downdating, and Applications
Tsung-Lin Lee(Michigan State University)
2007 AMS Session Meeting, Chicago
joint work with Tien-Yien Li and Zhonggang Zeng
![Page 2: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/2.jpg)
Rank determination problems appear in
1. Image Processing
2. Information Retrieval
3. Matrix Approximation
4. Least Squares Problems
5. Numerical Polynomial Algebra
……
![Page 3: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/3.jpg)
1
k
k
The rank
gap:
kAranknkk
)(11
Numerical rank: nmR nm ,
the rank decision threshold :
the approxi-rank w.r.t. the threshold : (numerical rank)
)(rank
![Page 4: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/4.jpg)
2k
kr k,rank(A) Assume
2})(|{12})(|{
minmin BABArBrankB
rrBrankB
Mirsky Theorem:
1k
A
k
![Page 5: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/5.jpg)
2k
The numerical rank w.r.t. threshold :
)(min2
BrankArankAB
1k
A
k
![Page 6: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/6.jpg)
SVD Algorithm (Golub-Reinsch)
In some applications, the matrix is large.
-The rank is close to full. (high rank)
-The rank is close to zero. (low rank)
=> efficient when the matrix size is moderate.
The goal: An efficient and stable algorithm
![Page 7: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/7.jpg)
The updating and downdating problems
=> It can’t solve them efficiently.
1989 Tony Chan => Rank Revealing QR algorithm for high rank matrices
![Page 8: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/8.jpg)
Updating problem:
krank )(
?)ˆ( rank
?)ˆ( rank
![Page 9: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/9.jpg)
Downdating problem:
krank )(
?)ˆ( rank
?)ˆ( rank
![Page 10: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/10.jpg)
1992 G.W. Stewart => rank revealing UTV decomposition. (URV/ULV)
1. Updating problems are applicable.
2. Downdating problems are difficult.
=> re-compute the UTV decomposition
F.D. Fierro, P.C. Hansen and P.S. K. Hansen (1999)UTV tools: Matlab templates for rank-revealing UTV decomposition
![Page 11: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/11.jpg)
2005, T.Y. Li and Zhonggang Zeng
=> rank-revealing algorithm for high rank matrices
1. The approxi-rank.2. The approxi-kernel.3. The method is more efficient and robust.4. Algorithms for updating and downdating problems are straightforward, stable and efficient.
![Page 12: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/12.jpg)
kernel-approxileftkernel-approxi
range-approxi rowspace-approxi
Tnkk
n
k
k
mkk vvvvuuuu
111
1
11
nkk
11
![Page 13: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/13.jpg)
Tsung-Lin Lee, T.Y. Li and Zhonggang Zeng
=> rank-revealing algorithm for low rank matrices
1. The approxi-rank.2. The approxi-range.3. The approxi-rowspace.4. The projections of left and right kernel.5. USV+E decomposition.6. The method is robust and more efficient.7. Algorithms for updating and downdating problems are straightforward, stable and efficient.
= +
![Page 14: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/14.jpg)
2
21
1
j
jj
jT
jT
j
xA
xy
yA
yx
1v
1u
Stop when thresholdepsyA
j
jT
:,
2
21
Power iteration on T
Random 0y
,3,2,1for j
![Page 15: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/15.jpg)
0y
1y
2y
4y1u
epsO
0
mR
3y
},,,{ 21 kuuuspan approxi-range
![Page 16: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/16.jpg)
},,,{)( 211 kuuuspanrangez
nnkk 1121
0' k
The implicit singular value deflation:
0'''1121
nkk
Tzz 11'
![Page 17: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/17.jpg)
Tp UUE kzzU ,,1
USV+E decomp.
TT LQU LQ pT EULQ
)(1 rangez
Tzz 11' )'(2 rangez
TT zzzz 2211'' )''(3 rangez
Tkk
Tp zzzzE 11
2pE
![Page 18: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/18.jpg)
perturbation= +
USV+E decomposition
kk
approxi-range
approxi-rowspace
![Page 19: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/19.jpg)
Numerical experiments and comparisonsMatlab 7.0, on Dell PC Pentium D 3.2MHz CPU, 1GB RAM
2n
n,20
2
400x200 800x400 1600x800 3200x1600
time error time error time error time error
SVD 0.31 3e-9 2.19 4e-9 16.6 3e-9 144. 7e-9
lurv 0.66 3e-9 1.52 4e-9 5.97 3e-9 32.5 7e-9
lulv 0.56 4e-9 1.52 6e-9 6.03 5e-9 31.9 5e-9
larank 0.05 3e-9 0.11 4e-9 0.39 3e-9 1.81 4e-9
3111
10 e
10)(,81 ranke
![Page 20: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/20.jpg)
A U= + pE
TV
Row updating
A U= + pE
TV
1
TV~
A U= + pE
TV
1
Tv~
2aVVa T
aVVa
vT
~
Ta
![Page 21: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/21.jpg)
row downdating
1)ˆ( korkrank )()ˆ( rowrow
QRUV )(min
R
deflate R
min
210
0ˆ
R
RGG
pT EQRV
= +
EUSV T
![Page 22: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/22.jpg)
Dominant(signal)A Perturbation
(noise)
= +
USV+E decomposition
)( 2nO )(nO
![Page 23: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/23.jpg)
Information retrieval
Latent Semantic Indexing method (LSI)
Library database
Webpage search engine (Google)
…
![Page 24: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/24.jpg)
![Page 25: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/25.jpg)
rank, revealing, updating, downdating, application
1
0
1
1
0
0
0
0
0
1
0
1
||
q12x8 term by document matrix
![Page 26: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/26.jpg)
2222
,cos
j
jT
j
j
jAeq
Aeq
Aeq
Aeq
EUSVA T
= +
222222
~cos
j
jT
jT
jTT
jT
jTT
jWeq
WeUq
eSVq
eSVUq
eUSVq
eUSVq
TSVW
![Page 27: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/27.jpg)
Image processing
Saving storage of photographs
FBI Fingerprint Image Database
Face Image Database
…
![Page 28: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/28.jpg)
A 480x640 monochrome (baseball picture)
Grey levels: 0 => 1
black white
![Page 29: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/29.jpg)
j
j
![Page 30: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/30.jpg)
Rank 20 approximation imageRank 480 image
20)(%35.1 1 kArank
![Page 31: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/31.jpg)
![Page 32: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/32.jpg)
7.477.920.735.38 : 1510.8%
2.014.945.140.388.07 : 1341.3%
k SVD
2.87
lulv
3.02
lurvlarank
0.17
Running time (seconds)
15.2 : 1
Compression ratio
18
Approxi-rank
2.1%
Threshold
1
1
1
1:)( knm
mn
![Page 33: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/33.jpg)
http://www.msu.edu/~leetsung/Software.htm
HighRankRev and LowRankRev Package
![Page 34: A Rank-Revealing Method for Low Rank Matrices with Updating, Downdating, and Applications](https://reader035.vdocuments.us/reader035/viewer/2022062518/56814331550346895dafa4d4/html5/thumbnails/34.jpg)
Thank you