parameterized complexity of matrix factorization - ibm · parameterized complexity of matrix...
TRANSCRIPT
![Page 1: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/1.jpg)
Parameterized Complexity of Matrix Factorization
David P. Woodruff
IBM Almaden
. .
. .
Based on joint works with Ilya Razenshteyn, Zhao Song, Peilin Zhong
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 1 / 50
![Page 2: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/2.jpg)
Outline
Frobenius norm factorization can be done in polynomial time
Intractable matrix problems
I Nonnegative matrix factorization(NMF)
F Arora-Ge-Kannan-Moitra’12, Moitra’13
I Weighted low rank approximation(WLRA)
F Razenshteyn-Song-W’16
I `1 low rank approximation(`1LRA)
F Song-W-Zhong’16
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 2 / 50
![Page 3: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/3.jpg)
Outline
Frobenius norm factorization can be done in polynomial timeIntractable matrix problems
I Nonnegative matrix factorization(NMF)
F Arora-Ge-Kannan-Moitra’12, Moitra’13
I Weighted low rank approximation(WLRA)
F Razenshteyn-Song-W’16
I `1 low rank approximation(`1LRA)
F Song-W-Zhong’16
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 2 / 50
![Page 4: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/4.jpg)
Outline
Frobenius norm factorization can be done in polynomial timeIntractable matrix problems
I Nonnegative matrix factorization(NMF)
F Arora-Ge-Kannan-Moitra’12, Moitra’13I Weighted low rank approximation(WLRA)
F Razenshteyn-Song-W’16
I `1 low rank approximation(`1LRA)
F Song-W-Zhong’16
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 2 / 50
![Page 5: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/5.jpg)
Outline
Frobenius norm factorization can be done in polynomial timeIntractable matrix problems
I Nonnegative matrix factorization(NMF)F Arora-Ge-Kannan-Moitra’12, Moitra’13
I Weighted low rank approximation(WLRA)
F Razenshteyn-Song-W’16
I `1 low rank approximation(`1LRA)
F Song-W-Zhong’16
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 2 / 50
![Page 6: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/6.jpg)
Outline
Frobenius norm factorization can be done in polynomial timeIntractable matrix problems
I Nonnegative matrix factorization(NMF)F Arora-Ge-Kannan-Moitra’12, Moitra’13
I Weighted low rank approximation(WLRA)
F Razenshteyn-Song-W’16I `1 low rank approximation(`1LRA)
F Song-W-Zhong’16
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 2 / 50
![Page 7: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/7.jpg)
Outline
Frobenius norm factorization can be done in polynomial timeIntractable matrix problems
I Nonnegative matrix factorization(NMF)F Arora-Ge-Kannan-Moitra’12, Moitra’13
I Weighted low rank approximation(WLRA)F Razenshteyn-Song-W’16
I `1 low rank approximation(`1LRA)
F Song-W-Zhong’16
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 2 / 50
![Page 8: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/8.jpg)
Outline
Frobenius norm factorization can be done in polynomial timeIntractable matrix problems
I Nonnegative matrix factorization(NMF)F Arora-Ge-Kannan-Moitra’12, Moitra’13
I Weighted low rank approximation(WLRA)F Razenshteyn-Song-W’16
I `1 low rank approximation(`1LRA)
F Song-W-Zhong’16
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 2 / 50
![Page 9: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/9.jpg)
Outline
Frobenius norm factorization can be done in polynomial timeIntractable matrix problems
I Nonnegative matrix factorization(NMF)F Arora-Ge-Kannan-Moitra’12, Moitra’13
I Weighted low rank approximation(WLRA)F Razenshteyn-Song-W’16
I `1 low rank approximation(`1LRA)F Song-W-Zhong’16
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 2 / 50
![Page 10: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/10.jpg)
Frobenius Norm Matrix Factorization
QuestionGiven a matrix A ∈ Rn×n and k > 1, the goal is to output two matricesU ∈ Rn×k ,V ∈ Rk×n such that
‖UV − A‖2F 6 (1 + ε) minrank−k A ′
‖A ′ − A‖2F ,
where ‖A‖2F = (∑n
i=1∑n
j=1 A2i,j)
2, ε ∈ [0,1)
Useful for data compression, easier to store factorizations U,VCan be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... timeFails if you want a nonnegative factorizationNot robust to noise, outliers, or missing entries
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 3 / 50
![Page 11: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/11.jpg)
Frobenius Norm Matrix Factorization
QuestionGiven a matrix A ∈ Rn×n and k > 1, the goal is to output two matricesU ∈ Rn×k ,V ∈ Rk×n such that
‖UV − A‖2F 6 (1 + ε) minrank−k A ′
‖A ′ − A‖2F ,
where ‖A‖2F = (∑n
i=1∑n
j=1 A2i,j)
2, ε ∈ [0,1)
Useful for data compression, easier to store factorizations U,V
Can be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... timeFails if you want a nonnegative factorizationNot robust to noise, outliers, or missing entries
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 3 / 50
![Page 12: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/12.jpg)
Frobenius Norm Matrix Factorization
QuestionGiven a matrix A ∈ Rn×n and k > 1, the goal is to output two matricesU ∈ Rn×k ,V ∈ Rk×n such that
‖UV − A‖2F 6 (1 + ε) minrank−k A ′
‖A ′ − A‖2F ,
where ‖A‖2F = (∑n
i=1∑n
j=1 A2i,j)
2, ε ∈ [0,1)
Useful for data compression, easier to store factorizations U,VCan be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... time
Fails if you want a nonnegative factorizationNot robust to noise, outliers, or missing entries
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 3 / 50
![Page 13: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/13.jpg)
Frobenius Norm Matrix Factorization
QuestionGiven a matrix A ∈ Rn×n and k > 1, the goal is to output two matricesU ∈ Rn×k ,V ∈ Rk×n such that
‖UV − A‖2F 6 (1 + ε) minrank−k A ′
‖A ′ − A‖2F ,
where ‖A‖2F = (∑n
i=1∑n
j=1 A2i,j)
2, ε ∈ [0,1)
Useful for data compression, easier to store factorizations U,VCan be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... timeFails if you want a nonnegative factorization
Not robust to noise, outliers, or missing entries
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 3 / 50
![Page 14: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/14.jpg)
Frobenius Norm Matrix Factorization
QuestionGiven a matrix A ∈ Rn×n and k > 1, the goal is to output two matricesU ∈ Rn×k ,V ∈ Rk×n such that
‖UV − A‖2F 6 (1 + ε) minrank−k A ′
‖A ′ − A‖2F ,
where ‖A‖2F = (∑n
i=1∑n
j=1 A2i,j)
2, ε ∈ [0,1)
Useful for data compression, easier to store factorizations U,VCan be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... timeFails if you want a nonnegative factorizationNot robust to noise, outliers, or missing entries
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 3 / 50
![Page 15: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/15.jpg)
. .
. .
Nonnegative Matrix Factorization
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 4 / 50
![Page 16: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/16.jpg)
Example of NMF. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 5 / 50
![Page 17: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/17.jpg)
Example of NMF. .
. .
Given : A ∈ Rn×n, n = 4, k = 2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 5 / 50
![Page 18: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/18.jpg)
Example of NMF. .
. .
Given : A ∈ Rn×n, n = 4, k = 2
A =
0 1 1 11 2 1 12 4 2 23 5 2 2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 5 / 50
![Page 19: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/19.jpg)
Example of NMF. .
. .
Given : A ∈ Rn×n, n = 4, k = 2
A =
0 1 1 11 2 1 12 4 2 23 5 2 2
Question : Are there two matrices U,V> ∈ Rn×k , such thatUV = A,U > 0,V > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 5 / 50
![Page 20: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/20.jpg)
Example of NMF. .
. .
Given : A ∈ Rn×n, n = 4, k = 2
A =
0 1 1 11 2 1 12 4 2 23 5 2 2
Question : Are there two matrices U,V> ∈ Rn×k , such thatUV = A,U > 0,V > 0
=
0 11 12 23 2
· [1 1 0 00 1 1 1
]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 5 / 50
![Page 21: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/21.jpg)
Example of NMF. .
. .
Given : A ∈ Rn×n, n = 4, k = 2
A =
0 1 1 11 2 1 12 4 2 23 5 2 2
Question : Are there two matrices U,V> ∈ Rn×k , such thatUV = A,U > 0,V > 0
=
0 11 12 23 2
· [1 1 0 00 1 1 1
]
= UV
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 5 / 50
![Page 22: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/22.jpg)
Main Question, Nonnegative Matrix Factorization
QuestionGiven matrix A ∈ Rn×n and k > 1, is there an algorithm that candetermine if there exist two matrices U,V> ∈ Rn×k ,
UV = A,U > 0,V > 0.
Or, are there any hardness results?
Equivalent to computing the nonnegative rank of A, rank+(A)Fundamental question in machine learningApplications
I Text mining, Spectral data analysis, Scalable Internet distanceprediction, Non-stationary speech denoising, Bioinformatics,Nuclear imaging, etc.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 6 / 50
![Page 23: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/23.jpg)
Main Question, Nonnegative Matrix Factorization
QuestionGiven matrix A ∈ Rn×n and k > 1, is there an algorithm that candetermine if there exist two matrices U,V> ∈ Rn×k ,
UV = A,U > 0,V > 0.
Or, are there any hardness results?
Equivalent to computing the nonnegative rank of A, rank+(A)
Fundamental question in machine learningApplications
I Text mining, Spectral data analysis, Scalable Internet distanceprediction, Non-stationary speech denoising, Bioinformatics,Nuclear imaging, etc.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 6 / 50
![Page 24: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/24.jpg)
Main Question, Nonnegative Matrix Factorization
QuestionGiven matrix A ∈ Rn×n and k > 1, is there an algorithm that candetermine if there exist two matrices U,V> ∈ Rn×k ,
UV = A,U > 0,V > 0.
Or, are there any hardness results?
Equivalent to computing the nonnegative rank of A, rank+(A)Fundamental question in machine learning
Applications
I Text mining, Spectral data analysis, Scalable Internet distanceprediction, Non-stationary speech denoising, Bioinformatics,Nuclear imaging, etc.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 6 / 50
![Page 25: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/25.jpg)
Main Question, Nonnegative Matrix Factorization
QuestionGiven matrix A ∈ Rn×n and k > 1, is there an algorithm that candetermine if there exist two matrices U,V> ∈ Rn×k ,
UV = A,U > 0,V > 0.
Or, are there any hardness results?
Equivalent to computing the nonnegative rank of A, rank+(A)Fundamental question in machine learningApplications
I Text mining, Spectral data analysis, Scalable Internet distanceprediction, Non-stationary speech denoising, Bioinformatics,Nuclear imaging, etc.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 6 / 50
![Page 26: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/26.jpg)
Main Question, Nonnegative Matrix Factorization
QuestionGiven matrix A ∈ Rn×n and k > 1, is there an algorithm that candetermine if there exist two matrices U,V> ∈ Rn×k ,
UV = A,U > 0,V > 0.
Or, are there any hardness results?
Equivalent to computing the nonnegative rank of A, rank+(A)Fundamental question in machine learningApplications
I Text mining, Spectral data analysis, Scalable Internet distanceprediction, Non-stationary speech denoising, Bioinformatics,Nuclear imaging, etc.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 6 / 50
![Page 27: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/27.jpg)
Rank vs. Nonnegative Rankrank(A) is the rank and rank+(A) is the nonnegative rank
I rank(A) 6 rank+(A)I Vavasis’09, determining whether rank+(A) = rank(A) is NP-hard.
.
.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 7 / 50
![Page 28: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/28.jpg)
Rank vs. Nonnegative Rankrank(A) is the rank and rank+(A) is the nonnegative rank
I rank(A) 6 rank+(A)
I Vavasis’09, determining whether rank+(A) = rank(A) is NP-hard.
.
.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 7 / 50
![Page 29: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/29.jpg)
Rank vs. Nonnegative Rankrank(A) is the rank and rank+(A) is the nonnegative rank
I rank(A) 6 rank+(A)I Vavasis’09, determining whether rank+(A) = rank(A) is NP-hard.
.
.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 7 / 50
![Page 30: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/30.jpg)
Rank vs. Nonnegative Rankrank(A) is the rank and rank+(A) is the nonnegative rank
I rank(A) 6 rank+(A)I Vavasis’09, determining whether rank+(A) = rank(A) is NP-hard.
.
.
A =
1 1 0 01 0 1 00 1 0 10 0 1 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 7 / 50
![Page 31: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/31.jpg)
Rank vs. Nonnegative Rankrank(A) is the rank and rank+(A) is the nonnegative rank
I rank(A) 6 rank+(A)I Vavasis’09, determining whether rank+(A) = rank(A) is NP-hard.
.
.
A =
1 1 0 01 0 1 00 1 0 10 0 1 1
=
1 1 01 0 10 1 00 0 1
·1 0 0 −1
0 1 0 10 0 1 1
rank(A) = 3,
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 7 / 50
![Page 32: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/32.jpg)
Rank vs. Nonnegative Rankrank(A) is the rank and rank+(A) is the nonnegative rank
I rank(A) 6 rank+(A)I Vavasis’09, determining whether rank+(A) = rank(A) is NP-hard.
.
.
A =
1 1 0 01 0 1 00 1 0 10 0 1 1
=
1 1 01 0 10 1 00 0 1
·1 0 0 −1
0 1 0 10 0 1 1
rank(A) = 3, but rank+(A) = 4
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 7 / 50
![Page 33: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/33.jpg)
Polynomial System Verifier
. .
. .
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 8 / 50
![Page 34: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/34.jpg)
Polynomial System Verifier
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 8 / 50
![Page 35: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/35.jpg)
Polynomial System Verifier
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a polynomial system P(x) over the real numbers
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 8 / 50
![Page 36: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/36.jpg)
Polynomial System Verifier
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a polynomial system P(x) over the real numbersv : #variables, x = (x1, x2, · · · , xv )
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 8 / 50
![Page 37: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/37.jpg)
Polynomial System Verifier
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a polynomial system P(x) over the real numbersv : #variables, x = (x1, x2, · · · , xv )
m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 8 / 50
![Page 38: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/38.jpg)
Polynomial System Verifier
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a polynomial system P(x) over the real numbersv : #variables, x = (x1, x2, · · · , xv )
m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomial constraints
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 8 / 50
![Page 39: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/39.jpg)
Polynomial System Verifier
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a polynomial system P(x) over the real numbersv : #variables, x = (x1, x2, · · · , xv )
m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomial constraintsH : the bitsizes of the coefficients of the polynomials
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 8 / 50
![Page 40: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/40.jpg)
Polynomial System Verifier
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a polynomial system P(x) over the real numbersv : #variables, x = (x1, x2, · · · , xv )
m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomial constraintsH : the bitsizes of the coefficients of the polynomials
In (md)O(v) poly(H) time, candecide if there exists a solution to polynomial system P
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 8 / 50
![Page 41: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/41.jpg)
Main Idea
. .
. .
1. Write minU,V>∈Rn×k ,U,V>0
‖UV − A‖2F as a polynomial system
that has poly(k) variables and poly(n) constraints and degree
2. Use polynomial system verifier to solve it
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 9 / 50
![Page 42: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/42.jpg)
Algorithm. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 43: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/43.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 44: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/44.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Question :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 45: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/45.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Question : Are there matrices U,V> ∈ Rn×k such that
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 46: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/46.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Question : Are there matrices U,V> ∈ Rn×k such thatUV = A
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 47: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/47.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Question : Are there matrices U,V> ∈ Rn×k such thatUV = AU > 0, V > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 48: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/48.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Question : Are there matrices U,V> ∈ Rn×k such thatUV = AU > 0, V > 0
Output :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 49: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/49.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Question : Are there matrices U,V> ∈ Rn×k such thatUV = AU > 0, V > 0
Output : Yes or No
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 50: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/50.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Question : Are there matrices U,V> ∈ Rn×k such thatUV = AU > 0, V > 0
Output : Yes or Noin n2O(k)
time Arora-Ge-Kannan-Moitra’12
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 51: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/51.jpg)
Algorithm. .
. .
Given : A ∈ Rn×n, k ∈ N
Question : Are there matrices U,V> ∈ Rn×k such thatUV = AU > 0, V > 0
Output : Yes or Noin n2O(k)
time Arora-Ge-Kannan-Moitra’12in 2O(k3)nO(k2) time Moitra’13
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 10 / 50
![Page 52: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/52.jpg)
k -SUM. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 11 / 50
![Page 53: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/53.jpg)
k -SUM. .
. .
k -SUM
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 11 / 50
![Page 54: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/54.jpg)
k -SUM. .
. .
k -SUM
Given: a set of n values s1, s2, · · · , sn each in the range [0,1]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 11 / 50
![Page 55: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/55.jpg)
k -SUM. .
. .
k -SUM
Given: a set of n values s1, s2, · · · , sn each in the range [0,1]
Determine:
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 11 / 50
![Page 56: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/56.jpg)
k -SUM. .
. .
k -SUM
Given: a set of n values s1, s2, · · · , sn each in the range [0,1]
Determine: if there is a set of k numbers that sum to exactly k/2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 11 / 50
![Page 57: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/57.jpg)
k -SUM. .
. .
k -SUM
Given: a set of n values s1, s2, · · · , sn each in the range [0,1]
Determine: if there is a set of k numbers that sum to exactly k/2
k -SUM hardness [Patrascu-Williams’10]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 11 / 50
![Page 58: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/58.jpg)
k -SUM. .
. .
k -SUM
Given: a set of n values s1, s2, · · · , sn each in the range [0,1]
Determine: if there is a set of k numbers that sum to exactly k/2
k -SUM hardness [Patrascu-Williams’10]
Assume: 3-SAT on n variables cannot be solved in 2o(n) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 11 / 50
![Page 59: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/59.jpg)
k -SUM. .
. .
k -SUM
Given: a set of n values s1, s2, · · · , sn each in the range [0,1]
Determine: if there is a set of k numbers that sum to exactly k/2
k -SUM hardness [Patrascu-Williams’10]
Assume: 3-SAT on n variables cannot be solved in 2o(n) time
then k -SUM cannot be solved in no(k) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 11 / 50
![Page 60: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/60.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 12 / 50
![Page 61: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/61.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
Given : A ∈ Rn×n, k ∈ N
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 12 / 50
![Page 62: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/62.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
Given : A ∈ Rn×n, k ∈ N
Output : Are there two matrices U,V> ∈ Rn×k , such thatUV = AU > 0,V > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 12 / 50
![Page 63: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/63.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
Given : A ∈ Rn×n, k ∈ N
Output : Are there two matrices U,V> ∈ Rn×k , such thatUV = AU > 0,V > 0
Assume : Exponential Time Hypothesis[Impagliazzo-Paturi-Zane’98]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 12 / 50
![Page 64: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/64.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
Given : A ∈ Rn×n, k ∈ N
Output : Are there two matrices U,V> ∈ Rn×k , such thatUV = AU > 0,V > 0
Assume : Exponential Time Hypothesis[Impagliazzo-Paturi-Zane’98]
Requires : nΩ(k) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 12 / 50
![Page 65: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/65.jpg)
Open Problems
The upper bound is nO(k2) while the lower bound is nΩ(k) - what isthe right answer?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 13 / 50
![Page 66: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/66.jpg)
. .
. .
Weighted Low Rank Approximation
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 14 / 50
![Page 67: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/67.jpg)
Weighted Low Rank Approximation. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 15 / 50
![Page 68: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/68.jpg)
Weighted Low Rank Approximation. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, k ∈ N, ε > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 15 / 50
![Page 69: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/69.jpg)
Weighted Low Rank Approximation. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, k ∈ N, ε > 0
Output :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 15 / 50
![Page 70: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/70.jpg)
Weighted Low Rank Approximation. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, k ∈ N, ε > 0
Output : rank-k A ∈ Rn×n s.t.
AW A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 15 / 50
![Page 71: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/71.jpg)
Weighted Low Rank Approximation. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, k ∈ N, ε > 0
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′‖W (A ′ − A)‖2F
AW A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 15 / 50
![Page 72: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/72.jpg)
Weighted Low Rank Approximation. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, k ∈ N, ε > 0
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′‖W (A ′ − A)‖2F
.
∑i,j
W 2i,j(A
′i,j − Ai,j)
2
AW A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 15 / 50
![Page 73: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/73.jpg)
Weighted Low Rank Approximation. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, k ∈ N, ε > 0
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′‖W (A ′ − A)‖2F
.
∑i,j
W 2i,j(A
′i,j − Ai,j)
2
.OPT
AW A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 15 / 50
![Page 74: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/74.jpg)
Weighted Low Rank Approximation. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, k ∈ N, ε > 0
Output :
..
U,V> ∈ Rn×k s.t.‖W (UV − A)‖2F 6 (1 + ε)OPT
U
V
W A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 15 / 50
![Page 75: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/75.jpg)
Motivation. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 76: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/76.jpg)
Motivation. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 77: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/77.jpg)
Motivation. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 78: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/78.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 79: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/79.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
.
9 8
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 80: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/80.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
..
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 81: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/81.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 82: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/82.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
....
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 83: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/83.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
.....
15
15
11
11
11
11
15
15
11
11
14
14
16
16
11
11
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 84: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/84.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
.....
15
15
11
11
11
11
15
15
11
11
14
14
16
16
11
11
.
11
11
11
11
11
11
16
16
13
13
12
12
12
12
14
14
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 85: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/85.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
.....
15
15
11
11
11
11
15
15
11
11
14
14
16
16
11
11
.
11
11
11
11
11
11
16
16
13
13
12
12
12
12
14
14
.
13
13
16
16
14
14
12
12
17
17
15
15
14
14
11
11
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 86: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/86.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
.....
15
15
11
11
11
11
15
15
11
11
14
14
16
16
11
11
.
11
11
11
11
11
11
16
16
13
13
12
12
12
12
14
14
.
13
13
16
16
14
14
12
12
17
17
15
15
14
14
11
11
.
12
12
11
11
12
12
11
11
15
15
11
11
13
13
14
14
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 87: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/87.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
.....
15
15
11
11
11
11
15
15
11
11
14
14
16
16
11
11
.
11
11
11
11
11
11
16
16
13
13
12
12
12
12
14
14
.
13
13
16
16
14
14
12
12
17
17
15
15
14
14
11
11
.
12
12
11
11
12
12
11
11
15
15
11
11
13
13
14
14
.
12
12
12
12
16
16
14
14
12
12
13
13
16
16
12
12
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 88: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/88.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
..........
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 89: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/89.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...........
8 8
8 8
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 90: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/90.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...........
8 8
8 8
.
4 4
4 4
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 91: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/91.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...........
8 8
8 8
.
4 4
4 4
.
1 1
1 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 92: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/92.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...........
8 8
8 8
.
4 4
4 4
.
1 1
1 1
.
0 0
0 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 93: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/93.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...........
8 8
8 8
.
4 4
4 4
.
1 1
1 1
.
0 0
0 0
.
8 8
8 8
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 94: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/94.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...........
8 8
8 8
.
4 4
4 4
.
1 1
1 1
.
0 0
0 0
.
8 8
8 8
.
4 4
4 4
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 95: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/95.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...........
8 8
8 8
.
4 4
4 4
.
1 1
1 1
.
0 0
0 0
.
8 8
8 8
.
4 4
4 4
.
2 2
2 2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 96: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/96.jpg)
Motivation. .
. .
Action Comedy Historical Cartoon Magical
...........
8 8
8 8
.
4 4
4 4
.
1 1
1 1
.
0 0
0 0
.
8 8
8 8
.
4 4
4 4
.
2 2
2 2
.
1 1
1 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 16 / 50
![Page 97: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/97.jpg)
Matrix Completion. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 17 / 50
![Page 98: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/98.jpg)
Matrix Completion. .
. .
Given : A ∈ R, ?n×n, Ω ⊂ [n]× [n] and k ∈ R
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 17 / 50
![Page 99: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/99.jpg)
Matrix Completion. .
. .
Given : A ∈ R, ?n×n, Ω ⊂ [n]× [n] and k ∈ R
A =
1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1
, k = 2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 17 / 50
![Page 100: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/100.jpg)
Matrix Completion. .
. .
Given : A ∈ R, ?n×n, Ω ⊂ [n]× [n] and k ∈ R
A =
1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1
, k = 2
Output : AΩ s.t. rank(A) = k
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 17 / 50
![Page 101: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/101.jpg)
Matrix Completion. .
. .
Given : A ∈ R, ?n×n, Ω ⊂ [n]× [n] and k ∈ R
A =
1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1
, k = 2
Output : AΩ s.t. rank(A) = k
.
A =
1 1 0 01 1 0 00 0 1 10 0 1 1
, A =
1 0 1 00 1 0 11 0 1 00 1 0 1
.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 17 / 50
![Page 102: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/102.jpg)
Matrix Completion. .
. .
Given : A ∈ R, ?n×n, Ω ⊂ [n]× [n] and k ∈ R
A =
1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1
, k = 2
Output : AΩ s.t. rank(A) = k
.
Algorithms : Candes-Recht’09, Candes-Recht’10, Keshavan’12Hardt-Wootters’14, Jain-Netrapalli-Sanghavi’13Hardt’15, Sun-Luo’15
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 17 / 50
![Page 103: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/103.jpg)
Matrix Completion. .
. .
Given : A ∈ R, ?n×n, Ω ⊂ [n]× [n] and k ∈ R
A =
1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1
, k = 2
Output : AΩ s.t. rank(A) = k
.
Algorithms : Candes-Recht’09, Candes-Recht’10, Keshavan’12Hardt-Wootters’14, Jain-Netrapalli-Sanghavi’13Hardt’15, Sun-Luo’15
Hardness : Peeters’96, Gillis-Glineur’11Hardt-Meka-Raghavendra-Weitz’14
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 17 / 50
![Page 104: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/104.jpg)
More Real-life Datasets. .
. .
RateBeer
Documents
Amazon
Biology
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 18 / 50
![Page 105: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/105.jpg)
More Real-life Datasets. .
. .
RateBeer
Tasters Beers Ratings
Documents
Amazon
Biology
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 18 / 50
![Page 106: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/106.jpg)
More Real-life Datasets. .
. .
RateBeer
Tasters Beers Ratings
Documents
Words Books Frequency
Amazon
Biology
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 18 / 50
![Page 107: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/107.jpg)
More Real-life Datasets. .
. .
RateBeer
Tasters Beers Ratings
Documents
Words Books Frequency
Amazon
Users Products Ratings
Biology
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 18 / 50
![Page 108: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/108.jpg)
More Real-life Datasets. .
. .
RateBeer
Tasters Beers Ratings
Documents
Words Books Frequency
Amazon
Users Products Ratings
Biology
Genes Cells Levels
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 18 / 50
![Page 109: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/109.jpg)
Main Results - Summary
Algorithm for Weighted low rank approximation(WLRA) problem
I W has r distinct rows and columnsI W has r distinct columnsI W has rank at most r
Hardness for Weighted low rank approximation(WLRA) problem
.
.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 19 / 50
![Page 110: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/110.jpg)
Main Results - Summary
Algorithm for Weighted low rank approximation(WLRA) problemI W has r distinct rows and columns
I W has r distinct columnsI W has rank at most r
Hardness for Weighted low rank approximation(WLRA) problem
.
.
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 19 / 50
![Page 111: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/111.jpg)
Main Results - Summary
Algorithm for Weighted low rank approximation(WLRA) problemI W has r distinct rows and columnsI W has r distinct columns
I W has rank at most r
Hardness for Weighted low rank approximation(WLRA) problem
.
.
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 19 / 50
![Page 112: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/112.jpg)
Main Results - Summary
Algorithm for Weighted low rank approximation(WLRA) problemI W has r distinct rows and columnsI W has r distinct columnsI W has rank at most r
Hardness for Weighted low rank approximation(WLRA) problem
.
.
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 19 / 50
![Page 113: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/113.jpg)
Main Results - Summary
Algorithm for Weighted low rank approximation(WLRA) problemI W has r distinct rows and columnsI W has r distinct columnsI W has rank at most r
Hardness for Weighted low rank approximation(WLRA) problem.
.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 19 / 50
![Page 114: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/114.jpg)
r Distinct Rows and Columns. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 115: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/115.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 116: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/116.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 117: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/117.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 118: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/118.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 119: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/119.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′‖W (A − A ′)‖2F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 120: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/120.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′
.
∑i,j
W 2i,j(Ai,j − A ′i,j)
2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 121: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/121.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPT
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 122: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/122.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 123: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/123.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r/ε) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 124: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/124.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r/ε) timefor an arbitrarily small constant γ > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 125: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/125.jpg)
r Distinct Rows and Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r/ε) timefor an arbitrarily small constant γ > 0fixed parameter tractable
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 20 / 50
![Page 126: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/126.jpg)
r Distinct Columns. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 127: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/127.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 128: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/128.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 129: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/129.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 130: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/130.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output : rank-k A ∈ Rn×n s.t.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 131: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/131.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′‖W (A − A ′)‖2F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 132: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/132.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′
.
∑i,j
W 2i,j(Ai,j − A ′i,j)
2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 133: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/133.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPT
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 134: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/134.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 135: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/135.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r2/ε) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 136: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/136.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r2/ε) timefor an arbitrarily small constant γ > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 137: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/137.jpg)
r Distinct Columns. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r2/ε) timefor an arbitrarily small constant γ > 0fixed parameter tractable
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 21 / 50
![Page 138: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/138.jpg)
Rank r. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 139: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/139.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 140: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/140.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 141: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/141.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Output :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 142: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/142.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 143: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/143.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′‖W (A − A ′)‖2F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 144: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/144.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min
rank−k A ′
.
∑i,j
W 2i,j(Ai,j − A ′i,j)
2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 145: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/145.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPT
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 146: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/146.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 147: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/147.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10
in nO(k2r/ε) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 148: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/148.jpg)
Rank r. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
, r = 3
Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)
..
OPTwith prob. 9/10
in nO(k2r/ε) timepreviously only r=1 was known to be in polynomial time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 22 / 50
![Page 149: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/149.jpg)
Random-4SAT Hypothesis. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 23 / 50
![Page 150: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/150.jpg)
Random-4SAT Hypothesis. .
. .
[Feige’02, Goerdt-Lanka’04]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 23 / 50
![Page 151: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/151.jpg)
Random-4SAT Hypothesis. .
. .
[Feige’02, Goerdt-Lanka’04]
Given : random 4-SAT formula S on n variables
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 23 / 50
![Page 152: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/152.jpg)
Random-4SAT Hypothesis. .
. .
[Feige’02, Goerdt-Lanka’04]
Given : random 4-SAT formula S on n variableseach clause has 4 literals
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 23 / 50
![Page 153: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/153.jpg)
Random-4SAT Hypothesis. .
. .
[Feige’02, Goerdt-Lanka’04]
Given : random 4-SAT formula S on n variableseach clause has 4 literalseach of Θ(n4) clauses is picked ind. with prob. Θ(1/n3)m = Θ(n) is the number of clauses
(x4 ∨ x32 ∨ x9 ∨ x20)S1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 23 / 50
![Page 154: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/154.jpg)
Random-4SAT Hypothesis. .
. .
[Feige’02, Goerdt-Lanka’04]
Given : random 4-SAT formula S on n variableseach clause has 4 literalseach of Θ(n4) clauses is picked ind. with prob. Θ(1/n3)m = Θ(n) is the number of clauses
Any algorithm that :
(x4 ∨ x32 ∨ x9 ∨ x20)S1
(x2 ∨ x13 ∨ x5 ∨ x25)S2
· · · · · ·(x11 ∨ x19 ∨ x24 ∨ x6)Sm
S = S1 ∧ S2 ∧ · · ·∧ Sm
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 23 / 50
![Page 155: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/155.jpg)
Random-4SAT Hypothesis. .
. .
[Feige’02, Goerdt-Lanka’04]
Given : random 4-SAT formula S on n variableseach clause has 4 literalseach of Θ(n4) clauses is picked ind. with prob. Θ(1/n3)m = Θ(n) is the number of clauses
Any algorithm that :outputs 1 with prob. 1 when S is satisfiableoutputs 0 with prob. > 1/2probability is over the input instances
(x4 ∨ x32 ∨ x9 ∨ x20)S1
(x2 ∨ x13 ∨ x5 ∨ x25)S2
· · · · · ·(x11 ∨ x19 ∨ x24 ∨ x6)Sm
S = S1 ∧ S2 ∧ · · ·∧ Sm
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 23 / 50
![Page 156: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/156.jpg)
Random-4SAT Hypothesis. .
. .
[Feige’02, Goerdt-Lanka’04]
Given : random 4-SAT formula S on n variableseach clause has 4 literalseach of Θ(n4) clauses is picked ind. with prob. Θ(1/n3)m = Θ(n) is the number of clauses
Any algorithm that :outputs 1 with prob. 1 when S is satisfiableoutputs 0 with prob. > 1/2probability is over the input instances
Requires : 2Ω(n) time
(x4 ∨ x32 ∨ x9 ∨ x20)S1
(x2 ∨ x13 ∨ x5 ∨ x25)S2
· · · · · ·(x11 ∨ x19 ∨ x24 ∨ x6)Sm
S = S1 ∧ S2 ∧ · · ·∧ Sm
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 23 / 50
![Page 157: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/157.jpg)
Maximum Edge Biclique(MEB). .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 24 / 50
![Page 158: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/158.jpg)
Maximum Edge Biclique(MEB). .
. .
Given : A bipartite graph G = (U,V ,E) with |U | = |V | = n
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 24 / 50
![Page 159: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/159.jpg)
Maximum Edge Biclique(MEB). .
. .
Given : A bipartite graph G = (U,V ,E) with |U | = |V | = n
Output : A k1-by-k2 complete bipartite subgraph of G
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 24 / 50
![Page 160: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/160.jpg)
Maximum Edge Biclique(MEB). .
. .
Given : A bipartite graph G = (U,V ,E) with |U | = |V | = n
Output : A k1-by-k2 complete bipartite subgraph of Gs.t. k1 · k2 is maximized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 24 / 50
![Page 161: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/161.jpg)
Maximum Edge Biclique(MEB). .
. .
Given : A bipartite graph G = (U,V ,E) with |U | = |V | = n
Output : A k1-by-k2 complete bipartite subgraph of Gs.t. k1 · k2 is maximized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 24 / 50
![Page 162: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/162.jpg)
Maximum Edge Biclique(MEB). .
. .
Given : A bipartite graph G = (U,V ,E) with |U | = |V | = n
Output : A k1-by-k2 complete bipartite subgraph of Gs.t. k1 · k2 is maximized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 24 / 50
![Page 163: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/163.jpg)
Maximum Edge Biclique(MEB). .
. .
Given : A bipartite graph G = (U,V ,E) with |U | = |V | = n
Output : A k1-by-k2 complete bipartite subgraph of Gs.t. k1 · k2 is maximized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 24 / 50
![Page 164: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/164.jpg)
Maximum Edge Biclique(MEB). .
. .
Given : A bipartite graph G = (U,V ,E) with |U | = |V | = n
Output : A k1-by-k2 complete bipartite subgraph of Gs.t. k1 · k2 is maximized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 24 / 50
![Page 165: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/165.jpg)
Maximum Edge Biclique(MEB). .
. .
[Goerdt-Lanka’04]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 25 / 50
![Page 166: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/166.jpg)
Maximum Edge Biclique(MEB). .
. .
[Goerdt-Lanka’04]
Assume : Random 4-SAT hardness hypothesis
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 25 / 50
![Page 167: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/167.jpg)
Maximum Edge Biclique(MEB). .
. .
[Goerdt-Lanka’04]
Assume : Random 4-SAT hardness hypothesis
there exist two constants ε1 > ε2 > 0 such that
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 25 / 50
![Page 168: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/168.jpg)
Maximum Edge Biclique(MEB). .
. .
[Goerdt-Lanka’04]
Assume : Random 4-SAT hardness hypothesis
there exist two constants ε1 > ε2 > 0 such that
Any algorithm that : distinguishes between
bipartite graphs G = (U,V ,E) with |U | = |V | = n in two cases
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 25 / 50
![Page 169: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/169.jpg)
Maximum Edge Biclique(MEB). .
. .
[Goerdt-Lanka’04]
Assume : Random 4-SAT hardness hypothesis
there exist two constants ε1 > ε2 > 0 such that
Any algorithm that : distinguishes between
bipartite graphs G = (U,V ,E) with |U | = |V | = n in two cases
1. there is a clique of size > (n/16)2(1 + ε1)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 25 / 50
![Page 170: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/170.jpg)
Maximum Edge Biclique(MEB). .
. .
[Goerdt-Lanka’04]
Assume : Random 4-SAT hardness hypothesis
there exist two constants ε1 > ε2 > 0 such that
Any algorithm that : distinguishes between
bipartite graphs G = (U,V ,E) with |U | = |V | = n in two cases
1. there is a clique of size > (n/16)2(1 + ε1)
2. all bipartite cliques are of size 6 (n/16)2(1 + ε2)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 25 / 50
![Page 171: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/171.jpg)
Maximum Edge Biclique(MEB). .
. .
[Goerdt-Lanka’04]
Assume : Random 4-SAT hardness hypothesis
there exist two constants ε1 > ε2 > 0 such that
Any algorithm that : distinguishes between
bipartite graphs G = (U,V ,E) with |U | = |V | = n in two cases
1. there is a clique of size > (n/16)2(1 + ε1)
2. all bipartite cliques are of size 6 (n/16)2(1 + ε2)
Requires :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 25 / 50
![Page 172: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/172.jpg)
Maximum Edge Biclique(MEB). .
. .
[Goerdt-Lanka’04]
Assume : Random 4-SAT hardness hypothesis
there exist two constants ε1 > ε2 > 0 such that
Any algorithm that : distinguishes between
bipartite graphs G = (U,V ,E) with |U | = |V | = n in two cases
1. there is a clique of size > (n/16)2(1 + ε1)
2. all bipartite cliques are of size 6 (n/16)2(1 + ε2)
Requires : 2Ω(n) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 25 / 50
![Page 173: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/173.jpg)
Reduction. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 174: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/174.jpg)
Reduction. .
. .
MEB
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 175: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/175.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = n
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 176: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/176.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 177: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/177.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized
( complement ) |E |− k1k2 is minimized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 178: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/178.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized
( complement ) |E |− k1k2 is minimized
WLRA
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 179: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/179.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized
( complement ) |E |− k1k2 is minimized
WLRA Given : A ∈ 0,1n×n,W ∈ Rn×n
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 180: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/180.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized
( complement ) |E |− k1k2 is minimized
WLRA Given : A ∈ 0,1n×n,W ∈ Rn×n
Output : u, v ∈ Rn s.t. ‖W (uv> − A)‖2F is minimized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 181: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/181.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized
( complement ) |E |− k1k2 is minimized
WLRA Given : A ∈ 0,1n×n,W ∈ Rn×n
Output : u, v ∈ Rn s.t. ‖W (uv> − A)‖2F is minimized
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 182: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/182.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized
( complement ) |E |− k1k2 is minimized
WLRA Given : A ∈ 0,1n×n,W ∈ Rn×n
Output : u, v ∈ Rn s.t. ‖W (uv> − A)‖2F is minimized
Ai,j =
10
if edge (Ui ,Vj) exists
otherwise
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 183: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/183.jpg)
Reduction. .
. .
MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized
( complement ) |E |− k1k2 is minimized
WLRA Given : A ∈ 0,1n×n,W ∈ Rn×n
Output : u, v ∈ Rn s.t. ‖W (uv> − A)‖2F is minimized
Ai,j =
10
if edge (Ui ,Vj) exists
otherwiseW i,j =
1n6
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 26 / 50
![Page 184: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/184.jpg)
Main Results - Hardness. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 27 / 50
![Page 185: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/185.jpg)
Main Results - Hardness. .
. .
Weighted Low Rank Approximation Hardness
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 27 / 50
![Page 186: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/186.jpg)
Main Results - Hardness. .
. .
Weighted Low Rank Approximation Hardness
Given : A ∈ Rn×n, distinct r columns W ∈ Rn×n
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 27 / 50
![Page 187: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/187.jpg)
Main Results - Hardness. .
. .
Weighted Low Rank Approximation Hardness
Given : A ∈ Rn×n, distinct r columns W ∈ Rn×n
k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 27 / 50
![Page 188: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/188.jpg)
Main Results - Hardness. .
. .
Weighted Low Rank Approximation Hardness
Given : A ∈ Rn×n, distinct r columns W ∈ Rn×n
k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTwith prob. 9/10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 27 / 50
![Page 189: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/189.jpg)
Main Results - Hardness. .
. .
Weighted Low Rank Approximation Hardness
Given : A ∈ Rn×n, distinct r columns W ∈ Rn×n
k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTwith prob. 9/10
Assume : Random-4SAT Hypothesis[Feige’02, Goerdt-Lanka’04]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 27 / 50
![Page 190: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/190.jpg)
Main Results - Hardness. .
. .
Weighted Low Rank Approximation Hardness
Given : A ∈ Rn×n, distinct r columns W ∈ Rn×n
k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTwith prob. 9/10
Assume : Random-4SAT Hypothesis[Feige’02, Goerdt-Lanka’04]
∃ε0 > 0, for any algorithm with ε < ε0 and k > 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 27 / 50
![Page 191: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/191.jpg)
Main Results - Hardness. .
. .
Weighted Low Rank Approximation Hardness
Given : A ∈ Rn×n, distinct r columns W ∈ Rn×n
k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTwith prob. 9/10
Assume : Random-4SAT Hypothesis[Feige’02, Goerdt-Lanka’04]
∃ε0 > 0, for any algorithm with ε < ε0 and k > 1Requires : 2Ω(r) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 27 / 50
![Page 192: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/192.jpg)
Algorithmic Techniques
Tools
I Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch
Algorithm, assuming a rank r weight matrix W
I Warmup, inefficient WLRA algorithmI “Guessing a sketch”
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 28 / 50
![Page 193: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/193.jpg)
Algorithmic Techniques
ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]
I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch
Algorithm, assuming a rank r weight matrix W
I Warmup, inefficient WLRA algorithmI “Guessing a sketch”
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 28 / 50
![Page 194: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/194.jpg)
Algorithmic Techniques
ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]
I Multiple regression sketchAlgorithm, assuming a rank r weight matrix W
I Warmup, inefficient WLRA algorithmI “Guessing a sketch”
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 28 / 50
![Page 195: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/195.jpg)
Algorithmic Techniques
ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch
Algorithm, assuming a rank r weight matrix W
I Warmup, inefficient WLRA algorithmI “Guessing a sketch”
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 28 / 50
![Page 196: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/196.jpg)
Algorithmic Techniques
ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch
Algorithm, assuming a rank r weight matrix W
I Warmup, inefficient WLRA algorithmI “Guessing a sketch”
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 28 / 50
![Page 197: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/197.jpg)
Algorithmic Techniques
ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch
Algorithm, assuming a rank r weight matrix WI Warmup, inefficient WLRA algorithm
I “Guessing a sketch”
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 28 / 50
![Page 198: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/198.jpg)
Algorithmic Techniques
ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch
Algorithm, assuming a rank r weight matrix WI Warmup, inefficient WLRA algorithmI “Guessing a sketch”
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 28 / 50
![Page 199: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/199.jpg)
Polynomial System Verifier (Recall)
. .
. .
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 29 / 50
![Page 200: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/200.jpg)
Polynomial System Verifier (Recall)
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 29 / 50
![Page 201: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/201.jpg)
Polynomial System Verifier (Recall)
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a real polynomial system P(x)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 29 / 50
![Page 202: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/202.jpg)
Polynomial System Verifier (Recall)
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a real polynomial system P(x)v : #variables, x = (x1, x2, · · · , xv )
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 29 / 50
![Page 203: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/203.jpg)
Polynomial System Verifier (Recall)
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a real polynomial system P(x)v : #variables, x = (x1, x2, · · · , xv )
m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 29 / 50
![Page 204: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/204.jpg)
Polynomial System Verifier (Recall)
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a real polynomial system P(x)v : #variables, x = (x1, x2, · · · , xv )
m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomial constraints
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 29 / 50
![Page 205: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/205.jpg)
Polynomial System Verifier (Recall)
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a real polynomial system P(x)v : #variables, x = (x1, x2, · · · , xv )
m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomial constraintsH : the bitsizes of the coefficients of the polynomials
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 29 / 50
![Page 206: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/206.jpg)
Polynomial System Verifier (Recall)
. .
. .[Renegar’92, Basu-Pollack-Roy’96]
Given : a real polynomial system P(x)v : #variables, x = (x1, x2, · · · , xv )
m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomial constraintsH : the bitsizes of the coefficients of the polynomials
It takes (md)O(v) poly(H) time todecide if there exists a solution to polynomial system P
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 29 / 50
![Page 207: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/207.jpg)
Lower Bound on the Cost
. .
. .
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 208: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/208.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 209: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/209.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 210: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/210.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 211: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/211.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 212: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/212.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomials
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 213: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/213.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomialsH : the bitsizes of the coefficients of the polynomials
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 214: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/214.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomialsH : the bitsizes of the coefficients of the polynomials
.
Rv
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 215: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/215.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomialsH : the bitsizes of the coefficients of the polynomials
.
Rv
.
T : x ∈ Rv |f1(x) > 0, · · · , fm(x) > 0
T
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 216: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/216.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomialsH : the bitsizes of the coefficients of the polynomials
.
Rv
.
T : x ∈ Rv |f1(x) > 0, · · · , fm(x) > 0
T
.
minimum value that nonnegative g takes over T ∩ Ball is
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 217: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/217.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomialsH : the bitsizes of the coefficients of the polynomials
.
Rv
.
T : x ∈ Rv |f1(x) > 0, · · · , fm(x) > 0
T
.
minimum value that nonnegative g takes over T ∩ Ball is
Ball
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 218: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/218.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomialsH : the bitsizes of the coefficients of the polynomials
.
Rv
.
T : x ∈ Rv |f1(x) > 0, · · · , fm(x) > 0
T
.
minimum value that nonnegative g takes over T ∩ Ball is
Ball
either 0 or > (2H + m)−dv
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 219: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/219.jpg)
Lower Bound on the Cost
. .
. .[Jeronimo-Perrucci-Tsigaridas’13]
Given: a real polynomial system P(x)
v : #variables, x = (x1, x2, · · · , xv )
m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]
d : maximum degree of all polynomialsH : the bitsizes of the coefficients of the polynomials
.
Rv
.
T : x ∈ Rv |f1(x) > 0, · · · , fm(x) > 0
T
.
minimum value that nonnegative g takes over T ∩ Ball is
Ball
either 0 or > (2H + m)−dv
g = (x1x2 − 1)2 + x22
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 30 / 50
![Page 220: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/220.jpg)
Multiple Regression Sketch. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 221: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/221.jpg)
Multiple Regression Sketch. .
. .
Given :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 222: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/222.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 223: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/223.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 224: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/224.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Let x(j) = arg minx∈Rk×1
‖A(j)x − b(j)‖,∀j ∈ [m]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 225: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/225.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Let x(j) = arg minx∈Rk×1
‖A(j)x − b(j)‖,∀j ∈ [m]
Choose : S to be a random Gaussian matrix
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 226: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/226.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Let x(j) = arg minx∈Rk×1
‖A(j)x − b(j)‖,∀j ∈ [m]
Choose : S to be a random Gaussian matrixDenote y(j) = arg min
y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 227: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/227.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Let x(j) = arg minx∈Rk×1
‖A(j)x − b(j)‖,∀j ∈ [m]
Choose : S to be a random Gaussian matrixDenote y(j) = arg min
y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]
Gaurantee :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 228: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/228.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Let x(j) = arg minx∈Rk×1
‖A(j)x − b(j)‖,∀j ∈ [m]
Choose : S to be a random Gaussian matrixDenote y(j) = arg min
y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]
Gaurantee : For all ε ∈ (0,1/2), one can set t = O(k/ε) such that
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 229: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/229.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Let x(j) = arg minx∈Rk×1
‖A(j)x − b(j)‖,∀j ∈ [m]
Choose : S to be a random Gaussian matrixDenote y(j) = arg min
y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]
Gaurantee : For all ε ∈ (0,1/2), one can set t = O(k/ε) such thatm∑
j=1‖A(j)y(j) − b(j)‖22 6 (1 + ε)
m∑j=1‖A(j)x(j) − b(j)‖22
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 230: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/230.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Let x(j) = arg minx∈Rk×1
‖A(j)x − b(j)‖,∀j ∈ [m]
Choose : S to be a random Gaussian matrixDenote y(j) = arg min
y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]
Gaurantee : For all ε ∈ (0,1/2), one can set t = O(k/ε) such thatm∑
j=1‖A(j)y(j) − b(j)‖22 6 (1 + ε)
m∑j=1‖A(j)x(j) − b(j)‖22
with prob. 9/10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 231: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/231.jpg)
Multiple Regression Sketch. .
. .
Given : A(1),A(2), · · · ,A(m) ∈ Rn×k
b(1),b(2), · · · ,b(m) ∈ Rn×1
Let x(j) = arg minx∈Rk×1
‖A(j)x − b(j)‖,∀j ∈ [m]
Choose : S to be a random Gaussian matrixDenote y(j) = arg min
y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]
Gaurantee : For all ε ∈ (0,1/2), one can set t = O(k/ε) such thatm∑
j=1‖A(j)y(j) − b(j)‖22 6 (1 + ε)
m∑j=1‖A(j)x(j) − b(j)‖22
with prob. 9/10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 31 / 50
![Page 232: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/232.jpg)
Warmup, inefficient WLRA Algorithm. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 233: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/233.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 234: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/234.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
.
Aij ∈ 0,±1,±2, · · · ,±∆
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 235: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/235.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
W ij ∈ 0,1,2, · · · ,∆
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 236: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/236.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPT
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 237: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/237.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 238: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/238.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 239: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/239.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
2. write polynomial g(x1, · · · , x2nk ) = ‖W (A − UV )‖2F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 240: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/240.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
2. write polynomial g(x1, · · · , x2nk ) = ‖W (A − UV )‖2F3. pick C ∈ [L−,L+], run polynomial verifier g(x) 6 C
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 241: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/241.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
2. write polynomial g(x1, · · · , x2nk ) = ‖W (A − UV )‖2F3. pick C ∈ [L−,L+], run polynomial verifier g(x) 6 C4. optimize C by binary search over [L−,L+]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 242: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/242.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
2. write polynomial g(x1, · · · , x2nk ) = ‖W (A − UV )‖2F3. pick C ∈ [L−,L+], run polynomial verifier g(x) 6 C4. optimize C by binary search over [L−,L+]
Time : 2Ω(nk)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 243: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/243.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
2. write polynomial g(x1, · · · , x2nk ) = ‖W (A − UV )‖2F3. pick C ∈ [L−,L+], run polynomial verifier g(x) 6 C4. optimize C by binary search over [L−,L+]
Time : 2Ω(nk)
How can we do better?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 244: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/244.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
Time : 2Ω(nk)
How can we do better?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 245: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/245.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
Time : 2Ω(nk)
How can we do better?polynomial verifier runs in (# constraints · degree)O(# variables)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 246: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/246.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
Time : 2Ω(nk)
How can we do better?polynomial verifier runs in (# constraints · degree)O(# variables)
lower bound on cost (# constraints)−degreeO(# variables)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 247: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/247.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
Time : 2Ω(nk)
How can we do better?polynomial verifier runs in (# constraints · degree)O(# variables)
lower bound on cost (# constraints)−degreeO(# variables)
write a polynomial with few #variables, i.e. poly(kr/ε)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 248: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/248.jpg)
Warmup, inefficient WLRA Algorithm. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :
1. create 2nk variables for U,V> ∈ Rn×k
Time : 2Ω(nk)
How can we do better?polynomial verifier runs in (# constraints · degree)O(# variables)
lower bound on cost (# constraints)−degreeO(# variables)
write a polynomial with few #variables, i.e. poly(kr/ε)without blowing up degree and #constraints too much
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 32 / 50
![Page 249: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/249.jpg)
Main Idea
. .
. .
To reduce the number of variables to poly(kr/ε) :
1. Multiple regression sketch with O(k/ε) rows
2. Weight matrix W has rank at most r
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 33 / 50
![Page 250: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/250.jpg)
Guess a Sketch. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 251: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/251.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 252: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/252.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
.
Aij ∈ 0,±1,±2, · · · ,±∆
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 253: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/253.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
W ij ∈ 0,1,2, · · · ,∆
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 254: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/254.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 255: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/255.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of WDW j be diagonal matrix with vector W j
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 256: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/256.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of WDW j be diagonal matrix with vector W j
.
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
DW 1 =
10
10
10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 257: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/257.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of WDW j be diagonal matrix with vector W j
..
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
DW 2 =
11
10
00
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 258: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/258.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of WDW j be diagonal matrix with vector W j
...
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
DW 3 =
12
34
56
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 259: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/259.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of WDW j be diagonal matrix with vector W j
....
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
DW 4 =
21
20
10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 260: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/260.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of WDW j be diagonal matrix with vector W j
.....
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
DW 5 =
22
44
66
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 261: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/261.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of WDW j be diagonal matrix with vector W j
......
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
DW 6 =
23
44
56
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 262: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/262.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.
U
V
W A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 263: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/263.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.
U
V
W A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 264: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/264.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
U
V
W A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 265: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/265.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
U
V
W A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 266: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/266.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
U
V
W A )− (
2
F
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 267: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/267.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×n
S T
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 268: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/268.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 269: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/269.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 270: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/270.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 271: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/271.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
.
SDW 1U = S
U
10
10
10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 272: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/272.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
..
SDW 2U = S
U
11
10
00
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 273: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/273.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
...
SDW 3U = S
U
12
34
56
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 274: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/274.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
....
SDW 4U = S
U
21
20
10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 275: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/275.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
.....
SDW 5U = S
U
22
44
66
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 276: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/276.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
......
SDW 6U = S
U
23
44
56
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 277: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/277.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
......
Create t × k variables for each of n SDW j U s?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 278: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/278.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
......
Create t × k variables for each of n SDW j U s?
.
×t × kn
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 279: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/279.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
......
Create t × k variables for each of n SDW j U s?
.
×t × k
.
?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 280: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/280.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
..
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :
......
.n∑j=1‖DW j UV j − DW j Aj‖22
n∑i=1‖U iVDW i − AiDW i‖22
Sketch by Gaussian S,T> ∈ Rt×nn∑
j=1‖SDW j UV j − SDW j Aj‖22
n∑i=1‖U iVDW i T − AiDW i T‖22
Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t
......
Create t × k variables for each of n SDW j U s?
.
×t × k
..
r
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 34 / 50
![Page 281: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/281.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of W
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 282: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/282.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of W.
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
1 1 10 1 21 1 30 0 41 0 50 0 6
column span of W
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 283: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/283.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of W..
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
101010
=
101010
W 1 = W 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 284: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/284.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of W...
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
111000
=
111000
W 2 = W 2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 285: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/285.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of W....
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
123456
=
123456
W 3 = W 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 286: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/286.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of W.....
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
212010
= +
101010
111000
W 4 = W 1 + W 2
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 287: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/287.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of W......
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
224466
= +
101010
123456
W 5 = W 1 + W 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 288: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/288.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm : W j be j th column of W.......
W =
1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6
234456
= +
111000
123456
W 6 = W 2 + W 3
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 289: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/289.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 290: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/290.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 291: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/291.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 292: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/292.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
.
S
U
10
10
10
create t × k variables for SDW 1U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 293: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/293.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
..
S
U
11
10
00
create t × k variables for SDW 2U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 294: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/294.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
...
S
U
12
34
56
create t × k variables for SDW 3U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 295: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/295.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
....
S
U
21
20
10
write SDW 4U as SDW 1U + SDW 2U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 296: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/296.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
.....
S
U
1 + 10 + 1
1 + 10 + 0
1 + 00 + 0
write SDW 4U as SDW 1U + SDW 2U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 297: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/297.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
......
S
U(
10
10
10
+
11
10
00
)
write SDW 4U as SDW 1U + SDW 2U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 298: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/298.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
.......
S
U
22
44
66
write SDW 5U as SDW 1U + SDW 3U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 299: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/299.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
........
S
U
1 + 10 + 2
1 + 30 + 4
1 + 50 + 6
write SDW 5U as SDW 1U + SDW 3U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 300: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/300.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
.........
S
U(
10
10
10
+
12
34
56
)
write SDW 5U as SDW 1U + SDW 3U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 301: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/301.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
.........
S
U
23
44
56
write SDW 6U as SDW 2U + SDW 3U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 302: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/302.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
.........
S
U
1 + 11 + 2
1 + 30 + 4
0 + 50 + 6
write SDW 6U as SDW 2U + SDW 3U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 303: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/303.jpg)
Guess a Sketch. .
. .
Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0
Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT
Algorithm :....... n∑
j=1‖DW j UV j − DW j Aj‖22
sketchn∑
j=1‖SDW j UV j − SDW j Aj‖22
1 + ε
create variables for SDW j U, ∀j ∈ [r ]
.........
S
U(
11
10
00
+
12
34
56
)
write SDW 6U as SDW 2U + SDW 3U
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 35 / 50
![Page 304: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/304.jpg)
Wrapping Up
Can use an explicit formula for the regression solution in terms ofthe variables created
The regression solution is a rational function, but can use tricks toclear the denominatorMultiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variablesTime : nO(rk2/ε)
(# constraints · degree)O(# variables) = (O(n) ·O(nk))O(krt)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 36 / 50
![Page 305: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/305.jpg)
Wrapping Up
Can use an explicit formula for the regression solution in terms ofthe variables createdThe regression solution is a rational function, but can use tricks toclear the denominator
Multiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variablesTime : nO(rk2/ε)
(# constraints · degree)O(# variables) = (O(n) ·O(nk))O(krt)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 36 / 50
![Page 306: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/306.jpg)
Wrapping Up
Can use an explicit formula for the regression solution in terms ofthe variables createdThe regression solution is a rational function, but can use tricks toclear the denominatorMultiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variables
Time : nO(rk2/ε)
(# constraints · degree)O(# variables) = (O(n) ·O(nk))O(krt)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 36 / 50
![Page 307: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/307.jpg)
Wrapping Up
Can use an explicit formula for the regression solution in terms ofthe variables createdThe regression solution is a rational function, but can use tricks toclear the denominatorMultiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variablesTime : nO(rk2/ε)
(# constraints · degree)O(# variables) = (O(n) ·O(nk))O(krt)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 36 / 50
![Page 308: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/308.jpg)
Wrapping Up
Can use an explicit formula for the regression solution in terms ofthe variables createdThe regression solution is a rational function, but can use tricks toclear the denominatorMultiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variablesTime : nO(rk2/ε)
(# constraints · degree)O(# variables) = (O(n) ·O(nk))O(krt)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 36 / 50
![Page 309: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/309.jpg)
Open Problems
For a rank-r weight matrix W , the upper bound is nO(k2r/ε) but thelower bound is only 2Ω(r) - can we close this gap?
Can we prove a hardness result with respect to the parameter k ,e.g., a 2Ω(k) lower bound for WLRA problem?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 37 / 50
![Page 310: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/310.jpg)
Open Problems
For a rank-r weight matrix W , the upper bound is nO(k2r/ε) but thelower bound is only 2Ω(r) - can we close this gap?Can we prove a hardness result with respect to the parameter k ,e.g., a 2Ω(k) lower bound for WLRA problem?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 37 / 50
![Page 311: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/311.jpg)
. .
. .
`1 Low Rank Approximation
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 38 / 50
![Page 312: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/312.jpg)
`1-Low Rank Approximation. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 39 / 50
![Page 313: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/313.jpg)
`1-Low Rank Approximation. .
. .
Given : A ∈ Rn×n, k ∈ N, α > 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 39 / 50
![Page 314: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/314.jpg)
`1-Low Rank Approximation. .
. .
Given : A ∈ Rn×n, k ∈ N, α > 1
Output :
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 39 / 50
![Page 315: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/315.jpg)
`1-Low Rank Approximation. .
. .
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : rank-k A ∈ Rn×n s.t.
A A−
1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 39 / 50
![Page 316: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/316.jpg)
`1-Low Rank Approximation. .
. .
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : rank-k A ∈ Rn×n s.t.‖A − A‖1 6 α · min
rank−k A ′‖A ′ − A‖1
A A−
1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 39 / 50
![Page 317: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/317.jpg)
`1-Low Rank Approximation. .
. .
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : rank-k A ∈ Rn×n s.t.‖A − A‖1 6 α · min
rank−k A ′‖A ′ − A‖1
.
∑i,j|A ′i,j − Ai,j |
A A−
1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 39 / 50
![Page 318: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/318.jpg)
`1-Low Rank Approximation. .
. .
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : rank-k A ∈ Rn×n s.t.‖A − A‖1 6 α · min
rank−k A ′‖A ′ − A‖1
.
∑i,j|A ′i,j − Ai,j |
.OPT
A A−
1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 39 / 50
![Page 319: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/319.jpg)
`1-Low Rank Approximation. .
. .
Given : A ∈ Rn×n, k ∈ N, α > 1
Output :
..
U ∈ Rn×k ,V ∈ Rk×n s.t.‖UV − A‖1 6 α · OPT
U
V
A−
1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 39 / 50
![Page 320: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/320.jpg)
Why is `1-low Rank Approximation Interesting?
The problem was shown to be NP-hard by Gillis-Vavasis’15 and anumber of heuristics have been proposed
Asked in several places if there are any approximation algorithms
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 40 / 50
![Page 321: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/321.jpg)
Why is `1-low Rank Approximation Interesting?
The problem was shown to be NP-hard by Gillis-Vavasis’15 and anumber of heuristics have been proposedAsked in several places if there are any approximation algorithms
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 40 / 50
![Page 322: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/322.jpg)
Thought Experiments. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 41 / 50
![Page 323: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/323.jpg)
Thought Experiments. .
. .
Given : A set of points in 2-dimensions
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 41 / 50
![Page 324: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/324.jpg)
Thought Experiments. .
. .
Given : A set of points in 2-dimensions
Goal : Find a line to fit those points
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 41 / 50
![Page 325: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/325.jpg)
Thought Experiments. .
. .
Given : A set of points in 2-dimensions
Goal : Find a line to fit those points
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 41 / 50
![Page 326: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/326.jpg)
Thought Experiments. .
. .
Given : A set of points in 2-dimensions
Goal : Find a line to fit those points
.
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 41 / 50
![Page 327: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/327.jpg)
Thought Experiments. .
. .
Given : A set of points in 2-dimensions
Goal : Find a line to fit those points
.
Output : `2 minimizer line,
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 41 / 50
![Page 328: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/328.jpg)
Thought Experiments. .
. .
Given : A set of points in 2-dimensions
Goal : Find a line to fit those points
.
Output : `2 minimizer line, `1 minimizer line
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 41 / 50
![Page 329: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/329.jpg)
Thought Experiments. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 42 / 50
![Page 330: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/330.jpg)
Thought Experiments. .
. .
Given : k = 1 and A =
4 0 0 00 1 1 10 1 1 10 1 1 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 42 / 50
![Page 331: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/331.jpg)
Thought Experiments. .
. .
Given : k = 1 and A =
4 0 0 00 1 1 10 1 1 10 1 1 1
Output : rank-1 ‖‖F−solution A =
4 0 0 00 0 0 00 0 0 00 0 0 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 42 / 50
![Page 332: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/332.jpg)
Thought Experiments. .
. .
Given : k = 1 and A =
4 0 0 00 1 1 10 1 1 10 1 1 1
Output : rank-1 ‖‖F−solution A =
4 0 0 00 0 0 00 0 0 00 0 0 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 42 / 50
![Page 333: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/333.jpg)
Thought Experiments. .
. .
Given : k = 1 and A =
4 0 0 00 1 1 10 1 1 10 1 1 1
Output : rank-1 ‖‖F−solution A =
4 0 0 00 0 0 00 0 0 00 0 0 0
Output : rank-1 ‖‖1−solution A =
0 0 0 00 1 1 10 1 1 10 1 1 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 42 / 50
![Page 334: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/334.jpg)
Thought Experiments. .
. .
Given : k = 1 and A =
4 0 0 00 1 1 10 1 1 10 1 1 1
Output : rank-1 ‖‖F−solution A =
4 0 0 00 0 0 00 0 0 00 0 0 0
Output : rank-1 ‖‖1−solution A =
0 0 0 00 1 1 10 1 1 10 1 1 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 42 / 50
![Page 335: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/335.jpg)
Heuristics and Nearly `1 Setting
`1-setting
I Heuristic algorithms Ke-Kanade’05, Ding-Zhou-He-Zha’06,Kwak’08, Brooks-Dula-Boone’13
I Robust PCA Candes-Li-Ma-Wright’11,Wright-Ganesh-Rao-Peng-Ma’09,Netrapalli-Niranjan-Sanghavi-Anandkumar-Jain’14,Yi-Park-Chen-Caramanis’16
None of them can achieve better than poly(n) approximation ratio
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 43 / 50
![Page 336: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/336.jpg)
Heuristics and Nearly `1 Setting
`1-settingI Heuristic algorithms Ke-Kanade’05, Ding-Zhou-He-Zha’06,
Kwak’08, Brooks-Dula-Boone’13
I Robust PCA Candes-Li-Ma-Wright’11,Wright-Ganesh-Rao-Peng-Ma’09,Netrapalli-Niranjan-Sanghavi-Anandkumar-Jain’14,Yi-Park-Chen-Caramanis’16
None of them can achieve better than poly(n) approximation ratio
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 43 / 50
![Page 337: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/337.jpg)
Heuristics and Nearly `1 Setting
`1-settingI Heuristic algorithms Ke-Kanade’05, Ding-Zhou-He-Zha’06,
Kwak’08, Brooks-Dula-Boone’13I Robust PCA Candes-Li-Ma-Wright’11,
Wright-Ganesh-Rao-Peng-Ma’09,Netrapalli-Niranjan-Sanghavi-Anandkumar-Jain’14,Yi-Park-Chen-Caramanis’16
None of them can achieve better than poly(n) approximation ratio
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 43 / 50
![Page 338: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/338.jpg)
Heuristics and Nearly `1 Setting
`1-settingI Heuristic algorithms Ke-Kanade’05, Ding-Zhou-He-Zha’06,
Kwak’08, Brooks-Dula-Boone’13I Robust PCA Candes-Li-Ma-Wright’11,
Wright-Ganesh-Rao-Peng-Ma’09,Netrapalli-Niranjan-Sanghavi-Anandkumar-Jain’14,Yi-Park-Chen-Caramanis’16
None of them can achieve better than poly(n) approximation ratio
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 43 / 50
![Page 339: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/339.jpg)
Main Question, `1 Low Rank Approximation
QuestionGiven matrix A ∈ Rn×n, is there an algorithm that is able to output arank-k matrix A such that
‖A − A‖1 6 α · minrank−k A ′
‖A ′ − A‖1?
Or, are there any inapproximability results?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 44 / 50
![Page 340: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/340.jpg)
Main Results - Algorithms
Upper bound (Algorithm) for an arbitrary n × n matrix A
I poly(k) log n-approximation, poly(n) timeI O(k)-approximation, nO(k)2O(k2) time.I Bicriteria algorithm
F rank-2k O(1)-approximation, nO(k2) running time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 45 / 50
![Page 341: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/341.jpg)
Main Results - Algorithms
Upper bound (Algorithm) for an arbitrary n × n matrix AI poly(k) log n-approximation, poly(n) time
I O(k)-approximation, nO(k)2O(k2) time.I Bicriteria algorithm
F rank-2k O(1)-approximation, nO(k2) running time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 45 / 50
![Page 342: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/342.jpg)
Main Results - Algorithms
Upper bound (Algorithm) for an arbitrary n × n matrix AI poly(k) log n-approximation, poly(n) timeI O(k)-approximation, nO(k)2O(k2) time.
I Bicriteria algorithm
F rank-2k O(1)-approximation, nO(k2) running time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 45 / 50
![Page 343: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/343.jpg)
Main Results - Algorithms
Upper bound (Algorithm) for an arbitrary n × n matrix AI poly(k) log n-approximation, poly(n) timeI O(k)-approximation, nO(k)2O(k2) time.I Bicriteria algorithm
F rank-2k O(1)-approximation, nO(k2) running time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 45 / 50
![Page 344: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/344.jpg)
Main Results - Algorithms
Upper bound (Algorithm) for an arbitrary n × n matrix AI poly(k) log n-approximation, poly(n) timeI O(k)-approximation, nO(k)2O(k2) time.I Bicriteria algorithm
F rank-2k O(1)-approximation, nO(k2) running time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 45 / 50
![Page 345: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/345.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 346: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/346.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
`1-Low Rank Approximation Hardness
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 347: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/347.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
`1-Low Rank Approximation Hardness
Given : A ∈ Rn×n, k ∈ N, α > 1
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 348: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/348.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
`1-Low Rank Approximation Hardness
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : U ∈ Rn×k , V ∈ Rk×n s.t. ‖UV − A‖1 6 α ·OPTwith prob. 9/10
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 349: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/349.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
`1-Low Rank Approximation Hardness
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : U ∈ Rn×k , V ∈ Rk×n s.t. ‖UV − A‖1 6 α ·OPTwith prob. 9/10
Assume : Exponential Time Hypothesis(ETH)[Impagliazzo-Paturi-Zane’98]
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 350: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/350.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
`1-Low Rank Approximation Hardness
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : U ∈ Rn×k , V ∈ Rk×n s.t. ‖UV − A‖1 6 α ·OPTwith prob. 9/10
Assume : Exponential Time Hypothesis(ETH)[Impagliazzo-Paturi-Zane’98]
for arbitrarily small constant γ > 0
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 351: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/351.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
`1-Low Rank Approximation Hardness
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : U ∈ Rn×k , V ∈ Rk×n s.t. ‖UV − A‖1 6 α ·OPTwith prob. 9/10
Assume : Exponential Time Hypothesis(ETH)[Impagliazzo-Paturi-Zane’98]
for arbitrarily small constant γ > 0for any algorithm running in nO(1) time
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 352: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/352.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
`1-Low Rank Approximation Hardness
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : U ∈ Rn×k , V ∈ Rk×n s.t. ‖UV − A‖1 6 α ·OPTwith prob. 9/10
Assume : Exponential Time Hypothesis(ETH)[Impagliazzo-Paturi-Zane’98]
for arbitrarily small constant γ > 0for any algorithm running in nO(1) time
Requires : α > (1 + 1log1+γ n
)
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 353: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/353.jpg)
Hardness - Under Exponential Time Hypothesis. .
. .
`1-Low Rank Approximation Hardness
Given : A ∈ Rn×n, k ∈ N, α > 1
Output : U ∈ Rn×k , V ∈ Rk×n s.t. ‖UV − A‖1 6 α ·OPTwith prob. 9/10
Assume : Exponential Time Hypothesis(ETH)[Impagliazzo-Paturi-Zane’98]
for arbitrarily small constant γ > 0for any algorithm running in nO(1) time
Requires : α > (1 + 1log1+γ n
)
Implies a 2Ω(1/ε1−γ) Hardness
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 46 / 50
![Page 354: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/354.jpg)
Open Problems
The hardness result is for (1 + 1/ poly(log n)) approximation, canwe obtain a hardness result for O(1) approximation ?
We have an nO(k)-time O(k)-approximation algorithm, can eitherthe approximation or the time be improved?We have a poly(n)-time O(log n) · poly(k)-approximationalgorithm. Can the approximation be improved?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 47 / 50
![Page 355: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/355.jpg)
Open Problems
The hardness result is for (1 + 1/ poly(log n)) approximation, canwe obtain a hardness result for O(1) approximation ?
We have an nO(k)-time O(k)-approximation algorithm, can eitherthe approximation or the time be improved?
We have a poly(n)-time O(log n) · poly(k)-approximationalgorithm. Can the approximation be improved?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 47 / 50
![Page 356: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/356.jpg)
Open Problems
The hardness result is for (1 + 1/ poly(log n)) approximation, canwe obtain a hardness result for O(1) approximation ?
We have an nO(k)-time O(k)-approximation algorithm, can eitherthe approximation or the time be improved?We have a poly(n)-time O(log n) · poly(k)-approximationalgorithm. Can the approximation be improved?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 47 / 50
![Page 357: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/357.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation
Algorithm techniques
I polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variables
Hardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 358: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/358.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorization
I weighted low rank approximationI `1-low rank approximation
Algorithm techniques
I polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variables
Hardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 359: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/359.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximation
I `1-low rank approximationAlgorithm techniques
I polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variables
Hardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 360: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/360.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation
Algorithm techniques
I polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variables
Hardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 361: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/361.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation
Algorithm techniques
I polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variablesHardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 362: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/362.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation
Algorithm techniquesI polynomial system verifier
I random matrices for dimensionality reduction to reduce the numberof variables
Hardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 363: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/363.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation
Algorithm techniquesI polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variables
Hardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 364: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/364.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation
Algorithm techniquesI polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variablesHardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 365: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/365.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation
Algorithm techniquesI polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variablesHardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 366: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/366.jpg)
Conclusions
Studied intractable matrix factorization problems through the lensof parameterized complexity
I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation
Algorithm techniquesI polynomial system verifierI random matrices for dimensionality reduction to reduce the number
of variablesHardness techniques
I random 4-SAT, ETH, etc.
Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 48 / 50
![Page 367: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/367.jpg)
Thank you!. .
. .
Questions?
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 49 / 50
![Page 368: Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix Factorization David P. Woodruff IBM Almaden. .. . Based on joint works with Ilya Razenshteyn,](https://reader030.vdocuments.us/reader030/viewer/2022021618/5b780cda7f8b9a8f698e38c6/html5/thumbnails/368.jpg)
Cauchy Distribution. .
. .
Cauchy distribution C(µ,γ), pdf f (x) = 1πγ
γ2
(x−µ)2+γ
µ
1π
For any three independent random variablesX ,Y ,Z drawn from Cauchy distribution.For any a,b ∈ R, aX + bY ∼ (|a|+ |b|)Z
Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 50 / 50