simple and practical algorithm for the sparse fourier transform -...
TRANSCRIPT
Simple and Practical Algorithm for the SparseFourier Transform
Haitham Hassanieh Piotr Indyk Dina Katabi Eric Price
MIT
2012-01-19
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 1 / 19
Outline
1 Introduction
2 Algorithm
3 Experiments
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 2 / 19
The Dicrete Fourier Transform
Discrete Fourier transform: given x ∈ Cn, find
xi =∑
xjωij
Fundamental toolI Compression (audio, image, video)I Signal processingI Data analysisI ...
FFT: O(n log n) time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 4 / 19
Sparse Fourier Transform
Often the Fourier transform is dominated by a small number of“peaks”
I Precisely the reason to use for compression.
If most of mass in k locations, can we compute FFT faster?
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 5 / 19
Previous work
Boolean cube: [KM92], [GL89]. What about C?[Mansour-92]: kc logc n.Long list of other work [GGIMS02, AGS03, Iwen10, Aka10]Fastest is [Gilbert-Muthukrishnan-Strauss-05]: k log4 n.
I All have poor constants, many logs.I Need n/k > 40,000 or ω(log3 n) to beat FFTW.I Our goal: beat FFTW for smaller n/k in theory and practice.I Result: n/k > 2,000 or ω(log n) to beat FFTW.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 6 / 19
Our result
Simple, practical algorithm with good constants.Compute the k -sparse Fourier transform in O(
√kn log3/2 n) time.
Get x ′ with approximation error
‖x ′ − x‖2∞ ≤1k‖x − xk‖22
If x is sparse, recover it exactly.Caveats:
I Additional ‖x‖2/nΘ(1) error.I n must be a power of 2.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 7 / 19
Structure of this section
If x is k -sparse with known support S, find xS exactly inO(k log2 n) time.
In general, estimate x approximately in O(√
nk) time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 9 / 19
IntuitionOriginal signal (time) Original signal (freq)
Cutoff signal (time) Cutoff signal (freq)
Cutoff signal (time) Cutoff signal, subsampled (freq)
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 10 / 19
n-dimensional DFT
n-dimensional DFTof first B terms.
B-dimensional DFTof first B terms.
FrameworkCutoff signal (time) Cutoff subsampled signals (freq)
“Hashes” into B buckets in B log B time.Issues:
I “Hashing” needs a random hash functionF Access x ′t = ω−atxσt , so x ′t = xσ−1t+a [GMS-05]
I CollisionsF Have B > 4k , repeat O(log n) times and take median. [Count-Sketch,
CCF02]I LeakageI Finding the support. [Porat-Strauss-12], talk at 9:45am.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 11 / 19
LeakageCutoff signal (time) Cutoff, subsampled signals (freq)
Let Fi =
{1 i < B0 otherwise
be the “boxcar” filter. (Used in
[GGIMS02,GMS05])Observe
DFT(F ·x ,B) = subsample(DFT(F ·x ,n),B) = subsample(F∗x ,B).
DFT F of boxcar filter is sinc, decays as 1/i .Need a better filter F !
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 12 / 19
Filters
0 20 40 60 80 1000.0
0.5
1.0
1.5
2.0 Filter (time)
Bin0
5
10
15
20
25 Filter (freq)
Observe subsample(F ∗ x ,B) in O(B log B) time.Needs for F :
I supp(F ) ∈ [0,B]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].
Gaussians:I Standard deviation σ = B/
√log n
B ·√
log n
I DFT has σ = (n/B)√
log n
(n/B)/√
log n
I Nontrivial leakage into O(log n) buckets.I But likely trivial contribution to correct bucket.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 13 / 19
Filters
0 20 40 60 80 1000.0
0.2
0.4
0.6
0.8
1.0 Filter (time)
Bin0
2
4
6
8
10
12
14
16
18 Filter (freq)
Observe subsample(F ∗ x ,B) in O(B log B) time.Needs for F :
I supp(F ) ∈ [0,B]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].
Gaussians:I Standard deviation σ = B/
√log n
B ·√
log n
I DFT has σ = (n/B)√
log n
(n/B)/√
log n
I Nontrivial leakage into O(log n) buckets.I But likely trivial contribution to correct bucket.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 13 / 19
Filters
0 50 100 150 2000.0
0.2
0.4
0.6
0.8
1.0 Filter (time)
Bin0
10
20
30
40
50
60
70
80 Filter (freq)
Observe subsample(F ∗ x ,B) in O(B log B) time.Needs for F :
I supp(F ) ∈ [0,B log n]I |F | < δ = 1/nΘ(1) except “near” 0.I F ≈ 1 over [−n/2B,n/2B].
Gaussians:I Standard deviation σ =�����B/
√log n B ·
√log n
I DFT has σ =������(n/B)
√log n (n/B)/
√log n
I Nontrivial leakage into 0 buckets.I But likely trivial contribution to correct bucket.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 13 / 19
Filters
0 50 100 150 2000.0
0.2
0.4
0.6
0.8
1.0 Filter (time)
Bin0
10
20
30
40
50
60
70
80 Filter (freq)
Let G be Gaussian with σ = B√
log nH be box-car filter of length n/B.
Use F = G ∗ H.I F = G · H, so supp(F ) ⊂ [0,B log n].I |F | < 1/nΘ(1) outside −n/B,n/B.I |F | = 1± 1/nΘ(1) within n/2B,n/B.
Hashes correctly to one bucket, leaks to at most 1 bucket.Replace Gaussians with “Dolph-Chebyshev window functions”:factor 2 improvement.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 14 / 19
Filters
0 50 100 150 2000.02
0.00
0.02
0.04
0.06
0.08
0.10 Filter (time)
Bin0.0
0.2
0.4
0.6
0.8
1.0
1.2 Filter (freq)
Let G be Gaussian with σ = B√
log nH be box-car filter of length n/B.
Use F = G ∗ H.I F = G · H, so supp(F ) ⊂ [0,B log n].I |F | < 1/nΘ(1) outside −n/B,n/B.I |F | = 1± 1/nΘ(1) within n/2B,n/B.
Hashes correctly to one bucket, leaks to at most 1 bucket.Replace Gaussians with “Dolph-Chebyshev window functions”:factor 2 improvement.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 14 / 19
Algorithm to estimate xS
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
Algorithm to estimate xS
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
Algorithm to estimate xS
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
Algorithm to estimate xS
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
Algorithm in general
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.
To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
Algorithm in general
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.To find S: choose all that map to the top 2k values.
nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
Algorithm in general
For O(log n) different permutations of x , computesubsample(F ∗ x ,B).Estimate each xi as median of values it maps to.To find S: choose all that map to the top 2k values.nk/B candidates to update at each iteration: total
(nkB
+ B log n) log n =√
nk log3/2 n
time.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 15 / 19
Empirical Performance: runtime
0.001
0.01
0.1
1
10
214
215
216
217
218
219
220
221
222
223
224
225
226
Ru
n T
ime
(se
c)
Signal Size (n)
Run Time vs Signal Size (k=50)
sFFT 1.0
sFFT 2.0
FFTW
FFTW OPT
AAFFT 0.9
0.01
0.1
1
10
100 1000
Ru
n T
ime
(se
c)
Number of Non-Zero Frequencies (k)
Run Time vs Sparsity (n=222
)
sFFT 1.0
sFFT 2.0
FFTW
FFTW OPT
AAFFT 0.9
Compare to FFTW, previous best sublinear algorithm (AAFFT).
Offer a heuristic that improves time to O(n1/3k2/3).I Filter from [Mansour ’92].I Can’t rerandomize, might miss elements.
Faster than FFTW for n/k > 2,000.Faster than AAFFT for n/k < 1,000,000.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 17 / 19
Empirical Performance: noise
1e-07
1e-06
1e-05
0.0001
0.001
0.01
0.1
1
-20 0 20 40 60 80 100
Ave
rage
L1
Err
or p
er E
nrty
SNR (dB)
Robustness vs SNR (n=222, k=50)
sFFT 1.0
sFFT 2.0
AAFFT 0.9
Just like in Count-Sketch, algorithm is noise tolerant.
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 18 / 19
Conclusions
Roughly: fastest algorithm for n/k ∈ [2× 103,106].Recent improvements [HIKP12b?]
I O(k log n) for exactly sparse xI O(k log n
k log n) for approximation.I Beats FFTW for n/k > 400 (in the exact case).
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 19 / 19
Hassanieh, Indyk, Katabi, and Price (MIT) Simple and Practical Algorithm for the Sparse Fourier Transform 2012-01-19 20 / 19