Download - Fast algorithm of unconvex compressde sensin
![Page 1: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/1.jpg)
Fast algorithms for nonconvexcompressive sensing
Rick Chartrand
Los Alamos National Laboratory
New Mexico Consortium
September 2, 2009
Slide 1 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 2: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/2.jpg)
Outline
Motivating Example
Nonconvex compressive sensing
Examples
Fast algorithm
Summary
Slide 2 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 3: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/3.jpg)
Motivating Example
Motivating exampleSuppose we want to reconstruct animage from samples of its Fouriertransform. How many samples dowe need?
Shepp-Logan phantom, x
Consider radial sampling,such as in MRI or (roughly)CT.
Ω
Slide 3 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 4: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/4.jpg)
Motivating Example
Nonconvexity is better
Fewer measurements are needed with nonconvex minimization:
minu‖Du‖pp, subject to (Fu)|Ω = (Fx)|Ω.
With p = 1, solution is u = x with 18 lines ( |Ω||x| = 6.9%).
With p = 1/2, 10 lines suffice ( |Ω||x| = 3.8%). (More than 104500
local minima.)
backprojection, 18 lines p = 1, 18 lines p = 12
, 10 lines p = 1, 10 lines
Slide 4 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 5: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/5.jpg)
Motivating Example
Nonconvexity is better
Fewer measurements are needed with nonconvex minimization:
minu‖Du‖pp, subject to (Fu)|Ω = (Fx)|Ω.
With p = 1, solution is u = x with 18 lines ( |Ω||x| = 6.9%).
With p = 1/2, 10 lines suffice ( |Ω||x| = 3.8%). (More than 104500
local minima.)
backprojection, 18 lines p = 1, 18 lines
p = 12
, 10 lines p = 1, 10 lines
Slide 4 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 6: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/6.jpg)
Motivating Example
Nonconvexity is better
Fewer measurements are needed with nonconvex minimization:
minu‖Du‖pp, subject to (Fu)|Ω = (Fx)|Ω.
With p = 1, solution is u = x with 18 lines ( |Ω||x| = 6.9%).
With p = 1/2, 10 lines suffice ( |Ω||x| = 3.8%). (More than 104500
local minima.)
backprojection, 18 lines p = 1, 18 lines p = 12
, 10 lines p = 1, 10 lines
Slide 4 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 7: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/7.jpg)
Motivating Example
New results
These are old results (Mar. 2006); what’s new?
I Reconstruction (to 50 dB) in 13 seconds (in Matlab; versusliterature-best 1–3 minutes).
I Exact reconstruction from 9 lines (3.5% of Fourier transform).
10 lines fastest 10-line re-covery
9 lines recovery fromfewest samples
Slide 5 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 8: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/8.jpg)
Motivating Example
New results
These are old results (Mar. 2006); what’s new?I Reconstruction (to 50 dB) in 13 seconds (in Matlab; versus
literature-best 1–3 minutes).
I Exact reconstruction from 9 lines (3.5% of Fourier transform).
10 lines fastest 10-line re-covery
9 lines recovery fromfewest samples
Slide 5 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 9: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/9.jpg)
Motivating Example
New results
These are old results (Mar. 2006); what’s new?I Reconstruction (to 50 dB) in 13 seconds (in Matlab; versus
literature-best 1–3 minutes).I Exact reconstruction from 9 lines (3.5% of Fourier transform).
10 lines fastest 10-line re-covery
9 lines recovery fromfewest samples
Slide 5 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 10: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/10.jpg)
Nonconvex compressive sensing
Outline
Motivating Example
Nonconvex compressive sensing
Examples
Fast algorithm
Summary
Slide 6 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 11: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/11.jpg)
Nonconvex compressive sensing
What is compressive sensing?
Ψ
A b
=
x x′
=
x
, .
I Compressive sensing is the reconstruction of sparse signals xfrom surprisingly few incoherent measurements b = Ax.
I We suppose the existence of an operator or dictionary Ψ suchthat most of the components of Ψx are (nearly) zero.
Slide 7 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 12: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/12.jpg)
Nonconvex compressive sensing
What is compressive sensing?
Ψ
A b
=
x x′
=
x
, .
I Compressive sensing is the reconstruction of sparse signals xfrom surprisingly few incoherent measurements b = Ax.
I We suppose the existence of an operator or dictionary Ψ suchthat most of the components of Ψx are (nearly) zero.
Slide 7 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 13: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/13.jpg)
Nonconvex compressive sensing
What is compressive sensing?
Ψ
A b
=
x x′
=
x
, .
I An undersampled measurement Ax is tantamount to acompressed version of x. If x is sufficiently sparse, it can berecovered perfectly.
I We exploit the fact that sparsity is mathematically special, yet ageneral property of natural or human signals.
Slide 7 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 14: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/14.jpg)
Nonconvex compressive sensing
What is compressive sensing?
Ψ
A b
=
x x′
=
x
, .
I An undersampled measurement Ax is tantamount to acompressed version of x. If x is sufficiently sparse, it can berecovered perfectly.
I We exploit the fact that sparsity is mathematically special, yet ageneral property of natural or human signals.
Slide 7 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 15: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/15.jpg)
Nonconvex compressive sensing
Optimization for sparse recovery
I Let x ∈ RN be sparse: ‖Ψx‖0 = K, K N .
I Suppose A is an M ×N matrix, M N , with A and Ψincoherent. For example, A = (aij), i.i.d. aij ∼ N(0, σ2). Letb = Ax.
minu‖Ψu‖0, s.t. Au = b.
minu‖Ψu‖1, s.t. Au = b.
minu‖Ψu‖pp, s.t. Au = b,
Unique solution is u = x with opti-mally small M , but is NP-hard.M ≥ 2K suffices with probability 1.
Can be solved efficiently; requiresmore measurements for reconstruc-tion. M ≥ CK log(N/K)
where 0 < p < 1. Solvable in prac-tice; requires fewer measurementsthan `1.M ≥ C1(p)K+pC2(p)K log(N/K)(with V. Staneva)
Slide 8 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 16: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/16.jpg)
Nonconvex compressive sensing
Optimization for sparse recovery
I Let x ∈ RN be sparse: ‖Ψx‖0 = K, K N .I Suppose A is an M ×N matrix, M N , with A and Ψ
incoherent. For example, A = (aij), i.i.d. aij ∼ N(0, σ2). Letb = Ax.
minu‖Ψu‖0, s.t. Au = b.
minu‖Ψu‖1, s.t. Au = b.
minu‖Ψu‖pp, s.t. Au = b,
Unique solution is u = x with opti-mally small M , but is NP-hard.M ≥ 2K suffices with probability 1.
Can be solved efficiently; requiresmore measurements for reconstruc-tion. M ≥ CK log(N/K)
where 0 < p < 1. Solvable in prac-tice; requires fewer measurementsthan `1.M ≥ C1(p)K+pC2(p)K log(N/K)(with V. Staneva)
Slide 8 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 17: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/17.jpg)
Nonconvex compressive sensing
Optimization for sparse recovery
I Let x ∈ RN be sparse: ‖Ψx‖0 = K, K N .I Suppose A is an M ×N matrix, M N , with A and Ψ
incoherent. For example, A = (aij), i.i.d. aij ∼ N(0, σ2). Letb = Ax.
minu‖Ψu‖0, s.t. Au = b.
minu‖Ψu‖1, s.t. Au = b.
minu‖Ψu‖pp, s.t. Au = b,
Unique solution is u = x with opti-mally small M , but is NP-hard.M ≥ 2K suffices with probability 1.
Can be solved efficiently; requiresmore measurements for reconstruc-tion. M ≥ CK log(N/K)
where 0 < p < 1. Solvable in prac-tice; requires fewer measurementsthan `1.M ≥ C1(p)K+pC2(p)K log(N/K)(with V. Staneva)
Slide 8 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 18: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/18.jpg)
Nonconvex compressive sensing
Optimization for sparse recovery
I Let x ∈ RN be sparse: ‖Ψx‖0 = K, K N .I Suppose A is an M ×N matrix, M N , with A and Ψ
incoherent. For example, A = (aij), i.i.d. aij ∼ N(0, σ2). Letb = Ax.
minu‖Ψu‖0, s.t. Au = b.
minu‖Ψu‖1, s.t. Au = b.
minu‖Ψu‖pp, s.t. Au = b,
Unique solution is u = x with opti-mally small M , but is NP-hard.M ≥ 2K suffices with probability 1.
Can be solved efficiently; requiresmore measurements for reconstruc-tion. M ≥ CK log(N/K)
where 0 < p < 1. Solvable in prac-tice; requires fewer measurementsthan `1.M ≥ C1(p)K+pC2(p)K log(N/K)(with V. Staneva)
Slide 8 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 19: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/19.jpg)
Nonconvex compressive sensing
Optimization for sparse recovery
I Let x ∈ RN be sparse: ‖Ψx‖0 = K, K N .I Suppose A is an M ×N matrix, M N , with A and Ψ
incoherent. For example, A = (aij), i.i.d. aij ∼ N(0, σ2). Letb = Ax.
minu‖Ψu‖0, s.t. Au = b.
minu‖Ψu‖1, s.t. Au = b.
minu‖Ψu‖pp, s.t. Au = b,
Unique solution is u = x with opti-mally small M , but is NP-hard.M ≥ 2K suffices with probability 1.
Can be solved efficiently; requiresmore measurements for reconstruc-tion. M ≥ CK log(N/K)
where 0 < p < 1. Solvable in prac-tice; requires fewer measurementsthan `1.M ≥ C1(p)K+pC2(p)K log(N/K)(with V. Staneva)
Slide 8 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 20: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/20.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 2:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.1p
Slide 9 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 21: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/21.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 2:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.2p
Slide 9 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 22: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/22.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 2:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.3p
Slide 9 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 23: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/23.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 2:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.4p
Slide 9 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 24: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/24.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 2:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.5p
Slide 9 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 25: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/25.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.1p
Slide 10 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 26: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/26.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.2p
Slide 10 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 27: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/27.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.3p
Slide 10 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 28: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/28.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.4p
Slide 10 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 29: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/29.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.5p
Slide 10 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 30: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/30.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.6p
Slide 10 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 31: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/31.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1:
x
Au = b
|u1|p + |u2|p + |u3|p = 0.7p
Slide 10 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 32: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/32.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.1p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 33: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/33.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.2p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 34: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/34.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.3p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 35: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/35.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.4p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 36: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/36.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.5p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 37: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/37.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.6p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 38: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/38.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.7p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 39: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/39.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.8p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 40: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/40.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 0.9p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 41: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/41.jpg)
Nonconvex compressive sensing
The geometry of `p
minu ‖u‖pp, subject to Au = bp = 1/2:
x
|u1|p + |u2|p + |u3|p = 1p
Au = b
Slide 11 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 42: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/42.jpg)
Nonconvex compressive sensing
Why might global minimization be possible?
Consider an ε-regularized objective, restricted to the feasible plane:N∑
i=1
(u2
i + ε)p/2
.
A moderate ε fills in the local minima.
Slide 12 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 43: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/43.jpg)
Nonconvex compressive sensing
Why might global minimization be possible?
Consider an ε-regularized objective, restricted to the feasible plane:N∑
i=1
(u2
i + ε)p/2
.
A moderate ε fills in the local minima.
ε = 0
Slide 12 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 44: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/44.jpg)
Nonconvex compressive sensing
Why might global minimization be possible?
Consider an ε-regularized objective, restricted to the feasible plane:N∑
i=1
(u2
i + ε)p/2
.
A moderate ε fills in the local minima.
ε = 1
Slide 12 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 45: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/45.jpg)
Nonconvex compressive sensing
Why might global minimization be possible?
Consider an ε-regularized objective, restricted to the feasible plane:N∑
i=1
(u2
i + ε)p/2
.
A moderate ε fills in the local minima.
ε = 0.1
Slide 12 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 46: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/46.jpg)
Nonconvex compressive sensing
Why might global minimization be possible?
Consider an ε-regularized objective, restricted to the feasible plane:N∑
i=1
(u2
i + ε)p/2
.
A moderate ε fills in the local minima.
ε = 0.01
Slide 12 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 47: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/47.jpg)
Nonconvex compressive sensing
Why might global minimization be possible?
Consider an ε-regularized objective, restricted to the feasible plane:N∑
i=1
(u2
i + ε)p/2
.
A moderate ε fills in the local minima.
ε = 0.001
Slide 12 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 48: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/48.jpg)
Examples
Outline
Motivating Example
Nonconvex compressive sensing
Examples
Fast algorithm
Summary
Slide 13 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 49: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/49.jpg)
Examples
3-D tomography
Six radiographs allow reconstruction of a stalagmite segment:
radiograph isosurface
iso from end
one z slice
lower z slice
x slice y slice
Slide 14 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 50: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/50.jpg)
Examples
Cortical activity reconstruction from EEG
eigenfunction basis wavelet basis not sparse, noisy
Cortical activity is reconstructed perfectly from synthetic EEG data,consisting of 256 scalp potential measurements. The syntheticsignals have 80 nonzero coefficients from graph-diffusioneigenfunction or wavelet bases. Making the signal onlyapproximately sparse and adding noise results in very littlereconstruction error.
Slide 15 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 51: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/51.jpg)
Examples
Numerical tests
Reconstruction frequency from 100 random measurements of256-dimensional signals, using IRLS with and withoutregularization.
Slide 16 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 52: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/52.jpg)
Fast algorithm
Outline
Motivating Example
Nonconvex compressive sensing
Examples
Fast algorithm
Summary
Slide 17 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 53: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/53.jpg)
Fast algorithm
Semiconvex regularization
Now we generalize an approach of J. Yang, W. Yin, Y. Zhang, and Y.Wang. Consider a componentwise, mollified `p objective:
ϕ(t) =
γ|t|2 if |t| ≤ α|t|p/p− δ if |t| > α
The parameters are chosen tomake ϕ ∈ C1.
p = 1/2p = 0p = −1/2
t
ϕ(t)
Now we seek ψ such that
ϕ(t) = minw
ψ(w) + (β/2)|t− w|22
This can be found by convex duality, as |t|22/2−ϕ(t)/β is convex ifβ = αp−2.
Slide 18 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 54: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/54.jpg)
Fast algorithm
Semiconvex regularization
Now we generalize an approach of J. Yang, W. Yin, Y. Zhang, and Y.Wang. Consider a componentwise, mollified `p objective:
ϕ(t) =
γ|t|2 if |t| ≤ α|t|p/p− δ if |t| > α
The parameters are chosen tomake ϕ ∈ C1.
p = 1/2p = 0p = −1/2
t
ϕ(t)
Now we seek ψ such that
ϕ(t) = minw
ψ(w) + (β/2)|t− w|22
This can be found by convex duality, as |t|22/2−ϕ(t)/β is convex ifβ = αp−2.
Slide 18 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 55: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/55.jpg)
Fast algorithm
A splitting approach
Now we consider an unconstrained `p minimization problem, andreplace
minu
N∑i=1
ϕ((Ψu)i) + (µ/2)‖Au− b‖22
with the split version
minu,w
N∑i=1
ψ(wi) + (β/2)‖Ψu− w‖22 + (µ/2)‖Au− b‖22,
which we solve by alternate minimization.
Slide 19 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 56: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/56.jpg)
Fast algorithm
Easy iterations
Holding u fixed, the w-subproblem is separable, and its solutioncomes from the convex duality:
wi = max
0, |(Ψu)i| −
|(Ψu)i|p−1
β
(Ψu)i
|(Ψu)i|.
This generalizes shrinkage ( or soft thresholding ) to `p.
Holding w fixed, the u-problem is quadratic:
(βΨ∗Ψ + µA∗A)u = βΨ∗w + µA∗b.
If A is a Fourier sampling operator and Ψ is Fourier-diagonalizable(such as a derivative operator or orthogonal wavelet transform), wecan solve this in the Fourier domain. This is very fast!
Slide 20 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 57: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/57.jpg)
Fast algorithm
Easy iterations
Holding u fixed, the w-subproblem is separable, and its solutioncomes from the convex duality:
wi = max
0, |(Ψu)i| −
|(Ψu)i|p−1
β
(Ψu)i
|(Ψu)i|.
This generalizes shrinkage ( or soft thresholding ) to `p.
Holding w fixed, the u-problem is quadratic:
(βΨ∗Ψ + µA∗A)u = βΨ∗w + µA∗b.
If A is a Fourier sampling operator and Ψ is Fourier-diagonalizable(such as a derivative operator or orthogonal wavelet transform), wecan solve this in the Fourier domain. This is very fast!
Slide 20 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 58: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/58.jpg)
Fast algorithm
Easy iterations
Holding u fixed, the w-subproblem is separable, and its solutioncomes from the convex duality:
wi = max
0, |(Ψu)i| −
|(Ψu)i|p−1
β
(Ψu)i
|(Ψu)i|.
This generalizes shrinkage ( or soft thresholding ) to `p.
Holding w fixed, the u-problem is quadratic:
(βΨ∗Ψ + µA∗A)u = βΨ∗w + µA∗b.
If A is a Fourier sampling operator and Ψ is Fourier-diagonalizable(such as a derivative operator or orthogonal wavelet transform), wecan solve this in the Fourier domain. This is very fast!
Slide 20 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 59: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/59.jpg)
Fast algorithm
Enforcing equality
minu,w
N∑i=1
ψ(wi) + (β/2)‖Ψu− w‖22 + (µ/2)‖Au− b‖22
Typically one enforces w = Ψu (and Au = b, if desired) byiteratively growing β (and µ). (continuation)
We get better results from an augmented Lagrangian (cf. splitBregman, T. Goldstein, S. Osher):
minu,w
N∑i=1
ψ(wi) + (β/2)‖Du−w−λ1‖22 + (µ/2)‖Au− b−λ2‖22,
and update λn+11 = λn
1 + w −Ψu, λn+12 = λn
2 + b−Au.
Slide 21 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 60: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/60.jpg)
Fast algorithm
Enforcing equality
minu,w
N∑i=1
ψ(wi) + (β/2)‖Ψu− w‖22 + (µ/2)‖Au− b‖22
Typically one enforces w = Ψu (and Au = b, if desired) byiteratively growing β (and µ). (continuation)
We get better results from an augmented Lagrangian (cf. splitBregman, T. Goldstein, S. Osher):
minu,w
N∑i=1
ψ(wi) + (β/2)‖Du−w−λ1‖22 + (µ/2)‖Au− b−λ2‖22,
and update λn+11 = λn
1 + w −Ψu, λn+12 = λn
2 + b−Au.
Slide 21 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 61: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/61.jpg)
Fast algorithm
Multiple penalty terms and other constraints
The same approach can easily handle two or more penalty terms inthe objective:
minu,w,v
N∑i=1
ψ1(wi) + (β1/2)‖Ψ1u− w‖22
+N∑
i=1
ψ2(vi) + (β2/2)‖Ψ2u− v‖22 + (µ/2)‖Au− b‖22
One can also use a similar splitting/augmented-Lagrangianapproach to handle inequality noise constraints, nonnegativityconstraints, etc. The method is very flexible.
Slide 22 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 62: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/62.jpg)
Fast algorithm
Multiple penalty terms and other constraints
The same approach can easily handle two or more penalty terms inthe objective:
minu,w,v
N∑i=1
ψ1(wi) + (β1/2)‖Ψ1u− w‖22
+N∑
i=1
ψ2(vi) + (β2/2)‖Ψ2u− v‖22 + (µ/2)‖Au− b‖22
One can also use a similar splitting/augmented-Lagrangianapproach to handle inequality noise constraints, nonnegativityconstraints, etc. The method is very flexible.
Slide 22 of 23
Operated by Los Alamos National Security, LLC for NNSA
![Page 63: Fast algorithm of unconvex compressde sensin](https://reader034.vdocuments.us/reader034/viewer/2022042615/55a8e5fb1a28ab852f8b4673/html5/thumbnails/63.jpg)
Summary
Summary
I Nonconvex compressive sensing allows compressible imagesto be recovered with even fewer measurements than“traditional” compressive sensing.
I Nonconvexity also improves robustness to noise and signalnonsparsity.
I Regularizing the objective appears to keep algorithms fromconverging to nonglobal minima.
I For Fourier-sampling measurements, the reconstruction can bedone very fast.
math.lanl.gov/~rick
Slide 23 of 23
Operated by Los Alamos National Security, LLC for NNSA