Total Variation and Geometric Regularization for Inverse ProblemsRegularization in StatisticsSeptember 7-11, 2003BIRS, Banff, Canada Tony Chan Department of Mathematics, UCLA

OutlineTV & Geometric Regularization (related concepts)PDE and Functional/Analytic basedGeometric Regularization via Level Sets TechniquesApplications (this talk):Image restorationImage segmentationElliptic Inverse problemsMedical tomography: PET, EIT

Regularization: Analytical vs StatisticalAnalytical: Controls smoothness of continuous functionsFunction spaces (e.g. Sobolov, Besov, BV)Variational models -> PDE algorithmsStatistical:Data driven priorsStochastic/probabilistic frameworksVariational models -> EM, Monte Carlo

Taking the Best from Each?Concepts are fundamentally related: e.g. Brownian motion Diffusion EquationStatistical frameworks advantages: General modelsAdapt to specific dataAnalytical frameworks advantages:Direct control on smoothness/discontinuities, geometryFast algorithms when applicable

Total Variation Regularization Measures variation of u, w/o penalizing discontinuities. |.| similar to Huber function in robust statistics. 1D: If u is monotonic in [a,b], then TV(u) = |u(b) u(a)|, regardless of whether u is discontinuous or not. nD: If u(D) = char fcn of D, then TV(u) = surface area of D. (Coarea formula) Thus TV controls both size of jumps and geometry of boundaries. Extensions to vector-valued functions Color TV: Blomgren-C 98; Ringach-Sapiro, Kimmel-Sochen

The Image Restoration ProblemA given Observed image z Related to True Image uThrough Blur KAnd Noise n Blur+NoiseInitial BlurInverse Problem: restore u, given K and statistics for n.Keeping edges sharp and in the correct location is a key problem !

Total Variation RestorationGradient flow: anisotropic diffusion data fidelity* First proposed by Rudin-Osher-Fatemi 92.* Allows for edge capturing (discontinuities along curves).* TVD schemes popular for shock capturing. Regularization:Variational Model:

Comparison of different methods for signal denoising & reconstruction

Image Inpainting (Masnou-Morel; Sapiro et al 99) Disocclusion Graffiti Removal

Unified TV Restoration & Inpainting model(C- J. Shen 2000)

TV Inpaintings: disocclusion

Examples of TV InpaintingsWhere is the Inpainting Region?

TV Zoom-inInpaint Region: high-res points that are not low-res pts

Edge Inpaintingedge tube TNo extra data are needed. Just inpaint!Inpaint region: points away from Edge Tubes

ExtensionsColor (S.H. Kang thesis 02)Eulers Elastica Inpainting (C-Kang-Shen 01) Minimizing TV + Boundary Curvature Mumford-Shah Inpainting (Esedoglu-Shen 01)Minimizing boundary + interior smoothness:

Geometric RegularizationMinimizing surface area of boundaries and/or volume of objectsWell-studied in differential geometry: curvature-driven flowsCrucial: representation of surface & volumeNeed to allow merging and pinching-off of surfacesPowerful technique: level set methodology (Osher/Sethian 86)

Level Set Representation (S. Osher - J. Sethian 87)Inside COutside COutside CCExample: mean curvature motion* Allows automatic topology changes, cusps, merging and breaking. Originally developed for tracking fluid interfaces.C= boundary of an open domain

Application: active contour Initial Curve Evolutions Detected Objects

Basic idea in classical active contoursCurve evolution and deformation (internal forces): Min Length(C)+Area(inside(C)) Boundary detection: stopping edge-function (external forces)Example:Snake model (Kass, Witkin, Terzopoulos 88)Geodesic model (Caselles, Kimmel, Sapiro 95)

Limitations - detects only objects with sharp edges defined by gradients - the curve can pass through the edge - smoothing may miss edges in presence of noise - not all can handle automatic change of topology Examples

A fitting term without edges where Fit > 0 Fit > 0 Fit > 0 Fit ~ 0Minimize: (Fitting +Regularization)Fitting not depending on gradient detects contours without gradient

An active contour model without edges Fitting + Regularization terms (length, area) C = boundary of an open and bounded domain |C| = the length of the boundary-curve C(C. + Vese 98)

Mumford-Shah Segmentation 89S=edges MS reg: min boundary + interior smoothness CV model = p.w. constant MS

Variational Formulations and Level Sets(Following Zhao, Chan, Merriman and Osher 96)The Heaviside functionThe level set formulation of the active contour model

The Euler-Lagrange equationsUsing smooth approximations for the Heaviside and Delta functions

AdvantagesAutomatically detects interior contours!Works very well for concave objects Robust w.r.t. noise Detects blurred contours The initial curve can be placed anywhere!Allows for automatical change of topolgy

Experimental Results

A plane in a noisy environmentEurope nightlights

4-phase segmentation2 level set functions2-phase segmentation1 level set function Multiphase level set representations and partitions allows for triple junctions, with no vacuum and no overlap of phases

Example: two level set functions and four phases

Phase 11 Phase 10 Phase 01 Phase 00 mean(11)=45 mean(10)=159 mean(01)=9 mean(00)=103 An MRI brain image

References for PDE & Level Sets in Imaging* IEEE Tran. Image Proc. 3/98, Special Issue on PDE Imaging* J. Weickert 98: Anisotropic Diffusion in Image Processing* G. Sapiro 01: Geometric PDEs in Image Processing Aubert-Kornprost 02: Mathematical Aspects of Imaging Processing Osher & Fedkiw 02: Bible on Level Sets Chan, Shen & Vese Jan 03, Notices of AMS

*