Submodular Optimization and Approximation Algorithms
Satoru Iwata (RIMS, Kyoto University)
Submodular Functions
• Cut Capacity Functions • Matroid Rank Functions • Entropy Functions
Finite Set
Entropy Functions
Information Sources
Entropy of the Joint Distribution
Conditional Mutual Information
Positive Definite Symmetric Matrices
Ky Fan’s Inequality
Extension of the Hadamard Inequality
Discrete Concavity
Diminishing Returns
Discrete Convexity
Convex Function
Discrete Convexity Lovász (1983)
: Linear Interpolation
: Convex
: Submodular
Murota (2003)
Discrete Convex Analysis
Submodular Function Minimization
Minimization Algorithm
Evaluation Oracle
Minimizer
Assumption:
Ellipsoid Method Grötschel, Lovász, Schrijver (1981)
Submodular Function Minimization Grötschel, Lovász, Schrijver (1981, 1988)
Iwata, Fleischer, Fujishige (2000) Schrijver (2000)
Iwata (2003)
Fleischer, Iwata (2000)
Orlin (2007)
Iwata (2002)
Fully Combinatorial
Ellipsoid Method
Cunningham (1985)
Iwata, Orlin (2009)
Symmetric Submodular Functions
Symmetric
Symmetric Submodular Function Minimization
Minimum Cut Algorithm by MA-ordering Nagamochi & Ibaraki (1992) Minimum Degree Ordering Nagamochi (2007)
Queyranne (1998)
Application to Clustering
Minimize
subject to
Greedy Split
-Approximation
Monotone, Submodular
Zhao, Nagamochi, Ibaraki (2005)
Submodurlar Function Maximization
Approximation Algorithms Nemhauser, Wolsey, Fisher (1978) Monotone SF Cardinality Constraint (1-1/e)-Approximation
Vondrák (STOC 2008) Monotone SF Matroid Constraint (1-1/e)-Approximation
Submodurlar Function Maximization
Approximation Algorithms Feige, Mirrokni, Vondrák (FOCS 2007) Nonnegative SF 2/5-Approximation
Lee, Mirrokni, Nagarajan, Sviridenko (STOC 2009) Nonnegative SF 1/4-Approximation (Matroid Constraint) 1/5-Approximation (Knapsack Constraints)
Feature Selection Random Variables
Predict from subset “Sick”
“Fever” “Rash” “Cough” Maximize subject to Conditionally
Independent
Submodular
Krause & Guestrin (2005)
Submodular Welfare Problem
Monotone, Submodular
Utility Functions
Maximize
subject to
-Approximation Vondrák (2008)
Algorithm with for monotone submodular functions
Construct a set function such that
For what function is this possible?
Approximating Submodular Functions Goemans, Harvey, Iwata & Mirrokni (2009)
Question What Kind of Approximation Algorithms Can Be Extended to Optimization Problems
with Submodular Cost or Constraints ?
Cf. Submodular Flow (Edmonds & Giles, 1977)
Svitkina & Fleischer (FOCS 2008) Sampling Algorithms, Lower Bounds Submodular Sparsest Cut Submodular Load Balancing
Question How Can We Exploit Discrete Convexity
in Design of Approximation Algorithms?
Cf. Ellipsoid Method (Grötschel, Lovász & Schrijver, 1981)
Chudak & Nagano (SODA 2007) Facility Location with Submodular Penalty
Submodular Function Minimization under Covering Constraints
• Submodular Vertex Cover Problem Rounding Algorithm • Submodular Cost Set Cover Problem Rounding Algorithm Primal-Dual Algorithm • Submodular Edge Cover Problem Inapproximability
Iwata & Nagano (2009)
The Vertex Cover Problem Graph
Vertex Cover
Find a Vertex Cover
Minimizing
The Vertex Cover Problem Integer Programming Formulation
Minimize
subject to
Linear Programming Relaxation (LPR)
The Vertex Cover Problem
Lemma (Nemhauser & Trotter, 1974)
LPR has a half-integral optimal solution.
Every nonsingular submatrix in the coefficient matrix has a half-integral inverse.
The Vertex Cover Problem
Optimal Solution of LPR
Vertex Cover
Vertex Cover
2-Approximation Solution
Submodular Vertex Cover Graph
Submodular
Find a Vertex Cover Minimizing
The Vertex Cover Problem
Relaxation Problem
Convex Programming Relaxation (CPR)
Minimize
subject to
Lemma
CPR has a half-integral optimal solution.
Proof of Half-Integrality
Every nonsingular submatrix in the coefficient matrix has a half-integral inverse.
Rounding Algorithm Half-Integral Optimal Solution of CPR
Vertex Cover
2-Approximation Solution
Vertex Cover
How to Solve CRP
Find a Vertex Cover
Minimizing
in
How to Solve CPR
Vertex Cover
Optimal Vertex Cover with
How to Solve CPR Vertex Cover with
Feasible to CPR
Lower Bounds Goel, Karande, Tripathi, Wang (FOCS 2009)
No polynomial algorithm can solve the submodular vertex cover problem approximately within a factor smaller than 2.
Rounding Algorithm Using Ellipsoid Method for Monotone Submodular Functions
Submodular Cost Set Cover Find Covering with Minimum
Rounding Algorithm -Approximation
Primal-Dual Algorithm
Greedy -Approximation Algorigm for Monotone Submodular Functions
Koufogiannakis & Young (ICALP 2009)
Relaxation Problem
Minimize
(SCP)
subject to
Ellipsoid Method
Rounding Algorithm
Optimal Solution of (SCP)
Set Cover
-Approximation Solution
Minimizer of among Supersets of
Rounding Algorithm
Set Cover
Set Cover
Monotone, Submodular
Primal-Dual Approximation
Maximize
subject to
Dual Problem of (SCP)
Submodular Polyhedron
Primal-Dual Approximation Initially,
Unique Maximal Set with
Primal-Dual Approximation
Set Cover
Set Cover
-Approximation
Submodular Cost Set Cover What if is not bounded?
For each Minimizer of Covering Set Cover
Set Cover
-Approximation
Greedy -Approx.?
Submodular Edge Cover Graph
Edge Cover
Submodular, Nonnegative,
Find an Edge Cover Minimizing
NP-hard MIN 2SAT
Inapproximability Result
Theorem No oracle-polynomial algorithm can solve the submodular edge cover problem approximately within a factor of
Summary Submodular Vertex Cover
CP-Rounding
2-Approx.
No Oracle-Polynomial Algorithm with an Approximation Ratio.
Submodular Cost Set Cover
CP-Rounding
Primal-Dual -Approx.
Submodular Edge Cover
-Approx.