duality in optimization and constraint satisfaction j. n. hooker carnegie mellon univ. pittsburgh,...

Duality in Optimization and Constraint Satisfaction

J. N. HookerCarnegie Mellon Univ.

Pittsburgh, USA

September 2006

Some Concepts of Duality

• Reflexive dual (mirror image)– Conservation of

parity• Refuted by C. S.

Wu, 1957


• Dual perspectives on one reality– Wave/particle duality


• Dual perspectives on one reality– Wave/particle duality– Mind/body duality

• Spinoza, Brahman/Atman

Lalita Mahatripurasundarirepresenting Brahman-Atman

consciousness


• Complementarity– Yin/yang


• Complementarity– Yin/yang– Good/evil in

Zoroastrianism

Aharamazda, Zoroastrian god

Duality in Mathematics

• Most mathematical duals are reflexive– Dual of dual = original


• Duality of polytopes– k-dimensional faces become (n k 1)-dimensional

faces

http://en.wikipedia.org/wiki/Image:Dual_Cube-Octahedron.jpg


• Duality of points and lines in projective geometry

7-point projective plane(Fano plane)


• Duality of rows and columns in matrix algebra.– Row rank of a matrix = column rank.


• Duality of rows and columns in matrix algebra.– Row rank of a matrix = column rank.

• Generalization: dual space of a vector space.– Space of linear functionals on a space has same

dimension as original space.– Dual of dual is a “natural transformation” in category

theory.

Duality in Optimization

• Most optimization duals offer dual perspectives.– Not reflexive in general.


• Reflexive:– Linear programming dual



• Not reflexive:– Surrogate dual– Lagrangean dual– Superadditive dual



• Not reflexive:– Surrogate dual– Lagrangean dual– Superadditive dual

• All of these are instances of inference duality and relaxation duality.


• Inference duality and relaxation duality are very different concepts.– Even though several optimization duals can be

viewed as either.



viewed as either.• Inference duality provides sensitivity analysis and

nogood-based search.– Will focus on nogood-based search here.





• Relaxation duality provides tight bounds.





• Relaxation duality provides tight bounds.• The constraint dual is neither type but gives rise to a

relaxation dual.


• Inference duality is a duality of search and inference.– The original problem can be solved by searching

over values of variables to find the best solution.– The dual is solved by inferring from the constraints

the best bound on the optimal value.


• Inference duality is a duality of search and inference.– The original problem can be solved by searching

over values of variables to find the best solution.– The dual is solved by inferring from the constraints

the best bound on the optimal value.

• Relaxation duality is a duality of restriction and relaxation.– The original problem can be solved by enumerating

restrictions.– The dual can be solved by enumerating

parameterized relaxations.


• Viewing duals in this way has three benefits.


• Viewing duals in this way has three benefits.• It reveals connections that might not otherwise be

noticed.– For instance, surrogate and Lagrangean duals are

seen to be closely related when viewed as inference duals rather than as relaxation duals.


• Viewing duals in this way has three benefits.• It reveals connections that might not otherwise be

noticed.– For instance, surrogate and Lagrangean duals are

seen to be closely related when viewed as inference duals rather than as relaxation duals.

• It can unify solution methods.– For instance, Benders decomposition and DPL with

clause learning are nogood-based search methods that result from different inference duals.


• It can suggest new solution methods.– For instance, any domain filter defines an inference

dual with an associated nogood-based search methods.

• Edge-finding gives rise to an effective Benders-like decomposition method for planning & scheduling, etc.

– A relaxation dual can be defined on a constraint dual formulation so as to generalize mini-bucket elimination.


• Outline…

Inference Dual

• An optimization problem minimizes an objective function subject to constraints.– It is solved by searching over values of the

variables.

• The inference dual finds the tightest lower bound on the objective function that can be deduced from the constraints.– It is solved by searching over proofs.

Inference Dual

• The optimization problem

has the inference dual

min ( )x D

f x C

max ( )

P Pv f x vC

P

Constraint set

Family of proofs

Proof P deduces f(x) v from C

“Primal” problem

Inference Dual

• Weak duality always holds:

Max value of dual problem

Min value of primal problem

Difference = duality gap

max ( )

P Pv f x vC

P

Inference Dual

• Strong duality sometimes holds:


= Min value of primal problem

P is a complete proof family

Strong duality

max ( )

P Pv f x vC

P

Linear Programming: Inference Dual

• A linear programming problem has the form

• The inference dual is

– The proof family P consists of domination by surrogates (nonnegative linear combinations).

0

minx

cx Ax b

max

P Pv Ax b cx v

P


• For any u 0, uAx ub is a surrogate of Ax b.

• uAx ub dominates cx v if uAx ub implies cx v (for x 0).– That is, uA c and ub v.

• This is a complete inference method.– Due to Farkas Lemma.


• So the inference dual

becomes

or

– This is the classical linear programming dual.

max

P Pv Ax b cx v

P

0

max ,u

v uA c ub v

0

maxu

ub uA c

when Ax b, x 0 is feasible


• Since the proof family is complete, we have strong duality:

0 0

mimax nxu

ub A cxu Axc b

except when uA c, u 0 and Ax b, x 0 are both infeasible.


• Since the proof family is complete, we have strong duality:

– In this case, the dual is symmetric• The dual of the dual is the primal.

– The dual therefore belongs to NP and the primal to co-NP.

0 0

mimax nxu

ub A cxu Axc b

except when uA c, u 0 and Ax b, x 0 are both infeasible.


• Example:

1 2

1 2 1

1 2 2

1 2

min 4 7

2 3 6 ( )

2 4 ( )

, 0

x x

x x u

x x u

x x

1 2

1 2 1

1 2 2

1 2

max 6 4

2 2 4 ( )

3 7 ( )

, 0

u u

u u x

u u x

u u

Dual:


• Example:

– Solution:

Dual:

1 2

1 2

1

1

2

2

(2)

(0)

2 3 6

2

4

4

4

6

7

12

12

x x

x x

x

x

x

xSurrogateDominates

1 2

1 2 1

1 2 2

1 2

12

2

min 4 7

2 3 6 ( )

2 4 ( )

, 0

0

x x

x x u

x x u

x x

1 2

1 2 1

1 2 2

1 2

12

3

max 6 4

2 2 4 ( )

3 7 ( )

, 0

0

u u

u u x

u u x

u u

Relaxation Dual

• A relaxation dual parameterizes relaxations of the optimization problem.– The relaxation parameters are dual variables.– Each relaxation provides a lower bound on the

optimal value of the primal.

• It seeks the relaxation that provides the tightest bound.– It is solved by searching over values of the dual

variables.

Relaxation Dual

• Given the optimization problem

a parameterized relaxation is

min ( )x D

f x C

min ( , ) ( )x D

f x u uC

Lower bound on f(x) for all x D satisfying C.

Relaxation of C.

u = vector of dual variables

Relaxation Dual


a parameterized relaxation is

– It provides a lower bound on the optimal value of the original problem, for any u.

min ( )x D

f x C

min ( , ) ( )x D

f x u uC

Relaxation Dual


a relaxation dual is

min ( )x D

f x C

max min ( , ) ( )

x Du Uf x u uC

Find relaxation that provides the largest lower bound.

Relaxation Dual

• Weak duality always holds:


Min value of primal problem

Difference = duality gap

max min ( , ) ( )

x Du Uf x u uC

Linear Programming: Relaxation Dual

• A surrogate relaxation of

replaces Ax b with a surrogate uAx ub.– The relaxation is parameterized by u.– The objective function is left unchanged.

0

minx

cx Ax b


• This yields the relaxation dual

– It is equivalent to the classical linear programming dual.

– So the classical dual is both an inference and a relaxation dual.

00max min

xucx uAx ub


• Example:

• Relaxation dual:

1 21 21 2 1 2 1 1 2 2 1 2, 0, 0

max min 4 7 (2 2 ) (3 ) 6 4x xu u

x x u u x u u x u u

1 2

1 2 1

1 2 2

1 2

min 4 7

2 3 6 ( )

2 4 ( )

, 0

x x

x x u

x x u

x x


• Example:

• Relaxation dual:

• Solution:

1 21 21 2 1 2 1 1 2 2 1 2, 0, 0

max min 4 7 (2 2 ) (3 ) 6 4x xu u

x x u u x u u x u u

1 2

1 2 1 2, 0min 4 7 124 6 12x x

x x x x

1 2Let ( , ) (2,0)u u

1 2

1 2 1

1 2 2

1 2

min 4 7

2 3 6 ( )

2 4 ( )

, 0

x x

x x u

x x u

x x

Surrogate Dual

• The surrogate dual is a relaxation dual of a general inequality-constrained problem:

• The parameterized relaxation is

• So the dual is

– In general there is a duality gap.

min ( ) ( ) 0x D

f x g x

min ( ) ( ) 0x D

f x ug x

0max min ( ) ( ) 0

u x Df x ug x

Surrogate Dual

• Example:

• The surrogate dual is

1 2

1 2

1 2

1 2

min 3 4

3 0

5 2 0

, {0,1,2,3}

x x

x x

x x

x x

1 2 1 1 2 2 1 2{0,1,2,3}0max min 3 4 ( 3 ) (5 2 ) 0

jxux x u x x u x x

Optimal value = 10

Surrogate Dual

• Example:

• The surrogate dual is

• Solution:

1 2

1 2

1 2

1 2

min 3 4

3 0

5 2 0

, {0,1,2,3}

x x

x x

x x

x x

1 2 1 1 2 2 1 2{0,1,2,3}0max min 3 4 ( 3 ) (5 2 ) 0

jxux x u x x u x x

1 2

1 2

( , ) (2,5)

8x 11

, which yields

the surrogate

and bound (no duality gap in

25

this ca10 se)

u u

x

Optimal value = 10

Surrogate Dual

• The surrogate dual is also an inference dual.• It uses the same proof system as the linear programming

dual.• Deduce f(x) v from g(x) 0 when some surrogate

ug(x) 0 dominates f(x) v.– i.e., when ug(x) 0 implies f(x) v.– An incomplete proof system.

Surrogate Dual

• This yields the inference dual

– which is equivalent to the relaxation dual

0

max ( ) 0 ( ) for u

v ug x f x v x D

0

max min ( ) ( ) 0x Du

f x ug x

Aside: Subadditive Dual

• Suppose multiplication by nonnegative vector u is replaced by a general subadditive function u( ).– u(a + b) u(a) + u(b)– u is also nondecreasing and homogeneous.

• The surrogate dual becomes the subadditive dual.– Repeated linear combinations and rounding give rise

to a family of subadditive functions for integer programming.

• They provide a complete inference method.• There is no duality gap.

Lagrangean Dual

• The Lagrangean dual is another relaxation dual for inequality-constrained problems

min ( ) ( ) 0x D

f x g x

Lagrangean Dual


• This time, remove the constraints entirely but “penalize” constraint violations in the objective function:

min ( ) ( ) 0x D

f x g x

0max min ( ) ( )

u x Df x ug x

Lagrangean Dual


• This time, remove the constraints entirely but “penalize” constraint violations in the objective function:

– This is a relaxation since

min ( ) ( ) 0x D

f x g x

0max min ( ) ( )

u x Df x ug x

( ) ( ) ( ) for all feasible f x ug x f x x

0 0

Lagrangean Dual

• The Lagrangean dual provides a weaker bound than than the surrogate dual.– But it can be solved by steepest ascent search.

• Write the dual

as

– Then (u) is concave, and its subgradient is g(x).

0max min ( ) ( )

u x Df x ug x

0

where max ( ) ( ) min ( ) ( )x Du

u u f x ug x

Lagrangean Dual

• Example:

• The dual is

where

1 2

1 2

1 2

1 2

min 3 4

3 0

5 2 0

, {0,1,2,3}

x x

x x

x x

x x

1 2

1 2, 0

max ( , )u u

u u

1 2 1 2 1 1 2 2 1 2{0,1,2,3}( , ) min 3 4 ( 3 ) (5 2 )

jxu u x x u x x u x x

Lagrangean Dual

• Example:

• The dual is

where

• Solution:

1 2

1 2

1 2

1 2

min 3 4

3 0

5 2 0

, {0,1,2,3}

x x

x x

x x

x x

1 2

1 2, 0

max ( , )u u

u u

1 2 1 2 1 1 2 2 1 2{0,1,2,3}( , ) min 3 4 ( 3 ) (5 2 )

jxu u x x u x x u x x

5 13 21 2 7 7 7

27

with optimal value

Duality gap

( , ) ( , ) 9

10 9of

u u

Lagrangean Dual

• The Lagrangean and surrogate duals appear unrelated.– But when viewed as inference duals, they are closely

related.• The Lagrangean dual uses a slightly weaker proof

system.– It follows that it yields a weaker bound.

Lagrangean Dual

• Deduce f(x) v from g(x) 0 when some surrogate ug(x) 0 dominates f(x) v.– i.e., some surrogate ug(x) 0 dominates v f(x) 0.

Lagrangean Dual


• In the surrogate dual, “dominates” meansug(x) 0 implies v f(x) 0.

Lagrangean Dual



• In the Lagrangean dual, it means something stronger:ug(x) v f(x).

– i.e., v f(x) + ug(x)

Lagrangean Dual



• In the Lagrangean dual, it means something stronger:ug(x) v f(x).

– i.e., v f(x) + ug(x)• So the resulting inference dual is the Lagrangean dual:

0

0

max ( ) ( ) for all

max min ( ) ( )

x Du

uv v f x ug x

x ug x

x D

f

Nogood-Based Search

• The inference dual provides the basis for nogood-based search.– Nogoods rule out solutions already examined.

• …and possibly other solutions that are no better.

Nogood-Based Search


• …and possibly other solutions that are no better.– Search proceeds by enumerating partial assignments

to the variables.– The next partial assignment must satisfy nogoods

generated so far.

Nogood-Based Search


• …and possibly other solutions that are no better.– Search proceeds by enumerating partial assignments

to the variables.– The next partial assignment must satisfy nogoods

generated so far.• The nogoods are obtained by analyzing the inference

dual of the restricted problem that results from each partial assignment.

Nogood-Based Search

• Solve:

• In each iteration, formulate a restriction of the problem:

min ( )x D

f x C

min ( )x D

f x

C B

Partial assignment that fixes some of the variables xj

Nogood-Based Search

• Solve:

• In each iteration, formulate a restriction of the problem:

– B may contain:• Branching constraints (in branching methods)• Solution of master problem (in Benders

decomposition)

min ( )x D

f x C

min ( )x D

f x

C B

Nogood-Based Search

• Let solve the inference dual of the restriction:

– Let fix the variables in B that serve as premises in

max ( ) ( )p P

f x f x v

C BP

( , )v P

BP

Nogood-Based Search


– Let fix the variables in B that are needed as premises in

– Create the nogood

max ( ) ( )p P

f x f x v

C BP

( , )v P

BP

( )f x v B

Nogood-Based Search


– Let fix the variables in B that serve as premises in

– Create the nogood

• The next partial assignment B must be consistent with all nogoods generated so far.– This avoids partial assignments already examined.

• …and others that are no better.

max ( ) ( )p P

f x f x v

C BP

( , )v P

BP

( )f x v B

Linear Programming: Nogood-Based Search

• Solve 0

min ( ) ( )F F

U

F U F Ux D

x

f x cx g x Ax b

Problem becomes linear subproblem when xF is fixed to .Fx


• Solve

• Nogood-based search becomes Benders decomposition.– The nogoods are formed by solving the linear

programming dual of the problem that results when xF is fixed.

0

min ( ) ( )F F

U

F U F Ux D

x

f x cx g x Ax b

Problem becomes linear subproblem when xF is fixed to .Fx


• Solve

• Let solve LP dual of the subproblem:

0

min ( ) ( )F F

U

F U F Ux D

x

f x cx g x Ax b

u

0

max ( )Fu

u b g x uA c


• Solve


– Then by strong duality ( = optimal value):

0

min ( ) ( )F F

U

F U F Ux D

x

f x cx g x Ax b

u

( ) ( )F Fz f x u b g x

0

max ( )Fu

u b g x uA c

z


• Solve


– Then by strong duality ( = optimal value):

– And since is dual feasible for any xF, weak duality implies the Benders cut

0

min ( ) ( )F F

U

F U F Ux D

x

f x cx g x Ax b

u

Fu ( ) ( )F Fz f x u b g x

( ) ( )F Fz f x u b g x

0

max ( )Fu

u b g x uA c

z

SAT: Nogood-Based Search

• Consider the propositional satisfiability problem (SAT), which can be written

• Branching search (DPL) with clause learning can be viewed as a nogood-based search.– Unit resolution (unit clause rule) is applied at every

node of the search tree.– The nogoods are conflict clauses…

{T,F}

min 0jx

C Set of logical clauses


• Example:

– Solve by branching on xj’s…


1 5

SAT problem

+ partial assignment

( , , ) (F,F,F,F,F)

is infeasible

x x


1 5

SAT problem


( , , ) (F,F,F,F,F)

is infeasible ( )

x x

v

Solution of inference dual is a unit resolution proof of infeasibility.


1 5

SAT problem


( , , ) (F,F,F,F,F)

is infeasible

x x

Solution of inference dual is a unit resolution proof of infeasiblity.

Proof remains valid when only x1 and x5 are fixed to F.


1 5

SAT problem


( , , ) (F,F,F,F,F)

is infeasible

x x



So we have the nogood

1 5( , ) (F,F) ( )x x f x


1 5

SAT problem


( , , ) (F,F,F,F,F)

is infeasible

x x



So we have the nogood

or conflict clause 1 5x x

1 5( , ) (F,F) ( )x x f x


1 5

Next partial assignment

( , , ) (F,F,F,F,T)

must satisfy nogood

x x


1 2

1 2

Next partial assignment

( , ) (F,T)

must satisfy previous

2 nogoods and

their resolvent

x x

x x


At this point the

accumulated nogoods

are unsatisfiable

and the search

is exhaustive

Domain Filtering Duals

• Since a domain filter is an inference method, it defines an inference dual and a nogood-based search method.

– Let [Lj,Uj] be the domain of xj.

– So x [L,U] where L = (L1,…,Ln), U = (U1,…,Un)

– Then the inference dual can be defined

where P contains the domain filtering operations.

max ( ( ) ) max ( ) ( [ , ])p P P P

v f x v f x x L U

C CP P


• Suppose we are branching.– Let B contain the current branching constraints.– Let solve the inference dual.– Let contain the constraints in B used as premises

in the filtering operation– We have the nogood

( , )v P

BP

( )f x v B


• Example: assign jobs to machines and schedule them to minimize makespan.

{1, , }, all

[ , ], all

, all min

disjunctive ( ),( ) , all

j

j

j j j

j x j

x m jj j ij j

s r d j

v s p jv

s x i p x i i

Start time of job jprocessing time of job j on machine i

Machine assigned to job j

Time window for job j

Planning & Scheduling: Filtering Dual

Suppose there are 5 jobs and 2 machines:

We will search overmachine assignments x.




At the current point in the search, we have assigned jobs 1,2,3,5 to machine A and job 4 to machine B.





Min makespan on machine B is 5.





Min makespan on machine B is 5.

Let’s compute min makespan on machine A…


Job 1

Job 2

Job 3

Job 5

Time window

Schedule

10

We find a feasible schedule with makespan 10.

Machine A


Job 1

Job 2

Job 3

Job 5

Time window

Schedule

9

We found a feasible schedule with makespan 10.

To prove optimality, we check whether a makespan of 9 is possible.

Machine A


Job 1

Job 2

Job 3

Job 5

Time window

Schedule

9

Edge finding discovers that job 2 must precede jobs 3 and 5.

Machine A


Job 1

Job 2

Job 3

Job 5

Time window

Schedule

9

Edge finding discovers that job 2 must precede jobs 3 and 5.

Since this reduces domain of s2 to empty set, there is no feasible solution with makespan 9.

Machine A


Job 1

Job 2

Job 3

Job 5

Time window

Schedule

9This edge finding procedure solves the inference dual of the scheduling problem.

Machine A

P


Job 1

Job 2

Job 3

Job 5

Time window

Schedule


derives infeasibility from the assignments Bof jobs 1,2,3,5 to machine A.

Machine A

P

P


Job 1

Job 2

Job 3

Job 5

Time window

Schedule


derives infeasibility from the assignments Bof jobs 1,2,3,5 to machine A.

Is there a smaller set of assignments that is sufficient to prove infeasibility?

Machine A

P

P

B


Job 1

Job 2

Job 3

Job 5

Time window

Schedule

9

Only jobs 2, 3, 5 were involved in this deduction. So we have the nogood:

Machine A

2 3 5 10x x x A v

B


Job 1

Job 2

Job 3

Job 5

Time window

Schedule

9


To improve the solution, the search must avoid this assignment.

Machine A

2 3 5 10x x x A v


Job 1

Job 2

Job 3

Job 5

Time window

Schedule

9


Since the nogoods contain a fixed set of variables x, this is a generalization of Benders decomposition

Machine A

2 3 5 10x x x A v

Constraint Dual

• The constraint dual is neither a relaxation dual nor an inference dual.

• However, one can define a relaxation dual for a constraint dual formulation.– Mini-bucket elimination is a special case.

• This is best explained by example…

Constraint Dual

• Consider the 0-1 problem

1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x x

x x x

x x x

x

Constraint Dual

• Consider the 0-1 problem

– Solution:

1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0

2

,1}j

x x x x

x x x

x x x

x x x

x

1 4( , , ) (0,1,0,0)x x

Constraint Dual

• One form of constraint dual (ignoring the objective function)is

1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x x

x x x

x x x

x

1 21 11 2 32 2 21 33 31 34 4

1 1 11 2 32 2 21 2 43 3 32 3 4

( , , )

( , , )

( , , )

x x

x x x

x x

x x

x x x D

x x x D

x x x D

where 3{0,1} {(0,0,0)}D

Standardize apart variables in different constraints and equate them in binary constraints.

Constraint Dual

12x 1

3x

21x

11x

22x

32x

24x

34x3

3x

Dependency graph for constraint dual:

1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x x

x x x

x x x

x

Induced width of dependency graph is 3.

This indicates complexity of solving the problem by nonserial dynamic programming.

Constraint Dual

1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x x

x x x

x x x

x

3x

2x1x

4x

Dependency graph of original problem also has induced width 3:

Constraint Dual

• One way to relax the dynamic programming model is to remove edges from the dependency graph.– and form “mini-buckets”

1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x x

x x x

x x x

x

1 1 2 3 2 1 2 4 3 2 3 4

1 2 3 1 2 31 1 2 3

1 2 4 1 2 42 1 2 4

2 3 4 2 3 43 2 3 4

min ( , , ) ( , , ) ( , , )

2 3 if 1( , , )

otherwise

0 0 4 if 1( , , )

otherwise

0 0 0 if 1( , , )

otherwise

f x x x f x x x f x x x

x x x x x xf x x x

x x x x x xf x x x

x x x x x xf x x x

Constraint Dual

1 1 2 3 2 1 2 4 3 2 3 4

1 2 3 1 2 31 1 2 3

1 2 4 1 2 42 1 2 4

2 3 4 2 3 43 2 3 4

min ( , , ) ( , , ) ( , , )

2 3 if 1( , , )

otherwise

0 0 4 if 1( , , )

otherwise

0 0 0 if 1( , , )

otherwise


x x x x x xf x x x

x x x x x xf x x x

x x x x x xf x x x

• The problem separates into 3 problems with induced width 2:

3x

2x1x 2x

1x

4x 3x

2x

4x

Constraint Dual

1 1 2 3 2 1 2 4 3 2 3 4

1 2 3 1 2 31 1 2 3

1 2 4 1 2 42 1 2 4

2 3 4 2 3 43 2 3 4

min ( , , ) ( , , ) ( , , )

2 3 if 1( , , )

otherwise

0 0 4 if 1( , , )

otherwise

0 0 0 if 1( , , )

1 0

o

0 1

t


x x x x x xf x x x

x x x x x xf x x x

x x x x x xf x x x

herwise

• But the resulting bound is weak

3x

2x1x 2x

1x

4x 3x

2x

4x

Constraint Dual

Another approach is to delete edges from the dependency graph of the constraint dual.

1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x x

x x x

x x x

x

12x 1

3x

21x

11x

22x

32x

24x

34x3

3x

Constraint Dual

12x 1

3x

21x

11x

22x

32x

24x

34x3

3x


1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x x

x x x

x x x

x

The previous mini-bucket scheme deletes all edges connecting variables.

Constraint Dual


1 2 3 4

1 2 3

1 2 4

2 3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x x

x x x

x x x

x

By deleting only one edge…

12x 1

3x

21x

11x

22x

32x

24x

34x3

3x

Constraint Dual


1 2 3 4

1 2

1 2 4

2

3

3 4

min 2 3 4

1

1

1

{0,1}j

x x x x

x x

x x

y

x

x x x

x

By deleting only one edge…one can reduce the induced width to 2.

12x 1

3x

21x

11x

22x

32x

24x

34x3

3x

Constraint Dual


1 2 3 4

1 2

1 2 4

2 3 4

3

min 2 3 4

1

1

1

{0,1

2

}j

x x x x

x x

x x x

x x

x

y

x

By deleting only one edge…one can reduce the induced width to 2.

The bound is now tight.

12x 1

3x

21x

11x

22x

32x

24x

34x3

3x

Constraint Dual

• One can now define a relaxation dual: choose a set of eliminated edges that provides the tightest bound.

• The choice of edges parameterizes the relaxation.

The dual can be solved by various heuristics, such as local search.

12x 1

3x

21x

11x

22x

32x

24x

34x3

3x

Primal-Dual Methods

• Primal-dual inference methods– Branch-and-cut for integer programming

• Combine branching (search over restrictions) with inferred cutting planes.

– Standard CP methods• Combine branching with filtering.

Primal-Dual Methods

• Primal-Dual Relaxation Methods– Branch-and-bound with Lagrangean relaxation.

• Solve Lagrangean relaxation at each node of search tree.

– Interleave search over restrictions with search over relaxations.

• Dual simplex method for linear programming.• “Out-of-kilter” method for network flows.• Dual ascent method for integer programming.

Duality of Duals?

• Why can so many duals be interpreted as both relaxation duals and inference duals?– Particularly math programming duals.

• Is there a duality of duals?

Duality of Duals?

• Why can so many duals be interpreted as both relaxation duals and inference duals?– Particularly math programming duals.

• Is there a duality of duals?

• No.– But there is a formal relationship between the duals…

Duality of Duals?

• A relaxation dual in which there is a finite algorithm for solving the relaxation is an inference dual.– The solution algorithm can be viewed as inferring the

best bound.• An inference dual in which the proofs are parameterized

is a relaxation dual.– The proof of the best bound can be viewed as solving

the tightest relaxation.

Duality of Duals?

• Math programming duals tend to be conceived as relaxation duals.– There is always an algorithm for solving the

relaxation, otherwise the dual would be useless.– So math programming duals tend to be both

relaxation and inference duals.• Inference methods are not ordinarily parameterized.

– So inference duals are not necessarily relaxation duals.

duality in optimization and constraint satisfaction j. n. hooker carnegie mellon univ. pittsburgh,...

Documents