1

Projection and inverse Projection and inverse projection as a method projection as a method of reformulating linear of reformulating linear

and integer and integer programmesprogrammesH.P. WilliamsH.P. Williams

London School of EconomicsLondon School of Economics

h.p.williams @ lse.ac.uk

www.lse.ac.uk/depts/op-research/personal/Williams

2

ExampleExample(Maximise (Maximise zz))

Subject to:

Project out x1

1 2 3

1 2 3

1 2 3

1

2

3

4 5 3 0 0

2 1

2 3 2

0 3

0 4

0 5

x x x z C

x x x C

x x x C

x C

x C

x C

3

2 3

2 3

2 3

2 3

2 3

7 8

5 3 0

2 5

2 3

, 0

x x z

x x z

x x

x x

x x

3

3

3

3

3

13 2 21

5

7 15

2 3

0

x z

x

x z

x

x

383

43

15

z

z

z

4

How to carry out ProjectionHow to carry out Projection

The following statements are equivalent:

jigf

xjigxfx

ji

ji

, all

:, all ][

Proof Immediate

Take )( j

j

ii

gMinx

fMaxx

or

(Decision Procedure of Langford for Theory of Dense Linear Order)

5

We can take fi and g i as linear expressions in the other variables (apart from x)

The constraints of any linear programme can be put in this form and x eliminated (projected out)

But need to combine every inequality of form

With every inequality of form

"" jgx

"" ifx

Can lead to combinatorial explosion in number of inequalities.

Equations and associated variables can be eliminated prior to this by Gaussian Elimination.

6

ExampleExample(Maximise (Maximise zz))

Subject to:1 2 3

1 2 3

1 2 3

1

2

3

4 5 3 0 0

2 1

2 3 2

0 3

0 4

0 5

x x x z C

x x x C

x x x C

x C

x C

x C

7

Write in formWrite in form

32

321

32

23

354

1

0

2

xx

zxxxxx

0

0

3

2

x

x

8

Eliminate Eliminate xx11

0

0

230

354

10

232

354

12

3

2

32

32

3232

3232

x

x

xx

zxx

xxxx

zxxxx

9

Have added (in suitable multiples) every inequality in which x1 has a positive coefficient to every inequality in which x1 has a negative coefficient

i.e. 2 3

2 3

2 3

2 3

2

3

7 8 0 4 1

5 3 0 0 4 3

2 5 1 2

2 3 2 3

0 4

0 5

x x z C C

x x z C C

x x C C

x x C C

x C

x C

10

C0 C1 C2 C3 C4 C5

1 1 11

1

1 1

4

Can continue process to eliminate x2 and x3

This is Fourier-Motzkin.

14

11

C0 C1 C2 C3 C4 C5

-- + + - 0

+ +- - 0 0

x2

Can be shown (Kohler) that if, after n variables have been eliminated, an inequality depends on more than n+1 of original inequalities it is redundant.

- + + + -

3/38Z 50 15Z 30 43Z

- -- - + - + -x3+

12

Can choose which variables to project out and order in which to do so.

13

EXAMPLE OF PARTIAL EXAMPLE OF PARTIAL PROJECTIONPROJECTIONBenders’ Decomposition

Partially project out some variables (for MIP, continuous variables) to give bound on objective (Benders Cut)

Potentially very large increase in constraints.

Performed partially and iteratively.

14

Application of Projection to Non-Application of Projection to Non-Exponential Formulations of the Exponential Formulations of the

Travelling Salesman ProblemTravelling Salesman Problem

Example

Sequential Formulation (Miller, Tucker and Zemlin)

1,1

1

1

),...,3,2(

visitediscity in which number sequence

tourofpart 1

jinxnuu

ix

jx

ni

u

jiiffx

ijji

jij

iij

i

ij

Constraints and Variables

)(0 2n

15

Project out iu variables

Gives n

SSx

Sjiij

,

all directed cycles

A relaxation of Conventional Formulation

all proper subsets

1,

Sxsji

ij

nS ...,2,1

nS ...,2,1

16

Other non-exponential formulations give Other non-exponential formulations give modifications of subtour elimination modifications of subtour elimination constraints of (exponential) constraints of (exponential) conventional formulationconventional formulation

Single Commodity Flow Formulation gives

nSn

SSx

Sjiij ,...,2,1 all

1,

Modified Single Commodity Flow Formulation

11

1

11

n

SSxx

nSjSi

SjSi

ijij

17

1,

SxSji

ij

Multi Commodity Flow Formulation

i.e. of equal strength to Conventional Flow Formulation

Time Staged Formulation

11

1

1

1

,11

n

SSxx

nx

n Sjiij

SjSi

SjSi

ijij

18

  Minimise 2y1 + 3y2    

           

subject to: -y1 + y2 4

  y1 + y2 5

  -y1 + 2y2 3

 y1, y2 0

INVERSE PROJECTION Apply the dual procedures to eliminate constraints (as opposed to variables).

-

19

Write in form 

                   

subject to: 4y0 - y1 + y2 - y3 = 0

  -5y0 + y1 + y2 - y4 = 0

  -3y0 - y1 + 2y2 - y5 = 0

  y0             = 1

 y1, y2, y3, y4, y5 0

Minimise 2y1 + 3y2           

 Eliminate homogeneous constraints by adding columns (in suitable multiples) where coefficients have opposite sign.

20

Implemented by a transformation of variables.

4y0

y2y3

y14u1

4u2

u3

u4

 

First homogeneous constraint vanishes.

21

Substitute 4y0 = 4u1 + 4u2

  y1 = 4u1 + u3

  y2 = u3 + u4

  y3 = 4u2 + u4

22

Model becomes 

Minimise 8u1 + 5u

3

+ 3u

4

             

                         

subject to - u1

+ 5u

2

+ 2u

3

+ u4 - y

4

= 0  

  - 7u1

- 3u

2

+ u3 + 2u

4

- y

5

= 0  

  u1 + u2             = 1  

 u1, u2, u3, u4, y4, y5 0

23

Repeat with transformations to eliminate second and third homogeneous constraints. Results in Minimise 43w1 + 5w2 + 38/3w3 + 15w4 + 3w5    

  w1     + w3 + w4     = 1

                       

w1, w2, w3, w4, w5 0

24

Applying corresponding transformations to identity matrix gives relation between final and original variables 

 

y0 1 0

1 1 0

y1 11 1 7/3

0 0

y2 7 18/

35 1

y3 0 013/

3 9 1

y4 13 20 0 1

y5 0 1 0 7 2

w1 w2 w3 w4 w5

25

Gives all vertices and extreme rays of model 

    y1 y2  

C: w1 = 1 gives (11 7) Objective = 43

  w2 gives extreme ray ( 1 1)  

B: w3 = 1 gives (7/3

8/3) Objective = 38/

3

A: w4 = 1 gives ( 0 5) Objective = 15

  w5 gives extreme ray ( 0 1)  

26

NB: Transformations mirror those of (Primal) Projection.

Redundant transformations if variable depends on more than n+1

of original variables after n constraints eliminated.

27

Example of Inverse Projection Example of Inverse Projection (On some constraints)

Dantzig-Wolfe Decomposition

.

.

.

28

Project out subproblems into single (convexity) constraints

1111….1

1111….

.

.

.

1111….1

29

Also Modal Formulations

i.e. Variables represent extreme modes of processes rather than quantities

NB Inverse Projection is not the same as Reverse Projection (not well defined).

30

INTEGER PROJECTIONINTEGER PROJECTION

0..

0

0

agbfeiagbf

agabxbf

gbx

fax

a, b positive integers

x an integer variable

f and g linear expressions

Need to state condition “a multiple of ab lies between bf and –ag”

The Elimination of Integer Variables

31

Can be done in a finite way by

]1-t, . . . 0,1,2,t

b) (mod 0,0

1-a, . . . 0,1,2,s

a) (mod 0,0

i.e.

1-a, . . . 0,1,2,s

a)(mod0,

tgatagbf

sfbsagbf

sfagbsbf

Presburger Arithmetic I.e. Arithmetic “without multiplication”)

Alternatively

32

Projection of an IP may not produce an IP in lower dimension

e.g. Project out x2

1

1

7

(mod3)

0,1,2

x u

x u

u

33

But if coefficient of x is unity in one of inequalities can apply F-M elimination

gaf

Z

gaxafx

gaxZ

gfxfx

:

sexpressioninteger

,:

NB The projection of an IP generally does not result in an IP in a lower dimension

34

INTEGER PROJECTION

Example Minimize z

i.e.

2 1 2

2 1 2

2 1 2

2 1 2

2 1 2

2 1 2

5 4 5 10 2 2

5 2 5 30 6 2

5 3 10 45 9 2

3 4 5 6 2 15

2 5 6 2 15

2 3 10 18 3(2 15)

x x z x

x x z x

x x z x

x x x

x x x

x x x

1 2

1 2

1 2

1 2

1 2

5 2 0

2 4 5

6 2 5

6 2 15

9 3 10

z x x

x x

x x

x x

x x

35

Eliminate 1x

2

2

2

2

2

2

2

2 16 25 2

6 2 25 6

9 33 50 9

14 0

4 10

0 65

2 (mod5) 0,..., 4

2 3 (mod 6) 0,...,5

z x u

z x u

z x u

x v

x v

v

z x u u

x v v

36

Eliminate x2 (taking into account congruence relations)

50 225 50 8

210 25 210 16

126 700 126 33 14

12 40 12 2

36 530 36 33 4

42 175 42 7

0 -65+v

z u w

z u s

z u v s

z u v w

z u v s

z u v w

37

Optimal Solution5,5,3,0

3,0,6 21

swvu

xxz

Optimal Solution a Lattice Point within Polytope. Reduces problem from an infinite number to a finite number of solutions.

1(mod2)

v 0(mod2)

v 2(mod3) 0,..., 4

0(mod5) 0,...,5

2(mod3) 0,..., 29

0(mod5) 0,...,989

6 5 7 (mod9)

4 7 2 1(mod11)

v

w

w u

w v

s w

s s

v s

z u s

38

C1

C2

C3

39

Inverse Integer Projection Inverse Integer Projection

0integer,,

1

01039

01526

0526

0542

654321

621

521

421

321

xxxxxx

y

yxxx

yxxx

yxxx

yxxx

Example Minimize z 21 25 xx

Subject to:

40

24x

3x

1u

2u

3u

12x

5y

41

Eliminate constraint 1 by a transformation of variables

1 2 3

1 1 2

2 2

3

1

2

3 1 3

1 2 3

1

3

1

21

(NB: , , not sign constrained)4

0 ( not sign constrained)

1

2 0 (mod 2)

0 (mod 4)

0 (mod 5)

x u u u

x u x x y

x u u u u

y u

u u u

u

u

42

Leads to:

Minimise 321 5542

1uuu

Subject to:

Eliminate constraints 2,3,4

1 2 3 4

1 2 5

1 2 3 6

3

1 2 3

1

4 5 6 1 3

5 6 4 2 0

7 6 2 0

21 18 26 4 0

5

0 (mod2)

0 (mod4)

, , , 0 ( , not sign constrained)

u u u x

u u x

u u u x

u

u u u

u

x x x u u

=

=

=

43

Leads to:

Minimise

Subject to:

0,,,

(0264923

(03969723

209300652342

4321

421

4321

432

wwww

www

wwww

www

83720) mod

251160) mod

Optimal LP Solution. w4 = 3220 Objective = 4.5

N.B. Could replace congruence conditions by Mixed Integer Constraints.

Feasible Solutions defined by Convexity Constraint + Congruence Conditions

Illustrates Hilbert Basis result

1 2 3 4

1253 259 345 351

251160w w w w

1 2 3

1 2

Optimal IP Solution. 630, 4830, 280 objective 6

giving 0, 3 (at P)

w w w

x x

1 2giving 0, 3 (At P)x x

44

Alternatively we could scale variables and write in form

14

13

12

11 1624501853036

9041760

1wwww Minimise

Subject to:

corresponds to the extreme ray

1 1 1 11 2 3 4

1 11 2

1 1 1 11 2 3 4

11

1 1 12 3 4

11

12

13

14

4550 16380 16380 0 (mod 32760)

15925 0 (mod 10920)

, , , 0

and , , to vertices

Optimal Solution: 1890

0.9692

(Objective 6)

0.0307

0

w w w w

w w

w w w w

w

w w w

w

w

w

w

1 1 12 3 4 1w w w

45

Projection and

Sign Patterns

+ -

+ -

: :

+ or -

0 0

0 0

: :

0 0

Can remove columns and all constraints in which non-negative entry.

+ + + 0 0 0 -

- - - 0 0 0 +

Can remove constraints and all variables in which non-negative entry

46

0

0

0

Can add constraints in suitable multiple and remove variable

Eg Network Flow

can remove node

Integer Variable (and other variables integer)

1 -1

1 -1

1 -1

+

or +

: :

+

0 0

0 0

: :

0 0

Can regard variable as continuous

And generalisations of above

47

REFERENCESREFERENCES

1. K.P. Martin Large Scale Linear and Integer Optimization : A Unified Approach, Kluwer 1999

2. Porta Version 2, Free Software Foundation, Boston 1991

3. H.P. Williams (1986) Fourier’s Method of Linear Programming and its Dual, Am. Math, Monthly, 93, 681-94

4. H.P. Williams (1976) Fourier-Motzkin Extension to Integer Programming Problems, Journal of Combinatorial Theory, 21, 118-123

5. H.P. Williams (1983) A Characterisation of all Feasible Solutions to an Integer Programme, Discrete Applied Mathematics, 5, 147-155