1

Valery I. Zorkaltsev,

Professor,Head of Laboratory,

Energy Systems Institute Siberian Branch of the Russian Academy of Sciences

E-mail: zork@isem.sei.irk.ru

International conference “Optimization and applications"

Montenegrio2009 г.

SYMMETRIC DUALITY IN OPTIMIZATION AND IT’S APPLICATIONS

2

For wide class of optimization problems they use special constructions called Dual optimization problems:

,

where

primal optimization problem;

dual optimization problem;

transition rule (often polysemantic).

*LL

L

*L

Definition of symmetric duality

3

For dual problem one can specify problem dual to it

Symmetric duality is event, when dual problem to dual problem coincides with primal problem

.*** LL

.** LL

4

Applications of dual problems:

• to prove optimality of obtained solutions;

• for justification of optimization algorithms;

• in solution interpretation;

• for making optimization algorithms;

• for researching and solving many complicated problems of operation research, including Nash equilibrium finding.

5

Lecture plan

1. Theory of symmetric duality in optimization:– Lagrangian multipliers;

– Theorems of alternative systems of linear inequalities;

– Legendre-Fenchel conjugate functions and their extensions.

2. Application of symmetric duality in optimization algorithms and regularization

6

Lecture plan (continuation)

3. Application in models:– load-flow models (electric circuits, hydraulic

circuits, nonlinear transportation problems);

– models of thermodynamic equilibrium and geometric programming;

– economic equilibrium models.

7

1. Lagrange multipliers of constraintsPrimal problem:

(1)

Lagrange problem:

where Lagrange multipliers, which satisfy conditions (1).

Modified Lagrange problem (Sh. Churkveidze):

Xx

n

iii xfxf

min)()(

10

.,...,1,0)(

min)(0

nixf

xf

i

Xx

i

Xx

n

iii xfxf

min))((

21

)(1

20

8

2. Theory of alternative systems of linear inequalities

Any system of linear inequalities can be confronted with an alternative system of linear inequalities S by formal rules

so that proposition is right: One and only one system of two is consistent: S or S*.Moreover, backward transformation takes place:

and .

That is alternative systems S and S* are symmetric.

*SS

*** SS SS **

9

Three examples of theorems of alternative systems of linear inequalities

It is assigned: А – matrix , b – vector in Rm. Sought vectors – , .

Remark. System of linear equations can be considered as special case of system of linear inequalities. The converse proposition is not correct.

nmnRx mRu

10

1. Fredgolm’s theorem (about alternative systems of linear inequalities)

Either there is

or there is

nRx,bAx

mRu

.1,0 ubuA TT

11

2. Farkas’ theorem

Either the following system possesses a solution

or the following system is solvable

,0, xbAx

.0,0 ubuA TT

12

3. Gail’s theorem

Either there is such, that

or there is vector such, that

nRx

,bAx mRu

.1,0,0 ubuuA TT

13

Applications of alternative systems of linear inequalities theory

1. Identification of system of linear inequalities incompatibility

– If a vector from the solution set of an alternative system S* will be obtained during the process of searching the solution of system S, then absence of the solution of initial system S will be proved. We have practical and effective (as computation has shown) method for identification of problem constraints inconsistency.

14

2. For determination of redundant constraints, exclusion of which doesn’t change the solution set, including situations in algorithms

– Gomory or Kelly cuts;– Fourier-Chernikov convolutions for description of

systems of linear inequalities solutions.

3. For identification of solutions of systems of linear inequalities with minimal set of active constraints – relative to interior points of systems of linear inequalities solution set.

15

4. For creation of new algorithms for solving systems of linear and on the basis of this nonlinear inequalities («Alternative approach», which is developed by U. Evtushenko, A. Golicov).

5. All theory of linear optimization duality is contained in theorems of alternative systems of linear inequalities. Duality of linear optimization is the basis for wide class of nonlinear problems.

16

Search of solutions and identification of inconsistence of system of double-sided linear inequalities

• The more restricted class of problems is considered the more interesting results about characteristics of this class of problems can be obtained

• Initial system: find satisfying the following conditions

• Alternative system of one inequality: find , such that

where

Here for vectors , have components:

nRx

xxxbAx , )(S

mRu,0)( u

.)()()( uAxuAxubu TTTTTmRy )(y )(y

},,0max{)( ii yy ....,,1},,0min{)( miyy ii

)( *S

17

Comparison of variants of interior point methods for problems of permissible regimes of electric power systems

Algo-rithm

Number of iterations for problems

inconsistent consistent

6*7 40*80 2*7 19*19 201*201

A 1 10 7 23 116

B 1 15 6 24 107

C 1 1 16 13 28

D 1 1 5 5 8

E 1 4 26 24 88A, B – primal algorithms C, D – dual algorithms

Е – primal-dual algorithm

18

Mutually dual problems of linear programming

Let be sets of optimal solutions of problems (P), (P*).

Let’s introduce sets of recession directions for this problems :

,

According to Farkas and Geil theorems pairs of sets and are alternative. Symmetric duality takes place for LP problems:

*)(P

)(P

UX ,

0,0* vbvA:RvU TTm

.*)*( PP

*, UX*, XU

,min, XxxcΤ ,max, UuubΤ

,0,: xbAxRxX n

.0)(: uAcugRuU Tm

.0,0,0:* scsAsRsX Tn

19

Theorem of duality for LP

Four events are possible for problems (Р), ( ) :

1. If , then , ,

, .

2. If , . Тогда , ,

, .

3. If , . Тогда , ,

, .

*P

X X U

X U

U

X U

*X

*U

*U

*X

*X *U

X U

X U

20

Theorem of duality for LP (continuation)

4. If , . Then , ,

, .

For any ,

There is , such that

In this and only this case .

X U U

Xx Uu .,...,1,0u njgx jj

*X *U

Xx

X

Uu .,...,1,0u njgx jj

U,X riurix

21

Equivalent representations of LP problem in the form of optimization problems

1. Primal problem

2. Dual problem

3. Self-dual problem

4. Symmetric problem (problem of complementa-rity)

.min, XxxcΤ

.max, UuubΤ

.,min, UuXxubxc ΤΤ

.,min,)(1

UuXxugxn

jjj

22

Reresentation of a linearprogramming problem as a system

of linear inequalities

,0 ubxc ΤΤ

,0, xbAx

.cuAT

It allows to consider problems of linear programming as

a special case of systems of linear inequalities.

23

3. Conjugate functions

for

1. Functions is Legendre conjugate of each other if

where

That is

., nRyx

,1 f

).(),( xFfx

.,))(())(( nRxxxfxf

)(),( yxF

)(),( yxF

24

3. Conjugate functions

2. Functions is Fenchel conjugate of each other, if

and},)({max)( xFyxy T

x

}.)({max)( yyxxF T

y

)(),( yxF

25

3. Generalization of conjugate functions of Legendre-Fenchel

3. Functions is conjugate of each other, if

where symmetric positive defined matrix.Following functions are mutually inverse

Following inequality is held

)(),( yxF

},)({max)( xFRyxy T

x

}.)({max)( yRyxxF T

y

R

).(),( 11 xFRfxR

.,)()( nT RyxRyxyxF

26

Symmetric duality

1. Primal problem (S)(S)(1)

2. Dual problem (S*)(S*)(2)

Note: problems (S) and (S*) have different structure of variables; dual to dual problem coincide with primal problem .

.0,

min,)(

xbAx

xFxcΤ

.

min,)(

cyuA

ubyT

Τ

SS **

27

Symmetric duality (continuation)

3. Self-dual problem: subject to (1), (2)

4. Symmetric problem: subject to (1), (2)

.min)()( yxFubxc ΤΤ

.min)()( yxyxF T

)(SD

)(SS

28

Symmetric duality. Equivalent system of equalities and inequalities

,0, xbAx

,cyuAT .0)()( yxFubxc ΤΤ

Constraint (3) can be substituted with еquality

)(

)3(

)2(

)1(

L

.0)()( yxyxF T )4(

29

ExamplesI. Symmetric duality for problems of

quadratic programming

where positive definite matrix

Mutually dual problems

Q.0,0 xQxxT

,2

1)(,

2

1)( 1 yQyyQxxxF TT

;0,min,)( xbAxxcxF T

.max,)( cyuAyub TT

30

ExamplesII. Especially important case of

separable functions

One form of writing the equivalent system of equalities and inequalities

.)()(,)()(11

n

jj

n

jjj yyxFxF

,0, xbAx

,

cuAy T

).(),( yxxfy

31

Theorem (for separable )

Let fj be continious increasing functions, then (S) is a problem of minimization of strictly convex function with linear constraints, (S*), (SD) are problems of minimization of convex function with linear constraints.

If, at the same time, and a system Ax=b, is consistent, then problems (S), (S*), (SD), (SS), (L) have coincident and unique (relatively to vectors x, y) solutions. Vector u is unique if rank A=m.

)(),( yxF

0x,0)0( jf )(jf

32

Applications, separable case

1. Regularization of linear programming problems: having small

Primal problem: regularization by Tihonov

Dual problem: search of pseudosolution of dual problem of linear programming

0

,)(1

2

n

jjxxF .

1)(

1

2

n

jjyy

.0,min,)(1

2

xbAxxxcn

ii

T

.min)(1 2

cuAub TT

33

Applications(with and self-conjugated functions of

the kind )

2. «Alternative» way of searching for normal solutions for system of inequalities with n variables

This is equivalent to problem with m variables

Such approach is preferable when

,0, xbAx .minx

,min,2

mTT RuubuA

.nm

0c

n

iixxxF

1

2)()()(

34

Applications

3. Load-flow models (nonlinear transportation problems, electric circuits, hydraulic circuits including heat, water and gas delivery problems)

indices of nodes,

indices of arcs,

incedence matrix,

vector of volumes of delivery in system and out of system,

mi ,...,1

nj ,...,1

A

b

35

Load-flow models

vector of pressure gains (or electro- motive forces, or conveyance tariffs) on arcs,

vector of flows on arcs, vector of pressures (tensions, prices) in

nodes, vector of pressure losses (tension losses,

price rises) on arcs.

c

xu

y

36

Load-flow models

flow balance in nodes (first Kirchhoff law),

balances of pressures on arcs,

interrelations of pressure loss and flow on arcs.

For example, Ohm law,

Darcy law.

bAx

)( cuAy T

)(),( yxxfy

jj

jjjj yr

xxry1

,

2/1

2 1,)( j

jjjjj yxxy

37

Results for hydraulic circuits obtained using the theory of

symmetric duality1. Conditions for existence and uniqueness of

classical load-flow model solution are clarified.2. Possibilities for choosing the form of

mathematical models representation are expanded.

3. Foundations for constructing and theoretical justification of algorithms for solving load-flow problems are obtained.

38

Results for hydraulic circuits obtained using the theory of symmetric duality

4. Theoretical research is held (including clarification of conditions for existence and uniqueness), algorithms for solving non-classical load-flow problems are developed, where some components of vectors x, y, u, b and c may be fixed аnd other components of these vectors should be found.

39

Transport model with piecewise defined nonlinear costs

• Model is applied in analysis of operation of natural gas and oil delivery systems to find and eliminate bottlenecks in proper time.

• Let be flow through the arc j,

− costs coefficient for the arc j, ,

− nonlinear function.

For each arc costs function will be

jsjx

0js

jF

),(~

)( jjjjjj xFxsxG

40

where

for

.),(

,,0)(

~

jjjjj

jjj

jj xxxxF

xxxxF

.,0 jjj xxx

jxjx

costs

flow

Normal regime

Extremal regime

)( jj xG

414141

Primal optimization problem

• bi – volume of delivery into the net at source node (if ) or out of the net at consumer node (if ),

• hi – penalty for incomplete delivery in node i, • Isrc – set of numbers of source nodes,• Icons – set of numbers of consumer nodes.

,0 bAx

min)())(~

(1

n

j Iiiiijjjj

cons

bbhxsxF )1(

)5(

,,...,1, njxx jj )2(

)3(

)4(,,0 srcii Iibb

,,0 consii Iibb

srcIi consIi

4242

Economic interpretation

jj sy

0 jx

jjj xsy )(

jjjj yxy )(~

jjjj xsxF )(

– profit (surplus) of transport company on arc j

– transportation costs on arc j– revenue from transportation on arc j

jjj sxf )(~

jjjj yxy )(~

jjjj xsxF )(~

jx

js

j arc

through flow

j arcon tariff

Fig. 1. Plot of marginal costs

4343

Calculation experiments results

Table 1. Results of computations

• Results of calculations for method of interior points on number of example networks

(Number of nodes,

number of arcs)

Amount of iterations of

interior points method

Time of compu-tation,

sec

Achieved accuracy of

equality constraints

Achieved accuracy of optimality conditions

(25, 30) 23 0.400 1.1053*10-7 0.00190019

(50, 67) 39 0.631 1.61968*10-10 0.00702529

(75, 109) 48 2.254 2.33416*10-8 0.00256343

(100, 116) 68 6.850 1.48463*10-7 0.00640007

(200, 240) 93 77.919 1.41919*10-5 0.00021684

44

Diagrams for computation results

Number of iterations

Amount of variables

Time of com-putation, sec

Amount of variables

Results of calculations is shown on two diagrams

4545

Problem of finding bottlenecks in natural gas delivery network in order to obtain system

reliability

• Two examples were computed for real networks:– Aggregated network for natural gas delivery system

(21 nodes, 28 arcs)

– Detailed network for the same system (337 nodes, 589 arcs)

• Two aims of computation for each example: – 1) to determine nodes with low supply and arcs with

utilized capacity when only normal regime is allowed

– 2) to determine abilities to increase supply of nodes with low supply and find arcs switched to extremal regime when extremal regime is allowed

4646

Aggregated network. Only normal regime is allowed

Aggregated network. Extremal regime is allowed

,, xxxbAx

min,)(1

iIi

ii

n

jjj bbhxs

cons

,,0 srcii Iibb

consii Iibb ,0

,, xxbAx ,,0 srcii Iibb

consii Iibb ,0

min)())(~

(1

n

j Iiiiijjjj

cons

bbhxsxF

Linear load-flow

Nonlinear load-flow

4747

Detailed network for natural gas delivery system

Number of nodes: 337 Number of arcs: 589Amount of iterations of interior points method: 82Time of calculation: 649.359 sec

48

Final word

• I’d like to give thank to people who helped me make this report:– Perjabinsky Sergey,– Medvezhonkov Dmitry.

• Thank you for your attention!