1Conditions for optimality

ProblemNecessary conditions

for optimalityAlso sufficient if:

One-variable unconstrained f (x) concave

Multivariable unconstrained f (x) concave

Constrained, nonnegative

constraints onlyf (x) concave

General constrained problem Karush-Kuhn-Tucker conditionsf (x) concave and gi(x)

convex ( j = 1, 2,..., n)

= 0df

dt

= =

0, 1,2, ,

i

fj n

x

= =

=

0, 1,2, ,

(or 0, if 0)i

j

fj n

xx

Joo Miguel da Costa Sousa / Alexandra Moutinho 396

Karush-Kuhn-Tucker conditions

Theorem: Assume that f(x), g1(x), g2(x),..., gm(x) are differentiable functions satisfying certain regularity conditions. Then

x = (x1*, x2

*,..., xn*)

can be an optimal solution for the NP problem if there are m numbers u1, u2,..., um such that all the KKT conditions are satisfied:

1.

2.

=

=

= = =

1*

*

1

0

at , for 1,2 , .

0

mi

i

ij j

mi

j i

ij j

f gu

x xj n

f gx u

x x

x x

Joo Miguel da Costa Sousa / Alexandra Moutinho 397

Karush-Kuhn-Tucker conditions

3.

4.

5.

6.

Conditions 2 and 4 require that one of the two quantities must be zero.

Thus, conditions 3 and 4 can be combined:

(3,4)

=

=

*

*

( ) 0for 1,2, , .

[ ( ) ] 0

i i

i i i

g bj m

u g b

x

x

= 0, for 1,2, , .iu j m

= * 0, for 1,2, , .jx j n

= = =

*( ) 0

(or 0, if 0), for 1,2, , .i i

i

g b

u j m

x

Joo Miguel da Costa Sousa / Alexandra Moutinho 398

Karush-Kuhn-Tucker conditions

Similarly, conditions 1 and 2 can be combined:

(1,2)

Variables ui correspond to dual variables in linear programming.

Previous conditions are necessary but not sufficient

to ensure optimality.

=

=

= =

1*

0

(or 0 if 0), for 1,2 , .

mi

i

ij j

j

f gu

x x

x j n

Joo Miguel da Costa Sousa / Alexandra Moutinho 399

Karush-Kuhn-Tucker conditions

Corollary: assume that f(x) is concave and g1(x), g2(x),..., gm(x) are convex functions, where all functions satisfy the regularity conditions. Then, x =

(x1*, x2

*,..., xn*) is an optimal solution if and only if all

the conditions of the theorem are satisfied.

Joo Miguel da Costa Sousa / Alexandra Moutinho 400

Example

= + +1 2Maximize ( ) ln( 1)f x xx

1 20, 0x x

subject to

+ 1 22 3x xand

Thus, m = 1, and g1(x) = 2x1 + x2 is convex.

Further, f(x) is concave (check it using Appendix 2).

Thus, any solution that verifies the KKT conditions is an optimal solution.

Joo Miguel da Costa Sousa / Alexandra Moutinho 401

2Example: KKT conditions

1. (j = 1) (j = 2)

2. (j = 1) (j = 2)

3.

4.

5.

6.

+ 11

12 0

1u

x

= +

1 1

1

12 0

1x u

x

11 0u

( ) =2 11 0x u

+ 1 22 3 0x x

+ =1 1 2(2 3) 0u x x

1 20, 0x x

1 0u

Joo Miguel da Costa Sousa / Alexandra Moutinho 402

Example: solving KKT conditions

From condition 1 (j = 2) u1 1. x1 0 from condition 5

Therefore,

Therefore, x1 = 0, from condition 2 (j = 1).

u1 0 implies that 2x1 + x2 3 = 0 from condition 4.

Two previous steps implies that x2 = 3.

x2 0 implies that u1 = 1 from condition 2 (j = 2).

No conditions are violated for x1 = 0, x2 = 3, u1 = 1.

Consequently x* = (0,3).