Linear Programming and Smoothed

Complexity

Richard Kelley

The Nevanlinna Prize

Has anyone heard of the Nevanlinna prize?

What about the Fields Medal?

Godel prize?

About the prize

Awarded at the International Congress of Mathematicians. Once every 4 years.

You have to be younger than 40.

Awarded for outstanding contributions in “information sciences” Mathematical aspects of computer science Scientific computing, optimization, and computer

algebra.

2010 Winner

Daniel Spielman Professor of computer science at Yale.

Spielman showed how to combine worst-case complexity analysis with average-case complexity analysis. The result is called “smoothed analysis.” Spielman used smoothed analysis to show that

linear programming is “basically” a polynomial-time algorithm.

Worst-Case Thinking

Think of this as a two-player game.

One player presents an algorithm.

The other player, the adversary, selects an input to the algorithm. The adversary wants the algorithm to run slowly.

The worst-case complexity is the running time of the algorithm on an input chosen by the best possible adversary.

Average-Case Thinking

Also a game, but against a cold and indifferent Nature.

Player one presents an algorithm.

Nature chooses an input. At random. What should “random” mean?

Because the input is random, the running time becomes a random variable.

The expected value of this random variable is the average-case complexity.

Can we do better?

Yes!

Combine the two!

Let’s pick up some background motivation first…

Linear Programming

A form of constrained optimization. Common in the real world.

You want to maximize a linear function.

You have a set of linear constraints.

Your variables are required to be nonnegative.

Important in the history of computer science. Developed during WWII for Army planning. State secret until 1947.

For Example…

max 3x + 5y

Subject to 5x + 7.2y <= 4

x + y <= 8

x, y >= 0

And in general

Linear Programming Geometry

Problems that reduce to linear programming

Economics Utility maximization, cost minimization, profit maximization, game theory (Nash equilibria).

Graph Theory Matching

Connectivity

Graph coloring

Maximum flows

Minimum cuts

Spanning trees

Scheduling and allocation.

Geometry (Convex Polytopes)

Sorting!!

Linear programming is “P-complete”

Example: Sorting

How is this an optimization problem? The biggest element in the array should have the

biggest index. The second biggest element should have the second

biggest index. Etc.

How is this a constrained optimization problem? No element should be duplicated (assuming unique

elements). No index should contain more than one element.

Sorting: The Linear Program

Solving Linear Programs

Even though we’re working in a continuous space, this is a discrete problem. Why?

The basic idea is to walk along the vertices of the feasible region, climbing to vertices that have better and better values for the objective function.

The Simplex Algorithm

Start at some point the feasible region. The vertex consisting of all 0’s usually works.

Look at all the neighboring vertices, find one with a higher objective function value. Involves keeping track of a set of “basis variables” Variables go in and out of the basis depending on

whether or not they make the objective function bigger.

Repeat until you can’t improve the objective value any more. “Easy” to show that you’re done.

More Geometry

Complexity?

How should we talk about the “size” of an instance of the linear programming problem?

Any guesses how long it should take to run the simplex algorithm?

Usually, it’s pretty quick. With n variables and m constraints, it takes about 2m iterations

In theory, this is a O(2^n) algorithm. Not a typo.

This is a perfect example of the difference between theory and practice!

Worst Case

We want to force the simplex algorithm to look at exponentially many vertices before finding the best one.

The trick is to use a (hyper-)cube.

Worst Case: Klee-Minty

The “Real World”

Huge LPs are solved all the time. And the simplex algorithm is usually linear.

This is one of the only exponential algorithms that is used widely in practice.

Longstanding Open Problem

For a long time, nobody had a clue.

The solution was pretty much worked out by 2005.

The Solution

Smoothed Analysis. Start with a arbitrarily chosen input

Could be from the adversary. “Shake it” a little bit.

The analysis then depends on the size of the input and the magnitude of the shaking.

The idea is that an algorithm with a low smoothed complexity is “almost always” well-behaved.

In Symbols

Graphically – Worst case Complexity

Smoothed Analysis

Questions?