everything you always wanted to know about theoretical computer science (but were afraid to ask)

Everything you always wanted to know about theoretical computer science

(but were afraid to ask)

Dimitris AchlioptasMicrosoft Research

Turing’s Machine

• Alphabet Σ, state space K

• f : K×Σ → K×Σ×{←,→,?} ×{Halt,Yes,No}

• Language: L µ Σ* is decided by ML

• Polynomial-time Robustness

Any reasonable attempt to model mathematically computer algorithms and their performance is bound to end up with a model of computation and associated time cost that is equivalent to Turing machines within a polynomial.

Church’s Thesis

Any reasonable attempt to model mathematically computer algorithms and their performance is bound to end up with a model of computation and associated time cost that is equivalent to Turing machines within a polynomial.

Church’s Thesis

Turing’s Idea

Programmable Hardware:

Describe f on the leftmost part of the tape.

Polynomial Time• N vertices• As,t distance from s to t

Find a spanning cycle of minimum length

Polynomial Time

• Edge lengths are written in binary. Lists vs. Matrices

• Assume M could decide if OPT<B in polynomial time– Do binary search to determine OPT – Probe edges one-by-one, replacing their cost with OPT+1

• N vertices• As,t distance from s to t

Find a spanning cycle of minimum length

P vs. NP

Polynomial Time PTIME = [k nk

L is in P if there exists a Turing Machine M which for every x, decides if x is in L in a polynomial number of steps.

P vs. NP

Polynomial Time PTIME = [k nk

L is in P if there exists a Turing Machine M which for every x, decides if x is in L in polynomially many steps.

Non-Deterministic Polynomial Time

L is in NP if there exists a Turing Machine M s.t. for every x • If x is in L then there exists w s.t. M(x,w)→“Yes” in PTIME.• If x is not in L then there is no such w.

Complexity

• Is there a spanning cycle of cost < C ? NP• Is there a spanning tree of cost < C ? P• Is there no spanning tree of cost < C ? P • Is there no spanning cycle of cost < C ? coNP

NPcoNP

P

Perebor

• 50-60s: Many optimization problems appear amenable only to brute force, i.e. (near-)exhaustive enumeration.

• Edmonds 1966: “The classes of problems which are respectively known and not known tohave good algorithms are of great theoretical interest […] I conjecture

thatthere is no good algorithm for the traveling salesman problem. My

reasonsare the same as for any mathematical conjecture: (1) It’s a legitimate mathematical possibility, and (2) I do not know.”

• Hartmanis, Stearns 1965:

“On the computational Complexity of Algorithms”, AMS Transactions

Input for Problem B

Output for Problem B

Algorithm for Problem B

Reductionfrom B to A

Algorithmfor B

x R(x) Yes/No

Reducibility

Input for Problem B

Output for Problem B

Algorithm for Problem B

Reductionfrom B to A

Algorithmfor B

x R(x) Yes/No

Shortest Spanning Path from s to t

Shortest Spanning Cycle

Reducibility

NP-completeness

A problem Π is NP-hard if every

problem in NP has a polynomial-time reduction to Π.

If Π is in ΝP and NP-hard then Π is NP-complete.

Moral: At least as hard as any other problem in NP

Cook’s Theorem

Satisfiability is NP-Complete

Idea:

Let variable Xp,t denote the content of tape-cell p after t steps

• By now thousands of problems have been shown NP-complete

• NP-completeness has migrated to other sciences, e.g. economics

Integer Linear Programming:

Ax·bIs there a satisfying vector x where all xi are non-negative INTEGERS?

NPcoNP

P TSP

PRIMALITY

More hard problems

Approximation

• Vertex Cover: Find a smallest set of vertices that touches every edge.– Algorithm: If an edge is still untouched, add

both its endpoints to the set.

Approximation

• Vertex Cover: Find a smallest set of vertices that touches every edge.– Algorithm: If an edge is still untouched, add

both its endpoints to the set.

• Max Cut: Partition the vertices of the graph to maximize number of cut edges.– Algorithm: Partition randomly.

Greed

Problem: Find a spanning tree of minimum length

Greed

Problem: Find a spanning tree of minimum length

Algorithm: Always add the cheapest allowable edge

Greed

Problem: Find a spanning tree of minimum length

Algorithm: Always add the cheapest allowable edge

Proof:

The cheapest edge emanating from each tree is always good.

Duality

Max-Flow = Min-Cut

Linear Programming:

Ax·bIs there a satisfying vector where all xi are non-negative rationals?

Linear Programming

Linear Programming:

Ax·bIs there a satisfying vector where all xi are non-negative rationals?

Linear Programming

An integer matrix A is Totally Unimodular (TUM) if every square, nonsingular submatrix B has det(B)=±1

Linear Programming:

Ax·bIs there a satisfying vector where all xi are non-negative rationals?

Linear Programming

An integer matrix A is Totally Unimodular (TUM) if every square, nonsingular submatrix B has det(B)=±1

If A is TUM then the LP solution is integral!

Linear Programming II

• Simplex method:– Most known rules can be made to take an

exponential number of steps.

• Ellipsoid algorithm:– Number of steps depends on size of integers

Is there an algorithm that solves LPs with m equations in n variables in Poly(m,n) steps?

Semidefinite Programming

Arrange n unit vectors so that their inner products satisfy a system of

linear inequalities.

Approximability

MAX CUT

• Assign a unit vector vs to each vertex s.• Maximize the sum of (1-vsvt)wst over all vertices• Take a random hyperplane through the origin.• Get an 0.87… approximation

Vertex Cover

• Conjecture: Cannot beat stupid algorithm!• Theorem: Need at least 1.36..*OPT

• Tool: NP = PCP(O(logn), O(1)).

Non-worst-case Inputs

• Random graphs

• Random pointsets

• Random formulas

Non-worst-case Inputs

• Random graphs

• Random pointsets

• Random formulas

Theoretical CS ProbabilisticCombinatorics

Non-worst-case Inputs

• Random graphs

• Random pointsets

• Random formulas

Theoretical CS ProbabilisticCombinatorics

Physics