introduction to computational complexity - math.sjtu.edu.cn · computation is everywhere...

Introduction to Computational Complexity

Jiyou [email protected]

Department of Mathematics, Shanghai Jiao Tong University

Sep. 24th, 2013

Computation is everywhere

Mathematics: addition; multiplication; functions; integrals;solving Diophantine or differential equations; provingtheorems...

Computer Science: scientific computing; algorithms and datastructures; information processing; artificial intelligence...

Physics & Biology: dynamical systems such as weatherevolution and SARS infectio; fetal development; emotionalreactions (such as music listening)...

Math Computer

Science

Biology , Physics...

Theory

of Computing

Fundamental Problems in Computation

1. Will a virus in an evolution system spread, or die out?2. What are the principles behind the development of life?3. Is there a program to solve all equations?4. Is there a program to prove all provable theorems?5. Which problems can be computed?6. Given any C-program, can we check if it will halt?

Algorithms in Mathematics

1. Inputs and outputs;2. Step-by-step;3. Finite language;4. Compute on every possible input.

History of Algorithms

B.C. 300: Euclid [proofs and algorithms: GCD algorithm ]A.D. 900 : al-Khwãrizmi [arithmetic]A.D. 1940: Turing [defined algorithms]

Alan Turing (1912-1954)

Father of Computing

1936: /On computable numbers, with an application to theentscheindungsproblem0, Foundations of Computer Science;1939-1945: Blechley Park, breaking Enigma;

1950: /Computing machinery and intelligence0.

Turing memorial statue plaque

Formal Languages

1. Σ = {σ1, σ2, . . . , σk} is a finite set of symbols;2. Σ∗ is the set of all words over Σ;3. A language L is a subset of Σ∗.

Definition of the Turing Machine

A Turing machine M is a five-tuple M = (Q; Γ; δ; q0; F )where

Q is a finite set of states;Γ is the tape alphabet including the blank � ∈ Γ;q0 ∈ Q is the initial state;F ⊂ Q is the set of final states;δ is the transition function:

δ : (Q − F )× Γ −→ Q × Γ× {R, N, L}.


A Turing machine M is a five-tuple M = (Q; Γ; δ; q0; F )whereQ is a finite set of states;

Γ is the tape alphabet including the blank � ∈ Γ;q0 ∈ Q is the initial state;F ⊂ Q is the set of final states;δ is the transition function:

δ : (Q − F )× Γ −→ Q × Γ× {R, N, L}.


A Turing machine M is a five-tuple M = (Q; Γ; δ; q0; F )whereQ is a finite set of states;Γ is the tape alphabet including the blank � ∈ Γ;

q0 ∈ Q is the initial state;F ⊂ Q is the set of final states;δ is the transition function:

δ : (Q − F )× Γ −→ Q × Γ× {R, N, L}.


A Turing machine M is a five-tuple M = (Q; Γ; δ; q0; F )whereQ is a finite set of states;Γ is the tape alphabet including the blank � ∈ Γ;q0 ∈ Q is the initial state;

F ⊂ Q is the set of final states;δ is the transition function:

δ : (Q − F )× Γ −→ Q × Γ× {R, N, L}.


A Turing machine M is a five-tuple M = (Q; Γ; δ; q0; F )whereQ is a finite set of states;Γ is the tape alphabet including the blank � ∈ Γ;q0 ∈ Q is the initial state;F ⊂ Q is the set of final states;

δ is the transition function:

δ : (Q − F )× Γ −→ Q × Γ× {R, N, L}.


A Turing machine M is a five-tuple M = (Q; Γ; δ; q0; F )whereQ is a finite set of states;Γ is the tape alphabet including the blank � ∈ Γ;q0 ∈ Q is the initial state;F ⊂ Q is the set of final states;δ is the transition function:

δ : (Q − F )× Γ −→ Q × Γ× {R, N, L}.

Configurations

Start configuration: (�; q0; w) where w is the input;End configuration: (v ; q; z) for q ∈ F where z is the output.

A Configuration

A state table

Current Scanned Print Action Next 5-tuplesA 0 1 R B (A, 0, 1, R, B)A 1 1 L C (A, 1, 1, L, C)B 0 1 L A (B, 0, 1, L, A)B 1 1 R B (B, 1, 1, R, B)C 0 1 L B (C, 0, 1, L, B)C 1 1 N z (C, 1, 1, N, z)

Table: Here Q = {A, B, C, z}, Γ = {0, 1} and z ∈ F .

The Turing Machine and Insights

Demo: http://morphett.info/turing/turing.html

Makes the concept of computation clear;Duality of program and input;The model is simple and can be easily simulated by a human;Turing machines are finite objects;Universality;The power of computation: computable functions;Church-Turing Thesis: Every function computable by anyreasonable device, is computable by a Turing machine;The limits of computation: uncomputable functions.

The Turing Machine and Insights

Algorithm: A Turing machine which halts in finite time on everypossible (finite) input. Machine M on input x computes M(x)Duality: A Turing Machine can be input to another TuringMachine.

The limits of computation

Does a given computer program P halt on all inputs?

Is a given computer program bug-free?Is a math statement provable?Is a given equation solvable?


Does a given computer program P halt on all inputs?Is a given computer program bug-free?

Is a math statement provable?Is a given equation solvable?


Does a given computer program P halt on all inputs?Is a given computer program bug-free?Is a math statement provable?

Is a given equation solvable?


Does a given computer program P halt on all inputs?Is a given computer program bug-free?Is a math statement provable?Is a given equation solvable?

Universal Turing Machine

TheoremThere exists a universal Turing machine U, such that for allTuring machines M and all words w ∈ Σ∗,

U(M; w) = M(w).

In particular, U does not halt iff M does not halt.

Computable functions

Interpret a Turing Machine M as a function

f : Γ∗ −→ Γ∗.

All such functions are called computable functions.

Acceptors

For decision problem L, suppose M halt in F (finite states),either state qyes for positive instances w ∈ L or state qno fornegative instances w ∈ L.We say M accepts the language L(M):

L(M) := {w ∈ Σ∗|∃a, b ∈ Σ∗, (ε; q0; w) `∗ (a; qyes; b)}.

L(M) := {w ∈ Σ∗|M accepts w}.

Decidability and Undecidability

A language L is called decidable, if there exists a TuringMachine with L(M) = L that halts on all inputs.

1. Turing: Halting Problem is undecidable;2. Turing: The question of if a math statement is provable isundecidable;3. Mattiasevich: The solvability of a Diophantine is undecidable;4. Conway: The problem of if a given epidemic will spread ordie is undecidable.

Non-deterministic Turing Machine

δ : (Q − F )× Γ −→ P(Q × Γ× {R, N, L}),

where P(X ) is the power set of X.

Given a non-deterministic Turing Machine N, one can constructa deterministic Turing Machine M with L(M) = L(N). Further, ifN(w) accepts after t(w) steps, then there is c such that M(w)accepts after at most ct(w) steps.

Complexity

What can algorithms do with restricted resources?How long can we solve a problem?How little disk-space can we use to solve a problem?What can we compute fast?

Computational Complexity

Runtime and required space;Hard and easy problems;The importance of efficient algorithms;The P vs. NP problem;The ubiquity of NP-complete problems;Proving vs. Verifying;Reductions, Hardness, Completeness.

P and NP

P: There is a deterministic Turing Machine M and a polynomialp(n) such that for each input w ∈ Σ∗ the running time ≤ p(|w |);NP: There is a nondeterministic Turing Machine M and apolynomial p(n) such that for each input w ∈ Σ∗ the runningtime ≤ p(|w |);

Complexity Classes for Deterministic Turing Machine

1. LINTIME: Linear time;2. P: Polynomial time:3. EXP: Exponential time;4. L: Logarithmic space;5. PSPACE: Polynomial space;6. EXPSPACE: Exponential space.

Complexity Classes for Non-deterministic Turing Machine

1. NLINTIME: Linear time;2. NP: Polynomial time:3. NEXP: Exponential time;4. NL: Logarithmic space;5. NPSPACE: Polynomial space;6. NEXPSPACE: Exponential space.

Complexity Classes Turing Machine

Theorem

LINTIME ⊆ P ⊆ EXP.

NLINTIME ⊆ NP ⊆ NEXP.

P ⊆ NP ⊆ PSPACE = NPSPACE .

Complexity Classes

Solvable NP

NP-complete

PLN

The equivalent definition of NP

1. A checking relation R ⊆ Σ∗ × Σ∗1;

2. Define a language

LR = {w#y | R(w , y)};

3. R is polynomial-time iff LR ∈ P;4. A language L over Σ is in NP iff there is k ∈ N and apolynomial-time checking relation R such that for all w ∈ Σ∗,

w ∈ L ⇐⇒ ∃y(|y | ≤ (|w |)k and R(w , y)).

P vs NP

Problems in P: solutions can be efficiently found;Problems in NP: solutions can be efficiently checked;Problems in NP-complete: The "hardest" NP problems.

Clay Math Institute Millennium Problems

Birch and Swinnerton-Dyer ConjectureHodge ConjectureNavier-Stokes EquationsP=NP?Poincare ConjectureRiemann HypothesisYang-Mills Theory

Examples in P

Matrix Multiplication (2.236); Determinant; LinearProgramming; Primality test; Many Graph Problems(Connectivity, Perfect Matching, Shortest Path)...

Examples in NP, not known to be in P

Factorization; Discrete Logarithm; Polynomial Identity Testing;Graph Isomorphism...

Theorem (Cook and Levin, 1970s)3-SAT is NP-complete.

Examples in NP-complete

3-SAT (The Satisfiability Problem); Subset Sum Problem;SuDoku; Theorem proving; Integer Linear Programming; ManyGraph Problems (TSP, Independent Set, Covering Set,Chromatic number)...

Exercises

1. Write a Turing Machine to compute the addition of twonatural numbers;2. Write a Turing Machine to compute the function f (x) = 3x ;3. Prove that both 1-SAT and 2-SAT are in P;4. Find more interesting examples according to complexityclasses.

introduction to computational complexity - math.sjtu.edu.cn · computation is everywhere...

Documents