Fall 2013 CMU CS 15-855Computational Complexity

Lecture 2

Diagonalization,

9/12/2013

9/12/2013 2

Languages• Let ∑ be any finite alphabet. • L µ ∑*

• L is a language.

• The decision problem corresponding to L is the function:

• f:∑* ! {yes, no} deciding membership in L, i.e., {x | f(x)=‘yes}

9/12/2013 3

Model for this class;Multi-tape Turing Machines

• Read-only Input tape; write only output . . . finite

controla b a b

a a

b b c d

. . .

. . .

k tapes

δ:Q x ∑k ! Q x ∑k x {L,R,-}k

(input tape)

9/12/2013 4

Turing Machines• Q finite set of states• ∑ alphabet including blank: “_”• qstart, qaccept, qreject in Q• δ:Q x ∑k ! Q x ∑k x {L,R,-}k transition fn.• input written on tape, head on 1st square,

state qstart

• sequence of steps specified by δ• if reach qaccept or qreject then halt

9/12/2013 5

TIME and SPACETIME(f(n)) = languages decidable by a multi-tape TM in at most f(n) steps, where n is the input length, and f :N ! N

SPACE(f(n)) = languages decidable by a multi-tape TM that touches at most f(n) squares of its work tapes, where n is the input length, and f :N ! N

9/12/2013 6

Time and Space Classes

L = SPACE(log n)PSPACE = k SPACE(nk)

P = k TIME(nk)

EXP = k TIME(2nk)

9/12/2013 7

Outline

• Why big-oh? Linear Speedup Theorem

• Hierarchy Theorems

• Robust Time and Space Classes

• Relationships between classes

• Some complete problems

9/12/2013 8

Linear SpeedupTheorem: Suppose TM M decides language L in

time f(n). Then for any > 0, there exists TM M’ that decides L in time

f(n) + n + 2.• Proof:

– simple idea: increase “word length”– M’ will have

• one more tape than M• m-tuples of symbols of M

∑new = ∑old ∑oldm

• many more states

9/12/2013 9

Linear Speedup

• part 1: compress input onto fresh tape

. . . a b a b b a a a

. . . aba bba aa_

9/12/2013 10

Linear Speedup

• part 2: simulate M, m steps at a timeb b a a b a b a a a b

. . . abb aab aba aab aba

. . . . . .

. . . m m

– 4 (L,R,R,L) steps to read relevant symbols, “remember” in state

– 2 (L,R or R,L) to make M’s changes

9/12/2013 11

Linear Speedup• accounting:

– part 1 (copying): n + 2 steps– part 2 (simulation): 6 (f(n)/m)– set m = 6/– total: f(n) + n + 2

Theorem: Suppose TM M decides language L in space f(n). Then for any > 0, there exists TM M’ that decides L in space f(n) + 2.

• Proof: same.

9/12/2013 12

Time and Space• Moral: big-oh notation necessary given our

model of computation– Recall: f(n) = O(g(n)) if there exists c such that f(n) ≤ c

g(n) for all sufficiently large n.– TM model incapable of making distinctions between

time and space usage that differs by a constant.• In general: interested in course distinctions not

affected by model– e.g. simulation of k-string TM running in time f(n) by

single-string TM running in time O(f(n)2)

9/12/2013 13

Hierarchy Theorems

• Does genuinely more time permit us to decide new languages?

• how can we construct a language L that is not in TIME(f(n))…

• idea: same as “HALT undecidable” diagonalization and simulation

9/12/2013 14

Recall proof for Halting Problem

Turing Machines

inputs

Yn

Yn

nY

n

Y n Y Y nn YH’ :

box (M, x): does M halt on x?

The existence of H which tells us yes/no for each box allows us to construct a TM H’ that cannot be in the table.

9/12/2013 15

Time Hierarchy Theorem

Turing Machines

inputs

Yn

Yn

nY

n

Y n Y Y nn YD :

box (M, x): does M accept x in time f(n)?

• TM SIM tells us yes/no for each box in time g(n)

• rows include all of TIME(f(n))

• construct TM D running in time g(2n) that is not in table

9/12/2013 16

Time Hierarchy Theorem

Theorem (Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n:

TIME(f(n)) ( TIME(f(2n)3).

• more on “proper complexity functions” later…

9/12/2013 17

Proof of Time Hierarchy Theorem

• Proof: – SIM is TM deciding language

{ <M, x> : M accepts x in ≤ f(|x|) steps }– Claim: SIM runs in time g(n) = f(n)3.– define new TM D: on input <M>

• if SIM accepts <M, M>, reject• if SIM rejects <M, M>, accept

– D runs in time g(2n)

9/12/2013 18

Proof of Time Hierarchy Theorem

• Proof (continued):– suppose M in TIME(f(n)) decides L(D)

• M(<M>) = SIM(<M, M>) ≠ D(<M>)• but M(<M>) = D(<M>)

– contradiction.

9/12/2013 19

Proof of Time Hierarchy Theorem• Claim: there is a TM SIM that decides

{<M, x> : M accepts x in ≤ f(|x|) steps}and runs in time g(n) = f(n)3.

• Proof sketch: SIM has 4 work tapes• contents and “virtual head” positions for M’s

tapes • M’s transition function and state• f(|x|) “+”s used as a clock• scratch space

9/12/2013 20

Proof of Time Hierarchy Theorem

• contents and “virtual head” positions for M’s tapes • M’s transition function and state• f(|x|) “+”s used as a clock• scratch space

– initialize tapes– simulate step of M, advance head on tape 3;

repeat.– can check running time is as claimed.

• Important detail: need to initialize tape 3 in time O(f(n))

9/12/2013 21

Proper Complexity Functions

• Definition: f is a proper complexity function if– f(n) ≥ f(n-1) for all n– there exists a TM M that outputs exactly f(n)

symbols on input 1n, and runs in time O(f(n) + n) and space O(f(n)).

9/12/2013 22

Proper Complexity Functions• includes all reasonable functions we will

work with– log n, √n, n2, 2n, n!, …– if f and g are proper then f + g, fg, f(g), fg, 2g

are all proper• can mostly ignore, but be aware it is a

genuine concern:• Theorem: 9 non-proper f such that

TIME(f(n)) = TIME(2f(n)).

9/12/2013 23

Hierarchy Theorems

• Does genuinely more space permit us to decide new languages?

Theorem (Space Hierarchy Theorem): For every proper complexity function f(n) ≥ log n:

SPACE(f(n)) ( SPACE(f(n) log f(n)).• Proof: same ideas.

9/12/2013 24

Best Hierarchy Theorems

Theorem (Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n:

TIME(f(n)) ( TIME( (f(n)log(f(n))).

Theorem (Space Hierarchy Theorem): For every proper complexity function f(n) ≥ log n:

SPACE(f(n)) ( SPACE( f(n)).

9/12/2013 25

Time and Space Classes

L = SPACE(log n)PSPACE = k SPACE(nk)

P = k TIME(nk)

EXP = k TIME(2nk)

9/12/2013 26

Hierarchy Separations

• Time Hierarchy Theorem impliesP ( EXP

– P µ TIME(2n) ( TIME(2(2n)3) µ EXP

• Space Hierarchy Theorem impliesL ( PSPACE

– L = SPACE(log n) ( SPACE(log2 n) µ PSPACE

9/12/2013 27

TIME(f(n)) SPACE(f(n))

• Any time step can only affect constant number of tape cells

: P µ PSPACE

9/12/2013 28

SPACE(f(n)) TIME(2 O(f(n)) )

• T in TIME(f(n)) says yes or no – never loops

• So T can’t repeat a configuration.• log(n) + klog(f(n))+ c + O(f(n) bits

• = O(f(n)) bits if f(n)>=log(n)28

input-tape head position work-tape head

position

state work-tape contents

SPACE(f(n)) TIME(2 O(f(n)) )

• L µ P

• PSPACE µ EXP

• So far:L µ P µ PSPACE µ EXP

• Open: L = P ??? P = PSPACE ??9/12/2013 29

9/12/2013 30

Reductions• reductions are the main tool for relating

problems to each other • given two languages L1 and L2 we say “L1

reduces to L2” and we write “L1 ≤ L2” to mean:– there exists an efficient (logspace) algorithm

that computes a function f s.t.• x L1 implies f(x) L2

• x L1 implies f(x) L2

9/12/2013 31

Reductions

• positive use: given new problem L1 reduce it to L2 that we know to be in P. Conclude L1 in P (how?)– e.g. bipartite matching ≤ max flow – formalizes “L1 as easy as L2”

yes

no

yes

noL1 L2

f

f

9/12/2013 32

Reductions

• negative use: given new problem L2 reduce L1 (that we believe not to be in P) to it. Conclude L2 not in P if L1 not in P (how?)– e.g. satisfiability ≤ graph 3-coloring– formalizes “L2 as hard as L1”

yes

no

yes

noL1 L2

f

f

9/12/2013 33

Reductions

• Example reduction:– 3SAT = { φ : φ is a 3-CNF Boolean formula

that has a satisfying assignment }(3-CNF = AND of OR of ≤ 3 literals)

– IS = { (G, k) | G is a graph with an independent set V’ µ V of size ≥ k }

(ind. set = set of vertices no two of which are connected by an edge)

9/12/2013 34

Ind. Set is as easy as 3-SAT

The reduction f: givenφ = (x y z) (x w z) … (…)

we produce graph Gφ:

x

y z

x

w z• one triangle for each of m clauses• edge between every pair of contradictory literals• set k = m

…

9/12/2013 35

Reductionsφ = (x y z) (x w z) … (…)

• Claim: φ has a satisfying assignment if and only if G has an independent set of size at least k– proof?

• Conclude that 3SAT ≤ IS.

x

y z

x

w z…

9/12/2013 36

Hardness

• L is C-hard if– every language in C reduces to L

9/12/2013 37

Completeness

• complexity class C• language L is C-complete if

– L is in C– L is C-hard