complexity theory - lecture 13: space complexity · space complexity space complexity review: space...

Space Complexity

Complexity TheorySpace Complexity

Daniel Borchmann, Markus Krötzsch

Computational Logic

2015-11-25

cba 2015 Daniel Borchmann, Markus Krötzsch Complexity Theory 2015-11-25 #1

Space Complexity Review

Review

Space Complexity Space Complexity

Space Complexity

Review: Space Complexity Classes

Recall our earlier definition of space complexities:

Definition 10.1

Let f : N→ R+ be a function.

DSpace(f(n)) is the class of all languages L for which there is anO(f(n))-space bounded Turing machine deciding L.

NSpace(f(n)) is the class of all languages L for which there is anO(f(n))-space bounded nondeterministic Turing machine deciding L.

Being O(f(n))-space bounded requires a (nondeterministic) TM

to halt on every input and

to use ≤f(|w |) tape cells on every computation path.

Space Complexity Classes

Some important space complexity classes:

L = LogSpace = DSpace(log n) logarithmic space

PSpace =⋃d≥1

DSpace(nd) polynomial space

ExpSpace =⋃d≥1

DSpace(2nd) exponential space

NL = NLogSpace = NSpace(log n) nondet. logarithmic space

NPSpace =⋃d≥1

NSpace(nd) nondet. polynomial space

NExpSpace =⋃d≥1

NSpace(2nd) nondet. exponential space

The Power Of Space

Space seems to be more powerful than timebecause space can be reused.

Example 10.2

Sat can be solved in linear space:Just iterate over all possible truth assignments (each linear in size) andcheck if one satisfies the formula.

Example 10.3

Tautology can be solved in linear space:Just iterate over all possible truth assignments (each linear in size) andcheck if all satisfy the formula.

More generally: NP ⊆ PSpace and coNP ⊆ PSpace

The Power Of Space

Example 10.2

Example 10.3

The Power Of Space

Example 10.2

Example 10.3

Linear Compression

Theorem 10.4

For every function f : N→ R+, for all c ∈ N, and for every f -spacebounded (deterministic/nondeterminsitic) Turing machineM:

there is a max{1, 1c f(n)}-space bounded (deterministic/nondeterminsitic)

Turing machineM′ that accepts the same language asM.

Proof idea.Similar to (but much simpler than) linear speed-up. �

This justifies using O-notation for defining space classes.

Linear Compression

Theorem 10.4

For every function f : N→ R+, for all c ∈ N, and for every f -spacebounded (deterministic/nondeterminsitic) Turing machineM:

there is a max{1, 1c f(n)}-space bounded (deterministic/nondeterminsitic)

Turing machineM′ that accepts the same language asM.

Proof idea.Similar to (but much simpler than) linear speed-up. �

This justifies using O-notation for defining space classes.

Tape Reduction

Theorem 10.5

For every function f : N→ R+ all k ≥ 1 and L ⊆ Σ∗:

If L can be decided by an f-space bounded k-tape Turing-machine,

it can also be decided by an f-space bounded 1-tape Turing-machine

Proof idea.Combine tapes with a similar reduction as for time. Compress space toavoid linear increase. �

Recall that we still use a separate read-only input tape to define somespace complexities, such as LogSpace.

Tape Reduction

Theorem 10.5

For every function f : N→ R+ all k ≥ 1 and L ⊆ Σ∗:

If L can be decided by an f-space bounded k-tape Turing-machine,

it can also be decided by an f-space bounded 1-tape Turing-machine

Proof idea.Combine tapes with a similar reduction as for time. Compress space toavoid linear increase. �

Recall that we still use a separate read-only input tape to define somespace complexities, such as LogSpace.

Time vs. Space

Theorem 10.6

For all functions f : N→ R+:

DTime(f) ⊆ DSpace(f) and NTime(f) ⊆ NSpace(f)

Proof.Visiting a cell takes at least one time step. �

Theorem 10.7

For all functions f : N→ R+ with f(n) ≥ log n:

DSpace(f) ⊆ DTime(2O(f)) and NSpace(f) ⊆ DTime(2O(f))

Proof.Based on configuration graphs and a bound on the number of possibleconfigurations.

Time vs. Space

Theorem 10.6

Theorem 10.7

Time vs. Space

Theorem 10.6

Theorem 10.7

Number of Possible Configurations

LetM := (Q ,Σ, Γ, q0, δ, qstart) be a 2-tape Turing machine(1 read-only input tape + 1 work tape)

Recall: A configuration ofM is a quadruple (q, p1, p2, x) where

q ∈ Q is the current state,

pi ∈ N is the head position on tape i, and

x ∈ Γ∗ is the tape content.

Let w ∈ Σ∗ be an input toM and n := |w |. Then also p1 ≤ n.

IfM is f(n)-space bounded we can assume p2 ≤ f(n) and |x | ≤ f(n)

Hence, there are at most

|Q | · n · f(n) · |Γ|f(n) = n · 2O(f(n)) = 2O(f(n))

different configurations on inputs of length n(the last equality requires f(n) ≥ log n).

Configuration Graphs

The possible computations of a TMM (on input w) form a directed graph:

Vertices: configurations thatM can reach (on input w)Edges: there is an edge from C1 to C2 if C1 `M C2

(C2 reachable from C1 in a single step)

This yields the configuration graph

Could be infinite in general.For f(n)-space bounded 2-tape TMs, there can be at most 2O(f(n))

vertices and 2 · (2O(f(n)))2 = 2O(f(n)) edges

A computation ofM on input w corresponds to a path in the configurationgraph from the start configuration to a stop configuration.

Hence, to test ifM accepts input w,

construct the configuration graph andfind a path from the start to an accepting stop configuration.

Configuration Graphs

The possible computations of a TMM (on input w) form a directed graph:

Vertices: configurations thatM can reach (on input w)Edges: there is an edge from C1 to C2 if C1 `M C2

(C2 reachable from C1 in a single step)

This yields the configuration graph

Could be infinite in general.For f(n)-space bounded 2-tape TMs, there can be at most 2O(f(n))

vertices and 2 · (2O(f(n)))2 = 2O(f(n)) edges

A computation ofM on input w corresponds to a path in the configurationgraph from the start configuration to a stop configuration.

Hence, to test ifM accepts input w,

construct the configuration graph andfind a path from the start to an accepting stop configuration.

Time vs. Space

Theorem 10.6

Theorem 10.7

Proof.

Build the configuration graph (time 2O(f(n))) and find a path from the startto an accepting stop configuration (time 2O(f(n))). �

Basic Space/Time Relationships

Applying the results of the previous slides, we get the following relations:

L ⊆ NL ⊆ P ⊆ NP ⊆ PSpace ⊆ NPSpace ⊆ ExpTime ⊆ NExpTime

We also noted P ⊆ coNP ⊆ PSpace.

Open questions:

What is the relationship between space classes and their co-classes?

What is the relationship between deterministic and non-deterministicspace classes?

Nondeterminism in Space

Most experts think that nondeterministic TMs can solve strictly moreproblems when given the same amount of time than a deterministic TM:Most believe that P ( NP

How about nondeterminism in space-bounded TMs?

Theorem 10.8 (Savitch’s Theorem, 1970)

For any function f : N→ R+ with f(n) ≥ log n:

NSpace(f(n)) ⊆ DSpace(f2(n)).

That is: nondeterminism adds almost no power to space-bounded TMs!

Nondeterminism in Space

Most experts think that nondeterministic TMs can solve strictly moreproblems when given the same amount of time than a deterministic TM:Most believe that P ( NP

How about nondeterminism in space-bounded TMs?

Theorem 10.8 (Savitch’s Theorem, 1970)

For any function f : N→ R+ with f(n) ≥ log n:

That is: nondeterminism adds almost no power to space-bounded TMs!

Consequences of Savitch’s Theorem

Savitch’s Theorem: For any function f : N→ R+ with f(n) ≥ log n:

Corollary 10.9

PSpace = NPSpace.

Proof.PSpace ⊆ NPSpace is clear. The converse follows since the square of apolynomial is still a polynomial. �

Similarly for “bigger” classes, e.g., ExpSpace = NExpSpace.

Corollary 10.10

NL ⊆ DSpace(O(log2 n)).

Note that log2(n) < O(log n), so we do not obtain NL = L from this.

Consequences of Savitch’s Theorem

Savitch’s Theorem: For any function f : N→ R+ with f(n) ≥ log n:

Corollary 10.9

PSpace = NPSpace.

Proof.PSpace ⊆ NPSpace is clear. The converse follows since the square of apolynomial is still a polynomial. �

Similarly for “bigger” classes, e.g., ExpSpace = NExpSpace.

Corollary 10.10

NL ⊆ DSpace(O(log2 n)).

Note that log2(n) < O(log n), so we do not obtain NL = L from this.

Proving Savitch’s Theorem

Simulating nondeterminism with more space:

Use configuration graph of nondeterministic space-bounded TM

Check if an accepting configuration can be reached

Store only one computation path at a time (depth-first search)

This still requires exponential space. We want quadratic space!What to do?

Things we can do:

Store one configuration:one configuration requires log n + O(f(n)) spaceif f(n) ≥ log n, then this is O(f(n)) space

Store log n configurations (remember we have log2 n space)

Iterate over all configurations (one by one)

Things we can do:

Proving Savitch’s Theorem: Key Idea

To find out if we can reach an accepting configuration, we solve a slighlymore general question:

Yieldability

Input: TM configurations C1 and C2, integer k

Problem: Can TM get from C1 to C2 in at most k steps?

Approach: check if there is an intermediate configuration C ′ such that

(1) C1 can reach C ′ in k/2 steps and

(2) C ′ can reach C2 in k/2 steps

{ Deterministic: we can try all C ′ (iteration){ Space-efficient: we can reuse the same space for both steps

An Algorithm for Yieldability

01 CanYield(C1,C2,k) {02 if k = 1 :03 return (C1 = C2) or (C1 `M C2)04 else if k > 1 :05 for each configuration C of M for input size n :06 if CanYield(C1,C,k/2) and07 CanYield(C,C2,k/2) :08 return true09 // eventually, if no success:10 return false11 }

We only call CanYield only with k a power of 2, so k/2 ∈ N

Space Requirement for the Algorithm

During iteration (line 05), we store one C in O(f(n))

Calls in lines 06 and 07 can reuse the same space

Maximum depth of recursive call stack: log2 k

Overall space usage: O(f(n) · log k)

Simulating Nondeterministic Space-Bounded TMs

Input: TMM that runs in NSpace(f(n)); input word w of length n

Algorithm:

ModifyM to have a unique accepting configuration Caccept

when accepting, erase tape and move head to the very left

Select d such that 2df(n) ≥ |Q | · n · f(n) · |Γ|f(n)

Return CanYield(Cstart,Caccept,k ) with k = 2df(n)

Space requirements:CanYield runs in

O (f(n) · log k) = O(f(n) · log 2df(n)

)= O(f(n) · df(n)) = O(f2(n))

Did We Really Do It?

“Select d such that 2df(n) ≥ |Q | · n · f(n) · |Γ|f(n)”

How does the algorithm actually do this?

f(n) was not part of the input!

Even if we knew f , it might not be easy to compute!

Solution: replace f(n) by a parameter ` and probe its value

(1) Start with ` = 1

(2) Check ifM can reach any configuration with more than ` tape cells(iterate over all configurations of size ` + 1; use CanYield on each)

(3) If yes, increase ` by 1; goto (2)

(4) Run algorithm as before, with f(n) replaced by `

Therefore: we don’t need to know f at all. This finishes the proof. �

Relationships of Space and Time

Summing up, we get the following relations:

L ⊆ NL ⊆ P ⊆ NP ⊆ PSpace = NPSpace ⊆ ExpTime ⊆ NExpTime

We also noted P ⊆ coNP ⊆ PSpace.

Open questions:

Is Savitch’s Theorem tight?

Are there any interesting problems in these space classes?

We have PSpace = NPSpace = coNPSpace.But what about L, NL, and coNL?

{ the first: nobody knows; the others: see upcoming lectures

complexity theory - lecture 13: space complexity · space complexity space complexity review: space...

Documents