csci 1590 intro to computational complexity - models of...

CSCI 1590Intro to Computational

ComplexityModels of Computation

John E. Savage

Brown University

January 26, 2009

John E. Savage (Brown University) CSCI 1590 Intro to Computational Complexity January 26, 2009 1 / 27

1 Problem Definitions

2 Models of Computation

3 Quiz on FSM Language Recognition

4 FSM Language Recognition

5 Quiz on TM Language Recognition

6 TM Language Recognition

7 Quiz on Classes P and NP

8 The Classes P and NP


Problem Definitions

Definition

A computational problem is specified by a function f : Σ∗ 7→ Σ∗ or alanguage. A language L is the set of strings x ∈ Σ∗ such that f(x) = 1.

Question

Does the size and nature of the alphabet Σ have a large impact on thetime complexity of a problem?


Models of Computation

Logic circuit

Deterministic finite-state machine (DFSM)

Nondeterministic finite-state machine (NFSM)

Deterministic Turing machine (TM)

Nondeterministic Turing machine (NTM)

Probabilistic Turing machine (PTM)

Random access machine (RAM)

Parallel random access machine (PRAM)

VLSI (very large scale integrated) model of computation


Logic Circuits

Definition

A circuit is a directed acyclic graph in which input vertices are associatedwith Boolean variables and non-input vertices carry the labels of Booleanfunctions such as and(∧), or (∨), not (¬), and exor (⊕).

pj

uj cj

gj

sj

cj+1

cj+1

vj uj

cj

sj

FAj

vj


The DFSM

Definition

A deterministic finite-state machine (DFSM) is a seven-tupleM = (Σ,Ψ, Q, δ, λ, s, F ) where Σ and Ψ are the input and outputalphabets, Q is the set of states of the machine, δ : Σ×Q 7→ Q is itsnext-state function, λ : Q 7→ Ψ is its output function, s is its initial state,and F ⊆ Q is a set of final states.

A DFSM makes transitions. Before each transition it is in a state q. Oninput x ∈ Σ it makes a transition to a state q′ = δ(q, x) ∈ Q, andproduces an output y = λ(q′) ∈ Ψ.

0

1

0

1

q0/0 q1/1

Start


Nondeterministic FSMs

Definition

A nondeterministic finite-state machine (NFSM) is a five-tupleM = (Σ, Q, δ, s, F ) where Σ is the input alphabet, Q is the set of statesof the machine, δ : Σ×Q 7→ 2Q is its next-state function, s is its initialstate, and F ⊆ Q is a set of final states. The next-state function maps astate and an input to a set of states, thereby providing a set of choices forthe next state.

0 q0/0 q1/1 q3/0

1 1

1

0

1

1 0

0

q2/1

Start


Deterministic Turing Machine

Definition

A deterministic Turing machine (DTM) is a six-tupleM = (Γ, β,Q, δ, s, h). Γ is tape alphabet, blank β 6∈ Γ, Q is finite set ofstates, δ : Q× (Γ ∪ β) 7→ (Q ∪ h)× (Γ ∪ β)× L, N, R isnext-state function, s is initial state, and h 6∈ Q is accepting haltstate (no exit from h). If M in state q reads a on tape andδ(q, a) = (q′, a′,C), it unit enters state q′, writes a′ in under head, andmoves head left (L) (if possible), right (R), or not (N).

Note that the control unit is a DFSM.0 1 2 m− 1

ControlUnit

Tape Unit

b


Nondeterministic Turing Machines

Definition

A nondeterministic TM (NTM) is a six-tuple M = (Γ, β,Q, δ, s, h)where δ : Q× (Γ ∪ β) 7→ (2Q ∪ h)× (Γ ∪ β)× L, N, R isnext-state function, s is initial state, and h 6∈ Q is accepting haltstate (no exit from h). If M in state q reads a on tape andδ(q, a) = (q′, a′,C), M may execute any triple in δ(q, a). A string xwritten left-adjusted on a blank tape is accepted if some sequence oftransitions leads to the accepting halt state h.

The control unit is a NFSM. A certificate for acceptance is the set oftransitions leading to an accept state.

0 1 2 m− 1

ControlUnit

Tape Unit

b


Randomness and Turing Machines

Definition

A probabilistic Turing machine (PTM) is a Turing machine in whichstate transitions occur probabilistically. That is, a PTM is a DTM withaccess to a second tape containing only random bits. While computing, aPTM reads and writes from its input/work tape or it reads random bitsfrom its random-input tape.

An NDTM can be modeled as a DTM with access to a properly chosencertificate. A PTM is similar to an NDTM but instead of accessing onewell-chosen certificate, a PTM accesses a randomly chosen sequence. APTM also allows us to quantify the amount of “randomness” that acomputation uses.

The goal when computing with a PTM can be to (a) always output thecorrect output, but to be efficient (in time or space) on average or (b) toalways execute efficiently, but to be correct with bounded probability


Random Access Machine (RAM)

Implements fetch-execute cycle

Uses direct and indirect addressing

Arithmetic logic unit (ALU) performs addition, single-bit shifts. Nomultiplication because it introduces parallelism.

Words increase at most one bit in length/per cycle

regb

rega

ALU

addr

in wrd

out wrd

b

CPUcmd

Random-Access Memory

m− 1m− 2

10

Decode

prog ctr


Parallel Random Access Machine (PRAM)

Implements fetch-execute cycle

Uses direct and indirect addressing

Arithmetic logic unit (ALU) performs addition, single-bit shifts. Nomultiplication because it introduces parallelism.

Words increase at most one bit in length/per cycle

Common Memory

P_p

RAM

P_2

RAM

P_1

RAM


The VLSI Model

Chips realize DFSMs – they are logic circuits with binary memorycells.

Wires have bounded width and separation λ.

Gates have area Ω(λ2), wires have area O(Lλ), L = length.

At most ν layers of wires.

Time for a signal to traverse a wire is constant, independent of length.


DFSM Language Recognition

Quiz

What does it mean that a DFSM accepts a string? recognizes alanguage?

Do the languages recognized by a DFSM change when the conditionfor accepting a string is changed from the DFMS entering an acceptstate to producing a particular value for the last output?

How would you define the complement of a language?

Given a language L recognized by a DFSM M , give a machine M ′

that recognizes the complement of L.


NFSM Language Recognition

Quiz

What does it mean that a NFSM accepts a string? recognizes alanguage?

Do the languages recognized by a NFSM change when the conditionfor accepting a string is changed from the NFMS entering an acceptstate to producing a particular value for the last output?

Given a language L recognized by an NFSM M , how would youconstruct a DFSM that recognizes the same language?


Computation by a DFSM

Definition

If a sequence of inputs x1, x2, . . . , xT causes a DFSM M to move from itsinitial state s to a state q ∈ F , the string x1x2 · · ·xT is accepted by theM .

The language L(M) recognized by M is the set of accepted strings.

A DFSM M computes a function fM defined by the mapping from inputsx1, x2, . . . , xT to outputs y1, y2, . . . , yT .

0

1

0

1

q0/0 q1/1

Start


NFSM Language Recognition

Definition

A string x1, x2, . . . , xT is accepted by an NFSM M if when started instate s, there are successor states that result in M arriving at a final statein F . A certificate to acceptance is the set of choices that lead to anaccept state.

The language recognized by an NFSM is the set of strings that it accepts.

0 q0/0 q1/1 q3/0

1 1

1

0

1

1 0

0

q2/1

Start


DTM and NTM Language Recognition

Quiz

What does it mean for a DTM to accept a string?

What does it mean for a NTM to accept a string?

What does it mean for a TM to recognize a language?

Is every language recognized by an DTM also recognized by a NTM?

Is every language recognized by an NTM also recognized by a DTM?

So why do we use NTMs?


Language Recognition on TMs

Definition

A DTM M accepts input string w ∈ Γ∗ if, when started in state s withw placed left-adjusted on blank tape with head on left cell, the last stateentered is h. M may not halt on some inputs.M accepts the language L(M) = strings accepted. Such languagesare called recursively enumerable (r.e.). If TM halts on all inputs andrecognizes L(M), L(M) is called recursive.A DTM computes a function f : Γ∗ 7→ Γ∗ if on input w it computes z(leftmost adjusted on non-blank portion of tape) where f(w) = z. f ispartial if the TM fails to halt on some input strings and completeotherwise.


Nondeterministic Turing Machines

A certificate to acceptance of a string x is the set of choices made bythe NFSM control unit of the NTM.

Here is an alternative definition of NTM language recognition.

Definition

A language L ⊂ Γ∗ is recognized by an DTM M if for every x ∈ Γ∗,

x ∈ L⇔ ∃u ∈ Γ∗ s.t. M(x,u) = 1

where M(x,u) = 1 means that M accepts x. u is a certificate toacceptance of x.


The Classes P and NP

Quiz

Is the class P the set of functions computable in polynomial time?

What is the parameter of the polynomial?

Is the class NP the languages recognized by an NDTM in polynomialtime?

Given a language L in NP, is there a certificate for each x ∈ L?

What does it mean that L is NP-complete?


The Class P

Definition

A language L ⊂ Γ∗ is in P if there exists a DTM M that halts on everyinput in polynomial time such that for every x ∈ Γ∗,

x ∈ L⇔M(x) = 1

where M(x) = 1(0) if M accepts (rejects) x in poly time in |x|.

Examples

The class of regular languages is in P.

The context-free language 0n1n is in P.


The Class NP

Here is an alternative definition of NP.

Definition

A language L ⊂ Γ∗ is in NP if there exists a polynomial p : N 7→ N and apolynomial time DTM M such that for every x ∈ Γ∗,

x ∈ L⇔ ∃u ∈ Γp(|x|) s.t. M(x,u) = 1

u is called a certificate for x when x ∈ L and u ∈ Γp(|x|) satisfyM(x,u) = 1.

Example

accept = < M,x, 1t >|M is a DTM, x ∈ Γ∗, ∃y ∈Γ∗ such that M accepts < x,y > in t steps.accept is in NP.


NP-complete Languages

Definition

Let L1 ⊂ Σ∗1 and L2 ⊂ Σ∗2. L1 is P-time reducible to L2 if there existsP-time computable f : Σ∗1 7→ Σ∗2 s.t. x ∈ L1 if and only if f(x) ∈ L2.

Definition

A language L2 is NP-complete if a) it is in NP and b) every language L1

in NP is P-time reducible to L2.

Exampletraveling salespersonInstance: An integer k and a set of n2 symmetric integer distancesdi,j | 1 ≤ i, j ≤ n between n cities where di,j = dj,i.Answer: “Yes” if there is a tour (an ordering) i1, i2, . . . , in of the citiessuch that the length l = di1,i2 + di2,i3 + · · ·+ din,i1 of the tour satisfiesl ≤ k.



More Examplescircuit satInstance: A circuit description with n input variables x1, x2, . . . , xn forsome integer n and a designated output gate.Answer: “Yes” if there is an assignment of values to the variables suchthat the output of the circuit has value 1.

satisfiability NP-complete problemInstance: A set of literals X = x1, x1, x2, x2, . . . , xn, xn, and a sequenceof clauses C = (c1, c2, . . . , cm) where each clause ci is a subset of X.Answer: “Yes” if for some assignment of Boolean values to variables inx1, x2, . . . , xn, at least one literal in each clause has value 1.



Theorem

accept is NP-complete.

Proof.

By definition accept is in NP.

A language L1 is in NP if there is an NTM M1 that accepts every x ∈ L1

in p(n) steps where n = |x| and p(n) is a polynomial. An instance of L1 is(M1,x, p(·)). Translate each such instance to an instance< M1,x, 1p(|x|) > of accept. The latter instance is accepted if and onlyif the former is accepted. The reduction takes time polynomial in thelength of (M1,x, p(·)) to transform it into < M1,x, 1p(|x|) >.


Course Management

Textbooks:

Models of Computation by John E Savage is available privately toBrown students.Complexity Theory: A Modern Approach by Sanjeev Arora andBoaz Barak is available athttp://www.cs.princeton.edu/theory/complexity/

Six or seven assignments

Perhaps a midsemester exam

A final exam


csci 1590 intro to computational complexity - models of...

Documents