Slide 1

Slide 2 Theoretical Foundation of Computation Instructor: Bin Fu Textbook: Introduction to the theory of computation, by Michael Siper Class Time: 5:45-8:25pm Tuesday Slide 3 Contents Computational models and languages finite automata, push automata regular languages, context free languages Computability theory decidable problem, un-decidable problem Complexity theory time, space, P, NP, PSpace Slide 4 Theory of Computation What is the computation? What problems are computable by computer in finite steps? What problems are computable by small number of steps? Slide 5 Mathematical Model Real computers are very complicate Develop simple mathematical model to define computation The mathematical models are equivalent to real computer under some transformation Slide 6 Important Boundary Computability theory boundary Un-computable problems Computable problems Slide 7 Hierarchy inside computable area Slide 8 Basic Concepts Set: a set is a group of objects represented as a unit. {7,21,57} Element: A object of a set is called an element. Subset: A is a subset of B if every member of A is also a member of B. Slide 9 Some sets Natural numbers set: N={1,2,3,} Integers set Z={, -2,-1,0,1,2,} Empty set: it is a set with no elements Slide 10 Set operations Union: A={2,4,9} B={1,2,5} ={1,2,4,5,9} Intersection: ={2 } Slide 11 Tuple Sequence: a list of objects in some order (7, 21, 57) Tuple: finite sequence (7, 21,57) Cartesian product: A x B is the set of all pairs with first element from A and second element from B A={1,2} B={x,y,z} A x B= {(1,x),(1,y), (1,z), (2,x), (2,y), (2,z) } Slide 12 Power Set Let A be a set. The power set of A is the set of subsets of A. For A={a,b}, its power set P(A) is {, {a}, {b}, {a,b}} Slide 13 Cartesian product Slide 14 Function A function f is mapping from one set D to another set R f: D R for every a in the set D, there is another element b in R such that a is mapped to b by f f(a)=b Domain: D Range: R Slide 15 Function example Function f: {0,1,2,3,4} {0,1,2,3,4} n f(n) 0 1 1 2 2 3 3 4 4 0 Slide 16 Relation For two sets A and B, a binary relation R is a subset AxB Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)} Slide 17 Represent binary relation If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3 Slide 18 Relation For two sets A and B, a binary relation R is a subset AxB Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)} Slide 19 Represent binary relation If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3 Slide 20 Equivalence relation A binary relation R is equivalence relation if Reflexive: xRx for every x Symmetric: if xRy, then yRx Transitive: if xRy and yRz, then xRz Example 1: = on {1,2,3}x{1,2,3} Example 2: = on NxN Slide 21 Example for equivalence Relation For two integers x and y, x y if (x-y) is a multiple of 7 In other words, there is another integer z such that (x-y)=7z. Slide 22 Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ } v1 v2 v3 v4 Slide 23 Path Graph G=(V,E) A path is a series of edges linked one by one Loop: Slide 24 Tree A graph is connected if every two nodes have a path to connect them A tree is a connected graph without loop Slide 25 Connected Graph Tree Every connected graph can be converted into tree by removing some edges Removing one edge on a loop does not damage the connectivity. Slide 26 A tree is a minimal connected graph Removing any edge on a tree damages the connectivity Proof. Tree T=(V,E). Let (v1, v2) be removed from T. T T=(V, E-{(v1,v2)}). If T is still connected, T has a loop containing v1 and v2. Contradiction! Slide 27 Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ } v1 v2 v3 v4 Slide 28 Node degree: The number of edges connecting to the node v1 v2 v3 v4 Slide 29 Mathematical approach Definition Mathematical statement Express some object with certain property Theorem A statement proved to be true Proof Slide 30 Mathematical proof Convincing logical argument that a statement is true Usually consists a series logical statements There is a small logical gap between the current logic statement with previous statements. Slide 31 3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction Slide 32 Sum of node degrees For every graph, the sum of the degrees of all nodes in G is an even number Sum=2+2+2=6 sum=2+3+4+3+2=14 Slide 33 Proof Let v1, v2, , vn are the n nodes of the graph. deg(vi) is the degree of node vi The sum of node degrees is sum=deg(v1)+deg(v2)++deg(vn) For each edge e=(vi, vj), it makes one contribution to both deg(vi) and deg(vj). If there are k edges, the sum is 2k. Slide 34 Regular graph A graph is k-regular if every node has degree equal to k Theorem: For each even number n>2, there exists a 3-regular graph with n nodes. Slide 35 Proof by construction n/2 nodes Let every point at top half connect to a point in the bottom half Slide 36 Proof Let be the n nodes of the graph. Add edges for i=0, 1, ,(n/2)-1, and Add edges for i=0,1, ,(n/2)-1 Slide 37 Proof by contradiction Assume the theorem is false Lead to an obviously false consequence Example: Jill just came in from outdoor and is complete dry Try to Prove: No rain Proof: Assume it were raining Jill would be wet. A contradiction! Slide 38 No 3-regular graph with odd nodes Theorem: There is no 3-regular graph with odd number of nodes Proof (by contradiction) If the graph G is a 3-regular graph with 2m+1 nodes, where m is an integer at least 0. The sum of the degrees of nodes is sum=3+3+.+3=3(2m+1)=6m+3=2(3m+1)+1 It is a contradiction to our previous theorem. Slide 39 3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction Slide 40 Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1) Slide 41 Example for induction Theorem: For every natural number n, Slide 42 Proof Basis: when n=1. The left side is 1 The right is Induction step: Assume Slide 43 Proof Slide 44 Computation models Finite automata Pushdown automata Turing machine Slide 45 3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction Slide 46 Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1) Slide 47 Example for induction Theorem: For every natural number n, Slide 48 Proof Basis: when n=1. The left side is 1 The right is Induction step: Assume Slide 49 Proof Slide 50 Computation models Finite automata Pushdown automata Turing machine Slide 51 Finite automata Supermarket entrance Front: A person is coming from the front. Rear: Both: front and rear Neither: neither front nor rear closed open Slide 52 Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states Slide 53 Automata example q2 q3q1 Slide 54 Acceptance Any sequence with at least one 1 and even number of 0s following the last 1 Slide 55 Notions for automata start state: q1 accept state: q2. In other words, F={q2} The set The transition function is 0 1 q1 q1 q2 q2 q3 q2 q3 q2 q2 Q={q1,q2,q3} p, tw"> Analysis Run the input string on a automata Input string: s1 s2 s3 s4 s5 s6 sn State q1 q2 q3 q9 q5 q9 q13 Let p be the number of states If n>p, two of the states must be equal, say q9 Repeat the substring between the two q9s reach the same accept state q13 Slide 145 Proof Let p be the number of the states in the Automata M for A Let (n>=p) be a string in A Let be the state transition sequence There are two equal states (j Proof Remove the for non-start symbol A Remove the unit rule A B For each at k>2, replace it by rules Slide 174 Proof For each at k>1, replace each terminal by the new symbol and add new rule Slide 175 Example A 0A1 Conversion: Slide 176 Remove the empty Slide 177 Slide 178 Slide 179 Slide 180 Arithmetic expression Grammar Slide 181 Example Generate Slide 182 Generate the expression Steps Slide 183 Arithmetic expression Grammar Start variable: Variable: Terminals: a,+,x,(,) Slide 184 Example Generate Slide 185 Example Generate Slide 186 Leftmost derivation A derivation of a string w is leftmost derivation if the left most variable is replaced at every step x + x a+ x a+a x a+a x a Slide 187 Ambiguity A string w is derived ambiguously in context-free grammar G if it has more than one leftmost derivation Grammar G is ambiguous if it generates some string ambiguously Ambiguity gives different interpretations by computer program Slide 188 Example Design the context free grammar for Slide 189 Grammar Partition the problem into two parts. One part is for The second part is Let and S1 is used to get via S2 is used to get via Slide 190 Pushdown Automata State control aabbbaabbb aaaabbbbbabaaabb Slide 191 Example The grammar generates the language It can be accepted by pushdown automata Slide 192 Remove input 0, push 0 into stack State control Slide 193 Remove input 0, push 0 into stack State control Slide 194 Match stack 0 with input 1. Then remove both of them State control Slide 195 Match stack 0 with input 1. Then remove both of them, accept State control Slide 196 Pushdown Automata Let M be 1. is the set of states 2. is the input alphabet 3. is the stack alphabet 4. is the state transition function 5. is the start state, 6. is the set of accept states q2 q3 q4 q1 Slide 197 States Explanation Q1 is the start state Q2 is used to push 0 symbol Q3 is used to match 0,1 pairs between stack and input tape Q4 is the accept state Slide 198 Transition a,b c: Used when the current input symbol a and the stack top symbol b, Remove the top symbol on the stack and push c on it The input is moved to the next after a Slide 199 Difference between Pushdown and finite state automata Pushdown automata has unlimited memory, which is last in, first out. Nondeterministic finite state automata is a special case of pushdown automata that has no memory Slide 200 Pushdown Automata 6-tuple 1.Q is the finite set of states 2. is the input alphabet (finite) 3. is the stack alphabet (finite) 4. 5. is the start state, 6. is the set of accept states Slide 201 A computation of pushdown automata Input string: State transition sequence: For i=0,,m-1, where and Final state accepts Slide 202 Theorem Every context free language can be accepted by a non- deterministic pushdown automata Slide 203 Example The grammar generates the language It generates the string 0011 Slide 204 Pushdown Automata State control Slide 205 Replace S by 0S1 at stack State control Slide 206 Match the stack top symbol with the input symbol, remove both if matched State control Slide 207 Replace S by 0S1 State control Slide 208 Remove after matching State control Slide 209 Replace S by empty State control Slide 210 Remove after matching State control Slide 211 Remove after matching, Accept State control Slide 212 Proof Idea Push $ and start symbol to the stack in the beginning. Repeat the three steps below Replace the top variable A on the stack with a the right side of a rule Match the top terminal on the stack with the input symbol, reject if not matched When stack has $ on the top and all input has been read, accepts Slide 213 How to replace the variable If the current stack has the top element s, replace it with the right of the rule State q a.. s....s.... Slide 214 New States The states are new states, specially added for the rule State moves from q to r after the variable is replaced on the top of the stack Slide 215 Pumping Lemma Lemma: If A is context-free language, there is a number p such that if s is in A and of length at least p, s may be divided into s=uvxyz, satisfying 1) for each 2) 3) Slide 216 Proof Idea Two variables are the same on a path Slide 217 Proof Idea A path from root to a leaf has all variables except the last one, which is a terminal When the path is too long, same variable has to happen twice. Repeat the part of the two equal variables area. Slide 218 Analysis Let V be the set of variables in a grammar. |V| is the number of variables in V If a path has at least |V|+1 variables, two of them will be equal. A path from root to a leaf has least |V|+1 variables if its length is at least |V|+2 A tree of depth |V|+2 has leaves, where b is the maximal length of right side among all rules. Slide 219 Proof Let If s has length, the parse tree T has height It has a path has at least |V|+1 variables. Two of the variables on the path are equal (=R). We have So, Slide 220 Proof The tree T is the parsing tree for s and has least size It is impossible that is empty otherwise, T is not least. Select the bottom |V|+2 symbols on the longest path of T, this makes the Slide 221 Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G. Slide 222 Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. v and y are in the area of the same symbol, say a. Contradiction for having more a than b. Case 2. v or y contains more than one symbol, contradiction for the incorrect order of symbols in Slide 223 Regular Pumping The language is not regular Proof. Assume it is a regular language. It can be accepted by automata M. Let p be the length of the pumping lemma. Consider s can be expressed s=xyz, where The string xz has the number of 0s no more than 1s (Pump it down to get the contradiction) Slide 224 Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G. Slide 225 Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. vy have intersection with the a area. Contradiction for having more a or b than c (pumping up). Case 2. vy does not contain symbol a, contradiction for having more a than b or c (pumping down). Slide 226 Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G. Slide 227 Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. vy is in the first half. Contradiction for moving 1 to the right area(pumping up). Case 2. vy is in the second half. Contradiction for moving 0 to the left area(pumping up). Slide 228 Proof The language is not context free Proof. Case 3. vy crosses the middle line. Contradiction for reducing 0 or 1 in the middle area (pumping down). The left half and right half have different number of 1s or 0s. Slide 229 Problem Write a context-free grammar for the following language Slide 230 Problem Using pumping lemma to disprove the following language is context-free: Slide 231 Algorithm An algorithm is a collection of simple instructions for carrying out some task Slide 232 Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms Slide 233 Unlimited Register Machine Statement1: x x+1 Statement2: x y Statement3: x 0 Statement4: if (x==y) jump m Each register can save an unbounded integer Slide 234 Turing Machine Proposed in 1936 An accurate model for the general purpose computer. Slide 235 Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control abab Slide 236 Language ww Design Turing machine for L={w#w|w is in {0,1}*} For example: 011#011 is in L 10011#10011 is in L Slide 237 Movement on the tape Move the head back and forth to match all pairs Slide 238 Movement on the tape Move the head Slide 239 Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet 4. 5. is the start state, 6. is the accept state 7. is the reject state, where Slide 240 State transition function For Slide 241 State transition function For Slide 242 Configuration Current state: q7 Current head position on the tape: 4 th cell Current tape content: abab q7 abab Slide 243 Configuration A configuration is represented by Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state Slide 244 Configuration Transition For Slide 245 Configuration Transition For Slide 246 Configuration Start configuration:, where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state Slide 247 Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields, and 3. is an accepting configuration Slide 248 Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it. Slide 249 a q1 q2 q3 q4 q6 q8 q10 q12 q13 q14 q5 q7 q9 q11 q_accept Slide 250 Multi-tape Turing Machine Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control ba aaa 01010 Slide 251 Multi-tape Turing Machine Transition for multi-tape Turing machine Slide 252 Simulate Multitape Turing machine Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 0101#aaa#b#a# Slide 253 Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow) Slide 254 Another simulation Assume there are k tapes Let the positions i, k+i, 2k+i, ., mk+i, be used for simulating the tape i. q7 ab1aa0a0 Slide 255 Simulate 3 tapes q7 ab1aa0a0 Slide 256 Simulate 3 tapes q7 ab1aa0a0 Slide 257 K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape is first j-cell information is at 2(j- 1)k+2i-1 and 2(j-1)k+2i Slide 258 Midterm October 18, 2010 Class Time Close book Chapter 0-Chapter 3 Slide 259 Problem 5 Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains an equal number of 0s and 1s}. Slide 308 Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable Slide 309 DFA ={ | B is a DFA that accepts input string w} Theorem: is a decidable language. Slide 310 Proof The input is, where B is a DFA, and w is a string. Simulate B on the input w: Start from the state of B and leftmost symbol w. Follow to transit the state and move the input symbols one by one Accepts if ends at an accept state; rejects otherwise. Slide 311 NFA ={ | B is a NFA that accepts input string w} Theorem: is a decidable language. Slide 312 Proof The input is, where B is a NFA and w is a string. Convert B into a DFA C via our previous algorithm Use our last TM to decide if C accepts w Accepts if it accepts; rejects otherwise. Slide 313 ={ | L(A) is not empty} Theorem: is a decidable language. Slide 314 Proof The input is, where A is a DFA. Mark the start state Repeat Mark a new state that has a transition to it from a marked one. Until no new state can be added Accept if an accept state is marked; otherwise reject Slide 315 ={ | A and B are DFA and L(A)=L(B)} Theorem: is a decidable language. Slide 316 Proof The input is, where A and B are DFA Check if is empty Slide 317 Problem Define L={ : L(A) is the union of L(B) and L(C)}, where A, B, and C are DFAs. Prove that L is decidable. Slide 318 One- one and onto Let A and B be two sets. For function f: A B, if whenever then f is called one-one. For function f: A B, say f is onto if f hits every element of B(In other words, for very b in B, there is a in A such that b=f(a)) Slide 319 Correspondence Let A and B be two sets. A and B are of the same size if there is a one-one and onto function f: A B For function f: A B, if it is both one-one and onto, then f is called correspondence. Slide 320 Examples {1,2,3,} and {2,4,6,} are of the same size via f(x)=2x. (0,1) and are of the same size Slide 321 Countable A set is countable if it is finite or it has the same size as N={1,2,3,} Theorem: The positive rational numbers set is coutable Slide 322 Proof Every positive rational number is in the table below Slide 323 Proof List all of them by and avoid repetition Slide 324 Examples For two positive rational numbers, ( p and q have no common divisor>1, and p and q have no common divisor >1), The number is listed before if p+q Composite A natural number is composite if it is the product of two integers >1 10=2*5 18=2*9 Composite={x| x=pq, for integers p,q>1} Slide 464 Verifier A verifier for a language L is an algorithm V, L={w| V accepts for some string c} For the verifier V for L, c is a certificate of w if V accepts If the verifier V for the language L runs in polynomial time, V is the polynomial time verifier for L. Slide 465 Verifier for Hamiltonian Path For, a certificate is a list of nodes of G: Verifier: check if m is the number of nodes of G check if and check if each is a directed edge of G for i=1,,m-1 If all pass, accept. Otherwise, reject. Slide 466 Verifier for Composite For integer x, a certificate is two integers p,q: Verifier: check if p>1 and q>1 check if x=pq If all pass, accept. Otherwise, reject. Slide 467 Class NP NP is the class of languages that have polynomial time verifiers. Examples: COMPOSITE is in NP HAMPATH is in NP Slide 468 Theorem A language has polynomial verifier iff it can be recognized by polynomial time nondeterministic Turing machine Proof: Assume L has verifier V, Assume L has NTM M, Slide 469 Proof Assume that L has polynomial time verifier V, which runs in time, where k is a constant NTM M Nondeterministically select string c of length Run V on If V accepts, accept; Otherwise, reject. Slide 470 Proof Assume that L is recognized by polynomial time NTM M, which runs in time, where k is a constant Verifier Let c determine a computation path of M Simulate M on the path c If M accepts, accept. Otherwise, reject. Slide 471 Clique Problem Given undirected graph G, a clique is a set of nodes of G such that every two nodes are connected by an edge. A k-clique is a clique with k nodes Slide 472 Clique Problem CLIQUE={ | G iss an undirected graph with k-clique} CLIQUE is in NP. Slide 473 Subset Sum Problem SUBSET-SUM={ | S= and for some, we have Slide 474 Polynomial Time Computable A function is a polynomial time computable function if some polynomial time Turing machine M exists that outputs on the tape for input w. Slide 475 Polynomial Time Reduction Assume that A and B are two languages on A is polynomial time mapping reducible to A if a polynomial time computable function exists such that Slide 476 Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses Slide 477 SAT A boolean formula is satisfiable if there exists assignments to its variables to make the formula true SAT={ | is satisfiable boolean formula} Slide 478 3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula} Slide 479 3SAT to CLIQUE Example: Slide 480 NP-completeness A language B is NP-complete if 1.B is in NP, and 2.Every A in NP is polynomial time reducible to B Theorem. If B is NP-complete and B is in P, then P=NP. Slide 481 Cook-Leving Theorem Theorem: SAT is NP-complete Proof. 1. SAT is in NP. 2. For every problem A in NP, Slide 482 Proof 1. The start configuration is legal 2. The final state is accept. 3. The movement is legal. 4. Each cell takes one legal symbol. Slide 483 Proof 1. 1 if The cell[i,j] holds symbol s; 0 otherwise 2. Time bound for the NTM M with constant k. 3. The movement is legal. 4. NTM M for accepting A. Slide 484 Nondeterministic Turing Machine 1.Q is the finite set of states 2. is the tape alphabet 3. 4. is the start state, 5. is the accept state. Slide 485 Configuration Transition For Slide 486 Configuration Transition For Slide 487 Configuration Start configuration:, where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state Slide 488 Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields, and 3. is an accepting configuration Slide 489 Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it. Slide 490 Proof Each cell has only one symbol 1.The symbol is selected from C: 2.Only one symbol is selected: 3.It is true for all cell at all configuration: Slide 491 Proof The start configuration is Slide 492 Proof Accept computation has reached. It makes sure the accept state will appear among the configuration transitions. Slide 493 Proof Characterize the legal move The whole move is legal if all windows are legal. Characterize one window is legal Slide 494 Proof The state transition Slide 495 Logic Demorgan Law: Slide 496 Truth table for y1 y2 x2 1 1 1 0 1 1 1 0 1 0 1 0 1 0 1 1 0 0 0 1 0 1 1 1 0 0 1 0 0 1 0 0 1 1 0 0 0 0 1 0 Slide 497 Convert to CNF Conversion: Slide 498 Convert to CNF Conversion: Slide 499 Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses Slide 500 Prepare for the Final Regular language and automata Context free language Decidability Undecidability Complexity theory Slide 501 Regular Language Concepts: Automata, regular expression Skills: Design automata to accept a regular language Disprove a language is a regular Slide 502 Context-free Language Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free Slide 503 Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable Slide 504 Undecidability Concepts: Countable, Turing undecidable, reduction Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable Slide 505 Complexity Concepts: Time on Turing machine PTIME(t(n)) NP-completeness Polynomial time reduction Polynomial time verifier Slide 506 Complexity Skill: Prove a problem is in P Prove a problem is in NP Use reduction to prove a problem is NP-complete. Slide 507 Grade A: B: C: Miss exam or homework Slide 508 SAT A conjunctive normal form is a conjunction of some clauses SAT={ | is satisfiable conjunctive normal form} Slide 509 Cook-Leving Theorem Theorem: SAT is NP-complete Proof. Same as that for SAT is NP-complete Slide 510 3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula} Slide 511 3SAT is NP-complete Theorem: There is polynomial time reduction from SAT to 3SAT. Slide 512 3SAT is NP-complete is satisfiable if and only if the following is satisfiable Slide 513 3SAT is NP-complete is satisfiable if and only if the following is satisfiable Slide 514 3SAT is NP-complete Convert every clause into 3cnf: Slide 515 3SAT is NP-complete Conjunctive normal form Each clause is convert into is satisfiable if and only if the following is satisfiable Slide 516 Problem Convert the formula F into 3SAT formula F such that F is satisfiable iff and F is satisfiable. Slide 517 Approximation Algorithms Slide 518 Outline and Reading Approximation Algorithms for NP-Complete Problems Approximation ratios Polynomial-Time Approximation Schemes 2-Approximation for Vertex Cover Approximate Scheme for Subset Sum 2-Approximation for TSP special case Log n-Approximation for Set Cover Slide 519 Approximation Ratios Optimization Problems We have some problem instance x that has many feasible solutions . We are trying to minimize (or maximize) some cost function c(S) for a solution S to x. For example, Finding a minimum spanning tree of a graph Finding a smallest vertex cover of a graph Finding a smallest traveling salesperson tour in a graph Slide 520 Approximation Ratios An approximation produces a solution T T is a k-approximation to the optimal solution OPT if c(T)/c(OPT) < k (assuming a min. prob.; a maximization approximation would be the reverse) Slide 521 Polynomial-Time Approximation Schemes A problem L has a polynomial-time approximation scheme (PTAS) if it has a polynomial-time (1+ )-approximation algorithm, for any fixed >0 (this value can appear in the running time). Subset Sum has a PTAS. Slide 522 Vertex Cover A vertex cover of graph G=(V,E) is a subset W of V, such that, for every (a,b) in E, a is in W or b is in W. OPT-VERTEX-COVER: Given an graph G, find a vertex cover of G with smallest size. OPT-VERTEX-COVER is NP-hard. Slide 523 Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Slide 524 Slide 525 A 2-Approximation for Vertex Cover Every chosen edge e has both ends in C But e must be covered by an optimal cover; hence, one end of e must be in OPT Thus, there is at most twice as many vertices in C as in OPT. That is, C is a 2-approx. of OPT Running time: O(m) Algorithm VertexCoverApprox(G) Input graph G Output a vertex cover C for G C empty set H G while H has edges e H.removeEdge(H.anEdge()) v H.origin(e) w H.destination(e) C.add(v) C.add(w) for each f incident to v or w H.removeEdge(f) return C Slide 526 Problem 1 Show that T={(i,j)| both I and j are positive rational numbers} is countable. Slide 527 Problem 2. Find a correspondence between (0,1) and [0,1] Slide 528 Solution Let for n=1, 2, Define the correspondence f(x) below: Slide 529 Problem 3 3. Let A={ | R and S are regular expressions and L(R ) is a subset of L( S)}. Show that A is decidable. Slide 530 Solution For two regular expression R and S, L( R) is a subset of L(S) if and only if the L(R ) has no intersection with. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Check if L(M4) is empty R S Slide 531 Problem 4 I={x| x is an irrational number in (0,1)}. Show that I is not countable. Slide 532 Homework 2009 Slide 533 Problem 4.2 The equivalence of a DFA and a regular expression is decidable Slide 534 Solution Proof. Given a regular language L and DFA M. We can construct a DFA N to accept L. The problem is converted into checking if two DFAs are equivalent. By Theorem 4.5, there is an algorithm to decide if L(M)=L(N). Slide 535 Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable Slide 536 Solution Assume M is a DFA with n states. L(M) is infinite iff it has path from the start state to accept with a loop on it, and the loop length is no more than n. L(M) is infinite iff M accepts a string of length between n and 2n. For every string x with length between n and 2n Run M on x If M accepts x, then output L(M) is infinite else output no Slide 537 Problem 4.7 The set of all infinite binary strings is not countable Slide 538 Proof. Proof by contradiction. Assume N={1,2,3,} and have the same size. There is one-one and onto map f: N Slide 539 Proof. Select such that and Since Slide 540 Problem 1 N={1,2,3,} NxNxN is countable Slide 541 Proof List all of them by Slide 542 Proof Use the similar method like the last slide to prove the NxN is countable Let f: NxN N be one-one and onto We have g: NxNxN N is one-one and onto, where g(i,j,k)=f(f(i,j),k) Slide 543 Problem 2. Find a correspondence between (0,1) and [0,1] Slide 544 Solution Let for n=1, 2, Define the correspondence f(x) below: Slide 545 Problem 3 3. Let A={ | R and S are regular expression and L(R ) = L( S)}. Show that A is decidable. Slide 546 Solution For two regular expression R and S, L( R)=L(S) if and only if their symmetric difference is empty. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Construct automata M5 to accept Check if L(M5) is empty Slide 547 Problem 4 S={ | M is a TM that accepts whenever it accepts w}. Show S is undecidable. Slide 548 Proof We are going to design a reduction from to S. For Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts is in iff N accepts both 01 and 10. Slide 549 Problem 5 5. Let S={ | M is a TM and L(M) = { }}. Show that neither S nor its complement is Turing-recognizable. Slide 550 Solution We first show that can be reducible to S. Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal then reject If x= then accepts if M(w) accepts. Slide 551 Solution For an input (M, w), derive M1(x, M1). Slide 552 Solution We show that can be reducible to. Let (M, w) be an input for Let Turing machine M2 be constructed below: M1 Input x If x = then accept If x is not equal then accept if M(w) accepts. Slide 553 Solution For an input (M, w), derive M2(x, M2). Slide 554 Problem 6 Show that A is a Turing recognizable if and only if Slide 555 Proof Part a) Let A be a Turing recognizable language. There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =. It is easy to see that Slide 556 Proof Part b) Assume that via computable function g. It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize. We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w). Slide 557 Proof We are going to design a reduction from to it For Design the TM N as follows Input x if x=0, enters all states except if x=1 run M on w, if M accepts w, enter is in iff N enters all states. Slide 558 Problem 4 c) The language is not a context free language Proof: Assume it is a context free language. Let p be the number from the pumping lemma Consider the string Slide 559 Final Exam December 13, 5:45-7:30pm, Monday Slide 560 Homework 4, 20 Slide 561 Problem 1 1.Let T1={ | M is a Turing machines and accepts infinite number of 0,1-strings of finite length}. Prove that T1 is undecidable. Slide 562 Proof We are going to design a reduction from to T1. For input Turing machine for Design the TM N as follows Input x run M on w, if M accepts w, acceptx is in iff N acccepts each input x. Slide 563 Problem 2 Let T2={ | M is a Turing machines and accepts five of 0,1-strings of finite length}. Prove that T2 is undecidable. Slide 564 Proof We are going to design a reduction from to T2. For Design the TM N as follows Input x if x is not 0,1, 00, 01 or 10, rejects. run M on w if M accepts w, accept x is in iff N accepts 0,1,00,01, and 10. Otherwise, N accepts no string Slide 565 Problem 3 S={ | M is a TM that accepts whenever it accepts w}. Show S is undecidable. Slide 566 Proof We are going to design a reduction from to S. For Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts is in iff N accepts both 01 and 10. Slide 567 Problem 4 Show that A is a Turing recognizable if and only if Slide 568 Proof Part a) Let A be a Turing recognizable language. There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =. It is easy to see that Slide 569 Proof Part b) Assume that via computable function g. It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize. We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w). Slide 570 Problem 5 5. Let S={ | M is a TM and L(M) = { }}. Show that neither S nor its complement is Turing-recognizable. Slide 571 Solution We first show that can be reducible to S. Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal then reject If x= then accepts if M(w) accepts. Slide 572 Solution For an input (M, w), derive M1(x, M1). Slide 573 Solution We show that can be reducible to. Let (M, w) be an input for Let Turing machine M2 be constructed below: M1 Input x If x = then accept If x is not equal then accept if M(w) accepts. Slide 574 Solution For an input (M, w), derive M2(x, M2). Slide 575 Homework 5, 20 Slide 576 Problem 1 Call graphs G and H isomorphic if the nodes of G may be reordered so that it is identical to H. Let ISO={ | G and H are isomorphic graphs}. Show that ISO is in NP. Slide 577 Solution for Problem 1 We design a polynomial time verification algorithm. A mapping f from the vertices of G to those of H is a witness. Check if 1) f is one-one, 2) f is onto, 3) (v,u) is an edge of G iff (f(u),f(v)) is an edge of H. It is easy to see that the verification takes polynomial time. Slide 578 Problem 2 Let MAX-CIQUE={ | the largest clique of G has k vertices}. Whether MAX-CLIQUE is in NP is unknown. Show that if P=NP, then MAX-CLIQUE is in P, and a polynomial time algorithm exists that, for a graph G, finds one of its largest cliques. Slide 579 Solution of Problem 2 Fist step is to find the largest k with (G,k) is in Clique. Try k from 1,2, Check if (G,k) is in Clique Select the largest k. Assume the largest k for (G,k) in Clique is obtained. Slide 580 Solution of Problem 2 Assume the largest k for (G,k) in Clique is obtained. Formulate the problem: (H, k,G) Determine if there is a clique of size k in G and contains all veritces in H. The problem is in NP. Extend H one by one until its size reaches k. Slide 581 Problem 3 Let G represent an undirected graph and let SPATH={ | G contains a simple path of length at most k from a to b} and LPATH={ Complexity D.Moshkovits 600 Example Question: How much space would a TM that decides {a n b n | n>0} require? Note: to count up to n, we need logn bits Slide 602 Complexity D.Moshkovits 601 Graph Connectivity CONN Instance: a directed graph G=(V,E) and two vertices s,t V Problem: To decide if there is a path from s to t in G? An undirected version is also worth considering Slide 603 Complexity D.Moshkovits 602 Graph Connectivity s t Slide 604 Complexity D.Moshkovits 603 CONN is in NL Start at s For i = 1,.., |V| { Non-deterministically choose a neighbor and jump to it Accept if you get to t } If you got here reject! Counting up to |V| requires log|V| space Storing the current position requires log|V| space Slide 605 Complexity D.Moshkovits 604 Configurations The content of the work tape The machines state The head position on the input tape The head position on the work tape The head position on the output tape Which objects determine the configuration of a TM of the new type? If the TM uses logarithmic space, there are polynomially many configurations Slide 606 Complexity D.Moshkovits 605 Log-Space Reductions Definition: A is log-space reducible to B, written A L B, if there exists a log space TM M that, given input w, outputs f(w) s.t. w A iff f(w) B the reduction Slide 607 Complexity D.Moshkovits 606 Do Log-Space Reductions Imply what they should? Suppose A 1 L A 2 and A 2 L; how to construct a log space TM which decides A 1 ? Wrong Solution: w f(w) Use the TM for A 2 to decide if f(w) A 2 Too Large! Slide 608 Complexity D.Moshkovits 607 Log-Space reductions Claim: if 1. A 1 L A 2 f is the log-space reduction 2. A 2 L M is a log-space machine for A 2 Then, A 1 is in L Proof: on input x, in or not-in A 1 : Simulate M and whenever M reads the i th symbol of its input tape run f on x and wait for the i th bit to be outputted Slide 609 Complexity D.Moshkovits 608 NL Completeness Definition: A language B is NL-Complete if 1. B NL 2. For every A NL, A L B. If (2) holds, B is NL-hard Slide 610 Complexity D.Moshkovits 609 Savitchs Theorem Theorem: S(n) log(n) NSPACE(S(n)) SPACE(S(n) 2 ) Proof: First well prove NL SPACE(log 2 n) then, show this implies the general case Slide 611 Complexity D.Moshkovits 610 Savitchs Theorem Theorem: NSPACE(logn) SPACE(log 2 n) Proof: 1.First prove CONN is NL-complete (under log-space reductions) 2.Then show an algorithm for CONN that uses log 2 n space Slide 612 Complexity D.Moshkovits 611 CONN is NL-Complete Theorem: CONN is NL-Complete Proof: by the following reduction: LL s t Does M accept x? Is there a path from s to t? Slide 613 Complexity D.Moshkovits 612 Technicality Observation: Without loss of generality, we can assume all NTMs have exactly one accepting configuration. Slide 614 Complexity D.Moshkovits 613 Configurations Graph A Computation of a NTM M on an input x can be described by a graph G M,x : st (u,v) E if M can move from u to v in one step A vertex per configuration the start configuration the accepting configuration Slide 615 Complexity D.Moshkovits 614 Correctness Claim: For every non-deterministic log-space Turing machine M and every input x, M accepts x iff there is a path from s to t in G M,x Slide 616 Complexity D.Moshkovits 615 CONN is NL-Complete Corollary: CONN is NL-Complete Proof: Weve shown CONN is in NL. Weve also presented a reduction from any NL language to CONN which is computable in log space (Why?) Slide 617 Complexity D.Moshkovits 616 A Byproduct Claim: NL P Proof: Any NL language is log-space reducible to CONN Thus, any NL language is poly-time reducible to CONN CONN is in P Thus any NL language is in P. Slide 618 Complexity D.Moshkovits 617 What Next? We need to show CONN can be decided by a deterministic TM in O(log 2 n) space. Slide 619 Complexity D.Moshkovits 618 The Trick u v... Is there a path from u to v of length d?Is there a vertex z, so there is a path from u to z of size d/2 and one from z to v of size d/2 ? d z d/2 d/2 Slide 620 Complexity D.Moshkovits 619 Recycling Space The two recursive invocations can use the same space Slide 621 Complexity D.Moshkovits 620 The Algorithm Boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if d=1 return FALSE for every vertex v { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } } Slide 622 Complexity D.Moshkovits 621 Example of Savitchs algorithm (a,b,c)=Is there a path from a to b, that takes no more than c steps. 3Log 2 (d) 1 4 3 2 boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } (1,4,3) (1,4,3)(1,2,2) (1,4,3)(1,2,2)TRUE (1,4,3)(2,4,1) (1,4,3)(2,4,1)FALSE (1,4,3)(1,3,2) (1,4,3)(1,3,2)(1,2,1) (1,4,3)(1,3,2)(1,2,1)TRUE (1,4,3)(1,3,2)(2,3,1) (1,4,3)(1,3,2)(2,3,1)TRUE (1,4,3)(1,3,2)TRUE (1,4,3)(3,4,1) (1,4,3)(3,4,1)TRUE (1,4,3) TRUE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2 ) and PATH(v,b, d/2 ) then return TRUE } return FALSE } Slide 623 Complexity D.Moshkovits 622 O(log 2 n) Space DTM Claim: There is a deterministic TM which decides CONN in O(log 2 n) space. Proof: To solve CONN, we invoke PATH(s,t,|V|) The space complexity: S(n)=S(n/2)+O(logn)=O(log 2 n) Slide 624 Complexity D.Moshkovits 623 Conclusion Theorem: NSPACE(logn) SPACE(log 2 n) How about the general case NSPACE(S(n)) SPACE(S 2 (n))? Slide 625 Complexity D.Moshkovits 624 The Padding Argument Motivation: Scaling-Up Complexity Claims space + non-determinism We have: space + determinism can be simulated by We want: space + non-determinism space + determinism can be simulated by Slide 626 Complexity D.Moshkovits 625 Formally NSPACE(s 1 (n)) SPACE(s 2 (n)) NSPACE(s 1 (f(n))) SPACE(s 2 (f(n))) Claim: For any two space constructible functions s 1 (n),s 2 (n) logn, f(n) n: s i (n) can be computed with space s i (n) simulation overhead E.g NSPACE(n) SPACE(n 2 ) NSPACE(n 2 ) SPACE(n 4 ) Slide 627 Complexity D.Moshkovits 626 Idea NTM...... n space: O(s 1 (f(n))).................... f(n) space: s 1 (.) in the size of its input DTM space: O(s 2 (f(n))) 0 0...... n Slide 628 Complexity D.Moshkovits 627 Padding argument Let L NPSPACE(s 1 (f(n))) There is a 3-Tape-NTM M L : babba Input Work |x| O(s 1 (f(|x|))) Slide 629 Complexity D.Moshkovits 628 Padding argument Let L = { x0 f(|x|)-|x| | x L } Well show a NTM M L which decides L in the same number of cells as M L. babba#00000000000000000000000000000000 Input Work f(|x|) O(s 1 (f(|x|)) Slide 630 Complexity D.Moshkovits 629 Padding argument M L 1.Count backwards the number of 0s and check there are f(|x|)-|x| such. 2.Run M L on x. babba00000000000000000000000000000000 Input Work f(|x|) O(s 1 (f(|x|))) In O(log(f(|x|)) space in O(s 1 (f(|x|))) space Slide 631 Complexity D.Moshkovits 630 Padding argument babba00000000000000000000000000000000 Input Work f(|x|) O(s 1 (f(|x|))) Total space: O(s 1 (f(|x|))) Slide 632 Complexity D.Moshkovits 631 Padding Argument We started with L NSPACE(s 1 (f(n))) We showed: L NSPACE(s 1 (n)) Thus, L SPACE(s 2 (n)) Using the DTM for L well construct a DTM for L, which will work in O(s 2 (f(n))) space. Slide 633 Complexity D.Moshkovits 632 Padding Argument The DTM for L will simulate the DTM for L when working on its input concatenated with zeros babba 00000000000000000000000 Input Slide 634 Complexity D.Moshkovits 633 Padding Argument When the input head leaves the input part, just pretend it encounters 0s. maintaining the simulated position (on the imaginary part of the tape) takes O(log(f(|x|))) space. Thus our machine uses O(s 2 (f(|x|))) space. NSPACE(s 1 (f(n))) SPACE(s 2 (f(n))) Slide 635 Complexity D.Moshkovits 634 Savitch: Generalized Version Theorem (Savitch): S(n) log(n) NSPACE(S(n)) SPACE(S(n) 2 ) Proof: We proved NL SPACE(log 2 n). The theorem follows from the padding argument. Slide 636 Complexity D.Moshkovits 635 Corollary Corollary: PSPACE = NPSPACE Proof: Clearly, PSPACE NPSPACE. By Savitchs theorem, NPSPACE PSPACE. Slide 637 Complexity D.Moshkovits 636 Space Vs. Time Weve seen space complexity probably doesnt resemble time complexity: Non-determinism doesnt decrease the space complexity drastically (Savitchs theorem). Well next see another difference: Non-deterministic space complexity classes are closed under completion (Immermans theorem). Slide 638 Complexity D.Moshkovits 637 NON-CONN Instance: A directed graph G and two vertices s,t V. Problem: To decide if there is no path from s to t. Slide 639 Complexity D.Moshkovits 638 NON-CONN Clearly, NON-CONN is coNL-Complete. (Because CONN is NL-Complete. See the coNP lecture) If well show it is also in NL, then NL=coNL. (Again, see the coNP lecture) Slide 640 Complexity D.Moshkovits 639 An Algorithm for NON-CONN 1.Count how many vertices are reachable from s. 2.Take out t and count again. 3.Accept if the two numbers are the same. Well see a log space algorithm for counting reachability Slide 641 Complexity D.Moshkovits 640 N.D. Algorithm for reach s (v, l) reach s (v, l) 1. length = l; u = s 2. while (length > 0) { 3. if u = v return YES 4. else, for all (u V) { 5. if (u, u) E nondeterministic switch: 5.1 u = u; --length; break 5.2 continue } } 6. return NO Takes up logarithmic space This N.D. algorithm might never stop Slide 642 Complexity D.Moshkovits 641 N.D. Algorithm for CR s CR s ( d ) 1. count = 0 2. for all u V { 3. count d-1 = 0 4. for all v V { 5. nondeterministic switch: 5.1 if reach(v, d - 1) then ++count d-1 else fail if (v,u) E then ++count; break 5.2 continue } 6. if count d-1 < CR s (d-1) fail } 7.return count Recursive call! Assume (v,v) E Slide 643 Complexity D.Moshkovits 642 N.D. Algorithm for CR s CR s ( d 1. count = 0 2. for all u V { 3. count d-1 = 0 4. for all v V { 5. nondeterministic switch: 5.1 if reach(v, d - 1) then ++count d-1 else fail if (v,u) E then ++count; break 5.2 continue } 6. if count d-1 < fail } 7.return count Main Algorithm: CR s C = 1 for d = 1..|V| C = CR(d, C) return C parameter C, C) Slide 644 Complexity D.Moshkovits 643 Efficiency Lemma: The algorithm uses O(log(n)) space. Proof: There is a constant number of variables ( d, count, u, v, count d-1 ). Each requires O(log(n)) space (range |V|). Slide 645 Complexity D.Moshkovits 644 Immermans Theorem Theorem[ Immerman/Szelepcsenyi ]: NL=coNL Proof: (1) NON-CONN is NL-Complete (2) NON-CONN NL Hence, NL=coNL. Slide 646 Complexity D.Moshkovits 645 Corollary Corollary: s(n) log(n), NSPACE(s(n))=coNSPACE(s(n)) Proof: By a padding argument. Slide 647 Complexity D.Moshkovits 646 TQBF We can use the insight of Savichs proof to show a language which is complete for PSPACE. We present TQBF, which is the quantified version of SAT. Slide 648 Complexity D.Moshkovits 647 TQBF Instance: a fully quantified Boolean formula Problem: to decide if is true x y z[(x y z) ( x y)] Example: a fully quantified Boolean formula Variables` range is {0,1} Slide 649 Complexity D.Moshkovits 648 TQBF is in PSPACE Theorem: TQBF PSPACE Proof: Well describe a poly-space algorithm A for evaluating : If has no quantifiers: evaluate it If = x( (x)) call A on (0) and on (1); Accept if both are true. If = x( (x)) call A on (0) and on (1); Accept if either is true. in poly time Slide 650 Complexity D.Moshkovits 649 Algorithm for TQBF x y[(x y) ( x y)] y[(0 y) ( 0 y)] y[(1 y) ( 1 y)] (0 0) ( 0 0) (0 1) ( 0 1) (1 1) ( 1 1 ) (1 0) ( 1 0) 101 1 0 1 1 Slide 651 Complexity D.Moshkovits 650 Efficiency Since both recursive calls use the same space, the total space needed is polynomial in the number of variables (the depth of the recursion) TQBF is polynomial-space decidable Slide 652 Complexity D.Moshkovits 651 PSAPCE Completeness Definition: A language B is PSPACE-Complete if 1. B PSPACE 2. For every A PSAPCE, A P B. If (2) holds, then B is PSPACE-hard standard Karp reduction Slide 653 Complexity D.Moshkovits 652 TQBF is PSPACE-Complete Theorem: TQBF is PSAPCE-Complete Proof: It remains to show TQBF is PSAPCE- hard: PP Will the poly-space M accept x? Is the formula true? x 1 x 2 x 3 [] Slide 654 Complexity D.Moshkovits 653 TQBF is PSPACE-Hard Given a TM M for a language L PSPACE, and an input x, let f M,x (u, v), for any two configurations u and v, be the function evaluating to TRUE iff M on input x moves from configuration u to configuration v f M,x (u, v) is efficiently computable Slide 655 Complexity D.Moshkovits 654 Formulating Connectivity The following formula, over variables u,v V and paths length d, is TRUE iff G has a path from u to v of length d (u,v,1) f M,x (u, v) u=v (u,v,d) w x y[((x=u y=w) (x=w y=v)) (x,y,d/2)] w is reachable from u in d/2 steps. v is reachable from w in d/2 steps. simulates AND of (u,w,d/2) and (w,v,d/2) Slide 656 Complexity D.Moshkovits 655 TQBF is PSPACE-Complete Claim: TQBF is PSPACE-Complete Proof: (s,t,|V|) is TRUE iff there is a path from s to t. is constructible in poly-time. Thus, any PSPACE language is poly-time reducible to TQBF, i.e TQBF is PSAPCE-hard. Since TQBF PSPACE, its PSAPCE-Complete. Slide 657 Complexity D.Moshkovits 656 Summary We introduced a new way to classify problems: according to the space needed for their computation. We defined several complexity classes: L, NL, PSPACE. Slide 658 Complexity D.Moshkovits 657 Summary Our main results were: Connectivity is NL-Complete TQBF is PSPACE-Complete Savitchs theorem (NL SPACE(log 2 )) The padding argument (extending results for space complexity) Immermans theorem (NL=coNL) By reducing decidability to reachability