theory of computation cs3102 spring 2014njb2b/theory/theory_lecture23_web.pdftheory of computation...
TRANSCRIPT
![Page 1: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/1.jpg)
Nathan Brunelle
Department of
Computer Science
University of Virginia
www.cs.virginia.edu/~njb2b/theory
Theory of Computation CS3102 – Spring 2014
A tale of computers, math, problem solving, life, love and tragic death
![Page 2: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/2.jpg)
Complexity Classes
Def: DTIME(t(n))={L | L is decidable within
time O(t(n)) by some deterministic TM}
Def: NTIME(t(n))={L | L is decidable within
time O(t(n)) by some non-deterministic TM}
Def: DSPACE(s(n))={L | L decidable within
space O(s(n)) by some deterministic TM}
Def: NSPACE(s(n))={L | L decidable within
space O(s(n)) by some non-deterministic TM}
![Page 3: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/3.jpg)
Non-Robustness of TM Complexity • Computability: all variations on TMs have the same
computing power – If there is a multi-tape TM that can decide L, there is a
regular TM that can decide L. – If there is a nondeterministic TM that can decide L, there is a
deterministic TM that can decide L.
• Complexity: variations on TM can solve problems in different times – Is a multi-tape TM faster than a regular TM? – Is a nondeterministic TM faster than a regular TM?
![Page 4: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/4.jpg)
Complexity Class P
P = U k
DTIME(nk)
P is the class of languages that can be decided in Polynomial Time on a deterministic, single-tape Turing machine.
![Page 5: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/5.jpg)
Classes in P
a) TIME(n2)
b) TIME(O(n7))
c) TIME(O(2n))
d) Class of languages that can be decided in Polynomial Time by a 2-tape TM
e) Class of languages that can be decided in Polynomial Time by a nondeterministic TM
Unknown! This is the P = NP question. Focus of this class…
Yes! We can simulate each step of a 2-tape TM by making 2 passes over the whole tape ~ 2(n+t(n))
(See Theorem 7.8)
![Page 6: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/6.jpg)
P Examples
• What are some examples of problems in P?
– Arithmetic
– Matrix Multiplication
– Matrix Inversion
– Sorting
![Page 7: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/7.jpg)
Non-deterministic polynomial time
• Deterministic Polynomial Time: TM takes at most O(nc) steps to accept a string of length n
• Non-deterministic Polynomial Time: The TM takes at most O(nc) steps on some path to accept a string of length n
![Page 8: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/8.jpg)
Complexity Class NP
NP = U k
NTIME(nk)
P is the class of languages that can be decided in Polynomial Time on a non-deterministic Turing Machine.
![Page 9: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/9.jpg)
Alternative Definition • A language is in NP if some execution path answers in
polynomial time
• What if we had a “list of directions”?
• Some information that told us which path accepted quickly
• L is in NP if there is a “verifier” which given a string x in L and a polynomial length “witness” it can verify x is a member of L in polynomial time
• NP = “easy to verify” problems
![Page 10: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/10.jpg)
Neil Immerman
Alternate Definition of NP Using Descriptive complexity
![Page 11: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/11.jpg)
Argue why these are in NP • Sets of integers which sum to 0
– Add them together, check if sum is 0
• Sets of integers which have a subset with sum 0 – Given a “witness” w (a particular subset which sums to 0) we can:
• Verify w is indeed a subset of the input • Verify that w sums to 0
• Sets of integers which have a subset of size 3 which sums to 0 – Try all subsets of size 3, check if any sum to 0 – Called “3-Sum”
• Primality testing: Given an integer, is it prime? – Given a “witness” (a pair of numbers) we can:
• Verify the product of these numbers gives the input (just 1 multiplication) • Surprisingly, this can be solved in Polynomial time (AKS, 2002)
![Page 12: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/12.jpg)
The Class P and the Class NP
• P = { L | L is accepted by a deterministic Turing Machine in polynomial time }
• NP = { L | L is accepted by a non-deterministic Turing Machine in polynomial time }
• They are sets of languages
![Page 13: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/13.jpg)
Most important CS Problem
• Does P = NP?
• Are all “easily verifiable problems” also “easily computable”?
• How much more efficient are non-deterministic Turing Machines?
![Page 14: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/14.jpg)
Progress • P = NP if every NP problem has a deterministic
polynomial time algorithm • We could find an algorithm for every NP problem • Seems… hard…
• Study an “archetypical” example
– An NP problem which is in P iff P=NP – Called Complete
• We need: At least one such problem
![Page 15: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/15.jpg)
NP
NP Hardness & Completeness Def: A problem L’ is NP-hard if:
(1) Every L in NP reduces to L’ in polynomial time.
Def: A problem L’ is NP-complete if:
(1) L is NP-hard; and
(2) L is in NP.
If one NPC problem is in P P=NP
P
NP-Hard
![Page 16: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/16.jpg)
NP Completeness Benefits 1. Saves time & effort of trying to solve intractable problems
efficiently;
2. Saves money by not separately working to efficiently solve different problems;
3. Helps systematically build on & leverage the work (or lack of progress) of others;
4. Transformations can be used to solve new problems by reducing them to known ones;
5. Illuminates the structure & complexity of seemingly unrelated problems;
![Page 17: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/17.jpg)
NP Completeness Benefits 6. Informs as to when we should use approximate solutions vs.
exact ones;
7. Helps understand the ubiquitous concept of parallelism (via non-determinism);
8. Enabled vast, deep, and general studies of other “completeness” theories;
9. Helps explain why verifying proofs seems to be easier than constructing them;
10. Illuminates the fundamental nature of algorithms and computation;
![Page 18: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/18.jpg)
NP Completeness Benefits 11. Gave rise to new and novel mathematical approaches, proofs,
and analyses;
12. Robustly decouples / abstracts complexity from underlying computational models;
13. Gives disciplined techniques for identifying “hardest” problems / languages;
14. Forged new unifications between computer science, mathematics, and logic;
15. NP-Completeness is interesting and fun!
![Page 19: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/19.jpg)
Our “Favorite” NP Complete Problem Satisfiability (SAT): Given a Boolean expression in conjuctive
normal form, is there some assignment of T/F to its variables such that the expression resolves to “True”?
Note: we may restrict each disjunction to have 3 variables (3-SAT)
Examples:
¬𝑎 ∨ 𝑏 ∨ ¬𝑐 ∧ 𝑎 ∨ 𝑑 ∨ 𝑐 ∧ (𝑑 ∨ 𝑏 ∨ ¬𝑒)
Satisfied if: 𝑎 = 𝐹, 𝑏 = 𝐹, 𝑐 = 𝑇, 𝑑 = 𝑇
𝑎 ∨ 𝑏 ∧ ¬𝑎 ∨ ¬𝑏 ∧ (¬𝑎 ∨ 𝑏) ∧ (𝑎 ∨ ¬𝑏)
Can’t be Satisfied!
![Page 20: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/20.jpg)
Stephen Cook
Leonid Levin
The Cook/Levin Theorem Theorem [Cook/Levin, 1971]: SAT is NP-complete. Proof idea: given a non-deterministic polynomial time TM M and input w, construct a CNF formula that is satisfiable iff M accepts w.
Create boolean variables:
q[i,k] at step i, M is in state k
h[i,k] at step i, M’s RW head scans tape cell k
s[i,j,k] at step i, M’s tape cell j contains symbol Sk
M halts in polynomial time p(n)
total # of variables is polynomial in p(n)
Qk
![Page 21: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/21.jpg)
Stephen Cook
Leonid Levin
Add clauses to the formula to enforce necessary restrictions on how M operates / runs:
• At each time i:
M is in exactly 1 state
r/w head scans exactly 1 cell
All cells contain exactly 1 symbol
• At time 0 M is in its initial state
• At time P(n) M is in a final state
• Transitions from step i to i+1 all obey M's transition function
Resulting formula is satisfiable iff M accepts w!
Qk
The Cook/Levin Theorem
![Page 22: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/22.jpg)
“Guess and Verify” Approach Note: SAT NP.
Idea: Nondeterministically “guess” each Boolean variable value, and then verify the guessed solution.
polynomial-time nondeterministic algorithm NP
This “guess & verify” approach is general.
Idea: “Guessing” is usually trivially fast ( NP)
NP can be characterized by the “verify” property:
NP set of problems for which proposed solutions can be quickly verified
set of languages for which string membership can be quickly tested.
![Page 23: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/23.jpg)
Historical Note The Cook/Levin theorem was independently proved by Stephen Cook
and Leonid Levin
• Denied tenure at Berkeley (1970) • Invented NP completeness (1971) • Won Turing Award (1982)
• Student of Andrei Kolmogorov • Seminal paper obscured by Russian, style, and Cold War
![Page 24: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/24.jpg)
![Page 25: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/25.jpg)
![Page 26: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/26.jpg)
An NP-Complete Encyclopedia Classic book: Garey & Johnson, 1979
• Definitive guide to NP-completeness
• Lists hundreds of NP-complete problems
• Gives reduction types and refs
Michael Garey David Johnson
![Page 27: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/27.jpg)
NP-Completeness Proof Method
• To show that Q is NP-Complete:
• 1) Show that Q is in NP – Usually by “guess and verify”
• 2) Pick an instance, R, of your favorite NP-Complete problem (ex: 3-SAT)
• 3) Show a polynomial algorithm to transform R into an instance of Q
![Page 28: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/28.jpg)
Reducing Reduction Proofs • Conjecture: A has some property Y.
• Proof by reduction from B to A: – Assume A has Y. Then, we know there is an M that decides A.
– We already know B does not have property Y.
– Show how to build S that solves B using M.
• Since we know B does not have Y, but having S would imply B has Y, S cannot exist. Therefore, M cannot exist, and A does not have Y.
![Page 29: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/29.jpg)
Undecidability Proofs • Conjecture: A has some property Y. • Proof by reduction from B to A:
– Assume A has Y. Then, we know an M exists. – We already know B does not have property Y. – Show how to build S that solves B using M.
• Since we know B does not have Y, but having S would imply B has Y, S cannot exist. Therefore, M cannot exist, and A does not have Y.
Undecidability: Y = “can be decided by a TM” B = a known undecidable problem (e.g., ATM, HALTTM, EQTM, …) M = “a TM that decides A”
![Page 30: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/30.jpg)
NP-Hardness Proofs • Conjecture: A has some property Y. • Proof by reduction from B to A:
– Assume A has Y. Then, we know an M exists. – We already know B does not have property Y. – Show how to build S that solves B using M.
• Since we know B does not have Y, but having S would imply B has Y, S cannot exist. Therefore, M cannot exist, and A does not have Y.
NP-Hardness: Y = “is NP-Hard” B = a known NP-Hard problem (e.g., 3-SAT, SUBSET-SUM, …) M = “a TM that decides A in polynomial-time”
![Page 31: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/31.jpg)
Example
• Suppose we know ATM is undecidable, but do not yet know if EQTM is.
EQTM = { <M1, M2> | M1 and M2 are TMs where L(M1) = L(M2) }
ATM = { <M>, w | M is TM, w is string, w in L(M) }
Conjecture: EQTM is undecidable.
What do we need to do to prove conjecture?
Reduce from ATM to EQTM: show that a solver for EQTM could be used to solve ATM.
Pitfall #1: Make sure you do reduction in right direction. Showing how to solve B using MA, shows A is as hard as B.
![Page 32: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/32.jpg)
Building Solvers
EQTM = { <M1, M2> | M1 and M2 are TMs where L(M1) = L(M2) }
ATM = { <M, w> | M is TM, w is string, w in L(M) }
Reduce from EQTM to ATM: show that MEQ, a solver for EQTM can be used to solve ATM.
Conjecture: EQTM is undecidable.
Pitfall #2: Get the inputs to the solver to match correctly. To solve B using MA, must transform inputs to B into inputs to A.
B =
A =
MB(<M, w>): machine that decides ATM
Simulate MEQ on <M1, M2>: M1 = a TM that simulates M running on w
M2 = a TM that always accepts If it accepts, accept; if it rejects, reject.
![Page 33: Theory of Computation CS3102 Spring 2014njb2b/theory/Theory_lecture23_web.pdfTheory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and](https://reader030.vdocuments.us/reader030/viewer/2022013110/5e2e55f7a24f7a1a6816345a/html5/thumbnails/33.jpg)
Legal Transformations • Undecidability proofs: your transformation can do
anything a TM can do, but must be guaranteed to terminate – E.g., cannot include, “simulate M and if it halts, accept”
• NP-Hardness proofs: your transformation must finish in polynomial time – E.g., cannot include, “do an exponential search to find the
answer, and output that”