1

UNIT -6

P, NP, and NP-Complete

2

Tractability

Some problems are intractable: as they grow large, we are unable to solve them in reasonable time

What constitutes reasonable time? » Standard working definition: polynomial time» On an input of size n the worst-case running time is O(nk)

for some constant k» O(n2), O(n3), O(1), O(n lg n), O(2n), O(nn), O(n!)» Polynomial time: O(n2), O(n3), O(1), O(n lg n) » Not in polynomial time: O(2n), O(nn), O(n!)

3

Polynomial-Time Algorithms

Are some problems solvable in polynomial time?» Of course: many algorithms we’ve studied provide

polynomial-time solutions to some problems

Are all problems solvable in polynomial time?» No: Turing’s “Halting Problem” is not solvable by any

computer, no matter how much time is given

Most problems that do not yield polynomial-time algorithms are either optimization or decision problems.

4

Optimization/Decision Problems

Optimization Problems» An optimization problem is one which asks, “What is

the optimal solution to problem X?”» Examples:

0-1 Knapsack Fractional Knapsack Minimum Spanning Tree

Decision Problems» An decision problem is one with yes/no answer» Examples:

Does a graph G have a MST of weight W?

5

Optimization/Decision Problems

An optimization problem tries to find an optimal solution A decision problem tries to answer a yes/no question Many problems will have decision and optimization

versions» Eg: Traveling salesman problem

optimization: find hamiltonian cycle of minimum weight decision: is there a hamiltonian cycle of weight k

Some problems are decidable, but intractable: as they grow large, we are unable to solve them in reasonable time» Is there a polynomial-time algorithm that solves the problem?

6

The Class P

P: the class of decision problems that have polynomial-time deterministic algorithms. » That is, they are solvable in O(p(n)), where p(n) is a polynomial on n

» A deterministic algorithm is (essentially) one that always computes the correct answer

Why polynomial?» if not, very inefficient

» nice closure properties the sum and composition of two polynomials are always polynomials too

7

Sample Problems in P

Fractional Knapsack MST Sorting Others?

8

The class NP

NP: the class of decision problems that are solvable in polynomial time on a nondeterministic machine (or with a nondeterministic algorithm)

(A determinstic computer is what we know) A nondeterministic computer is one that can “guess” the right

answer or solution » Think of a nondeterministic computer as a parallel machine that can

freely spawn an infinite number of processes Thus NP can also be thought of as the class of problems

» whose solutions can be verified in polynomial time Note that NP stands for “Nondeterministic Polynomial-time”

9

Sample Problems in NP

Fractional Knapsack MST Sorting Others?

» Hamiltonian Cycle (Traveling Salesman)

» Graph Coloring

» Satisfiability (SAT) the problem of deciding whether a given

Boolean formula is satisfiable

10

NP-complete problems

A decision problem D is NP-complete iff1. D NP

2. every problem in NP is polynomial-time reducible to D

11

NP-Hard and NP-Complete

If R is polynomial-time reducible to Q, we denote this R p Q

Definition of NP-Hard and NP-Complete: » If all problems R NP are polynomial-time reducible to

Q, then Q is NP-Hard

» We say Q is NP-Complete if Q is NP-Hard and Q NP

If R p Q and R is NP-Hard, Q is also NP-Hard (why?)

12

Boolean satisfiability problem

In computer science, the Boolean Satisfiability Problem (sometimes called Propositional Satisfiability Problem and abbreviated as SATISFIABILITY or SAT) is the problem of determining if there exists an interpretation that satisfies a given Boolean formula.

In other words, it asks whether the variables of a given Boolean formula can be consistently replaced by the values TRUE or FALSE in such a way that the formula evaluates to TRUE.

If this is the case, the formula is called satisfiable.

13

On the other hand, if no such assignment exists, the function expressed by the formula is identically FALSE for all possible variable assignments and the formula is unsatisfiable.

For example, the formula "a AND NOT b" is satisfiable because one can find the values a = TRUE and b = FALSE, which make (a AND NOT b) = TRUE. In contrast, "a AND NOTa" is unsatisfiable.

14

SAT is one of the first problems that was proven to be NP-complete.

This means that all problems in the complexity classNP, which includes a wide range of natural decision and optimization problems, are in a technical sense at most that difficult to solve as SAT.

There is no known algorithm that efficiently solves SAT, and it is generally believed that no such algorithm exists; yet this belief has not been proven mathematically, and resolving the question whether SAT has an efficient algorithm is equivalent to the P versus NP problem, which is the most famous open problem in the theory of computing.

15

Despite the fact that no algorithms are known that solve SAT efficiently, correctly, and for all possible input instances, many instances of SAT that occur in practice, such as in artificial intelligence, circuit design and automatic theorem proving, can actually be solved rather efficiently using heuristical (intelligent guesswork) SAT-solvers.

Such algorithms are not believed to be efficient on all SAT instances, but experimentally these algorithms tend to work well for many practical applications.

16

Basic definitions and terminology

A propositional logic formula, also called Boolean expression, is built from variables, operators AND (conjunction, also denoted by ), OR (disjunction, ), NOT (negation, ¬), and parentheses. ∧ ∨A formula is said to be satisfiable if it can be made TRUE by assigning appropriate logical values (i.e. TRUE, FALSE) to its variables.

The Boolean satisfiability problem (SAT) is, given a formula, to check whether it is satisfiable.

This decision problem is of central importance in various areas of computer science, including theoretical computer science, complexitytheory, algorithmics, cryptography and artificial intelligence.

17

There are several special cases of the Boolean satisfiability problem in which the formulas are required to have a particular structure.

A literal is either a variable, then called positive literal, or the negation of a variable, then called negative literal. A clause is a disjunction of literals (or a single literal).

A clause is called Horn clause if it contains at most one positive literal.

A formula is in conjunctive normal form (CNF) if it is a conjunction of clauses (or a single clause).

18

For example, "x1" is a positive literal, "¬x2" is a negative literal, "x1  ¬∨ x2" is a clause, and "(x1  ¬∨ x2) (¬∧ x1   ∨ x2   ∨ x3) ¬∧ x1" is a formula in conjunctive normal form, its 1st and 3rd clause are Horn clauses, but its 2nd clause is not.

The formula is satisfiable, choosing x1=FALSE, x2=FALSE, and x3 arbitrarily, since (FALSE ¬FALSE) (¬FALSE FALSE  ∨ ∧ ∨ ∨ x3) ¬FALSE ∧evaluates to (FALSE TRUE) (TRUE FALSE  ∨ ∧ ∨ ∨ x3) TRUE, and in ∧turn to TRUE TRUE TRUE (i.e. to TRUE). In contrast, the CNF ∧ ∧formula a  ¬∧ a, consisting of two clauses of one literal, is unsatisfiable, since for a=TRUE anda=FALSE it evaluates to TRUE ¬TRUE (i.e. to FALSE) ∧and FALSE ¬FALSE (i.e. again to FALSE), respectively.∧

19

The Satisfiability (SAT) Problem

Satisfiability (SAT):» Given a Boolean expression on n variables, can we

assign values such that the expression is TRUE?

» Ex: ((x1 x2) ((x1 x3) x4)) x2

» Seems simple enough, but no known deterministic polynomial time algorithm exists

» Easy to verify in polynomial time!

20

Example: CNF satisfiability This problem is in NP. Nondeterministic algorithm:

» Guess truth assignment» Check assignment to see if it satisfies CNF formula

Example: (A¬B ¬C ) (¬A B) (¬ B D F ) (F ¬ D)

Truth assignments: A B C D E F 0 1 1 0 1 0 1 0 0 0 0 1 1 1 0 0 0 1 ... (how many more?)

Checking phase: Θ(n)

21Complexity©D.Moshkovitz

21

Conclusion: SAT is NP-Complete

For any language A in NP,

testing membership in A

can be reduced to...

satisfiability problem

22Complexity©D.Moshkovitz

22

Co-NP

Revisiting the Map

NPP

NPC

SAT

23Complexity©D.Moshkovitz

23

Looking Forward

From now on, in order to show some NP problem is NP-Complete, we merely need to reduce SAT to it.

any NP problem

can be reduced to...

SAT

new NP problem

can be reduced to...We’ve already shown this

will imply the new problem is in NPC

24Complexity©D.Moshkovitz

24

... and Beyond!

Moreover, every NP-Complete problem we discover, provides us with a new way for showing problems to be NP-Complete.

any NP problem

can be reduced to...

Known NP-hard problem

new NP problem

can be reduced to...

25

26

Review: P And NP Summary

P = set of problems that can be solved in polynomial time» Examples: Fractional Knapsack, …

NP = set of problems for which a solution can be verified in polynomial time» Examples: Fractional Knapsack,…, Hamiltonian Cycle, CNF

SAT, 3-CNF SAT Clearly P NP Open question: Does P = NP?

» Most suspect not» An August 2010 claim of proof that P ≠ NP, by Vinay

Deolalikar, researcher at HP Labs, Palo Alto, has flaws

27

8- 27

Chapter 8

The Theory of NP-Completeness

28

KINDS OF PROBLEMS Decision problems Optimization problems

8- 28

29

8- 29

Decision problems The solution is simply “Yes” or “No”. Optimization problems are more difficult. e.g. the traveling salesperson problem

» Optimization version:

Find the shortest tour

» Decision version:

Is there a tour whose total length is less than or equal to a constant c ?

30

8- 30

Solving an optimization problem by a decision algorithm :

Solving TSP optimization problem by a decision algorithm :» Give c1 and test (decision algorithm)

Give c2 and test (decision algorithm)

Give cn and test (decision algorithm)

» We can easily find the smallest ci

31

8- 31

CATEGORIES OF PROBLEMS

Undecidable problems

Intractable problems

Tractable problems

32

Non deterministic Algorithms

Deterministic Algorithms : Result of every operation is uniquely defined.

Nondeterministic Algorithms : Results of operations are not uniquely defined but are limited to specified sets of possibilities.

To specify nondeterministic algorithms, we introduce three new functions:

Choice(S) – arbitrarily chooses one of the elements of set S. Failure() – signals an unsuccessful completion. Success() – signals a successful completion.

Nondeterministic machine : A machine capable of executing a non

deterministic algorithm.

8- 32

33

Nondeterministic search

j := Choice(1, n);

if A[j] = x then

{

write(j);

success();

}

write(0);

Failure();

8- 33

34

Class P

A decision problem D is solvable in polynomial time or in the class P, if there exists an algorithm A such that

A takes instances of D as inputs. A always outputs the correct answer “Yes” or “No”. There exists a polynomial p such that the execution of A

on inputs of size n always terminates in p(n) or fewer step

8- 34

35

Class NP

A decision problem solvable by non deterministic algorithms in polynomial Time or in the class NP if there exists an algorithm A such that

A takes as inputs potential witnesses for “yes” answers to problem D. A correctly distinguishes true witnesses from false witnesses. There exists a polynomial p such that for each potential witnesses of

each instance of size n of D, the execution of the algorithm A takes at most p(n) steps.

8- 35

36

P and NP

P is the set of all decision problems solvable by deterministic algorithms in polynomial time.

NP is the set of all decision problems solvable by nondeterministic algorithms in polynomial time.

P is subset of NP

8- 36

37

Polynomial Time Reduction

Problem L1 reduces to L2 (L1 α L2) if and only if there is a way to solve L1 by a deterministic polynomial time algorithm using a deterministic algorithm that solves L2 in polynomial time.

If there is a polynomial time algorithm for L2, then we can solve L1 in polynomial time. α is a transitive relation.

Example: The Hamiltonian cycle problem is polynomial-time reducible

to the decision version of TSP.

8- 37

38

NP-Hard and NP-Complete

NP-HARD : Problem P is said to be NP-hard, If all problems Q Є NP, are reducible to P. Informally, P is “as hard as” all the problems in NP.

NP-COMPLETE : Problem P is said to be NP-Complete, If P is NP-hard, and P Є NP.

Informally, P is one of the hardest problems in NP.

If P α R, and P is NP-Complete, then R is also NP-Complete.

8- 38

39

8- 39

P: the class of problems which can be solved by a deterministic polynomial algorithm.

NP : the class of decision problem which can be solved by a non-deterministic polynomial algorithm.

NP-hard: the class of problems to which every NP problem reduces.

NP-complete (NPC): the class of problems which are NP-hard and belong to NP.

40

8- 40

Some concepts of NPC

Definition of reduction: Problem A reduces to problem B (A B) iff A can be solved by a deterministic polynomial time algorithm using a deterministic algorithm that solves B in polynomial time.

Up to now, none of the NPC problems can be solved by a deterministic polynomial time algorithm in the worst case.

It does not seem to have any polynomial time algorithm to solve the NPC problems.

41

8- 41

The theory of NP-completeness always considers the worst case.

The lower bound of any NPC problem seems to be in the order of an exponential function.

Not all NP problems are difficult. (e.g. the MST problem is an NP problem.)

If A, B NPC, then A B and B A.

Theory of NP-completeness:If any NPC problem can be solved in polynomial time, then all NP problems can be solved in polynomial time. (NP = P)

42

8- 42

The satisfiability problem

The satisfiability problem» The logical formula :

x1 v x2 v x3

& - x1

& - x2 the assignment :

x1 ← F , x2 ← F , x3 ← Twill make the above formula true .

(-x1, -x2 , x3) represents x1 ← F , x2 ← F , x3 ← T

43

8- 43

If there is at least one assignment which satisfies a formula, then we say that this formula is satisfiable; otherwise, it is unsatisfiable.

An unsatisfiable formula : x1 v x2

& x1 v -x2

& -x1 v x2

& -x1 v -x2

44

8- 44

Definition of the satisfiability problem: Given a Boolean formula, determine whether this formula is satisfiable or not.

  A literal : xi or -xi A clause : x1 v x2 v -x3 Ci A formula : conjunctive normal form (CNF)

C1& C2 & … & Cm

45

8- 45

Nondeterministic algorithms A nondeterminstic algorithm consists of phase 1: guessing phase 2: checking If the checking stage of a nondeterministic algorithm is

of polynomial time-complexity, then this algorithm is called an NP (nondeterministic polynomial) algorithm.

NP problems : (must be decision problems)» e.g. searching, MST

sortingsatisfiability problem (SAT)traveling salesperson problem (TSP)

46

8- 46

Decision problems

Decision version of sorting:

Given a1, a2,…, an and c, is there a permutation of ai

s ( a1, a2

, … ,an ) such that a∣ 2

–a1 + a∣ ∣ 3

–a2 + … ∣

+ a∣ n–an-1

∣＜ c ? Not all decision problems are NP problems

» E.g. halting problem : Given a program with a certain input data, will the program

terminate or not? NP-hard Undecidable

47

8- 47

Nondeterministic operations and functions

[Horowitz 1998] Choice(S) : arbitrarily chooses one of the elements in set S Failure : an unsuccessful completion Success : a successful completion Nonderministic searching algorithm:

j ← choice(1 : n) /* guessing */

if A(j) = x then success /* checking */

else failure

48

8- 48

A nondeterministic algorithm terminates unsuccessfully iff there does not exist a set of choices leading to a success signal.

The time required for choice(1 : n) is O(1). A deterministic interpretation of a non-deterministic

algorithm can be made by allowing unbounded parallelism in computation.

49

8- 49

Nondeterministic sorting

B ← 0 /* guessing */for i = 1 to n do

j ← choice(1 : n) if B[j] ≠ 0 then failure B[j] = A[i]

/* checking */for i = 1 to n-1 do

if B[i] > B[i+1] then failure success

50

8- 50

Nondeterministic SAT

/* guessing */

for i = 1 to n do

xi ← choice( true, false )

/* checking */

if E(x1, x2, … ,xn) is true then success

else failure

51

8- 51

Cook’s theorem

NP = P iff the satisfiability problem is a P problem.

SAT is NP-complete. It is the first NP-complete

problem. Every NP problem reduces to

SAT.

Stephen Arthur Cook

(1939~)