computational complexity dr. colin campbell course: emat20531
Post on 28-Mar-2015
234 Views
Preview:
TRANSCRIPT
Computational Complexity
Dr. Colin CampbellCourse: EMAT20531
Order Notation for Functions
The order notation is used to give a bound on the limiting behaviour of a function.
We write f(x) is O(g(x)) iff
xgmxfmx that such 00
Order example f=100x3+50x2+10
x+100 g=x3
f-mg
m=110
m=120
m=130
m=140
m=150
The big O Notation for Algorithms
We can represent the complexity of an algorithm A by a function
fA(n)=number of steps (or time) it takes for A to complete given an input of size n.
The size of input may be measured in a number of different ways.
Big O NotationA=Do i=1,..,nPrint “hello world”ClearRepeatPrint “End” f(n)=2n+1 O(f(n))=n A is linear in n
Big O Notation: 2
A=Do i=1,…,nDo j=1,…,nPrint “Hello World”ClearRepeatRepeatPrint “End” f(n)=2n2+1 O(f(n) is n2
The Quicksort Algorithm
Input: a list L=(a1,…,an) of numbers Choose an number p at random from L Determine L1 all numbers ai less than p
and L2 all numbers ai greater than or equal to p
Recursively sort L1 and L2
Solution=concatenate L1 and L2
Quicksort Example
L=(1,5,2,7,6,4) sort p=4 [(1,2)(4,5,6,7)=(1,2,4,5,6,7)]L1=(1,2) L2=(5,7,6,4) sort L=(5,7,6,4) p=6 [(4,5)(6,7)=(4,5,6,7)] L1=(5,4), L2=(7,6) sort L=(5,4) [(4,5)] sort L=(7,6) [(6,7)]
Sort the list (1,5,2,7,6,4)
Quicksort: The Worst Case
Given p partitioning into L1 and L2 is O(n) The worst possible case would be that
every time we partition the list we have L1 length n-1 and L2 length 1
For the i’th partition we have a computation O(n-i)
Now (n-i)=n2. Hence the entire computation is O(n2).
Quicksort: On Average On average the split between L1
and L2 will result in two lists of n/2. This is based on the assumption that
the elements of the list are selected from an interval [x,y] according to a uniform distribution
Consider P(pai), the expected number of elements in L2 is nP(pai)
Quicksort: On Average (2) p is a uniformly distributed random
variable taking values in [x,y] Therefore
xy
xaapP ii
ai is also a uniformly distributed random variable from [x,y] with expected value (x+y)/2.
Therefore, the expected value of P(pai) is ½ and the expected number of elements in L2 is n/2
Quicksort: On Average (3)
Consider, the number of such partitions before we end up with strings of length 1. i.e. find k for which n/2k=1
Log(n/2k)=0 log(n)=log(2k)=k Hence, there are on average log(n)
partitions each requiring O(n) computations
This gives O(nlog(n))
Polynomial Time
An algorithm runs in polynomial time if fA(n) is O(nk) for some positive integer k.
E.g. Quicksort always runs in polynomial time even in the worst case when it is O(n2)
The class P refers to those algorithms which run in polynomial time on a Turing Machine as described in the earlier section.
Non-deterministic Turing Machines
A Non-deterministic Turing machine is a Turing machine where for some state qi and symbol S there is more than one instruction beginning qiS.
q1
q2
q3
1:R
0:R
Computation Paths A deterministic Turing machine follows
a computation path. A Non-Deterministic generates a tree of
computations. We can either think of a NDTM as
following all paths simultaneously until it finds an accepting state
Or being a very lucky guesser between when choosing between states
Accepting States and Paths
For NDTM we usually think of binary yes no responses from a computation.
We define a set of accepting state AQ. An accepting path is a path which
terminates in an accepting state. A NDTM accepts an input x if there is
an accepting path for that input.
NDTM Example Let TMP be a deterministic Turing machine
that given a string of 1s and 0s terminates in state q+ if there are an even number of 1 and q- otherwise. AQ={q+}
Further suppose that the start state of TMP is q3
Given a string of 1s we mean substrings to be a string of the same length where some or all of the 1s have been switched to 0.
Eg 111 has substrings 111,110,101,100, 011,010,001,000
NDTM Example: 2
q1 q2
q3
0:R
1:0
1:RTMP
1111
111
10211
011
11111
102
1
11131103 1011
Several deterministic steps1013
1003
0111
0011
0113
0103
0013
0003
q- q+
q+ q- q+ q- q- q+
Non-Deterministic Polynomial
Intuitively, we see that a NDTM is more efficient than a DTM since it can explore many computational paths simultaneously.
NP refer to the class of algorithms which run in polynomial time on a NDTM.
Clearly PNP but does P=NP?
NP Complete
NP-Complete are a collection of decision problems for which there is an algorithm in NP but no known algorithm in P.
Formally, a problem in NP is NP-Complete if any other problem in NP can be reduced to it.
Problem C is reducible to NP-Complete problem D if there is an algorithm A which solves C with a subroutine A+ that solves D, such if A+ ran in polynomial time so would A.
SAT: An NP Complete Problem
SAT: Given a sentence of Propositional Logic S in CNF, is there an allocation of truth values for which S is true?
There is no known algorithm in P which solves SAT
This is basically because all (known) approaches require us (in the worst case) to check all possible truth value allocations. If S has n propositional variables there are 2n possible allocations of truth values.
A NDTP algorithm for SAT Simplify things by assuming that S is in
CNF and each clause has exactly k literals for k>2. (k-SAT)
Notice that for any given allocation of truth values we can check if satisfies S in O(nk) – n is the number of variables in S.
There are 2n(2n-1)..(2n-k+1)/3! Different clauses.
Assume we can check if each is satisfied in 1 step then the computation is O(nk)
SAT Algorithm: 2
Algorithm• For each variable guess either t or f. [O(n) on NDTM]• Check if allocation of truth values satisfies S [O(nk)]
S=(PR Q)(PRQ)
Pt f
Q Qt f t f
R R R Rt t t tf f f f
acc
ept
acc
ept
reje
ct
acc
ept
acc
ept
reje
ct
acc
ept
acc
ept
Proving NP-Completeness
Usually a problem D is shown to be NP-complete by proving that a known NP-complete problem C reduces to it.All NP problems
reduce
NP complete C
D
All NP problems
reduce
Dreduce
CLIQUE
In graph theory a CLIQUE is a set of vertices (nodes) where each is connected to the others.
6
4
3
5
2
1
5
2
1
Clique of size 3
The CLIQUE Problem
For a given graph decide whether or not there is a clique of size k.
This has application in chip design where for example nodes are wires on a chip and edges signify that two wires can overlap.
CLIQUE tells the designer something about how much space is required on the chip.
We shall show that SAT reduces to CLIQUE So CLIQUE is NP-Complete.
CLIQUE and SAT
Let S be a sentence in CNF with k clauses. S=C1C2…Ck We construct a graph G where Vertices={Li:LiCi} i.e. every literal from
every clause Edges={<Li,Lj> LiLj and ij} i.e. edges
between literals from different clauses provides they do not conflict.
S is satisfiable if and only if G has a clique of size k.
CLIQUE and SAT: 2
Example S=(PQ)(PQ)(PR) L1=P, L2=Q, L3=P, L4=Q, L5=P, L6=R
L2
L5
L3
L6
L4
L1Graph G
CLIQUE and SAT: 3 (If) Suppose there is a clique of size k then there
are k non-conflicting literals, one from each clause in S.
Since they are non-conflicting there is a truth allocation that makes each true and hence each clause in S true.
(Only If) Suppose there is an allocation of truth values for which S is true.
At least one literal from each clause must be true
These cannot be conflicting, hence they must form a clique of size k in G
CLIQUE and SAT: 4
L2
L5
L3
L6
L4
L1Graph G
L2
L5L4
L2L6
L4
L6
L4
L1
L5L4
L1
L2
L3
L6
P=t,Q=t,R={t,f}
P=t,Q=t,R=t
P=t,Q=t,R={t,f}P={t,f},Q=t,R=tP=f,Q=t,R=t
Other NP-Complete Problems: The Knapsack Problem
500g100
Weight: wi
Utility: ui
1kg200
300g400
750g80
Maximize 1i
iiux
Quantity: xi
subject to Cwx ii
i
Decision problem: is there a solution for which Vuxi
ii 1and Cwx i
ii ?
Other NP-Complete Problems: The Travelling Salesman Problem
A B
C D
E
F
10
8 718
12
16 25
4
9
Starting at A find the shortest route which visits all towns and then returns to A.
top related