Presentation is loading. Please wait.

Presentation is loading. Please wait.

Computational Complexity Dr. Colin Campbell Course: EMAT20531.

Similar presentations

Presentation on theme: "Computational Complexity Dr. Colin Campbell Course: EMAT20531."— Presentation transcript:

1 Computational Complexity Dr. Colin Campbell Course: EMAT20531

2 Order Notation for Functions The order notation is used to give a bound on the limiting behaviour of a function. We write f(x) is O(g(x)) iff

3 Order example f=100x 3 +50x 2 +10x +100 g=x 3 f-mg m=110 m=120 m=130 m=140 m=150

4 The big O Notation for Algorithms We can represent the complexity of an algorithm A by a function f A (n)=number of steps (or time) it takes for A to complete given an input of size n. The size of input may be measured in a number of different ways.

5 Big O Notation A= Do i=1,..,n Print hello world Clear Repeat Print End f(n)=2n+1 O(f(n))=n A is linear in n

6 Big O Notation: 2 A= Do i=1,…,n Do j=1,…,n Print Hello World Clear Repeat Print End f(n)=2n 2 +1 O(f(n) is n 2

7 The Quicksort Algorithm Input: a list L=(a 1,…,a n ) of numbers Choose an number p at random from L Determine L 1 all numbers a i less than p and L 2 all numbers a i greater than or equal to p Recursively sort L 1 and L 2 Solution=concatenate L 1 and L 2

8 Quicksort Example L=(1,5,2,7,6,4) sort p=4 [(1,2) (4,5,6,7)=(1,2,4,5,6,7)] L1=(1,2) L2=(5,7,6,4) sort L=(5,7,6,4) p=6 [(4,5) (6,7)=(4,5,6,7)] L1=(5,4), L2=(7,6) sort L=(5,4) [(4,5)] sort L=(7,6) [(6,7)] Sort the list (1,5,2,7,6,4)

9 Quicksort: The Worst Case Given p partitioning into L 1 and L 2 is O(n) The worst possible case would be that every time we partition the list we have L 1 length n-1 and L 2 length 1 For the ith partition we have a computation O(n-i) Now (n-i)=n 2. Hence the entire computation is O(n 2 ).

10 Quicksort: On Average On average the split between L1 and L2 will result in two lists of n/2. This is based on the assumption that the elements of the list are selected from an interval [x,y] according to a uniform distribution Consider P(p a i ), the expected number of elements in L 2 is n P(p a i )

11 Quicksort: On Average (2) p is a uniformly distributed random variable taking values in [x,y] Therefore a i is also a uniformly distributed random variable from [x,y] with expected value (x+y)/2. Therefore, the expected value of P(p a i ) is ½ and the expected number of elements in L 2 is n/2

12 Quicksort: On Average (3) Consider, the number of such partitions before we end up with strings of length 1. i.e. find k for which n/2 k =1 Log(n/2 k )=0 log(n)=log(2 k )=k Hence, there are on average log(n) partitions each requiring O(n) computations This gives O(nlog(n))

13 Polynomial Time An algorithm runs in polynomial time if f A (n) is O(n k ) for some positive integer k. E.g. Quicksort always runs in polynomial time even in the worst case when it is O(n 2 ) The class P refers to those algorithms which run in polynomial time on a Turing Machine as described in the earlier section.

14 Non-deterministic Turing Machines A Non-deterministic Turing machine is a Turing machine where for some state q i and symbol S there is more than one instruction beginning q i S. q1q1 q2q2 q3q3 1:R 0:R

15 Computation Paths A deterministic Turing machine follows a computation path. A Non-Deterministic generates a tree of computations. We can either think of a NDTM as following all paths simultaneously until it finds an accepting state Or being a very lucky guesser between when choosing between states

16 Accepting States and Paths For NDTM we usually think of binary yes no responses from a computation. We define a set of accepting state AQ. An accepting path is a path which terminates in an accepting state. A NDTM accepts an input x if there is an accepting path for that input.

17 NDTM Example Let TMP be a deterministic Turing machine that given a string of 1s and 0s terminates in state q+ if there are an even number of 1 and q- otherwise. AQ={q+} Further suppose that the start state of TMP is q 3 Given a string of 1s we mean substrings to be a string of the same length where some or all of the 1s have been switched to 0. Eg 111 has substrings 111,110,101,100, 011,010,001,000

18 NDTM Example: 2 q1q1 q2q2 q3q3 0:R 1:0 1:R TMP Several deterministic steps q- q+ q-q+q- q+

19 Non-Deterministic Polynomial Intuitively, we see that a NDTM is more efficient than a DTM since it can explore many computational paths simultaneously. NP refer to the class of algorithms which run in polynomial time on a NDTM. Clearly P NP but does P=NP?

20 NP Complete NP-Complete are a collection of decision problems for which there is an algorithm in NP but no known algorithm in P. Formally, a problem in NP is NP-Complete if any other problem in NP can be reduced to it. Problem C is reducible to NP-Complete problem D if there is an algorithm A which solves C with a subroutine A+ that solves D, such if A+ ran in polynomial time so would A.

21 SAT: An NP Complete Problem SAT: Given a sentence of Propositional Logic S in CNF, is there an allocation of truth values for which S is true? There is no known algorithm in P which solves SAT This is basically because all (known) approaches require us (in the worst case) to check all possible truth value allocations. If S has n propositional variables there are 2 n possible allocations of truth values.

22 A NDTP algorithm for SAT Simplify things by assuming that S is in CNF and each clause has exactly k literals for k>2. (k-SAT) Notice that for any given allocation of truth values we can check if satisfies S in O(n k ) – n is the number of variables in S. There are 2n(2n-1)..(2n-k+1)/3! Different clauses. Assume we can check if each is satisfied in 1 step then the computation is O(n k )

23 SAT Algorithm: 2 Algorithm For each variable guess either t or f. [O(n) on NDTM] Check if allocation of truth values satisfies S [O(n k )] S=(P R Q) (P R Q) P t f Q Q tf tf R R R R tt tt ff f f accept reject accept reject accept

24 Proving NP-Completeness Usually a problem D is shown to be NP- complete by proving that a known NP- complete problem C reduces to it. All NP problems reduce NP complete C D All NP problems reduce D

25 CLIQUE In graph theory a CLIQUE is a set of vertices (nodes) where each is connected to the others Clique of size 3

26 The CLIQUE Problem For a given graph decide whether or not there is a clique of size k. This has application in chip design where for example nodes are wires on a chip and edges signify that two wires can overlap. CLIQUE tells the designer something about how much space is required on the chip. We shall show that SAT reduces to CLIQUE So CLIQUE is NP-Complete.

27 CLIQUE and SAT Let S be a sentence in CNF with k clauses. S=C 1 C 2 … C k We construct a graph G where Vertices={L i :L i C i } i.e. every literal from every clause Edges={ L i L j and i j} i.e. edges between literals from different clauses provides they do not conflict. S is satisfiable if and only if G has a clique of size k.

28 CLIQUE and SAT: 2 Example S=(P Q) ( P Q) (P R) L1=P, L2=Q, L3= P, L4=Q, L5=P, L6=R L2L2 L5L5 L3L3 L6L6 L4L4 L1L1 Graph G

29 CLIQUE and SAT: 3 (If) Suppose there is a clique of size k then there are k non-conflicting literals, one from each clause in S. Since they are non-conflicting there is a truth allocation that makes each true and hence each clause in S true. (Only If) Suppose there is an allocation of truth values for which S is true. At least one literal from each clause must be true These cannot be conflicting, hence they must form a clique of size k in G

30 CLIQUE and SAT: 4 L2L2 L5L5 L3L3 L6L6 L4L4 L1L1 Graph G L2L2 L5L5 L4L4 L2L2 L6L6 L4L4 L6L6 L4L4 L1L1 L5L5 L4L4 L1L1 L2L2 L3L3 L6L6 P=t,Q=t,R={t,f} P=t,Q=t,R=t P=t,Q=t,R={t,f} P={t,f},Q=t,R=t P=f,Q=t,R=t

31 Other NP-Complete Problems: The Knapsack Problem 500g 100 Weight: w i Utility: u i 1kg g g 80 Maximize Quantity: x i subject to Decision problem: is there a solution for which and ?

32 Other NP-Complete Problems: The Travelling Salesman Problem AB CD E F Starting at A find the shortest route which visits all towns and then returns to A.

Download ppt "Computational Complexity Dr. Colin Campbell Course: EMAT20531."

Similar presentations

Ads by Google