CS 461 – Nov. 21 Sections 7.1 – 7.2 Measuring complexity Dividing decidable languages into complexity classes. Algorithm complexity depends on what kind of TM you use. Formal definition of “P” Next: Concepts of NP and NP-complete

Examples Some graph problems –Finding the shortest path –Finding the cheapest network (spanning tree) –Hamiltonian and Euler cycles –Traveling salesman problem Why do similar sounding problems have vastly different complexity?

“O” review Order of magnitude upper bound Rationale: 1000 n 2 is fundamentally less than n 3, and is in the same family as n 2. Definition: f(n) is O(g(n)) if there exist integer constants c and n 0 such that f(n)  c g(n) for all n  n 0. –In other words: in the long run, f(n) can be bounded by g(n) times a constant. –e.g. 7n is O(n 2 ) and also O(n 3 ) but not O(n). O(n 2 ) would be a tight or more useful upper bound. –e.g. Technically, 2 n is O(3 n ) but 3 n is not O(2 n ). –log(n) is between 1 and n. Ordering of common complexities: O(1), O(log n), O(n), O(n log n), O(n 2 ), O(n 3 ), O(2 n ), O(n!)

Measuring complexity Complexity can be defined: –as a function of n, the input length –It’s the number of Turing machine moves needed. –We’re interested in order analysis, not exact count. E.g. About how many moves would we need to recognize { 0 n 1 n } ? –Repeatedly cross of outermost 0 and 1. –Traverse n 0’s, n 1’s twice, (n-1) 0’s twice, (n-1) 1’s twice, etc. –The total number of moves is approximately: 3n + 4((n-1)+(n-2)+(n-3)+…+1) = 3n + 2n(n-1) = 2n 2 + n ~ 2n 2 –2n 2 steps for input size 2n  O(n 2 ).

Complexity class We can classify the decidable languages. TIME(t(n)) = set of all languages that can be decided by a TM with running time O(t(n)). –{ 0 n 1 n }  TIME(n 2 ). –1*00*1(0+1)*  TIME(n). –{ 1 n 2 n 3 n }  TIME(n 2 ). –CYK algorithm  TIME(n 3 ). –{ ε, 0, 1101 }  TIME(1). Technically, you can also belong to a “worse” time complexity class. L  TIME(n)  L  TIME(n 2 ). It turns out that { 0 n 1 n }  TIME(n log n). (p. 252)

TM variants It turns out that the complexity depends on the variant! –Ordinary one-tape TM. (for definition of TIME(t(n)) –Multi-tape TM.(useful for programming) –Non-deterministic TM. E.g. { 0 n 1 n } can be decided in O(n) on a multi-tape TM. –See page 253. In general, a multi-tape TM that runs in O(t(n)) can be converted to a single-tape TM that runs in O((t(n) 2 ). –See page 254. In general, a non-deterministic TM that runs in O(t(n)) can be converted to an ordinary TM running in 2 O(t(n)). –See page 256. –This is why we say that NP problems are exponential.

Multi-tape { 0 n 1 n } We have seen an O(n 2 ) algorithm on a single-tape TM. And a better algorithm that runs in O(n log n), also with single tape. With multiple tapes, we can do it in O(n). –Scan input to make sure of the form 0*1*. –Return to beginning. –Read 0’s and copy them to 2 nd tape. –When you reach the first 1, start the 2 nd tape’s head at the first 0. –Now we match 0’s and 1’s simultaneously. So, the problem is O(n), although technically it’s still in TIME(n log n). Not a big deal in the long run.

Single vs. multi tape O(t(n)) multi-tape  O(t(n) 2 ) single tape. Proof: We simulate the multi-tape operation on single tape TM. –Each of the k tapes is performing a computation that runs in t(n) time. In other words, we take up to t(n) steps. Scan the tape, taking note of where virtual heads are. How long is the tape contents? –Each virtual section of the tape could be t(n) symbols long, because each step could write a symbol on the tape. Final calculation: –Scanning the entire tape takes O(t(n)) time. –And we have to scan the tape t(n) times! –Multiply together to obtain O(t(n) 2 ).

DTM vs. NTM O(t(n)) NTM  2 O(t(n)) DTM. Proof: The NTM has a tree of possible computational futures. For input length n, the depth of tree is up to t(n). Each node of the tree can have up to c children. –Total number of nodes in the tree is on the order of O(c t(n) ). The time it takes to reach the appropriate node in the tree is t(n) –Multiply this by the number of nodes to obtain the upper bound on the complexity: O(t(n) c t(n) ). –Claim: this is equivalent to 2 O(t(n)). Actually it’s greater but ok. Finally, need to convert multi-tape to single tape. But squaring an exponential is still just an exponential.

Definition of P The union of all language classes of the form: TIME(1)  TIME(n)  TIME(n 2 )  TIME(n 3 ) … Thus, it’s all algorithms that can be decided by an ordinary (one-tape, deterministic) TM in polynomial time. –Even if we allowed multiple tapes, the time complexity would still by polynomial, by our earlier theorem. –We also generalize “TM” to computer program. When we speak of a O(f(n)) algorithm, this is generally assumed. But it’s still deterministic! So, it’s ok to say { 0 n 1 n } is O(n) problem. Many algorithms are in P. This is where we like to be. –No recursion –Nested loops ok.