Complexity A decidable problem is computationally solvable. But what resources are needed to solve the problem? –How much time will it require? –How much memory will it require? In this chapter, we study time complexity. Chapter 8 covers the space complexity of a problem. –Space corresponds to memory. –We do not cover space complexity; this topic is rarely covered in introductory theory courses.

Goal: transfer all n disks from peg A to peg C Rules: –move one disk at a time –never place larger disk above smaller one Recursive solution: –transfer n  1 disks from A to B –move largest disk from A to C –transfer n  1 disks from B to C Total number of moves: –T(n)  2T(n  1)  1 Towers of Hanoi

Towers of Hanoi (2) Recurrence relation: T(n)  2 T(n  1)  1 T(1)  1 Solution by unfolding: T(n) =  2 (2 T(n - 2) + 1) + 1 = = 4 T(n - 2) = = 4 (2 T(n - 3) + 1) = = 8 T(n - 3) =... = 2 i T(n - i) + 2 i-1 +2 i the expansion stops when i  n  1 T(n) = 2 n – n – n –

Towers of Hanoi (3) This is a geometric sum, so that we have T(n)  2 n  1 Good or bad news? –the Tibetans were confronted with a tower problem of 64 rings... –Assuming the priests move one ring per second, it would take ~585 billion years to complete the process!

Reasonable vs. Unreasonable Processing 1 elements takes 0.01 sec function/ n n2n2 1/10,000 second 1/2,500 second 1/400 second 1/100 second 9/100 second n5n5 1/10 second 3.2 seconds 5.2 minutes 2.8 hours 28.1 days 2n2n 1/1000 second 1 second 35.7 years 400 trillion centuries a 75 digit- number of centuries n 2.8 hours 3.3 trillion years a 70 digit- number of centuries a 185 digit- number of centuries a 728 digit- number of centuries Exponential Polynomial

Goals Basics of time complexity theory Introduce method for measuring the time needed to solve a problem. Show how to classify problems according to the amount of time required. Show that certain classes of problems require enormous amounts of time. We will also see how to determine whether we have this type of problem.

Language vs. Problem We will use “Language” and “Problem” interchangeably Any Problem can be converted to a Language, and vice-versa –Problem: Determine if a number is prime –Language: L = {p : p is prime } –Problem: Determine if a Turing Machine M halts on an input string w –Language: L = {e(M), e(w) : M halts on w}

Calculating Running Time of a Program To determine the total time  add up the time required for each statement. The running time of a simple statement is O(1). The running time of an if statement is the larger of the running times of the two branches, plus the time required to evaluate the condition The running time of a sequence of statements is the running time of the longest step. The running time of a loop is the time to execute one iteration times the number of iterations.

Why Algorithm Analysis Solve a problem on a computer  seek an algorithm that makes efficient use of the computer resources –time –space Algorithm analysis - mathematical approach to describe (in terms of the problem size): –The amount of resources used by the algorithm Space Computational (running) time –Running time: The number of primitive operations (steps) executed before termination –Order of growth The leading term of a formula Expresses the behavior of a function toward infinity

Slides

Asymptotic upper bound Suppose f and g are two nonnegative functions Definition [Big “oh”]: –f(n)=O(g(n)) iff. there exist positive constants c and n 0 such that f(n)≤ c g(n) for all n, n≥ n 0. Examples –3n+2=O(n) as 3n+2 ≤ 4n for n ≥ 2 –10n 2 +4n+2 = O(n 2 ) as 10n 2 +4n+2 ≤ 11n 2 for n ≥ 5 g(n) should be as small as possible –f(n) = O(n 2 ) –f(n) = O(n)

Logarithms The notation "ln(x)" means loge(x); The notation "lg(x)" means log10(x); The notation "lb(x)" means log2(x).

Logarithms

Towers of Hanoi (3) This is a geometric sum, so that we have T(n)  2 n  1  O(2 n ) The running time of this algorithm is exponential (k n ) rather than polynomial (n k ), where n is input size. Good or bad news? –the Tibetans were confronted with a tower problem of 64 rings... –Assuming the priests move one ring per second, it would take ~585 billion years to complete the process!