1. Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure

2. Outline  Review of recursive algorithm running time—cheat sheet  Homogeneous Second-Order Linear Recurrence  Brute Force Strategy for Algorithm Design  The art of lazy algorithm is to count/estimate its running time  Is it doable within given timeframe?

3. Time Efficiency of Recursive Algorithms Steps in mathematical analysis of recursive algorithms:  Decide on parameter n indicating input size  Identify algorithm’s basic operation  Determine worst, average, and best case for input of size n  Set up a recurrence relation and initial condition(s) for C(n)-the number of times the basic operation will be executed for an input of size n (alternatively count recursive calls).  Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution (see Appendix B)

4. Three Recurrence Types We know How to Find the Closed-Form Solution Please related them to the following algorithms we learned in the last class Recursive algorithm for n! Recursive algorithm for Tower of Hanoi Recursive algorithm for finding the number of digits in the binary representation of a decimal integer Recursive algorithm for finding the Fibbonacci numbers

5. Important Recurrence Types: One (constant) operation reduces problem size by one. T(n) = T(n-1) + c T(1) = d Solution: T(n) = (n-1)c + d linear A pass through input reduces problem size by one. T(n) = T(n-1) + cn T(1) = d Solution: T(n) = [n(n+1)/2 – 1] c + d quadratic One (constant) operation reduces problem size by half. T(n) = T(n/2) + c T(1) = d Solution: T(n) = c lg n + d logarithmic A pass through input reduces problem size by half. T(n) = 2T(n/2) + cn T(1) = d Solution: T(n) = cn lg n + d n n log n

6. A General Divide-and-Conquer Recurrence: Master Theorem T(n) = aT(n/b) + f (n) where f (n) ∈ Θ(n k ) a < b k T(n) ∈ Θ(n k ) a = b k T(n) ∈ Θ(n k lg n ) a > b k T(n) ∈ Θ( n log b a ) Note: the same results hold with O instead of Θ.

7. Solutions to a Homogeneous Second-Order Linear Recurrence with Constant Coefficients Characteristic equation  roots Then

8. Fibonacci numbers The Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … Fibonacci recurrence: F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1 Another example: A(n) = 3A(n-1) – 2A(n-2) A(0) = 1 A(1) = 3 2nd order linear homogeneous recurrence relation with constant coefficients

9. How to Find the Constants and Using the initial conditions For example, for Fibonacci numbers F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1 Characteristic equation  roots We have Using F(0)=0 and F(1)=1 

10. Computing Fibonacci numbers Definition based recursive algorithm Nonrecursive brute-force algorithm Explicit formula algorithm Logarithmic algorithm based on formula: F(n-1) F(n) F(n) F(n+1) 0 1 1 1 = n for n≥1, assuming an efficient way of computing matrix powers.

11. Another Example: calculating a^n Construct an algorithm for computing a n, where a>0 is a constant and the integer n>=0 is the input size. Then analyze its time efficiency Basic operation  a multiplication of two float numbers Two choices Nonrecursive algorithm Recursive algorithm Any idea to make it faster, e.g., with a better time efficiency? Can we solve this problem in time?

12. Empirical Analysis of Algorithm Time Efficiency  Why?  Mathematical analysis is difficult for many algorithms  Difficult for average case analysis  Difficult for special type of data distribution  How?  Purpose: compare efficiency  Efficiency measures M: operation counts or time unit  Determine characteristics of typical sample input  Implement the algorithm and run it  Different sample sizes, multiple samples for each size  Profiling tools: measuring time spent on different segments of program can pinpoint bottleneck in a program..

13. Empirical Analysis of Algorithm Time Efficiency Data analysis for the empirical data Scatter Plot

14. Now We Get to Chapter 3 -- Brute Force From now on, we are going to learn some basic and general strategies in designing algorithms to solve some typical computing problems. We will analyze the efficiency of these algorithms using the tools learned in the past several classes We will learn how to design algorithms with better efficiency First, let’s talk about the Brute Force strategies– the simplest

15. Brute Force Strategy for Algorithm Design Brute Force is a straightforward approach to solving a problem, usually directly based on the problem’s statement and definitions of the concepts involved In many cases, Brute Force does not provide you a very efficient solution Brute Force may be enough for moderate size problems with current computers….

16. Sorting Algorithm We have discussed two sorting algorithms: Selection Sort and Insertion Sort What are the basic idea behind the Selection-Sort algorithm? Scanning the entire given list to find its smallest element and swap it with the first element This is a straightforward solution – Brute Force strategy What is its time efficiency – An example: sorting the numbers [89 45 68 90 29 34 17] See pseudocode in Section 3.1

17. Selection sort

18. Insertion Sort

19. Another Brute-Force Application: Bubble Sort Compare adjacent elements and exchange them if they are out of order This the result after the first pass, which moves the largest as the rightmost element

20. Algorithm in Pseudocode What is the time efficiency?

21. Sequential Search – Brute Force Find whether a search key is present in an array What is time efficiency of this algorithm?

22. Brute-Force String Matching Find a pattern in the text: Pattern – ‘NOT’, text – ‘NOBODY_NOTICED_HIM’ Typical Applications – ‘find’ function in the text editor, e.g., MS-Word, Google search What is the time efficiency of this algorithm?

23. Closest-Pair and Convex Hull Problems by Brute Force Closest-Pair problem Given n points in a plane, find the closest pair How to solve this problem and what is the time efficiency of this algorithm? Convex-Hull problem Convex hull is the tightest convex polygon that bounds a set of n points in a plane Convex polygon – any two points in this polygon results in the inclusion of the segment that links these two points also in this polygon

24. Convex/NonConvex Polygons

25. Convex Hull Imagine a rubber band around a set of nails Nails touched by the band  extreme points

26. Solve Convex-Hull Problem  Connect any pair of points by a line segment.  Each line segment partitions the plane to the two half planes  If all the n points are on the same side of this line segment Such a line segment is an edge of the convex-hull polygon  What is the time efficiency of this Brute-Force algorithm?  For each possible pair of points, we need to check whether all the remaining n-2 points are on the same side of the line segment that connects these pair of points.  For Sorting, String Matching, and Convex-Hull problems, we will revisit them by designing more efficient algorithms.

27. Exhaustive Search A brute-force approach to combinatorial problem Generate each and every element of the problem’s domain Then compare and select the desirable element that satisfies the set constraints Involve combinatorial objects such as permutations, combinations, and subsets of a given set The time efficiency is usually bad – usually the complexity grows exponentially with the input size Three examples Traveling salesman problem Knapsack problem Assignment problem

28. Traveling Salesman Problem Find the shortest tour through a given n cities that visits each city exactly once before returning to the starting city Using graph model: city  vertex, road  edge, length of the road  edge weight. TSP  shortest Hamiltonian Circuit – a cycle that passes through all the vertices of the graph exactly once Exhaustive search: List all the possible Hamiltonian circuits (starting from any vertex) Ignore the direction How many candidate circuits do we have?  (n-1)!/2 Very high complexity

29. TSP Example