1. 5/15/2015COT 54071 COT 5407: Introduction to Algorithms Tao Li ECS 318; Phone: x6036 taoli@cs.fiu.edu http://www.cs.fiu.edu/~taoli/class/COT5407-F09/index.html

2. 8/28/07COT 54072 Why should I care about Algorithms? Cartoon from Intractability by Garey and Johnson

3. 8/28/07COT 54073 More questions you should ask Who should know about Algorithms? Is there a future in this field? Would I ever need it if I want to be a software engineer or work with databases?

4. 8/28/07COT 54074 Why are theoretical results useful? Cartoon from Intractability by Garey and Johnson

5. 8/28/07COT 54075 Why are theoretical results useful? Cartoon from Intractability by Garey and Johnson

6. 5/15/2015COT 54076 Self-Introduction Ph.D. in Computer Science from University of Rochester, 2004 –Research Interests: data mining, machine learning, information retrieval, bioinformatics and more ? Associate Professor in the School of Computer Science at Florida International University Industry Experience: –Summer internships at Xerox Research (summer 2001, 2002) and IBM Research (Summer 2003, 2004)

7. 5/15/2015COT 54077 Student Self-Introduction Name –I will try to remember your names. But if you have a Long name, please let me know how should I call you Major and Academic status Programming Skills –Java, C/C++, VB, Matlab, Scripts etc. Research Interest Anything you want us to know

8. 5/15/2015COT 54078 What this course is about Introduction to Algorithms Analysis of Algorithms How does one design programs and ascertain their efficiency? Divide-and-conquer techniques, string processing, graph algorithms, mathematical algorithms. Advanced data structures such as balanced tree schemes.

9. 5/15/2015COT 54079 Course Logistics Meeting Time and Location: Tuesday and Thursday 17:00pm-18:15pm, ECS134 Office Hours: Tuesday and Thursday 14:30pm-15:30pm TA: Yali Wu Textbook: Introduction to Algorithms, (Third Edition) Thomas Cormen, Charles Leiserson, Ronald Rivest, and Clifford Stein. MIT Press.

10. 5/15/2015COT 540710 Evaluation Class participation and Quizzes: 10% Midterm Exam: 30% Final Exam: 30% Assignments:30% You may work with one other person on homeworks, but you must each write up your solutions separately. If you work with another person, indicate who you worked with on your solution. Please start a new page for each problem on your solutions, and include your name on each page, so the TA can choose the problems for grading. Exams are open/closed book ??

11. Algorithm A computational problem is a mathematical problem, specified by an input/output relation. An algorithm is a computational procedure for solving a computational problem. 5/15/2015COT 540711 “computer” problem algorithm inputoutput

12. 1-12 Some Well-known Computational Problems Sorting Searching Shortest paths in a graph Minimum spanning tree Primality testing Traveling salesman problem Knapsack problem Chess Towers of Hanoi Program termination

13. 8/28/07COT 540713 History of Algorithms The great thinkers of our field: Euclid, 300 BC Bhaskara, 6 th century Al Khwarizmi, 9th century Fibonacci, 13 th century Babbage, 19 th century Turing, 20 th century von Neumann, Knuth, Karp, Tarjan, …

14. 8/28/07COT 540714 Euclid’s Algorithm GCD(12,8) = 4; GCD(49,35) = 7; GCD(210,588) = ?? GCD(a,b) = ?? Observation: [a and b are integers and a  b] –GCD(a,b) = GCD(a-b,b) Euclid’s Rule: [a and b are integers and a  b] –GCD(a,b) = GCD(a mod b, b) Euclid’s GCD Algorithm: –GCD(a,b) If (b = 0) then return a; return GCD(a mod b, b)

15. 1-15 Basic Issues Related to Algorithms How to design algorithms How to express algorithms Proving correctness Efficiency –Theoretical analysis –Empirical analysis Optimality

16. 1-16 Algorithm design strategies Brute force Divide and conquer Decrease and conquer Transform and conquer Greedy approach Dynamic programming Backtracking Branch and bound Space and time tradeoffs

17. 1-17 Analysis of Algorithms How good is the algorithm? –Correctness –Time efficiency: amount of work done –Space efficiency: amount of space used –Simplicity, clarity Does there exist a better algorithm? –Lower bounds –Optimality

18. 5/15/2015COT 540718 Correctness Proving correctness is dreadful for large algorithms. A strategy that can be used is: divide the algorithm into smaller pieces, and then clarify what the preconditions and postconditions are and prove correct assuming everything else is correct.

19. 5/15/2015COT 540719 Amount of Work Done Rather than counting the total number of instructions executed, we'll focus on a set of key instructions and count how many times they are executed. Use asymptotic notation and pay attention only to the largest growing factor in the formula of the running time. Two major types of analysis: worst-case analysis and average- case analysis

20. 5/15/2015COT 540720 More Amount of space used: The amount of space used can be measured similarly. Consideration of this efficiency is often important. Simplicity, clarity: Sometimes, complicated and long algorithms can be simplified and shortened by the use of recursive calls. Optimality: For some algorithms, you can argue that they are the best in terms of either amount of time used or amount of space used. There are also problems for which you cannot hope to have efficient algorithms.

21. 5/15/2015COT 540721 Asymptotic Growth Rates of Functions Big O Big Omega Little O Little Omega Theta Notation

22. 5/15/2015COT 540722 Notations

23. 5/15/2015COT 540723 Other mathematical background The ceiling function The floor function The exponentials and logarithms Fibonacci number Summations and Series

24. Why study algorithms? Theoretical importance –the core of computer science Practical importance –A practitioner’s toolkit of known algorithms –Framework for designing and analyzing algorithms for new problems

25. Two main issues related to algorithms How to design algorithms How to analyze algorithm efficiency

26. 1-26 Algorithm design strategies Brute force Divide and conquer Decrease and conquer Transform and conquer Greedy approach Dynamic programming Backtracking Branch and bound Space and time tradeoffs

27. Analysis of algorithms How good is the algorithm? –time efficiency –space efficiency Does there exist a better algorithm? –lower bounds –optimality

28. 1-28 Important problem types sorting searching string processing graph problems combinatorial problems geometric problems numerical problems

29. 1-29 Fundamental data structures list –array –linked list –string stack queue, priority queue Graph Tree set and dictionary

30. 5/15/2015COT 540730 Search You are asked to guess a number X that is known to be an integer lying between integers A and B. How many guesses do you need in the worst case? –Number of guesses = log 2 (B-A) You are asked to guess a positive integer X. How many guesses do you need in the worst case? –NOTE: No upper bound B is known for the number.

31. 5/15/2015COT 540731 Search You are asked to guess a number X that is known to be an integer lying between integers A and B. How many guesses do you need in the worst case? –Number of guesses = log 2 (B-A) You are asked to guess a positive integer X. How many guesses do you need in the worst case? –NOTE: No upper bound B is known for the number. –Algorithm: figure out B (by using Doubling Search) perform binary search in the range B/2 through B. –Number of guesses = log 2 B + log 2 (B – B/2) –Since X is between B/2 and B, we have: log 2 (B/2) < log 2 X, –Number of guesses < 2log 2 X - 1

32. 5/15/2015COT 540732 Polynomials Given a polynomial –p(x) = a 0 + a 1 x + a 2 x 2 + … + a n-1 x n-1 + a n x n compute the value of the polynomial for a given value of x. How many additions and multiplications are needed? –Simple solution: Number of additions = n Number of multiplications = 1 + 2 + … + n = n(n+1)/2 –Improved solution using Horner’s rule:

33. 5/15/2015COT 540733 Polynomials Given a polynomial –p(x) = a 0 + a 1 x + a 2 x 2 + … + a n-1 x n-1 + a n x n compute the value of the polynomial for a given value of x. How many additions and multiplications are needed? –Simple solution: Number of additions = n Number of multiplications = 1 + 2 + … + n = n(n+1)/2 –Improved solution using Horner’s rule: p(x) =p(x) = a 0 + x(a 1 + x(a 2 + … x(a n-1 + x a n ))…)) Number of additions = n Number of multiplications = n