Introduction to Algorithms Jiafen Liu Sept. 2013

About me Teacher ： Liu Jiafen Mail: Research Interest: –Information security –Formal Method and Verification of Correctness –Data Mining and Business Intelligence –Development of soft under iOS Homepage: – 09/24/ html

Now it’s your turn Please introduce yourself –Name –Age –Graduated School –Specialty and Interest Background Investigation Have you studied data structure and other prerequisite courses of CS? Have you programmed in practice before?

About the Course Site: – go.htmlLecture –Notes can be found here. Related Resources can also be found on you can also watch lectures online No Textbook

About the Course 16 weeks (the last 2 is reserved for exam) Site: Tongbo Building 206 Time: Can we choose another proper time? –Tuesday afternoon （ 5-7 or 6-8 ） ? –Tuesday night （ ） ? –Thursday night （ ） –Or we have to split into 2 period(first 2 in classroom, and last 1 on computers)?

Grading 20%Attendance 50%Performance –1 presentation at least (10%) –assignments (30%) 30% Final paper

Course Objectives What is an algorithm? What’s the difference between algorithm and program? Why we need this course? How to evaluate a algorithm? How can we design a good algorithm? How to make an algorithm implemented?

What is an algorithm? Algorithm is a problem-solving method or process. It helps us to understand scalability. Algorithm satisfy the following properties: –Input –Output –Unambiguous –Finite –Feasible

Algorithm and Program Program is an implementation of algorithm using a programming language. Algorithm design takes the most important place in Software Engineering. Programmer VS Coder? How to become an excellent programmer?

What is important for a program? correctness functionality maintainablity robustness simplicity user- friendliness Performance scalability

Why we need performance? currencyPerformance is the currency of computing. Example: C family VS Java This course makes theoretical study of computer program performance and resource usage. –How to evaluate an algorithm? –How can we design a good algorithm? –How to make an algorithm implemented?

The problem of sorting InputInput: sequence 〈 a 1, a 2, …, a n 〉 of numbers. OutputOutput: permutation 〈 a 1 ', a 2 ', …, a n ' 〉 such that a 1 ' ≤ a 2 ' ≤…≤ a n '. Example: Input : Output: How can we do that? –Recall the process of playing cards.

Insertion sort

Insertion sort

Insertion sort

Insertion sort

Insertion sort

Insertion sort Done!

Insertion sort

Running time The running time depends on the input: an already sorted sequence is easier to sort. Since short sequences are easier to sort than long ones, we will parameterize the running time by the size of the input.

Kinds of analyses Worst-case: (usually) –T(n) =maximum time of algorithm on any input of size n. Average-case: (sometimes) –T(n) =expected time of algorithm over all inputs of size n. –Need assumption of statistical distribution of inputs. Best-case: (bogus) –Cheat with a slow algorithm that works fast on some input.

Machine-independent time Generally, we seek upper bounds on the running time, Why? –Because everybody likes a guarantee. What is insertion sort’s worst-case time? It depends on the speed of our computer: –relative speed (on the same machine) –absolute speed (on different machines)

Big Idea Asymptotic Analysis Ignore machine-dependent constants. growthLook at growth of T(n) as n→∞.

Θ-notation Math: Θ(g(n)) = { f(n) : there exist positive constants c 1, c 2, and n 0 such that 0 ≤ c 1 g(n) ≤ f (n) ≤ c 2 g(n) for all n ≥ n 0 } Engineering: Drop low-order terms; Ignore leading constants. Example: 3n n 2 –5n = Θ(n 3 )

Θ-notation Θ-notation is fabulous because it satisfies our issue of being able to compare both relative and absolute speed.

Asymptotic performance When n gets large enough, a Θ (n 2 ) algorithm always beats a Θ(n 3 ) algorithm. We shouldn’t ignore asymptotically slower algorithms, however. Real-world design situations often call for a careful balancing of engineering objectives. Asymptotic analysis is a useful tool to help to structure our thinking.

Insertion sort analysis Assume that every elemental operation is going to take some constant amount of time. Worst case : Input reverse sorted. [arithmetic series] Average case: All permutations equally likely.

About Θ-notation It is more of a descriptive notation than it is a manipulative notation. Is insertion sort a fast sorting algorithm? –Moderately so, for small n. –Not at all, for large n.

Another method : Merge sort Merging two sorted arrays Time = Θ(n) to merge a total of n elements (linear time)

Merge Sort T(n) Θ(1) 2T(n/2) Θ(n) Θ(1) : Abuse, but we can ignore it. 2T(n/2) : Should be, but it turns out not to matter asymptotically.

Cost for merge sort We shall usually omit stating the base case when T(n) = Θ(1) because it has no effect on the asymptotic solution to the recurrence. Lecture 2 will provide several ways to find a good upper bound on T(n), for example, Recursion Tree.

Recursion Tree We can write as T(n) = 2T(n/2) + cn, where c > 0 and it is constant. T(n)

Recursion Tree We can write as T(n) = 2T(n/2) + cn, where c > 0 and it is constant. cn T(n/2) T(n/2)

Recursion Tree We can write as T(n) = 2T(n/2) + cn, where c > 0 and it is constant. cn cn/2 cn/2 T(n/4) T(n/4) T(n/4) T(n/4)

Recursion Tree We can write as T(n) = 2T(n/2) + cn, where c > 0 and it is constant. cn cn/2 cn/2 T(n/4) T(n/4) T(n/4) T(n/4) …… Θ(1) Height = ？ cn #(leaves) = ? n logn+1 Θ(n) Total = cn*logn+ Θ(n)Θ (n logn)

Conclusion Θ(nlgn) grows more slowly than Θ(n 2 ). Therefore, merge sort asymptotically beats insertion sort in the worst case. In practice, merge sort beats insertion sort for n > 30 or so. Go test it out for yourself!

Homework Read Chapter1 and Chapter 2. Recall another sort method “Bubble sort” and express with pseudocode, then compute its cost using Θ -notation (Hand in paper edition the next class).