1. Lecture 3 Induction & Sort(1) Algorithm design techniques: induction Selection sort, Insertion sort, Shell sort...

2. ROADMAP Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons 5/27/2016Xiaojuan Cai2

3. Techniques based on Recursions Induction or Tail recursion Iteration, Proof: invariants by induction Non-overlapping subproblems Divide-and-conquer Overlapping subproblems Dynamic programming 5/27/2016Xiaojuan Cai3

4. Techniques based on Recursions Induction or Tail recursion ‣ Iteration, ‣ Proof: invariants by induction Non-overlapping subproblems ‣ Divide-and-conquer Overlapping subproblems ‣ Dynamic programming 5/27/2016Xiaojuan Cai4

5. Induction example 9, 8, 9, 6, 2, 56 Selection sort Problem size: 6 2, 8, 9, 6, 9, 56 Problem size: 5 Algorithm Design: problem size n --> n-1 Correctness Proof: by mathematical induction. 5/27/2016Xiaojuan Cai5

6. Permutation 1 2 3 1 3 2 2 1 3 2 3 1 3 1 2 3 2 1 5/27/2016Xiaojuan Cai6 Problem: Permuation Input: an array of n elements Output: the permutations of n elements

7. Permutation1 5/27/2016Xiaojuan Cai7

8. Permutation2 5/27/2016Xiaojuan Cai8

9. Permutation 1 2 3 1 3 2 2 1 3 2 3 1 3 1 2 3 2 1 a1a1 a2a2 a3a3 a4a4 Permutation 1: Permutation 2: a1a1 a2a2 a3a3 a4a4 5/27/2016Xiaojuan Cai9

10. Permutation 1 5/27/2016Xiaojuan Cai10 Ω(n ⋅ n!)

11. Permutation 2 5/27/2016Xiaojuan Cai11 Ω(n ⋅ n!)

12. Quiz Which output is in alphabetical order? A. permutation1 B. permutation2 C. both D. none of above

13. Polynomial Problem: PolynomialEval Input: a[0...n], and x Output: the value of p(x) William George Horner (source from Wikipedia) Horner’s rule 5/27/2016Xiaojuan Cai13

14. Horner’s rule 5/27/2016Xiaojuan Cai14

15. Finding majority element 1 2 3 3 4 2 2 2 3 2 Problem: MajorityElement Input: a sequence of integers A[1...n] Output: An integer a in A that appears more than n/2 times, otherwise, output none. 1 5 1 1 4 2 1 3 1 1 none 1 Observation If two different elements in the original sequence are removed, then the majority in the original sequence remains the majority in the new sequence. 5/27/2016Xiaojuan Cai15

16. Finding majority element Θ(n) 5/27/2016Xiaojuan Cai16

17. Proof of correctness Lemma (Invariants) Given a[1..N], Candidates(i) returns the majority element of a[i...N] if there exists a majority element. Proof. (Inductive proof) Base step. Lemma holds for i = N Inductive step. Induction Hypothesis. Assume for j > i, the lemma holds. Prove it holds for i. 5/27/2016Xiaojuan Cai17

18. Radix sort All numbers in the array consists of EXACTLY k digits. 7467 1247 3275 6792 9187 9134 4675 1239 6792 9134 3275 4675 7467 1247 9187 1239 9134 1239 1247 7467 3275 4675 9187 6792 9134 9187 1239 1247 3275 7467 4675 6792 1239 1247 3275 4675 6792 7467 9134 9187

19. Proof of correctness Lemma (Invariants) In Radix sort, if the i-th digits are sorted, then the numbers consisting of i, i-1,..,1 digits are sorted.. Proof. (Inductive proof) Base step. Lemma holds for i = 1 Inductive step. Induction Hypothesis. Assume for j < i, the lemma holds. Prove it holds for i. 5/27/2016Xiaojuan Cai19

20. Where are we? Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons 5/27/2016Xiaojuan Cai20

21. ? 5/27/2016Xiaojuan Cai21

22. ? 5/27/2016Xiaojuan Cai22

23. ? 5/27/2016Xiaojuan Cai23

24. Selection sort 2 1362443 1 519 i jmin Best caseWorst case #Comparison: #Interchange: 5/27/2016Xiaojuan Cai24

25. Selection sort in final order ↑ Lemma (Invariants) Entries the left of ↑ (including ↑) fixed and in ascending order. No entry to right of ↑ is smaller than any entry to the left of ↑. 5/27/2016Xiaojuan Cai25

26. Insertion sort 2 13624431519 Best caseWorst case #Comparison: #Move: 5/27/2016Xiaojuan Cai26

27. Insertion sort in order ↑ not yet seen Lemma (Invariants) Entries to the left of ↑ (including ↑) are in ascending order. Entries to the right of ↑ have not yet been seen. 5/27/2016Xiaojuan Cai27

28. Shell sort (3-1) 2 13624431519 10 Correctness: The last round of shell sort is insertion sort. 5/27/2016Xiaojuan Cai28

29. Comparison http://www.sorting-algorithms.comhttp://www.sorting-algorithms.com/ worstaveragebestremarks selection N 2 / 2 N exchanges insertion N 2 / 2N 2 / 4Nfor small N or partially ordered shell ??Ntight code, subquadratic 5/27/2016Xiaojuan Cai29

30. Stability and in-place Definition: Stability A stable sort preserves the relative order of items with equal keys. Definition: In-place A sorting algorithm is in-place if the extra memory it uses ≤ c log N. in-place?stable? selection ？？ insertion ？？ shell ？？ 5/27/2016Xiaojuan Cai30

31. Comparison in- place? stable?worst averag e bestremarks selection xN 2 / 2 N exchanges insertion xxN 2 / 2N 2 / 4N for small N or partially ordered shell x??Ntight code, subquadratic 5/27/2016Xiaojuan Cai31

32. Conclusion 5/27/2016Xiaojuan Cai32 Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons

33. Next permutation 5/27/2016Xiaojuan Cai33 1 3 2 Next_perm 2 1 3 1 4 3 2 5 Next_perm 1 4 3 5 2

34. Exercises Download hw3.pdf from our course homepage Due on next Tuesday 5/27/2016Xiaojuan Cai34