Download presentation
Presentation is loading. Please wait.
Published byFranklin Potter Modified over 9 years ago
1
3101 3.0A Lecture 1 - 1 10/24/2015 COSC3101A: Design and Analysis of Algorithms Tianying Ji Lecture 1
2
3101 3.0A Lecture 1 - 2 10/24/2015 Part 1 The course General information Introduction to algorithms Algorithm analysis basics Asymptotic notation
3
3101 3.0A Lecture 1 - 3 10/24/2015 The Course Purpose: The theoretical study of design and analysis of algorithms Not a programming course Not a math course, either Textbook: Introduction to Algorithms, Cormen, Leiserson, Rivest, Stein Second edition An excellent reference you should own
4
3101 3.0A Lecture 1 - 4 10/24/2015 The Course Instructor: Tianying Ji ji@cs.yorku.ca Office: CSEB 2022 Office hours: 5:30pm-7:00pm, Tuesday TA: TBA TA Office hours and location TBA
5
3101 3.0A Lecture 1 - 5 10/24/2015 The Course Grading policy (Probably, TBD): Assignments: 20% Mid-term Exam: 30% Final Exam: 50%
6
3101 3.0A Lecture 1 - 6 10/24/2015 The Course Format One lecture/week About four Assignments oProblem sets oMaybe occasional programming assignments Mid-term + final exam
7
3101 3.0A Lecture 1 - 7 10/24/2015 The Course Course overview Mathematical background (asymptotic notation, recurrence relations, bounding sums, induction, etc.) [Ch 1, 2.2, 3, 4] Proofs of correctness, including loop invariants [Ch 2.1] Recursive algorithms; divide and conquer technique [Ch 2.3, 4] Sorting algorithms and lower bounds [Ch 6,7,8] Selection [Ch 9] Greedy algorithms [Ch 16] Dynamic programming [Ch 15] Graph algorithms (searching, spanning trees, shortest path, etc.) [Ch 22,23,24,25,26] (If time permits) A brief introduction to NP-completeness. [Ch 34]
8
3101 3.0A Lecture 1 - 8 10/24/2015 Part 1 Questions?
9
3101 3.0A Lecture 1 - 9 10/24/2015 Part 2 The course ☻ Introduction to algorithms [Ch 1] What are Algorithms? Analysis vs. Design RAM Algorithm analysis basics Asymptotic notation
10
3101 3.0A Lecture 1 - 10 10/24/2015 What are algorithms? Webster definition: a procedure for solving a mathematical problem (as of finding the greatest common divisor) in a finite number of steps that frequently involves repetition of an operation; broadly : a step-by-step procedure for solving a problem or accomplishing some end especially by a computer
11
3101 3.0A Lecture 1 - 11 10/24/2015 Basics of Algorithms Well-defined computational procedure Takes input values Produces output values Results are always correct for any input event Always terminates
12
3101 3.0A Lecture 1 - 12 10/24/2015 Musical analogy ComputingMusical SoftwareMusic HardwareMusical instruments AlgorithmsChord progressions Source codeMusical notation ProgrammingComposing
13
3101 3.0A Lecture 1 - 13 10/24/2015 Basic goals for an algorithm Always correct Always terminates This class: performance Performance often draws the line between what is possible and what is impossible.
14
3101 3.0A Lecture 1 - 14 10/24/2015 Analysis vs. Design Analysis: predict the cost of an algorithm in terms of resources and performance Design: design algorithms which minimize the cost
15
3101 3.0A Lecture 1 - 15 10/24/2015 Machine model: Generic Random Access Machine (RAM) Executes operations sequentially Set of primitive operations: Arithmetic. Logical, Comparisons, Function calls Simplifying assumption: all ops cost 1 unit Eliminates dependence on the speed of our computer, otherwise impossible to verify and to compare
16
3101 3.0A Lecture 1 - 16 10/24/2015 Part 2 Questions?
17
3101 3.0A Lecture 1 - 17 10/24/2015 Part 3 The course ☻ Introduction to algorithms ☻ Algorithm analysis basics [Ch 2.1, 2.2] Sorting Insertion sort Asymptotic notation
18
3101 3.0A Lecture 1 - 18 10/24/2015 The problem of sorting Input: sequence a 1, a 2, …, a n of numbers. Output: permutation a' 1, a' 2, …, a' n such that a' 1 a' 2 … a' n. Example: Input: 8 2 4 9 3 6 Output: 2 3 4 6 8 9
19
3101 3.0A Lecture 1 - 19 10/24/2015 Insertion sort I NSERTION -S ORT (A, n) ⊳ A[1.. n] for i ← 2 to n dokey ← A[i] j ← i – 1 while j > 0 and A[j] > key doA[j+1] ← A[j] j ← j – 1 A[j+1] ← key “pseudocode” ji key sorted A:A: 1n
20
3101 3.0A Lecture 1 - 20 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 30104020 1234 i = j = key = A[j] = A[j+1] =
21
3101 3.0A Lecture 1 - 21 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 30104020 1234 i = 2j = 1key = 10 A[j] = 30 A[j+1] = 10
22
3101 3.0A Lecture 1 - 22 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 30 4020 1234 i = 2j = 1key = 10 A[j] = 30 A[j+1] = 30
23
3101 3.0A Lecture 1 - 23 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 30 4020 1234 i = 2j = 1key = 10 A[j] = 30 A[j+1] = 30
24
3101 3.0A Lecture 1 - 24 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 30 4020 1234 i = 2j = 0key = 10 A[j] = A[j+1] = 30
25
3101 3.0A Lecture 1 - 25 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 2j = 0key = 10 A[j] = A[j+1] = 10
26
3101 3.0A Lecture 1 - 26 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 3j = 0key = 10 A[j] = A[j+1] = 10
27
3101 3.0A Lecture 1 - 27 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 3j = 0key = 40 A[j] = A[j+1] = 10
28
3101 3.0A Lecture 1 - 28 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 3j = 0key = 40 A[j] = A[j+1] = 10
29
3101 3.0A Lecture 1 - 29 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 3j = 2key = 40 A[j] = 30 A[j+1] = 40
30
3101 3.0A Lecture 1 - 30 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 3j = 2key = 40 A[j] = 30 A[j+1] = 40
31
3101 3.0A Lecture 1 - 31 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 3j = 2key = 40 A[j] = 30 A[j+1] = 40
32
3101 3.0A Lecture 1 - 32 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 4j = 2key = 40 A[j] = 30 A[j+1] = 40
33
3101 3.0A Lecture 1 - 33 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 4j = 2key = 20 A[j] = 30 A[j+1] = 40
34
3101 3.0A Lecture 1 - 34 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 4j = 2key = 20 A[j] = 30 A[j+1] = 40
35
3101 3.0A Lecture 1 - 35 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 4j = 3key = 20 A[j] = 40 A[j+1] = 20
36
3101 3.0A Lecture 1 - 36 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10304020 1234 i = 4j = 3key = 20 A[j] = 40 A[j+1] = 20
37
3101 3.0A Lecture 1 - 37 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 103040 1234 i = 4j = 3key = 20 A[j] = 40 A[j+1] = 40
38
3101 3.0A Lecture 1 - 38 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 103040 1234 i = 4j = 3key = 20 A[j] = 40 A[j+1] = 40
39
3101 3.0A Lecture 1 - 39 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 103040 1234 i = 4j = 3key = 20 A[j] = 40 A[j+1] = 40
40
3101 3.0A Lecture 1 - 40 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 103040 1234 i = 4j = 2key = 20 A[j] = 30 A[j+1] = 40
41
3101 3.0A Lecture 1 - 41 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 103040 1234 i = 4j = 2key = 20 A[j] = 30 A[j+1] = 40
42
3101 3.0A Lecture 1 - 42 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 1030 40 1234 i = 4j = 2key = 20 A[j] = 30 A[j+1] = 30
43
3101 3.0A Lecture 1 - 43 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 1030 40 1234 i = 4j = 2key = 20 A[j] = 30 A[j+1] = 30
44
3101 3.0A Lecture 1 - 44 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 1030 40 1234 i = 4j = 1key = 20 A[j] = 10 A[j+1] = 30
45
3101 3.0A Lecture 1 - 45 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 1030 40 1234 i = 4j = 1key = 20 A[j] = 10 A[j+1] = 30
46
3101 3.0A Lecture 1 - 46 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10203040 1234 i = 4j = 1key = 20 A[j] = 10 A[j+1] = 20
47
3101 3.0A Lecture 1 - 47 10/24/2015 An Example: Insertion Sort InsertionSort(A, n) for i ← 2 to n do key ← A[i] j ← i - 1 while j > 0 and A[j] > key do A[j+1] ← A[j] j ← j - 1 A[j+1] ← key 10203040 1234 i = 4j = 1key = 20 A[j] = 10 A[j+1] = 20 Done!
48
3101 3.0A Lecture 1 - 48 10/24/2015 Example of insertion sort 824936 284936 248936 248936 234896 234689done
49
3101 3.0A Lecture 1 - 49 10/24/2015 Kinds of analyses Worst-case: (usually) T(n) = maximum time of algorithm on any input of size n. Average-case: (sometimes) T(n) = expected time of algorithm over all inputs of size n. Need assumption of statistical distribution of inputs. Best-case: (NEVER) Cheat with a slow algorithm that works fast on some input.
50
3101 3.0A Lecture 1 - 50 10/24/2015 Insertion Sort Statement Cost Times InsertionSort(A, n) for i ← 2 to n c 1 n do key ← A[i] c 2 n-1 j ← i - 1 c 3 n-1 while j > 0 and A[j] > key c 4 T do A[j+1] ← A[j] c 5 T-(n-1) j ← j - 1 c 6 T-(n-1) 0 A[j+1] ← key c 7 n-1 T = t 2 + t 3 + … + t n = where t i is number of while expression evaluations for the i th for loop iteration
51
3101 3.0A Lecture 1 - 51 10/24/2015 Analyzing Insertion Sort T(n) = c 1 n + c 2 (n-1) + c 3 (n-1) + c 4 T + c 5 (T - (n-1)) + c 6 (T - (n-1)) + c 7 (n-1) = c 8 T + c 9 n + c 10 What can T be? Best case -- inner loop body never executed ot i = 1 T(n) is a linear function Worst case -- inner loop body executed for all previous elements ot i = i T(n) is a quadratic function Average case o???
52
3101 3.0A Lecture 1 - 52 10/24/2015 Analysis Simplifications Ignore actual and abstract statement costs Order of growth is the interesting measure: oHighest-order term is what counts ] Remember, we are doing asymptotic analysis ] As the input size grows larger it is the high order term that dominates
53
3101 3.0A Lecture 1 - 53 10/24/2015 Part 3 Questions?
54
3101 3.0A Lecture 1 - 54 10/24/2015 Part 4 The course ☻ Introduction to algorithms ☻ Algorithm analysis basics [Ch 2.1, 2.2] ☻ Asymptotic notation[Ch3]
55
3101 3.0A Lecture 1 - 55 10/24/2015 Asymptotic Complexity Running time of an algorithm for large input sizes. Expressed using only the highest-order term in the expression for the exact running time. Instead of exact running time, say (n 2 ). Describe behavior of functions in the limit. Described using Asymptotic Notation.
56
3101 3.0A Lecture 1 - 56 10/24/2015 Asymptotic Notation , O, , o, Defined in terms of functions whose domain is the set of natural numbers. Ex: f(n) = (n 2 ). Describes how f(n) grows in comparison to n 2. Determine sets of functions, in practice used to compare two function sizes. The notations differ in the rate-of-growth relation they describe between the defining function and the defined set of functions.
57
3101 3.0A Lecture 1 - 57 10/24/2015 -notation (g(n)) = {f(n): +ve constants c 1, c 2, and n 0 such that 0 c 1 g(n) f(n) c 2 g(n), n n 0 } For function g(n), (g(n)) is given by: g(n) is an asymptotically tight bound for f(n). Intuitively: Set of all functions that have the same rate of growth as g(n).
58
3101 3.0A Lecture 1 - 58 10/24/2015 O -notation O(g(n)) = {f(n): +ve constants c and n 0 such that 0 f(n) cg(n), n n 0 } For function g(n), O(g(n)) is given by: g(n) is an asymptotic upper bound for f(n). Intuitively: Set of all functions whose rate of growth is the same as or lower than that of g(n). f(n) = (g(n)) f(n) = O(g(n)). (g(n)) O(g(n)).
59
3101 3.0A Lecture 1 - 59 10/24/2015 -notation (g(n)) = {f(n): +ve constants c and n 0 such that 0 cg(n) f(n), n n 0 } For function g(n), (g(n)) is given by: g(n) is an asymptotic lower bound for f(n). Intuitively: Set of all functions whose rate of growth is the same as or higher than that of g(n). f(n) = (g(n)) f(n) = (g(n)). (g(n)) (g(n)).
60
3101 3.0A Lecture 1 - 60 10/24/2015 Relations Between , O,
61
3101 3.0A Lecture 1 - 61 10/24/2015 Upper Bound Notation We say InsertionSort’s run time is O(n 2 ) Properly we should say run time is in O(n 2 ) Read O as “Big-O” (you’ll also hear it as “order”) In general a function f(n) is O(g(n)) if there exist positive constants c and n 0 such that f(n) c g(n) for all n n 0 Formally O(g(n)) = { f(n): positive constants c and n 0 such that f(n) c g(n) n n 0 }
62
3101 3.0A Lecture 1 - 62 10/24/2015 Insertion Sort Is O(n 2 ) Proof Suppose runtime is an 2 + bn + c oIf any of a, b, and c are less than 0 replace the constant with its absolute value an 2 + bn + c (a + b + c)n 2 + (a + b + c)n + (a + b + c) 3(a + b + c)n 2 for n 1 Let c’ = 3(a + b + c) and let n 0 = 1 Question Is InsertionSort O(n 3 )? Is InsertionSort O(n)?
63
3101 3.0A Lecture 1 - 63 10/24/2015 Big O Fact A polynomial of degree k is O(n k ) Proof: Suppose f(n) = b k n k + b k-1 n k-1 + … + b 1 n + b 0 oLet a i = | b i | f(n) a k n k + a k-1 n k-1 + … + a 1 n + a 0
64
3101 3.0A Lecture 1 - 64 10/24/2015 Lower Bound Notation We say InsertionSort’s run time is (n) In general a function f(n) is (g(n)) if positive constants c and n 0 such that 0 c g(n) f(n) n n 0 Proof: Suppose run time is an + b oAssume a and b are positive (what if b is negative?) an an + b
65
3101 3.0A Lecture 1 - 65 10/24/2015 Asymptotic Tight Bound A function f(n) is (g(n)) if positive constants c 1, c 2, and n 0 such that c 1 g(n) f(n) c 2 g(n) n n 0 Theorem f(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n))
66
3101 3.0A Lecture 1 - 66 10/24/2015 Practical Complexity
67
3101 3.0A Lecture 1 - 67 10/24/2015 Practical Complexity
68
3101 3.0A Lecture 1 - 68 10/24/2015 Practical Complexity
69
3101 3.0A Lecture 1 - 69 10/24/2015 Practical Complexity
70
3101 3.0A Lecture 1 - 70 10/24/2015 Other Asymptotic Notations A function f(n) is o(g(n)) if positive constants c and n 0 such that f(n) < c g(n) n n 0 A function f(n) is (g(n)) if positive constants c and n 0 such that c g(n) < f(n) n n 0 Intuitively, o() is like < O() is like () is like > () is like () is like =
71
3101 3.0A Lecture 1 - 71 10/24/2015 Part 4 Questions?
72
3101 3.0A Lecture 1 - 72 10/24/2015 Review: Induction Suppose S(k) is true for fixed constant k oOften k = 0 S(n) S(n+1) for all n >= k Then S(n) is true for all n >= k
73
3101 3.0A Lecture 1 - 73 10/24/2015 Proof By Induction Claim:S(n) is true for all n >= k Basis: Show formula is true when n = k Inductive hypothesis: Assume formula is true for an arbitrary n Step: Show that formula is then true for n+1
74
3101 3.0A Lecture 1 - 74 10/24/2015 Induction Example: Gaussian Closed Form Prove 1 + 2 + 3 + … + n = n(n+1) / 2 Basis: oIf n = 0, then 0 = 0(0+1) / 2 Inductive hypothesis: oAssume 1 + 2 + 3 + … + n = n(n+1) / 2 Step (show true for n+1): 1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1) = n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2 = (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2
75
3101 3.0A Lecture 1 - 75 10/24/2015 The End Read your text Ch1, Ch2.1, Ch2.2, Ch3!
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.