1. Discrete Mathematics Chapter 2 The Fundamentals : Algorithms, the Integers, and Matrices. 大葉大學 資訊工程系 黃鈴玲

2. 2 2.1 Algorithms Def 1. An algorithm is a finite sequence of precise instructions for performing a computation or for solving a problem. Example 1. Describe an algorithm for finding the maximum value in a finite sequence of integers.( 假設給定的 sequence 是 a 1, a 2,…, a n )

3. 3 Solution : ( English language) 1.Set the temporary maximum equal to the first integer in the sequence. 2.Compare the next integer in the sequence to the temporary maximum, and if it is larger than the temporary maximum, set the temporary maximum equal to this integer. 3.Repeat the previous step if there are more integers in the sequence. 4.Stop when there are no integers left in the sequence. The temporary maximum at this point is the largest integer in the sequence.

4. 4 Solution : (pseudo-code) Algorithm 1. Finding the Maximum Element procedure max(a 1, a 2, …, a n : integers) max := a 1 for i := 2 to n if max < a i then max := a i { max is the largest element}

5. 5 ※ There are several properties that algorithms generally share : Input Output Definiteness : The steps of an algorithm must be defined precisely. Correctness : produce correct output values Finiteness : produce the desired output after a finite number of step. Effectiveness Generality : The procedure should be applicable for all problems of the desired form, not just for a particular set of input values.

6. 6 Problem : Locate an element x in a list of distinct elements a 1, a 2,…, a n, or determine that it is not in the list. 做法 : linear search, binary search. Algorithm 2. The linear search algorithm procedure linear_search( x : integer, a 1,a 2,…,a n : distinct integers) i := 1 While ( i ≤ n and x≠a i ) i := i + 1 if i ≤ n then location := i else location := 0 { location = j if x = a j ; location = 0 if x≠a i, ∀ i } ※ Searching Algorithms

7. 7 兩種 search 方式的概念 : Linear Search : 從 a 1 開始，逐一比對 x 是否等於 a i ，若找到 則 location = i, 若到 a n 比完後還找不到，則 location = 0 。 Binary Search : ( 必須具備 a 1 a m 表示 x 應在右半，否則在左半。 (2) 重覆上一步驟至 list 只剩一個元素 a i ， 若 x = a i 則 location = i ，否則 location = 0 。

8. 8 Example 3. Search 19 from a 1 a 2 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 11 a 12 a 13 a 14 a 15 a 16 1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22 12 13 15 16 18 19 20 22 18 19 20 22 18 19 19 1. ( 切兩半 ) ( 因 19 > 10 ，取右半 ) 2. ( 再切二半 ) ( 因 19 > 16 ，取右半 ) 3. ( 再切二半 ) ( 因 19 ≦ 19 ，取左半 ) 4. ( 再切二半 ) ( 因 19 > 18 ，取右半 ) 5 此時只剩一個元素 a 14 = 19 因 19 = 19 ，故 location =14 Note : a i, a i +1, …, a j 數列的切法 : 令 m = 則 a m 即切開紅線左邊那點。

9. 9 Algorithm 3. The Binary Search Algorithm procedure binary_search( x : integer, a 1,a 2,…,a n : increasing integers) i :=1 { i is left endpoint of search interval } j := n { j is right endpoint of search interval } while i < j begin m := if x > a m then i := m+1 else j := m end if x = a i then location := i else location := 0 { location = i if x = a i, location = 0 if x≠a i, ∀ i }

10. 10 Problem : Suppose that we have a list of elements, a sorting is putting these elements into a list in which the elements are in increasing order. eg. 7, 2, 1, 4, 5, 9 => 1, 2, 4, 5, 7, 9 d, t, c, a, f => a, c, d, f, t 解法有很多，此處僅介紹 : bubble sort ( 氣泡排序 法 ) ，及 insertion sort ( 插入排序法 ) 。 Bubble Sort 概念 : 設原 list 為 a 1,…,a n 。  從 a 1,a 2 開始，向後兩兩比較，若 a i > a i+1 則交換，當檢查 完 a n 時， a n 必定是最大數。  再從 a 1,a 2 開始向後比較，若 a i > a i+1 則交換，此時只需檢 查到 a n- 1 即可。  依此類推。 ※ Sorting Algorithms

11. 11 Example 4. Use the bubble sort to put 3, 2, 4, 1, 5 into increasing order. Sol : 3241532415 First pass ( i=1 ) : 2341523415 2341523415 2314523145 Second pass ( i=2 ) : Third pass ( i=3 ) :Fourth pass ( i=4 ) : 1234512345 2314523145 2314523145 2134521345 2134521345 2314523145 2134521345 1234512345 1234512345 1234512345

12. 12 Algorithm 4 The Bubble Sort procedure bubble_sort (a 1,…,a n ) for i := 1 to n  1 for j := 1 to n  i if a j > a j+1 then interchange a j and a j+1 { a 1,a 2,…,a n is in increasing order }

13. 13 Insertion Sort 的概念 :  從 j = 2 開始，將 a j 插入已排序好的 a 1,…,a j  1 間的 位置，使得 a 1,…,a j 都由小 → 大排好。  j 逐次遞增，重複上一步驟至做完。

14. 14 Example 5. Use insertion sort to sort 3, 2, 4, 1, 5 Sol :  (j=2 時， a 1 =3 可看成已經排序好的數列，此時要插 入 a 2 ) : 3 < 2  2, 3 交換  2, 3, 4, 1, 5  (j=3 時， a 1,a 2 已經排序好，此時要插入 a 3 ) : 4 > 2, 4 > 3  4 的位置不變  2, 3, 4, 1, 5  (j=4 時， a 1,a 2,a 3 已經排序好，此時要插入 a 4 ) : 1 < 2  將 1 插在最前面  1, 2, 3, 4, 5  (j=5 時， a 1,a 2,a 3,a 4 已經排序好，此時要插入 a 5 ) : 5 > 1, 5 > 2, 5 > 3, 5 > 4  5 不變  1, 2, 3, 4, 5 a 1 a 2 a 3 a 4 a 5

15. 15 Algorithm 5 The Insertion Sort procedure insertion_sort ( a 1,…,a n : real numbers with n ≥ 2 ) for j := 2 to n begin i := 1 while a j > a i i := i + 1 m := a j for k := 0 to j – i – 1 a j  k := a j  k  1 a i := m end { a 1,a 2,…,a n are sorted } ( Exercise : 9, 13, 23, 35, 39 ) 找出 a j 應插入的位置 最後 a i  1 < a j <= a i 將 a i, a i+1, …, a j  1 全部往右移一格

16. 16 2.2 The Growth of Functions To analyze the practicality of the program, we need to understand how quickly the function (number of operations used by this algorithm) grows as n (number of input elements) grows. eg. sort n objects  Alg. 1 : n 2 次計算  Alg. 2 : 8n 次計算 n 123…8910 # of op. Alg.1 149…6481100 Alg.2 81624…647280 better!

17. 17 Def 1. ( Big- O notation ) Let f and g be functions from the set of integers to the set of real numbers. We say that f (x) is O(g(x)) if there are constants C and k such that | f (x) | ≤ C | g(x) | whenever x > k. ( read as “ f ( x ) is big-oh of g ( x )” )

18. 18 Example 1. Show that f (x) = x 2 +2x+1 is O(x 2 ) Sol : Since x 2 +2x+1 ≤ x 2 +2x 2 +x 2 = 4x 2 whenever x > 1, it follows that f (x) is O(x 2 ) (take C = 4 and k =1 ) 另法： If x > 2, we see that x 2 +2x+1 ≤ x 2 +x 2 +x 2 = 3x 2 ( take C = 3 and k = 2 )

19. 19 Figure 2. The function f (x) is O(g(x)) Example 1( 補充 ). Show that f (n)= n 2 +2n +2 is O(n 3 ) Sol : Since n 2 +2n+2 ≤ n 3 +n 3 +n 3 = 3n 3 whenever n > 1, we see that f (n) is O(n 3 ) ( take C = 3 and k = 1 ) k Cg(x) f (x) g(x)g(x) f (x) k Note. The function g is chosen to be as small as possible.

20. 20 Example 4. How can big- O notation be used to estimate the sum of the first n positive integers? ( i.e.,  ) Sol : 1 + 2 + 3 + … + n ≤ n + n + … + n = n 2 ∴ is O(n 2 ), taking C =1 and k =1. Theorem 1. Let f (x) = a n x n +a n  1 x n  1 +…+a 1 x+a 0 where a 0, a 1, …, a n are real numbers. Then f (x) is O(x n ).

21. 21 Example 5. Give big- O estimates for f (n) = n! Sol : n! = 1  2  3  …  n ≤ n  n  …  n = n n ∴ n! is O(n n ), taking C =1 and k =1. Theorem 2,3 Suppose that f 1 (x) is O(g 1 (x)) and f 2 (x) is O(g 2 (x)), then (f 1 +f 2 )(x) is O(max(|g 1 (x)|, |g 2 (x)|)), (f 1 f 2 )(x) is O(g 1 (x) g 2 (x)). Example 6. (see Figure 3) 常見 function 的成長速度由小至大排列： 1 < log n < n < n log n < n 2 < 2 n < n!

22. 22 Exercise 7,11,19 Exercise 19(c) : f (n) = (n!+2 n )(n 3 +log(n 2 +1))  (n!+n!)(n 3 +n 3 ) = 4n 3  n! ∴ f (n) is O(n 3  n!) 取 C = 4, k = 3

23. 23 2.3 Complexity of Algorithms Q : How can the efficiency of an algorithm be analyzed ? Ans : (1) time (2) memory Def :  Time complexity : an analysis of the time required to solve a problem of a particular size. ( 評量方式 : 計算 # of operations ，如 “comparison” 次數， “ 加法 ” 或 “ 乘法 ” 次數等 )  Space complexity : an analysis of the computer memory required to solve a problem of a particular size. ( 通常是資 料結構探討的範圍 )

24. 24 Example 1. Describe the time complexity of Algorithm 1. Algorithm 1. ( Find Max ) procedure max(a 1,…,a n : integers) max := a 1 for i := 2 to n if max < a i then max := a i { max is the largest element } Sol : ( 計算 # of comparisons) i 值一開始 = 2 逐次加一，並比較是否 >n. 當 i 變成 n+1 時 因比 n 大，故結束 for 迴圈。 ∴ 共有 n 次 comparison 共有 n  1 次 comparison 故整個演算法共做 2n  1 次 comparison 其 time complexity 為 O(n).

25. 25 Example 2. Describe the time complexity of the linear search algorithm. Algorithm 2 ( Linear Search ) procedure ls ( x : integer, a 1,…,a n : distinct integers ) i := 1 While ( i  n and x ≠a i ) i := i +1 if i  n then location := i else location := 0 location = i  x = a i = 0  x  a i  i Sol : ( 計算 # of comparisons ) (Case 1) 當 x = a i for some i  n 時 此行只執行 i 次，故此行共 2i 次比較 加上 if ，共計 2i +1 次 comparisons. (Case 2) 當 x ≠ a i for all i 時 此行執行 n 次後 第 n + 1 次時 i = n + 1 > n 即跳出 ∴共計 2n+2 次 comparisons 由 (1) 、 (2) 取 worst-case 演算法的 time complexity 為 O(n)

26. 26 Example 4. Describe the average-case performance of the linear search algorithm, assuming that x is in the list. Sol : ( 計算 “ 平均比較次數 ” ) 已知當 x = a i 時，共需 2i + 1 次比較. ( by Example 2 ) x = a 1,a 2, …, 或 a n 的機率都是 1/n. ∴平均比較次數 ( 即期望值 ) = ( x = a 1 的比較次數 ) × ( x = a 1 的機率 ) + ( x = a 2 的比較次數 ) × ( x = a 2 的機率 ) + … + ( x = a n 的比較次數 ) × ( x = a n 的機率 ) = 3 × 1/n + 5 × 1/n + … + ( 2n+1) × 1/n = ( 3+5+…+(2n+1)) / n = / n = n + 2 ∴ average-case 的 time complexity 為 O(n) Alg. 2 ( Linear Search ) procedure ls ( x,a 1,…,a n ) i := 1 While ( i  n and x ≠a i ) i := i +1 if i  n then location := i else location := 0

27. 27 Example 3. Describe the time complexity of the binary search algorithm. Sol : 設 n = 2 k 以簡化計算 ( 若 n < 2 k ，其比較次數必小於等 於 n = 2 k 的情況 ) 因 while 迴圈每次執行後 整個 list 會切成兩半 故最多只能切 k 次 就會因 i = j 而跳出迴圈 ∴共比較 2k+2 次 time complexity 為 O(k) = O(log n) Alg. 3 ( Binary Search ) procedure bs ( x : integer, a 1,…,a n : increasing integers ) i := 1{ left endpoint } j := n{ right endpoint } while i < j /* ( k+1 次 ) begin m :=  ( i + j ) / 2  if x > a m then i := m+1 /* ( k 次 ) else j := m end if x = a i then location := i /* ( 1 次 ) else location := 0

28. 28 Example 5. What is the worst-case complexity of the bubble sort in terms of the number of comparisons made ? procedure bubble_sort ( a 1,…,a n ) for i := 1 to n  1 for j := 1 to n – i if a j > a j+1 then interchange a j and a i+1 { a 1,…,a n is in increasing order } Sol : 共 n  1 個 pass 第 i 個 pass 需 n – i 次比較 ∴共計 (n  1)+(n  2)+…+1 = 次比較 ∴ O(n 2 ) Note 1. 不管何種 case 都需做 次 比較。 Note 2. For 迴圈所需比較次數通常會省略，因此 Example 5,6 不再考慮。

29. 29 Example 6. What is the worst-case complexity of the insertion sort in terms of the number of comparisons made ? procedure insertion_sort ( a 1,…,a n ) for j := 2 to n begin i := 1 while a j > a i i := i +1 m := a j for k := 0 to j  i  1 a j  k := a j  k  1 a i := m end { a 1,…,a n are sorted } Sol : 做最多次比較的情況如下： 在考慮 a j 時 a 1 < a 2 < … < a j  1 < a j 此時共做 j 次比較 故共計 2+3+…+n =  1 次比較  O(n 2 ) ( 即 worst case 是 a 1 < a 2 < … < a n )

30. 30 Table 1. Commonly Used Terminology ComplexityTerminology O(1) constant complexity O(log n) Logarithmic complexity O(n)O(n) Linear complexity O(n log n) n log n complexity O(nb)O(nb) Polynomial complexity O(b n ), b >1 Exponential complexity O(n!) Factorial complexity Exercise : 7,8,13

31. 31 2.4 The integers and division ※探討一些 Number Theory 的基本觀念 Def 1. a, b : integers, a ≠ 0. a divides b (denote a | b ) if  c  Z, b=ac. ( a : a factor of b, b : a multiple of a ) ( a b if a does not divide b ) Corollary 1. If a, b, c  Z and a | b, a | c. then a | mb + nc whenever m, n  Z Def 2. p  Z + \ {1} is called prime ( 質數 ) if a p,  1<a< p, a  Z +. p is called composite ( 合成數 ) otherwise. Thm 2. (The fundamental theorem of arithmetic) 所有大於 1 的正整數，都是質數或可分解為質數 的乘積 ( 分解方式唯一 ) 。

32. 32 Thm 3. If n is a composite integer, then n has a prime divisor less than or equal to. Thm 4. There are infinitely many primes. Pf. 假設質數只有 n 個： p 1, p 2, …, p n ， Let Q = p 1 p 2 …p n +1. 證明 p 1, …, p n 都不能整除 Q. ※目前為止所知最大的質數是 2 p  1 的形式, where p is prime. 稱為 Mersenne primes ( 梅森質數 ). 梅森質數 Example 7. 2 2  1=3, 2 3  1=7, 2 5  1=31 are primes, but 2 11  1=2047=23  89 is not a prime. Def 3. If a=dq+r, where a,q,r  Z, d  Z +, 0  r<d then r = a mod d. Def 4. gcd ( greatest common divisor ) Def 7. lcm ( least common multiple ) Def 5. relatively prime ( 互質 )

33. 33 Def 8. If a, b  Z, m  Z +, then a is congruent to b modulo m if m|(a  b). (denote a ≡ b (mod m) ). Thm 9. Let m  Z +, a,b  Z. a≡b (mod m) iff  k  Z, a=b+km. Exercise 14. How many zeros are there at the end of 100! ? Sol : 計算 1  2  3  …  100=10 k  m, where 10 m ∵ 10=2  5 ，又 2 的次數必定比 5 多 ∴ 計算 1  2  3  …  100=5 k  n, where 5 n ∵ 5,10,15,20,…,100 才有因數 5, 而 25,50,75,100 有因數 25 ∴ k=24  共有 24 個 0 Homework : 試寫一 alg. 求出  n 的所有質數

34. 34 2.5 Integers and Algorithms ※ The Euclidean Algorithm ( 輾轉相除法求 gcd ) Example : Find gcd(91,287) Sol: 287 = 91  3 + 14 91 = 14  6 + 7 14 = 7  2 ∴ gcd(91,287) = 7 Example 12. Find gcd(416,662) ( 做做看 !) Ans : 2 if x |91 & x |287  x |14 ∴ gcd (91,287) = gcd(91,14) gcd (91,14) = gcd (14,7) gcd (14,7) = 7

35. 35 Algorithm 6. ( The Euclidean Algorithm) procedure gcd ( a, b : positive integers) x := a y := b while y≠0 begin r := x mod y ( if y > x then r = x) x := y y := r end { gcd (a, b) = x } eg. 求 gcd (6,12) x = 6 y = 12 while y≠0 r = 6 mod 12 =6 x = 12 y = 6 while y≠0 r = 12 mod 6 = 0 x = 6 y = 0 while y = 0, end. ∴ gcd (6,12) = 6 Exercise : 21,23

36. 36 2.6 Applications of Number Theory ※介紹中國餘數定理 Example 5. 孫子算經 : 「某物不知其數，三三數之餘二，五五數之餘 三，七七數之餘二，問物幾何 ? 」 i.e. x ≡ 2 (mod 3) x ≡ 3 (mod 5) x = ? x ≡ 2 (mod 7) Theorem 4. (The Chinese Remainder Theorem) Let m 1,m 2,…,m n be pairwise relatively prime positive integers. The system x ≡ a 1 (mod m 1 ) x ≡ a 2 (mod m 2 ) : x ≡ a n (mod m n ) has a unique solution modulo m = m 1 m 2 …m n. ( 即有一解 x, where 0  x < m, 且所有其他解 mod m 都等於 x )

37. 37 Proof of Thm 4: Let M k = m / m k  1  k  n ∵ m 1, m 2,…, m n are pairwise relatively prime ∴ gcd ( M k, m k ) = 1  integer y k s.t. M k y k ≡1 (mod m k ) ( by Thm. 3, 此處不証 )  a k M k y k ≡a k (mod m k ),  1  k  n Let x = a 1 M 1 y 1 +a 2 M 2 y 2 +…+a n M n y n ∵ m i | M j,  i≠j ∴ x≡a k M k y k ≡a k (mod m k )  1  k  n x 即為一解

38. 38 Example 6. ( 承 Example 5 題目敘述，求解 ) Let m = 3  5  7 = 105 M 1 = m / 3 = 35 M 2 = m / 5 = 21 M 3 = m / 7 = 15 ∵ 35≡2 (mod 3)  35  2 ≡ 1 (mod 3) 21≡1 (mod 5)  21  1 ≡ 1 (mod 5) 15≡1 (mod 7)  15  1 ≡ 1 (mod 7) ∴ x = a 1 M 1 y 1 + a 2 M 2 y 2 + a 3 M 3 y 3 = 2  35  2 + 3  21  1 + 2  15  1 = 233 ≡ 23 (mod 105) ∴ 最小的解為 23 ，其餘解都等於 23+105t for some t  Z + Exercise : 19 M1M1 y1y1 M2M2 y2y2 M3M3 y3y3

39. 39 Exercise 18. Find all solutions to the system of congruences x≡2 (mod 3) x≡1 (mod 4) x≡3 (mod 5) Sol : a 1 =2, a 2 =1, a 3 =3, m 1 =3, m 2 =4, m 3 =5 m=3  4  5=60 M 1 =20, M 2 =15, M 3 =12 20≡2 (mod 3)  20  2≡1 (mod 3) 15≡3 (mod 4)  15  3≡1 (mod 4) 12≡2 (mod 5)  12  3≡1 (mod 5) ∴ x = 2  20  2+1  15  3+3  12  3 = 80+45+108=233≡53 (mod 60)

40. 40 2.7 Matrices Algorithm 1. Matrix multiplication procedure matrix_multiplication(A : m  k matrix, B : k  n matrix ) for i := 1 to m for j :=1 to n begin c ij := 0 for q := 1 to k c ij := c ij + a iq b qj end { c =[c ij ] = A  B } Exercise : 23, 25