Presentation on theme: "1/24 Advanced Graph Algorithms (I) What we do not cover but you are expected to know –Mathematical induction, basic data structure, sorting, shortest path,"— Presentation transcript:
1/24 Advanced Graph Algorithms (I) What we do not cover but you are expected to know –Mathematical induction, basic data structure, sorting, shortest path, minimum spanning tree, dynamic programming, divide-and-conquer –Homework assignment Better if you know NP-complete, NP-hard –Pick it up yourself if not. It becomes a common sense in computer science.
2/24 Advanced Graph Algorithms (II) Who should take – 對演算法有興趣 – 對抽象思考有興趣 – 培養基礎，以後有志於從事研究工作 – 有興趣知道如何將理論應用到實際問題 生物資訊，網路搜尋，自然語言 對任職於 Google, Facebook, Microsoft 有興趣者 課程網站（所有的 announcement 都放在此） – 我在研究院的網頁下面的教學網站 –http://iasl.iis.sinica.edu.tw/hsu/#teachhttp://iasl.iis.sinica.edu.tw/hsu/#teach
4/24 Biography 許聞廉 Wen-Lian Hsu 中研院資訊所特聘研究員 許聞廉 Wen-Lian Hsu 1973 台大數學學士 1980 康乃爾 Operations Research 博士 美國西北大學工業工程系 中央研究院資訊所 – 發展「自然輸入法」、許氏鍵盤 Research interests: –Design of algorithms, artificial intelligence, natural language processing, bioinformatics, knowledge management
5/24 Scope Consecutive ones test PQ-trees and PC- trees Planar graphs Maximal Planar Subgraph Algorithm Chordal graphs Interval graphs Applications –Sequence assembly –Motif discovery –de novo sequencing –Protein structure prediction
6/24 A few Examples of Mathematical Induction Most algorithms we designed are “recursive.” N! = N x (N-1)! The theoretical basis is “Mathematical Induction.”
7/24 A Hat Problem (I) No strategy In the worst case, all men were shot. Strategy 1 (with collaboration) In the worst case, half of the men will be shot. N prisoners lined up in a row, each one can see the hats of all people in front of him. A person who guesses the color of his hat correctly can survive
8/24 A Hat Problem (II) Design a strategy so that as few men will die as possible. Strategy 1 (at least half can survive, probably ¾ will) Divide the men into two groups: odd-numbered and even-numbered. Each odd-numbered person should tell the person in front the correct color (since he can see it). As for the person himself, there is still ½ chance that he will survive)
9/24 A Hat Problem (III) Message Passing Suppose we use 0 to indicate white hat and 1 for black hat Let the original sequence be Then the sequences each man will see are as follows How do you let each man guess the right # (except the first one)? odd-evenness (or parity) of the # of 0’s and 1’s.
10/24 A Hat Problem (IV) If the current hat is 0, then moving to the next sequence will only change the parity of 0 (the parity of 1 stays the same) Everyone knows the parity of 0 and 1 for the sequence in front of him. If the 1 st person says the parity of 1 for his sequence (either odd or even), then by checking whether the parity of 1 changes, the 2 nd person knows his hat color By induction, everyone afterward can compute his hat color
11/24 Marriage Theorem The following condition is both necessary and sufficient: Every set of r girls, 1 r n, like at least r boys. There are n girls and n boys. Each girl has a list of boys she can marry. Assume a boy never rejects a girl’s offer. Under what condition can you find a perfect match? Prove by induction (for the sufficient part, since necessity is clear). How do you reduce the problem size to a smaller one? Easy case: There is a subset of k girls who like exactly k boys By induction, can match these k girls with the k boys. Again, by induction, the remaining n-k girls can be matched to the n-k boys Otherwise: we have: Every set of r girls, 1 r n, likes > r boys. Marry a girl with a boy first, then for the remaining n-1 girls and n-1 boys, the condition still holds. Sufficient: If the condition holds, then can find a perfect match Necessary: If there is a perfect match, then the condition must hold.
12/24 Maximum Subsequence Sum Problem Given (possibly negative) integers A 1, A 2, …, A N, find the maximum value of over all i, j For input – 2, 11, -4, 13, -5, -2, the answer is 20 k= i j AkAk
13/24 Algorithm 1: Brute-force method Given n integers A 1, A 2, …, A N, how many subsequences can you form? For example: 1, 2, 3, 4 –The possible sums include: 1, 1+2, 1+2+3, , 2+3, , –Find the maximum in the above sums –An O(N 3 ) solution
14/24 Algorithm 2: A Bit Clever Algorithm By noting = A j + We can reuse partial computation in previous steps k=i j AkAk j-1 AkAk
15/24 Algorithm 2: A Bit Clever Algorithm Sum (i,j) = 0 for all i, j For i = 0 to n For j = i to n Sum (i,j) Sum (i, j-1) + A j end An O(N 2 ) algorithm
16/24 Algorithm 3: Divide and Conquer Divide Part: –Split the problem into two roughly equal subproblems, which are then solved recursively. Conquer Part: –Patching together the two solutions of the subproblems with small amount of additional work.
17/24 Algorithm 3: Divide and Conquer Maximum subsequence sum problem –Divide the sequence into two equal parts –The maximum subsequence sum can be found in one of three places: Entirely in the left half of the input Entirely in the right half of the input Crosses the middle and in both halves
18/24 Algorithm 3: Divide and Conquer In first half: 6 First HalfSecond Half
19/24 Algorithm 3: Divide and Conquer In second half: 8 First HalfSecond Half
20/24 Algorithm 3: Divide and Conquer Crosses middle: 11 First HalfSecond Half
21/24 Algorithm 3: Divide and Conquer When the maximum subsequence sum crosses the middle T(n) = 2T(n/2) + 2n/2 An O(n log n) algorithm The maximum subsequence sum from the end
22/24 Algorithm 4: The most clever one An improvement over algorithm 2 Clever observations: –No negative subsequence can possibly be a prefix of the optimal subsequence
23/24 Algorithm 4: The most clever one Use induction. Keep two sums at each iteration and update them based on the three conditions below. Note that maxendSUM is the optimal subsequence sum from the right end If maxendSUM n-1 + A n 0, then maxendSUM n = 0 If maxendSUM n-1 + A n > 0, then maxendSUM n maxendSUM n-1 + A n If maxendSUM n-1 + A n > maxSUM n-1, then maxSUM n = maxendSUM n-1 + A n Else maxSUM n = maxendSUM n-1 maxSUM n-1 maxendSUM n-1 1 n - 1 maxSUM n-1 maxendSUM n-1 + A n 1 n When would adding A n change the maxSUM n ?
24/24 Algorithm 4: The most clever one An O(N) algorithm –It takes constant time to update the two sums at each iteration Example: Sequence: -2, 11, -4, 13, -5, -2 Initially, maxendSUM0 = 0, maxSUM0 = 0 i = 1: (a) maxendSUM1 = 0 i = 2: (b) maxendSUM2 = 11, (c) maxSUM2 = 11 i = 3: (b) maxendSUM3 = 7, (c) maxSUM3 = 11 i = 4: (b) maxendSUM4 = 20, (c) maxSUM4 = 20 i = 5: (b) maxendSUM5 = 15, (c ) maxSUM5 = 20 i = 6: (b) maxendSUM6 = 13, (c ) maxSUM6 = 20
25/24 Common Computational Models Discrete algorithm –Probabilistic, approximation, on-line, randomized Non-linear programming (numerical) Statistical –Regression –Machine learning Neural net, SVM, Hidden Markov Model, Maximum entropy, Conditional random fields, –Evolutionary Genetic algorithm, particle swarm Areas: NLP, ASR, IR, IE, DM