1. 1 Optimization Spring 2010

2. 2 Some practical remarks Lecturers: Kristoffer Arnsfelt Hansen and Peter Bro Miltersen. Homepage: www.daimi.au.dk/dOptwww.daimi.au.dk/dOpt Exam: Written, 3 hours. There will be three compulsory assignments. If you want to transfer credit from last year, let me know as soon as possible and before March 1. The solution to the compulsory assignments should be handed in at specific tutorial and given to the instructor in person. Text: “Kompendium” available at GAD.

3. 3 FAQ about compulsory assignments Q: Do I really have to hand in all three assignments? A: YES! Q: Do I really have to hand in all three assignments on time? A: YES! Q: What if I can’t figure out how to solve them? A: Ask your instructor. Start solving them early, so that you will have sufficient time. Q: What if I get sick or my girlfriend breaks up or my hamster dies? A: Start solving them early, so that you will have sufficient time in case of emergencies. Q: Do I really have to hand in all three assignments? A: YES!

4. 4 Maximum matching Max Flow Shortest paths Min Cost Flow Linear Programming Mixed Integer Linear Programming Polynomial time (easy to solve) by Local Search. Worst case exponential time (more difficult to solve, but not impossible). = reduction The course in one slide

5. Max Flow Max Flow was already covered in dADS The Ford-Fulkerson and Edmonds-Karp algorithms for solving max flow instances are not exam curriculum. Modelling concrete optimization problem using max flow is still exam curriculum. 5

6. 6 The Max Flow Problem

7. 7 Flow networks Flow networks are the problem instances of the max flow problem. A flow network is given by 1) a directed graph G = (V,E) 2) capacities c: E ! R +. 3) The source s 2 V and the sink t 2 V. Convention: c(u,v)=0 for (u,v) not in E.

8. 8 Flows Given flow network, a flow is a feasible solution to the max flow problem. A flow is a map f: V £ V ! R satisfying capacity constraints: 8 (u,v): f(u,v) · c(u,v). Skew symmetry: 8 (u,v): f(u,v) = – f(v,u). Flow conservation: 8 u 2 V – {s,t}:  v 2 V f(u,v) = 0

9. 9 A Flow 5 10 2 -2 7 1 9 4 5 11

10. 10 Skew Symmetry is modeled as 2 4 -2 2 Our flows are net flows.

11. 11 Flow Conservation 7 15 11  u f(v 3, u) = 0?

12. 12 Flow Conservation 7 15 11  u f(v 3, u) = (-7)+1+(-5)+11=0 -7 -5

13. 13 7 15 11 -7 -5 Flow entering v 3 = 12 Flow leaving v 3 = 12 Flow conservation expresses that v 3 is in balance.

14. 14 The max flow problem f(X,Y) :=  u 2 X, v 2 Y f(u,v). Value of f: |f| := f(s,V). The Max Flow Problem: Given a flow network (V,E,c,s,t), find the feasible flow f maximizing |f|.

15. 15 Algorithms for solving the Max Flow problem Ford-Fulkerson algorithm Edmonds-Karp algorithm Examples of the local search algorithmic pattern.

16. 16 Local Search Pattern LocalSearch(ProblemInstance x) y := feasible solution to x; while 9 z ∊ N(y): v(z) > v(y) do y := z; od; return y; N(y) is a neighborhood of y.

17. 17 Local Search

18. 18 Local search checklist Design: How do we find the first feasible solution? Neighborhood design? Which neighbor to choose? Analysis: Partial correctness? (termination ) correctness) Termination? Complexity?

19. 19 The first flow?

20. 20 The first flow? 0 0 0 0 0 0 0 0 0

21. 21 Neighborhood design Given a flow, how can we find a slightly different (and hopefully slightly better) flow?

22. 22 The first flow? 0 0 0 0 0 0 0 0 0

23. 23 4 4 4 Path Flows Path Flow = flow with positive values only on a simple path from s to t

24. 24 Idea Let N(y) be the flows that can be obtained from y by adding a path flow (without violating the capacity bounds). The path flow we add should use some path in (V,E) along which every edge has some unused capacity.

25. 25 Example 1 1 1 1 1 sst

26. 26 Example 1 1 1 1 1 0 0 0 0 0 st

27. 27 Example 1 1 1 1 1 0 0 0 0 0 st

28. 28 Another Path? 1 1 1 1 1 0 1 1 0 1 No. st

29. 29 Optimal? 1 1 1 1 1 0 1 1 0 1 st

30. 30 No! 1 1 1 1 1 1 1 0 1 1 st

31. 31 Some remarks When designing a local search algorithm, the most obvious neighborhood relation is not necessarily the right one. That a solution cannot be improved by using some specified set of changes does not necessarily mean it is globally optimal.

32. 32 Better Idea The graph (V,E) is not really the right one to find paths in. The path flow we add should use some path in (V,E f ) where E f is the set of edges that has unused capacity under the current flow f. E f may include edges (u,v) 2 E as well as back-edges (u,v) for which (v,u) 2 E.

33. 33 The residual network Let G=(V,E,c,s,t) be a flow network and let f be a flow in G. The residual network is the flow network with edges and capacities E f = {(u,v) 2 V £ V| f(u,v) < c(u,v)} c f (u,v) = c(u,v) - f(u,v)

34. 34 Augmenting Paths A simple path p from s to t in G f is called an augmenting path. Let c f (p) = min{ c f (u,v) : (u,v) is on p} Let f p (u,v) be c f (p) if (u,v) is on p -c f (p) if (v,u) is on p 0 otherwise Then f p is a path flow in G f with value c f (p)

35. 35 Ford-Fulkerson method Ford-Fulkerson(G) f = 0 while( 9 simple path p from s to t in G f ) f := f + f p output f

36. 36 Design: How do we find the first feasible solution? Neighborhood design? Which neighbor to choose? Analysis: Partial correctness? (termination ) correctness) Termination? Complexity? Local search checklist ٧ ٧

37. 37 Cuts A cut (S,T) in G is a partition of V into S and T=V-S with s 2 S and t 2 T. Its capacity is c(S,T) =  u 2 S, v 2 T c(u,v) A minimum cut is a cut with smallest capacity among all cut.

38. 38 ST c(S,T)=26 A cut

39. 39 Lemma 26.5 Let f be a flow in G and let (S,T) be a cut in G. Then |f| = f(S,T).

40. 40 Corollary 26.6 Let f be a flow in G and let (S,T) be a cut in G. Then |f| · c(S,T). This is a weak duality theorem.

41. 41 Max Flow – Min Cut Theorem Let f be a flow in G. The following three conditions are equivalent: 1. f is a maximum flow 2. G f contains no augmenting path 3. There is a cut (S,T) so that |f|=c(S,T)

42. 42 Max Flow – Min Cut Theorem The value of the maximum flow in G is equal to the capacity of the minimum cut in G. This is a strong duality theorem.

43. 43 Flows vs Cuts The solution values agree, not the solutions themselves – flows and cuts are completely different objects. Given a max flow we can easily find a min cut (follows from proof of max flow-min cut theorem). Going the other way is less obvious.

44. 44 Consequence The Ford-Fulkerson method is partially correct, i.e., if it terminates it produces the flow with the maximum value.

45. 45 Local search checklist Design: How do we find the first feasible solution? Neighborhood design? Which neighbor to choose? Analysis: Partial correctness? (termination ) correctness) Termination? Complexity? ٧ ٧ ٧

46. 46 Termination Suppose all capacities are integers. We start with a flow of value 0. In each iteration, we get a new flow with higher integer value. We always have a legal flow, i.e., one of value at most |f|. Hence we can have at most |f| iterations.

47. 47 Correctness of Ford-Fulkerson Since Ford-Fulkerson is partially correct and it terminates if capacities are integers it is a correct algorithm for finding the maximum flow if capacities are integers.

48. 48 Integrality Theorem (26.11) If a flow network has integer valued capacities, there is a maximum flow with an integer value on every edge. The Ford- Fulkerson method will yield such a maximum flow. The integrality theorem is often extremely important when “programming” and modeling using the max flow formalism.

49. 49 Local search checklist Design: How do we find the first feasible solution? Neighborhood design? Which neighbor to choose? Analysis: Partial correctness? (termination ) correctness) Termination? Complexity? ٧ ٧ ٧ ٧

50. 50 Complexity The Ford-Fulkerson can be implemented to run in time at most O(|E| |f|). Is this fast?

51. 51 Polynomial time algorithms Defintion: A polynomial time algorithm is an algorithm than runs in time polynomial in n, where n is the number of bits of the input. How we intend to encode the input influences if we have a polynomial algorithm or not. Usually, some “standard encoding” is implied. In this course: Polynomial ¼ Fast Exponential ¼ Slow

52. 52 java MaxFlow ?????????? How to encode max flow instance?

53. 53 java MaxFlow 6#0|16|13|0|0|0#0|0|10|12|0|0 #0|4|0|0|14|0#0|0|9|0|0|20 #0|0|0|7|0|4|#0|0|0|0|0|0 How to encode max flow instance?

54. 54 Complexity of Ford-Fulkerson With standard (decimal or binary) representation of integers, Ford-Fulkerson is an exponential time algorithm.

55. 55 java MaxFlow 111111 #|1111111111111111|1111111111111||| #||1111111111|111111111111|| #|1111|||11111111111111| #||111111111|||11111111111111111111 #|||1111111||1111 #|||||

56. 56 Complexity of Ford-Fulkerson With unary (4 ~ 1111) representation of integers, Ford- Fulkerson is a polynomial time algorithm. Intuition: When the input is longer it is easier to be polynomial time as a function of the input length. An algorithm which is polynomial if integer inputs are represented in unary is called a pseudo-polynomial algorithm. Intuitively, a pseudo-polynomial algorithm is an algorithm which is fast if all numbers in the input are small.

57. 57 Edmonds-Karp Edmonds-Karp algorithm for Max Flow: Implement Ford-Fulkerson by always choosing the shortest possible augmenting path, i.e., the one with fewest possible edges.

58. 58 Complexity of Edmonds-Karp The total running time is O(|V| |E| 2 ) and Edmonds-Karp is a polynomial time algorithm for Max Flow.

59. 59 Modeling with Max Flow: A scheduling problem A set of jobs must be scheduled on M identical machines. Each job j has an arrival date r j, a required delivery date d j and a processing time p j · d j - r j. A job can be preemptively moved from one machine to another. Can the jobs be scheduled on the machines so that the deadlines are met?

60. 60 M = 3

61. 61

62. 62 Reduction: Maximum Matching ! Max Flow What is the maximum cardinality matching in G?

63. 63 G

64. 64 G’ G’ st All capacities are 1

65. 65 Finding a balanced set of Representatives A city has clubs C 1, C 2,…,C n and parties P 1, P 2,…,P m. A citizen may be a member of several clubs but may only be a member of one party. A balanced city council must be formed by including exactly one member from each club and at most u k members from party P k. (Ahuja, Application 6.2)

66. 66

67. 67 Processes p 1, p 2, …, p n must be assigned to one of two processors. Assigning p i to processor k gives computation cost a ik. If p i and p k are assigned to different processors, communication cost c ik is incurred. Minimize the total cost. Distributed Computation on Two- Processor Computer (Ahuja, Application 6.5)

68. 68