1. ::Network Optimization:: Minimum Spanning Trees and Clustering Taufik Djatna, Dr.Eng. 1

2. Spanning Tree A spanning tree of a graph is a tree and is a subgraph that contains all the vertices. A graph may have many spanning trees; for example, the complete graph on four vertices has sixteen spanning trees: 2

3. Spanning Tree – cont. 3

4. Minimum Spanning Trees Suppose that the edges of the graph have weights or lengths. The weight of a tree will be the sum of weights of its edges. Based on the example, we can see that different trees have different lengths. The question is: how to find the minimum length spanning tree? 4

5. Minimum Spanning Trees The question can be solved by many different algorithms, here is three classical minimum-spanning tree algorithms : –Boruvka's Algorithm –Kruskal's Algorithm –Prim's Algorithm 5

6. 6 Kruskal's Algorithm Joseph Bernard Kruskal, Jr Kruskal Approach: –Select the minimum weight edge that does not form a cycle Kruskal's Algorithm: sort the edges of G in increasing order by length keep a subgraph S of G, initially empty for each edge e in sorted order if the endpoints of e are disconnected in S add e to S return S

7. Kruskal's Algorithm - Example 7

8. 8

9. 9 Prim’s Algorithm Robert Clay Prim Prim Approach: –Choose an arbitrary start node v –At any point in time, we have connected component N containing v and other nodes V-N –Choose the minimum weight edge from N to V-N Prim's Algorithm: let T be a single vertex x while (T has fewer than n vertices) { find the smallest edge connecting T to G-T add it to T }

10. 10 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

11. 11 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

12. 12 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

13. 13 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

14. 14 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

15. 15 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

16. 16 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

17. 17 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

18. 18 Prim's Algorithm - Example 13 50 11 7 2 812 9 10 14 40 3 1 6 20

19. 19 Boruvka's Algorithm Otakar Borůvka –Inventor of MST –Czech scientist –Introduced the problem –The original paper was written in Czech in 1926. –The purpose was to efficiently provide electric coverage of Bohemia.

20. 20 Boruvka’s Algorithm Boruvka Approach: –Prim “in parallel” –Repeat the following procedure until the resulting graph becomes a single node. For each node u, mark its lightest incident edge. Now, the marked edges form a forest F. Add the edges of F into the set of edges to be reported. Contract each maximal subtree of F into a single node.

21. 21 Boruvka’s Algorithm - Example 2.1 1.3 2.3 1.2 2.2 3.1 2.4 3 1 1.5 1.4 2.6 2.7 2.5 3.2 5 3.3 4 4.1 5.1

22. Usage of Minimum Spanning Trees Network design: –telephone, electrical, hydraulic, TV cable, computer, road Approximation algorithms for NP-hard (non-deterministic polynomial-time hard) problems: –traveling salesperson problem Cluster Analysis 22

23. Clustering Definition –Clustering is “the process of organizing objects into groups whose members are similar in some way”. –A cluster is therefore a collection of objects which are “similar” between them and are “dissimilar” to the objects belonging to other clusters. –A data mining technique 23

24. Why Clustering? Unsupervised learning process Pattern detection Simplifications Useful in data concept construction 24

25. The use of Clustering Data Mining Information Retrieval Text Mining Web Analysis Marketing Medical Diagnostic Image Analysis Bioinformatics 25

26. Network Optimization Models: Maximum Flow Problems The problem statement Solving by linear programming Augmenting path algorithm 26

27. Maximum Flow Problem Given:Directed graph G=(V, E), Supply (source) node O, demand (sink) node T Capacity function u: E  R. Goal:Given the arc capacities, send as much flow as possible from supply node O to demand node T through the network. Example: 4 4 5 6 4 4 5 5 O A D B C T 27

28. Characteristics of a feasible flow Let x ij denote the flow through arc i  j. Capacity u ij of arc i  j is the upper bound on the flow shipped through arc i  j. Thus, we have the following constraints: 0  x ij  u ij, for any arc i  j Every node i, except the source and the sink, should satisfy the conservation-of-flow constraint, i.e., flow into node i = flow out of node i In terms of x ij the constraint is For any flow that satisfies the conservation-of-flow constraints, flow out of the source = flow into the sink This is the amount we want to maximize. 28

29. Linear Program for the Maximum Flow Problem Summarizing, we have the following linear program: 0  x ij  u ij, for any arc i  j This linear program can be solved by a special type of the Simplex method, called Network Simplex. In this class, we will cover a more efficient algorithm for the Maximum Flow Problem, the Augmenting Path Algorithm. 29

30. Towards the Augmenting Path Algorithm Idea: Find a path from the source to the sink, and use it to send as much flow as possible. In our example, 5 units of flow can be sent through the path O  B  D  T ; Then use the path O  C  T to send 4 units of flow. The total flow is 5 + 4 = 9 at this point. Can we send more? O A D B C T 4 4 56 4 45 5 5 5 4 4 5 30

31. Towards the Augmenting Path Algorithm If we redirect 1 unit of flow from path O  B  D  T to path O  B  C  T, then the freed capacity of arc D  T could be used to send 1 more unit of flow through path O  A  D  T, making the total flow equal to 9+1=10. To realize the idea of redirecting the flow in a systematic way, we need the concept of residual capacities. O A D B C T 4 4 56 4 45 5 5 5 5 4 4 4 4 1 5 1 1 31

32. Residual capacities Suppose we have an arc with capacity 6 and current flow 5: Then there is a residual capacity of 6-5=1 for any additional flow through B  D. On the other hand, at most 5 units of flow can be sent back from D to B, i.e., 5 units of previously assigned flow can be canceled. In that sense, 5 can be considered as the residual capacity of the reverse arc D  B. To record the residual capacities in the network, we will replace the original directed arcs with undirected arcs: B D 6 5 B D 1 5 The number at B is the residual capacity of B  D; the number at D is the residual capacity of D  B. 32

33. Residual Network The network given by the undirected arcs and residual capacities is called residual network. In our example, the residual network before sending any flow: Note that the sum of the residual capacities on both ends of an arc is equal to the original capacity of the arc. How to increase the flow in the network based on the values of residual capacities? O A D B C T 4 4 5 6 4 4 5 5 0 0 0 0 0 0 0 0 33

34. Augmenting paths An augmenting path is a directed path from the source to the sink in the residual network such that every arc on this path has positive residual capacity. The minimum of these residual capacities is called the residual capacity of the augmenting path. This is the amount that can be feasibly added to the entire path. The flow in the network can be increased by finding an augmenting path and sending flow through it. 34

35. Updating the residual network by sending flow through augmenting paths Continuing with the example, Iteration 1: O  B  D  T is an augmenting path with residual capacity 5 = min{5, 6, 5}. After sending 5 units of flow through the path O  B  D  T, the new residual network is: O A D B C T 4 4 4 4 5 0 0 0 0 0 0 1 0 5 5 5 5 6 5 0 0 0 35

36. Updating the residual network by sending flow through augmenting paths Iteration 2: O  C  T is an augmenting path with residual capacity 4 = min{4, 5}. After sending 4 units of flow through the path O  C  T, the new residual network is: O A D B C T 4 4 0 0 0 4 5 0 0 0 1 0 5 5 5 0 1 4 4 4 36

37. Updating the residual network by sending flow through augmenting paths Iteration 3: O  A  D  B  C  T is an augmenting path with residual capacity 1 = min{4, 4, 5, 4, 1}. After sending 1 units of flow through the path O  A  D  B  C  T, the new residual network is: O A D B C T 0 0 5 5 0 4 4 4 0 0 0 1 5 1 4 4 3 3 1 1 1 2 4 0 5 3 37

38. Terminating the Algorithm: Returning an Optimal Flow There are no augmenting paths in the last residual network. So the flow from the source to the sink cannot be increased further, and the current flow is optimal. Thus, the current residual network is optimal. The optimal flow on each directed arc of the original network is the residual capacity of its reverse arc: flow( O  A )=1, flow( O  B )=5, flow( O  C )=4, flow( A  D )=1, flow( B  D )=4, flow( B  C )=1, flow( D  T )=5, flow( C  T )=5. The amount of maximum flow through the network is 5 + 4 + 1 = 10 (the sum of path flows of all iterations). 38

39. The Summary of the Augmenting Path Algorithm Initialization: Set up the initial residual network. Repeat –Find an augmenting path. –Identify the residual capacity c* of the path; increase the flow in this path by c*. –Update the residual network: decrease by c* the residual capacity of each arc on the augmenting path; increase by c* the residual capacity of each arc in the opposite direction on the augmenting path. Until no augmenting path is left Return the flow corresponding to the current optimal residual network 39

40. 40 Multicommodity Flows: a brief glimpse at an important problem 1 2 3 4 5 6 3366 $5, 4 $2, 4 $6, 4 $0, 8 Send 6 units of commodity 1 from node 1 to node 4. The costs and capacities are given. What is the minimum cost multicommodity flow? Send 3 units of commodity 2 from node 3 to node 6. Two minimum cost flows on a shared network

41. 41 Multicommodity Flows 1 2 3 4 5 6 33 66 $5, 4 $2, 4 $6, 4 $0, 8 Send 5 units of commodity 1 from node 1 to node 4. Send 3 units of commodity 2 from node 3 to node 6. 4 2 1 2 The shared arc capacities make the problem much more complex. Optimum flows may be fractional.

42. 42 Multicommodity flow Problems can be large Consider a communication network with 1000 nodes, and 10,000 arcs. Suppose each node wants to send a different message to each other node. 1,000,000 messages. 1,000,000 commodities 10 billion variables. (10,000 variables per commodity)

43. 43 Multicommodity Flow Problems are important in practice Manufacturing Networks Road and Rail Transportation Networks Air Transportation Networks Communication Networks

44. Maximum Flow Problems Maximum Flow Problems Ford-Fulkerson Demo 44

45. 45 Ford-Fulkerson Algorithm s 2 3 4 5t 10 9 8 4 6 2 0 0 0 0 0 0 0 0 G: Flow value = 0 0 flow capacit y

46. 46 Ford-Fulkerson Algorithm s 2 3 4 5t 10 9 8 4 6 2 0 0 0 0 0 0 0 0 G: s 2 3 4 5t 10 9 4 6 2 G f : 10 8 8 8 8 X X X 0 Flow value = 0 capacit y residual capacity flow

47. 47 Ford-Fulkerson Algorithm s 2 3 4 5t 10 9 8 4 6 2 8 0 0 0 0 8 8 0 0 G: s 2 3 4 5t 10 4 6 G f : 8 8 8 9 2 2 2 10 2 X X X 2 X Flow value = 8

48. 48 0 Ford-Fulkerson Algorithm s 2 3 4 5t 10 9 8 4 6 2 0 0 0 2 8 2 G: s 2 3 4 5t 4 2 G f : 10 8 2 7 6 X 6 6 6 X X 8 X Flow value = 10

49. 49 Ford-Fulkerson Algorithm s 2 3 4 5t 10 9 8 4 6 2 0 6 6 8 8 2 G: s 2 3 4 5t 1 6 G f : 10 8 8 6 6 6 4 4 4 2 X 8 2 8 X X 0 X Flow value = 16

50. 50 Ford-Fulkerson Algorithm s 2 3 4 5t 10 9 8 4 6 2 2 8 8 8 8 0 G: s 2 3 4 5t 6 2 G f : 10 8 6 8 8 2 2 1 2 8 2 X 9 7 9 X X 9 X X 3 Flow value = 18

51. 51 Ford-Fulkerson Algorithm s 2 3 4 5t 10 9 8 4 6 2 3 9 9 9 7 0 G: s 2 3 4 5t 1 9 1 1 6 2 G f : 10 7 6 9 9 3 1 Flow value = 19

52. 52 Ford-Fulkerson Algorithm s 2 3 4 5t 10 9 8 4 6 2 3 9 9 9 7 0 G: s 2 3 4 5t 1 9 1 1 6 2 G f : 10 7 6 9 9 3 1 Flow value = 19 Cut capacity = 19