1. I/O-Efficient Batched Union-Find and Its Applications to Terrain Analysis Pankaj K. Agarwal, Lars Arge, and Ke Yi Duke University University of Aarhus

2. The Union-Find Problem A universe of N elements: x 1, x 2, …, x N Initially N singleton sets: {x 1 }, {x 2 }, …, {x N } Each set has a representative Maintain the partition under –Union( x i, x j ) : Joins the sets containing x i and x j –Find( x i ) : Returns the representative of the set containing x i

3. The Solution d bja eg h fl n m i srczk p representatives d bja eg h fl n m Union(d, h) : link-by-rank d bja eg h fl n Find(n) : path compression m

4. Complexity O(N α(N)) for a sequence of N union and find operations [Tarjan 75] – α() : Inverse Ackermann function (very slow!) –Optimal in the worst case [Tarjan79, Fredman and Saks 89] Batched (Off-line) version –Entire sequence known in advance –Can be improved to linear on RAM [Gabow and Tarjan 85] –Not possible on a pointer machine [Tarjan79]

5. Simple and Good, as long as … The entire data structure fits in memory

6. The I/O Model Main memory of size M Disk of infinite size One I/O transfers B items between memory and disk

7. Our Results An I/O-efficient algorithm for the batched union-find problem using O(sort( N )) = O( N/B log M/B (N/B) ) I/Os expected –Same as sorting –optimal in the worst case A practical algorithm using O(sort( N ) log(N/M) ) I/Os Applications to terrain analysis –Topological persistence : O(sort( N )) I/Os –Contour trees : O(sort( N )) I/Os

8. I/O-Efficient Batched Union-Find Assumption: No redundant unions –Each union must join two different sets –Will remove later Two-stage algorithm –Convert to interval union-find Compute an order on the elements s.t. each union joins two adjacent sets –Solve batched interval union-find

9. Union Graph r ab cdef ghi 1: Union(d, g) 2: Union(a, c) 3: Union(r, b) 4: Union(a, e) 5: Union(e, i) 6: Union(r, a) 7: Union(a, d) g 8: Union(d, h) r 9: Union(b, f) 3 1 2 4 5 7 8 9 6 r ab cde f g h i 3 1 2 4 5 7 8 9 6 Equivalent union trees (Tree if no redundant unions)

10. Transforming the Union Tree r ab cdef ghi 3 1 2 4 5 7 8 9 6 r ab cdef g h i 3 1 2 4 5 7 8 9 6 r ab c d efg h i 3 1 2 4 5 7 8 9 6 r ab c d e f g h i 3 1 2 4 5 7 8 9 6 Weights along root-to-leaf path decrease

11. Formulating as a Batched Problem r ab cdef ghi 3 1 2 4 5 7 8 9 6 r ab c d e f g h i 3 1 2 4 5 7 8 9 6 For each edge, find the lowest ancestor edge with a higher weight

12. Cast in a Geometry Setting r ab cdef ghi 3 1 2 4 5 7 8 9 6 Euler Tour In O(sort( N )) I/Os [Chiang et al. 95] 1 2 3 4 5 6 7 8 9 x : positions in the tour y : weight

13. Cast in a Geometry Setting r ab cdef ghi 3 1 2 4 5 7 8 9 6 1 2 3 4 5 6 7 8 9 For each edge, find the lowest ancestor edge with a higher weight For each segment, find the shortest segment above and containing it

14. Distribution Sweeping M/B vertical slabs checked here checked recursively Total cost: O(sort( N ))

15. In-Order Traversal r ab c d e f g h i 3 1 2 4 5 7 9 6 Weights along root-to-leaf path decrease At u, with child u 1,…, u k (in increasing order of weight) 1.Recursively visit subtree at u 1 2.Return u 3.For i=2,…, k Recursively visit subtree at u i br 8 aceigdhf Claim: this traversal produces the right order

16. Solving Interval Union-Find Union: x : two operands y : time stamp Find: x : operand y : time stamp representative

17. Solving Interval Union-Find Union: x : two operands y : time stamp Find: x : operand y : time stamp Four instances of batched ray shooting: O(sort( N ))

18. Solving Interval Union-Find Union: x : two operands y : time stamp Find: x : operand y : time stamp Four instances of batched ray shooting: O(sort( N ))

19. Handling Redundant Unions Union tree becomes a general graph Compute the minimum spanning tree –O(sort( N )) I/Os (randomized) [Chiang et al. 95] O(sort( N ) loglog B ) I/Os (deterministic) [Arge et al. 04] –Deterministic O(sort( N )) I/Os if graph is planar –Only MST edges are non-redundant

20. Applications 1.Topological Persistence 2.Contour Trees

21. Application: Topological Persistence Introduced by Edelsbrunner et al. 2000 Measure importance on a surface –Feature extraction –Topological de-noising Many applications –Surface modeling –Shape analysis –Terrain analysis –Computational Biology

22. Topological Persistence Illustrated

23. Formulated as Batched Union-Find Represented as a triangulated mesh Consider minimum-saddle pairs When reach –A minimum or maximum: do nothing –A regular point u : Issue union( u,v ) for a lower neighbor v –A saddle u : let v and w be nodes from u ’s two connected pieces in its lower link Issue: find( v ), find( w ), union( u,v ), union( u,w ) lower link

24. Experiment 1: Random Union-Find 128MB memory

25. Experiment 2: Topological Persistence on Terrain Data Neuse River Basin of North Carolina: ~ 0.5 billion points

26. Experiment 2: Topological Persistence on Terrain Data Entire data set (0.5b): IM fails and EM takes 10 hours 128MB memory

27. Contour Trees

28. Summary An I/O-efficient algorithm for the batched union-find problem using O(sort( N )) = O( N/B log M/B (N/B) ) I/Os –optimal in the worst case A practical algorithm using O(sort( N ) log(N/M) ) I/Os Applications to terrain analysis –Topological persistence : O(sort( N )) I/Os –Contour trees : O(sort( N )) I/Os Open Question: –On-line case: Can we get below O(N α(N)) I/Os?

29. Thank you!

30. Previous Results Directly maintain contours –O( N log N ) time [van Kreveld et al. 97] –Needs union-split-find for circular lists –Do not extend to higher dimensions Two sweeps by maintaining components, then merge –O( N log N ) time [Carr et al. 03] –Extend to arbitrary dimensions

31. Join Tree and Split Tree 9 8 7 6 5 4 3 2 1 Join tree 9 8 7 6 5 4 3 2 1 Split tree Qualified nodes 9 8 7 6 5 4 3 1 Join tree 9 8 7 6 5 4 3 1 Split tree

32. Final Contour Tree 9 8 7 6 5 4 3 2 1 Join tree 9 8 7 6 5 4 3 2 1 Split tree 9 8 7 6 5 4 3 2 1 Contour tree Hard to BATCH!

33. Another Characterization 9 8 7 6 5 4 3 2 1 Join tree 9 8 7 6 5 4 3 2 1 Split tree 9 8 7 6 5 4 3 2 1 Contour tree u v w u v w u u w Let w be the highest node that is a descendant of v in join tree and ancestor of u in split tree, (u, w) is a contour tree edge Now can BATCH!

34. Map to Rectangles 9 8 7 6 5 4 3 2 1 Join tree 9 8 7 6 5 4 3 2 1 Split tree u v w u v w u v w Can be solved in O(sort(N)) I/Os (practical, too)

35. Topological Persistence

36. Label Nodes with Intervals Using Euler tour (O(sort(N) I/Os) 9 8 7 6 5 4 3 2 1

37. Map to Rectangles 9 8 7 6 5 4 3 2 1 Join tree 9 8 7 6 5 4 3 2 1 Split tree u v w u v w u v w Can be solved in O(sort(N)) I/Os (practical, too)

38. Formulated as Batched Union-Find Represented as a triangulated mesh Consider minimum-saddle pairs When reach –A minimum or maximum: do nothing –A regular poin u : Issue union( u,v ) for a lower neighbor v –A saddle u : let v and w be nodes from u ’s two connected pieces in its lower link Issue: find( v ), find( w ), union( u,v ), union( u,w ) lower link

39. Experiment 1: Random Union-Find

40. Experiment 2: Topological Persistence on Terrain Data