1. I/O-Algorithms Lars Arge Spring 2012 April 17, 2012

2. Lars Arge I/O-algorithms 2 I/O-Model Parameters N = # elements in problem instance B = # elements that fits in disk block M = # elements that fits in main memory T = # output size in searching problem We often assume that M>B 2 I/O: Movement of block between memory and disk D P M Block I/O

3. Lars Arge I/O-Algorithms 3 Fundamental Bounds Internal External Scanning: N Sorting: N log N Permuting Searching: Note: –Linear I/O: O(N/B) –Permuting not linear –Permuting and sorting bounds are equal in all practical cases –B factor VERY important: –Cannot sort optimally with search tree

4. Scalability Problems: Block Access Matters Example: Traversing linked list (List ranking) –Array size N = 10 elements –Disk block size B = 2 elements –Main memory size M = 4 elements (2 blocks) Large difference between N and N/B large since block size is large –Example: N = 256 x 10 6, B = 8000, 1ms disk access time N I/Os take 256 x 10 3 sec = 4266 min = 71 hr N/B I/Os take 256/8 sec = 32 sec Algorithm 2: N/B=5 I/OsAlgorithm 1: N=10 I/Os 1526 7 34108912 9 8 54763 4Lars Arge I/O-Algorithms

5. Lars Arge I/O-algorithms 5 List Ranking Problem: –Given N-vertex linked list stored in array –Compute rank (number in list) of each vertex One of the simplest graph problem one can think of Straightforward O(N) internal algorithm –Also uses O(N) I/Os in external memory Much harder to get external algorithm 3459 6 82710 1526 7 34 89

6. Lars Arge I/O-algorithms 6 List Ranking We will solve more general problem: –Given N-vertex linked list with edge-weights stored in array –Compute sum of weights (rank) from start for each vertex List ranking: All edge weights one Note: Weight stored in array entry together with edge (next vertex) 1526 7 341089 1 11 11 1 11 1 1

7. Lars Arge I/O-algorithms 7 List Ranking Algorithm: 1.Find and mark independent set of vertices 2.Bridge-out independent set: Add new edges 3.Recursively rank resulting list 4.Bridge-in independent set: Compute rank of independent set Step 1, 2 and 4 in I/Os Independent set of size αN for 0 < α 1 I/Os 111 1 1 111 11 222 1 3468910 2 57

8. Lars Arge I/O-algorithms 8 List Ranking: Bridge-out/in Obtain information (edge or rang) of successor –Make copy of original list –Sort original list by successor id –Scan original and copy together to obtain successor information –Sort modified original list by id I/Os 11 2 3459 6 8 2710 2 3 4 95 8 6 7 3459 6 8 27 3489 6 27

9. Lars Arge I/O-algorithms 9 List Ranking: Independent Set Easy to design randomized algorithm: –Scan list and flip a coin for each vertex –Independent set is vertices with head and successor with tails Independent set of expected size N/4 Deterministic algorithm: –3-color vertices (no vertex same color as predecessor/successor) –Independent set is vertices with most popular color Independent set of size at least N/3 3-coloring I/O algorithm 3459 6 82710

10. Lars Arge I/O-algorithms 10 List Ranking: 3-coloring Algorithm: –Consider forward and backward lists (heads/tails in two lists) –Color forward lists (except tail) alternately red and blue –Color backward lists (except tail) alternately green and blue 3-coloring 3459 6 82710

11. Lars Arge I/O-algorithms 11 List Ranking: Forward List Coloring Identify heads and tails For each head, insert red element in priority-queue (priority=position) Repeatedly: –Extract minimal element from queue –Access and color corresponding element in list –Insert opposite color element corresponding to successor in queue Scan of list O(N) priority-queue operations I/Os ` 3459 6 82710

12. Lars Arge I/O-algorithms 12 Summary: List Ranking Simplest graph problem: Traverse linked list Very easy O(N) algorithm in internal memory Much more difficult external memory –Finding independent set via 3-coloring –Bridging vertices in/out Permuting bound best possible –Also true for other graph problems 3459 6 82710 1526 7 34 89

13. Lars Arge I/O-algorithms 13 Summary: List Ranking External list ranking algorithm similar to PRAM algorithm –Sometimes external algorithms by PRAM algorithm simulation Forward list coloring algorithm example of time forward processing –Use external priority-queue to send information forward in time to vertices to be processed later 3459 6 82710

14. Lars Arge I/O-algorithms 14 Algorithms on Trees TBD

15. Lars Arge I/O-algorithms 15 References External-Memory Graph Algorithms Y-J. Chiang, M. T. Goodrich, E.F. Grove, R. Tamassia. D. E. Vengroff, and J. S. Vitter. Proc. SODA'95 –Section 3-6 I/O-Efficient Graph Algorithms Norbert Zeh. Lecture notes –Section 2-4 Cache-Oblivious Priority Queue and Graph Algorithm Applications L. Arge, M. Bender, E. Demaine, B. Holland-Minkley and I. Munro. SICOMP, 36(6), 2007 –Section 3.1-3-2