1. Fall 2008Parallel Sorting1

2. Fall 2008Parallel Sorting2 Sort  Sorting operation is frequently used for database processing.  For example sorting may be requested by users through the use of Distinct, Order By and Group By clauses in SQL.  In general, parallel database sorting consists of multiple rounds of local sort, performed in each PN in parallel, followed by movement of tuples among PNs.  A dynamic load balancing approach has been proposed, recently.

3. Fall 2008Parallel Sorting3  Traditional Technologies  Parallel Merge-All Sort  Parallel Binary-Merge Sort  Parallel Partitioned Sort  Dynamic Load Balancing Approaches  Partitioned with Redistribution Sort (PRS)  Load Balancing Optimization Sort (LBOS)  Load Balancing Optimization Sort with Fragment Feature (LBOS-F)

4. Fall 2008Parallel Sorting4 Parallel Merge-All Sort

5. Fall 2008Parallel Sorting5 Parallel Binary-Merge Sort

6. Fall 2008Parallel Sorting6 Parallel Partitioned Sort

7. Fall 2008Parallel Sorting7 Partitioned with Redistribution Sort (PRS)  Partition Phase: each PN first scans its local tuples and distributes them as buckets according to range partitioning.  Optimization Phase: a coordinator PN is used to monitor the load for every PN. If every PN has the similar amount of data to process, Go to Sorting Phase. Otherwise, the imbalanced work load is detected, thus, a redistributed partitioning is needed.  Redistribution Phase: data redistribution.  Sorting Phase: local sort is carried out in each PN.

8. Fall 2008Parallel Sorting8 PRS

9. Fall 2008Parallel Sorting9 Load Balancing Optimization Sort (LBOS)  Phase 1. Bucket Sizes Counting : Each PN reads its local data and computes each bucket size according to range distribution. Thereafter, each PN transfers the bucket information to a coordinator.  Phase 2. Load Balancing Optimization : coordinator decides the distribution strategy for assigning each bucket to PNs according to their bucket IDs sequentially. Whether coordinator assigns one more next bucket (size B) to current PN i will be decided on: if (P i +B)-P avg < d, it will assign next bucket to PN i. Otherwise, the bucket will be assigned to next PN i+1. This phase only decides the distribution strategy without any data transferred.

10. Fall 2008Parallel Sorting10  Phase 3. Data Partitioning : Each PN transfers and receives data by using the distribution strategy decided in Phase 2.  Phase 4. Sorting : Each PN performs local sort.

11. Fall 2008Parallel Sorting11 LBOS

12. Fall 2008Parallel Sorting12 Effect of Bucket Skew PPS ： PRS → 37.1% PPS ： LOBS → 60.9% Bucket skew=1 25.23 ( 秒 ) 15.93 ( 秒 ) 9.87 ( 秒 )

13. Fall 2008Parallel Sorting13 Effect of No. of Processing Nodes 24.72 ( 秒 ) 19.32 ( 秒 ) 14.9 ( 秒 ) 4.41 ( 秒 ) 3.45 ( 秒 ) 1.77 ( 秒 ) PRS ： PPS → 21.8% LOBS ： PPS → 59.9% PRS ： PPS → 21.8% LOBS ： PPS → 39.7% PNs=16PNs=256

14. Fall 2008Parallel Sorting14 Load Balancing Optimization Sort with Fragment Feature (LBOS-F)  When the skew condition is severe, LBOS performs not very well.  LBSO-F is used to address this drawback and it also consists of four phases.  Its Phase 1, 3, and 4 are exactly the same as in LBOS except the Phase 2 in which LBOS-F can fragment oversize buckets and distribute them to different PNs.

15. Fall 2008Parallel Sorting15 Phase 2 for LBOS-F  Phase 2. Load Balancing Optimization with Fragment Feature : coordinator decides the distribution strategy for assigning each bucket to PNs according to their bucket IDs sequentially. Suppose that the next bucket (size B and P i +B>P avg ), with the interval of data values from rs to re ( rs < re ), is being considered for the assignment. The coordinator will fragment this bucket into two portions. Suppose that PN i already has total size P i of tuples which is d tuples less than P avg (P i +d=P avg ). The first portion which consists of those data values within the range of rs to rs+(re-rs)d/B of the bucket will be assigned to current PN i. The second portion which consists of the remaining data of the bucket will be assigned to next PN i+1. Again, this phase is without any data transferred.

16. Fall 2008Parallel Sorting16 Effect of Bucket Skew

17. Fall 2008Parallel Sorting17 Effect of Number of PNs