# Yili Gong, Marlon E. Pierce and Geoffery C. Fox Community Grids Lab, Indiana University.

## Presentation on theme: "Yili Gong, Marlon E. Pierce and Geoffery C. Fox Community Grids Lab, Indiana University."— Presentation transcript:

Yili Gong, Marlon E. Pierce and Geoffery C. Fox Community Grids Lab, Indiana University

Workflow Matchmaking in Grids Target Execution Environment TeraGrid Target Problem Decide when a job in the workflow should run on which resource. Assumptions Once two jobs have no logic or data dependency, they can run simultaneously on a computing resource, if not exceeding any limit. Jobs can only run on some of the resources. Motivation It is better to group resource-critical jobs together than mapping them individually.

System Architecture

The Resource-Critical Workflow Matchmaking Algorithm Ranking Sort the jobs in a non-ascending order of the rank values. Grouping Get the first ungrouped node in the sorted node list as the first node of a new group. Check each of its children: if its ancestor are grouped and its resource match ratio is below a certain threshold, add it into the group, and check its children further on. Matchmaking Matchmaking nodes in each group

Example 0 6 54321 87 9 1418221325 152620142117 262019 R0 R1R2 1.40.9 1.0 Data transfer rates between resources. Execution times for jobs on resources. DAG for workflow and sizes of data transferred between jobs. NodeR0R1R2 0171921 1222723 215 9 3 ∞ 89 4171420 5 ∞∞ 30 6171615 749 25 81622 ∞ 923 ∞ 19

Example 0 6 54321 87 9 19.0 18.3 22.7 13.2 28.5 15.226.819.114.220.029.0 27.621.221.7 R0 R1R2 1.40.9 1.0 Data transfer rates between resources. Execution times for jobs on resources. The weight of a node is the average of the job’s execution times on all possible resources. The weight of an edge is the average of the jobs’ communication times on all possible resource combinations. NodeR0R1R2 0171921 1222723 215 9 3 ∞ 89 4171420 5 ∞∞ 30 6171615 749 25 81622 ∞ 923 ∞ 19 14.2 24.0 13.0 8.5 17.0 30.0 16.041.019.0 21.0 Step 1: Ranking

Example 0 6 54321 87 9 168.2 R0 R1R2 1.40.9 1.0 Data transfer rates between resources. Execution times for jobs on resources. NodeR0R1R2 0171921 1222723 215 9 3 ∞ 89 4171420 5 ∞∞ 30 6171615 749 25 81622 ∞ 923 ∞ 19 Step 1: Ranking 0, 1, 5, 4, 3, 2, 7, 6, 8, 9 122.593.8 99.9114.5 120.8 64.683.261.7 21.0 Using the same upward rank computing approach as in HEFT. Sort the jobs in a non-ascending order of the rank values.

Example 0 6 54321 87 9 R0 R1R2 1.40.9 1.0 Data transfer rates between resources. Execution times for jobs on resources. NodeR0R1R2 0171921 1222723 215 9 3 ∞ 89 4171420 530 ∞∞ 6171615 749 25 8 ∞ 2216 923 ∞ 19 Groups: 0, 3, 5 1 4 2, 8 7 6, 9 Step 1: Ranking 0, 1, 5, 4, 3, 2, 7, 6, 8, 9 Step 2: Grouping Match Ratio: the ratio of the number of resources the job can run on and the total resource number. Match Ratio Threshold α = 0.8 1 11 1 11 0.67 0.33 0.67 0.33

Example 0 6 54321 87 9 R0 R1R2 1.40.9 1.0 Data transfer rates between resources. Execution times for jobs on resources. NodeR0R1R2 0171921 1222723 215 9 3 ∞ 89 4171420 530 ∞∞ 6171615 749 25 8 ∞ 2216 923 ∞ 19 Step 3: Matchmaking a group 0, 3, 5

Experimental Evaluation -- Setting DAG Generator Parameter Sweep Applications Heterogeneity Model Match Ratio Communication Bandwidth Communication-to-Computation-Ratio Match Ratio Threshold (MRT)

Experimental Evaluation -- Metrics Compare our resource critical algorithm with the minimum EFT algorithm. Difference ratio of NSL Normalized Schedule Length (NSL) The ratio of the real makespan divided by a fixed cost of the critical path. Average Improvement Ratio The average of difference ratios of all (200) cases in a certain setting.

Results -- Influence of MRT Branch Number = 4, Depth = 8 The resource-critical algorithm performs worse than the min EFT algorithm. The resource-critical algorithm performs Better than the min EFT algorithm.

Results -- Influence of CCR Branch Number = 4, Depth = 8 The average improvement ratio increases from 23% to 43% as CCR varies from 0.1 to 1. This shows that the resource-critical algorithm works better when communication cost plays a bigger role.

Results -- Influence of the Shape of DAGs CCR = 1.0, MRT = 0.5 Depth = 24Branch Number = 4 The branch number has little influence on the performance of the resource-critical algorithm while depth does.

Contact: Yili Gong: gongy@indiana.edu Website: http://grids.ucs.indiana.edu/ptl iupages/publications/ Questions?

Download ppt "Yili Gong, Marlon E. Pierce and Geoffery C. Fox Community Grids Lab, Indiana University."

Similar presentations