Resource Allocation and Scheduling for Workflows Gurmeet Singh, Carl Kesselman, Ewa Deelman

Problem Given a workflow and a Grid, how to allocate resources and schedule the workflow in order to optimize performance. –Which resources should be provisioned –When they should be provisioned –How much capacity should be provisioned –Scheduling on provisioned resources

Previous Work No allocation in case of time sharing systems. Combined allocation and scheduling in case of queuing based systems. Scheduling of data is coupled with scheduling of computations. Operations Research (inventory control, MRP)

Formal Model An allocation plan consisting of allocation requests and cost AP = {ar 1, ar 2,, ar n } ar = C A (AP) = f(ar 1, ar 2, ar n ) e.g. utilization, wait time A policy, S, for scheduling partial workflows and associated cost, C SCH costs can be makespan, lateness, reliability etc. The goal is to minimize the total cost of allocation and scheduling. C T = g(C A (AP), C SCH )

Relation to previous work The AP defines the Virtual Grid. Can be used to explain previous scheduling approaches based on the appropriate cost formulations. –Dedicated resources imply constant allocation cost –Others imply the cost of allocation is the wait time associated with it.

Approaches to Allocation Deterministic (using reservations) Stochastic (using predictions) A combination of both The space of possible allocation plans is likely to be exponential even with reservations. Search based heuristics might be useful.

Glidein TG/OSG Condor-G Execution stack for SCEC, Montage workflows Grid Pegasus DAGMan Provisioned Resource AP S

Preliminary Work Java based Grid Simulator A parametric task graph generator 4 Grid sites with background load Backfill based FCFS scheduling policy for Grid resources. Resources can be queried for earliest start time of a request.

Preliminary Work Preliminary results show good results with very simple allocation strategy and small workflows.

Current Research Modeling resource availability in Grids Heuristics for creating an allocation plan –Greedy approach –Search based metaheuristics Experiments to evaluate the performance. Implementation to be integrated with current planning and execution systems such as Pegasus.