IBM Thomas J. Watson Research Center © 2004 IBM Corporation Analysis of Parallel-Server Systems with Dynamic Affinity Scheduling and Load Balancing Mark S. Squillante Mathematical Sciences Department April 18, 2004

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 2 Problem Motivation Cache Affinity [SquillanteLazowska90]

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 3 Problem Motivation Key Points of Fundamental Tradeoff  Customers can be served on any server of a parallel-server queueing system  Each customer is served most efficiently on one the servers  Load imbalance among queues occurs due to stochastic properties of system Cache Affinity [SquillanteLazowska90]

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 4 General Overview  Optimal Dynamic Threshold Scheduling Policy –Fluid limits –Diffusion limits  Analysis of Dynamic Threshold Scheduling –Consider generalized threshold scheduling policy –Matrix-analytic analysis and fix-point solution, asymptotically exact –Numerical experiments –Optimal settings of dynamic scheduling policy thresholds  Stochastic Derivative-Free Optimization

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 5 Scheduling Policy ()() ()() 1 P

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 6 Scheduling Model ()() ()() 1 P ()()

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 7 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … … State Vector ( i, j, v ): i: total number of customers waiting or receiving service at the processor of interest j: number of customers in the process of being migrated to the processor of interest v: K-bit vector denoting customer type of up to the first K customers at the processor

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 8 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … …

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 9 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … …

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 10 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … …

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 11 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … …

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 12 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … …

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 13 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … … Final Solution Obtained via Fix-Point Iteration 1.Initialize (p s, s, p r,  r ) 2.Compute stationary probability vector in terms of (p s, s, p r,  r ) 3.Compute new values of (p s, s, p r,  r ) in terms of stationary vector 4.Goto 2 until differences between iteration values are arbitrarily small

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 14 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … …

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 15 Mathematical Analysis T s u,1,0 T s u,0,0 T s u +1,0,1 T s u +1,0,0 rr rr (1-p s ) rr rr   s 0,1,0 0,0,0 1,0,1 1,0,0   (1-p r )  (1-p r )  prpr  prpr … … … T r u,1,0 T r u,0,0 T r u +1,0,1 T r u +1,0,0  rr   rr … … … … … …

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 16 Mathematical Analysis General K, Pure Sender-Initiated Policy

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 17 Mathematical Analysis General K

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 18 Mathematical Analysis General K

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 19 Numerical Results

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 20 Numerical Results

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 21 Numerical Results

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 22 Numerical Results

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 23 Numerical Results

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 24 Stochastic Derivative-Free Optimization  Internal Model  External Model  Trust Region

Title/subtitle/confidentiality line: 10pt Arial Regular, white Maximum length: 1 line Information separated by vertical strokes, with two spaces on either side Copyright: 10pt Arial Regular, white IBM Thomas J. Watson Research Center © 2004 IBM Corporation Optional slide number: 10pt Arial Bold, white Analysis of Dynamic Affinity Scheduling and Load Balancing | Mark S. Squillante 25 General Overview  Optimal Dynamic Threshold Scheduling Policy –Fluid limits –Diffusion limits  Analysis of Dynamic Threshold Scheduling –Consider generalized threshold scheduling policy –Matrix-analytic analysis and fix-point solution, asymptotically exact –Numerical experiments –Optimal settings of dynamic scheduling policy thresholds  Stochastic Derivative-Free Optimization