1. Flexible and Efficient Control of Data Transfers for Loosely Coupled Components Joe Shang-Chieh Wu http://meou.us Department of Computer Science University of Maryland, USA

2. What & How Obtain more accurate results by coupling existing (parallel) physical simulation components Different time and space scales for data produced in shared or overlapped regions Runtime decisions for which time-stamped data objects should be exchanged Performance might be a concern

3. Roadmap Approximate Match [Grid 2004] Collective Buffering [IPDPS 2007] Distributed App Match + Eager Transfer [under submission] Conclusion

4. Matching is OUTSIDE components Separate matching (coupling) information from the participating components  Maintainability – Components can be developed/upgraded individually  Flexibility – Change participants/components easily  Functionality – Support variable-sized time interval numerical algorithms or visualizations

5. Distributed Array Transfer Library Basic Operation Runtime-based Approximate Match Library Importer component Request Array for T = 2.5 Matched Array for T = 3 Approximate Match Exporter component T=4 T=3 T=2 Exported Distributed Array Imported Distributed Array Arrays are distributed among multiple processes T=1

6. Separate codes from matching define region R1 define region R4 define region R5... Do t = 1, N, Step0... // computation jobs export(R1,t) export(R4,t) export(R5,t) EndDo define region R2... Do t = 1, M, Step1 import(R2,t)... // computation jobs EndDo Importer App1 Exporter App0 Configuration file # App0 cluster0 /bin/App0 2... App1 cluster1 /bin/App1 4... App2 cluster2 /bin/App2 16... App4 cluster4 /bin/App4 4 # App0.R1 App4.R0 REGL 0.05 App0.R1 App2.R0 REG 0.1 App0.R4 App1.R2 REGU 0.5 # Connection-Wise Approximate Match Policy Precision Find t’ in App0, s.t. (a) t <= t’ <= t + 0.5 (b) minimize t’ – t Sourc e Sink

7. Execution time is composed of  Computation time (T comp )  Buffering time (T buf )  Matched data transfer time (T tran ) T buf matters when exporter components (data sources) run more slowly T tran matters when import components (data sinks) run more slowly Dissection of Execution Time

8. Collective Buffering (when exporters run more slowly) Fastest export process sends runtime match results to slower processes in the same program Unnecessary memory copies can be avoided in slower processes Optimal State: only required exported data are buffered

9. Collective Buffering Result Data Exporting Time for the Slowest Process Copy All Copy Some Only Copy Required Optimal State

10. Eager Transfer + Distributed Match (when importer runs more slowly) Bandwidth and Latency both contribute matched data transfer time Eager transfer, transferring predicted data in advance, solves bandwidth issue Distributed approximate match, running on both exporter and importer, solves latency issue

11. Original ET Only ET+DM

12. Conclusion Runtime-based approximate match is a solution to couple different time scale components Performance can be improved –When exporter runs more slowly, avoid unnecessary memory copies –When importer runs more slowly, transfer predicted data and meta-data in advance

13. The End

14. Questions ? (http://meou.us)

15. Distributed Array Transfer Library Basic Operation Runtime-based Approximate Match Library Importer component Request Array for T = 2.5 Matched Array for T = 3 Approximate Match Exporter component T=4 T=3 T=2 Exported Distributed Array Imported Distributed Array Arrays are distributed among multiple processes T=1

16. On-Demand Approach Import Component Makes Request Perform Approx Match on Export Component, and then Transfer Matched Data Need Data Transfer Time (T 3 – T 2 ) and 2 one-way delays (T 2 – T 1 )

17. Eager Transfer Only Get permission to push predicted data Transfer predicted data in advance Import component makes request Perform approx match on export component Need 2 one-way delays ( T 16 – T 15 )

18. Eager Transfer With Distributed Match … Transfer predicted data + meta-data in advance Import component makes request becomes local operations Local operation time T 26 – T 25 is needed, independent to one- way delay

19. All Together

20. Supported matching policies = LUBminimum f(x) with f(x) ≥ x GLBmaximum f(x) with f(x) ≤ x REGf(x) minimizes |f(x)-x| with |f(x)-x| ≤ p REGUf(x) minimizes f(x)-x with 0 ≤ f(x)-x ≤ p REGLf(x) minimizes x-f(x) with 0 ≤ x-f(x) ≤ p FASTRany f(x) with |f(x)-x| ≤ p FASTUany f(x) with 0 ≤ f(x)-x ≤ p FASTLany f(x) with 0 ≤ x-f(x) ≤ p