Presentation is loading. Please wait.

Presentation is loading. Please wait.

Matching Memory Access Patterns and Data Placement for NUMA Systems Zoltán Majó Thomas R. Gross Computer Science Department ETH Zurich, Switzerland.

Similar presentations


Presentation on theme: "Matching Memory Access Patterns and Data Placement for NUMA Systems Zoltán Majó Thomas R. Gross Computer Science Department ETH Zurich, Switzerland."— Presentation transcript:

1 Matching Memory Access Patterns and Data Placement for NUMA Systems Zoltán Majó Thomas R. Gross Computer Science Department ETH Zurich, Switzerland

2 Non-uniform memory architecture 2 Processor 1 Core 4Core 5 Core 6Core 7 IC MC DRAM Processor 0 Core 0Core 1 Core 2Core 3 MC IC DRAM

3 Non-uniform memory architecture Local memory accesses bandwidth: 10.1 GB/s latency: 190 cycles 3 Processor 1 Core 4Core 5 Core 6Core 7 IC MC DRAM Processor 0 Core 0Core 1 Core 2Core 3 MC IC DRAM T Data All data based on experimental evaluation of Intel Xeon 5500 (Hackenberg [MICRO ’09], Molka [PACT ‘09])

4 Non-uniform memory architecture Local memory accesses bandwidth: 10.1 GB/s latency: 190 cycles Remote memory accesses bandwidth: 6.3 GB/s latency: 310 cycles 4 Processor 1 Core 4Core 5 Core 6Core 7 IC MC DRAM Processor 0 Core 0Core 1 Core 2Core 3 MC IC DRAM T Data Key to good performance: data locality All data based on experimental evaluation of Intel Xeon 5500 (Hackenberg [MICRO ’09], Molka [PACT ‘09])

5 Data locality in multithreaded programs 5 Remote memory references / total memory references [%]

6 Data locality in multithreaded programs 6 Remote memory references / total memory references [%]

7 Outline  Automatic page placement  Memory access patterns of matrix-based computations  Matching memory access patterns and data placement  Evaluation  Conclusions 7

8 Automatic page placement  Current OS support for NUMA: first-touch page placement  Often high number of remote accesses  Data address profiling  Profile-based page-placement  Supported in hardware on many architectures 8

9 Profile-based page placement Based on the work of Marathe et al. [JPDC 2010, PPoPP 2006] 9 Processor 1 DRAM Processor 0 DRAM T0 Profile P0: accessed 1000 times by P1:accessed3000 times by T0 T1 P1 P0

10 Automatic page placement  Compare: first-touch and profile-based page placement  Machine: 2-processor 8-core Intel Xeon E5520  Subset of NAS PB: programs with high fraction of remote accesses  8 threads with fixed thread-to-core mapping 10

11 Profile-based page placement 11

12 Profile-based page placement 12

13 Inter-processor data sharing 13 Processor 1 DRAM Processor 0 DRAM T0 Profile P0: accessed 1000 times by P1 :accessed 3000 times by T0 T1 P0P1 P2: accessed 4000 times by accessed 5000 times by T0 T1 P2 P2: inter-processor shared

14 Inter-processor data sharing 14 Processor 1 DRAM Processor 0 DRAM T0 Profile P0: accessed 1000 times by P1 :accessed 3000 times by T0 T1 P0P1 P2: accessed 4000 times by accessed 5000 times by T0 T1 P2 P2: inter-processor shared

15 Inter-processor data sharing 15 Shared heap / total heap [%]

16 Inter-processor data sharing 16 Shared heap / total heap [%]

17 Inter-processor data sharing 17 Shared heap / total heap [%] Performance improvement [%]

18 Inter-processor data sharing 18 Shared heap / total heap [%] Performance improvement [%]

19 Automatic page placement  Profile-based page placement often ineffective  Reason: inter-processor data sharing  Inter-processor data sharing is a program property  Detailed look: program memory access patterns  Loop-parallel programs with OpenMP-like parallelization  Matrix processing  NAS BT 19

20 Matrix processing Process m sequentially m[NX][NY] 20 NX NY for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_20.jpg", "name": "Matrix processing Process m sequentially m[NX][NY] 20 NX NY for (i=0; i

21 for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_21.jpg", "name": "for (i=0; i

22 Thread scheduling Remember: fixed thread-to-core mapping 22 Processor 1 DRAM Processor 0 DRAM T0 T1 T2 T3 T4 T5 T6 T7

23 #pragma omp parallel for for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_23.jpg", "name": "#pragma omp parallel for for (i=0; i

24 #pragma omp parallel for for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_24.jpg", "name": "#pragma omp parallel for for (i=0; i

25 NX NY for (t=0; t { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_25.jpg", "name": "NX NY for (t=0; t

26 NX NY for (t=0; t { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_26.jpg", "name": "NX NY for (t=0; t

27 Solution? 1.Adjust data placement High overhead of runtime data migration cancels benefit 2.Adjust iteration scheduling Limited by data dependences 3.Adjust data placement and iteration scheduling together 27

28 API  Library for data placement  Set of common data distributions  Affinity-aware loop iteration scheduling  Extension to GCC OpenMP implementation  Example use case: NAS BT 28

29 Use-case: NAS BT  Remember: BT has two incompatible access patterns  Repeated x-wise and y-wise access to the same data  Idea: data placement to accommodate both access patterns 29 NX NY Allocated at Processor 0 Allocated at Processor 1 Blocked-exclusive data placement

30 distr_t *distr; distr = block_exclusive_distr(m, sizeof(m), sizeof(m[0]/2)); distribute_to(distr); Use-case: NAS BT for (t=0; t { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_30.jpg", "name": "distr_t *distr; distr = block_exclusive_distr(m, sizeof(m), sizeof(m[0]/2)); distribute_to(distr); Use-case: NAS BT for (t=0; t

31 Use-case: NAS BT 31 distr_t *distr; distr = block_exclusive_distr(m, sizeof(m), sizeof(m[0]/2)); distribute_to(distr); #pragma omp parallel for for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_31.jpg", "name": "Use-case: NAS BT 31 distr_t *distr; distr = block_exclusive_distr(m, sizeof(m), sizeof(m[0]/2)); distribute_to(distr); #pragma omp parallel for for (i=0; i

32 x_wise() Matrix processed in two steps 32 Step 1: left half all accesses local Step 2: right half all accesses local Allocated at Processor 1 Allocated at Processor 0 NY / 2 NX Allocated at Processor 0 Allocated at Processor 1 NY / 2 T0 T1 T2 T3 T4 T5 T6 T7

33 #pragma omp parallel for for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_33.jpg", "name": "#pragma omp parallel for for (i=0; i

34 #pragma omp parallel for for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_34.jpg", "name": "#pragma omp parallel for for (i=0; i

35 for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_35.jpg", "name": "for (i=0; i

36 for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_36.jpg", "name": "for (i=0; i

37 m[0.. NX/8 - 1][*] m[NX/8.. 2*NX/8 - 1][*] m[2*NX/8.. 3*NX/8 - 1][*] m[3*NX/8.. 4*NX/8 - 1][*] m[4*NX/8.. 5*NX/8 - 1][*] m[5*NX/8.. 6*NX/8 - 1][*] m[6*NX/8.. 7*NX/8 - 1][*] m[7*NX/8.. NX - 1][*] #pragma omp parallel for schedule(static) for (i=0; i { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3395267/slides/slide_37.jpg", "name": "m[0.. NX/8 - 1][*] m[NX/8.. 2*NX/8 - 1][*] m[2*NX/8..", "description": "3*NX/8 - 1][*] m[3*NX/8.. 4*NX/8 - 1][*] m[4*NX/8.. 5*NX/8 - 1][*] m[5*NX/8.. 6*NX/8 - 1][*] m[6*NX/8.. 7*NX/8 - 1][*] m[7*NX/8.. NX - 1][*] #pragma omp parallel for schedule(static) for (i=0; i

38 y_wise() Matrix processed in two steps 38 Allocated at Processor 0 Allocated at Processor 1 NX / 2 Allocated at Processor 0 Allocated at Processor 1 NY NX / 2 T4T5T6T7 Step 1: upper half all accesses local Step 2: lower half all accesses local T0T1T2T3

39 Outline  Profile-based page placement  Memory access patterns  Matching data distribution and iteration scheduling  Evaluation  Conclusions 39

40 Evaluation 40 Performance improvement over first-touch [%]

41 Evaluation 41 Performance improvement over first-touch [%]

42 Evaluation 42 Performance improvement over first-touch [%]

43 Scalability Machine: 4-processor 32-core Intel Xeon E7-4830 43 Performance improvement over first-touch [%]

44 Scalability Machine: 4-processor 32-core Intel Xeon E7-4830 44 Performance improvement over first-touch [%]

45 Conclusions  Automatic data placement (still) limited  Alternating memory access patterns  Inter-processor data sharing  Match memory access patterns and data placement  Simple API: practical solution that works today  Ample opportunities for further improvement 45

46 Thank you for your attention! 46


Download ppt "Matching Memory Access Patterns and Data Placement for NUMA Systems Zoltán Majó Thomas R. Gross Computer Science Department ETH Zurich, Switzerland."

Similar presentations


Ads by Google