Presentation on theme: "Scheduling and Performance Issues for Programming using OpenMP"— Presentation transcript:
1Scheduling and Performance Issues for Programming using OpenMP Alan RealISS, University of Leeds
2Overview Scheduling for worksharing constructs Performance. STATIC. GUIDED.RUNTIME.Performance.
3SCHEDULE clauseGives control over which loop iterations are executed by which thread.Syntax:Fortran: SCHEDULE(kind[,chunksize])C/C++: schedule(kind[,chunksize])kind is one of STATIC, DYNAMIC, GUIDED or RUNTIMEchunksize is an integer expression with positive value; e.g.: !$OMP DO SCHEDULE(DYNAMIC,4)Default is implementation dependent.
4STATIC schedule With no chunksize specified If chunksize is specified Iteration space is divided into (approximately) equal chunks, and one chunk is assigned to each thread (block schedule)If chunksize is specifiedIteration space is divided into chunks, each of chunksize iterations. The chunks are assigned cyclically to each thread (block cyclic schedule)
5STATIC schematic SCHEDULE(STATIC) SCHEDULE(STATIC,4) T0 T1 T2 T3 46 1 Block-cyclic schedule is good for triangular matrices to avoid load imbalance; e.g: do i=1,n do j=i,n block end do end do
6DYNAMIC scheduleDivides the iteration space up into chunks of size chunksize and assigns them to threads on a first-come first-served basis.i.e. as a thread finishes a chunk, it is assigned the next chunk in the list.When no chunksize is specified it defaults to 1.
7GUIDED scheduleSimilar to DYNAMIC, but the chunks start off large and get smaller exponentially.The size of the next chunk is (roughly) the number of remaining iterations divided by the number of threads.The chunksize specifies the minimum size of the chunksWhen no chunksize is specifies it defaults to 1.
8DYNAMIC and GUIDED SCHEDULE(DYNAMIC,3) SCHEDULE(GUIDED,3) T0 T1 T2 T3 46SCHEDULE(DYNAMIC,3)T3T0T1T2T3T3T2T3T1T2T0461SCHEDULE(GUIDED,3)
9RUNTIME scheduleAllows the choice of schedule to be deferred until runtimeSet by environment variable OMP_SCHEDULE: % setenv OMP_SCHEDULE “guided,4”There is no chunksize associated with the RUNTIME schedule
10Choosing a schedule STATIC is best for balanced loops –least overhead. STATIC,n good for loops with mild or smooth load imbalance – but can introduce “false sharing” (see later).DYNAMIC useful if iterations have widely varying loads, but ruins data locality (can get cache hit).GUIDED often less expensive than DYNAMIC, but beware of loops where first iterations are the most expensive!Use RUNTIME for convenient experimentation
11PerformanceEasy to write OpenMP but hard to write an efficient program!5 main causes of poor performance:Sequential codeCommunicationLoad imbalanceSynchronisationCompiler (non-)optimisation.
12Sequential codeAmdahl’s law: Parallel code will never run faster than the parts which can only be executed in serial.Limits performance.Need to find ways of parallelising it!In OpenMP, all code outside of parallel regions and inside MASTER, SINGLE and CRITICAL directives is sequential.This code should be as small as possible.
13CommunicationOn Shared memory machines, communication is “disguised” as increased memory access costs.It takes longer to access data in main memory or another processor’s cache than it does from local cache.Memory accesses are expensive!~100 cycles for a main memory access compared to 1-3 cycles for a flop.Unlike message passing, communication is spread throughout the program.Much harder to analyse and monitor.
14Caches and coherencyShared memory programming assumes that a shared variable has a unique value at a given time.Caching means that multiple copies of a memory location may exist in the hardware.To avoid two processors caching different values of the same memory location, caches must be kept coherent.Coherency operations are usually performed on the cache lines in the level of cache closest to memory(level 2 cache on most systems)Maxima has:64kb Level 1 cache, 32byte cache lines (4 d.p. words).8Mb Level 2 cache, 512byte cache lines (64 d.p. words).
15Memory hierarchy P P P P L1C L1C L1C L1C L2C L2C L2C L2C Interconnect
16Coherency Highly simplified view: Each cache line can exist in one of 3 states:Exclusive: the only valid copy in any cache.Read-only: a valid copy but other caches may contain it.Invalid: out of date and cannot be used.A READ on an invalid or absent cache line will be cached as read-only.A READ on an exclusive cache line will be changed to read only.A WRITE on a line not in an exclusive state will cause all other copies to be marked invalid and the written line to be marked exclusive.
17Coherency example Exclusive Read-Only Invalid W P R P W P P R L1C L1C InterconnectMemoryCoherency example
18False sharing Cache lines consist of several words of data. What happens when two processors are both writing to different words on the same cache line?Each write will invalidate the other processors copy.Lots of remote memory accesses.Symptoms:Poor speedupHigh, non-deterministic numbers of cache misses.Mild, non-deterministic, unexpected load imbalance.
19False sharing example 1 integer count(maxthreads) !$OMP PARALLEL : count(myid)=count(myid)+1Each thread writes a different element of the same arrayWords with consecutive addresses are more than likely in the same cache line.Solution:Pad array by dimension of cache line:parameter (linesize=16)integer count(linesize,maxthreads)count(1,myid) = count(1,myid) + 1
20False sharing example 2 Small chunk sizes: !OMP DO SCHEDULE(STATIC,1) do j = 1,ndo i=1,jb(j)=b(j)+a(i,j)end doWill induce false sharing on b.
21Data affinity Data is cached on the processors which access it. Must reuse cached data as much as possible.Write code with good data affinity:Ensure the same thread accesses the same subset of program data as much as possible.Try to make these subsets large, contiguous chunks of data.Will avoid false sharing and other problems.
22Data affinity example i 1 2 3 4 j i !$OMP DO PRIVATE(i) do j=1,n do i=1,na(i,j)= i+jend do!$OMP DO SCHEDULE(STATIC,16)!$OMP& PRIVATE(i)do i=1,jb(j)=b(j)+a(i,j)1st 16 j’s OK – rest are cache misses!ji16ijj
23Distributed shared memory systems Location of data in main memory is important.OpenMP has no support for controlling this.Normally data is placed on the first processor which accesses it.Data placement can be controlled indirectly by parallelisation of data initialisation.Not an issue on Maxima as CPUs have reasonably uniform memory access within each machine.
24Load imbalanceLoad imbalance can arise from both communication and computation.Worth experimenting with different scheduling optionsCan use SCHEDULE(RUNTIME).If none are appropriate, may be best to do your own scheduling!
25Synchronisation Barriers can be very expensive Avoid barriers via: Typically 1000s cycles to synchronise 32 processors).Avoid barriers via:Careful use of the NOWAIT clauseParallelise at the outermost level possible.May require re-ordering of loops +/ indicies.Consider using point-to-point synchronisation for nearest-neighbour type problems.Choice of CRITICAL / ATOMIC / lock routines may impact performance.
26Compiler (non-)optimisation Sometimes the addition of parallel directives can inhibit the compiler from performing sequential optimisations.Symptoms:1-thread parallel code has longer execution and higher instruction count than sequential code.Can sometimes be cured by making shared data private, or local to a routine.
27Performance tuningMy code is giving me poor speedup. I don’t know why. What do I do now?A:Say “this machine/language is a heap of junk”Give up and go back to your workstation/PCB:Try to classify and localise the sources of overhead.What type of problem is it and where in the code does it occurUse any available tools that can help you:Add timers, use profilers – Sun Performance analyser.Fix problems that are responsible for large overheads first.IterateGood Luck!