1. Automatic Thread Extraction with Decoupled Software Pipelining
Ottoni, Rangan, Stoler and August [2005]

2. DSWP Long-running Concurrently executing Non-speculative
Truly decoupled
Reduce inter-core communication and per-core resources

3. Hardware Additions Software Additions Other techniques
Two new instructions (produce and consume)
Software Additions
DSWP is added in the back-end of existing compilers
Other techniques
Either too limiting in implementation (for only a few cases of parallelization problems)
Speculative methods => hardware support

4. Comparison with DOACROSS

5. DOACROSS: Communications cost between cores
DSWP : Critical path remains in core 0, no communication latency because X will already have been executed in core 1

6. Key Concepts
Flow of data between cores is acyclic : instructions in each recurrence are executed in the same core
Decoupling of acyclic flows with the help of inter-core queues for communication
Different recurrences assigned to different cores
Better overall utilization of cores
Better execution speed
Limited number of restrictions (in comparison with DOACROSS and other techniques)

7. Communication and Synchronization
“Produce” instruction to send data to other core
“Consume” instruction to obtain data from other core
Produce and consume are matched in order
Data are stored in software implemented inter-core queues
Synchronization blocks only when enqueuing to a full queue or dequeuing from empty queue to avoid overhead
Queue latency measured of no importance

8. DSWP Algorithm

9. Transformed Code Partition of instructions to multiple threads
Replication of the necessary ones
Insertion of produce and consume to deal with dependencies
Dependences go only in one direction, from the producer to the consumer thread
Main thread produces loop live-in values for the other thread and consumes live-out values after consumer loop termination

10. Transforming the original code

11. Identify (data, control) dependencies and produce necessary arcs
SCCs : sets of instructions that have “reverse” dependences between them (recurrences)
All instructions in an SCC are executed in the same thread
Creation of SCC sequences (Pn)

12. Rules for Grouping SCCs in Pn parttitions
1<=n<=t where t max simultaneous threads
Each vertex belongs to exactly one partition in P
Each dependence arc goes from Pi to Pj where i<j => each partition Pi can be executed separately thus forming a pipeline

13. Load Balancing (TPP) NP-complete
Crucial for good performance (thread pipeline is limited by the longest average latency)
Solution: heuristic method
Count each SCC latency E
Add consecutive latencies until Ei -> n:maximum threads
Prefered partitions with minimum outgoing dependences in order to reduce communication and synchronization
Total estimation to determine if it is profitable to create multiple threads

14. Splitting the code
Computation of Basic Blocks (BB) through adding all instructions that belong to Pi
Addition of produce and consume for data and control (branch flags) dependence
Creation of BBs
Fix branch targets and replicate instructions where necessary to respect control flow

15. Extra Control Dependencies
Loop-iteration : to use the correct queues across iterations (different queues can be used in each iteration)
Two-loop graph
Extract classic dependences
Coalesce final graph

17. Conditional control dependence: to communicate the condition under which a dependence may occur or a live-out value can origin from
Insertion of an extra dependence arc for the first
Do not ignore output dependences for the second

19. Measurements Not sensitive to communication latency
Fairly insensitive to queue size
Can be used along with current ILP techniques for better results

21. IMPROVING DATA LOCALITY WITH LOOP TRANSFORMATIONS
McKinley, Carr and Tseng [1996]

22. Main Focus Processor speed increases faster than memory speed
Memory hierarchy with fast but small memory levels (cache)
Goal of model: To enhance spatial and temporal locality of programs through estimating reuse of cache lines and transforming code for desirable loop organization

23. Compiler Optimization
Improve order of memory accesses to exploit all levels of the memory hierarchy through loop permutation, fusion, distribution and reversal
Machine independent
Only knowledge of cache line size

24. Data Dependence
δ = {δ1...δκ} hybrid distance/direction vector, represents dependence between two array references from the outermost to the innermost loop enclosing the references
Direction: outer to inner loop positive
Distance: the minimum difference of iteration numbers of the pair of references when dependence occurs

25. Measure of Locality Number of cache lines a loop nest accesses
We assume there will be no conflict or capacity cache misses in one iteration of the innermost loop
Algorithms RefGroup, RefCost, LoopCost to determine the total number of cache lines accessed when a candidate loop l is placed innermost

26. RefGroup Calculates group-reuse
Two references are in the same group if they exhibit spatial/temporal locality
Ref1 and Ref2 are in the same group if
(temporal) there is Ref1δRef2 and
δ is a loop independent dependence
δ l is a small constant d (|d|<=2) and all other entries are zero
(spatial) They refer to the same array and the first subscript differs by at most d’ where d’ is less than or equal to the cache line size in terms of array elements
A reference can only be in one group

27. RefGroup Example

28. RefCost
Calculates locality for a loop l, number of cache lines l uses:
1 for loop invariant references
Trip/(cls/stride) for consecutive references where trip= (ub-lb+step)/step and stride is the step size of l multiplied by the subscript coeficient of the loop index variable
Trip for non-consecutive references

29. LoopCost
Calculates total number of cache lines accessed by all references when l is innermost loop
Sums RefCost for all reference groups then multiplies the result by the trip counts of all remaining loops

31. Loop Transformations Permutation: least cost memory model
Most reuse means lower LoopCost so we want the most reused in the innermost loop
Rank the loops using LoopCost from outermost to innermost so that
LoopCost(li-1)>= LoopCost(l)
Permute the distance/direction vector to see of the permutation of the best loop as innermost is legal
If not legal, second best is tried

32. Legal Permutations
Place a loop l at position k+1 for a single dependence
Legal if direction vector entry at position l is positive or zero
Legal if entry at l is negative and {p1,…,pk} positive
Illegal if entry at l is negative and {p1,…,pk} zero

33. Permutation Complexity
Tests the legality of n(n-1) loop permutations (worst case)
Most expensive part is evaluating the locality of the loop (LoopCost) so actually is O(n) in the number of LoopCost invocations where n is the number of loops in the nest

34. Loop Reversal
Reverses the order in which the iterations of a loop nest execute
Does not improve by itself, is only an enhancer for permutation
If Permute cannot legally position a loop in the desired position, it tests if reversal is legal and if it enables the desired permutation

35. Loop Fusion Takes multiple loop nests and combines them into one
Legal only if no data dependences are reversed
Improves locality directly by moving accesses to the same cache line to the same loop iteration
Loops must have same number of iterations
Compute RefGroup and LoopCost for both fused and non-fused code and compare them
Memory order may differ
May enable permutation

36. Fusion

37. Fusion Complexity
Loop nests may be reordered due to fusion so we may need to calculate LoopCost for every pair of loop nests (O(m2)), m number of candidate nests for fusion
For only adjacent loops O(m)

38. Loop Distribution
Separates independent statements in a single loop into multiple loops with identical headers
Recurrences must be placed in the same loop (DSWP-like)
Finest granularity and test if it enables permutation
Invoked only if memory order cannot be achieved on a nest and if not all of the inner nests can be fused
Implemented only if combines with permutation to improve the actual LoopCost

39. Distribution Complexity
LoopCost is calculated for each individual partition
O(m) where m is the number of individual partitions created by distribution

40. Compound Transformation
Combines all the previously mentioned algorithms into one

41. Compound Algorithm

42. Complexity
O(nm2) in the number of LoopCost invocations where n is the number of loops and m the number of adjacent loop nests
m2 is due to fusion
Fusion and distribution are only occasionally applied

43. Effect of Permutation

44. Speed-up