1. Complexity 20-1 Complexity Andrei Bulatov Parallel Arithmetic

2. Complexity 20-2 Arithmetic Operations A sequential algorithm can compute the sum of two integers, n and m, in O(max{log n, log m}) the product of two integers, n and m, in O(log n  log m) Our goal is to find parallel algorithms that compute the sum of two integers, n and m, in O(max{log log n, log log m}) the product of two integers, n and m, in O(log log n  log log m)

3. Complexity 20-3 Prefix Sums Instance: A sequence of positive integers Objective: Compute the prefix sums: Prefix Sum The straightforward algorithm requires n – 1 additions Unfortunately, it cannot be parallelized

4. Complexity 20-4 Parallel Prefix Sums Our algorithm is recursive Suppose that n is a power of 2 (otherwise add some 0 s to the sequence) Given a sequence of numbers compute recursively compute the prefix sums for (after this step we have half of the prefix sums for ) compute the remaining sums by adding to

5. Complexity 20-5 Analysis The depth of recursion for n numbers is log n On each iteration the algorithm performs additions in one step and then addition in another step Thus, the time complexity of the algorithm is 2·log n and the total amount of work is Therefore, by Brent’s principle, the number of processors needed is

6. Complexity 20-6 Associative Operations Note that this algorithm can be used not only for addition, but also for other operations like multiplication and others The only property of addition we used is associativity Definition An operation  is said to be associative if, for any a, b and c (a  (b  c)) = ((a  b)  c) Definition An operation  is said to be associative if, for any a, b and c (a  (b  c)) = ((a  b)  c)

7. Complexity 20-7 Addition We compute the sum of two binary numbers carry 1 carry 1 carry 1 carry 0 carry 0 … This algorithm requires a time O(log n) and cannot be parallelized in an obvious way

8. Complexity 20-8 Fast Addition Let and be binary representations of two integers such that Let be their sum and let the carry out of the i th position If we know the carries, we can compute the sum in two parallel steps:

9. Complexity 20-9 Computing Carries The carry is 1 if and only if (a) or (b) at least one of them is 1 and the previous carry is 1 If we define and then We also assume that

10. Complexity 20-10 We define a binary operation on pairs of bits: Now, denoting we get Then That is where

11. Complexity 20-11 Claim. The operation  is associative

12. Complexity 20-12 Finally, we define Then

13. Complexity 20-13 Analysis The carries can be computed as prefix sums in a parallel time 2 + 2 log log n Then we complete the computation in 2 more steps The total amount of work is O(log n) and the number of processors needed

14. Complexity 20-14 Fast Multiplication To multiply two numbers and in binary representation, we need to compute the following sum This at most n additions can be performed in parallel by grouping summands in pairs. The depth of such a binary tree is Therefore two numbers can be multiplied in a parallel time