1. SORTING NETWORKS

2. Overview Sorting Network Components Sorting Network
Comparison Networks
Comparator
Zero-one Principle
Bitonic Sorting Network
Half-Cleaner
Bitonic Sorter
Merging Network
Sorting Network
Previously we have seen several sorting algorithms that operate in a serial fashion by executing one operation at a time. What we will be presenting today is a look at a form of sorting algorithm based on the comparison-network model which allows for multiple operations to be performed simultaneously.
A comparison network differs from a Random Access Machines implementation of an algorithm in two main areas.
First, a comparison network can only perform comparisons. This means that an algorithm with a counting component such as counting sort cannot be implemented using a comparison network.
Secondly, operations in a comparison network can be performed at the same time or in a parallel fashion, which differs significantly from the RAM model in which operations occur one after another or serially .
Our presentation will cover the building blocks of a sorting network and some helpful principles and conclude with the development of a sorting network.
We’ll begin our discussion of sorting networks by first taking a look at what is a comparison network and it’s components, in particular the comparator device.
Then will discuss the zero-one principle which will assist in generalizing the operation of a comparison network to work correctly on arbitrary input.
We’ll then begin the construction of the a parallel version of merge sort using a bitonic sorter.
And we’ll make a modification of the bitonic sorter into a merging network.
And conclude with the implementation of a sorting network, using these components, that can sort n values in O(lg2 n) time.

3. Comparison Networks
Sorting networks are Comparison Networks that always sort their inputs
Comparison Network composed of:
wires
comparators
Wires: transmits values from place to place
Comparators: two inputs  two outputs
First up is a look at what is comparison network and it’s relation to sorting networks.
Essentially, a sorting network is a comparison network that always sorts it inputs.
A comparison network is comprised solely of wires and comparators.
The wires transmit the input and subsequent values through the network to the output. Traditionally, we view the input traveling along horizontal wires in a left to right direction and we’ll maintain that format throughout our presentation.
Between the input and output wires we have devices called comparators. A comparator operates on two inputs from the wires. The sole purpose of the comparator is to compare it’s inputs and deliver it’s output back to the wires and in our case it orders the outputs in a decending fashion.

4. Comparison Networks Comparators: inputs: x and y outputs: x’ and y’
x’ = min(x, y)
y’ = max(x, y)
input output
x   x’ = min(x, y)
y   y’ = max(x, y)
x x’ = min(x, y)
y y’ = max(x, y)
comparator
If we consider our inputs to the comparators to be the values contained in x and y, the output from the comparators is held in x’ and y’.
The output values in x’ and y’ are calculated by the functions:
x’ = min(x, y) and
y’ = max(x, y)
This results in the output being ordered in decending order in the network.
So if we look at the diagram:
We have our inputs x and y entering the comparator and the sorted values x’ and y’ exiting the comparator.
To make matters a little easier to depict we will illustrate the comparators as vertical lines across the wires with nodes on the wires that the comparator is receiving input from and delivering output to.

5. Comparison Networks Comparators n input wires: a1, a2, …, an
input sequence: < a1, a2, …, an>
n output wires: b1, b2, …, bn
output sequence: <b1, b2, …, bn>
Properties:
requirement: graph is acyclic
output produced only when input is available
comparators process in parallel if input is available
Looking at the design of a comparison network we have our comparators and our input wires.
Since we have the same number of inputs as outputs across the network we can represent our input wires as:
a1, a2, …, an ( a1, a2 through an)
And the output wires as:
b1, b2, …, bn (b1, b2 through bn)
For simplicity sake we represent the values input to the network as the sequence < a1, a2, …, an>
And the output from the network as the sequence <b1, b2, …, bn>
Some requirements for representing the comparison network graph is that:
The graph of the wires and comparators interconnections must be acyclic, so we can never go through the same comparator twice.
The output from a comparator can only be produced when then input is made available to it.
And the comparators are able to process the input in parallel as the input is made available.

6. Comparison Networks comparators
a b1
a b2
a b3
a b4
values A C D E B
So taking a look at a simple example of a comparison network made up of 4 inputs(a1 – a4) and outputs(b1 - b4) with 5 comparators operating in the network we have the following diagram.
Our input sequence is <9, 5, 2, 6> and we state that this sequence arrives at the input wires at time 0.
We designate our five comparators as A, B, C, D and E.

7. Comparison Networks outputs
a b1
a b2
a b3
a b4
depth:
depth starts at 0
input wires depth: dx and dy
output wires depth: max(dx, dy) + 1 5 2 A C 9 6 D E 2 5 B 6 9
This comparison network happens to also be a sorting network and the input should be sorted into the order on the right 2, 5, 6 and 9
Walking through the networks operations:
At time 0 the comparators A and B both have their inputs available.
If we assume that each comparator takes 1 time unit to compute it’s output values.
Then at time 1 both comparators A and B produce their output.
Resulting in A computing the min and max of the outputs and producing 5 on the top and 9 on the bottom from the inputs 9 and 5
And B generating 2 and 6 respectively on the output wires unchanged from the input since they were already sorted.
At time 1, when A and B have finished, both comparators C and D have their input available.
C receives 5 and 2 and outputs 2 and 5 at time 2 since it took 1 time for the comparator to compute
And D receives 9 and 6 and outputs 6 and 9 also at time 2
At this point both the top and bottom wires have completed and the values on the middle wires are available to comparator E.
So at time 2, E receives 6 and 5 and outputs 5 and 6 at time 3.
So under the assumption that each comparator takes unit time, we can define the running time of a comparison network, as the time it takes for all the output wires to receive their values once the input wires receive theirs.
Informally, this would be the largest number of comparators that any input element can pass through as is travels across the network.
Defining this more formally we use the depth of a wire. Where the depth of the initial input wire to the network has depth 0 and if a comparator as two input wires with depths dx and dy then the comparators output wires have depth max(dx, dy) + 1.
So we define the depth of a comparator to be the depth of it’s output wires.
Looking at the depths from the example.
We can see that since both comparators A and B receive their inputs at time 0 and it takes 1 unit of time to process the output, then since the depth of the comparators is defined by their output wires, both A and B have depth 1. It follows that C and D would receive their inputs at time 1 resulting in their depths both being 2 when they deliver their output. And finally E would have a depth of 3.
Therefore, in this example the comparison network would have a depth of 3.

8. Comparison Networks
Sorting Network is a Comparison Network where the output is sorted
output sequence is monotonically increasing
(b1 <= b2 …<= bn) for every input sequence
Not all Comparison Networks are Sorting Networks
Comparison Network is like a procedure in that it specifies how comparisons are to occur
So to review.
A sorting network is a comparison network that sorts it’s output. That is the output sequence is monotonically increasing for every input sequence.
Now the example that we saw of a comparison network just happened to also be a sorting network, since it would sort all of it’s inputs, however, not all comparison networks are sorting networks.
So essentially a comparison network is like a procedure in that it specifies how comparisons are to occur from the arrangement of the comparators. But it is unlike a procedure in that its size, or the number of comparators it contains, depends on the number of inputs and outputs.

9. Zero-One Principle
If a Sorting Network works correctly when each input is drawn from the set {0, 1}, then it works correctly on arbitrary input numbers
(integers, reals, or values from any linearly sorted set)

10. Zero-One Principle Lemma 27.1
If a Comparison Network transforms the input sequence a = <a1, a2, …, an> into the output sequence b = <b1, b2, …, bn>, then for any monotonically increasing function f, the network transforms the input sequence:
f(a) = <f(a1), f(a2), …, f(an)>
into the output sequence:
f(b) = <f(b1), f(b2), …, f(bn)>

11. Zero-One Principle Proof:
prove claim: if f is a monotonically increasing function, then a single comparator with inputs f(x) and f(y) produces output f(min(x, y)) and f(max(x, y))
f(x) min(f(x), f(y))
f(y) max(f(x), f(y))
min(f(x), f(y)) = f(min(x, y))
max(f(x), f(y)) = f(max(x, y))

12. Zero-One Principle outputs f(x) = ceiling(x/2)
a b1
a b2
a b3
a b4 3 1 A C 5 3 D E 1 3 B 3 5

13. Zero-One Principle Theorem 27.2 (Zero-One Principle)
If a Comparison Network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

14. Zero-One Principle Proof: By contradiction:
Suppose that the network sorts all zero-one sequences, but there exists a sequence of arbitrary numbers that the network does not sort:
<a1, a2, …, an> contains elements ai and aj
where ai < aj
but the network places aj before ai in the output sequence

15. Zero-One Principle Define monotonically increasing function f:
f(x) = { 0 if x <= ai,
1 if x > ai.
aj is placed before ai in the output sequence
f(aj) placed before f(ai)
when input is <a1, a2, …, an>
since f(aj) = 1 and f(ai) = 0, from above we have a contradiction

16. Bitonic Sorting Network
Bitonic Sequence:
a sequence that monotonically increases and then decreases, or can be circularly shifted to become monotonically increasing and then monotonically decreasing
examples:
<1, 4, 6, 8, 3, 2>, < 6, 9, 4, 2, 3, 5>, and
<9, 8, 3, 2, 4, 6>
zero-one: 0i1j0k or 1i0j1k when i,j,k >= 0

17. Bitonic Sorting Network
Half-Cleaner
a bitonic sorter is composed of several stages, each of which is called a half-cleaner
half-cleaner is a Comparison Network of depth 1
line i compared with i + n/2 for i = 1, 2,…, n/2
assume that n is even

18. Bitonic Sorting Network
Example: Half-Cleaner[8] all 1’s or 0’s
bitonic
clean
bitonic ________

19. Bitonic Sorting Network
Lemma 27.3
If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisfies the following properties:
both the top half and the bottom half are bitonic
every element in the top half is at least as small as every element in the bottom half
at least one half is clean (either all 0’s or 1’s)

20. Bitonic Sorting Network
Proof:
the Comparison Network Half-Cleaner[n] compares inputs i and i + n/2 for i = 1, 2, … n/2
suppose input is of the form:
00…011…100…0
there are 3 possible cases for the n/2 midpoint to fall
and the situation where the midpoint falls in a block of 1’s is separated into 2 cases
So 4 cases in total

21. Bitonic Sorting Network
Cases:
bitonic
n/2
top
top
bitonic clean 1 1
bitonic 1 1
bottom
bottom
top
top 1
bitonic clean 1 1
bitonic 1
bottom
bottom

22. Bitonic Sorting Network
Cases:
bitonic
n/2
top
top 1
bitonic 1 1 1 1 1
bitonic clean 1
bottom
bottom
top
top
bitonic clean 1 1 1 1
bitonic 1 1
bottom
bottom 1

23. Bitonic Sorting Network
Bitonic Sorter
by recursively combining half-cleaners, we can build a bitonic sorter, a network that sorts bitonic sequences
first stage: Half-Cleaner[n]
subsequent sorts with Bitonic-Sorter[n/2]
Depth D(n) of Bitonic-Sorter[n]:
D(n) = { if n =1,
D(n/2) if n = 2k and k>= 1

24. Bitonic Sorting Network
Example:
Half-Cleaner[n]
Bitonic-
Sorter[n/2]
Bitonic-
Sorter[n/2]

25. Bitonic Sorting Network
Example:
sorted
bitonic 1 1 1 1 1 1 1 1 1 1 1 1

26. Bitonic Sorting Network
Bitonic-Sorter can be used to sort a zero-one sequence
By the Zero-One principle it follows that any bitonic sequence of arbitrary numbers can be sorted using this network

27. A Merging Network
Networks that can merge two sorted input sequences into one sorted output sequence
The merging network is based on the following
Given two sorted sequences, if we reverse the second sequence and then concatenate the two, the resulting sequence is bitonic.
ie.
given : X = and Y =
YR =
X● YR =
Thus to merge the two input sequences X and Y it suffices to perform a bitonic sort on X concatenated with Y reverse

28. A Merging Network
We can construct MERGER[n] by modifying the first half cleaner of BITONIC-SORTER[n].
The key is to perform the reversal of the second half of the inputs implicitly.
Given two sorted sequences <a1, a2, …, an/2> and <an/2+1, an/2+2, …, an>
Want the effect of bitonically sorting the sequence < a1, a2, …, an/2, an, an-1, an-2, ..., an/2+1>
Given two sorted sequences <a1, a2, …, an/2> and <an/2+1, an/2+2, …, an> to be merged we want the effect of bitonically sorting the sequence < a1, a2, …, an/2, an, an-1, an-2, ..., an/2+1>

29. A Merging Network
BITONIC-SORTER[n] compares inputs i and n/2+i for i= 1,2,3,…,n/2.
a b1
a b2 bitonic
a b clean
bitonic a b4
a b8
a b7 bitonic
a b6
a b5
Review of the first half cleaner of the Bitonic-Sorter[n], with the reverse of the second sequence of values

30. A Merging Network
First stage of the merging network compares inputs i and n/2 + i for i= 1,2,3,…,n/2.
a b1
sorted a b2 bitonic
a b clean
a b4
a b5
sorted a b6 bitonic
a b7
a b8
First stage of the merging network compare inputs I and n – I + 1.
The only subtlety is that the order of the outputs from the bottom of the first stage of Merger[n] are reversed compared with the order of the outputs from an ordinary half-cleaner.

31. A Merging Network
Since the reversal of a bitonic sequence is bitonic sequence.
Both top and bottom outputs from the first stage of the merging network satisfy the properties of Lemma 27.3
both the top half and the bottom half are bitonic
every element in the top half is at least as small as every element in the bottom half
at least one half is clean (either all 0’s or 1’s)

32. A Merging Network
The top and bottom outputs can be bitonically sorted in parallel to produce the sorted output of the merging network.

33. A Merging Network Bitonic- Sorter[n/2] Bitonic- Sorter[n/2]
The first stage of the meger decomposed followed by two parrallel copies of bitonic sorter[n/2]

34. A Merging Network sorted 1 1 1 1 1 sorted 1 1 1 1 sorted 1 1 1 1 1 1 1
sorted 1 1 1 1 1
sorted 1 1 1 1
sorted 1 1 1 1 1 1 1 1 1 1
The same network as the previous slide, but recursively unrolled. Sample zero-one values are shown

35. A Merging Network First stage is different
Consequently the depth is the same as BITONIC-SORTER[n]
Depth D(n) of MERGER[n]:
D(n) = { if n =1,
D(n/2) if n = 2k and k>= 1
D(n) = lg n
The only thing different from the Bitonic Sorter[n] is the first stage. Consequently the depth of Merger[n] is lg n the same as the Bitonic Sorter

36. A Sorting Network
Now we have all the necessary tools to construct a network that can sort any input sequence
We are going to use the a fore mentioned merging network to implement a parallel version of merge sort, called SORTER[n]

37. A Sorting Network Sorter[n/2] Merger[n] Sorter[n/2]
The sorting network Sorter is constructed by recursievely combining merging networks.
This the recursive construction

38. A Sorting Network Merger[2] Merger[4] Merger[8] Merger[2] Merger[2]
If we unroll the recursion the network looks like this for n = 8

39. A Sorting Network 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 3 4 4 4 4 5 5 6 depth1 1 1 1 1 1 1 1 1 1 1
This is what the Merger boxes look like with the actual merging networks.
Notice that all the mergers keep with the properties that are necessary for them to work.
A sorted input, and gives out a sorted output.
Show the bitonic clean areas that would occour. The first stage of the Mergers…… 1 2 2 3 4 4 4 4 5 5 6
depth

40. A Sorting Network

41. Conclusion Sorting Network Components Sorting Networks
Comparison Networks
Zero-one Principle
Bitonic Sorting Network
Half-Cleaner
Bitonic Sorter
Merging Network
Sorting Networks