Comparison Networks Sorting Sorting binary values Sorting arbitrary numbers Implementing symmetric functions

Sorting Algorithms Example Mergesort(array[1,…,n] of Integers): begin Mergesort(array[1,…,n/2]); Mergesort(array[n/2+1,…,n]); Merge(array[1,…,n/2], array[n/2+1,…,n]); end comparisons required to merge two arrays of size m/2 comparisons to sort n elements Order of comparisons not fixed in advance. Not readily implementable in hardware.

Sorting Networks Sorting Network C D B A A B C D Order of comparisons fixed in advance. Readily implementable in hardware.

Sorting Networks (binary values) inputs outputs 1 sorted Sorting Network 1 1 1 1 1 1 1

Comparator (2-sorter) inputs outputs x min(x, y) C y max(x, y)

Comparator (2-sorter) inputs outputs x min(x, y) y max(x, y)

Comparison Network 1 1 1 1 width n depth d

Comparison Network 1 1 1 1 n / 2 comparisons per stage d stages

Sorting Network Any ideas?

Sorting Network inputs outputs Sorting Network . n . n 1

Insertion Sort Network inputs outputs depth 2n 3

Batcher Sorting Network Next Lecture

Sorting Arbitrary Numbers inputs outputs x min(x, y) y max(x, y) x, y can be values from any linearly ordered set, e.g., integers, reals, etc.

Integer Comparator X, Y: integers represented as m-bit binary strings. Comparison function: C(X,Y) = 1 if X > Y, 0 otherwise. Idea: use C(X,Y) to select the min and the max of X and Y.

Sorting Arbitrary Numbers 9 6 2 2 9 6 6 9 2 2 6 9 sorted

Sorting Arbitrary Numbers 1 4 5 1 5 4 4 5 1 1 4 5 sorted

Sorting Arbitrary Numbers 3 7 3 7 7 3 3 7 not sorted How can we verify if a network sorts all possible input sequences?

Sorting Arbitrary Numbers inputs outputs Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 000 000 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 100 001 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 100 001 101 011 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 100 001 101 011 110 101 not sorted! Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 100 001 101 011 110 101 not sorted! 111 111 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers

Sorting Arbitrary Numbers inputs outputs Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 000 000 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 100 001 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 100 001 101 011 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 100 001 101 011 110 011 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 1 1 1 1 000 000 001 001 010 001 011 011 100 001 101 011 110 011 111 111 Try all possible 0/1 sequences.

Sorting Arbitrary Numbers inputs outputs 000 000 001 001 010 001 011 011 100 001 101 011 110 011 111 111 all sorted! Try all possible 0/1 sequences.

Zero-One Principle If a comparison network sorts all possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Lemma Given For a monotonically increasing function f,

Lemma Given For a monotonically increasing function f,

Proof: Lemma

Proof: Lemma

Proof: Lemma f is monotonically increasing:

Proof: Lemma f is monotonically increasing:

Proof: Lemma f is monotonically increasing:

Generalization Given

Generalization For a monotonically increasing function f, (by induction)

Proof: Zero-One Principle Suppose b) there exists a sequence that it doesn’t sort, i.e., such that but is placed before in the output. a) the network sorts all sequences of 0’s and 1’s, Define f (x) = 0 if 1 otherwise

Proof: Zero-One Principle Sorting Network .

Proof: Zero-One Principle . Sorting Network . . . .

Proof: Zero-One Principle . Sorting Network . . 1 contradiction! . .

Batcher Sorting Network, n = 4

Batcher Sorting Network, n = 8

Lemma 1 Any subsequence of a sorted sequence is a sorted sequence. sorted sorted 1 1 1 1 1 1 1 1

Lemma 2 For a sorted sequence, the number of 0’s in the even subsequence is either equal to, or one greater than, the number of 0’s in the odd subsequence. sorted 1 1 1 1 1 1 even odd

Lemma 3 For two sorted sequences and : denotes the the number of 0’s in denotes the even subsequence of denotes the odd subsequence of

Lemma 3 1 1 1 x ¢ 1 E x ¢ 1 1 O x ¢

Lemma 3 For two sorted sequences and : (by Lemma 2) (by Lemma 2)

Merge Network Merge[4] sorted sorted sorted

Merge Network (pf.) sorted sorted sorted sorted Merge[4] (by Lemma 1)

Merge Network (pf.) sorted sorted Merge[4] By Lemma 3 and differ by at most 1 By Lemma 3 sorted

Merge Network (pf.) Merge[4] sorted and differ by at most 1 By Lemma 3

Merge Network (pf.) 1 1 1 1 1 1 Merge[4] By Lemma 3 and Merge[4] 1 1 and differ by at most 1 By Lemma 3 1 1 1 1

Batcher Sorting Network Merge[8] sorted Sort[4]

Batcher Sorting Network, n = 4 Merge[4]

Batcher Sorting Network, n = 8 Merge[8] Sort[4]

Sorting Networks AKS (Ajtai, Komlós, Szemerédi) Network: based on expander graphs. AKS (Chvátal) Batcher AKS better for

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs 1 bit of memory

Asynchronous token routing device Balancer Asynchronous token routing device inputs outputs balanced token counts 1 bit of memory

Balancer Snapshot inputs outputs x y 1 bit of memory

Data structure for multiprocessor coordination Counting Network Data structure for multiprocessor coordination c c d d c a a d d g c g step sequence f e g a e b f g e b b f b f a e

Execution trace: token counts on all wires Counting Network Execution trace: token counts on all wires 3 1 2 1 2 1 2 1 step sequence

Counting Network Tokens are assigned value based on the output wire number. 4, 8, 12, 16, . . . 3, 7, 11, 15, . . . High throughput Low contention Advantages Counting Network 2, 6, 10, 14, . . . 1, 5, 9, 13, . . .

Comparator inputs outputs x min(x, y) y max(x, y)

Balancer inputs outputs x y

Balancer inputs outputs 7 4 2 5

Balancing Network 3 1 2 1 2 1 2 1 width n depth d

Smooth Sequences for any smooth property: Balancing Network .

Smooth Sequences . smooth property: for any Balancing Network 3 4 3 3

Step Sequences for any step property: Balancing Network . .

Step Sequences . for any step property: Balancing Network 3 3 3 3 4 4

Counting Network . . Balancing network with step output sequences: for any for all inputs

Sorting vs. Counting . . ? Counts Sorts Balancing Network Comparison isomorphic

Sorting vs. Counting . . . . ? Counts Sorts Balancing Network Comparison Network . . . . isomorphic

Sorting vs. Counting . . . . ? Counts Sorts Balancing Network Comparison Network . . . . isomorphic

Sorting vs. Counting Theorem If a balancing network counts, then its isomorphic comparison network sorts, but not vice-versa.

Sorting vs. Counting Counts Sorts By 0/1 principle, we need only consider 0/1 inputs. x y balancer x y min(x, y) max(x, y) comparator

Sorting vs. Counting Counts Sorts By 0/1 principle, we need only consider 0/1 inputs. A step sequence of 0’s and 1’s is a sorted sequence of 0’s and 1’s. 1 1 1 1 comparator balancer

Sorting vs. Counting Sorts Counts Insertion Sort: a network which sorts but doesn’t count. 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

Sorting vs. Counting Sorts Counts 32 16 8 16 12 4 8 12 2 10 5 2 7 8 10 32 16 8 16 12 4 8 12 2 10 5 2 7 8 10 5 2 9 8 5 2

Data structure for multiprocessor coordination Counting Network Data structure for multiprocessor coordination c c d d c a a d d g c g step sequence f e g a e b f g e b b f b f a e

Execution trace: token counts on all wires Counting Network Execution trace: token counts on all wires 3 1 2 1 2 1 2 1 step sequence

Counting Network Tokens are assigned value based on the output wire number. 4, 8, 12, 16, . . . 3, 7, 11, 15, . . . High throughput Low contention Advantages Counting Network 2, 6, 10, 14, . . . 1, 5, 9, 13, . . .

Batcher Counting Network 3 1 2 1 2 1 2 1 n = 4 inputs Batcher sorting network also works as a counting network!

Lemma 1 Any subsequence of a step sequence is a step sequence. 3 3 4 4 4 4 4 4 4 4

Lemma 2 For a step sequence, the sum of the even subsequence is either equal to, or one less than, the sum of the odd subsequence. 3 3 step 3 3 Sum of even subseq.: 3+3+4+4=14 3 3 4 4 4 4 Sum of odd subseq.: 3+4+4+4=15 4 4 4 4 4 4 even odd

Lemma 3 For two step sequences and : denotes the sum of denotes the even subsequence of denotes the odd subsequence of Follows from: (by Lemma 2) (by Lemma 2)

Lemma 3 9 x ¢ 9 E x ¢ 3 4 3 4 9 O x ¢ 3 4

Merge Network Merge[4] step step step

Merge Network (pf.) Merge[4] step step step step

Merge Network (pf.) step step Merge[4] By Lemma 3 and differ by at most 1 By Lemma 3 step

Merge Network (pf.) Merging two step sequences whose sums differ by at most one: 3 4 3 4 step 3 4

Merge Network (pf.) Merging two step sequences whose sums differ by at most one: 3 4 3 4 not quite step 3 4

Merge Network (pf.) Merge[4] step

Batcher Counting Network Merge[8] step Count[4]

Verifying that a Network Counts Theorem A balancing network with m balancers is a counting network if and only if it satisfies the step property for all input sequences with sum .