1 Datamation Sort 1 Million Record Sort using OpenMP and MPI Sammie Carter Department of Computer Science N.C. State University November 18, 2004

2 Background  The Datamation sorting benchmark was introduced in 1985 by a group of database experts as a test of a processor's I/O subsystem and operating system.  The performance metric is the time to sort 1 million 100-byte records, where the first 10- bytes are the key.

3 Sorting Review  The common sorting algorithms can be divided into two classes by the complexity of their algorithms. Algorithmic complexity is a complex subject (imagine that!) that would take too much time to explain here, but suffice it to say that there's a direct correlation between the complexity of an algorithm and its relative efficiency. Algorithmic complexity is generally written in a form known as Big-O notation, where the O represents the complexity of the algorithm and a value n represents the size of the set the algorithm is run against.  The two classes of sorting algorithms are O(n 2 ), which includes the bubble, insertion, selection, and shell sorts; and O(n log n) which includes the heap, merge, and quick sorts.  O(logn) < O(n ) < O(nlogn) < O(n2) < O(n3) < O(2n)

4 O(n 2 ) Sorts

5 O(n log n) Sorts

6 Sorting Example  Sorting Example Sorting Example

7 OpenMP  OpenMP is an industry standard application programming interface (API) for writing parallel applications for shared memory computers. At the heart of OpenMP are directives, or pragmas, programmers insert to incrementally add parallelism to a program.

8 OpenMP – Hello World #include main () { int nthreads, tid; /* Fork a team of threads giving them their own copies of variables */ #pragma omp parallel private(nthreads, tid) { /* Obtain thread number */ tid = omp_get_thread_num(); printf("Hello World from thread = %d\n", tid); /* Only master thread does this */ if (tid == 0) { nthreads = omp_get_num_threads(); printf("Number of threads = %d\n", nthreads); } } /* All threads join master thread and disband */ }

9 MPI – Message Passing Interface  MPI defines a standard library for message passing that can be used to develop portable message passing programs using either C or Fortran.  The MPI stand defines both the syntax as well as the semantics of a core set of library routines that are very useful in writing message-passing programs.  MPI was developed by a group of researchers from academia and industry, and has enjoyed wide support by almost all the hardware vendors. Vendor implementations of MPI are available on almost all commercial parallel computers.

10 MPI – Hello World /* mpicc –o helloworld helloworld.c */ /* bsub –W 2 –I –n 4 mpiexec./helloworld */ #include "mpi.h" #include int main(int argc,char *argv[]) { int myrank, numprocs; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD,&myrank); MPI_Comm_size(MPI_COMM_WORLD,&numprocs) printf("Hello World from process %d of %d\n", myrank, numprocs); MPI_Finalize(); return 0; }

11 OpenMP and MPI Examples  OpenMP Hello World omp_hello.c (mcrae)  OpenMP Sort ompmerge3.c (mcrae)  MPI Hello World helloworld.c (henry2)  MPI Sort mpimerge5.c (henry2)

12 Conclusion  Final Sorting Algorithm Description  Questions / Comments / Suggestions  Contact Information: Sammie Carter

13 References  Sorting Algorithms  Sorting Algorithms Demo  Parallel Programming with OpenMP  Grid Computing (Barry Wilkinson)