1. Chansup Byun, William Arcand, David Bestor, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Andrew McCabe, Peter Michaleas, Julie Mullen, David O’Gwynn, Andrew Prout, Albert Reuther, Antonio Rosa, Charles Yee 2012 IEEE High Performance Extreme Computing Conference 10 - 12 September 2012 Driving Big Data With Big Compute This work is sponsored by the Department of the Air Force under Air Force contract FA8721-05-C-0002. Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the United States Government.

2. Presentation Name - 2 Author Initials MM/DD/YY Introduction LLGrid MapReduce Dynamic Distributed Dimensional Data Model (D4M) Demonstration –Data Ingestion Performance –Database Query Performance Summary Outline

3. Presentation Name - 3 Author Initials MM/DD/YY The Big Four Cloud Ecosystems Enterprise Big DataDBMS Supercomputing Each ecosystem is at the center of a multi-$B market Pros/cons of each are numerous; diverging hardware/software Some missions can exist wholly in one ecosystem; some can’t Each ecosystem is at the center of a multi-$B market Pros/cons of each are numerous; diverging hardware/software Some missions can exist wholly in one ecosystem; some can’t IaaS - Interactive - On-demand - Elastic PaaS - High performance - Parallel Languages - Scientific computing PaaS - Java - Map/Reduce - Easy admin SaaS - Indexing - Search - Security IaaS: Infrastructure as Service PaaS: Platform as a Service SaaS: Software as a Service

4. Presentation Name - 4 Author Initials MM/DD/YY LLGrid provides interactive, on-demand supercomputing Accumulo database provides high performance indexing, search, and authorizations within a Hadoop environment LLGrid provides interactive, on-demand supercomputing Accumulo database provides high performance indexing, search, and authorizations within a Hadoop environment LLGridEnterprise Big DataDBMS The Big Four Cloud Ecosystems IaaS - Interactive - On-demand - Elastic PaaS - High performance - Parallel Languages - Scientific computing PaaS - Java - Map/Reduce - Easy admin SaaS - Indexing - Search - Security IaaS: Infrastructure as Service PaaS: Platform as a Service SaaS: Software as a Service Supercomputing

5. Presentation Name - 5 Author Initials MM/DD/YY LLGridEnterprise Big DataDBMS MapReduce The Big Four Cloud Ecosystems Supercomputing IaaS - Interactive - On-demand - Elastic PaaS - High performance - Parallel Languages - Scientific computing PaaS - Java - Map/Reduce - Easy admin SaaS - Indexing - Search - Security IaaS: Infrastructure as Service PaaS: Platform as a Service SaaS: Software as a Service LLGrid MapReduce provides map/reduce interface to supercomputing D4M provides an interactive parallel scientific computing environment to databases LLGrid MapReduce provides map/reduce interface to supercomputing D4M provides an interactive parallel scientific computing environment to databases

6. Presentation Name - 6 Author Initials MM/DD/YY Big Compute + Big Data Stack High Level Composable API: D4M (“Databases for Matlab”) Weak Signatures, Noisy Data, Dynamics Novel Analytics for: Text, Cyber, Bio Interactive Super- computing High Performance Computing: LLGrid + Hadoop Distributed Database/ Distributed File System Distributed Database: Accumulo/HBase (triple store) Combining Big Compute and Big Data enables entirely new domains A C E B Array Algebra D

7. Presentation Name - 7 Author Initials MM/DD/YY Introduction LLGrid MapReduce Dynamic Distributed Dimensional Data Model (D4M) Demonstration –Data Ingestion Performance –Database Query Performance Summary Outline

8. Presentation Name - 8 Author Initials MM/DD/YY Hadoop Architecture Overview Hadoop cluster

9. Presentation Name - 9 Author Initials MM/DD/YY LLGrid_MapReduce Diagram input ab output a.outb.out LLGrid_MapReduce Mapper Task 1 Mapper Task 2 Reduce Task scan Reduce Out 1 2 4 5 3 Set dependency on Mapper Tasks Scheduler

10. Presentation Name - 10 Author Initials MM/DD/YY LLGrid_MapReduce –-np nTasks \ –-mapper MyMapper \ –-input input_dir \ –-output output_dir \ [--reducer MyReducer \] [--redout output_filename] –MyMapper must have two inputs: input filename and output filename. –LLGrid_MapReduce creates an array job to process all the input files in the input directory for MyMapper –[Optional] LLGrid_MapReduce creates a job for MyReducer to process the output from MyMapper. LLGrid_MapReduce API

11. Presentation Name - 11 Author Initials MM/DD/YY Introduction LLGrid MapReduce Dynamic Distributed Dimensional Data Model (D4M) Demonstration –Data Ingestion Performance –Database Query Performance Summary Outline

12. Presentation Name - 12 Author Initials MM/DD/YY High Level Language: D4M Distributed Database Query: Alice Bob Cathy David Earl Query: Alice Bob Cathy David Earl Associative Arrays Numerical Computing Environment D4M Dynamic Distributed Dimensional Data Model A C D E B A D4M query returns a sparse matrix or a graph… …for statistical signal processing or graph analysis in MATLAB D4M binds associative arrays to databases, enabling rapid prototyping of data-intensive cloud analytics and visualization

13. Presentation Name - 13 Author Initials MM/DD/YY Graphs can be represented as a sparse matrices –Multiply by adjacency matrix  step to neighbor vertices –Work-efficient implementation from sparse data structures Triple Store Representation: Graphs as Matrices 1 2 3 4 7 6 5 xATxATx ATAT 

14. Presentation Name - 14 Author Initials MM/DD/YY Like Perl associative arrays but in 2D and mixed data types A('alice ','bob ') = 'talked ' or A('alice ','bob ') = 47.0 1-to-1 correspondence with triple store ('alice ','bob ’,'talked ’) or ('alice ','bob ’,47.0) Associative Arrays Concept alice bob talked alice bob

15. Presentation Name - 15 Author Initials MM/DD/YY Composable mathematical operations A + B A - B A & B A|B A*B Composable query operations via array indexing A('alice ', :) alice row A('alice bob ', :) alice and bob row A('al* ', :) rows beginning with al A('alice : bob ', :) rows alice to bob A(1:2, :) first two rows A == 47.0 all entries equal to 47.0 Associative Arrays Implementation Complex queries with ~50x less effort than Java/SQL Naturally leads to high performance parallel implementation Complex queries with ~50x less effort than Java/SQL Naturally leads to high performance parallel implementation

16. Presentation Name - 16 Author Initials MM/DD/YY Introduction LLGrid MapReduce Dynamic Distributed Dimensional Data Model (D4M) Demonstration –Data Ingestion –Database Query Summary Outline

17. Presentation Name - 17 Author Initials MM/DD/YY A Python application to parse ASCII files and to ingest the result into an Accumulo database Data Ingestion With LLGrid MapReduce Presplit the table by letters+numbers+punctuation Prepend random string (32 in this case) to row keys Presplit the table by letters+numbers+punctuation Prepend random string (32 in this case) to row keys

18. Presentation Name - 18 Author Initials MM/DD/YY Accumulo Ingestion Scalability Study LLGrid MapReduce With A Python Application Data #1: 5 GB of 200 files Data #2: 30 GB of 1000 files 4 Mil e/s Accumulo Database: 1 Master + 7 Tablet servers

19. Presentation Name - 19 Author Initials MM/DD/YY Scalable benchmark specified by graph community Very large power law graph –Local Rows, Cols, Vals: 220790, 220935, 2047790 Graph500 Benchmark Adjacency Matrix Vertex In Degree Distribution Power Law In Degree Number of Vertices

20. Presentation Name - 20 Author Initials MM/DD/YY Accumulo Data Ingestion Scalability pMATLAB Application Using D4M

21. Presentation Name - 21 Author Initials MM/DD/YY Ingestion Rate History Time (HH:MM) Number of entries/second Ingestion Rate History with 6 Tablet Servers Ingestion Rate History with 1 Tablet Server

22. Presentation Name - 22 Author Initials MM/DD/YY Effect of Pre-Split Accumulo with 8 tablet servers

23. Presentation Name - 23 Author Initials MM/DD/YY Effect of Ingestion Block Size Accumulo with 8 tablet servers

24. Presentation Name - 24 Author Initials MM/DD/YY Accumulo Column Query Time pMATLAB Application Using D4M

25. Presentation Name - 25 Author Initials MM/DD/YY Accumulo Row Query Time pMATLAB Application Using D4M

26. Presentation Name - 26 Author Initials MM/DD/YY Scan Rate History Accumulo DB With 1 Tablet Server Time (HH:MM) Scan Rate (entries/second) End of the query operation Start of the query operation

27. Presentation Name - 27 Author Initials MM/DD/YY Scan Rate History Accumulo DB With 6 Tablet Server Time (HH:MM) Scan Rate (entries/second) End of the query operation Start of the query operation

28. Presentation Name - 28 Author Initials MM/DD/YY We have demonstrated using an MPI cluster (LLGrid) environment to drive big data application on a Hadoop cluster environment. –LLGrid MapReduce –Parallel Matlab with D4M (Dynamic Distributed Dimensional Data Model ) Data ingestion and database query results show good scalability in the following use-case scenarios. –A Python application with LLGrid MapReduce –A parallel Matlab application with D4M Graph500 benchmark Summary