MySQL Users Conf MIT Lincoln Laboratory Measuring MySQL Server Performance for Sensor Data Stream Processing Jacob Nikom MIT Lincoln Laboratory The MySQL Users Conference April 2006 This work was sponsored by the U.S. Army Space and Missile Defense Command under Air Force Contract# FA C Opinions, interpretations, recommendations and conclusions are that of the author and are not necessarily endorsed by the United States Government.

MIT Lincoln Laboratory MySQL Users Conf Outline Introduction Benchmark for real-time streamed data recording Benchmark for real-time streamed events processing Summary

MIT Lincoln Laboratory MySQL Users Conf Outline Introduction –Reagan Test Site (RTS) and its instrumentation –RTS Operations Coordination Center (ROCC) –ROCC architecture and real-time data flow –ROCC backend design Benchmark for real-time streamed data recording Benchmark for real-time streamed events processing Summary

MIT Lincoln Laboratory MySQL Users Conf Reagan Test Site (RTS) and its Instrumentation –Multiple RF sensors collecting data in several regions of electromagnetic spectrum –Multiple optical sensors collecting objects metric and spectral characteristics –Telemetry systems capable of tracking multiple targets –Mobile and fixed ground safety instrumentation

MIT Lincoln Laboratory MySQL Users Conf ROCC RTS Operations Coordination Center (ROCC) ROCC controls RTS instrumentation –Manages data flow from multiple sensors –Analyses the acquired data –Displays tracks and predicts the path of space objects –Stores acquired data for later analysis and reporting –Simulates past and future activities Activity Simulation Data Analysis Algorithms Database Displays Network Sensors Primary operations:

MIT Lincoln Laboratory MySQL Users Conf ROCC Publish-subscribe middleware ROCC Architecture and Real-Time Data Flow Sensors Alg_1 Alg_M ROCC Interface Box_1 ROCC Interface Box_N Historian MySQL server Operational Data Store (ODS)

MIT Lincoln Laboratory MySQL Users Conf ROCC Backend Design Network/middleware ODS MySQL server How well Historian/MySQL server pair could satisfy current performance requirements for particular hardware platform and operating system? How well it could satisfy future requirements (how scalable is the solution)? How server performance depends upon specific hardware? How server performance depends upon specific software? Questions for benchmark to answer (then) Messages Queries Historian subscription threads

MIT Lincoln Laboratory MySQL Users Conf Outline Introduction Benchmark for real-time streamed data recording –Insertion measurement benchmark (IMB) –Testing system configurations –Analysis of IMB results Comparing one and multiple tables performance Comparing local and remote servers performance Comparing MySQL 4.1 and 5.0 servers performance Comparing MySQL 4.1 and 4.0 servers performance Comparing MySQL 4.0, 4.1 and 5.0 servers performance Comparing InnoDB engine vs. MyISAM engine performance Comparing JDK bit vs. JDK bit performance Benchmark for real-time streamed events processing Summary

MIT Lincoln Laboratory MySQL Users Conf Insertion measurement benchmark (IMB) Goals –Multiple clients writing into the same table (up to 100 clients) –Multiple clients writing into different tables (up to 100 tables) –Different record lengths (from 10 bytes up to 10 MB – 1,000,000 times) –Reading performance was not the major issue Design parameters –Store up to 10 MB/s with various record lengths –Simulate the asynchronous message generation in the acquisition system Server Table Client 1 … 100 Multiple clients – one table (M1) Client 1 … 100 Multiple clients – multiple tables (MM)

MIT Lincoln Laboratory MySQL Users Conf Testing system configurations QuantityCPUMemoryDisksNetwork 2Quad CPU Opteron, 64-bit, 1 MB Cache 8 GBRAID 5 SCSI Ultra 320, 300 GB 144 GB, 10 K rpm, 4.7 ms 1 Gb/sec 4Dual CPU Opteron, 64-bit, 1 MB Cache GBRAID5, SATA, 250 GB, 80 GB, 7.2 K rpm, 10.9 ms, UltraATA-100, 7.2 K rpm, 11.5 ms 1 Gb/sec 2Single and dual CPU Intel(R) Xeon(TM) CPU, 2.0 GHz, 2 MB Cache GBUltra SCSI 320, 146 GB, 15K rpm, 3.5 ms SCSI Ultra 160, 73 GB, 10K rpm, 7.8 ms 1 Gb/sec #Operating SystemMySQL ServerJava Client 1RHEL 4 AS, kernel ELsmp, 64-bit standard 32/64-bitbuild 1.4.2_03, 32-bit 2RHEL 4 ES, kernel ELsmp 32-bit standard 32/64-bitbuild 1.4.2_10, 32-bit 6SLES 8 kernel , 64-bit standard 32/64-bitbuild 1.5.0_01, 64-bit Software tested Hardware tested

MIT Lincoln Laboratory MySQL Users Conf Outline Introduction Benchmark for real-time streamed data recording –Insertion measurement benchmark (IMB) –Testing system configurations –Analysis of IMB results (MyISAM tables) Comparing one and multiple tables performance Comparing local and remote servers performance Comparing MySQL 4.1 and 5.0 servers performance Comparing MySQL 4.1 and 4.0 servers performance Comparing MySQL 4.0, 4.1 and 5.0 servers performance Comparing InnoDB engine vs. MyISAM engine performance Comparing JDK bit vs. JDK bit performance Benchmark for real-time streamed events processing Summary

MIT Lincoln Laboratory MySQL Users Conf Comparing One and Multiple Tables Performance (MyISAM) 4-CPU Opteron – local client and server (the same machine)

MIT Lincoln Laboratory MySQL Users Conf Comparing Local and Remote Servers Performance (MyISAM) 2-CPU server; 2-CPU remote client / 2-CPU local client and server 2-CPU server; 4-CPU remote client / 2-CPU local client and server

MIT Lincoln Laboratory MySQL Users Conf Comparing MySQL 4.1 and 5.0 Servers Performance (MyISAM) 1-CPU Intel Xeonlocal client and server

MIT Lincoln Laboratory MySQL Users Conf Comparing MySQL 4.0 and 4.1 Servers Performance (MyISAM) 1-CPU Intel Xeonlocal client and server

MIT Lincoln Laboratory MySQL Users Conf Comparing MySQL 4.0, 4.1, and 5.0 Servers Performance (MyISAM) 1-CPU Intel Xeonlocal client and server

MIT Lincoln Laboratory MySQL Users Conf Comparing InnoDB vs MyISAM and JDK bit vs JDK bit 2-CPU Opteron client; 4-CPU Opteron server; MySQL

MIT Lincoln Laboratory MySQL Users Conf Introduction Benchmark for real-time streamed data recording Benchmark for processing real-time streamed events – Client/Server (CS)–based ROCC architecture – Using indexes for real-time data storing and retrieving – Records retrieval acceleration using secondary query and indexes – Event measurement benchmark (EMB) – Analysis of EMB Results – Comparing MySQL 4.1 and 5.0 servers performance (MyISAM) Summary Outline

MIT Lincoln Laboratory MySQL Users Conf Client/Server (CS)–based ROCC architecture CS drawbacks Data centralization More workload for the network – UDP vs. TCP/IP, no multicast More workload for the database server Require indexes and multilevel queriesmore complex requests Sensors CS advantages Dont need special recording component (Historian) Pull modelall components get the data when they want them The same API for the current and historical data Excellent filtering capabilities (SQL) High throughput, very simple settings, no load balancing Very standard, very low cost, low demand for resources ROCC Interface Box_N ROCC Interface Box_1 Alg_1 Alg_M ODS MySQL server

MIT Lincoln Laboratory MySQL Users Conf Using Indexes for Storing and Retrieving Real-Time Data Traditional (interactive and batch) approach 1.Store data into the relational table 2.Add indexes to the table once the data accumulation is finished 3.Retrieve the data using added indexes Real-time (continuous) approach 1. Store data into the relational table using primary key with AUTO_INCREMENT option 2. Retrieve the latest data using primary key, ORDER and LIMIT query parameters 3. Store the latest retrieved data in the temporary derived table 4. Make the search of the record using all available fields only in the derived table Advantages Simplicity Independence of indexes addition from data creation Drawbacks Adding indexes to all data in the table is a slow process Indexes will be recalculated every time a new record is stored Advantages Drawbacks Primary key has to be numeric to allow AUTO_INCREMENT option Only one index per table could be used for selecting the derived table Very fast! Index creation time is spread over all insertion operations (10% overhead) LIMIT parameter defines the width of the window (number of records for the second SELECT statement) Primary key is indexed automatically

MIT Lincoln Laboratory MySQL Users Conf Records Retrieval Acceleration Using Inner Query and Indexes recordIDcolumn1………. N301 N-1299 N-2303 ……. N-M400 N-M-1……. 1 recordIDcolumn1………. N301 N-1299 N-2303 ……. N-M400 N-M-1……. 1 recordIDColumn1………. N301 N-1299 N-2303 ……. N-M400 N-M-1……. 1 SELECT * FROM table1 ORDER BY recordID DESC LIMIT M Table for record insertion and retrieval SELECT * FROM () t WHERE t.column1<300 second query window window width = M<<N Primary autoincrement key (indexed) Last record in the table First record in the table First record in the second query window Last record in the second query window Found record

MIT Lincoln Laboratory MySQL Users Conf Event Measurement Benchmark (EMB) Goals and Design Goals –Multiple writing clients write into multiple tables time-stamping each record –Multiple reading clients read from into different tables while the writing clients continue to write into the tables –Reading clients compare each record time stamp with current time and calculate the latency –Reading clients write the results into result tables Design –Measure the latencies between freshly stored and retrieved events –Demonstrate that more than 2 MB/sec throughput could be achieved –Demonstrate the sustainable latencies were less that 300 msec

MIT Lincoln Laboratory MySQL Users Conf MySQL server MySQL database EMB Goals and Design WritingClient_3 WritingClient_2 WritingClient_1 Computer 0 Writing tables 0 Computer 2 Reading Client_6 Reading Client_5 Reading Client_4 Reading Client_3 Reading Client_2 Reading Client_1 Result tables 2 Computer 1 Reading Client_6 Reading Client_5 Reading Client_4 Reading Client_3 Reading Client_2 Reading Client_1 Result tables 1

MIT Lincoln Laboratory MySQL Users Conf Analysis of EMB Results Database server4 CPU Opteron, writing/reading clients2 CPU Opterons MySQL server version –standard, Network 1 Gb/sec, JDK 1.4.2_10 Latencies histograms for writing/reading threads for 4 clients Throughput = 4 MB/sec, 90% latencies within 100 msec Throughput = 2 MB/sec, 90% latencies within 100 msec Throughput = 3 MB/sec, 90% latencies within 40 msec

MIT Lincoln Laboratory MySQL Users Conf Comparing MySQL 4.1 and 5.0 Servers Performance (MyISAM) Database server2 CPU Xeon, writing and reading clientstwo 2CPU Opterons Network 1 Gb/sec, JDK 1.4.2_10 MySQL MySQL

MIT Lincoln Laboratory MySQL Users Conf Outline Introduction Benchmark for recording real-time streamed data Analysis of Insertion Measurement Benchmark (IMB) Benchmark for processing real-time streamed events Summary

MIT Lincoln Laboratory MySQL Users Conf Summary IMB and EMB were designed to model major data flows in ROCC IMB demonstrated: –Java/MySQL client/server architecture satisfies ROCC requirements –MySQL was the fastest version for insertion transactions –MyISAM showed superior performance relative to InnoDB –JDK bit delivers much better performance than bit EMB demonstrated: –Java/MySQL-based client/server architecture with appropriate queries and indexation could be used for ESP applications –It delivered better throughput and latencies than currently employed publish/subscribe architecture –MySQL exhibited better linearity and performance for large number of writing/reading threads than version Benchmarks showed that MySQL server is capable to handle both real-time data recording and real-time event stream processing