©2010 PROS Holdings, Inc. All rights reserved. My JDBC Performance Sucks and What I Did About It!

©2010 PROS Holdings, Inc. All rights reserved. Who am I? 2 Peter Tran Senior Performance Technical Lead Over 12 years with PROS 8 years working on server-side performance 6 years Oracle tuning 3 years SQL Server tuning

©2010 PROS Holdings, Inc. All rights reserved. Topics 3 Assumptions Transaction Isolation Levels Connection Batching Queries Parsing Misc

©2010 PROS Holdings, Inc. All rights reserved. Assumptions 4 Will not compare different driver type Recommendations should be driver type agnostic Examples in presentation uses Type IV Thin Driver Examples uses SQL Server 2008 and Oracle 10g databases Will not cover SQL optimization Will not cover JDBC resource leak

©2010 PROS Holdings, Inc. All rights reserved. Transaction Isolation Levels 5 Big impact on performance Critical for correct business logic java.sql.Connection Transaction Isolation LevelDirty-ReadsNon-RepeatablePhantomPerformance TRANSACTION_NONE N/A Fastest TRANSACTION_READ_UNCOMMITED Yes Fast TRANSACTION_READ_COMMITTED NoYes Medium TRANSACTON_REPEATABLE_READ No YesSlow TRANSACTION_SERIALIZABLE No Slowest

©2010 PROS Holdings, Inc. All rights reserved. Connection 6 java.sql.Connection Use connection pooling Big performance penalty to create new physical connection to database

©2010 PROS Holdings, Inc. All rights reserved. Batching 7 Batch UPDATE and INSERT SQLs Use java.sql.(Prepared/Callable)Statement Added benefit of preventing SQL Injection (security) Do not use java.sql.Statement Remember to setAutoCommit to FALSE SQL statement reuse Reduces parse

©2010 PROS Holdings, Inc. All rights reserved. Batch Insert Example – Wrong! 8 Wrong way to do batch update. //turn off autocommit con.setAutoCommit(false); Statement stmt = con.createStatement(); stmt.addBatch("INSERT INTO employees VALUES (1000, 'Joe Jones')"); stmt.addBatch("INSERT INTO departments VALUES (260, 'Shoe')"); stmt.addBatch("INSERT INTO emp_dept VALUES (1000, 260)"); // submit a batch of update commands for execution int[] updateCounts = stmt.executeBatch();

©2010 PROS Holdings, Inc. All rights reserved. Batch Insert Example 9 Right way to do batch update using PreparedStatement // turn off autocommit con.setAutoCommit(false); PreparedStatement stmt = con.prepareStatement( "INSERT INTO employees VALUES (?, ?)"); stmt.setInt(1, 2000); stmt.setString(2, "Kelly Kaufmann"); stmt.addBatch(); stmt.setInt(1, 3000); stmt.setString(2, "Bill Barnes"); stmt.addBatch(); // submit the batch for execution int[] updateCounts = stmt.executeBatch();

©2010 PROS Holdings, Inc. All rights reserved. Batch Query Example 10 Right way to do batch query using CallableStatement // turn off autocommit con.setAutoCommit(false); CallableStatement stmt = con.prepareCall("sp_GetEmployees (?, ?)"); stmt.setInt(1, 2000); stmt.setString(2, "Kelly Kaufmann"); ResultSet rs = stmt.executeQuery()

©2010 PROS Holdings, Inc. All rights reserved. Batch Insert Test Results 11 Test 1: Statement – No Batch Test 2: PreparedStatement – No batch/no reuse Test 3: PreparedStatement – No batch/reuse Test 4: PreparedStatement – With batch/reuse

©2010 PROS Holdings, Inc. All rights reserved. Batch Pitfalls 12 Forgetting to set autocommit to FALSE Excessively large batch size Writer blocks readers Use up more memory on database server ORA snapshot too old

©2010 PROS Holdings, Inc. All rights reserved. Query Performance 13 Set fetch size Settable java.sql.ResultSet java.sql.Statement java.sql.(Prepared/Callable)Statement Oracle 11.2g driver defaults to 10 SQL Server v3.0 driver defaults to 128 Reduces Physical/Logical I/O Uses more memory in client to hold data

©2010 PROS Holdings, Inc. All rights reserved. Query Performance 14 Fetch only data that is needed SQL Optimization – Physical/Logical IOs Reduce Network Use appropriate getXXX() method for retrieving data Minimize data conversion guessing Eliminates nasty conversion bug

©2010 PROS Holdings, Inc. All rights reserved. Microsoft Query Set Fetch Size Test Results 15 Test 1: Query 1M rows – fetch size = 10 Test 2: Query 1M rows – fetch size = 128 (default) Test 3: Query 1M rows – fetch size = 1M

©2010 PROS Holdings, Inc. All rights reserved. Oracle Query Set Fetch Size Test Results 16 Test 1: Query 1M rows – fetch size = 10 (default) Test 2: Query 1M rows – fetch size = 500 Test 3: Query 1M rows – fetch size = 1M

©2010 PROS Holdings, Inc. All rights reserved. Parsing 17 Parsing is expensive! Scalability killer for high volume OLTP applications Not as big of concern for OLAP applications Consumes CPU intensive Database engine not doing real-work Increase latch contention Increase memory usage SQL statement cache is a singleton Oracle: Shared Pool SQL Server: Plan Cache

©2010 PROS Holdings, Inc. All rights reserved. Parsing 18 Best place to fix is in the application code Use PreparedStatement or CallableStatement No Parse – Predicates uses bind variables and Statement reused Soft Parse – Predicate uses bind variables but Statement not reused Hard Parse – Predicate with literal values Parse TypePerformance Parse OnceBest Soft ParseGood Hard ParseBad

©2010 PROS Holdings, Inc. All rights reserved. Parse Fix - Hack 19 Do not have access to application code May select a suboptimal query plan Buggy

©2010 PROS Holdings, Inc. All rights reserved. Oracle Parsing Example 20 Oracle 11.2g JDBC Driver Test 1: Hard Parse (using Statement with literals) Test 2: Soft Parse Test 3: Parse Once

©2010 PROS Holdings, Inc. All rights reserved. Microsoft Parsing Example 21 Microsoft JDBC Driver version 3.0 Test 1: Hard Parse (using Statement with literals) Test 2: Soft Parse Test 3: Parse Once

©2010 PROS Holdings, Inc. All rights reserved. jTDS Parsing Example for SQL Server 22 jTDS JDBC Driver version Test 1: Hard Parse (using Statement with literals) Test 2: Soft Parse Test 3: Parse Once

©2010 PROS Holdings, Inc. All rights reserved. Parse Fix - Hack 23 SQL Server Use PARAMETERIZATION = FORCED Set at database level (Database Properties -> Options) Restrictive: Check Books-On-Line (BOL) YMMV – 5% improvements

©2010 PROS Holdings, Inc. All rights reserved. Parse Fix - Hack 24 Oracle CURSOR_SHARING = SIMILAR | FORCED Instance level or session level Use trigger to hack session level for each login CREATE OR REPLACE TRIGGER after_usr_logon AFTER LOGON ON.SCHEMA BEGIN EXECUTE IMMEDIATE 'ALTER SESSION SET CURSOR_SHARING=FORCE'; END; Super buggy Use SIMILAR – mixed work-load OLTP/OLAP Use FORCE – pure OLTP Use EXACT (default) – DSS/DW OLAP

©2010 PROS Holdings, Inc. All rights reserved. Driver Specific Improvements 25 Requires casting to drive specific implementation Your Performance May Vary (YPMV) Microsoft JDBC Driver version 3.0 Adaptive Buffer – reduces memory footprint in client to minimize Out of Memory but requires more round-trip to database (default=adaptive as of version 2.0) Oracle 11.2g JDBC Driver Statement and ResultSet Caching Connection Caching jTDS JDBC Driver version Implicitly caches PreparedStatement – not a new feature

©2010 PROS Holdings, Inc. All rights reserved. Database JDBC Driver Tuning Parameters 26 SQL Server Implicit Unicode conversion Use setStringParametersAsUnicodesetStringParametersAsUnicode Bypass index and use table scan

©2010 PROS Holdings, Inc. All rights reserved. My JDBC Performance Workout Plan 27 Maximize the usage of batching Set appropriate fetch size for queries Set the appropriate isolation level Reduce the hard/soft parses Before After

©2010 PROS Holdings, Inc. All rights reserved. 28 Q&A