@ Carnegie Mellon Databases Data-oriented Transaction Execution VLDB 2010 Ippokratis Pandis Ryan Johnson Nikos Hardavellas Anastasia Ailamaki Carnegie Mellon University École Polytechnique Fédérale de Lausanne Northwestern University

© 2010 Ippokratis Pandis Scaling OLTP on multicores/sockets Shared Everything –Hard to scale 2 Multicore multisocket machine Core CPU Core CPU

© 2010 Ippokratis Pandis Scaling OLTP on multicores/sockets Shared Everything –Hard to scale Shared Nothing –Multiple processes, physically separated data –Explicit contention control –Perfectly partition-able workload –Memory pressure: redundant data/structures 3 Multicore multisocket machine Core CPU Core CPU Two approaches complementary Focus on scalability of a single (shared everything) instance

© 2010 Ippokratis Pandis Shore-MT Multithreaded version of SHORE State-of-the-art DBMS features Two-phase row-level locking ARIES-style logging/recovery Similar at instruction-level with commercial DBMSs 4 High-performing, scalable conventional engine Available at: Sun Niagara T1 Insert-only mbench

© 2010 Ippokratis Pandis Problems on Even Higher Parallelism 5 Sun Niagara T2 TPC-C Payment Lock manager overhead dominant Contention for acquiring locks in compatible mode Linear

© 2010 Ippokratis Pandis Data-oriented Transaction Execution It is not the transaction which dictates what data the transaction-executing thread will access Break each transaction into smaller actions –Depending on the data they touch Execute actions by data-owning threads Distribute and privatize locking, data accesses across threads New transaction execution model Reduce overhead of locking and data accesses 6

© 2010 Ippokratis Pandis DORA vs. Conventional – At 100% CPU 7 Eliminate contention on the centr. lock manager Significantly reduced work (lightweight locks) Sun Niagara T2 64 HW contexts

© 2010 Ippokratis Pandis 8 Roadmap Introduction Conventional execution Data-oriented execution Evaluation Conclusions

© 2010 Ippokratis Pandis Example CPU-0 I D CPU L1 CPU-1 I D CPU-2 I D L2 MEM I/O WH CUSTORD Transaction: I D u(wh) u(cust) u(ord) I = Instruction D = Data 9

© 2010 Ippokratis Pandis Conventional - Example CPU-0 CPU L1 CPU-1CPU-2 L2 MEM I/O WH CUSTORD Transaction: I D u(wh) u(cust) u(ord) u(wh) u(cust) u(ord) 10

© 2010 Ippokratis Pandis Typical Lock Manager The higher the HW parallelism Longer Queues of Requests Longer CSs Higher Contention L1EX L2EX T1 Lock Head Lock Hash Table Queue Lock Requests Xcts Lock Requests 11

© 2010 Ippokratis Pandis Time Inside the lock manager Latch contention on the lock manager Sun Niagara T2 TPC-B 12

© 2010 Ippokratis Pandis Thread-to-transaction – Access Pattern 13 Unpredictable access pattern Source of contention

© 2010 Ippokratis Pandis 14 Roadmap Introduction Conventional execution Data-oriented execution Evaluation Conclusions

© 2010 Ippokratis Pandis Thread-to-data – Access Pattern 15 Predictable access patterns Optimizations possible (e.g. no centralized locks)

© 2010 Ippokratis Pandis Transaction Flow Graph Each transaction input is a graph of Actions & RVPs Actions –Table/Index it is accessing –Subset of routing fields Rendezvous Points –Decision points (commit/abort) –Separate different phases –Counter of the # of actions to report –Last to report initiates next phase –Enqueue the actions of the next phase 16 TPC-C Payment Upd(WH)Upd(DI)Upd(CU) Ins(HI) Phase 1 Phase 2

© 2010 Ippokratis Pandis Partitions & Executors Routing table at each table –{Routing fields executor} Executor thread –Local lock table {RoutingFields + partof(PK), LockMode} List of blocked actions –Input queue New actions –Completed queue On xct commit/abort Remove from local lock table –Loop completed/input queue –Execute requests in serial order Completed Input Local Lock Table Pref LM Own Wait AAB A {1,0}EX A {1,3}EX B A 17 Routing fields: {WH_ID, D_ID} RangeExecutor {0,0}-{5,2}1 {5,2}-{6,1}2

© 2010 Ippokratis Pandis DORA - Example CPU-0 CPU L1 CPU-1CPU-2 L2 MEM I/O WH CUSTORD u(wh) u(cust) u(ord) Transaction: I D u(wh) u(cust) u(ord) 18 Centralized lock free Improved data reuse

© 2010 Ippokratis Pandis Non-partition aligned accesses Secondary index probes without part of the primary key in their definition Each entry stores the part of the primary key needed to decide routing, along with the RID Do the probe with the RVP-executing thread Enqueue a new action to the corresponding thread Do the record retrieve with the RID 19

© 2010 Ippokratis Pandis 20 Roadmap Introduction Conventional execution Data-oriented execution Evaluation Conclusions

© 2010 Ippokratis Pandis 21 Sun Niagara T2 TM1 BASELINE DORA Baseline DORA Eliminates contention on the lock manager Linear scalability to 64 HW ctxs Immune to oversaturation Eliminating contention on the lock mgr

© 2010 Ippokratis Pandis Response time for single client 22 better Exploits intra-xct parallelism Lower response times on low loads

© 2010 Ippokratis Pandis Peak Performance 23 better

© 2010 Ippokratis Pandis Peak Performance 24 better Higher peak performance Always close to 100% CPU utilization

© 2010 Ippokratis Pandis 25 Roadmap Introduction Conventional execution Data-oriented execution Evaluation Conclusions

© 2010 Ippokratis Pandis DORA vs. Conventional 26 Avoid expensive (centralized) lock manager operations 20% Immune to centr. lock manager 40+% Intra-xaction parallelism on light loads 2x Higher performance in the entire load spectrum

© 2010 Ippokratis Pandis 27 Reducing overheads of locking Rdb/VMS –Distributed DBMS –Lock carry-over reduces network traffic DB2 keep table lock setting –Connection holds all table locks until close –Leads to poor concurrency H-Store –Single-threaded, non-interleaved execution model –No locking or latching at all

© 2010 Ippokratis Pandis Summary 28 Large number of active threads stress scalability of database system Data-oriented transaction execution (DORA) Decentralize locking & data accesses Benefits of shared-nothing w/o physical data partitioning Small modifications on a conventional engine Higher performance on the entire load spectrum