Presentation is loading. Please wait.

Presentation is loading. Please wait.

Cache Craftiness for Fast Multicore Key-Value Storage

Similar presentations

Presentation on theme: "Cache Craftiness for Fast Multicore Key-Value Storage"— Presentation transcript:

1 Cache Craftiness for Fast Multicore Key-Value Storage
Yandong Mao (MIT), Eddie Kohler (Harvard), Robert Morris (MIT)

2 Let’s build a fast key-value store
KV store systems are important Google Bigtable, Amazon Dynamo, Yahoo! PNUTS Single-server KV performance matters Reduce cost Easier management Goal: fast KV store for single multi-core server Assume all data fits in memory Redis, VoltDB KV

3 Feature wish list Clients send queries over network
Persist data across crashes Range query Perform well on various workloads Including hard ones! clin

4 Hard workloads Skewed key popularity Small key-value pairs Many puts
Hard! (Load imbalance) Small key-value pairs Hard! Many puts Arbitrary keys String (e.g. or integer Some work handle integers much better than string keys, we want to handle both keys well Small KV: the per-query processing overhead is dominating, and it requires the server to process queries efficiently

5 First try: fast binary tree
140M short KV, cores Throughput (req/sec, millions) Network/disk not bottlenecks High-BW NIC Multiple disks 3.7 million queries/second! Better? What bottleneck remains? DRAM! Where in the talk is? I am showing this because of ???

6 Cache craftiness goes 1.5X farther
140M short KV, cores Throughput (req/sec, millions) To address DRAM bottleneck, we applied cache-craftiness. By cache-craftiness, I mean … We find that … can 1.5X, which is the focus of the rest of the talk. Cache-craftiness: careful use of cache and memory

7 Contributions Masstree achieves millions of queries per second across various hard workloads Skewed key popularity Various read/write ratios Variable relatively long keys Data >> on-chip cache New ideas Trie of B+ trees, permuter, etc. Full system New ideas + best practices (network, disk, etc.) .. hard workloads, including workloads w/ skewed….

8 Experiment environment
A 16-core server three active DRAM nodes Single 10Gb Network Interface Card (NIC) Four SSDs 64 GB DRAM A cluster of load generators Let me first introduce the experiment environment. The performance numbers in this talk is

9 Potential bottlenecks in Masstree
Network DRAM Less best practice log log Disk Single multi-core server

10 NIC bottleneck can be avoided
Single 10Gb NIC Multiple queue, scale to many cores Target: 100B KV pair => 10M/req/sec Use network stack efficiently Pipeline requests Avoid copying cost Our target is 10M

11 Disk bottleneck can be avoided
10M/puts/sec => 1GB logs/sec! Single disk Multiple disks: split log See paper for details Write throughput Cost Mainstream Disk MB/sec 1 $/GB High performance SSD up to 4.4GB/sec > 40 $/GB Single multi-core server Fusion io drive, 750M/s, $7,100 FusionIO ioDrive Octal, Fastest SSD, see

12 DRAM bottleneck – hard to avoid
140M short KV, cores Cache-craftiness goes 1.5X father, including the cost of: Network Disk Throughput (req/sec, millions)

13 DRAM bottleneck – w/o network/disk
140M short KV, cores Throughput (req/sec, millions) Cache-craftiness goes 1.7X father! To focus on DRAM bottleneck, let’s forget about network/disk, and now cache-craftiness can do 1.7X better! Now this factor of 1.7 is what rest of this talk is about. Now let me remind you that the rest of the talk is about DRAM bottleneck. What I am going to do is to show you each of the steps we took before arriving at Masstree

14 DRAM latency – binary tree
140M short KV, cores Throughput (req/sec, millions) B A C O 𝑙𝑜𝑔2𝑁 serial DRAM latencies! Y X Z The numbers does not include the cost of network/logging. We start with a binary tree, which is a reasonable tree. 2.7 us/lookup 380K lookups/core/sec 10M keys => VoltDB

15 DRAM latency – Lock-free 4-way tree
Concurrency: same as binary tree One cache line per node => 3 KV / 4 children X Y Z ½ levels as binary tree ½ DRAM latencies as binary tree A B

16 4-tree beats binary tree by 40%
140M short KV, cores Throughput (req/sec, millions) 4-tree does not work well! No balance, terrible performance! -> balanced tree, we still want wide fanout => B+tree

17 4-tree may perform terribly!
Unbalanced: O 𝑁 serial DRAM latencies e.g. sequential inserts Want balanced tree w/ wide fanout A B C D E F O(N) levels! G H I

18 B+tree – Wide and balanced
Concurrent main memory B+tree [OLFIT] Optimistic concurrency control: version technique Lookup/scan is lock-free Puts hold ≤ 3 per-node locks

19 Wide fanout B+tree is 11% slower!
140M short KV, put-only Fanout=15, fewer levels than 4-tree, but # cache lines from DRAM >= 4-tree 4-tree: each internal node is full B+tree: nodes are ~75% full Serial DRAM latencies >= 4-tree Throughput (req/sec, millions) 11%

20 B+tree – Software prefetch
Same as [pB+-trees] Masstree: B+tree w/ fanout 15 => 4 cache lines Always prefetch whole node when accessed Result: one DRAM latency per node vs. 2, 3, or 4 4 lines = 1 line

21 140M short KV, put-only, @16 cores
B+tree with prefetch 140M short KV, cores Beats 4-tree by 9% Balanced beats unbalanced! Throughput (req/sec, millions) Summarize, Better + balanced!

22 Concurrent B+tree problem
Lookups retry in case of a concurrent insert Lock-free 4-tree: not a problem keys do not move around but unbalanced insert(B) Intermediate state! A C D A C D A B C D

23 B+tree optimization - Permuter
Keys stored unsorted, define order in tree nodes A concurrent lookup does not need to retry Lookup uses permuter to search keys Insert appears atomic to lookups Permuter: 64-bit integer insert(B) A C D A C D B A C D B 1 2 3 1 2 Keys are stored unsorted, and Permuter defines the order of keys in the tree node More clear

24 140M short KV, put-only, @16 cores
B+tree with permuter 140M short KV, cores Improve by 4% Throughput (req/sec, millions)

25 Performance drops dramatically when key length increases
Short values, 50% cores, no logging Throughput (req/sec, millions) Why? Stores key suffix indirectly, thus each key comparison compares full key extra DRAM fetch One common problem with most B+tree design is that … Here is an example B+tree Keys differ in last 8B Key length

26 Masstree – Trie of B+trees
Trie: a tree where each level is indexed by fixed-length key fragment Masstree: a trie with fanout 264, but each trie node is a B+tree Compress key prefixes! B+tree, indexed by k[0:7] B+tree, indexed by k[8:15] Explain trie B+tree, indexed by k[16:23]

27 Case Study: Keys share P byte prefix – Better than single B+tree
𝑃 8 trie levels each has one node only A single B+tree with 8B keys Complexity DRAM access Masstree O(log N) Single B+tree O(P log N)

28 Masstree performs better for long keys with prefixes
Short values, 50% cores, no logging Throughput (req/sec, millions) 8B key comparison vs. full key comparison Key length

29 Does trie of B+trees hurt short key performance?
140M short KV, cores 8% faster! More efficient code – internal node handle 8B keys only Throughput (req/sec, millions)

30 Evaluation Masstree compare to other systems?
Masstree compare to partitioned trees? How much do we pay for handling skewed workloads? Masstree compare with hash table? How much do we pay for supporting range queries? Masstree scale on many cores? Move setup earlier, simplify expr. I have shown you some evaluation in previous slides. Now there are bunch of other interesting questions to answer about the final design

31 Masstree performs well even with persistence and range queries
20M short KV, uniform dist., cores, w/ network Memcached: not persistent and no range queries Throughput (req/sec, millions) Redis: no range queries Unfair: both have a richer data and query model 0.04 0.22

32 Multi-core – Partition among cores?
Multiple instances, one unique set of keys per inst. Memcached, Redis, VoltDB Masstree: a single shared tree each core can access all keys reduced imbalance B Y A C X Z B A C Y X Z

33 A single Masstree performs better for skewed workloads
140M short KV, cores, w/ network Throughput (req/sec, millions) No remote DRAM access No concurrency control Partition: 80% idle time 1 partition: 40% 15 partitions: 4% Hit the point Start x-axis from 1 Mention disabled all concurrency control One partition receives δ times more queries δ

34 Cost of supporting range queries
Without range query? One can use hash table No resize cost: pre-allocate a large hash table Lock-free: update with cmpxchg Only support 8B keys: efficient code 30% full, each lookup = 1.1 hash probes Measured in the Masstree framework 2.5X the throughput of Masstree Range query costs 2.5X in performance Range

35 Scale to 12X on 16 cores Short KV, w/o logging
Perfect scalability Throughput (req/sec/core, millions) Scale to 12X Put scales similarly Limited by the shared memory system May scale better w/ fewer DRAM nodes Change to get, say put similarly Number of cores

36 Related work [OLFIT]: Optimistic Concurrency Control
[pB+-trees]: B+tree with software prefetch [pkB-tree]: store fixed # of diff. bits inline [PALM]: lock-free B+tree, 2.3X as [OLFIT] Masstree: first system combines them together, w/ new optimizations Trie of B+trees, permuter OLFIT: Does not support variable length keys pB+-trees: Not concurrent pkB-tree: Not concurrent, Masstree is 20% 1 core PALM: higher latency (100us v.s. Masstree’s 1us) PALM has nice … Masstree builds upon on several previous idea

37 Summary Masstree: a general-purpose high-performance persistent KV store 5.8 million puts/sec, 8 million gets/sec More comparisons with other systems in paper Using cache-craftiness improves performance by 1.5X

38 Thank you!

Download ppt "Cache Craftiness for Fast Multicore Key-Value Storage"

Similar presentations

Ads by Google