Coding and Algorithms for Memories Lecture 6 1

Class Overview What have we studied so far? – Background on memories – Flash memories: properties, structure and constraints – Rewriting codes – WOM codes What’s next? – Other rewriting codes – Lecture 6 (today) – Rank modulation codes – Lecture 7 & 8 – ECC and constrained codes – Lecture 9 – Wear leveling & memory management – Lecture 10 – Storage – Lectures HW 1 – Due today HW 2 – Will be released tomorrow, due May 1 st 2

Array of cells, made of floating gate transistors ─ Each cell can store q different levels ─ Today, q typically ranges between 2 and 16 ─ The levels are represented by the number of electrons ─ The cell’s level is increased by pulsing electrons ─ To reduce a cell level, all cells in its containing block must first be reset to level 0 A VERY EXPENSIVE OPERATION Rewriting Codes 3

Problem: Cannot rewrite the memory without an erasure However… It is still possible to rewrite if only cells in low level are programmed 4

Rewriting Codes Store 3 bits once Store 1 bit 8 times Store 4 bits once Store 1 bit 16 times Rewrite codes significantly reduce the number of block erasures 5

One of the most efficient schemes to decrease the number of block erasures Floating Codes Buffer Codes Trajectory Codes Rank Modulation Codes WOM Codes Rewriting Codes 6

Write-Once Memories (WOM) Introduced by Rivest and Shamir, “How to reuse a write-once memory”, 1982 The memory elements represent bits (2 levels) and are irreversibly programmed from ‘0’ to ‘1’ Q: How many cells are required to write 100 bits twice? P1: Is it possible to do better…? P2: How many cells to write k bits twice? P3: How many cells to write k bits t times? P3’: What is the total number of bits that is possible to write in n cells in t writes? 1 st Write 2 nd Write 7

Buffer Codes An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Enables to write any t bits before resetting – After every write, the last r bits are recoverable What is the problem? Maximize the value of t: – Find upper bound on the number of writes t – Find constructions getting close to this bound 8

Construction of Buffer Codes An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 9

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 10

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 11

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 12

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 13

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) 1(1,0,0,1)(1,1,0,0,1,1,0,0,1,0,0) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 14

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) 1(1,0,0,1)(1,1,0,0,1,1,0,0,1,0,0) 0(0,0,1,0)(1,1,1,0,1,1,0,0,1,0,0) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 15

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) 1(1,0,0,1)(1,1,0,0,1,1,0,0,1,0,0) 0(0,0,1,0)(1,1,1,0,1,1,0,0,1,0,0) 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 16

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) 1(1,0,0,1)(1,1,0,0,1,1,0,0,1,0,0) 0(0,0,1,0)(1,1,1,0,1,1,0,0,1,0,0) 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) 1(1,0,0,1)(1,1,1,1,1,1,1,1,1,1,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? Before writing the next bit, the memory is reset to (1,1,1,1,1,1,1,1,1,1,1) and the buffer (1,0,0,1) is written bit by bit to the memory 17

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) 1(1,0,0,1)(1,1,0,0,1,1,0,0,1,0,0) 0(0,0,1,0)(1,1,1,0,1,1,0,0,1,0,0) 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) 1(1,0,0,1)(1,1,1,1,1,1,1,1,1,1,1) (1,1,1,1,2,1,1,1,1,1,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? Before writing the next bit, the memory is reset to (1,1,1,1,1,1,1,1,1,1,1) and the buffer (1,0,0,1) is written bit by bit to the memory 18

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) 1(1,0,0,1)(1,1,0,0,1,1,0,0,1,0,0) 0(0,0,1,0)(1,1,1,0,1,1,0,0,1,0,0) 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) 1(1,0,0,1)(1,1,1,1,1,1,1,1,1,1,1) (1,1,1,1,2,1,1,1,1,1,1) (2,1,1,1,2,1,1,1,1,1,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? Before writing the next bit, the memory is reset to (1,1,1,1,1,1,1,1,1,1,1) and the buffer (1,0,0,1) is written bit by bit to the memory 19

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) 1(1,0,0,1)(1,1,0,0,1,1,0,0,1,0,0) 0(0,0,1,0)(1,1,1,0,1,1,0,0,1,0,0) 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) 1(1,0,0,1)(1,1,1,1,1,1,1,1,1,1,1) (1,1,1,1,2,1,1,1,1,1,1) (2,1,1,1,2,1,1,1,1,1,1) (2,2,1,1,2,1,1,1,1,1,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? Before writing the next bit, the memory is reset to (1,1,1,1,1,1,1,1,1,1,1) and the buffer (1,0,0,1) is written bit by bit to the memory 20

Construction of Buffer Codes Next BitBuffer StateMemory State (0,0,0,0)(0,0,0,0,0,0,0,0,0,0,0) 1(0,0,0,1)(0,0,0,0,1,0,0,0,0,0,0) 1(0,0,1,1)(0,0,0,0,1,1,0,0,0,0,0) 0(0,1,1,0)(1,0,0,0,1,1,0,0,0,0,0) 0(1,1,0,0)(1,1,0,0,1,1,0,0,0,0,0) 1(1,0,0,1)(1,1,0,0,1,1,0,0,1,0,0) 0(0,0,1,0)(1,1,1,0,1,1,0,0,1,0,0) 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) 1(1,0,0,1)(1,1,1,1,1,1,1,1,1,1,1) (1,1,1,1,2,1,1,1,1,1,1) (2,1,1,1,2,1,1,1,1,1,1) (2,2,1,1,2,1,1,1,1,1,1) (2,2,1,1,2,1,1,2,1,1,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? Before writing the next bit, the memory is reset to (1,1,1,1,1,1,1,1,1,1,1) and the buffer (1,0,0,1) is written bit by bit to the memory 21

Construction of Buffer Codes Next BitBuffer StateMemory State (1,0,0,1)(2,2,1,1,2,1,1,2,1,1,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 22

Construction of Buffer Codes Next BitBuffer StateMemory State (1,0,0,1)(2,2,1,1,2,1,1,2,1,1,1) 1(0,0,1,1)(2,2,1,1,2,1,1,2,2,1,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 23

Construction of Buffer Codes Next BitBuffer StateMemory State (1,0,0,1)(2,2,1,1,2,1,1,2,1,1,1) 1(0,0,1,1)(2,2,1,1,2,1,1,2,2,1,1) 1(0,1,1,1)(2,2,1,1,2,1,1,2,2,2,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? 24

Construction of Buffer Codes Next BitBuffer StateMemory State (1,0,0,1)(2,2,1,1,2,1,1,2,1,1,1) 1(0,0,1,1)(2,2,1,1,2,1,1,2,2,1,1) 1(0,1,1,1)(2,2,1,1,2,1,1,2,2,2,1) 0(1,1,1,0)(2,2,2,1,2,1,1,2,2,2,1) An (n,q,r,t) Buffer Code: – Contains n q-ary cells – Guarantees t bits writes – The last r bits are recoverable Example: n=11, q=3, r=4, t=? t=7+4=11 25

Construction of Buffer Codes What is the number of writes? – On the first layer: n-r writes – On every consecutive write: n-2r+1 – Together: n-r+(q-2)(n-2r+1) = (q-1)(n-2r+1)+r-1 Trivial Upper bound: t ≤ (q-1)n 26

How to Improve? Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) 27

How to Improve? Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) (1,1,1,1,1,1,1,0,1,0,0) Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Before writing the next bit, all cells besides the last 4 are raised to level 1 28

How to Improve? Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) (1,1,1,1,1,1,1,0,1,0,0) 1(1,0,0,1)(1,1,1,1,2,1,1,1,1,0,0) Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Before writing the next bit, all cells besides the last 4 are raised to level 1 29

How to Improve? Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) (1,1,1,1,1,1,1,0,1,0,0) 1(1,0,0,1)(1,1,1,1,2,1,1,1,1,0,0) 1(0,0,1,1)(1,1,1,1,2,2,1,1,1,0,0) Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Before writing the next bit, all cells besides the last 4 are raised to level 1 30

How to Improve? Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) (1,1,1,1,1,1,1,0,1,0,0) 1(1,0,0,1)(1,1,1,1,2,1,1,1,1,0,0) 1(0,0,1,1)(1,1,1,1,2,2,1,1,1,0,0) 1(0,1,1,1)(1,1,1,1,2,2,2,1,1,1,0) Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Before writing the next bit, all cells besides the last 4 are raised to level 1 31

How to Improve? Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) (1,1,1,1,1,1,1,0,1,0,0) 1(1,0,0,1)(1,1,1,1,2,1,1,1,1,0,0) 1(0,0,1,1)(1,1,1,1,2,2,1,1,1,0,0) 1(0,1,1,1)(1,1,1,1,2,2,2,1,1,1,0) 0(1,1,1,0)(2,1,1,1,2,2,2,1,1,1,1) Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Before writing the next bit, all cells besides the last 4 are raised to level 1 32

How to Improve? Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) (1,1,1,1,1,1,1,0,1,0,0) 1(1,0,0,1)(1,1,1,1,2,1,1,1,1,0,0) 1(0,0,1,1)(1,1,1,1,2,2,1,1,1,0,0) 1(0,1,1,1)(1,1,1,1,2,2,2,1,1,1,0) 0(1,1,1,0)(2,1,1,1,2,2,2,1,1,1,1) 1(1,1,0,1)(2,1,1,1,2,2,2,1,2,1,1) Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Before writing the next bit, all cells besides the last 4 are raised to level 1 33

How to Improve? Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) (1,1,1,1,1,1,1,0,1,0,0) 1(1,0,0,1)(1,1,1,1,2,1,1,1,1,0,0) 1(0,0,1,1)(1,1,1,1,2,2,1,1,1,0,0) 1(0,1,1,1)(1,1,1,1,2,2,2,1,1,1,0) 0(1,1,1,0)(2,1,1,1,2,2,2,1,1,1,1) 1(1,1,0,1)(2,1,1,1,2,2,2,1,2,1,1) 1(1,0,1,1)(2,1,1,1,2,2,2,1,2,2,1) Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Before writing the next bit, all cells besides the last 4 are raised to level 1 34

How to Improve? Next BitBuffer StateMemory State 0(0,1,0,0)(1,1,1,1,1,1,0,0,1,0,0) (1,1,1,1,1,1,1,0,1,0,0) 1(1,0,0,1)(1,1,1,1,2,1,1,1,1,0,0) 1(0,0,1,1)(1,1,1,1,2,2,1,1,1,0,0) 1(0,1,1,1)(1,1,1,1,2,2,2,1,1,1,0) 0(1,1,1,0)(2,1,1,1,2,2,2,1,1,1,1) 1(1,1,0,1)(2,1,1,1,2,2,2,1,2,1,1) 1(1,0,1,1)(2,1,1,1,2,2,2,1,2,2,1) 0(0,1,1,0)(2,2,1,1,2,2,2,1,2,2,1) Example: n=11, q=3, r=4, t=? Perform the first 7 writes as before Before writing the next bit, all cells besides the last 4 are raised to level 1 t=7+7=14 35

How to Improve? The number of writes now is t = (q-1)(n-r) (instead of (q-1)(n-2r+1)+r-1) The upper bound is (q-1)n 36

Flash/Floating Codes k bits are stored using n cells A write is a change 0 → 1 or 1 → 0 of one of the k bits Definition – Flash Codes: An (n,k,t) q Flash/Floating Code is a coding scheme that accommodates any sequence of up to t writes of k bits, using n q-level cells, in such a way that a block erasure is never required Goal: Given k, n, q maximize the number of writes t 37

,0 0,1 0,20,30,40,50,60,7 1,01,11,21,31,41,51,61,7 2,0 2,12,22,32,42,52,62,7 3,0 3,13,23,33,43,53,63,7 4,0 4,14,24,34,44,54,64,7 5,0 5,15,25,35,45,55,65,7 6,0 6,16,26,36,46,56,66,7 7,0 7,17,27,37,47,57,67,7 Flash Codes Example: Storing three bits using two 8-level cells Bits Diagram Cells Diagram 38

Write Deficiency A trivial upper bound on the number of writes: t ≤ n(q – 1) Write Deficiency: The difference between the trivial upper bound, n(q – 1), and the guaranteed number of writes t δ = n(q – 1) – t The write deficiency shows how close a given flash code is to the trivial upper bound Theorem: For any (n,k,t) q flash code where n≥k t ≤ n(q – 1) – [½(q – 1)(k – 1)] For n ≥ k, δ ≥ [½(q – 1)(k – 1)] For n large enough, the lower bound on the write deficiency does not depend on n! 39

Example – Two Bits Construction t = Every cell is filled to the top before moving to the next one When the cells coincide, the last cell represents two bits. The cell’s residue modulo 4 sets the bits value: 0 – (0,0) 1 – (0,1) 2 – (1,0) 3 – (1,1) The maximum number of writes (worst case) is n(q-1) – [(q-1)/2] (optimal) before erasing is required. 40

Indexed Flash Codes v3v3 v2v2 v4v4 v1v1 c1c1 c2c2 c3c3 c4c4 Bits Cells Index Cells vkvk v5v5 cncn Partition the k bits into groups of k’ ≤ 6 bits Partition the cells into blocks of n’< n cells Each group of bits is represented by one of the blocks When a block gets full, start using the next empty block Indexing is needed to indicate for each block which bit-group it represents Leads to overhead due to index cells 41

Index-Less Indexed Flash Codes Use the same idea but without the index cells How? – Each block consists of k cells and stores one bit – The blocks are used differently in such a way that it is possible to decode which bit each block stores – The bit value is the parity of its block Example: Storing 4 bits – The memory consists of 4-cell blocks – Assume each cell has 5 levels: 0,1,2,3,4 First BitSecond BitThird BitFourth Bit

Index-Less Indexed Flash Codes Example: Writing 4 bits using 24 cells of 3 levels Bit Value Modified Bit When writing stops, at most (k – 1)(k(q – 1) – 1) levels are not used The write deficiency order is δ = O(k 2 q) 43

Nearly Optimal Construction How to continue when writing stops? – Every block is divided into two sub-blocks of k/2 cells – Every sub-block stores one bit – It is not possible to write the bits as before – Now… use the index cells! Index Cells k cells k/2 44

Nearly Optimal Construction How to continue when writing stops? – Every block is divided into two sub-blocks of k/2 cells – Every sub-block stores one bit – It is not possible to write the bits as before – Now… use the index cells! – For each sub-block, its bit index is stored in the index cells – Repeat the process recursively log 2 k steps – At each step, at most 2k – 2 sub-blocks need to be indexed Theorem: The write deficiency order is δ = O(qklog 2 k/ logq) Index Cells X 3 X 22 k/2 k/4 45

Hot+Cold Rewrite WOM hot/cold WOM k bits t writes k bits The cold bits can be written only once The hot bits can be written multiple times 46

1 Hot + 1 Cold, n=1 Hot-bit write: go UP Cold-bit write: switch RIGHT 1 cold write, q /2-1 hot writes (optimal) x1.5 than 2 hot, +1 than floating codes ,00,00,10, ,01,01,11,1cold,hot = 47

1 Hot + 1 Cold, n=2 Hot-bit write: alternate RIGHT-UP Cold-bit write: +2 UP 2q-3 total writes, up to 1 cold write (optimal) x2 than 2 hot, x1.33 than floating codes c1c : 1,1 2: 1,0 1: 0,1 0: 0,0 cold=0 cold= c2c cold=0 cold=1 48