1. Lecture 15 Flash Memory File Systems Dec. 13, 2016 Kyu Ho Park

2. Outline NAND flash memory characteristics NAND flash file systems
JFFS2, YAFFS2
CFFS
PFFS design and implementation
NAND vs. PRAM characteristics
PFFS design goal
PFFS implementation
Metadata separation
Virtual management of metadata

3. PRAM and Flash Memory PRAM NOR SLC NAND MLC NAND Volatility
Non-volatile
Random access
Yes No
Unit of write
Word (2bytes)
Page (2Kbytes)
Read speed
100ns/word
25us/page
60us/page
Write speed
8us/word
11.5us/word
200us/page
800us/page
Erase speed
No erase
0.7s/64KB
2ms/128KB
Program endurance
106
105
Size
32MByte
~1GB
4GB+
Others
Serial program
Paired page damage

4. Flash memory charateristics

5. NAND Flash Memory NAND flash memory is beauty
Non-volatility
Shock-resistance
Low power consumption
Relatively large capacity
Fast access time (No seek latency)
NAND flash memory is beast
Limited life time(100K ~ 1M erase cycles)
Slow program/erase operation

6. Introduction } … NAND Flash Memory NAND flash memory characteristics
Non-volatile, shock-resistant, power-economic nature
Treated as block-oriented storage media
NAND flash memory characteristics
Page – unit of transfer (Read/Write)
Block – unit of Erase, consist of several pages
In-Place Update is not allowed
Erase is required before update
Requires Mapping management
Garbage collection
Read/write vs. Erase mismatch
make free pages using Erase operation
page }
block
spare …

7. NAND Flash Memory Characteristics
Operations
Read
Write (or Program) : Change state 1 to 0
Erase : Change state 0 to 1
Units
Page (2KB) : Read and Write Unit
Block (64 Pages =128KB) : Erase Unit
 The write operation should be preceded by erase operation which takes much longer time than a page write
Ref) K9F1G08X0A NAND flash memory datasheet

8. NAND flash file systems

9. NAND flash file system
File system is aware of flash memory characteristics
File system performs flash memory specific optimization
Using LFS (Log-structured File System) approach
No in-place update
Any file system modification (data and metadata) is appended to log
Need invalid page management : garbage collection
Need Garbage Collection

10. Log Structured File Systems

11. Log Structured File System
inode 1
File 2
inode 2
Log

12. LFS New inode map block! Fixed Checkpoint region tracks location
area
File 1
inode map
File 2
inode 1
inode 2
Fixed Checkpoint region tracks location
of inode map blocks in log
New inode map block!

13. LFS
Empty segment
Dead

14. LFS
Cleaner runs

15. LFS
Cleaner runs

16. Flash Translation Layer
Related Works
Flash memory based file system approach
Block Level Approach [FTL:Flash Translation Layer][NFTL]
Emulate a standard block device
Portable, file system independent : Ext3, FAT can use on top of it
File system direct interface Approach [JFFS][YAFFS][CFFS][HFFS][PFFS]
OS file system design for Flash memory as storage system
Virtual File System
Ext3
FAT
JFFS
Yaffs
CFFS
Flash Translation Layer
Flash Driver
User application
Kernel
User …
Block Layer
HFFS
PFFS

17. Reference
CFFS : S. H. Lim and K. H. Park, “An efficient nand flash file system for flash memory storage,” IEEE Transactions on Computers, vol. 55, no. 7, pp. 906–912, 2006.
HFFS : C. Lee, S. H. Baek, K. H. Park, “A Hybrid Flash File System Based on NOR and NAND Flash Memories for Embedded Devices,” IEEE Transactions on Computers, vol. 57, no. 7, pp , 2008.
PFFS : Y. Park, S. H. Lim, C. Lee, K. H. Park, et. Al., “PFFS: A Scalable Flash Memory File System for the Hybrid architecture of Phase change RAM and NAND Flash,” The 23rd Annual ACM Symposium on Applied Computing, pp , 2008.

18. Flash File System Consideration
Reliability
Robustness on power failure and unclean shutdown
Wear leveling
Balancing the number of erase cycles
Flash Memory Scan time
Memory footprint
Garbage collection
Throughput

19. Flash File Systems JFFS (Journaling Flash File System)
Designed for both NOR & NAND, which is not optimized for NAND flash memory, designed for NOR at the first time.
Better than FTL (reliability and wear leveling)
High RAM usage
Slow mount time due to Data region scan operation
YAFFS (Yet Another Flash File System)
Developed for the NAND Flash Memory
Well use the OOB region (Spare)
FAST scan reduces mount time
Wear leveling 닳는 정도의 균등화, 한블록을 100번 지우는 것보다 100블록을 한번씩 지우는 것이 수명이 오래간다.

20. JFFS Designed for NOR at the first time.
Ideal log-structured (circular)
High Reliability
Full scan at a mount time
Very long time
Requires many RAM
Linux 2.0, Axis corp.,
Linux 2.2, Redhat
Long test, Robust
JFFS2: eCos, Linux 2.4 & 2.6 by Redhat

21. JFFS: Log-structured A Operations NAND Flash 150 C 250 B Version : 1A
Operations
NAND Flash
150 C
250 B
Version : 1
Store A (200 Bytes) at 0
Offset :
400
Len :
200
Data :
AAAA ..
특정한 위치의 지정없이 순차적으로 저장된다.
그래서 전 파일과 디렉토리 구조를 파악하기 위해서 전 영역의 스캔이 필요
NOR는 inode의 header만 읽고 데이터는 스킵할 수 있으나.
NAND는 page단위로 읽어야 하므로 상당히 불필요한 데이터를 곁들어서 읽어야 한다.
최근 version의 노드가 옛 version의 노드가 가지는 데이터의 일부 구간을 포함할 수 있다.
전 노드를 스캔에서 update된 데이터 구조를 추출해진다.
이렇게 하는 이유: 1 안정성, 2 wear leveling
Version : 2
Offset :
200
Store B(200 Bytes) at 200
jffs _
raw _
inode
Len :
200
Data :
BBBB ..
Version : 3
Offset :
150
Store C(100Bytes) at 150
Len :
100
Data :
CCC ..

22. Garbage Collection (cont’d)
head
가장 오래된 노드: head
가장 최근의 노드끝: tail
너무도 순차적 – 이데아적 log-structured
하나의 블록이 모두 clean해도 GC는 지운다.
GC를 위해서 최소한 2개이상의 emtpy block이 있어야 한다.
메모리로 읽기 – 지우기 – 쓰기는 안된다.
메모리로 읽이 – 쓰기 - 지우기
block

23. JFFS2 Compression Inode
Recoding JFFS, but similar structure
Efficient storage utilization
Fast if the flash is slow
Inode
inode and dirent (reducing space overhead, supporting hard links)
Circular log -> non sequential log structure
randomly select a dirty block
Clean block marker: more reliable at a power-failure
memory usage
4MB for 128MB flash
압축: flash가 느리면 빠를 수 있다. 반대로 CPU가 느리면 전체 성능은 느려진다.
그러나 omap5912kit에서 동일 성능, SC1100에서 압축이 더 빨랐다.
Reduce space overhead
노드에서 파일이름이 차지하는 부분이 상당히 크다 그래서 파일 이름과 노드에서 중복해서 반복되는 것을 dirent로 옮겼다.

24. YAFFS File System Design
Object Header Page : means inode
512 bytes for each inode
Inode number is stored in OOB region as Object Id
Chunk Id is always 0
Data Page
Store data information of File
Object Id stand for their inode
Chunk Id means their position in file structure starting from 1 (1,2,3,4 … etc)
OOB data (16 bytes)
Object Id – inode number (file id)
Chunk Id – file data position (page id)
Write serial number – for integrity
Bytes – amount of stored data
ECC for data, ECC for tag
Data status byte, block status byte
Updating the relevant pages
replaced by writing new pages with new data
But same OOB data
OOB region (Spare)
Data region
page
block …
Object Header Data Free

25. YAFFS log structure Writing data A(0 – 799) to a file “file.doc”
parent object ID: 0 (top)
Name: file.doc
Attributtes: …
Object ID: 100
Page ID: 0
Byte count: 512
OOB region
Data region
A(0~511)
Page ID: 1
A(512~799)
Page ID: 2
Byte count: 288
unused (800~1023)
Page 0
Page 1
Page 2
Writing data A(0 – 799) to a file “file.doc”

26. YAFFS log structure (cont’d)
1. Writing data A(0 – 799) to a file “file.doc”
2. Writing data B( ) to the file
Data region (512 bytes)
OOB region (16 bytes)
parent object ID: 0 (top)
Name: file.doc
Attributes: …, File size
Object ID: 100
Page ID: 0
Byte count: 512
Page 0
A (0~511)
Object ID: 100
Page ID: 1
Byte count: 512
Page 1
A(512~799)
Object ID: 100
Page ID: 2, discarded
Byte count: 288
Page 2
unused (800~1023)
A(512~699)
Object ID: 100
Page ID: 2
Byte count: 488
Page 3
B(700~999)
unused (1000~1023)

27. YAFFS: Garbage Collection
Dirty block: a block that contains at least one discarded page.
Arbitrarily select a dirty block
Set all pages of the dirty block to be discarded.
Erase the dirty block, then the block is empty
JFFSx보다 GC가 매우 간단하다. Fixed sized page때문에 간단
2 3 개의 reserved empty block이 있다.
가장 맨 앞의 empty block부터 채워 나간다.
Discarded page
allocated page
empty
full block

28. YAFFS: RAM Structure RAM usage: 784 kB for 128MB flash
Store as Object Header
Object
Header
Internal Tnode ..
Leaf Tnode .. .. …
Store as Data page
Page number in NAND
Object Header Data Free
RAM usage: 784 kB for 128MB flash

29. New Flash File System
CORE Flash File System(CFFS)

30. CFFS (Core Flash File System )
Top Document Accessed

31. Motivation Problems of Flash Memory based File System
Garbage collection
Performance is degraded when garbage collection happens
Affect the write performance
Mounting (booting) time
Reduce exhaustive mounting time
YAFFS: 24 seconds for 1 GB flash
Generally, file system characteristics affect the flash memory performance
File size distribution
File access patterns
Inode & data characteristics

32. Core Flash File System (CFFS)
Flash File system Design consideration
File size distribution & File access patterns
Most files are small and most accesses are to small files
Most storage is consumed by large files which are ‘read’ most of time
Different characteristics of inode & data
Inode is much hotter than data
Every update of data cause the inode update
Even not update of data, inode update is required
File rename, file move, update of file attributes
Minimize extra write operation
Efficient garbage collection
Fast boot time
Inode classification : i-class 1 for small files, i-class 2 for large files
Inode & data separation & InodeMapblock

33. CFFS Design Inode classification Each inode (file) Occupies one page
Small files : many write operations
i-class 1 : direct index only
Large files : read operations in most of time
i-class 2 : indirect index only
As index level increase, extra write operation increases in NAND flash memory
Reduce extra write operations
i-class 1
Attributes
File Name
Parent Inode
Direct index
i-class 2
Attributes
File Name
Parent Inode
Indirect index
NAND Flash Memory

34. CFFS: Indirect Indexing
i-class 1: Almost files are covered by i-class 1
i-class 2: User files such as multimedia files
Indexing
Basic idea that enables to discriminate Inode blocks and data blocks

35. CFFS Design (cont’d) … … Inode & data Separation NAND structure
Inode and data is written to different blocks in flash memory
Inode is much hotter than data
Hot-cold separation is good for garbage collection
Fast scan can be made
Only the scan of inode blocks can setup the file system
InodeMapBlock manages the inode block lists
NAND structure
InodeMapBlock
Inode block에는 inode만 들어간다. YAFFS는 그렇지 않다 …
RAM structure
Stored in Inode Block
Inode block
Inode
Index entries
Data block
Data .. .. …
Stored in Data Block
Page number in NAND
Inode Data Free

36. CFFS: mount Two step scan operations 1st step: Superblock scan
Superblock: the first valid eraseblock.
Read superblock pages
Finding inode-stored blocks.
2nd step: Inode block scan
For each inode block
Read spare regions of each pages in the inode block
If the page is a valid inode block
Scan the index entries
Build Tnode structure
Inode block
Data block
InodeMapBlock

37. CFFS: mount (cont’d) If the InodeMapBlock is invalid or corrupted.
Read the spare region of the first page of each eraseblocks
if the eraseblock is a inode block.
2nd step: Inode block scan
Else
Skip the eraseblock
Continue for the next eraseblock
Because we can identify if a eraseblock is a inode block by the spare region of the first page.
Read part
Unread part

38. CFFS: Garbage Collection
Garbage collection is performed when
When Block contains only discarded pages – garbage collection
If only small free pages exist, copy it to other region and garbage collection
Inode block & data block separation
When the reclaimed block is a inode block
Live inode page is copied to somewhere other inode block
Eliminate from the InodeBlockList linked list
Inode block pointer in superblock is set to zero
Erase the block
When the reclaimed block is data block
Live data page is copied to somewhere other data block

39. CFFS: Garbage Collection
Dirty block: a block that contains at least one discarded page.
Select the dirties block
Copy a valid block to a free space
Set all pages of the dirty block to be discarded.
Erase the dirty block, then the block is empty
JFFSx보다 GC가 매우 간단하다. Fixed sized page때문에 간단
2 3 개의 reserved empty block이 있다.
가장 맨 앞의 empty block부터 채워 나간다.
Discarded page
allocated page
empty
full block

40. Performance Evaluation
Experimental environment
Embedded EVB (SMDK2410)
ARM9 SoC processor (S3C2410) : 200MHz
Main Memory : 64MB
SAMSUNG NAND K9S1208VOM SMC card
Flash Memory Characteristics
Embedded Evaluation Board

41. Scan Coverage > > > JFFS2 YAFFS CFFS: Invalid InodeMapBlock
Read part
Read part
Read part
Read part
Unread part
Unread part
Unread part
Unread part

42. Performance Evaluation
Performance metric
File system Mount time (boot time)
Scan time estimation using file distribution
64MB NAND Flash memory
80% of file system consists of small files under 1MB
Files, device nod, symbolic links
20% of file system consists of large files over 1MB
Comparison
FTL, NFTL, JFFS, YAFFS, CFFS

43. Experimental Results Garbage collection PostMark benchmark results
Performance metric
Garbage collection
PostMark benchmark
files with file size range 512 bytes to 70KB with distribution 80:20
1000 transactions
Read/write with creation/deletions
Varying the ratio Files : transactions
1:10, 1:3, 1:2
results

44. Experimental Results CFFS vs. YAFFS NAND flash file system comparison
Boot time (scan operation)
Garbage collection (system performance)

45. CFFS: Performance Parameters JFFS2 YAFFS CFFS Scan Time 22.6s 3s 593ms
Postmark Throughput
0.426
0.485
0.487
# of flash page writes
69105
68055
68577
# of extra writes per GC
0.139
0.148
0.119
RAM usage
4.1MB
768kB
784kB

46. Summary of CFFS Core Flash File System(CFFS)
NAND Flash Memory based File System
Designed for NAND flash memory considering general file system characteristics
Architecture
Inode classification : i-class 1, i-class 2
inode & data separation
InodeMapBlock: fast searching inode blocks
Fast boot time (scan operation)
Efficient garbage collection
Hot page locality
Reduced extra writes

47. Previous flash file systems
Feature
Pros.
Cons.
JFFS2
[2001]
LFS approach
Metadata saved with data as a node
Reliable
Scalability problem
Node management overhead
YAFFS2
[2003]
Using spare region of NAND flash memory
Reduced mounting time
JFFS3
[2005]
Move the data index to flash memory
No scalability problem
Fast mount time
Extra write operation for updating index
Snow rolling effect of index
Complex garbage collection
Not implemented yet
CFFS
[2006]
Inode/Data separation
Inode classification along with data size
Minimized mount time
Reduced scalability problem
Reduced GC overhead
Flash scanning is remained
Using main memory for directory management

48. Problems in flash file system
The scalability problems
They have to find where the metadata of file system in NAND flash memory
The mounting time is increased according to the size of NAND flash memory
They store temporal metadata in the main memory
The memory usage is increased according to the number of files.
The performance problems
Each metadata update cause one page write (2KB)
However metadata is usually under 10 bytes and frequently updated
It impacts on the overall flash file system performance

49. The merits of PRAM Non-volatile, Byte addressable In-place update
Erase operation is not necessary
Fast read/write speed of byte level operations
In-place update
We can save and update the data in a fixed region
Long endurance
Coarse-grained wear-leveling technique is enough

50. NAND Flash file system with PRAM
Metadata, which includes Inode and directory structure, is small size data that is frequently updated and mostly needs byte level updates
다행스럽게도
NAND flash memory has many limitation and it is bed to handle the metadata
PRAM is good for small size data that is frequently updated
Use PRAM as metadata storage of NAND flash file system!!

51. PFFS design Store metadata in a fixed region of PRAM
No scanning overhead
All metadata updates are occurred in PRAM as byte level
No additional NAND page writes for metadata updates
Directory and file indexing structures are composed in PRAM
No main memory use for temporal metadata

52. Comparison: Scalability
1. Accessing a file ‘/dir/a.txt’
Open(“/dir/a.txt”)
i-number
Location of inode
Location of data
Type of Index
JFFS [‘01], YAFFS[’03]
CFFS[‘06]
PFFS[‘07]
1. Find i-number using path name
In memory directory structures
In-PRAM directory structures
2. Find inode using i-number
In memory inode map
Simple calculation using i-number
3. Find file data
In memory file structures
In memory data pointers
In-PRAM data pointers
JFFS
YAFFS
CFFS
PFFS
>
1. Scan area comparison
>
>
Scan area
Non-scan area

53. Comparison: Metadata updates
There is no NAND page write for metadata!!

54. Embedded Software Market Size