1. YAFFS: Yet Another Flash File System 2008-01-18 Speaker : Yunjung Yoo

2. Contents Introduction History Design Terminology Layout – NAND / RAM Log-structured File System (1 ~ 5) Mount / Scan Read / Write Read Syscall Flowchart (1 ~ 2) Garbage Collection ECC YAFFS2 OOB Data Checkpointing Conclusion

3. Introduction Charles Manning, Aleph One The first file system designed for NAND YAFFS1 : 512 byte pages + 16 byte spare areas YAFFS2 : 2048 byte pages + 64 byte spare areas Background –SmartMedia + FAT : low memory footprint / not robust –NOR + JFFS : based on JFS / long boot-time, RAM overhead

4. History Decided to create ’YAFFS’ - Dec 2001 Working on NAND emulator - March 2002 Working on real NAND (Linux) - May 2002 WinCE version - Aug 2002 ucLinux use - Sept 2002 Linux rootfs - Nov 2002 pSOS version - Feb 2003 Shipping commercially - Early 2003 Linux 2.6 supported - Aug 2004 YAFFS2 - Dec 2004 Checkpointing - May 2006

5. Terminology Flash-defined –Page - 2k flash page (512 byte YAFFS1) –Block - Erasable set of pages (typically 64 on 2K NAND) YAFFS-defined –Chunk - YAFFS tracking unit. –Usually == page.

6. Design 각 파일은 id 를 갖는다. – inode, id 가 0 이면 유효하지 않은 파일임 파일은 chunk(= page) 로 나뉘어 저장된다. (2K / 512B) Chunk 는 0 과 그 이상의 숫자를 갖는다. 0 은 그 파일의 헤더이다. 객체를 나타낸 자료구조 : –yaffs_ObjectHeader(NAND), yaffs_Object(RAM) 플래시의 각 페이지는 file id : chunk # 로 구분한다. Chunk 에 대한 정보는 Tags 로 OOB 영역에 저장된다. 파일의 내용을 변경할 경우, 관련된 chunk 는 새로운 데이터와 같은 태그로 새로운 위치에 쓰여진다. 이때, 이전의 페이지와 새로운 페이지를 구분하기 위해 write 가 일어 날 때마다 2-bit serial#(YAFFS1) / sequence(YAFFS2) 가 순차적으로 증가한다. 이것은 crash-recovery 할 때 분별할 수 있게 한다. “Discarded” page 가 꽉 찬 block 은 GC 대상이 된다. “Deleted” object 는 “Unlinked dir” 로 옮겨져 삭제된다.

7. Layout | NAND PageSpare ChunkTag ObjectHeader ChunkId == 0 Data ChunkId > 0 Byte#Usage 0~511Data. (File data or header) 512~515Tags 516Data Status 517Block Status 518~519Tags 520~522ECC 523~524Tags 525~527ECC Bit#Usage 18File id 22-bit serial number 20Page-id(chunk id) 10Counter (the # of bytes used in the page 12ECC on tags 2Unused

8. Layout | RAM Header(/) Header(file1) Data(file1) Header(dir1) Header(file2) Data(file2) Object(file2) Object(file1) Object(dir1) Tnode(file1) Object(/) Tnode(file2) Child Sibling Parent

9. Log-structured FS(1) Supposed the flash with 4 pages per block First, Create the file Then, Write a few chunks BlockChunkObjIDChunkIDDelete?Comment 001000Live Object header for this files BlockChunkObjIDChunkIDDelete?Comment 001000Live Object header for this files 011001Live First chunk of data 021002Live Second chunk of data 031003Live Third chunk of data

10. Log-structured FS(2) Close the file >> Write a new object header for the file BlockChunkObjIDChunkIDDelete?Comment 001000Del Obsoleted object header 011001Live First chunk of data 021002Live Second chunk of data 031003Live Third chunk of data 101000Live New object header

11. Log-structured FS(3) Open the file for read/write and overwrite part of the first chunk in the file and close the file. The replaced data and object header chunks become deleted. BlockChunkObjIDChunkIDDelete?Comment 001000Del Obsoleted object header 011001Del Obsoleted First chunk of data 021002Live Second chunk of data 031003Live Third chunk of data 101000Del Obsoleted object header 111001Live New first chunk of data 121000Live New object header

12. Log-structured FS(4) Now resize the file to zero by opening the file with O_TRUNC and closing the file. This writes a new object header with length 0 and marks the data chunks deleted. BlockChunkObjIDChunkIDDelete?Comment 001000Del Obsoleted object header 011001Del Obsoleted First chunk of data 021002Del Obsoleted 2nd chunk of data 031003Del Obsoleted 3rd chunk of data 101000Del Obsoleted object header 111001Del Obsoleted first chunk of data 121000Del Obsoleted object header 131000Live New object header

13. Log-structured FS(5) Block#1 is dirty block!! Rename the file, then we need new object header BlockChunkObjIDChunkIDDelete?Comment 001000Del Obsoleted object header 011001Del Obsoleted First chunk of data 021002Del Obsoleted 2nd chunk of data 031003Del Obsoleted 3rd chunk of data 101000Del Obsoleted object header 111001Del Obsoleted first chunk of data 121000Del Obsoleted object header 131000Del Obsoleted object header 201000Live New object header

14. Mount / Scan Mount 시 Spare 만 읽는다. (chunkId == 0) 인 경우, Object Header 가 저장된 Page 이므로, 이를 읽어서 Object 정보를 주 메모리에 동적으 로 생성. YAFFS2 의 경우 저장된 checkpoint 를 읽어들인다. Object Header ChunkID=0 File data ChunkID=1 Object Header ChunkID=0.... Page1Page2Page 32

15. Read / Write Write –Page 단위로 기록한다. –Page 보다 작거나 파일의 마지막 단편 부분은 정책상 Cache 된다. Read –Cache 에 읽고자 하는 데이터를 찾는다. – 없는 경우는 실제 Flash 로부터 읽어온다. Read/Write 성능 향상을 위해 ChunkCache 라는 간단한 자료구조 사용 –Read/write 회수 감소, Flash memory 수명 증가. –Response Time 감소.

16. Read Syscall Flowchart(1) sys_read generic_file_read generic_file_directIOdo_generic_file_read file_read_actor yaffs_readpage VFS Layer File System Specific Layer no yes noyes 캐시의 내용을 사용자 영역에 복사 페이지 캐시 해시테이블에 요청된 페이지가 존재하는가 ? 페이지 캐시를 사용하는가 ? System Call handling Layer

17. Read Syscall Flowchart(2) yaffs_readpage VFS Interface Layer Core Layer MTD Interface Layer yaffs_readpage_unlock yaffs_readpage_nolock yaffs_ReadDataFromFile yaffs_ReadChunkDataFromObject yaffs_ReadChunkWithTagsFromNAND nandmtd2_ReadChunkWithTagsFromNAND

18. Garbage Collection 3 blocks reserved for GC (384K) If no deleted blocks, GC dirtiest Soft Background deletion: –Delete/Resize large files can take up to 0.5s –Incorporated with GC –Spread over several writes GC is deterministic - does one block for each write (default)

19. ECC Needs Error Correction Codes for reliable use ECC on Tags and data 22bits per 256 bytes, 1-bit correction, 2-bit detection Lots of options: –Hardware or software –YAFFS or MTD –New MTD, old MTD or YAFFS/Smartmedia positioning Make sure bootloader, OS and FS generation all match! Can be disabled - not recommended!

20. YAFFS2 Designed for new hardware: –>=1k page size –No re-writing –Can support Toshiba/Sandisk MLC parts Main difference is ‘discarded’ status tracking ECC done by driver (MTD in Linux case) Extended Tags (Extra metadata to improve performance) RAM footprint 25-50% less faster (write 1-3x, read 1-2x, delete 4-34x, GC 2-7x)

21. OOB Data YAFFS1: –Derived from Smartmedia, (e.g byte 5 is bad block marker) –16 bytes: 7 tags, 2 status, 6 ECC –YAFFS/Smartmedia or JFFS2 format ECC YAFFS2: –64 bytes available in 2k page –MTD-determined layout –MTD or hardware does ECC –Tags normally 28 bytes (16 data, 12ecc)

22. Checkpointing RAM structures saved on flash at unmount (10 blocks) Structures re-read, avoiding boot scan Invalidated by any write Lazy Loading also reduces mount time.

23. Conclusion Very simple NAND-friendly Relatively frugal with resources (especially RAM) Boot quickly Journaling (robustness)