Outline for today Topic: MEMStore paper Administrative: No class on Wednesday!

MEMS-based Storage David Nagle, Greg, Ganger, Steve Schlosser, and John Griffin

David NagleDecember, 2000http:// What if a “disk drive” could … Storage 10 Gbytes of data In the size of a penny Deliver 100 MB – 1 GB/sec bandwidth Deliver access times 10X faster than today’s drives Consume ~100X less power than low-power disk drives Integrate storage, RAM, and processing on the same die The drive is the computer Cost less than $10

David NagleDecember, 2000http:// CHIPS CHIPS - Center for Highly Integrated Information Processing and Storage Systems Goals Processor (> 1,000 MIPS) RAM (> 100 MB) Nonvolatile Mass Memory (> 10 GB) Communications (> 100 MB/s) Storage access time (< 1 ms) IC Processing Based Manufactured by IC photolithographic techniques Advantages of Integration Less expensive & smaller volume and mass Lower power and high shock resistance new markets / new applications Co-location of Storage and Processing archival data storage / secure data Today I’ll focus just on MEMS-based storage 2 cm

David NagleDecember, 2000http:// How do you put a “Disk Drive” on a chip? Build storage using MEMS MEMS are MicroElectricMechanicalSystems Physical sensor and actuator systems with features measured in microns Built using process technologies similar to current CMOS fabs Enable co-location of nonvolatile storage, RAM and processing on same physical chip

David NagleDecember, 2000http:// Example The world's smallest guitar is 10 micrometers long – about the size of a single cell -- with six strings each about 50 nanometers, or 100 atoms, wide. Made by Cornell University researchers from crystalline silicon, it demonstrates a new technology for a new generation of electromechanical devices. Photo by D. Carr and H. Craighead, Cornell.The above image (508 x 327 pixels) is the digital image created by the electron microscope, and is the highest-resolution version available.

David NagleDecember, 2000http:// MEMS From Sandia National Labs

David NagleDecember, 2000http:// MEMS 200  m

David NagleDecember, 2000http:// MEMS

David NagleDecember, 2000http:// Applications of MEMS Sensors accelerometers gyroscopes Actuators micromirror arrays for LCD projectors heads for inkjet printers optical switches microfluidic pumps for delivering medicine

David NagleDecember, 2000http:// MEMS-based Storage On-chip Magnetic Storage - using MEMS for media positioning Read/Write tips Read/Write tips Magnetic Media Magnetic Media Actuators

David NagleDecember, 2000http:// MEMS-based Storage Read/write tips Read/write tips Media Bits stored underneath each tip Bits stored underneath each tip side view

David NagleDecember, 2000http:// MEMS-based Storage 1  m probe tip 100  m group of six tips Read/write probe tips

David NagleDecember, 2000http:// MEMS-based Storage Media Sled X Y

David NagleDecember, 2000http:// MEMS-based Storage Springs X Y

David NagleDecember, 2000http:// MEMS-based Storage Anchors attach the springs to the chip. Anchors attach the springs to the chip. Anchor X Y

David NagleDecember, 2000http:// MEMS-based Storage Sled is free to move Sled is free to move X Y

David NagleDecember, 2000http:// MEMS-based Storage Sled is free to move Sled is free to move X Y

David NagleDecember, 2000http:// MEMS-based Storage Springs pull sled toward center Springs pull sled toward center X Y

David NagleDecember, 2000http:// MEMS-based Storage X Y Springs pull sled toward center Springs pull sled toward center

David NagleDecember, 2000http:// MEMS-based Storage Actuators pull sled in both dimensions Actuators pull sled in both dimensions Actuator X Y

David NagleDecember, 2000http:// MEMS-based Storage Actuators pull sled in both dimensions Actuators pull sled in both dimensions X Y

David NagleDecember, 2000http:// MEMS-based Storage Actuators pull sled in both dimensions Actuators pull sled in both dimensions X Y

David NagleDecember, 2000http:// MEMS-based Storage Actuators pull sled in both dimensions Actuators pull sled in both dimensions X Y

David NagleDecember, 2000http:// MEMS-based Storage Actuators pull sled in both dimensions Actuators pull sled in both dimensions X Y

David NagleDecember, 2000http:// MEMS-based Storage Probe tips are fixed Probe tips are fixed Probe tip X Y

David NagleDecember, 2000http:// MEMS-based Storage X Y Probe tips are fixed Probe tips are fixed

David NagleDecember, 2000http:// MEMS-based Storage X Y Sled only moves over the area of a single square Sled only moves over the area of a single square One probe tip per square One probe tip per square Each tip accesses data at the same relative position Each tip accesses data at the same relative position

David NagleDecember, 2000http:// Why Use MEMS-based Storage? Cost ! 10X cheaper than RAM Lower cost-entry point than disk $10-$30 for ~10 Gbytes New product niches Can be merged with DRAM & CPU(s) Example Applications: “throw-away” sensors / data logging systems infrastructure monitoring; e.g., bridge monitors, concrete pours, smart highways, condition-based maintenance, security systems, low-cost speaker-independent continuous speech recognition, etc. Ubiquitous use in everyday world … every appliance will be smart, store information, and communicate 0.01 GB 0.1 GB 1 GB 10 GB 100 GB $1 $10 $100 $1000 CACHE RAM DRAM HARD DISK Entry Cost Entry Cost MEMS

David NagleDecember, 2000http:// Why Not EEPROM? We have computers on a chip now - Embedded computers Billions of embedded CPUs sold today How are HI 2 PS 2 different today’s “embedded computer”? Currently nonvolatile memory is EEPROM (FLASH memory) MEMS >> increase in nonvolatile mass memory (many GB) EEPROM* Feature Size Scaling vs. Time : NOR Cell Area (um 2 ) (density MB/cm 2 ) EEPROM cost $/MB$4 $2 $1.5$1$0.53$0.27 (Best Case - no increase in fab cost / cm 2 ) Taking EEPROM prices as $0.27/MB --> 10GB = $2,700 For IC-Based Storage in 2009 we predict cost ~$25 / 10GB > 100X better than EEPROM * From Semiconductor Industries Association (SIA) Roadmap 1997

David NagleDecember, 2000http:// Why Use MEMS-based Storage? 10 GByte/cm 2 = 65 GB/in 2 density (100x CD-ROM) 30 nm x 30 nm bit size Example Applications: Space / satellite use - store data when not in line of site act as packet buffer for communications satellites, etc. Human portable applications - e.g., medical implants, super PDA Law enforcement / monitoring devices / security surveillance 100,000 Occupied volume [cm 3 ] , ,000 Storage Capacity [GByte] 3.5” Disk Drive Flash memory, 0.4 µm 2 cell Chip-sized data 10 GByte/cm 2 1 Volume !

David NagleDecember, 2000http:// Why Use MEMS-based Storage? Lower Data Latency ! Conventional disk drives: worst-case rotational latency 5-11ms IC-Based Mass Storage: depends on design - 100’s of  s possible Example Applications Transaction-processing storage, Non-volatile storage hierarchies, network- buffers Worst-Case Access Time (Rotational Latency) Cost $ / GB $1 / GB $3 / GB $10 / GB $30 / GB $100 / GB 10ns 1µs 100µs 10ms DRAM HARD DISK Prediction 2008 $300 / GB EEPROM (Flash) MEMS

David NagleDecember, 2000http:// Managing MEMS-based Storage MEMS Data Layout Sector is 8 data bytes + ECC + servo Sector is 8 data bytes + ECC + servo Media area divided into “regions” Media area divided into “regions” 2500 Data stored in “sectors” of ~100 bits Data stored in “sectors” of ~100 bits

David NagleDecember, 2000http:// Data layout Optimized for: Sequential access Local access … Serpentine layout

David NagleDecember, 2000http:// Read-modify-write example Read-modify-write example …

David NagleDecember, 2000http://

David NagleDecember, 2000http://

David NagleDecember, 2000http://

David NagleDecember, 2000http://

David NagleDecember, 2000http://

David NagleDecember, 2000http://

David NagleDecember, 2000http://

David NagleDecember, 2000http://

David NagleDecember, 2000http:// Fast Read-Modify-Write Disks must wait an entire disk rotation to perform a read-modify-write MEMS devices can quickly turn around and write (or rewrite a sector) Example: Read-modify-write of 8 sectors (4KBytes) in msecs Atlas 10KMEMS Read Reposition Write Total

David NagleDecember, 2000http:// X-dimension Settling Time Consider a simple seek... Sweep area of one probe tip Oscillations in X Oscillations in Y Why do we only care about the X dimension? Why do we only care about the X dimension?

David NagleDecember, 2000http:// X-dimension Settling Time Oscillations in X lead to off-track interference! Oscillations in X lead to off-track interference! In Y, the oscillations appear as slight variations in velocity, which can be tolerated. In Y, the oscillations appear as slight variations in velocity, which can be tolerated. Sled is moving in Y Sled is moving in Y Why do we only care about the X dimension? Why do we only care about the X dimension?

David NagleDecember, 2000http:// Seek Time from Center Seek time (ms) X displacement (bits)

David NagleDecember, 2000http:// Seek Time from Center

David NagleDecember, 2000http:// The Effect of Settle Time Seek time (ms) Displacement (bits) Seek time in Y Seek time in X without settling constant with settling constant

David NagleDecember, 2000http:// Seek Time Without Settle

David NagleDecember, 2000http:// Access data and then turn around and access same data Turn-around

David NagleDecember, 2000http:// Access data and then turn around and access same data Turn-around

David NagleDecember, 2000http:// Access data and then turn around and access same data Turn-around

David NagleDecember, 2000http:// Access data and then turn around and access same data Turn-around Turning “Turn-around”, No data is accessed Turning “Turn-around”, No data is accessed

David NagleDecember, 2000http:// Access data and then turn around and access same data Turn-around

David NagleDecember, 2000http:// Access data and then turn around and access same data Turn-around

David NagleDecember, 2000http:// Access data and then turn around and access same data Turn-around Turning “Turn-around”, No data is accessed Turning “Turn-around”, No data is accessed

David NagleDecember, 2000http:// Sustained Data Rate

David NagleDecember, 2000http:// Sustained Data Rate 1.6 Mbits / sec * 1280 tips = 2048 Mbits / sec

David NagleDecember, 2000http:// Sustained Data Rate

David NagleDecember, 2000http:// OS view of MEMS-based storage High-level MEMS characteristics: Long positioning times High streaming rate Logical block interface works well Opportunities for device optimization, but convoluted tricks not necessary

FAST 2004 Paper Specificity test – are the benefits of a policy or role MEMS- specific? If fails (performance same when compared to fast disk), MEMStore considered just like a fast disk wrt this role or policy Merit test If MEMS-specific, is it worth it (>10% improvement)? Standard Storage Interface (interoperability) Linear array of logical blocks (512 bytes) Exact mapping of LBN to physical media is hidden Contract for the Standard Interface (performance model) Sequential access is best Access to nearby LBN is more efficient that distant Ranges of LBN are interchangeable Good qualitative arguments for MEMStores to be block-oriented and the contract stays valid

David NagleDecember, 2000http:// Request scheduling 0-MAXMAX

David NagleDecember, 2000http:// Request scheduling 0-MAXMAX Seek time in X Seek time in Y

Substituting/Migrating in Disk Array

David NagleDecember, 2000http:// MEMS scheduling Saturation point (first come, first served)

David NagleDecember, 2000http:// MEMS scheduling (shortest “seek time” first)

David NagleDecember, 2000http:// MEMS scheduling (shortest positioning time)

David NagleDecember, 2000http:// Disk scheduling X-axis shift Curves saturate in same order, relative position Curves saturate in same order, relative position

FAST 2004 Scheduling Results SDF is Shortest Distance First

David NagleDecember, 2000http:// Data layout Basically as for disks Sequential access >>> not sequential Local access > not local Some interesting differences File size vs. physical location

David NagleDecember, 2000http:// Small requests 0.42 ms/move in this subregion 0.37 ms/move in this subregion

David NagleDecember, 2000http:// Large requests: 256KB Transfer time dominates positioning time 0MAX Short seek Long seek

David NagleDecember, 2000http:// Bipartite layout Metadata or small objects Large/streaming objects

FAST 2004: MEMStore Specific Features Tip – subset parallelism 2D data structures Quick turnarounds (read-modify-write operations) Device scan 2D Data Structure Accesses

David NagleDecember, 2000http:// Failure Management MEMS devices will have internal failures Tips will break during fabrication/assembly … and during use Media can wear With multiple tips, data and ECC can be striped across the tips ECC can be both horizontal and vertical On tip or tip-media failure, ECC prevents data loss Could then use spares to regain original level of reliability

David NagleDecember, 2000http:// Failure Management MEMS devices will have internal failures Tips will break during fabrication/assembly … and during use Media can wear Probe Tip

David NagleDecember, 2000http:// Failure Management MEMS devices will have internal failures Tips will break during fabrication/assembly … and during use Media can wear Probe Tip Spare Tip

David NagleDecember, 2000http:// Failure Management MEMS devices will have internal failures Tips will break during fabrication/assembly … and during use Media can wear Probe Tip Spare Tip

David NagleDecember, 2000http:// Failure Management MEMS devices will have internal failures Tips will break during fabrication/assembly … and during use Media can wear Probe Tip Spare Tip

David NagleDecember, 2000http:// MEMS in Computer Systems MEMS-based storage device simulator Uses first-order mechanics Integrated into DiskSim Models events, busses, cache Compare against simulated disks SimOS-Alpha Full machine simulator with DiskSim as storage subsystem

David NagleDecember, 2000http:// Random Workload - 15X Speedup 10,000 small random requests, 67% reads, exponentially sized with mean 4KB. 10,000 small random requests, 67% reads, exponentially sized with mean 4KB.

David NagleDecember, 2000http:// Random Workload - 15X Speedup 10,000 small random requests, 67% reads, exponentially sized with mean 4KB. MEMS has small positioning variability MEMS has small positioning variability

David NagleDecember, 2000http:// PostMark - 5X Speedup

David NagleDecember, 2000http:// MEMS-based Storage as Disk Cache File System Disk MEMS Cache MEMS Cache HP Cello trace has 8 disks 10.4GB total capacity HP Cello trace has 8 disks 10.4GB total capacity 1999 Disk (Quantum Atlas 10K) 9 GB 1999 Disk (Quantum Atlas 10K) 9 GB Baseline MEMS 3 GB Baseline MEMS 3 GB

David NagleDecember, 2000http:// Baseline Configuration File System Disk

David NagleDecember, 2000http:// Disk Cache Configuration File System MEMS

David NagleDecember, 2000http:// Disk Cache Configuration File System Disk MEMS Cache MEMS Cache Disk MEMS Cache MEMS Cache Disk MEMS Cache MEMS Cache Disk MEMS Cache MEMS Cache

David NagleDecember, 2000http:// MEMS-based Storage As a Disk Cache

David NagleDecember, 2000http:// File System-managed Layout File system could allocate data directly MEMS Disk File system Metadata Small files Paging Large, streaming files

David NagleDecember, 2000http:// Perf Idle Fast Idle Low power Idle Standby Active Low-power Disk Drives IBM Travelstar 8GS Time (s) Power (W) Command stream ends 40 ms 2 s 400 ms

David NagleDecember, 2000http:// MEMS-based Storage Lower operating power 100 mW for sled positioning 1 mW per active tip For 1000 active tips, total power is 1.1 watt 50 mW standby mode 0.5 ms Active Time (s) Power (W) Standby (not to scale) Standby (not to scale) Fast transition from standby

David NagleDecember, 2000http:// PostMark

David NagleDecember, 2000http:// PostMark Performance Idle Active

David NagleDecember, 2000http:// Netscape

David NagleDecember, 2000http:// Netscape Lots of transitions Largely idle Active

David NagleDecember, 2000http:// Future of MEMS-based Storage Perfect for portable devices Size, capacity, power

David NagleDecember, 2000http:// Archival Storage Amount of archived data is growing Tape High latency Legacy tape drives Write-once MEMS-based storage devices 100X the density of write-many MEMS Very low latency Integrated processing can break down legacy barrier

David NagleDecember, 2000http:// Personal Flight Data Recorder Enabled by Camera Networking High-capacity, low-power portable storage Enables research in Building the FDR Indexing, data mining “What was that meeting about?” Privacy

David NagleDecember, 2000http:// System-on-a-Chip Filling memory gap Operating system support Scheduling Data layout Fault management New applications PDA, digital music, video, archival storage 2 cm

David NagleDecember, 2000http:// Active MEMS-based Storage Devices Massively parallel computation directly integrated with storage on chip Chip area available for processing Large potential bandwidth

David NagleDecember, 2000http:// Active MEMS-based Storage Devices Massively parallel computation directly integrated with storage on chip Cap (total) Processors BW (per device) Time 20 X 50 GB 8 50 MB/sec 6,250 sec Enabled by better chip packaging Enabled by better chip packaging 20 X MB/sec 1,000 sec 100 x 10 GB GB/s 10 sec Multiprocessor Active Disk Active MEMS

David NagleDecember, 2000http:// MEMS-based Storage Is On the Way Interesting new storage technology Gigabytes of non-volatile data in a single IC Sub-millisecond average access time Low power Can fill various roles Augment memory hierarchy Portable devices Archival storage Active storage devices

David NagleDecember, 2000http:// MEMS-based Storage at CMU lcs.web.cmu.edu/research/MEMS