Storage Systems CSE 598d, Spring 2007 Rethink the Sync April 3, 2007 Mark Johnson

Goals of a File System Durability –Programmers expect that what they write to the file system will exist at a later point Performance –Programmers expect a certain level of performance from the file system Contrasting roles!

What is a Synchronous File System Pretty much what we are used to. Block Calling application until modifications are committed to disk. –Application will not think data was written, when it has not been –Some exception taken for disk level caching because true synchronous writes are very very slow.

Asynchronous File System Modifications are committed long after the call completes Complicates applications that require durability or ordering guarantees. –Shifts burden to programmers. However, most file systems provide an abstraction layer in spite of potential problems.

Interesting Compromises Fsynch does not provide against data loss in the event of power failure Is a conscious decision because real synchronous behavior is too slow. Apps that require very strong durability guarantees (Dbs), often recommend disabling drive cache.

External Synchronization API provides a set of guarantees provided to the clients of the file system. Application Centric view –Synchronous IO File system provides guarantees to the applications Seems simple, what else is there?

External Sync User Centric View! –File System provides guarantees to the USER of an application Ties output to commit strategy –Network –Screen –Other I/O

External Sync Returns control to the application before data is committed. But it buffers all output until the data is committed. Modifications are committed in order, and only then are external output buffers flushed. External I/O rarely blocks (screen/net)

xsynchfs File system developed as part of the Speculator project –“Speculative Execution in a distributed FS” xsynch adds commit dependency –Also has inheritance of these dependencies across processes Most output is delayed no more than the time to commit a single transaction

Output Triggered Commits When output is used, it triggers a commit of any pending disk data. Relies on a causal relationship between FS writes and external output.

Design Defined by externally observable behavior Application State is an internal property of the computer system. “An IO is externally synchronous if the external output produced by the computer system cannot be distinguished from output had the IO been synchronous.”

More Design Traditional systems have two partitions, kernel and user. Usually the return from a system call is an 'external event' This approach adds a third layer, external interfaces. Only on the boundary between app/external is a commit done

Still More Design Requires that all applications access services through the OS Must be an abstraction layer for everything Thoughts? Does this prevent direct I/O?

Correctness Must ensure that external output occurs in same causal order that would have occurred had the IO been synchronous.

Limitations Complicates app recovery after catastrophic failure because application continues after failure, but before errors detected. Possible Idea to checkpoint and rollback –Was rejected due to overhead concerns User may have temporal expectations. –xsyncfs commits every 5 seconds anyway

Bottom Line Uncommitted Output never visible to external observer.

Implementation Uses much from Speculator –Avoids overhead of rollback/checkpoints Propogates dependencies New transactions created on commit of previous transaction Uses journaled mode to guaranteed causuality.

Shared Memory Sticky issue since more than one process could be using the memory to output. Easy Approach: All processes using shared memory become part of the dependency chain. This is very conservative, though.

Correctness Results Actually provided more visible durability than the synchronous file system.

Performance Within 7% of true asynchronous file system Analysis shows that overhead of tracking accounts for most of the 7% Synchronous was an order of a magnitude slower –Interesting since xsyncfs provides better durability guarantees.

Conclusion This seems to be a very good idea Performance indicators show that xsyncfs is a good, if not better replacement for traditional synchronous file systems However, given the current technology of caching in drives, is it necessary?