1. Bibisect on Demand Bjoern Michaelsen Bjoern.michaelsen@canonical.com
Ubuntu LibreOffice package maintainer
Deputy Member, Board of Directors, The Document Foundation

2. No idea

3. Everything is a file "Everything is a file"
describes one of the defining features of Unix, and its derivatives — that a wide range of input/output resources such as documents, directories, hard- drives, modems, keyboards, printers and even some inter-process and network communications are simple streams of bytes exposed through the filesystem name space.
– Wikipedia

4. ...but not everyone sees the same tree
Lots of containers, views and isolations:
chroot (1984)
VMWare, VirtualBox
KVM, Xen
Docker, LXC, cgroups and private mounts
Ubuntu snappy

5. Atomic updates

6. Content addressable filesystem
In many ways you can just see git as a filesystem — it is content- addressable, and it has a notion of versioning, but I really really designed it coming at the problem from the viewpoint of a filesystem person (hey, kernels is what I do), and I actually have absolutely zero interest in creating a traditional SCM system.
– Linus Torvalds
git object database
commit
tree
blob

7. Bibisect and ostree (2011)
LibreOffice bibisect and Gnome ostree independently started using git as a binary storage in 2011 for testing and QA
Fully known stack for stacktraces and error reporting
Current bibisect data:
One image is 400MB installed
755 images of LibreOffice build on Ubuntu in 12 GB
Thus: 16 MB per image (compressed by factor 25)
Ancient
Current Data:
Over builds from LibreOffice 3.5 (2012) to ~now available
A living git repo that is fed with debug builds for every commit of the current release

8. Xkcd 1367

10. First Proof of Concept (04/2013)
Full image in DSCM using efficient storage
Export as a read-only mount
Atomic switching of state
Per process isolation
On demand, on-the-fly install
Implemented in Python/Fuse/git commands => awfully slow
archive
local client
local cache
old instances – frozen state
new instances – cached state
first instances – on-the-fly DL

11. First Proof of Concept (04/2013)
Implemented in Python/Fuse/git commands => awfully slow
Navigates paths remotely
Downloads blobs and stores them uncompressed in the local cache
Python maps through file access to these blobs
FUSE
Python
git commands

12. A more serious attempt (2016)
Implemented in a system language (rust) for performance
Using FUSE and libgit2
Export as a read-only mount
Each commit is a subdirectory
On demand, on-the-fly install
archive
local client
local cache
old instances – frozen state
new instances – cached state
first instances – on-the-fly DL

13. A more serious attempt (2016)
Implemented in rust/FUSE/libgit2 => better performance
Caches not only blobs, but also commit and directory nodes
Downloads blobs and stores them uncompressed in the local cache
rust maps through file access to these blobs
FUSE
Rust
libgit2

14. A more serious attempt (2016)
Implemented in rust/FUSE/libgit2 => better performance
Caches not only blobs, but also commit and directory nodes
Downloads blobs and stores them uncompressed in the local cache
rust maps through file access to these blobs
FUSE
Rust
libgit2
sshfs/httpfs2
Local Cache
Remote Repo
Local “Checkouts”

15. Challenges and Missing Bits
Artificially creating stat for the filesystem
Directory nodes do not contain filesizes => stat needs to download content
Supporting insecure transports:
requires amending directory nodes with hashes of contents
requires checking signed tags
Garbage collection/Cache Management/Quotas
Adding the git repo structure on top of this to allow using git bisect as expected

16. Application developer pet peeve

17. https://code.launchpad.net/~bjoern-michaelsen/+git/gitfs-rust