Section 4 Block Storage with SES

Objectives Understand Block Storage in SES Configure storage for use as a block device Access storage from a Linux host

Understand Block Storage Objective Notes:

What is Block Storage SES supports clients accessing the RADOS storage cluster as a block device Provides clients with access to the cluster as if it was a disk drive Supported in the Linux kernel since 2.6.32 Supported by KVM/QEMU 16Eb file system sizes Thin provisioning Snapshots Copy on Write clones

Architecture Block devices Up to 16 EiB Thin Provisioning Snapshots (Kernel module) librados RADOS Cluster

RBD Storage VM VM Storage Block Device QEMU/KVM Linux Kernel librbd Kernel module LIBRADOS

Configure Block Storage

Configure a Block Device To use block storage in Ceph, an rbd image needs to be created This image is then mapped to the host as a block device, appearing under the /dev filesystem /dev/rbdn rbd images can be created in different formats Format 1 was the original format Format 2 is the default from Jewel onwards and supports copy-on-write clones Default can be changed if required Add the following lines to ceph.conf to set the rbd default [client] rbd default image format = 1 or 2 as required

Command: rbd Syntax: rbd OPTIONS command Option Description create -create an rbd image in the default pool info -view details of an existing rbd image map -connect a client node to an rbd image showmapped -display local /dev/rbdx → rbd image mapping Notes:

Creating a new rbd image If required, first create a new pool on the RADOS cluster Or use the default or an existing pool Use the rbd create option to make the new rbd image rbd create -p <poolname> <rbdimagename> --size <xxxx> --image format <1|2> If no pool is specified the default rbd pool will be used Size can be given in various units (MB, GB etc). Defaults to MB Check that the image is created rbd list

Access Storage from a Linux Host

Accessing an rbd image Devices are mapped using the pool and image names The rbd map option is used to configure access rbd map -p <poolname> <rbdimagename> If no pool is specified the default rbd pool will be used Check that the image is mapped correctly rbd showmapped

Deploying VMs with Block Storage

Using Block Storage with KVM/QEMU Virtual Machine images are a common use of Ceph block storage Direct support from KVM/QEMU Copy on Write (COW) snapshot support makes it easy to provision many VMs efficiently Preferred format is raw disk Other formats such as qcow are supported but introduce additional overhead and can cause problems with live VM migration if caching is enabled Creating QEMU image files on rbd is supported directly from the command line and from virt-manager via libvirt

Command: qemu-img Syntax: qemu-img command OPTIONS rbd-information Option Description create -create a qemu image file resize -change the size of a qemu image file info -view details of an image file convert -change format of an image file such as qcow to raw This is NOT a ceph specific command, it is part of the qemu project To use with rbd, provide the rbd specific details in place of the more usual filename qemu-img create -f raw rbd:poolname/imagename size Notes:

Snapshots with rbd

Creating Snapshots with rbd An rbd snapshot is a read-only copy of an rbd image taken at a given point in time Snapshots can be rolled back to restore the file-system state Difference files can be created to provide a delta between two snapshot states Similar to btrfs and ZFS functionality

Command: rbd snap Syntax: rdb snap OPTIONS <pool/image> <@snapshot> Option Description create -create a snapshot of an rbd image ls -list snapshots in a pool rollback -return the rbd image to an earlier snapshot state rm -delete a given snapshot purge -delete all snapshots of an image Notes:

Creating a snapshot rbd snap create <poolname>/<image-name>@<snapname> Creates a snapshot of the image, with the name snapname snapname is used to identify the individual snapshot Notes:

Restoring a snapshot rbd snap rollback<poolname>/<image- name>@<snapname> Restores the filesystem to the condition at the time <snapname> was created Need to remount filesystem to update filesystem status Notes:

Copy On Write Clones

Copy on Write Also known as 'snapshot layering' Requires the rbd image to be created as a format 2 image --image format 2 option when creating the rbd image COW Images are fully independent and writeable But require 'parent' snapshot to be present RBD Image (VM OS Base) Snapshot The graphic shows the common use case of hosting multiple VM systems from a single golden base image. Clone images VMs

Copy on Write - dependency Parent snapshot is required to remain present and unchanged Deletion of the parent snapshot will break all COW clones RBD Image (VM OS Base) Snapshot Notes: Clone images VMs

Creating a copy-on-write clone First, ensure that the rbd image is format 2 rbd snap –pool <poolname> --image <imagename> info Before creating any clones, protect the parent snapshot Deletion or changing the source snapshot will break all COW clones rbd snap protect <poolname>/<image-name>@<snapname> Create a COW Clone Basically a writeable snapshot rbd clone <sourcepool>/<sourceimage>@<snapshot> <destinationpool>/<destinationimage> Cloning is almost instantaneous Only a pointer is created back to the source snapshot

View details of a COW Clone rbd COW clones store details of their parent snapshot and how much data has been changed due to writes to the clone Use the info option to view the details of a COW clone Parent snapshot is listed Overlap is also listed to show overlap between clone and parent

Flattening COW Clones The COW clone approach is very effective Rapid creation of each clone Small disk overhead when a clone is created Only pointers to parent No additional data until changed data is written Sometimes there is a need to remove a COW clone from the dependency on the parent shapshot Perhaps to move a VM to a different host system Flatten option combines the parent snapshot with the clone image to make the COW Clone a fully independent rbd image Parent snapshot is unaffected, as are any other COW clones based on the same parent

COW Clone Flattening Use the rbd flatten command to convert from a COW Clone to a full, standalone rbd image rbd flatten <pool>/<imagename> Writes all data from the parent snapshot to the image Takes time and bandwidth as the entire image needs to be recreated rbd image remains available during the process as do all other COW images based on the same parent

Section 4 Exercises Objective Notes: