1. Section 4 Block Storage with SES

2. Objectives Understand Block Storage in SES
Configure storage for use as a block device
Access storage from a Linux host

3. Understand Block Storage
Objective Notes:

4. What is Block Storage
SES supports clients accessing the RADOS storage cluster as a block device
Provides clients with access to the cluster as if it was a disk drive
Supported in the Linux kernel since
Supported by KVM/QEMU
16Eb file system sizes
Thin provisioning
Snapshots
Copy on Write clones

5. Architecture Block devices Up to 16 EiB Thin Provisioning Snapshots
(Kernel module)
librados
RADOS Cluster

6. RBD Storage VM VM Storage Block Device QEMU/KVM Linux Kernel librbd
Kernel module
LIBRADOS

7. Configure Block Storage

8. Configure a Block Device
To use block storage in Ceph, an rbd image needs to be created
This image is then mapped to the host as a block device, appearing under the /dev filesystem
/dev/rbdn
rbd images can be created in different formats
Format 1 was the original format
Format 2 is the default from Jewel onwards and supports copy-on-write clones
Default can be changed if required
Add the following lines to ceph.conf to set the rbd default
[client]
rbd default image format = 1 or 2 as required

9. Command: rbd Syntax: rbd OPTIONS command Option Description
create -create an rbd image in the default pool
info -view details of an existing rbd image
map connect a client node to an rbd image
showmapped -display local /dev/rbdx → rbd image mapping
Notes:

10. Creating a new rbd image
If required, first create a new pool on the RADOS cluster
Or use the default or an existing pool
Use the rbd create option to make the new rbd image
rbd create -p <poolname> <rbdimagename> --size <xxxx> --image format <1|2>
If no pool is specified the default rbd pool will be used
Size can be given in various units (MB, GB etc). Defaults to MB
Check that the image is created
rbd list

11. Access Storage from a Linux Host

12. Accessing an rbd image
Devices are mapped using the pool and image names
The rbd map option is used to configure access
rbd map -p <poolname> <rbdimagename>
If no pool is specified the default rbd pool will be used
Check that the image is mapped correctly
rbd showmapped

13. Deploying VMs with Block Storage

14. Using Block Storage with KVM/QEMU
Virtual Machine images are a common use of Ceph block storage
Direct support from KVM/QEMU
Copy on Write (COW) snapshot support makes it easy to provision many VMs efficiently
Preferred format is raw disk
Other formats such as qcow are supported but introduce additional overhead and can cause problems with live VM migration if caching is enabled
Creating QEMU image files on rbd is supported directly from the command line and from virt-manager via libvirt

15. Command: qemu-img Syntax: qemu-img command OPTIONS rbd-information
Option Description
create -create a qemu image file
resize -change the size of a qemu image file
info -view details of an image file
convert -change format of an image file such as qcow to raw
This is NOT a ceph specific command, it is part of the qemu project
To use with rbd, provide the rbd specific details in place of the more usual filename
qemu-img create -f raw rbd:poolname/imagename size
Notes:

16. Snapshots with rbd

17. Creating Snapshots with rbd
An rbd snapshot is a read-only copy of an rbd image taken at a given point in time
Snapshots can be rolled back to restore the file-system state
Difference files can be created to provide a delta between two snapshot states
Similar to btrfs and ZFS functionality

18. Command: rbd snap
Syntax: rdb snap OPTIONS <pool/image>
Option Description
create -create a snapshot of an rbd image
ls list snapshots in a pool
rollback -return the rbd image to an earlier snapshot state
rm delete a given snapshot
purge -delete all snapshots of an image
Notes:

19. Creating a snapshot
rbd snap create
Creates a snapshot of the image, with the name snapname
snapname is used to identify the individual snapshot
Notes:

20. Restoring a snapshot
rbd snap rollback<poolname>/<image-
Restores the filesystem to the condition at the time <snapname> was created
Need to remount filesystem to update filesystem status
Notes:

21. Copy On Write Clones

22. Copy on Write Also known as 'snapshot layering'
Requires the rbd image to be created as a format 2 image
--image format 2 option when creating the rbd image
COW Images are fully independent and writeable
But require 'parent' snapshot to be present
RBD Image
(VM OS Base)
Snapshot
The graphic shows the common use case of hosting multiple VM systems from a single golden base image.
Clone images
VMs

23. Copy on Write - dependency
Parent snapshot is required to remain present and unchanged
Deletion of the parent snapshot will break all COW clones
RBD Image
(VM OS Base)
Snapshot
Notes:
Clone images
VMs

24. Creating a copy-on-write clone
First, ensure that the rbd image is format 2
rbd snap –pool <poolname> --image <imagename> info
Before creating any clones, protect the parent snapshot
Deletion or changing the source snapshot will break all COW clones
rbd snap protect
Create a COW Clone
Basically a writeable snapshot
rbd clone <destinationpool>/<destinationimage>
Cloning is almost instantaneous
Only a pointer is created back to the source snapshot

25. View details of a COW Clone
rbd COW clones store details of their parent snapshot and how much data has been changed due to writes to the clone
Use the info option to view the details of a COW clone
Parent snapshot is listed
Overlap is also listed to show overlap between clone and parent

26. Flattening COW Clones The COW clone approach is very effective
Rapid creation of each clone
Small disk overhead when a clone is created
Only pointers to parent
No additional data until changed data is written
Sometimes there is a need to remove a COW clone from the dependency on the parent shapshot
Perhaps to move a VM to a different host system
Flatten option combines the parent snapshot with the clone image to make the COW Clone a fully independent rbd image
Parent snapshot is unaffected, as are any other COW clones based on the same parent

27. COW Clone Flattening
Use the rbd flatten command to convert from a COW Clone to a full, standalone rbd image
rbd flatten <pool>/<imagename>
Writes all data from the parent snapshot to the image
Takes time and bandwidth as the entire image needs to be recreated
rbd image remains available during the process as do all other COW images based on the same parent

28. Section 4 Exercises
Objective Notes: