1. Device Mapper tools: RAID, Logical Volume Manager, and LUKS encryption Alan Porter 2011-01-13 Triangle Linux Users Group

2. « RAID · LVM · LUKS » device mapper · losetup Device Mapper Maps one block device to another one. Does some sort of transformation position of blocks (RAID, LVM, kpartx, multipath) scrambling content (LUKS, cryptoloop, Truecrypt) caching content (dmcache)

3. « RAID · LVM · LUKS » device mapper · losetup Get familiar with block tools without using a bunch of disks. Loopback devices $ dd if=/dev/zero of=disk1 bs=1M seek=1023 count=1 $ losetup /dev/loop1 disk1 $ losetup -a /dev/loop1: [fc02]:1848179 (/home/alan/disk1) $ losetup -d /dev/loop1

4. « RAID · LVM · LUKS » levels · mdadm · failures · spares What RAID is: A “high-availability” feature. Insurance against disk failure. Used when you can not afford downtime when a disk goes bad. Sometimes used for speed. What RAID is NOT: Insurance against fat-fingering a file (rm *). A backup.

5. « RAID · LVM · LUKS » levels · mdadm · failures · spares RAID Levels 0 - striping, used for performance A single disk failure will destroy the array. 1 - mirroring, same data on all disks All but one device is allowed to fail. 4 - striped array with a parity device One device is allowed to fail. 5 - striped array with distributed parity One device is allowed to fail. 6 - striped array with dual distributed parity. Two devices are allowed to fail.

6. « RAID · LVM · LUKS » levels · mdadm · failures · spares “mdadm” syntax $ mdadm -C /dev/md0 --level=5 --raid-devices=/dev/loop1 /dev/loop2 /dev/loop3 note - 'missing' is a possible device Then mount the /dev/md0 device. $ mount /dev/md0 /mnt/mountpoint

7. « RAID · LVM · LUKS » levels · mdadm · failures · spares Dealing with failed disks: To remove a failed device from the array: $ mdadm /dev/md0 -r /dev/loop1 To add the fixed drive back: $ mdadm /dev/md0 -a /dev/loop1 To simulate a drive failure: $ mdadm /dev/md0 --fail /dev/loop1

8. « RAID · LVM · LUKS » levels · mdadm · failures · spares Spare disks Can be specified when the array is created with the --spare-devices=N option. $ mdadm --create /dev/md1 --level=raid5 --raid-devices=3 --spare- devices=1 /dev/loop1 /dev/loop2 /dev/loop3 /dev/loop4 Can also be added at a later time with the hotadd option. $ mdadm /dev/md1 -a /dev/loop5

9. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Logical Volume Manager Allocate disk space as you need it. (Then resize filesystems that sit on LV's) You can move volumes from disk to disk. Cloning & snapshots.

10. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean

11. Physical volumes $ pvcreate /dev/loop1 /dev/loop2 /dev/loop3 $ pvscan PV /dev/loop1 lvm2 [1.00 GiB] PV /dev/loop2 lvm2 [1.00 GiB] PV /dev/loop3 lvm2 [1.00 GiB] Total: 3 [3.00 GiB] / in use: 0 [0.00 GiB] / in no VG: 3 [3.00 GiB]

12. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Volume groups $ vgcreate trilug /dev/loop1 /dev/loop2 Volume group "trilug" successfully created $ vgscan Reading all physical volumes. This may take a while... Found volume group "trilug" using metadata type lvm2

13. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean “vgdisplay” tells more info than “vgscan” $ vgdisplay --- Volume group --- VG Name trilug System ID Format lvm2 Metadata Areas 3 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 3 Act PV 2 VG Size 1.99 GiB PE Size 4.00 MiB Total PE 510 Alloc PE / Size 0 / 0 Free PE / Size 510 / 1.99 GiB VG UUID odpp2n-XmED-AOq6-qpMH-tMYv-7PnX-vEU5WQ

14. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Logical volumes $ lvcreate trilug --name=music --size=1.5G Logical volume "music" created $ lvscan ACTIVE '/dev/trilug/music' [1.50 GiB] inherit

15. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Using your new Logical Volume (It's just like any other block device) $ mkfs.ext4 /dev/trilug/music $ mkdir mnt ; mount /dev/trilug/music mnt

16. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Moving stuff around $ vgextend /dev/trilug /dev/loop3 Volume group "trilug" successfully extended $ vgreduce /dev/trilug /dev/loop1 Physical volume "/dev/loop1" still in use $ pvmove /dev/loop1 /dev/loop1: Moved: 100.0% $ vgreduce /dev/trilug /dev/loop1 Removed "/dev/loop1" from volume group "trilug"

17. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Practical example #1: Replacing a disk - minimal downtime Plug in USB drive, add PV Move stuff from old HD to USB Remove old HD from VG Replace HD Reboot (carefully, with a Live CD) Add new HD to VG Move stuff to new HD Remove USB drive from VG

18. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Practical example #2: “fdisk” a HD without rebooting - while it's in use! Add a USB drive as a new PV. Pvmove the volume group off of the HD. Remove the HD from the VG Run fdisk. Then do it all again in reverse (vgextend, pvmove, vgreduce, pvremove) The system stays up the whole time!

19. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Snapshots Creates a “copy on write” or “diffs” area. When a block is written to the original volume, the original block is stored in the “diffs” area. $ lvcreate -s /dev/trilug/music --name=snap --size=100M Logical volume "snap" created $ lvscan ACTIVE Original '/dev/trilug/music' [1.50 GiB] inherit ACTIVE Snapshot '/dev/trilug/snap' [100.00 MiB] inherit $ dd if=/dev/trilug/snap of=/dev/tape # takes 4 hours $ lvremove /dev/trilug/snap

20. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Visualization tools system-config-lvm

21. « RAID · LVM · LUKS » pv · vg · lv · move · snap · tools · clean Cleaning up LVM - destructive! $ umount /dev/trilug/music $ lvremove /dev/trilug/music $ vgremove trilug $ pvremove /dev/loop{1,2,3}

22. « RAID · LVM · LUKS » format · mount · unmount · info · keys Linux Unified Key Setup (LUKS) LUKS is a standard for the disk encryption. It specifies how the data is laid out on the disk. LUKS can be used on any OS. Some Linux distros (Ubuntu “alternate”) support installing on LUKS partitions.

23. « RAID · LVM · LUKS » format · mount · unmount · info · keys Formatting a disk or partition (method 1): $ sudo cryptsetup luksFormat /dev/loop1 Formatting a disk or partition (method 2): $ sudo cryptsetup luksOpen /dev/loop1 mylabel $ sudo mkfs.vfat /dev/mapper/mylabel $ sudo cryptsetup luksClose mylabel Formatting a disk or partition (method 3): $ sudo luksformat -t ext2 /dev/loop1

24. « RAID · LVM · LUKS » format · mount · unmount · info · keys Mounting a LUKS partition (method 1): $ sudo cryptsetup luksOpen /dev/loop1 mylabel $ sudo mount -t ext2 /dev/mapper/mylabel /mnt/mylabel Mounting a LUKS partition (method 2): $ sudo pmount /dev/loop1 mylabel

25. « RAID · LVM · LUKS » format · mount · unmount · info · keys Unmounting a LUKS partition (method 1): $ sudo umount /mnt/mylabel/ $ sudo cryptsetup luksClose mylabel Unmounting a LUKS partition (method 2): $ sudo pumount mylabel

26. « RAID · LVM · LUKS » format · mount · unmount · info · keys Showing info about a LUKS partition: $ sudo cryptsetup luksDump /dev/loop1 $ sudo cryptsetup luksUUID /dev/loop1 $ sudo cryptsetup isLuks /dev/loop1 ; echo $? What? There are EIGHT keys?

27. « RAID · LVM · LUKS » format · mount · unmount · info · keys LUKS passwords The disk/partition is encrypted using a master key. Eight user keys are stored on the disk. Any user key can be used to decrypt the master key, and thus access the disk/partition. Changing a LUKS password (user key): $ sudo cryptsetup luksAddKey /dev/loop1 $ sudo cryptsetup luksDelKey /dev/loop1

28. « RAID · LVM · LUKS » closing thoughts · cleanup · questions Recovery Boot Ubuntu Live CD, then “apt-get” necessary packages (mdadm, lvm2, cryptsetup, pmount) Make a bootable USB (“startup disk creator”), boot into it, apt-get the needed packages. They will be saved in the “reserved extra space” (an overlay filesystem that is added to the static filesystem image from the ISO). Be careful... there are lots of ways to very quickly get into trouble! Practice with loop devices before jumping into your main HD.

29. « RAID · LVM · LUKS » closing thoughts · cleanup · questions Cleaning up our loop devices $ losetup -d /dev/loop{1,2,3} $ rm disk1 disk2 disk3

30. « RAID · LVM · LUKS » closing thoughts · cleanup · questions Questions?