Linux Software RAID & LVM Patrick Ladd pladd@optonline.net 5/2/2007

Terminology : RAID RAID: Redundant Array of Inexpensive Disks Combine 2 or more smaller disks into one larger disk Why? Redundancy Reduce or eliminate exposure to disk failure Performance More data under the drive heads Ease of management Single disk simplifies file system layout Capacity: n: Number of disks s: Size of disks

Software RAID vs Hardware RAID Implemented at kernel level in OS Pros: More flexible Hardware independence Cheap Cons: Requires properly configured kernel Uses system resources Hardware RAID Built into card or motherboard Pros: Faster Battery backed option Cons: Hardware dependency Drivers? Expensive

RAID 0: Striping Not really RAID No redundancy Performance: Best Capacity: Best (n x s) Reliability: Worst

RAID 1: Mirroring Fastest Least efficient disk usage Disks added in pairs Performance: Best Capacity: Worst (n x s) / 2 Reliability: Best

RAID 5: Striping with Distributed Parity Efficient space usage Good balance of features Requires n >= 3 Performance: Good Capacity: Good (n-1) x s Reliability: Good

Layered RAID RAID 51 RAID 10 RAID 01 RAID made up of RAID disks Examples: RAID 01 / 0+1 – RAID 1 of RAID 0's RAID 10 – RAID 0 of RAID 1's RAID 05 – RAID 0 of RAID 5's RAID 50 – RAID 5 of RAID 0's RAID 51 – RAID 5 of RAID 1's Combine benefits of multiple RAID setups Hopefully to offset the drawbacks Tends to be storage inefficient RAID 10 RAID 01

LVM – What & Why LVM: Logical Volume Manager Abstract the interface to storage devices Separate physical layer from file system layers Why? Ease of Management Flexible partitioning Human readable disk names Data migration flexibility Snapshots Alternate way to do RAID 0 (Striping)

LVM Objects Physical Volume (PV) Represents physical hard drive partition Broken up into Physical Extents (PE) Logical Volume (LV) Virtual “partition” for O/S Broken up into Logical Extents (LE) Volume Group (VG) Collection of PVs and LVs Maps LEs to PEs

LVM Architecture OS LV LV LV LV LV VG VG PV PV PV PV PV Hard Drive

LVM Metadata UUID - Unique identifier stored in every VG, LV, PV Header contained at start of every PV UUIDs of all other PVs UUIDs of all LVs Allocation map of PE-->LE

Mapping Modes Linear: Data from LV laid out sequentially in PVs Striped: Data laid out alternately among 2 or more PVs Possibly some performance improvement if PVs are on separate physical hardware Concatenation: LVM2 allows extending LVs using one of the above two mapping modes with more data in another layout

Ubuntu Install Has LVM / RAID support Must be done from alternate ISO image (i.e. ubuntu-6.10-alternate-i386.iso) Must be done in text mode

Fedora Core 6 Install LVM / RAID support Includes Anaconda graphical installer support

LVM Common Commands Display object information - [pv|vg|lv]display Create object - [pv|vg|lv]create Delete object – [pv|vg|lv]remove Change object – [pv|vg|lv]change Display metadata– [pv|vg|lv]s Rename – [vg|lv]rename

pvdisplay Output [root@laptop ~]# pvdisplay --- Physical volume --- PV Name /dev/hda3 VG Name MainVG PV Size 7.79 GB / not usable 9.59 MB Allocatable yes (but full) PE Size (KByte) 32768 Total PE 249 Free PE 0 Allocated PE 249 PV UUID XH408a-y8Pz-RoHs-rEOD-RZcW-AFwB-vytZ5I

vgdisplay Output [root@laptop ~]# vgdisplay --- Volume group --- VG Name MainVG System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 8 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 3 Max PV 0 Cur PV 1 Act PV 1 VG Size 7.78 GB PE Size 32.00 MB Total PE 249 Alloc PE / Size 249 / 7.78 GB Free PE / Size 0 / 0 VG UUID JYEdd7-mKC2-qAqS-1jsP-EWD4-UGVC-ZGn07n

lvdisplay Output [root@laptop ~]# lvdisplay --- Logical volume --- LV Name /dev/MainVG/RootLV VG Name MainVG LV UUID DgwUgn-wb2F-K9Kn-PKcj-Ra0p-CwoO-FHpH44 LV Write Access read/write LV Status available # open 1 LV Size 6.56 GB Current LE 210 Segments 3 Allocation inherit Read ahead sectors 0 Block device 253:0

Resizing a Partition Pre LVM2 – e2fsadm Nice tool to automatically take filesystem offline, resize logical volume, then resize filesystem, put volume back online Now - has to be done manually umount /tmp lvresize +10M /dev/MainVG/TmpLV ext2resize /tmp mount /tmp

Adding a drive Give drive a partition table Safer – prevents other OS that don't recognize LVM from doing strange things to it Setup PV on partition (pvcreate) Add PV to VG (vgextend)

Converting to LVM Add 2nd Drive Set up LVM on it Mount it Copy contents of existing drive to LVM Boot to new LVM image Ensure things work Add old drive as LVM physical volume Extend LVM onto old drive

Snapshots Create an exact copy of a LV Can be read-only or read-write (default) Caution: copy on write means that snapshot must be large enough to handle writes lvcreate -L 500M -s -n BackupLV /dev/MainVG/TmpLV

Uses for Snapshots Read only Good for backups Make sure applications / file system are consistent Force write of data to disk sync file system Read/Write Speculative experiments Try something out, if it doesn't work, remove snapshot and try again Virtual Machine images?

Sources Packages Needed RAID Support in kernel mdadm LVM - Current incarnation is LVM2 http://sources.redhat.com/lvm2/ Device mapper in kernel (most kernels built in) Device mapper in user space (device-mapper) Device mapper support library (libdevmapper) LVM Tools (lvm2) HOWTOs

EVMS EVMS – Enterprise Volume Management System http://evms.sourceforge.net/ Competitor to LVM Integrated support for RAID 0,1,4,5 Handles LVM & LVM2 volumes Better user-space tools LVM has better internals Not included in most pre-built kernels Has a nice GUI – works with LVM

EVMS GUI

EVMS Text GUI