Presentation is loading. Please wait.

Presentation is loading. Please wait.

OReilly Open Source Convention 2000 Linux Filesystems for Real-World Needs Æleen Frisch e x ponential Consulting,

Similar presentations


Presentation on theme: "OReilly Open Source Convention 2000 Linux Filesystems for Real-World Needs Æleen Frisch e x ponential Consulting,"— Presentation transcript:

1 OReilly Open Source Convention 2000 Linux Filesystems for Real-World Needs Æleen Frisch e x ponential Consulting, LLC North Haven, Connecticut, USA

2 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 2 Itinerary u The Kernels VFS u Local Filesystems u Network Shared Filesystems u Advanced Filesystems u Futures

3 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 3 Virtual File System (VFS) u Kernel subsystem u Provides a consistent interface for low-level file I/O u Filesystem needs only use that API and provide specified functionality, and it is automatically supported.

4 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 4 Filesystems for Local Disks u Lots of supported types: v ext2 v CD-ROM: iso9660, Microsoft Joliet extensions v ufs, fat, vfat, umsdos, ntfs, sysv, affs, adfs, hfs, hpfs, qnx4, … v minix v procfs v /dev/pts

5 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 5 Journaled Filesystems u ReiserFS in SuSE 6.3 u ext3 u dtfs: v u SGIs xfs being ported

6 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 6 Network Filesystems u NFS u AFS u smbfs u ncpfs

7 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 7 Linux NFS u Longtime weak spot v Unreliable v Poor performance F nfsstones benchmark v Unable to communicate with non-Linux clients u knfsd is what you want to use v Need to upgrade all current distributions as delivered to latest available version F Kernel patches F New daemon F New utilities

8 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 8 Production Filesystem Features u Bigger than one disk partition (flexibility) u Faster I/O u Fault tolerant u Distributed v Load balancing u Enhanced Security v Access Control Lists v Encryption

9 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 9 Logical Volumes u Dynamically-resizable filesystem consisting of multiple, independent disk partitions (physical volumes), upon which a virtual structure is imposed: v Volume groups--virtual disks-- divisible into … v Logical volume--virtual partitions--which hold … v Filesystems u Common in modern OSs: AIX, HP, True64 u Logical Volume Manager (lvm): alpha code u Veritas Volume Manager: $$$$

10 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 10 lvm u linux.msede.com/lvm/ u Basics seem to work, but still Alpha code (version 0.7) u Maximum (theoretical) filesystem size: > 1000 TB

11 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 11 lvm Commands u pv: create, change, display u vg: create, change, display, ck, cfgbackup/restore, export, extend, reduce, remove, split u lv: create, change, display, extend, reduce, remove v e2fsadm (needs PartitionMagics resize2fs)

12 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 12 lvm Example u Run fdisk to set partition type to 0xFE u pvcreate /dev/sdb1 /dev/sdc1 u vgcreate new_vg /dev/sd{b,c}1 v /dev/new_vg/group v /etc/lvmconf/new_vg.conf u lvcreate –L 2g –n new_lv –r 4 –C y u mke2fs /dev/new_vg/new_lv u mount /dev/new_vg/new_lv /somewhere u e2fsadm –L +500m /dev/new_vg/new_lv v vs. lvextend

13 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 13 RAID u Redundant Arrays of Inexpensive Disks u Choices v Software v Hardware F Controller F RAID device

14 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 14 RAID Levels u 0: Disk stripingperformance only v Best large transfer I/O bandwidth v No loss of storage capacity u 1: Disk mirroring v Best data redundancy v Good performance on small transfers u 3: Disk striping with parity disk v Optimizes I/O bandwidth for large transfers v Low impact of single disk failure F Obsolete! u 5: Parity block rotates among component disks v Optimizes I/O operations/sec v Not vulnerable to single disk failure

15 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 15 Linux RAID u 0, 1, 4(!), 5 u Linear u Kernel support must be enabled v Patches needed (still): F ftp.*.kernel.org/pub/linux/daemons/raid/alpha F RedHat installs for you u Special files: /dev/mdx u Configuration file: /etc/raidtab v mkraid [device] u Persistent superblock v Automatic detection v raidstart and raidstop to control manually

16 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 16 Kernel Support

17 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 17 Sample /etc/raidtab Entries v Raid 0: Striping raiddev /dev/md0 raid-level 0 nr-raid-disks 2 chunk-size 64 persistent-superblock 1 device /dev/sdc1 raid-disk 0 device /dev/sdd1 raid-disk 1 v Raid 1: Mirroring raiddev /dev/md0 raid-level 1 nr-raid-disks 2 persistent-superblock 1 device /dev/sdc1 raid-disk 0 device /dev/sdd1 raid-disk 1

18 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 18 Mirroring a Striped Disk: RAID 10 raiddev /dev/md0 raid-level 0 nr-raid-disks 2 chunk-size 64k … device /dev/sdc1 raid-disk 0 device /dev/sdd1 raid-disk 1 raiddev /dev/md1 raid-level 0 nr-raid-disks 2 chunk-size 64k device /dev/sde1 raid-disk 0 device /dev/sdf1 raid-disk 1 raiddev /dev/md2 raid-level 1 nr-raid-disks 2 device /dev/md0 raid-disk 0 device /dev/md1 raid-disk 1 u But why?

19 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 19 RAID 5 raiddev /dev/md0 raid-level 5 nr-raid-disks 3 nr-spare-disks 1 persistent-superblock 1 device /dev/sdc1 raid-disk 0 device /dev/sdd1 raid-disk 1 device /dev/sde1 raid-disk 2 device /dev/sdf1 spare-disk 0

20 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 20 General RAID Considerations u Stripe size matters! v Best value depends on typical I/O transfer size v Testing: trial and error u Underlying filesystem block size = 4K v mke2fs –b 4 … u Dont overload controllers

21 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 21 Coda u Follow-on to the Andrew File System (AFS) v u Distributed filesystem: v /coda on client can have components (volumes) from all over the network (transparent to user) u Lots of metadata, stored in a database, using logging and other transaction-based procedure to ensure integrity u Complex configuration task

22 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 22 Coda Features u Read-write data replication v Automatic modification propogation and synchronization/repair u Caching (performance, network failures, mobile computing) v Client-requested files F Automatic update requests v Client-modifications when server is unavailable; attempts automatic synchronization of conflicts u Kerberos-like authentication tokens u Access control lists

23 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 23 Intermezzo u u Distributed file system v Designed for high availability v Palimpsest on Coda

24 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 24 Global File System (GFS) u u Network-shared storage

25 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 25 Access Control Lists u POSIX ACLs u Alpha code v Recent call for testing … u Specifies file access on per-user/group basis u ACLs inherited from parent directory u Deny>Allow (null=>deny)

26 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 26 Enabling ACLs u Patch and build kernel v Prompt for development option u Get/build utilities u Patch and build ext2fs utilities u mount … -o acl u setfacl and getfact u Do frequent backups (files and ACLs)

27 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 27 Encryption u Encryption v Steganography F Filesystem hidden in the low bits of each byte of an audio file u EFS v ftp.csua.berkeley.edu/pub/cypherpunks/ filesystems/linux u PPDD v linux01.gwdg.de/~alatham/ppdd.html

28 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 28 Setup u Patch/configure/build kernel u Get/build utilities u Patch/rebuild standard tools u dd to create file u losetup to set algorithm and pass phrase u mke2fs

29 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 29 Using an Encrypted filesystem u fstab: device mount-pt ext2 defaults,noauto,loop,encryption=type 0 0 u Prompted for pass phrase when filesystem is mounted

30 Copyright © , Exponential Consulting, LLC Linux Filesystems for Real-World Needs 30 Filesystem Futures


Download ppt "OReilly Open Source Convention 2000 Linux Filesystems for Real-World Needs Æleen Frisch e x ponential Consulting,"

Similar presentations


Ads by Google