Presentation is loading. Please wait.

Presentation is loading. Please wait.

Windows File Systems and Improving Performance APCUG 1/5/2008 Greg Hayes Raxco Software

Similar presentations

Presentation on theme: "Windows File Systems and Improving Performance APCUG 1/5/2008 Greg Hayes Raxco Software"— Presentation transcript:

1 Windows File Systems and Improving Performance APCUG 1/5/2008 Greg Hayes Raxco Software

2 Introduction Company Background –Raxco founded 1978 –Largest provider of OpenVMS system management software –Spun off Axent Technologies (now part of Symantec) in 1996 –First Windows product released in 1998

3 Introduction Company Background –Microsoft Gold Certified Partner –Microsoft ISV –Member of Microsoft Development Network –Software used by some of the largest companies and government organizations in the world

4 Introduction Speaker background –System manager for 10+ years –Raxco customer for 10 years –Started at Raxco in 1996 –Manager, Technical Solutions –Microsoft MVP 2003-2007 - Windows File Systems –Online communities –Speaking/Education

5 Topic Background Who remembers their first PC? PC hardware performance continues to rise –CPU clock speeds > 2Ghz –2GB RAM –500GB+ capacity hard drives Limiting factor is still disk I/O Anything that speeds up access to disk improves performance

6 Windows File Systems FAT16 FAT32 exFAT (new) NTFS

7 Windows File Systems FAT16/32 The File Allocation Table is created when you format a FAT drive and contains information about all of the files found on the drive and where they logically reside. There are actually 2 FATs per partition.

8 Windows File Systems – FAT16 Benefits: –Faster performance on smaller partitions - less than 4GB –Best Performance for Pagefile Drawbacks: –Little or no security –High rate of file system corruption –Limits on partition size (FAT16 – 4GB)

9 Windows File Systems – FAT16 FAT16 –4GB FAT16 partition has a cluster size of 64k. –Wastes space as even the smallest files take up a full cluster. –Maximum file size is 2GB –Files per volume 65,536

10 Windows File Systems – FAT32 FAT32 –Maximum file size is 4GB –Allows smaller cluster sizes to be used – resulting in less space “waste” –Maximum partition size is 32GB (as formatted by XP/Vista/2K3) –Up to 2TB can be written/read (XP/Vista/2K3) –Files per volume ~4,177,920

11 Windows File Systems – exFAT exFAT –“Extended FAT” –Designed for flash media – exchange media with PCs. –Free space bitmap for faster allocation –Maximum theoretical file size is 16EB –More than 1000 files per folder

12 Windows File Systems - NTFS Journaling/Transaction Based File System “Self Describing” File System Metadata - data that describes data

13 Windows File Systems - NTFS Benefits: –Better security –Better performance with large partitions – greater than 4GB –Better resistance to file system corruption Drawbacks: –Can’t natively boot to MSDOS and get access to partition

14 Windows File Systems - NTFS NTFS –Maximum file size is approx 16TB –Maximum partition size is 2TB – 256TB (depends on cluster size used) –Files per volume 4,294,967,295

15 Windows File Systems - NTFS NTFS –Disk Quotas (Win2K and newer) –File/Folder compression built into the file system –Encryption built into the file system (Win2K and newer) –Volume Shadow Copy (VSS – WinXP and newer) –txNTFS

16 NTFS Metadata $MFT - Master File Table (0) –The $MFT contains at least 1 record for each file that exists on the partition. $MFTMirr – Master File Table Mirror (1) –Duplicate of the 1 st 4 records of the $MFT –Used for redundancy/recoverability

17 NTFS Metadata $LogFil – NTFS Transaction Log (2) –Updates to the file system first get posted to the transaction log –Gives NTFS its “self-healing” abilities $Volume – Volume Information (3) –Volume Label/Version

18 NTFS Metadata $AttrDef – Attribute Definitions (4) –Names, Numbers, Descriptions $Root – (5) –Root folder on the drive. (I.e. C:\)

19 NTFS Metadata $Bitmap – Cluster Bitmap (6) –Indicates whether a cluster is used or empty –Uses this to quickly find free space $Boot – Partition Boot Sector (7) –Location of the partition boot sector –Bootstrap loader code if bootable drive

20 NTFS Metadata $BadClus – Bad Cluster File (8) –Clusters the file system has reported as bad –Updated by CHKDSK $Secure – Security Descriptors (9) $Upcase – Upcase Table (10) –Translation of lowercase characters to their equivalent Unicode upper case characters

21 NTFS Metadata $Extend – Optional Extensions (11) –User Change Journal –Quotas

22 Anatomy of a $MFT Record Fixed size records – 1K Attribute Records –$FILE_NAME 8.3 file name and parent directory Long File Name –$DATA Extent information for file. Includes Logical Cluster Number (LCN) and Run Length $DATA record contains information on file size

23 Anatomy of a $MFT Record DiskEdit

24 MFT Reserved Zone Created when partition is formatted $MFT will “grow into” as needed File system will keep free if possible Defragmenters can not use free space inside of MFT Reserved Zone (NT4/Win2K)

25 MFT Reserved Zone NT4 –Fixed Size –12.5% by default –Can be increased up to 50% via registry key

26 MFT Reserved Zone Windows 2003/2000/XP/Vista/2008 –Dynamically created every time partition is mounted –First record of $MFT to first non-free cluster - up to a default max of 12.5%

27 Logical vs Physical Clusters Logical Clusters –File system level –Every partition starts at logical cluster 0 –No idea of hard drive technology in use IDE SCSI RAIDx # platters or read/write heads

28 Logical vs Physical Clusters Physical Clusters –Hard drive level –Hard drive controller translates logical-to- physical and positions heads


30 Cluster Size and Performance Smaller clusters –less wasted space –Worse performance – especially large files Larger clusters –more wasted space –Better performance – especially large files

31 Conversion from FATx to NTFS NT4/Win2K –Results in 512byte clusters –Not “the best” for performance - especially with video/image applications WinXP/Vista –Will try to convert using 4k clusters. –Best for general file system performance.

32 OEM Vendors and Cluster Size IBM/Dell/HP –Provide Software to install/configure Windows Server –Formats system drive with 512byte cluster –Absolute worse for system drive performance –Only way to convert is 3 rd party tools and have server unavailable for extended period (HP SmartStart, Dell OpenManage, IBM ServerGuide)

33 Cluster Size and Performance Larger cluster sizes result in better performance when reading large files (video,audio,images) The larger the cluster size, the more space can be “wasted” if you store small files on the partition With cluster sizes greater than 4k under NT4 and Win2K, you will NOT be able to defragment With cluster sizes greater than 4k, you will not be able to use NTFS compression

34 Biggest Cause of Poor File System Performance Fragmentation!

35 Fragmentation Causes What causes fragmentation? –Occurs when files are created, extended or deleted –Happens regardless of how much free space is available (After XP/SP2 installation – 944 files/2943 fragments) –More than one Logical I/O request has to be made to the hard drive controller to access a file

36 Fragmentation Impacts What does fragmentation do to my system? –Slows down access to files –Extra CPU/Memory/Disk resource usage –Some applications may not run –Slow system boot/shutdown –Audio/Video record/playback drops frames or “skips”

37 Measuring Impact of Fragmentation Measuring the performance loss in reading a fragmented file

38 What Can I Do About Fragmentation? You can’t stop fragmentation from happening (you can only slow it down)! What you CAN do is to defragment

39 Defragmenting - Results What does defragmenting do? –Locates logical pieces of a file and brings them together Faster to access file and takes less resources Improves read performance –Consolidates free space into larger pieces New files get created in 1 piece Improves write performance

40 Measuring Impact of Fragmentation Measuring the performance difference in reading a contiguous file

41 Defragmenting - Issues to Consider Safety, Safety, Safety! –No loss of data –No corruption of data

42 Defragmenting - Issues to Consider Free Space –How much is enough? –Where is free space located? Inside MFT Reserved Zone Outside of MFT Reserved Zone –Consolidation of free space

43 Safety - Microsoft’s Defrag APIs Provided as part of the operating system Defragmenters do not actually move files Integrated with caching system and Memory Manager Performs all I/O synchronization Allows even files in use to be defragmented

44 Safety - Microsoft’s Defrag APIs Restrictions –Move Granularity –Free Space in MFT Reserved Zone –NTFS Cluster Size –Special OS files Pagefile Hibernate file NTFS Metadata

45 Microsoft Defrag API Restrictions

46 Microsoft’s Defrag APIs “Quirks” –Compressed files (NT4/Win2K) –File System Cache (NT4/Win2K) –Encrypted Files (Win2K) –NTFRS/DFS (Win2K) –VSS (Vista,Win2K3)

47 Built In Defragmenter Windows 2000/XP Very Basic –Hard to schedule/No scheduling –Multiple passes –Inability to defragment certain files –High free space requirement –Lack of effective free space consolidation –Resource intensive –Not designed for large drives

48 Built In Defragmenter Vista Very Basic –Limited scheduling options –Multiple passes –Inability to defragment certain files –High free space requirement –Incomplete free space consolidation –Not designed for large drives –Not easy to get fragmentation information

49 Advanced Defrag Technology Complete Defrag of All Files Free Space Consolidation Single Pass Defragmentation File Placement Strategy Free Space Requirement Minimal Resource Usage Large Drive Support Easy to Schedule and Manage OS Certification Robust/Easy Reporting

50 Defrag Completeness Data Files Directories System Files –Pagefile –Hibernate File –NTFS metadata

51 Free Space Consolidation Allows new files to be created contiguously –Maintains file system performance longer –Requires less frequent defrag passes

52 Free Space Consolidation Defragmenting files improves read performance Free space consolidation improves write performance

53 Single Pass Defragmentation As my dad always told me… –If you are going to do a job – do it right the first time After defrag has completed, you don’t have to “wonder” if you need to run again Fragmentation issue is solved – done!

54 File Placement Strategy When you play chess, do you have a strategy or do you just start moving pieces around? A good file placement strategy accomplishes several things: –Slows down re-fragmentation –Speeds up future defrag passes –Works with the operating system – not against

55 Free Space Requirement Built in defragmenter requires 10-20% usable free space (outside MFT Reserved Zone – NT4/Win2K) Most commercial defragmenters require about the same – 10-20% Free space consolidation allows you to only actually need about 3%

56 Resource Usage Run in the background Low Memory Usage Low CPU Usage

57 OS Certification Certified for Windows Independently tested to ensure that –Application “plays well” with the operating system –Works with Windows power management –Drivers written to MS standards (if applies)

58 Windows Prefetch What is Prefetching? –Windows monitors system boots and application launches –Uses information gathered to perform driver load optimization (speeds up boot process) –Uses information gathered to speed up application launches

59 Windows Prefetch Windows stores this information in the \Windows\Prefetch folder –Layout.ini – Every 3 days, Windows will automatically perform a “partial” defrag of the files indicated in layout.ini

60 Windows Prefetch This process depends on a large enough piece of contiguous free space being available. If there isn’t, then it doesn’t happen!

61 Windows Prefetch Defragmenters - working WITH the operating system NOT against. Microsoft says: –Either respect where Windows places these files OR –Manage these files yourself

62 “Tweaking” - BootVis BootVis –One of the most mis-understood “tweaking” tools around –Available from Microsoft –Used to profile system boots Optimize driver loading for faster boots Calls built-in defragmenter to “optimize” boot files Also can be used to identify drivers that load slowly

63 “Tweaking” - BootVis Windows XP does this boot optimization over a period of boots (after 3 boots have been performed) BootVis does it “now” End result is the same

64 “Tweaking” – Disable Last Access HKLM\System\CurrentControlSet\Control\ FileSystem\Disablelastaccess =1 Improves NTFS performance by 2% Last Access updating turned off by default in future versions of NTFS (2008 Server)

65 “Tweaking” – 8.3 Name Creation Win2k3/XP/Vista: fsutil.exe behavior set disable8dot3 1 Win2k:HKLM\System\CurrentControlSet\C ontrol\FileSystem\NtfsDisable8dot3NameC reation = 1 Provides faster directory enumeration Legacy apps may no longer install/work.

66 “Tweaking” – Purging Prefetch Belief is that a large number of entries in layout.ini slows down system. Windows will manage effectively. Use of entries in layout.ini actually speeds up system boots and application launch.

67 “Tweaking” – Disabling Vista’s SuperFetch Belief is that not having a large amount of free memory available is a bad thing. SuperFetch caches frequently accessed programs/data and uses available RAM. Unused RAM is a wasted resource. SuperFetch survives system restarts. XP caching doesn’t.

68 Volume Shadow Copy (VSS) VSS and defragmentation Multiple of 16k cluster size Default cluster size is 4k because NTFS compression hasn’t been modified to support greater than 4k cluster size. BitLocker (Vista) also restricted to 4k cluster size

69 DiskPar/DiskPart DiskPar - Win2k DiskPart – Win2k3 Want to avoid crossing track boundaries Align on 64k for best MS SQL performance Win2k8 – default is 64k when creating volumes Contact storage vendor – i.e. EMC recommends 64k

70 Disk Technology SATA worse in high write environments – however cheap RAID1+0 best for high read/write environments AND redundancy – however expensive More onboard cache – better performance

71 Performance Measuring Tools Windows Performance Monitor –Split I/O Count (fragmentation) –Disk Queue Length (<= 2/spindle) hIOMon – –Device AND File based metrics SQLio – Microsoft –Stress Test I/O subsystem

72 Cluster Size Recommendations * You can’t use compression if cluster size greater than 4k

73 Conclusion To improve file system/drive performance –Use appropriate disk technology –Use the most appropriate file system –Use the most appropriate cluster size –Align on cluster boundaries –Make sure free space is consolidated –When you defragment, make sure that it is being done effectively.

74 Conclusion Questions/Answers

Download ppt "Windows File Systems and Improving Performance APCUG 1/5/2008 Greg Hayes Raxco Software"

Similar presentations

Ads by Google