Presentation on theme: "An Exercise in Improving SAS Performance on Mainframe Processors"— Presentation transcript:
1 An Exercise in Improving SAS Performance on Mainframe Processors SAS BLKSIZE and BUFSIZE Options
2 ForwardAt the last KCASUG meeting, George Hurley presented “Customizing Your SAS Initialization II.”In this presentation, George suggested that it is possible to save CPU in SAS jobs by tuning the BUFSIZE parameter.With our current interest in saving CPU and stretching the life of mainframe equipment, I decided to investigate what kind of savings were possible in our environment.
3 BackgroundIn the 1990s and earlier, disk storage for mainframes consisted of a stack of 14” platters arranged in what was called a disk drive.There was a separate read/write head for each surfaceAll read/write heads were aligned at the same relative position and moved togetherDisk drives were organized into tracks and cylinders.A track represented the data that could be accessed from one surface with one revolution of the diskA cylinder was all the tracks that could be accessed from the same relative location of the read/writes heads.Data was stored with gaps between records in a CKD format
6 Background3390s were the final generation of IBM classical disk drivesEach track could hold up to 56,664 bytesThe largest size record was 32,767 bytesWhile records could be larger, records were rarely larger than 27,998 bytesThis is the largest record size that allowed 2 records per trackRecord sizes approaching 27,998 bytes provided optimal use of disk storage on 3390 devicesThis is commonly referred to as a “half track” record size
7 BackgroundWhen modern storage controllers started replacing classical mainframe storage, the storage controllers emulated classical storage devices, particularly the 3390While data is actually stored in stripes with multiple layers of virtualization, access to the data still follows the protocol of classical mainframe storage
8 Mainframe SAS FilesTwo factors have the most influence on the performance of I/Os for SAS datasetsBLKSIZE – the size of the block (physical record)BUFSIZE – the size of the storage bufferShould be a multiple of BLKSIZE
9 SAS BLKSIZE BLKSIZE Larger block sizes are more efficient With smaller block sizes, there is additional overhead in SAS to manage each blockSAS files can have any BLKSIZE up to 32,760The optimal BLKSIZE for SAS files is 27,648Largest “half-track” size for SAS filesProvides optimal balance of performance and disk storage utilization
10 SAS BUFSIZEBUFSIZEWhen SAS schedules an I/O for a SAS dataset, it builds the I/O command to transfer as much data as will fit in the buffer as a single I/O commandThis saves the operating system overhead related to managing multiple I/OsSAS uses its own channel programming (EXCPs) for SAS files, not normal operating system access methodsFor example, with a BLKSIZE of 27,648 and a BUFSIZE of 110,592, SAS would build I/O commands to transfer 4 blocks with each I/O command
11 SAS BUFSIZEBUFSIZEBuffer sizes of between 110,592 and 221,184 tend to be fairly efficientMEMSIZE may need to be increased when BUFSIZE is increased
12 Controlled Tests Performed some controlled tests One controlled test Wrote 250,000 records to a SAS file (each about 1.6K of data)In separate step, read the records (in a _NULL_ data step, SET the input to the file just created)Varied BLKSIZE and BUFSIZE in each run
13 Controlled TestsTests showed that a BLKSIZE of 27,648 performed better than a BLKSIZE of 6,144 for similar buffer sizesA BLKSIZE of 6,144 was the old standard in our shopTests also suggested limited improvements in CPU and run times with buffer sizes above 110,592 to 221,184In fact, sometimes performance appeared to deteriorate with larger buffer sizes
18 Production PilotsIdentified the jobs that were using the largest total amount of CPURan pilots on 2 of the top 5 jobs to explore potential benefits with real jobsChanged BKLSIZE from 6,144 to 27,648Increased BUFSIZE to 221,184Ran several parallel runs of the MXG job with various BLKSIZE and BUFSIZE (MXG is a common SAS-based mainframe tool to capture and manage mainframe performance data)Experimented with various block sizesHave not placed changes to MXG in production yetRewrote one job in another language
19 Pilot Results Pilot results were quite favorable Job using largest amount of CPU (runs many times each day) – see charts for Job 16% reduction in CPU25% improvement in run timeJob using 5th largest amount of CPU (runs many times each day) – see charts for Job 29% reduction in CPU43% improvement in run timeMXG (2nd largest user of CPU – runs once daily)5% reduction in CPU~ 10% improvement in run time
24 Production Implementation Changed BLKSIZE to 27,648Changed both CONFIG member and SAS PROCChanged BUFSIZE to 221,184Changed CONFIG memberMade changes to ensure jobs would not fail with memory issuesMEMSIZE parameter removed from CONFIGDefaults to 0 (no limitation on memory)Changed REGION to 0M in SAS PROCMade mass change to production SAS jobs to remove REGION parameter overrides
25 Implementation Results Measured results based on production jobs that ran dailyCompared results on job / weekday basisFor jobs that ran during the day:10% average reduction in CPUVaried from no gain to 15-20% improvement30% average improvement in run timesVaried considerably from job to jobFor jobs that ran at night3% reduction in CPU10% improvement in run times
26 Issues and Opportunities Many production jobs reuse same SAS files without ever deleting and recreating themBLKSIZE remains smaller sizeMany production jobs use their own customized SAS PROCs or CONFIG membersCannot easily take advantage of changesWill need to look for opportunities to tune these jobs later
27 Thinking Outside the Box One very large SAS job runs dailyJob would read million rowsSort data on 4 keysSummarize 32 columns using PROC UNIVARIATERewrote job in another languageTook advantage of partial natural order of data and used hashing algorithm to organize dataInitial level summary done in summary programSummarized data was then input to SAS
28 Changes in Rewritten Job Reduced CPU 95%Improved run time 97%It is worth noting that I could find only two large SAS jobs that could take advantage of this technique. All other SAS jobs that I looked at were far too complex to consider doing this.
Your consent to our cookies if you continue to use this website.