2Building an MXG PDB from NTSMF Data ‘Classic’ MXG PDB StructureWhy it doesn’t workWhat needed to changeHow it is implemented for NTSMFUsing BLDNTPDB
3Classic MXG PDB Structure Read ALL the data and place it in a PDB with possibly CICSTRAN on the sideRun daily - daily contains detail of ALL data readWEEKBLD consolidates dailies into weekliesMONTHBLD consolidates weeklies into monthlies
4Classic MXG PDB Structure DASD storage required:8 daily PDB5 weekly PDBenough WORK space for the largest dataset (STEPS or TYPE74)
5Why It Doesn’t Work 40 GB of SMF data daily Needs 160GB of WORK space (SORTWK and WORK)320GB of space for daily datasets1+ Terror Bytes for weekly datasetsTime - Daily jobs that run for more than a day are not practical
6What Needed to Change Multiple Parallel Threads Run week-to-date Run month-to-dateEliminate weekly/monthly jobsExploit summarization macros where needed to reduce volume
7How Is It Implemented for NTSMF? BLDNTPDB - a very large and complex collection of SAS MACROs that allows the user to choose how the PDB should be structured.Can use the classic MXG structure or...Can use virtually any structure the user chooses
8Using BLDNTPDB Parameters and Meanings Examples of use Future Directions?????????????????????
9Using BLDNTPDB - Parameters and Meanings NTSMF=C:\MXG\NTSMF\NTSMF.SMF - where is your RAW NTSMF data?WEEKSTRT=MON - on what day of the week does your week start? The normal MXG default is Monday. This controls when weekly processing takes place.RUNNTINT=YES - should we run the NTINTRV summarization?
10Parameters and Meanings ASUMDUR=HOUR - what level of summarization should we apply? Valid values are QTRHOUR, HALFHOUR, HOUR, SHIFT, DATE, WEEK, MONTHKEEPERS=null - A list of datasets to be ‘kept’ (usually not used)DROPPERS=null - A list of datasets to be ‘dropped’ (built but not kept in the output PDB.)
11Parameters and Meanings ZEROOBS=null - Similar to DROPPERS but datasets are created with 0 observationsMAXDRVS=26 - Maximum number of logical drives/physical drive (limits and array size.)RERUN=null - If YES is specified forces the rewriting of an old PDB USE WITH CAUTION
12Parameters and Meanings MON=C:\MXG\MONTUE=C:\MXG\TUEWED=C:\MXG\WEDTHU=C:\MXG\THUFRI=C:\MXG\FRISAT=C:\MXG\SATSUN=C:\MXG\SUNDataset placement for DAILY PDBs. If a null string is used, the dataset WILL NOT BE BUILT.
13Parameters and Meanings WEEK=C:\MXG\WEEKWEEK2=C:\MXG\WEEK2WEEK3=C:\MXG\WEEK3WEEK4=C:\MXG\WEEK4WEEK5=C:\MXG\WEEK5Placement for weekly datasets. If a null string is used, the dataset WILL NOT BE BUILT.
14Parameters and Meanings TREND=C:\MXG\TRENDMONTH=C:\MXG\MONTHMONTH1=nullPlacement for monthly and TREND datasets. A null string results in the dataset NOT BEING BUILT.
15Parameters and Meanings RUNDAY=YES - Run the BUILD process daily and build the specified daily databasesRUNWEEK=NO - YES means run the weekly BUILD process if it is the day specified by WEEKDATE and WTD means build WTD datasets dailyRUNMONTH=NO - YES means run the monthly BUILD process on the 2nd day of the month and MTD means build MTD datasets dailyRUNTREND=WEEKLY - Build the TREND datasets when the WEEKLY BUILD is run. If DAILY is specified, build the TREND datasets when the DAILY datasets are built.
16Parameters and Meanings RPTRAW=NO - Build a table of the distribution of the RAW NTSMF record typesRPTDAY=PDB - Create reports for the most recent PDB at the hourlyRPTWEEK=NO - Create reports for the WEEKLY PDB if it has just beenRPTMONTH=NO - Create reports for the MONTHLY database if it has just been createdRPTTREND=NO - Create reports for the TREND database if it has just been updated
17Parameters and Meanings GRAFDAY=PDB - Create reports for the most recent PDB at the hourlyGRAFWEEK=NO - Create reports for the WEEKLY PDB if it has just beenGRAFMNTH=NO - Create reports for the MONTHLY database if it has just been createdGRAFTRND=NO - Create reports for the TREND database if it has just been updated
18Parameters and Meanings Any combination of RUNxxxx, RPTxxxx, and GRAFxxxx specifications may be used. You can do DAILY, WEEKLY, and MONTHLY runs in a single execution as well as creating reports and graphs at each level.
19Parameters and Meanings Null strings passed as some parameters means ‘dont do this!’
20Examples of Use Classic MXG Structure PDB for each day of week, 5 weeks, monthly with week run on Monday%BLDNTPDB(RUNDAY=YES, RUNWEEK=YES, RUNMNTH=YES, RUNTRND=WEEKLY);
21Examples of Use Single daily PDB no WEEKLY/MONTHLY/TREND %BLDNTPDB(RUNDAY=YES,MON=, TUE=, WED=, THU=, FRI=, SAT=, SUN=);
22Examples of UseSingle daily PDB with week-to-date, month-to-date, and trending done daily%BLDNTPDB(RUNDAY=YES, RUNWEEK=WTD,RUNMNTH=MTD, RUNTRND=DAILY,MON=,TUE=,WED=, THU=, FRI=, SAT=, SUN=);
23A Few Words About MXG 16.04 and Later Each dataset now has an up to 6 character mnemonic identifier. In the case of NTSMF data it starts with NT and then has 4 unique characters.This is used to identify what datasets to KEEP DROP or set to ZEROOBS.YOU MUST REVIEW THE CHANGES and DOCUMENTATION!
24Examples of Use Build and keep only the datasets needed for NTINTRV %BLDNTPDB(KEEPERS=NTBROW NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROC NTPROR NTRDIR NTSERV NTSYST NTTCP);
25Examples of UseBuild only the datasets needed for NTINTRV and DROP after NTINTRV is created:%BLDNTPDB(DROPPERS=NTBROW NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROC NTPROR NTRDIR NTSERV NTSYST NTTCP);
26Examples of UseBuild the datasets needed for NTINTRV but build the PROCESS dataset with zero observations:%BLDNTPDB(KEEPERS=NTBROW NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROR NTRDIR NTSERV NTSYST NTTCP, ZEROOBS=NTPROC);
27Examples of Use Reporting can be added to any execution: %BLDNTPDB(xxxxxxxxxxxxxxx, RPTDAY=YES);
28Examples of Use Reporting can be done against any PDB: %BLDNTPDB(xxxxxxxxxxxxxxx, RPTDAY=MON);
29Examples of UseGRAPHS can be created for daily, weekly and trend datasets (CPU usage only at this time - more to come.)%BLDNTPDB(xxxxxxxxxxxxx, GRAFDAY=YES, GRAFWEEK=YES);
30Examples of Use - A Case Study Suppose you want the following:Run the PDB DAILY keeping all datasetsMaximum of 5 logical drives/physical driveStart the week on SundayKeep a WTD/MTD/and TREND datasetsRun reports of Daily usageCreate Graphs of Daily usage
31Examples of Use - A Case Study %BLDNTPDB(MON=,TUE=,WED=,THU=,FRI=,SAT=,SUN=,WEEK2=,WEEK3=,WEEK4=,WEEK5=,RUNDAY=YES,WEEKSTRT=SUN,RUNWEEK=WTD,RUNMNTH=MTD,RUNTRND=DAILY,MAXDRVS=5,RPTDAY=PDB,GRAFDAY=PDB);
32Future Directions Largely up to YOU! My knowledge of the data is limitedMy experience with the data is limitedI need to know what YOU want/need
33Future Directions What reports are needed Data sources Data elements Levels of summarization
34Future Directions What graphs are needed What data sources What kinds of graphsWhat levels of summarization
35Future Directions Currently on the development list: Selection of datasets to place in the WEEKLY/MONTHLY databases - a WEEKKEEP= operand or something along those linesMore automated summarization (this is a tough one because it supposes knowledge of the datasets and variables that is difficult to have programmatically.)
36Future Directions Currently on the Development List Ability to limit variables in WTD/MTD datasets using DROP=Ability to output reports as HTML