1. Building an MXG PDB from NTSMF Data
Chuck Hopf

2. Building an MXG PDB from NTSMF Data
‘Classic’ MXG PDB Structure
Why it doesn’t work
What needed to change
How it is implemented for NTSMF
Using BLDNTPDB

3. Classic MXG PDB Structure
Read ALL the data and place it in a PDB with possibly CICSTRAN on the side
Run daily - daily contains detail of ALL data read
WEEKBLD consolidates dailies into weeklies
MONTHBLD consolidates weeklies into monthlies

4. Classic MXG PDB Structure
DASD storage required:
8 daily PDB
5 weekly PDB
enough WORK space for the largest dataset (STEPS or TYPE74)

5. Why It Doesn’t Work 40 GB of SMF data daily
Needs 160GB of WORK space (SORTWK and WORK)
320GB of space for daily datasets
1+ Terror Bytes for weekly datasets
Time - Daily jobs that run for more than a day are not practical

6. What Needed to Change Multiple Parallel Threads Run week-to-date
Run month-to-date
Eliminate weekly/monthly jobs
Exploit summarization macros where needed to reduce volume

7. How Is It Implemented for NTSMF?
BLDNTPDB - a very large and complex collection of SAS MACROs that allows the user to choose how the PDB should be structured.
Can use the classic MXG structure or...
Can use virtually any structure the user chooses

8. Using BLDNTPDB Parameters and Meanings Examples of use
Future Directions
?????????????????????

9. Using BLDNTPDB - Parameters and Meanings
NTSMF=C:\MXG\NTSMF\NTSMF.SMF - where is your RAW NTSMF data?
WEEKSTRT=MON - on what day of the week does your week start? The normal MXG default is Monday. This controls when weekly processing takes place.
RUNNTINT=YES - should we run the NTINTRV summarization?

10. Parameters and Meanings
ASUMDUR=HOUR - what level of summarization should we apply? Valid values are QTRHOUR, HALFHOUR, HOUR, SHIFT, DATE, WEEK, MONTH
KEEPERS=null - A list of datasets to be ‘kept’ (usually not used)
DROPPERS=null - A list of datasets to be ‘dropped’ (built but not kept in the output PDB.)

11. Parameters and Meanings
ZEROOBS=null - Similar to DROPPERS but datasets are created with 0 observations
MAXDRVS=26 - Maximum number of logical drives/physical drive (limits and array size.)
RERUN=null - If YES is specified forces the rewriting of an old PDB USE WITH CAUTION

12. Parameters and Meanings
MON=C:\MXG\MON
TUE=C:\MXG\TUE
WED=C:\MXG\WED
THU=C:\MXG\THU
FRI=C:\MXG\FRI
SAT=C:\MXG\SAT
SUN=C:\MXG\SUN
Dataset placement for DAILY PDBs. If a null string is used, the dataset WILL NOT BE BUILT.

13. Parameters and Meanings
WEEK=C:\MXG\WEEK
WEEK2=C:\MXG\WEEK2
WEEK3=C:\MXG\WEEK3
WEEK4=C:\MXG\WEEK4
WEEK5=C:\MXG\WEEK5
Placement for weekly datasets. If a null string is used, the dataset WILL NOT BE BUILT.

14. Parameters and Meanings
TREND=C:\MXG\TREND
MONTH=C:\MXG\MONTH
MONTH1=null
Placement for monthly and TREND datasets. A null string results in the dataset NOT BEING BUILT.

15. Parameters and Meanings
RUNDAY=YES - Run the BUILD process daily and build the specified daily databases
RUNWEEK=NO - YES means run the weekly BUILD process if it is the day specified by WEEKDATE and WTD means build WTD datasets daily
RUNMONTH=NO - YES means run the monthly BUILD process on the 2nd day of the month and MTD means build MTD datasets daily
RUNTREND=WEEKLY - Build the TREND datasets when the WEEKLY BUILD is run. If DAILY is specified, build the TREND datasets when the DAILY datasets are built.

16. Parameters and Meanings
RPTRAW=NO - Build a table of the distribution of the RAW NTSMF record types
RPTDAY=PDB - Create reports for the most recent PDB at the hourly
RPTWEEK=NO - Create reports for the WEEKLY PDB if it has just been
RPTMONTH=NO - Create reports for the MONTHLY database if it has just been created
RPTTREND=NO - Create reports for the TREND database if it has just been updated

17. Parameters and Meanings
GRAFDAY=PDB - Create reports for the most recent PDB at the hourly
GRAFWEEK=NO - Create reports for the WEEKLY PDB if it has just been
GRAFMNTH=NO - Create reports for the MONTHLY database if it has just been created
GRAFTRND=NO - Create reports for the TREND database if it has just been updated

18. Parameters and Meanings
Any combination of RUNxxxx, RPTxxxx, and GRAFxxxx specifications may be used. You can do DAILY, WEEKLY, and MONTHLY runs in a single execution as well as creating reports and graphs at each level.

19. Parameters and Meanings
Null strings passed as some parameters means ‘dont do this!’

20. Examples of Use Classic MXG Structure
PDB for each day of week, 5 weeks, monthly with week run on Monday
%BLDNTPDB(RUNDAY=YES, RUNWEEK=YES, RUNMNTH=YES, RUNTRND=WEEKLY);

21. Examples of Use Single daily PDB no WEEKLY/MONTHLY/TREND
%BLDNTPDB(RUNDAY=YES,MON=, TUE=, WED=, THU=, FRI=, SAT=, SUN=);

22. Examples of Use
Single daily PDB with week-to-date, month-to-date, and trending done daily
%BLDNTPDB(RUNDAY=YES, RUNWEEK=WTD,RUNMNTH=MTD, RUNTRND=DAILY,MON=,TUE=,WED=, THU=, FRI=, SAT=, SUN=);

23. A Few Words About MXG 16.04 and Later
Each dataset now has an up to 6 character mnemonic identifier. In the case of NTSMF data it starts with NT and then has 4 unique characters.
This is used to identify what datasets to KEEP DROP or set to ZEROOBS.
YOU MUST REVIEW THE CHANGES and DOCUMENTATION!

24. Examples of Use Build and keep only the datasets needed for NTINTRV
%BLDNTPDB(KEEPERS=NTBROW NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROC NTPROR NTRDIR NTSERV NTSYST NTTCP);

25. Examples of Use
Build only the datasets needed for NTINTRV and DROP after NTINTRV is created:
%BLDNTPDB(DROPPERS=NTBROW NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROC NTPROR NTRDIR NTSERV NTSYST NTTCP);

26. Examples of Use
Build the datasets needed for NTINTRV but build the PROCESS dataset with zero observations:
%BLDNTPDB(KEEPERS=NTBROW NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROR NTRDIR NTSERV NTSYST NTTCP, ZEROOBS=NTPROC);

27. Examples of Use Reporting can be added to any execution:
%BLDNTPDB(xxxxxxxxxxxxxxx, RPTDAY=YES);

28. Examples of Use Reporting can be done against any PDB:
%BLDNTPDB(xxxxxxxxxxxxxxx, RPTDAY=MON);

29. Examples of Use
GRAPHS can be created for daily, weekly and trend datasets (CPU usage only at this time - more to come.)
%BLDNTPDB(xxxxxxxxxxxxx, GRAFDAY=YES, GRAFWEEK=YES);

30. Examples of Use - A Case Study
Suppose you want the following:
Run the PDB DAILY keeping all datasets
Maximum of 5 logical drives/physical drive
Start the week on Sunday
Keep a WTD/MTD/and TREND datasets
Run reports of Daily usage
Create Graphs of Daily usage

31. Examples of Use - A Case Study
%BLDNTPDB(MON=,
TUE=,WED=,THU=,FRI=,
SAT=,SUN=,WEEK2=,WEEK3=,
WEEK4=,WEEK5=,
RUNDAY=YES,WEEKSTRT=SUN,
RUNWEEK=WTD,RUNMNTH=MTD,
RUNTRND=DAILY,MAXDRVS=5,
RPTDAY=PDB,GRAFDAY=PDB);

32. Future Directions Largely up to YOU!
My knowledge of the data is limited
My experience with the data is limited
I need to know what YOU want/need

33. Future Directions What reports are needed Data sources Data elements
Levels of summarization

34. Future Directions What graphs are needed What data sources
What kinds of graphs
What levels of summarization

35. Future Directions Currently on the development list:
Selection of datasets to place in the WEEKLY/MONTHLY databases - a WEEKKEEP= operand or something along those lines
More automated summarization (this is a tough one because it supposes knowledge of the datasets and variables that is difficult to have programmatically.)

36. Future Directions Currently on the Development List
Ability to limit variables in WTD/MTD datasets using DROP=
Ability to output reports as HTML

37. ???????????