1. JPMorganChase1 Care and Feeding of SMF in a Large System Environment Joe Babcock

2. JPMorganChase2 Agenda Define the problem. Define the problem. A model for SMF capture. A model for SMF capture. Processing the SMF data (Parsing). Processing the SMF data (Parsing). Summarization of SMF parsed data. Summarization of SMF parsed data.

3. JPMorganChase3 Total SMF Processing If Single Threaded.

4. JPMorganChase4 SMF Incremental Processing 3 types of SMF dumps: 3 types of SMF dumps: –F – dump and clear the MANx DSN’s. –A – AM incremental starts the AM parse (0601). –P – PM incremental starts the PM parse (1401). –D – mid-night incremental starts the daily parse (0001). Batch scheduler, schedules the dumps. Batch scheduler, schedules the dumps.

5. JPMorganChase5 SMF Incremental Processing Batch scheduler, receives the event upon SMFDUMP completion. Batch scheduler, receives the event upon SMFDUMP completion. For incremental runs the batch scheduler, starts a copy to collect all SMF data dumped sense the last incremental. For incremental runs the batch scheduler, starts a copy to collect all SMF data dumped sense the last incremental.

6. JPMorganChase6 SMFDUMP Flow SMFDUMPLOG Post to Scheduler MANx ZEKBATCH SMFDUMP SSYSx.**.gdg SSYSx.**.gdg SSYSx.**.gdg All, NET, RMF, STO, etc.

7. JPMorganChase7 SMFCOPY SSYSx.**.gdg SSYSx.**.gdg SSYSx.**.gdg SSYSx.**.gdg Single Sub-system i.e. RMF ISYSx.**.gdg Input for CPE Parse jobs IEBGENER SMFBARMF IEFBR14Delete IDCAMS Def null DS

8. JPMorganChase8 Why Copy the SMF data? SMF is a pseudo VBS file. SMF is a pseudo VBS file. –Has short blocks in the file. Verify the data. Verify the data. Minimizes contention for the GDG base, used by SMFDUMP. Minimizes contention for the GDG base, used by SMFDUMP.

9. JPMorganChase9 CPE Parse Jobs 3 times per day (A,P & D) ISYSx.**.gdg MXG/SAS job Parses the data DSN Below CPEP.PMST&sysplex..NMXG.&inc..RMF.&mmmdd CPEBARMF

10. JPMorganChase10 CPE Copy Jobs MXG/SAS job Parses the data DetailFile CPEP.PMST&sysplex..NMXG.&inc..RMF.&mmmdd CPEP.PMST&sysplex..NMXG.RMF.DETAIL.&mmmdd IncrementalFiles CPEBARMF

11. JPMorganChase11 CPE Summary Jobs MXG/SAS job summarizes the data CPEP.PMST&sysplex..NMXG.RMF.DETAIL.&mmmdd CPEP.PMST&sysplex..NMXG.RMF.SUMMARY.&mmmdd Other summary datasets CPEP.PMST.&sysplex..NMXG.RMF.SUMMARY.&mmmdd.IO15MCPEP.PMST.&sysplex..NMXG.RMF.SUMMARY.&mmmdd.SMFINTRV DetailFile SummaryFile CPEBSRMF

12. JPMorganChase12 CPE Report Jobs MXG/SAS job Report process CPEP.PMST&sysplex..NMXG.RMF.SUMMARY.&mmmdd Other summary datasets CPEP.PMST.&sysplex..NMXG.RMF.SUMMARY.&mmmdd.IO15MCPEP.PMST.&sysplex..NMXG.RMF.SUMMARY.&mmmdd.SMFINTRV SummaryFile CSV Daily / Weekly / Monthly Reports CPEBRRMF

13. JPMorganChase13 Overview CPE CPEAACIC Parse job CPEAPCIC CPEASCIC Summary job CPEACCIC Copy job CPEADCIC Parse job SMFAiCIC Incremental SMF job CPEARCIC Report job

14. JPMorganChase14 CPE Environment JCL for CPE Jobs. JCL for CPE Jobs. Macro’s – AUTOMAC. Macro’s – AUTOMAC. Source -- SOURCE (Lib). Source -- SOURCE (Lib). Based on MXG. Based on MXG. –CPEP.PSRC0.PMXG.SOURCLIB.

15. JPMorganChase15 CPE Control There are 8 sub-systems in CPE processing. There are 8 sub-systems in CPE processing. –CIC - CICS processing. –DB2 - DB2 processing. –DCL - Dcollect processing (not part of SMF data). –IMS - IMS processing (from IMS log file).

16. JPMorganChase16 CPE Control There are 8 sub-systems in CPE processing. There are 8 sub-systems in CPE processing. –NET - communications. MQ / NJE / WebSphere etc. MQ / NJE / WebSphere etc. –RMF - CMF/RMF, type30’s & JES. –SEC - security records (RACF & ACF2). –STO - storage related data (not TYPE74 records).

17. JPMorganChase17 Standard JCL //#CPEACIC JOB (01089400000AF0002),'OH1-1281.JL.BABCOCK', // CLASS=X,MSGCLASS=0,MSGLEVEL=(1,1),NOTIFY=AF0002 //SAS EXEC SAS,WORK='1111,1111', // CONFIG='CPEP.PSRC0.PV1M0.SOURCE(CONFIG)' //WORK DD VOL=(,,,6) //SMF DD DISP=SHR,DSN=SMFP.ISYSA.N.A.CIC(00) //SASAUTOS DD DISP=SHR,DSN=CPEP.PSRC0.PV1M0.AUTOMAC //SOURCLIB DD DISP=SHR,DSN=CPEP.PSRC0.PV1M0.SOURCE // DD DISP=SHR,DSN=CPEP.PSRC0.PMXG.SOURCLIB //LIBRARY DD DISP=SHR,DSN=CPEP.PSRC0.NMXG.FORMATS //USERFMTS DD DSN=CPEP.PSRC0.N.USERFMTS,DISP=SHR //SYSIN DD DSN=CPEP.PSRC0.PV1M0.SOURCE(OPTIONS),DISP=SHR // DD * %SMFDATES(NEWDATE=0,INCR=A,PLEX=B) ; RUN ; %SMFDATES(NEWDATE=0,INCR=A,PLEX=B) ; RUN ; *** %SMFDATES(NEWDATE=‘DDMMMYY'D,INCR=A,PLEX=B) ; RUN ; %INCLUDE SOURCLIB(CICPARSE) ; %INCLUDE SOURCLIB(CICPARSE) ;/*

18. JPMorganChase18 AutoMac Members SMFDATES *** ALCLIBALCRANGERDRANGECLRANGEJWEEKMMMDD1 REPORTSRUNWHENSHIFT

19. JPMorganChase19 Questions???

20. JPMorganChase20 SMFDATES %MACRO SMFDATES(NEWDATE=0,INCR=F,PLEX=B,HLVQ=CPEP); * *********************************************************** *; * MACRO: SMFDATES ; * INPUTS REQUIRED NONE ; * INPUTS OPTIONAL - ; * NEWDATE - SAS DATE VARIABLE TO BE RETURNED ; * IN THE OUTPUT FORMATS. ; * INCR - INCREMENT TYPE (A,P,D) ; * IF NEWDATE=0 ; * THEN SELECT (&INCR) ('A','P') ; * WHEN ('A','P') THEN DATE = TODAY() ; * WHEN ('D') ) THEN DATE = TODAY()-1 ; * OTHERWISE ---- ERROR ; * END ; * ELSE DATE = NEWDATE ; * PLEX - THE PLEX ID FOR THE EXECUTION. ; * HLVQ - THE HIGH LEVEL QUAL. FOR THE LIBNAMES ; * CREATED BY THIS SYSTEM. ; * ;

21. JPMorganChase21 SMFDATES (Cont.) * OUTPUT FIELD EXAMPLES: ; * &MMMDD - JUL20 ; * &RUNDATE - 07/20/2003 ; * &MMM - JUL ; * &DD - 20 ; * &INC - A,P,D ; * &SYSPLEX - A,B,D ; * &JULDATE - 03201 ; * &HLQ - CPEP ; * &LAST_MONTH_FIRST_DAY - FIRST DAY OF LAST MONTH ; * &LAST_MONTH_LAST_DAY - LAST DAY OF LAST MONTH ; * &THIS_MONTH_FIRST_DAY - FIRST DAY OF THIS MONTH ; * &THIS_MONTH_LAST_DAY - LAST DAY OF THIS MONTH ; * &LAST_MONTH_DAYS - NUMBER OF DAYS LAST MONTH ; * &THIS_MONTH_DAYS - NUMBER OF DAYS THIS MONTH ; * *********************************************************** *;

22. JPMorganChase22 ALCLIB %MACRO ALCLIB (DDNAME=, ENGINE=, ENGINE=, DSN=, DSN=, UNIT=, UNIT=, SPACE=, SPACE=, DISP=, DISP=, RETPD=0 RETPD=0 ) ; ) ; ********************************************************** ; * THIS MACRO CHECKS THE LIBRARY AND IF NOT PRESENT THEN ; * ALCLIBES THE FILE. ; * DDNAME - THE DDNAME FOR THE FILE. ; * ENGINE - THE SAS I/O ENGINE. ; * DSN - THE FOURTH NODE OF THE DATASET NAME. ; * UNIT - UNIT JUST LIKE JCL. ; * SPACE - SPACE ALLOCAITON IF NEEDED. ; * RETPD - THE RETENTION PERIOD. ; ********************************************************** ; %ALCLIB (DDNAME=PDB,DSN=&HLQ..PMST&SYSPLEX..NMXG.RMF.DETAIL.&MMMDD);

23. JPMorganChase23 ALCRANGE %MACRO ALCRANGE (STARTDT=, ENDDT=0, ENDDT=0, SUBSYS=, SUBSYS=, LEVEL=, LEVEL=, MEMBER=) ; MEMBER=) ; ******************************************************** ; * ALCRANGE ALLOCATES A RANGE OF CPEP FILES FOR THE USER ; * AND COUNTS THE ALLOCATION FOR RDRANGE MACRO. ; * USE THE RDRANGE MACRO FOR THE SET COMMAND TO ; * READ THE DATASETS ALLOCATED. ; * STARTDT = THE START DATE FOR THE FILE. ; * ENDDT = THE END DATE FOR THE RANGE OF FILES. ; * IF LEFT BLANK OR OFF IT WILL DEFAULT TO ; * TODAYS DATE. ; * SUBSYS = THE SMF GROUPING FOR THE FILE (RMF, NET,... ; * LEVEL = THE LEVEL OF THE FILE TO BE USED ; * (DETAIL, SUMMARY, ETC.) ; * MEMBER = THE 6TH NODE OF THE DATASET ; * (SMFINTRV, IO15M ETC.) ; ******************************************************** ;

24. JPMorganChase24 ALCRANGE Example %ALCRANGE(STARTDT=&DATADATE-6, ENDDT=&DATADATE, ENDDT=&DATADATE, SUBSYS=RMF, SUBSYS=RMF, LEVEL=DETAIL ) ; LEVEL=DETAIL ) ;OR %ALCRANGE((STARTDT=&DATADATE-6, ENDDT=&DATADATE, ENDDT=&DATADATE, SUBSYS=RMF, SUBSYS=RMF, LEVEL=SUMMARY, LEVEL=SUMMARY, MEMBER=SMFINTRV ) ; MEMBER=SMFINTRV ) ; Only one SAS file type at a time!!! You can free and allocate another dataset range for later processing.

25. JPMorganChase25 RDRANGE %MACRO RDRANGE (FILENM) ; ************************************************************ ; * RDRANGE - CREATES THE SET STATEMENT TO READ A RANGE OF * ; * CPEP FILES (DATE RANGE). * ; * FILENM = THE MEMBER OF THE SAS LIB YOU ARE READING. * ; ************************************************************ ; %RDRANGE (JOBS) ; OR %RDRANGE (SMFINTRV) ; The member must be in the SAS file allocated by %ALCRANGE!!

26. JPMorganChase26 CLRANGE %MACRO CLRANGE ; ****************************************************** ; * CLRANGE - FREES THE LAST ALLOCATION MADE BY * ; * ALCRANGE MACRO. * ; ****************************************************** ; %CLRANGE ;

27. JPMorganChase27 JWEEK – Julian Week %MACRO JWEEK (DATE=, WEEK=, YEAR= ) ; %MACRO JWEEK (DATE=, WEEK=, YEAR= ) ; ********************************************************** ; * INPUT THE DATE FOR THIS MACRO JWEEK * ; * RETURNS : * ; * &WEEK = WEEK OF THE YEAR * ; * &YEAR = THE YEAR OF THE DATE. * ; ********************************************************** ;

28. JPMorganChase28 MMMDD1 %MACRO MMMDD1 (DATE) ; ****************************************************** ; ****************************************************** ; * MMMDD1 - CREATES VARIABLES FOR LATER USE IN SAS * ; * MMMDD1 - CREATES VARIABLES FOR LATER USE IN SAS * ; * CODE: * ; * CODE: * ; * &MMMDD1 - DATE * ; * &MMMDD1 - DATE * ; * &MMM1 - MONTH * ; * &MMM1 - MONTH * ; * &DD1 - DAY * ; * &DD1 - DAY * ; ****************************************************** ; ****************************************************** ; %MMMDD1 (&DATADATE-35) ; Then you can use: %ALCLIB (DDNAME=PDB, DISP=(OLD,DELETE), DSNAME=CPEP.PMST&SYSPLEX..NMXT.RMF.DETAIL.&MMMDD1) ; DSNAME=CPEP.PMST&SYSPLEX..NMXT.RMF.DETAIL.&MMMDD1) ; Deleted the detail file that is 36 days old.

29. JPMorganChase29 Reports %MACRO REPORTS(MEMBER=, SS1=, LS=256, %MACRO REPORTS(MEMBER=, SS1=, LS=256, VARLIST=, PRINTBY=, PAGEBY=, VARLIST=, PRINTBY=, PAGEBY=, DATE=, CSVLABEL=YES, SUM=); DATE=, CSVLABEL=YES, SUM=); **************************************************************** ; * MEMBER - THE SAS FILE MEMBER TO PRINT (MUST BE IN THE * ; * WORK FILE). * ; * SS1 - SUB-SYSTEM ID OR 4TH NODE OF THE REPORTS DSN * ; * IF NO SS1 VALUE IT LOOKS FOR &SSS. * ; * VARLIST - LIST OF VARIABLES IN THE ORDER YOU WANT PRINTED * ; * PRINTBY - VALUES FOR THE BY STATMENT OF THE PROC PRINT. * ; * PAGEBY - VALUE FOR THE PAGEBY STATEMENT OF THE PROC PRINT. * ; * SUM - VALUES FOR THE SUM STATMENT OF THE PROC PRINT. * ; **************************************************************** ; You must create the files: &HLQ..PORP&SYSPLEX..E.&SS1..REPORTS.&MMMDD &HLQ..PORP&SYSPLEX..E.&SS1..REPORTS.&MMMDD &HLQ..PSTG&SYSPLEX..E.&SS1..REPORTS.&MMMDD &HLQ..PSTG&SYSPLEX..E.&SS1..REPORTS.&MMMDD Then code: %REPORTS(MEMBER=EXCPTAN2, LS=132, SS1=CIC, VARLIST = STRTTIME TRANNAME ENDTIME RESPTM APPLID ) ; VARLIST = STRTTIME TRANNAME ENDTIME RESPTM APPLID ) ;

30. JPMorganChase30 RUNWHEN %MACRO RUNWHEN (DATE=, TYPE=, DAY=, MEM=) ; ******************************************************** ; * RUNWHEN - SCHEDULE CODE FOR DAY OF WEEK OR MONTH. * ; * TYPE - WEEK OR OTHER * ; * DATE - RUN DATE * ; * DAY - DAY OF WEEK OR DATE FOR THE RUN * ; * MEM - MEMBER TO INCLUDE AND RUN * ; ******************************************************** ; %RUNWHEN (TYPE=WEEK, DAY=7, DATE=&DATDATE, MEM=IMSRPT01) ; Or %RUNWHEN (TYPE=MONTH, DAY=&THIS_MONTH_LAST_DAY, DATE=&DATADATE, MEM=RMFMTHLY) ;

31. JPMorganChase31 SHIFT %MACRO SHIFT(DTVALUE); ********************************************************* ; * SHIFT - CREATES THE VARIABLE SHIFT WITH THE VALUE OF * ; * 1,2,3,OR 4 * ; ********************************************************* ; ATTRIB SHIFT LENGTH=$1 LABEL='SHIFT*ID' ; ATTRIB SHIFT LENGTH=$1 LABEL='SHIFT*ID' ; %SHIFT (STARTIME) ;

32. JPMorganChase32 SOURCE (Lib) CPEP.PSRC0.PV1M0.SOURCE most likely we will create an alias for our libraries to allow for releases. CPEP.PSRC0.PV1M0.SOURCE most likely we will create an alias for our libraries to allow for releases. –CPEP.PSRC0.P.SOURCE pointing to the real dataset.

33. JPMorganChase33 Source (Lib) CICCOPYCICPARSECICSUMRYDB2COPYDB2PARSEDB2SUMRYDCLCOPYDCLPARSEIMSCOPYIMSMSUMMIMSPARSEIMSSUMRYNETCOPYNETPARSERMFCOPYRMFINTRVRMFPARSERMFSUMRYSECCOPYSECPARSESTOCOPYSTOPARSE CONFIGOPTIONS EXTY167IMACRMFIIMACSHFTIMACSPINIMAC167IMSCDE EnvironmentMembers MXG Control CPE Control Members

34. JPMorganChase34 CONFIG * THIS IS AN EXACT COPY OF CONFIGV8, NO ALTERATIONS * THIS COPY WAS MADE TO ELIMINATE A VERSION REFERENCE IN PROD JCL * * COPYRIGHT (C) 2000,2002 MERRILL CONSULTANTS DALLAS TEXAS 75229 * LAST UPDATED JAN 28, 2002. CHANGE 19.279. * * THIS IS THE CONFIG OPTIONS FOR SAS VERSION 8 UNDER MVS OR OS/390. * * THESE OPTION VALUES REPLACE THE SAS DEFAULT VALUES * YEARCUTOFF=1960 YEARCUTOFF=1960 SEQENGINE=V6SEQ SEQENGINE=V6SEQ BLKSIZE(3380)=23040 BLKSIZE(3380)=23040 BLKSIZE(3390)=27648 BLKSIZE(3390)=27648 ETC… ETC … ETC…

35. JPMorganChase35 OPTIONS ***********************************************; * MEMBER OPTIONS IS INCLUDED AT THE BEGINNING * * OF THE SYSIN CONCATENATION LIST FOR ALL * * PRODUCTION JOBS * ***********************************************; OPTIONS NOS99NOMIG MAUTOSOURCE SASAUTOS=(SASAUTOS SOURCLIB ) SASAUTOS=(SASAUTOS SOURCLIB ) FMTSEARCH=(WORK USERFMTS LIBRARY); FMTSEARCH=(WORK USERFMTS LIBRARY); OPTIONS COMPRESS=YES ; OPTIONS COMPRESS=YES ;

36. JPMorganChase36 MXG Control Exty167. Exty167. –Exit for type 167 records. IMACRMFI. IMACRMFI. –The _DURSET macro for setting intervals & grouping. IMACSHFT. IMACSHFT. –Set shift variable in some code. IMACSPIN. IMACSPIN. –Control for the spin logic for JOBS and STEPS PDB’s. Imac167. Imac167. –Control for the type 167 record. IMSCDE. IMSCDE. –Macros and code for IMS processing (daily detail, summary, & monthly processing.

37. JPMorganChase37 CPE Source Member Naming Standard The first 3 characters of the member name is the sub- system. The first 3 characters of the member name is the sub- system. This is followed by the type of process. This is followed by the type of process. –PARSE. Parse the data and create the incremental PDB’s. Parse the data and create the incremental PDB’s. –Copy. Copy the A, P, D incremental datasets to the daily DETAIL libraries. Copy the A, P, D incremental datasets to the daily DETAIL libraries. –SUMM. Summarize the DETAIL data into summary files (daily). Summarize the DETAIL data into summary files (daily). –RPT. Stages the sssRPTnn members to run for daily, weekly etc reports. Stages the sssRPTnn members to run for daily, weekly etc reports.

38. JPMorganChase38 CPE Source Member Naming Standard (Cont.) Examples Examples »CICPARSE »CICCOPY »CICSUMRY »RMFPARSE »RMFCOPY »RMFSUMM

39. JPMorganChase39 CPE Job Naming Standards Jobs Jobs –CPEpiyyy »p – is the plex »i - is the incremental value (A,P,D) or C – Copy S – Summary R – Report »yyy – is the sub system (STO, RMF, etc.)

40. JPMorganChase40 CPE Job Naming Standards (Cont.) Datasets. Datasets. »CPEP.PMST&sysplex..NMXG.&Inc..&Sub..&Mmmdd. &Inc – the incremental. &Inc – the incremental. &Sub – the sub-system (STO, RMF, etc). &Sub – the sub-system (STO, RMF, etc). &Mmmdd – the date in month,day format. &Mmmdd – the date in month,day format. »CPEP. PMST&sysplex..NMXG.&Sub..DETAIL.&Mmmdd. Refer to the previous definitions. Refer to the previous definitions. This is the daily detail file. This is the daily detail file. »CPEP. PMST&sysplex..NMXG.&Sub..SUMMARY.&Mmmdd. Daily summary file. Daily summary file.

41. JPMorganChase41 CPE Job Naming Standards (Cont.) Exceptions: Exceptions: –CPEP.PMST&sysplex..NMXG.&Inc..Sto42ds.&Mmmdd. »This is the type 42 dataset records only. –CPEP.PMST&sysplex..NMXG.STO42DS.DETAIL.&Mmmdd. »This is the daily detail file for type 42 dataset records only. –CPEP.PMST&sysplex..NMXG.RMF.SUMMARY.&Mmmdd..SMFINTRV. »Daily summary file containing the SMFINTRV member only. –CPEP.PMST&sysplex..NSMF.&Inc..Cat.&Mmmdd. »Raw SMF (60,61,65,66) records for catalog recovery.

42. JPMorganChase42 Overview CPE CPEAACIC Parse job CPEAPCIC CPEASCIC Summary job CPEACCIC Copy job CPEADCIC Parse job SMFAiCIC Incremental SMF job CPEARCIC Report job

43. JPMorganChase43 Questions???