1. Code Generation

2. 2 Overview of presentation Goal Background Dynamic SQL Method Examples

3. 3 Goal The goal of this presentation is to cover several methods of having SAS write code for you The method I will use is to cover the concept, followed by some examples, then some tips Data is faked up or at the least obfuscated As much as possible all ideas have been stolen, borrowed, downloaded from the internet or otherwise cribbed Think about it: would you trust a presentation on “not writing your own code” from someone who actually came up with all of the ideas themselves?

4. 4 Background I’ve been programming SAS for years, but prior to that I spend a lot of time dealing with Oracle SQL  SQL programming has little or no macro facility  To do task that are simple in SAS you have to be able to code Dynamically  Dynamic coding translates will into dealing with SAS macros, but still comes up on its own accord The next few slides are an overview of Dynamic SQL as a background.

5. 5 SELECT ALM_KEY, 1 Cycle, Prefix,Current_account_number FROM OPS$U35234. DEBIT_ACT_CARD_ALM_JUN_1999 where rownum < 11 and statement_cycle_number = 1 order by Prefix,Current_Account_Number; SELECT ALM_KEY, 2 Cycle, Prefix,Current_account_number FROM OPS$U35234. DEBIT_ACT_CARD_ALM_JUN_1999 where rownum < 11 and statement_cycle_number = 2 order by Prefix,Current_Account_Number; SELECT ALM_KEY, 3 Cycle, Prefix,Current_account_number FROM OPS$U35234. DEBIT_ACT_CARD_ALM_JUN_1999 where rownum < 11 and statement_cycle_number = 3 order by Prefix,Current_Account_Number;. SPOOL OPTIONS spool code.sql Select 'SELECT ALM_KEY,',rownum,' Cycle, Prefix,Current_account_number' coll1, 'FROM &owner. &ALM_table' coll2, 'where rownum < 11 and statement_cycle_number = '||rownum coll3, 'order by Prefix,Current_Account_Number;' FROM snapshot where rownum < 29; spool off Dynamic SQL: Example 1 get 10 accounts of each type

6. 6 /* -------------------------------------------------------------------- */ /* Spool a file of SQL code to be run later */ /* -------------------------------------------------------------------- */ spool code.sql SELECT 'spool '||AGENCY||'.lst ' CC, 'select r1_account_num||account_prefix||account_number ' CC, 'from nbr_co_seg1 ' CC, 'where exclude_flag is null and move = 1 ' CC, 'and agency = '''||AGENCY||''' ' CC, 'and final_level in (201,301,202,203,302,303) ' CC, 'and keep_flag in (2,3,4); ' CC, 'spool off ' CC FROM NBR_CO_SEG1 WHERE EXCLUDE_FLAG IS NULL AND FINAL_LEVEL IN (201,301,202,203,302,303) AND MOVE = 1 AND KEEP_FLAG in (2,3,4) GROUP BY AGENCY; spool off /* -------------------------------------------------------------------- */ Dynamic SQL: Example 2 spool a separate file for each AGENCY

7. 7 The Principles: How it works Using SAS as a way to write other code is something that is most effective in situations meeting : 1. You have a lot of repetitive tasks 2. These core details that change from task to task can be expressed in a dataset 3. The dataset needed is easy to create 4. The code to complete these tasks needs to be saved and/or the macro facility by itself cannot complete the task For the sake of clarity let us establish some names: Program 1: The sas code we write in order to produce program 2 Program 2: The sas code Program 1 writes in order to do stuff [Program 1]  [Program 2] Did I mention that Program 2 does not need to be SAS code?

8. 8 What is the difference between this and writing macros The interaction between using SAS generated SAS code and a SAS macro is a little complicated.  Sometimes they are two ways to solve the same issue.  Often SAS written code is a tactic in macro programming, either explicitly or via the CALL EXECUTE. Oddly enough, code generating seems a bit more stable and platform independent

9. 9 Basic outline of program 2 1. Set-up / Initialization Could include library statements, options, etc. Could include ODS statements 2. List of actions to be taken Could be a data steps for each of 100 situations Could be a report procedure for each combination of variables Could be a sas macro for each variable Could be a simple assignment statement or if/then 3. Clean-up Delete any temporary datasets Close out any ODS files Restore normal system options

10. 10 Basic outline of program 1 1. Initialize options and set up 2. Gather information about each instance of the task 3. Organize data set of information for each task 4. Employ a data null step to fill in constants (i.e. the tasks) in between the variables (i.e. the specifics of each situation) 5. Clean up

11. 11 Examples Proc Google Getting a format from proc rank Generating SAS code from Specs Using SAS’s built in coding features

12. 12 Proc Google While this is technically not a SAS thing per say if searching for your problem on the internet solves the issue, who cares? I’ve lifted lots of good stuff over the years, but I want to share a great example next because it involves generating SAS code. Originally it generated a proc format statement, but I modified it for other stuff…

13. 13 Getting a format from proc rank For the purposed of reporting and analysis we use proc rank all the time.  It is perfect for computing quantiles such as deciles or twentile.  Sometimes we either do not want to add the ranked variables to our dataset, or more likely we want to look at a number of variables from a number of datasets broken down the same way. Attempt 1: check the SAS documentation to see if proc rank will produce a format instead of a variable. It seems reasonable, but the attempt is in vain Attempt 2: Check Google: search “proc rank format” (2 nd link is a macro written by Ken Moore that does the trick) Attempt 3: Write the code yourself (thankfully we are spared)…well I did modify this to produce the SAS code to produce the ranges

14. 14 Getting a format from proc rank Let’s look at an outline of how this process works: 1. Rank the data into a temporary data set 2. Run proc means on this data set, with the ranked variable as the only class variable. Output to a dataset 3. Adjust the data so that the Min of Rank N becomes the Max of Rank N-1. Treat the first and last rankings specially (along with missing values. All of this is in one dataset. 4. Input this summary dataset into proc format to create a format 5. Create a summary report using proc freq and the format we just created 6. Use a data _null_ step to write the code for the format

15. 15 Getting a format from proc rank: Program 1 As an example I’m using the dataset sashelp.prdsal2 %include "path\rankformat.sas"; %FORMAT(SASHELP.Prdsal3, PREDICT, PF, 10 ); Input dataset Variable to rank Output format name Number of groups needed

16. 16 Getting a format from proc rank: Program 2 As an example I’m using the dataset sashelp.prdsal2 PROC FORMAT; VALUE PF. -<. ="Missing " LOW -< 174.000 ="< 174.000 " 174.000 -< 361.000 ="174.000 -< 361.000 " 361.000 -< 547.800 ="361.000 -< 547.800 " 547.800 -< 728.000 ="547.800 -< 728.000 " 728.000 -< 916.300 ="728.000 -< 916.300 " 916.300 -< 1098.900 ="916.300 -< 1098.900 " 1098.900 -< 1304.000 ="1098.900 -< 1304.000 " 1304.000 -< 1557.000 ="1304.000 -< 1557.000 " 1557.000 -< 1855.000 ="1557.000 -< 1855.000 " 1855.000 - HIGH ="1855.000 + "; RUN; %macro PREDICT; if PREDICT =. then PREDICTr =0 ; else if PREDICT <= 174.000 then PREDICTr =1 ; else if PREDICT > 174.000 and PREDICT <= 361.000 then PREDICTr =2 ; else if PREDICT > 361.000 and PREDICT <= 547.800 then PREDICTr =3 ; else if PREDICT > 547.800 and PREDICT <= 728.000 then PREDICTr =4 ; else if PREDICT > 728.000 and PREDICT <= 916.300 then PREDICTr =5 ; else if PREDICT > 916.300 and PREDICT <= 1098.900 then PREDICTr =6 ; else if PREDICT > 1098.900 and PREDICT <= 1304.000 then PREDICTr =7 ; else if PREDICT > 1304.000 and PREDICT <= 1557.000 then PREDICTr =8 ; else if PREDICT > 1557.000 and PREDICT <= 1855.000 then PREDICTr =9 ; else if PREDICT > 1855.000 then PREDICTr =10 ; %mend; Output format statement Output macro statement

17. 17 Getting a format from proc rank How the program works ******************************************************************************; * Program: format.sas *; * Author: Ken Moore *; * Created: 08/28/03 *; ******************************************************************************; %MACRO FORMAT(DATA, /* INPUT DATA SET */ VAR, /* CONTINUOUS NUMERIC VARIABLE TO BE FORMATTED */ FMT, /* FORMAT NAME */ GROUPS /* NUMBER OF GROUPS */ ); **********************************************************************; * ASSIGN RANK FOR NON-MISSINGS ; **********************************************************************; proc rank data=&DATA (keep=&VAR where=(&VAR ne.)) out=rank groups=&GROUPS; var &VAR; ranks RANK; run;

18. 18 Getting a format from proc rank How the program works *********************************************************************; * OBTAIN GROUP MEANS FOR FORMAT BOUNDARIES ; *********************************************************************; proc means data=rank missing noprint; var &VAR; class RANK; output out=mnrank (where=(RANK ne.)) min=min; run; data mnrank; set mnrank; RANK_=_N_; run; data getrange; set mnrank; RANK_=RANK_-1; run; proc sort data=getrange; by RANK_; run; proc sort data=mnrank out=smnrank; by RANK_; run;

19. 19 Getting a format from proc rank How the program works *********************************************************************; * CREATE FORMAT DATASET ; *********************************************************************; data miss; format START END $10. LABEL $20.; FMTNAME="&FMT"; START='.'; END='.'; LABEL='Missing'; run; data getrange2; format START END $10. LABEL $20. MIN MAX 8.3; merge getrange (in=a rename=(MIN=MAX_)) smnrank (in=b rename=(MIN=MIN_)); by RANK_; retain FMTNAME "&FMT" SEXCL EEXCL 'Y'; MAX=MAX_; MIN=MIN_; if RANK_>0; START=PUT(MIN,8.3); END=PUT(MAX,8.3); LABEL=left(compbl(put(MIN,8.3)||' -< '||put(MAX,8.3))); if RANK=0 then do; START='LOW'; HLO='L'; SEXCL='N'; EEXCL='N'; LABEL=compbl('< '||put(MAX,8.3)); end; if RANK=%eval(&GROUPS-1) then do; END='HIGH'; LABEL=compbl(put(MIN,8.3)||' +'); HLO='H'; SEXCL='N'; EEXCL='N'; end; run; data getrange3; set miss getrange2; run; proc format cntlin=getrange3; run;

20. 20 Getting a format from proc rank How the program works *********************************************************************; * OUTPUT SUMMARY TABLE ; *********************************************************************; title "VAR: %UPCASE(&VAR.), FORMAT NAME: %UPCASE(&FMT.), GROUPS: &GROUPS "; proc freq data=&data.; tables &var. / missing; format &var. &fmt..; run; title; *********************************************************************; * OUTPUT PROC FORMAT TO PRINT ; *********************************************************************; data _null_; set getrange3 end=eof; file print; TO=' -< '; if END='HIGH' then TO=' - '; SEMI=''; if eof then SEMI=';'; if _n_=1 then put "PROC FORMAT;" / " VALUE %quote(&FMT.)" / " " START TO END '="'LABEL'"'; else put " " START TO END '="'LABEL'"' SEMI; if eof then put "RUN;"; run;

21. 21 Getting a format from proc rank How the program works *********************************************************************; * OUTPUT CODE IN SAS TO PROGRAM ; *********************************************************************; data _null_; set getrange3 end=eof; file print; R=_N_-1; if _N_=1 then put '%macro ' "&VAR.;"/ " if &VAR. =. then &VAR.r =" R "; " ; else if START="LOW" then put " else if &VAR. <= "END" then &VAR.r =" R "; " ; else if END="HIGH" then put " else if &VAR. > "START" then &VAR.r =" R ";" ; else put " else if &VAR. > "START" and &VAR. <= "END" then &VAR.r =" R ";"; if eof then put '%mend;'; run; *********************************************************************; * CLEANUP ; *********************************************************************; proc datasets library=work nolist; delete rank mnrank smnrank getrange miss getrange2 getrange3; run; quit; %MEND FORMAT;

22. 22 Generating SAS code from Specs This example came from having to have code written in SAS and other languages from programming specs All of the specs involve one of two structures 1. A single variable cut into ranges 2. A matrix of two variables cut into ranges The first step in this process was to standardize the specs: Ranges Matrix (truncated: actually 20 by 20 is used today)

23. 23 Generating SAS code from Specs Program 1: outline Now that we have standard specs, we can outline a process to generate the code: 1.Specify metadata  Pathnames for specs and for macros  Names of Range1-RangeN  Names of Matrix1-MatrixM  Variable ranges to be used in each matrix 2.Read Specs  Read each range%Import_range  Read each matrix%Import_matrix 3.Create code  Code for each range %Code_ranges  Code for each matrix %Code_matrix 4.Output code  Code in SAS data _null_...  Code in other languages data _null_... 5.Clean up proc datasets %Run_Autocode

24. 24 Generating SAS code from Specs Program 1: Specify metadata %Let Path1={c:\...\}\Code; %Let Path2={c:\...\}\Specs; %Let Num_Ranges=16; %Let Range1 =Score_0;%Let var1 =Score; %Let Range2 =Severity_0;%Let var2 =Severity; %Let Range3 =Score_1;%Let var3 =Score; %Let Range4 =Severity_1;%Let var4 =Severity;..................... %Let Range15 =Score_91;%Let var15 =Score; %Let Range16 =Severity_91;%Let var16 =Severity; %Let Num_matrixes=8; %Let Matrix1_var1=Score_0;%Let Matrix1_var2=Severity_0; %Let Matrix2_var1=Score_1;%Let Matrix2_var2=Severity_1;..................... %Let Matrix7_var1=Score_90;%Let Matrix7_var2=Severity_90; %Let Matrix8_var1=Score_91;%Let Matrix8_var2=Severity_91;

25. 25 Generating SAS code from Specs Program 1: %Run_autocode %macro Run_autocode; %do index = 1 %to &Num_Ranges.; %Import_range(&&Range&index..);%end; %do index = 1 %to &Num_matrixes.; %Import_matrix(&&Matrix&index._var1,&&Matrix&index._var2)%end; %do index = 1 %to &Num_Ranges.; %Code_ranges(&&Range&index..,&&var&index..);%end; %do index = 1 %to &Num_matrixes.; %Code_matrix(&&Matrix&index._var1,&&Matrix&index._var2);%end; data _null_; set sas_code; file "&Path1.\RangeMatrix.sas" ; put @1 code $120.; run; data _null_; set bds_code;file "&Path1.\bds_code.txt" ; put @1 code $120.; run; proc datasets; delete MTX: (gennum=all); delete Range: (gennum=all); delete Sas_code: (gennum=all);delete bds: (gennum=all); quit; %mend Run_autocode;

26. 26 Generating SAS code from Specs Program 1: %Import stuff; %macro Import_range(range); PROC IMPORT OUT= WORK.&range. DATAFILE= "&Path2.\&range..xls" DBMS=EXCEL REPLACE; SHEET="Sheet1$"; GETNAMES=YES; MIXED=NO; SCANTEXT=YES; USEDATE=YES; SCANTIME=YES; RUN; %mend Import_range; %macro Import_matrix(var1,var2); PROC IMPORT OUT= WORK.MTX_&var1._&var2._defn DATAFILE= "&Path2.\matrix_&var1._&var2..xls" DBMS=EXCEL REPLACE; SHEET="Sheet1$"; GETNAMES=YES; MIXED=NO; SCANTEXT=YES; USEDATE=YES; SCANTIME=YES; RUN; %mend Import_matrix;

27. 27 Generating SAS code from Specs Program 1: %Code_ranges; %macro Code_ranges(range,var); data &range._sascode (keep=code); set &range. nobs=n; where min*1 ne. and max*1 ne.; length code $ 120.; if _n_=1 then code=" "; if _n_=1 then output; if _n_=1 then code='%'||"macro CodeRange&range.; "; if _n_=1 then output; if _n_=1 then code="*----------| Sas Code for &range. using &var. |---*;"; if _n_=1 then output; if _n_=1 then code=" &range. =99;"; if _n_=1 then output; code=" if &var. >="||compress(min)||" and &var. <= "||compress(max)||" then &Range. ="||compress(range)||";"; output; if _n_=n then code="%mend CodeRange&range.;"; if _n_=n then output; run; proc append base=sas_code data = &range._sascode force;run; %mend Code_ranges;

28. 28 Generating SAS code from Specs Program 1: %Code_Matrix; %macro Code_Matrix(v1,v2); data MTX_&v1._&v2._defn2; set MTX_&v1._&v2._defn (keep=_numeric_);run; data MTX_&v1._&v2._code (keep=code ); length code $ 120.; set MTX_&v1._&v2._defn2 nobs=n; array columns[*] F: ; dim=dim(columns); Range1=columns[1]; if _n_=1 then code=" "; if _n_=1 then output; if _n_=1 then code='%'||"macro CodeMatrix_&v1._&v2.; "; if _n_=1 then output; if _n_=1 then code="*------|Matrix of variables &v1. and &v2. |----*;"; if _n_=1 then output; if _n_=1 then code=" Matrixout =99;"; if _n_=1 then output; do i= 2 to dim; if i=2 then Range2=99; if i>2 then Range2=i-3; RiskRank=columns[i]; code=" if &v1.="||compress(range1)||" and &v2. = "||compress(range2)||" then Matrixout="||compress(RiskRank)||";"; if RiskRank ne. then output; end; if _n_=n then code="%mend CodeMatrix_&v1._&v2.; "; if _n_=n then output; if _n_=n then code="*--------------------------------------------------*;"; if _n_=n then output; run; proc append base=sas_code data = MTX_&v1._&v2._code force;run; %mend Code_Matrix;

29. 29 Generating SAS code from Specs Program 2: The Actual Code Ranges %macro CodeRangeScore_0; *----------| Sas Code for Score_0 using Score |---*; Score_0 =99; if Score >=0 and Score <= 589 then Score_0 =0; if Score >=590 and Score <= 601 then Score_0 =1; if Score >=602 and Score <= 610 then Score_0 =2; if Score >=611 and Score <= 618 then Score_0 =3; if Score >=619 and Score <= 626 then Score_0 =4; if Score >=627 and Score <= 662 then Score_0 =5; %mend CodeRangeScore; %macro CodeRangeSeverity_0; *----------| Sas Code for Severity_0 using Severity |---*; Severity_0 =99; if Severity >=0 and Severity <= 4994.6326 then Severity_0 =0; if Severity >=4994.6327 and Severity <= 5000 then Severity_0 =1; if Severity >=5000.0001 and Severity <= 7651.3413 then Severity_0 =3; if Severity >=7651.3414 and Severity <= 11027.44 then Severity_0 =4; if Severity >=11027.4401 and Severity <= 15335.9424 then Severity_0 =5; if Severity >=15335.9425 and Severity <= 20625.4816 then Severity_0 =6; if Severity >=20625.4817 and Severity <= 28399.5717 then Severity_0 =7; if Severity >=28399.5718 and Severity <= 41538.5343 then Severity_0 =8; if Severity >=41538.5344 and Severity <= 10000000 then Severity_0 =9; %mend CodeRangeSeverity;

30. 30 Generating SAS code from Specs Program 2: The Actual Code Matrix %macro CodeMatrix_Score_0_Severity_0; *------|Matrix of variables Score_0 and Severity_0 |----*; Matrixout =99; if Score_0=0 and Score_0 = 1 then Matrixout=3; if Score_0=0 and Score_0 = 2 then Matrixout=3; if Score_0=0 and Score_0 = 3 then Matrixout=3; if Score_0=0 and Score_0 = 4 then Matrixout=3; if Score_0=0 and Score_0 = 5 then Matrixout=3; if Score_0=0 and Score_0 = 6 then Matrixout=2; if Score_0=0 and Score_0 = 7 then Matrixout=2; if Score_0=0 and Score_0 = 8 then Matrixout=1; if Score_0=0 and Score_0 = 9 then Matrixout=1; if Score_0=0 and Score_0 = 10 then Matrixout=1; if Score_0=1 and Score_0 = 1 then Matrixout=4; if Score_0=1 and Score_0 = 2 then Matrixout=3; if Score_0=1 and Score_0 = 3 then Matrixout=3;..................... if Score_0=5 and Score_0 = 2 then Matrixout=4; if Score_0=5 and Score_0 = 3 then Matrixout=4; if Score_0=5 and Score_0 = 4 then Matrixout=4; if Score_0=5 and Score_0 = 5 then Matrixout=4; if Score_0=5 and Score_0 = 6 then Matrixout=4; if Score_0=5 and Score_0 = 7 then Matrixout=4; if Score_0=5 and Score_0 = 8 then Matrixout=4; if Score_0=5 and Score_0 = 9 then Matrixout=4; if Score_0=5 and Score_0 = 10 then Matrixout=3; %mend CodeMatrix_Score_0_Severity_0; *--------------------------------------------------*;

31. 31 Using SAS’s built in coding features There are several SAS features that will write code for you: www.sas.com Import/Export wizard SUGI papers that are online Enterprise Guide 1. Click your way through your task 2. View the code 3. Fix the code to get what you really want 4. Save

32. 32 Useful tools and tips Dimensionless arrays Proc SQL select into External files Proc Contents