Writing Robust SAS Macros Yan Qiao Principle Programmer, BeiGene Wuhan,12SEP2019

Author Biography Yan Qiao Currently a Principle Programmer with BeiGene. Worked as applications & tools developer for more than 6 years with Merck Serono. Developed a series of statistical applications & tools, including SAS macros, R functions and so on.

Robust macros should… Validate inputs Handle potential failures Notify the caller of the outcome of execution Clean up environment Be easily maintained

Macro developer should… Define parameters Validate parameters Check and react to outcomes of each step Return status of the execution Restore environment Test and document

A simple macro to find duplicates… %macro finddups(data=_last_, out=_dups, by=, where=) ; %local i lastby ; proc sort data=&data out=&out ; by &by ; %if (%quote(&where) ^= %str()) %then %do ; where &where ; %end ; run ; /* determine the last by variable for use with first. and last. */ %let lastby=%scan(&by,-1,%str( )); data &out ; set &out end=__end; if first.&lastby + last.&lastby < 2 then output &out ; %mend ;

What if… The caller forgets to specify value for parameter by. The input dataset specified in parameter data dose not exist. The variable name specified in parameter by dose not exist in the input dataset. The condition specified in parameter where makes no sense. There is no duplicate observation in the input dataset.

Define Parameters Use named parameters instead of positional parameters. Choose suitable parameter names by following naming conventions used for options and statements in SAS procedures: data, out, infile, file, var, by. Define valid values for parameters: Y/N.

Validate Parameters Required parameters are not missing. Parameters take valid values. Existence of inputs.

Parameter Validation %local _pi _params _param rc; %let _params=.data.out.by.; %let _pi=1; %do %while(%scan(&_params,&_pi,.)^=%str()) ; %let _param=%scan(&_params,&_pi,.); %if %quote(&&&_param) = %str() %then %do ; %put ERROR: &sysmacroname: Parameter %upcase(&_param) is required.; %let rc=1; %end; %else %if &_param=data %then %do ; %if %sysfunc(exist(&&&_param))=0 %then %do ; %put ERROR: &sysmacroname: %upcase(&&&_param) does not exist.; %let rc=2; %end ; %let _pi=%eval(&_pi+1) ; %if &rc=1 or &rc=2 %then %return ;

Check and React to Outcomes syserr contains a return code status set by some SAS procedures and the data step. (0=no error/warning; 4=warning; >4=error). sysfilrc contains the return code from the last filename statement. syslibrc contains the return code from the last libname statement.

Check and React to Outcomes proc sort data=&data out=&out ; by &by ; %if (%quote(&where) ^= %str()) %then %do ; where &where ; %end ; run ; %if &syserr > 4 %then %do ; %put ERROR: &sysmacroname: error sorting the dataset. ; %return ; %end;

Return Code Define a global macro variable for return code. Use consistent return code in macro system.

Return Code %macro finddups(data=_last_,out=_dups,by=, rc=rc_finddups) ; %if %quote(&rc) = %str() %then %do ; %put NOTE: &sysmacroname: Using RC_&sysmacroname for the RC parameter ; %let rc=rc_&sysmacroname ; %end ; %global &rc ; %let &rc=9999 ;   proc sort data=&data out=&out ; by &by ; run ; %if &syserr>4 %then %do ; %let &rc=3 ; %return ; ... %let &rc=0; %mend;

Restore Environment Resetting options to their original values. Deleting temporary files and data sets. Clearing file or library references used internally by the macro. Ensuring that no undesired global macro variables have been defined.

Restore Environment proc sort data=&data out=_finddups_tmp ; by &by ; run ; %if &syserr>4 %then %do ; %let &rc=3 ; %goto DONE ; %end ; data &out ; set _finddups_tmp ; ... %let &rc=4 ; %let &rc=0 ; %DONE: %if %sysfunc(exist(_finddups_tmp)) %then %do ; proc datasets lib=work nolist ; delete _finddups_tmp; quit ; %end ; %mend;

Test and Document Tests on error handling/functionality of the macro. Validation protocol, plan and report. User documentation: a concise description of the purpose of the macro; type, expected values, default values and detailed descriptions of parameters; description of inputs; description of the outputs; examples. aefreq_fpg_v1.pdf

To make the simple example ROBUST… /* Function: Create a data set containing duplicates of an input data set. This is almost the complement of PROC SORT NODUPKEY- it throws away all records having a unique key. Return codes: 0: success; 1: missing value for parameter DATA; 2: missing value for parameter OUT; 4: missing value for parameter BY; 3,5,6,7: combination of parameters missing; 8: input dataset dose not exist; 9: by variables do not exist in input dataset; 101: error during the sort step; 102: error during the data step searching for duplicates; 9999: unexpected error. */

To make the simple example ROBUST… %macro finddups(data=_last_, /* input data set */ out=_dups, /* output data set */ by=, /* by variables */ where=, /* where clause */ rc=rc_finddups /* global macro variable for return code */ ); /* define return code*/ %if %quote(&rc) = %str() %then %do ; %put NOTE: &sysmacroname: Using RC_&sysmacroname for the RC parameter ; %let rc=rc_dups; %end ; %global &rc ; %let &rc=9999 ;

To make the simple example ROBUST… /* parameter validation */ %Local i lastby nmiss _params _param ; %let _params=.data.out.by.; %let i=1; %let nmiss=0 ; %do %while(%scan(&_params,&i,.)^=%str()) ; %let _param=%nrbqupte(%scan(&_params,&i,.)); %if &&&_param = %str() %then %do ; %put ERROR: &sysmacroname: Parameter %upcase(&_param) is required.; %let nmiss=%eval(&nmiss+2**(&i-1)) ; %end; %let i=%eval(&i+1) ; %if &nmiss>0 %then %do ; %let &rc=&nmiss ; %return; %end ;

To make the simple example ROBUST… /* input dataset/variables validation */ %let dsid = %sysfunc(open(&data.)); %if &dsid=0 %then %do ; %put ERROR: &sysmacroname: input dataset &data does not exist.; %let rc=8; %return; %end ; %Local j _by_var ; %let j=1; %do %while(%scan(&by,&j, ,%str( ));)^=%str()) ; %let _by_var=%nrbqupte(%scan(&by,&j,%str())); %if %sysfunc(varnum(&dsid., &_by_var.))=0 %then %do ; %put ERROR: &sysmacroname: variable &_by_var does not exist in the input dataset.; %let rc=9; %let j=%eval(&j+1) ; %end;

To make the simple example ROBUST… %let lastby=%scan(&by,-1,%str( )); data &out; set _finddups_tmp end=__end; by &by ; if first.&lastby + last.&lastby < 2 then output &out ; run; %if &syserr > 4 %then %do ; %put ERROR: &sysmacroname: error building duplicates data set &out. ; %let &rc=102 ; %goto DONE ; %end; proc sort data=&data out=_finddups_tmp ; by &by ; %if (%quote(&where) ^= %str()) %then %do ; where &where ; %end ; run ; %if &syserr > 4 %then %do ; %put ERROR: &sysmacroname: error sorting the data set. ; %let &rc=101 ; %goto DONE; %end;

To make the simple example ROBUST… /* Set return code to 0 for success*/ %let &rc=0 ; %DONE: /* restore environment */ %if %sysfunc(exist(_finddups_tmp)) %then %do ; proc datasets lib=work nolist ; delete _finddups_tmp; quit ; %end ; %mend;

Q&A Thank you! Name: Yan Qiao Organization: BeiGene E-mail: yan.qiao@beigene.com Paper Number xx-xxx 12/5/2019