1. Increase Your Productivity by Doing Less
Next Presentation:
Increase Your Productivity by Doing Less
Presenter: Arthur Tabachneck
Art holds a PhD from Michigan State University, has been a SAS user since 1974, is president of the Toronto Area SAS Society and has received such recognitions as the SAS Customer Value Award (2008), SAS-L Hall of Fame (2011), SAS Circle of Excellence (2012) and, in 2013, was recognized as being the first SAS Discussion Forum participant to be awarded more than 10,000 points
Copyright © 2010, SAS Institute Inc. All rights reserved. 1

2. Increase Your Productivity by Doing Less
Arthur Tabachneck
Thornhill, ON Canada
Xia Ke Shan
Beijing, China
Joe Whitehurst Robert Virgile
Atlanta, GA Lexington, MA
Copyright © 2010, SAS Institute Inc. All rights reserved.

3. our Truth in Advertising commitment
WARNING: The paper I'm about to present:
was originally a minor subset of a major paper I presented in the Beyond the Basics section
it was written because we thought of the idea for the original paper too late to submit to SGF2013 but, due to a cancellation, SGF had an opening for a 10 minute Coder's Corner paper
but, while it is clearly a disturbing example of extreme opportunism, it may be the most significant 10 minute paper/presentation I've had the privilege of co-authoring
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 3

4. What the %transpose() macro is
It's a SAS macro
Looks and feels like PROC TRANSPOSE
Has all of the PROC TRANSPOSE options and statements + two additional ones
Can run between 9 and 12 as much as 200 times or more faster than using PROC SORT and PROC TRANSPOSE
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 4

5. Have you ever had to flip a SAS dataset from being tall to being wide?
i.e., from:
idnum
date
var1 1
2001JAN SD
2001FEB EF
2001MAR HK 2 GH
2001APR MM
2001MAY JH
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 5

6. flipping a SAS dataset to:
idnum
var1_2001JAN
var1_2001FEB
var1_2001MAR
var1_2001APR
var1_2001MAY 1 SD EF HK 2 GH MM JH
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 6

7. if you have, you are probably already familiar with PROC TRANSPOSE
Note to self:
remember to first sort the data
proc transpose
data=have
out=want (drop=_:)
prefix=var1_;
by idnum;
var var1;
id date;
run;
Not difficult, but you need to know
which are options
which are statements
and, if needed, remember to sort your data before hand
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 7

8. How many of you have ever:
forgotten to run proc sort before running another proc that required sorted data?
run proc sort but didn't include the options that can make the process more efficient (e.g., noequals, presorted and tagsort)?
run a proc that only used a few of a file's variables, but didn't include a keep dataset option to limit the amount of data that had to be processed?
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 8

9. Compare the performance of the following two sets of almost identical code run on a file with 40,000 records and 1,002 variables
PROC SORT data=have out=need;
by idnum date;
run; took 2.41 seconds CPU time
PROC TRANSPOSE data=need out=want (drop=_:)
prefix=var1_Qtr;
by idnum;
var var1;
id date;
format date Qtr1.;
run; took 0.74 seconds CPU time
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 9

10. Compare the performance of the following two sets of almost identical code run on a file with 40,000 records and 1,002 variables
PROC SORT data=have (keep=idnum date var1)
out=need noequals;
by idnum date;
run; took 0.33 seconds CPU time
PROC TRANSPOSE data=need out=want (drop=_:)
prefix=var1_Qtr;
by idnum;
var var1;
id date;
format date Qtr1.;
run; took 0.16 seconds CPU time
6.39 times faster
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 10

11. More importantly compare the real time performance of running the two versions of PROC SORT
PROC SORT data=have out=need;
by idnum date;
run; took seconds real time
PROC SORT data=have (keep=idnum date var1)
out=need noequals;
by idnum date;
run; took 0.57 seconds real time
192.2 times faster
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 11

12. You can achieve the same performance gains
by running the following code
%transpose(data=have, out=want (drop=_:),
prefix=var1_Qtr, by=idnum, var=var1,
id=date, sort=yes, format=date Qtr1.)
The macro's benefits
less code thus less chance for error
same efficiency as the optimized code
no need to know which PROC TRANSPOSE features are options and which are statements
further code reductions possible by setting common defaults for the named parameters
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 12

13. How the %transpose() macro works
First, all of the parameters are declared:
%macro transpose( data=, out=, var=, prefix=,
suffix=, let=, name=, label=,
sort_options=, id=, sort=, idlabel=,
delimiter=, by=, copy=, format=);
the same options and statements you
would use with PROC TRANSPOSE +
two new named parameters (sort and sort_options)
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 13

14. If &var is null populate it with all numeric variables
data _temp;
set &data. (obs=1 drop=&by. &id. &copy.);
run;
%if %length(&var.) eq 0 %then %do;
proc sql noprint;
select name
into :var separated by " "
from dictionary.columns
where libname="WORK" and
memname="_TEMP" and
type="num" ;
quit;
%end;
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 14

15. If the sort parameter eq 'yes' run proc sort
%if %sysfunc(upcase("&sort.")) eq "YES" %then %do;
proc sort data=&data
out=_temp
by &by. &id.;
run;
%let data=_temp;
%end;
(keep=&by. &id. &var. &copy.)
&sort_options. noequals;
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 15

16. using all of the specified parameters
Run PROC TRANSPOSE
using all of the specified parameters
proc transpose data=&data. (keep=&by. &id. &var. &copy.)
%if %length(&delimiter.) gt 0 %then delimiter=&delimiter.;
%if %length(&label.) gt 0 %then label=&label.;
%if %length(&let.) gt 0 %then let;
%if %length(&name.) gt 0 %then name=&name.;
%if %length(&out.) gt 0 %then out=&out.;
%if %length(&prefix) gt 0 %then prefix=&prefix.;
%if %length(&suffix) gt 0 %then suffix=&suffix.; ;
%if %length(&by.) gt 0 %then by &by.;;
%if %length(&copy.) gt 0 %then copy &copy.;;
%if %length(&id.) gt 0 %then id &id.;;
%if %length(&idlabel.) gt 0 %then idlabel &idlabel.;;
%if %length(&var.) gt 0 %then var &var.;;
%if %length(&format.) gt 0 %then format &format.;;
run;
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 16

17. finally, delete temporary file
proc delete data=work._temp;
run;
%mend transpose;
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 17

18. Potential Applications
transposing files quicker
using the macro as a template for creating similar macros which incorporate sort and efficiency options for other SAS procs that could benefit from the approach
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 18

19. Presentation Overview
How this paper came about 
What the %transpose macro is 
The macro's benefits 
How the macro works 
Potential applications of the macro 
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 19

20. Questions?
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved. 20

21. Your comments and questions are valued and encouraged
Contact the Authors
Arthur Tabachneck, Ph.D.
President, myQNA, Inc.
Thornhill, ON
code name: art297
Joe Whitehurst
High Impact Technologies
Atlanta, GA
code name: joe whitehurst
Robert Virgile
Robert Virgile Associates, Inc.
Lexington, MA
code name: astounding
Xia Ke Shan
Beijing, China
code name: ksharp
#SASGF11
Copyright © 2010, SAS Institute Inc. All rights reserved.