Presentation on theme: "Multiprocessing with SAS ® Software Now Bill Fehlner, Kathleen Wong, Kifah Mansour SAS Toronto."— Presentation transcript:
Multiprocessing with SAS ® Software Now Bill Fehlner, Kathleen Wong, Kifah Mansour SAS Toronto
Multiprocessing n Purpose – complete a job sooner n Benchmarks – it really works! n Hidden sources of computing power n MP CONNECT functionality n Examples – how hard is it? n How to learn more
Complete a job sooner n It takes the same amount of resource, but the task is done sooner.
The Purpose of Multiprocessing n Complete a job in less total elapsed time. n Increase usage of available CPUs.
The Practice of Multiprocessing In years past, life was simple. Each computer had one central processing unit (CPU) and was not connected to another computer. SAS was created as a single-threaded application, which means the program executes in a top-down approach on the one processor.
The Practice of Multiprocessing Today a computer can have multiple processors, or be part of a network. Version 8 and previous versions of SAS software are still single-threaded applications, but now have the ability through MP CONNECT to take advantage of additional processors for different steps.
The Practice of Multiprocessing n Divide application into sub-units of work. FREQ ProcedureTABULATE ProcedureUNIVARIATE Procedure FREQ Procedure TABULATE Procedure UNIVARIATE Procedure n Execute independent tasks in parallel.
What if not every task is independent? Extract Oracle Data Read/Summarize SAS Data Set Read/Summarize Raw Data File 0elapsed time Merge Data
Benchmarks n SUN Solaris n IBM RS/6000 n HP 9000
Benchmark – SUN Solaris n Solaris 7 with twelve 400 MHZ Ultrasparc processors n 14 SAS steps operating on one table, including SORT, SUMMARY, FREQ n 480 MB per data set
Hidden Sources of Computing Power n Under utilized servers n Workstations after hours
Hidden Sources of Computing Power n Under utilized servers
Hidden Sources of Computing Power n Workstations after hours
MP CONNECT functionality n Start and finish asynchronous sessions n Communicate between sessions n Process return codes n Implement program dependencies n Manage the SAS sessions
SAS/CONNECT Monitor Window The list of tasks is dynamically updated as new tasks start, and the Status field changes from “Running” to “Complete”, as appropriate.
NOTIFY= YES Option displays a message window that indicates the completion of a task.
Examples n Scaling up – concurrent sessions on one machine n Scaling out – concurrent sessions on distributed processors
Scaling up – one machine Statements required: What’s new in MP CONNECT n Options... ; Autoconnect = Sascmd = n Rsubmit taskname... ; Wait=no Persist = n Waitfor.... ; n Signoff taskname;
Scaling up - one machine Task 1: convert and sort a text file. Task 2: convert and sort a text file. 0elapsed time Interleave Data
Scaling up – one machine options SASCMD= '/bin/SAS/V8/sas.exe -nosyntaxcheck ' autosignon=YES ; rsubmit task1 wait=no sysrputsync=yes; data sample1; length type1 $12 region $13;
Scaling out – distributed processing Statements required: n Filename rlink... ; 3 diff remote systems n %let taskname =... ; n Signon taskname; n Rsubmit taskname... ; Wait=no n Waitfor.... ; n Signoff taskname;
Client (Local) Server (Remote) Task 1 Task 2 Asynchronous Execution on Multiple Machines Server (Remote)
Scaling out – distributed processing filename rlink '!sasroot\connect\saslink\tcpunix.scr'; %let task1 = bcom1; signon task1; /* unix host */ rsubmit task1 wait=no ; data sample1; length type1 $12 region $13;
How to learn more n SUGI articles www.sas.com/usergroups/sugi/proceedings/ n Cheryl Garner, SUGI 25, paper 16 n John E. Bentley, SUGI 26, paper 269 n John E. Bentley, SUGI 27, paper 107
How to learn more n SAS courses www.sas.com/service/edu/cantrain/ n Multiprocessing with SAS ® Software n SAS ® Macro Language n Optimizing SAS ® Programs