© 2012 IBM Corporation 1 IBM PureData for Analytics Clustering three ways with Open Source R

© 2012 IBM Corporation 2 Using R with Puredata for Analytics Small data outside database Single Model, Serial Model Processing Large data inside database Single Model, Serial Model Processing Many small data inside database Many Model, Parallel Model Processing e.g. Bulk Parallel Execution Pull data down from database Run R on desktop or dedicated server Call INZA functions from R Process data directly against DB tables Push R into database Process data directly against DB tables Small data inside database Single Model, Serial Model Processing Push R into database Process data directly against DB tables

© 2012 IBM Corporation 3 Using R with Puredata for Analytics Small data outside database Single Model, Serial Model Processing Large data inside database Single Model, Serial Model Processing Many small data inside database Many Model, Parallel Model Processing e.g. Bulk Parallel Execution Pull data down from database Run R on desktop or dedicated server Call INZA functions from R Process data directly against DB tables Push R into database Process data directly against DB tables Small data inside database Single Model, Serial Model Processing Push R into database Process data directly against DB tables Analysis only looks at the last three scenarios

© 2012 IBM Corporation 4 Comparing performance for single model in-database Number of Observations INZA wrapper from R: nzKMeans cclust run IDB with nzSingleModel 500,000 user system elapsed user system elapsed ,000,000user system elapsed user system elapsed ,000,000user system elapsed user system elapsed ,000,000user system elapsed user system elapsed ,000,000user system elapsed user system elapsed Would expect nzKMeans to outperform cclust in-database between 5M and 6M observations Note: Tests run on a first-gen twin-fin Note: performance numbers variations are relative due to system being used during the testing

© 2012 IBM Corporation 5 Bulk-parallel execution of cclust (10K observations for each) Number of Models cclust run IDB with nzBulkModel Average time per model 50user system elapsed user system elapsed user system elapsed In general, these results would be significantly superior to running cclust serially in a dedicated environment simply due to R execution overhead and accounting for additional time required for data movement and/or partitioning

© 2012 IBM Corporation 6 Clustering three ways with Open R and IBM Puredata for Analytics  Using wrapper for INZA KMEANS (Stores resulting model in-database), single model data.nz <- nz.data.frame("BENCHMARK_DATA") system.time( nz.clust5 <- nzKMeans(data.nz, k=5,maxiter=1000,distance="euclidean",id="ID", getLabels=F,randseed=1234, outtable="admin.DATA_2_clust5d", format="kmeans",dropAfter=T) )  Running R in-database, single model (Returns resulting model to client.) system.time( data.cclust <- nzSingleModel(data.nz[,2:16], function(df){ require(cclust); cclust(as.matrix(df),5,iter.max=1000, verbose=FALSE,dist="euclidean",method="kmeans") }, force=TRUE ))  Running R in-database, bulk parallel model (Stores resulting models in-database, returns list of models by INDEX) # ua_ct is col 6, the “index” or grouping column system.time( data.cclust <- nzBulkModel(data.nz[data.nz$ID< ,2:16], 6, function(df){ require(cclust); cclust(as.matrix(df),5,iter.max=1000,verbose=FALSE,dist="euclidean",method="kmeans") }, output.name="CCLUSTBULKMODEL", clear.existing=TRUE ) )

© 2012 IBM Corporation 7 Bulk-parallel execution of cclust: Result Details Number of Rows Number of Models TimingsOverall Average Elapsed per Model Rows per Model 0.5 M50user system elapsed K 1 M100user system elapsed K 2 M100user system elapsed K 4 M500user system elapsed K 5 M500user system elapsed K