NSF DMS VOStat - HEAD 2004 Ashish Mahabal VOStat Arming Astronomers with Advanced Statistics Caltech: A. Mahabal, M. Graham, S.G.Djorgovski, R. Williams Penn State: J. Babu (PI), E. Feigelson CMU: R. Nichol, D. Van DenBerk, L.Wasserman

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Use of statistics astronomical studies per year 5% have “statistics” in their abstract 20% treat variable objects or multivariate datasets

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Traditional methods Fourier transform (Fourier 1807) Least sq. and chisq (Legendre 1805, Pearson 1901) Kolmogorov-Smirnov test (Kolomogrov 1933) Principal Component Analysis (Hotelling 1936)

NSF DMS VOStat - HEAD 2004 Ashish Mahabal VOStat Web based service Simple and sophisticated statistical routines Large datasets Public domain (R)/ specially written General purpose and Virtual Observatory

NSF DMS VOStat - HEAD 2004 Ashish Mahabal VOStat ASCII / VOTABLE as input (can be used as an intermediate block for a VO based pipeline) CGI routines as prototypes (few 1000 lines) Webservices (Java GUI) - hundreds of thousands of lines (limited by R’s capabilities) - distributed, multi-OS, multi-language

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Examples of available functions Descriptive statistics (e.g. boxplot) Two- and k-sample tests (e.g. Wilcoxon rank-sum test) Density estimation (e.g. Kernel smoothing) Correlation and regression (e.g. PCA) Censored data (e.g. Survival) Multivariate classification (e.g. H clustering) External functions (e.g. K-density)

NSF DMS VOStat - HEAD 2004 Ashish Mahabal User-friendly GUI Columns are autoselected (and can be deselected) Parameter choices for functions are conveniently placed Can be used from your own webpages on tables residing elsewhere

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Toy Demos Rediscovering HR diagram Rediscovering FP of Globular Clusters Looking for outliers in color-color space

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Rediscovering HR diagram Hyades stars (Hipparcus main catalog) Mean/median/boxplot Density estimation (Histogram) Kernel smoothing Correlation matrix X-Y plot Multivariate clustering

NSF DMS VOStat - HEAD 2004 Ashish Mahabal X-Y plot between Vmag and B-V reveals the famous structure in the dataset: the color-magnitude of bright stars showing the main sequence, giant branch (with red clump stars), and a few Hyades white dwarfs.

NSF DMS VOStat - HEAD 2004 Ashish Mahabal FP of Globular clusters Matrix of pairwise correlation coefficients Pairwise plots Principal Component Analysis

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Core parameters as a group tend to be highly correlated, unlike the half-light parameters. This is indicative of the dynamical evolution driven by the core collapse.

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Exploring outliers Palomar-QUEST synoptic sky survey 9 mix-and-match colors from 8 filters Aim: finding outliers in color-color space for spectroscopic follow-up 1000 random objects

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Boxplot Reveals relationships between colors (mean, median, overlap, outliers)

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Clustering K-means provides various cluster centers along with withinss and a list of possible outliers

NSF DMS VOStat - HEAD 2004 Ashish Mahabal

NSF DMS VOStat - HEAD 2004 Ashish Mahabal K-density Probability - density association for outliers

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Visual confirmation (found from 1000 random objects)

NSF DMS VOStat - HEAD 2004 Ashish Mahabal Summary Web-based VO compatible Public domain and specialized routines