Interactive Supercomputing Update IDC HPC User’s Forum, September 2008

Agenda Why am I here? Some trends… What does Interactive Supercomputing do? What’s new? (and app examples if there is time) 2

Why I’m here (at least partly) At the April User’s Forum meeting, somebody on a panel said something like; ‘I don’t want to learn MPI, I wish computer scientists would build tools to make my life easier.’ At that very moment, I was interviewing with Interactive Supercomputing… 3 

HPC Conventional Wisdom Includes; Computing cost continues to decline while reality cost continues to rise – creating pull for “in silico” techniques More compute power is needed for multiple reasons; More fidelity; multi-physics; data explosion… Increasing complexity in the compute engine More cores, not faster cores; Potentially less capability / core; Multi-threading HW; The usual pain points are only getting worse. E.g. memory and i/o BW/FLOP, latencies… Creating a more difficult strategy choice for development; multicore, manycore, gpu, thin, thick or fat nodes… 4 There is a strong need for new development tools -- even for experienced parallel programmers. But in the meantime…

The Domain Expert View Swamped by the velocity of their own domain Long ago moved from 3GL’s to VHLL’s E.g. from FORTRAN to some variant of the M language (most likely Matlab®) … and don’t want to move back Now have enough data and math to need more than one desktop worth of compute Our surveys show as many as 40% of users are performance limited for some applications 5

What we do: Make high performance computing accessible to the widest possible range of users; enable domain experts to develop and deploy high performance parallel applications easily 6 Note: “server” includes “cluster”

Star-P Value Proposition Higher Productivity, Quicker Results, No complex programming

Star-P Open Software Platform

What’s New? (courtesy PNNL) 9 We call this step Knowledge Discovery

Why Star-P for Knowledge Discovery? Need to match algorithm to data means users need to experiment with multiple algorithms VHLL makes code changes easy Note, we see this requirement often – e.g. in finance and intelligence where codes must be continually adapted Size of data means HPC is required for experiments With Star-P, good enough speed-up is achieved quickly Star-P includes KD functions which run in parallel and Parallel I/O to remove that potential bottleneck 10

Factoring network flow behavior [Karpinski, Almeroth, Belding]

Algorithmic exploration Many NMF variants exist in the literature –Not clear how useful on large data –Not clear how to calibrate (i.e., number of iterations to converge) NMF algorithms combine linear algebra and optimization methods Basic and “improved” NMF factorization algorithms implemented: –euclidean (Lee & Seung 2000) –K-L divergence (Lee & Seung 2000) –semi-nonnegative (Ding et al. 2006) –left/right-orthogonal (Ding et al. 2006) –bi-orthogonal tri-factorization (Ding et al. 2006) –sparse euclidean (Hoyer et al. 2002) –sparse divergence (Liu et al. 2003) –non-smooth (Pascual-Montano et al. 2006)

NMF traffic analysis results NMF identifies essential components of the traffic Analyst labels different types of external behavior

Computational Ecology Modeling dispersal of species within a habitat (to maximize range) Large geographic areas, linked with GIS data Blend of numerical and combinatorial algorithms Brad McRae and Paul Beier, “Circuit theory predicts gene flow in plant and animal populations”, PNAS, Vol. 104, no. 50, December 11, 2007

Results Solution time reduced from 3 days (desktop) to 5 minutes (14p) for typical problems Aiming for much larger problems: Yellowstone-to-Yukon (Y2Y)

16 Thank You! David Rich VP Marketing