Presentation on theme: "Why do I build CI and what did it teach me? Miron Livny Wisconsin Institutes for Discovery Madison-Wisconsin."— Presentation transcript:
Why do I build CI and what did it teach me? Miron Livny Wisconsin Institutes for Discovery Madison-Wisconsin
At the WIDs we Provide Researchers with State Of The Art CI Through Leadership in High Throughput Computing (HTC)
CI = Hardware + Software + People
Because I like building things! Because I like building things!
CI We Have Been Building : DENet – Discrete event simulation environment for distributed systems Present: Condor – Technologies for High Throughput Computing 2003-Present: OSG –High Throughput Computing at a national scale 2003-Present: GLOW – High Throughput Computing at the campus (UW) level 2006-Present: BaTLab – Build and Test facility for the NMI community
Better Computer Science!
“Why are you leaving academia and taking a job in industry?” “I want to have impact! ” 14
In the words of Mike Carey “ I left academia for industry because I was drawn to the idea of getting more direct access to real problems - from customers and challenges encountered while building commercial-grade software - because I felt like I was in somewhat of a mode of inventing and solving problems, at least w.r.t. some of the things I'd been working on. Sure, that was leading to many written/submitted/accepted papers, but it was somehow less than satisfying after awhile.” 15
P. Brady, the LIGO Data Analysis Software Chair, summarized the role of Condor technologies – “Condor manages LIGO compute-intensive data analysis jobs on more than 23,000 CPU slots offered by nine Linux clusters operated by the LSC. More than 250 LSC scientists rely heavily on Condor technologies to manage complex data analysis workflows. Over the years, LIGO and the Condor team have developed a strategic partnership resulting in many new software features that benefit LIGO and the entire Condor user community. We eagerly look forward to continuing this partnership since Condor technologies are critical to the continued success of the LIGO Data Grid as a platform for gravitational wave science. ”
Better hearing with Cochlear Implants Normal hearing listeners need sound’s temporal fine structure information to understand speech in noise, localize sounds, and recognize talkers and melodies. Conventional cochlear implant speech processing strategies discard temporal fine structure and only represent the temporal envelopes of sound. Experimental question: If we set the timing the stimulating electric pulses of cochlear implants to represent this temporal fine structure, will cochlear implant listeners hear better? $ 15 minutes The algorithm for extracting the fine structure and shifting the pulses is computationally expensive, and creating the over 25,000 stimuli for a cochlear implant experiment would take a lab computer 260 days to complete. OSG allowed us to make all the stimuli within a day. Tyler Churchill Binaurial Hearing and Speech Lab Waisman Center
21 C HT C team 2011
“The members of the OSG are united by a commitment to promote the adoption and to advance the state of the art of distributed high throughput computing (DHTC) – shared utilization of autonomous resources where all the elements are optimized for maximizing computational throughput.”
It is hard work, a long term commitment and extremely rewarding!
Real and hard Computer Science problems are only exposed when you do it for “real” and at “scale”
Open Science Grid (OSG)OSG HTC at the National Level
Stick to Principals!
The words of Koheleth son of David, king in Jerusalem ~ 200 A.D. Only that shall happen Which has happened, Only that occur Which has occurred; There is nothing new Beneath the sun! Ecclesiastes Chapter 1 verse 9 Ecclesiastes, (קֹהֶלֶת, Kohelet, "son of David, and king in Jerusalem" alias Solomon, Wood engraving Gustave Doré (1832–1883)
Perspectives on Grid Computing Uwe Schwiegelshohn Rosa M. Badia Marian Bubak Marco Danelutto Schahram Dustdar Fabrizio Gagliardi Alfred Geiger Ladislav Hluchy Dieter Kranzlmüller Erwin Laure Thierry Priol Alexander Reinefeld Michael Resch Andreas Reuter Otto Rienhoff Thomas Rüter Peter Sloot Domenico Talia Klaus Ullmann Ramin Yahyapour Gabriele von Voigt We should not waste our time in redefining terms or key technologies: clusters, Grids, Clouds... What is in a name? Ian Foster recently quoted Miron Livny saying: "I was doing Cloud computing way before people called it Grid computing", referring to the ground breaking Condor technology. It is the Grid scientific paradigm that counts!
Claims for “benefits” provided by Distributed Processing Systems – High Availability and Reliability – High System Performance – Ease of Modular and Incremental Growth – Automatic Load and Resource Sharing – Good Response to Temporary Overloads – Easy Expansion in Capacity and/or Function P.H. Enslow, “What is a Distributed Data Processing System?” Computer, January 1978
Questions Is building CI in support of research science or engineering? Should CI for research be built and operated by commercial organizations? Should all scientific software funded by federal agencies be open source? How should NSF decide how much to invest in in building and operating CI?