Software Technology Workshop Brent Fultz Welcome to the DANSE How to explain DANSE Component example Immediate tasks

DANSE (distributed data analysis for neutron scattering experiments) Component-Based Runtime Environment Pyre framework makes it possible to support interoperable components Science Subfield Development focused development has real leverage today

New Science Better Science Ease of Use Software Stability Why DANSE?

Aivazis Billinge The DANSE Effort Fultz Ustundag Kienzle Butler Fultz/Trouw

DANSE / SNS Interactions Dysfunctional

ISIS Paolo Radaelli, Chick Wilson,Toby Perring, Steve Bennington TUM Jens Krueger, Joern Beckmann ANSTO Nick Hauser Chalk River Mel Potter Open University (UK) Lyndon Edwards KEK Toshiya Otomo JAERI Yukio Morii A component framework offers a way for these persons to make real contributions. Real International Collaborators

8 Workshops, 2 Polls, >3 Years We Have Learned 1. Users Want to See Data Now Raw, I(Q), I(E), S(Q,E) Responsive, intuitive interface Reasonable defaults Customization 2. Users Want to Do Science With the Data Self-evident, but the subfields must plan their own paths forward.

z - Field of Research y - Science Sophistication x - Software Sophistication Orthogonal Dimensions of Users

Golden Rules You should be allowed to use your favorite tools in the SNS software system. Please don’t take away my tools. Please allow for new tools in the future. Please allow for development and maintenance. You should be allowed to use your favorite tools in the SNS software system. Please don’t take away my tools. Please allow for new tools in the future. Please allow for development and maintenance. Expectation Darwinian evolution will prevail.

Can We All Get Along? – Benefits Power to do new science: integration with theory optimize experimental procedures Coherence and ease of use: local or distributed with favorite interface Power to do new science: integration with theory optimize experimental procedures Coherence and ease of use: local or distributed with favorite interface Requirement We need a common framework so we all can participate. We need a common framework so we all can participate.

NSSA Software User Poll Questions on: remote control, collaboratory, remote data analysis No support for remote control (too risky) Strong support for: remote data analysis collaboratories Overwhelming support for: standardization of data analysis

Inelastic Subproject Survey

ARCS Software Roadmap Sept. 2001

Experimental Inelastic Neutron Scattering

Component Templates Standard Data Streams Python objects Standard communication protocol between components that can reside anywhere Data Flow Paradigm histograms tables meta-data Code Place Name Place Initiate, terminate, error properties

'''Multiphonon.py Calculates the multiphonon scattering, using a phonon DOS... ''' from mpFunctions import * def run(All_Inputs_List): """Multiphonon.py main loop...""" # check user inputs for validity, get data from disk checkUserInput(arglist) arglist = setupRun(arglist) # 1-phonon quantities, multiphonon terms arglist = onePhonon(arglist) arglist = multiPhonon(arglist) # prepare results for output, send to disk, etc. arglist = prepareResults(arglist) outputResults(arglist) return if __name__ == '__main__': """Run main loop if launched standalone.""" from mpUserInput import * run(All_Inputs_List) Encapsulation Abstraction Launched standalone or Inside Analysis Procedure

Show Code Run Code

grid services light gsl Built from technologies that exist today Extension of Pyre framework ssh, scp, XML-RPC, (SOAP substitution planned) Adaptable later to OGSI To a remote sysadmin, the gsl client on your laptop seems like an unusually smart and fast user logon to remote systems find environments on remote systems Service Factory daemon spawns processes client communicates with individual processes cvs checkout and build needed components

Quality Standards and Validated Code: define specifications OOP design testing and validation Version Control (cvs access) Release Management (first case: Let’s coordinate our component submissions: Aug 13, 2004 ) Software Project Issues

When analysis components are building blocks, what will scientists build?

Cost to Build Software We wrote a draft WBS in Nov estimate of 125 man-years (!) Five-Year Budget 8 M$ scientific subfields (4.5 programs) 7 M$ technology (7 FTE + hardware)

Software Development

Software Project Schedule

Conceptual Engineering Design Phase Test component framework with gsl extensions. Component development by subproject leaders (they can then estimate effort for tasks). Coordinate with SNS on precise tasks and technical requirements (facilities infrastructure, GUI, etc.). Write the WBS and proposal for DANSE CONST. Project initiation tasks (staffing, controls, comm, twiki). Teach technology, open to worldwide participation.

Work for Subproject Leaders Poll your community: Aug 20, Coordinate with SNS instrument scientist on specific tasks and requirements. Enumerate the tasks and estimate effort: Sept. 10, Adopt an instrument (ARCS adopted Pharos). Learn OOP (Part VI in Learning Python). cvs experience: This workshop. Component submission date: Aug 13, 2004.

Path to the Proposal

Summary and Issues The Road from Data to Science is Paved by Computing. The Road to DANSE is Paved by Money. The Road to Money is Paved by a Proposal. The Road to a Proposal is: - Short and fast, requiring high acceleration - Starts today. Let’s go!

End Presentation