1. Software Project Brent Fultz California Institute of Technology ARCS Project Intellectual Coherence Distributed Computing Tasks Underway Coordination between Caltech and SNS

2. ARCS Spectrometer Moderator Shutter Guide s Choppers Sample Detector Array Beamstop

3. ARCS Spectrometer Headstock Neck Frets Pickups Volume/Tone Solid Body Tailpiece (Whammy Optional)

4. ARCS Baseline Review Hardware - Detectors inside vacuum - Shielding Project Management - Project Execution Plan - Project Baseline Plan - Memorandum of Agreement (Caltech-SNS) - Budget Authority Software - Plan Dates: Review: March Committee Report: May DOE Instructions: September ARCS Response: November

5. Preamp Prototype

6. Hardware Issue #2: Shielding Design Effort on: Beamstop design Crosstalk between instruments Background from proton transport line and choppers

7. Project Schedule

8. Software Project Plan

9. Software Roadmap v. 1.0

10. Software Survey...... 10 responses, from serious users. very detailed comments.

11. User Ratings of Software Packages User Opinion (–2 to +2) Strength of Opinion (0 to +3) Number of Respondents Rating = UO*SO/N

12. Software Survey Results Interesting: CS: ISAW, Unisoft, Computational Crystallographic Toolkit User: ISAW, DAVE, Mslice, McStas, Gulp, Chop Problems: 1.Code Quality and User Opinion are not necessarily correlated (McStas is an exception) 2. We are still looking at details, but we can already see that there will be more rewriting than expected

13. Issues in Code Development X hours to develop a code for yourself 3 X hours to make it convenient for others 9 X hours to make it run for others on several platforms Adding personnel to a late software project makes the project later. (Brooks Law)

14. Architectural Coherence Left: Saint-Sernin, Toulouse, Romanesque Pilgrimage Church, c. 1080-1120 Right: Notre-Dame, Amiens, French Gothic Cathedral, begun 1220 Inconsistencies in Architecture Cathedrals --- Charming :-) Software --- Annoying :-(

15. Enforce Coherence -- Write the Manual Now (Brooks, Mythical Man Month) Define needs and scope Reference for developers Identify and show interrelationships Standardize notation Documentation

16. Experimental Inelastic Neutron Scattering Contents: Theory of neutron scattering Theory of excitations in condensed matter Data analysis procedures Software architecture Reference manual Book Project Plan: We write it as we learn it LATEX Presently ~130 pages, on www Intellectual follow-on to diffraction textbook Publish electronically?

17. library.caltech.edu Caltech Library System Caltech Collection of Open Digital Archives (CODA) Coda: Mus. A passage of more or less independent character introduced after the completion of the essential parts of a movement, so as to form a more definite and satisfactory conclusion. -- Oxford English Dictionary Caltech CODA uses the Open Archives Initiative - Protocol for Metadata Harvesting (OAI-PMH). Archives in production are registered as OAI Data Providers. As registered Data Providers, they may participate in federated searching interfaces and other end user services offered by a growing number of OAI Service Providers. Several OAI Service Providers are listed below. Caltech CODA runs on freely available electronic archiving software. The Caltech ETD collection runs on ETD-db software developed at Virginia Tech; all other collections run on EPrints software developed at the University of Southampton. In Production

18. Python Modern language with clear syntax Open source Object-oriented Interpreted Very high level data types “Easy” interface to C++ C++ Industry standard language with cryptic syntax Low-level control allows optimum machine performance Compiled and dangerous, but the only choice XML Syntax for organizing data (does no work itself) Java Some similarities to Python. The choice for browser applets. Java 1.4 integrates XML-RPC scripting.

19. Key Concept: Data Analysis as a Web Service Data analysis is a service controlled by a web browser Computation is arranged by the server user’s web browser issues commands and receives results

20. XML and the Network User’s Web Browser Web Server Database Server Beowulf Cluster XML-RPC is an open standard for remote procedure calls The user’s web browser issues commands to a server The server distributes the work to appropriate computers

21. The User Interface Users navigate the web site with the left frame (content is generated dynamically by the web server). Users select data and calculational tools, and add them to the Java applet on the right

22. The User Interface (“Labview”-like) The user “wires” the boxes together to represent data flows We intend to allow users to insert new code in empty boxes, and archive sessions and procedures

23. Data Analysis Execution User hits “Run” Applet translates wiring diagram into XML-RPC commands Server receives commands, arranges Python script, starts data processing.

24. Data Analysis as a Web Service The server can provide access to the best combination of hardware and software Experimental data and analysis codes reside on the servers, so little data bandwidth is needed Platform independent (a huge savings in effort!) Computing resources can be changed without affecting the user Clean separation of GUI from analysis code One web portal for all SNS instruments?

25. Software Project Decisions Made Python (for users and instrument scientists) C++ (core modules) Web service using remote procedure calls Immediate Tasks to Test Architecture Demonstration web portal (Java/XML - XML/Python) Rebinning software (Python/C++) Lattice dynamics (Python/C++) Contingencies minimized after we write some test modules A baseline plan is possible by January, 2003.

26. Software Roadmap v. 1.0

27. Born - von Kármán Lattice Dynamics Simplicity: Complexity: 1.Undergrowth of indices for tensor quantities: 2.Crystal structure: lattice, l, basis,  allowed elements depend on symmetry

28. J. M. Ziman, Electrons and Phonons

29. Data Rebinning Account for incident flux Remove background Convert from time to energy Correct for detector efficiency Bin into rings of constant scattering angle Convert from angle to momentum Subtract multiphonon and multiple scattering Correct for absorption

30. Data Rebinning – Tim’s Test Tested rebin procedure in several forms. 300 seconds with IDL 60 seconds with IDL using a DLL compiled from C++ 2 seconds with C++ “Rebinner” class

31. First-Order Approx. for SNS – Caltech Interaction Universities codify science National Labs generate science Detectors Dispersions SNS Caltech ?

32. Model Fails in Second Order Project Phase of SNS User interface requires tremendous care from SNS and Caltech Common standards and documentation Analysis software and hardware now at Caltech, but must migrate to SNS, ORNL resources Operations Phase of SNS Role of Caltech and broad community in future software maintenance or development?

33. Electronic Publishing server user documentation library instrument and data ? ? codes ?

34. Discussion on SNS - Caltech Interactions Coordination of software projects at Caltech and SNS? Scope and scope management. Expectations of users for software support by SNS and instrument scientists. Consistency of GUI for SNS instruments? Single web portal? Who maintains the code? Standards for maintainable code with "open source coalition"? Issues with distributed computing. Lab policies, security, graphics, user permissions. Issues with releasing software to run on users' machines Status of storage and archiving of raw data by SNS? Institutional arrangements with the ORNL supercomputing center? On-line control of a neutron spectrometer. Technical and policy issues.

35. SNS - Caltech Interactions Communication is the key. Thanks for coming! Please visit the ARCS wiki http://viz.cacr.caltech.edu:8000/arcs/1

38. Agenda 8:00 AM Breakfast or coffee 8:30 AM Introductions 8:40 AM Overview (Brent Fultz) 9:20 AM Directions (Michael Aivazis) 10:10 AM Break 10:30 AM Demo of web portal for data analysis (Jonny Lin, Tabitha Swan-Wood) 10:50 AM Physical concepts as software classes (Tim Kelley) 12:00 Lunch at the Athenaeum (hosted) Discussion who …what …why …when … how NeXus classes for analysis classes (Ray Osborn) 6 PM Heritage Wines? 7 PM Dinner at Cafe Santorini (no host)