1. Experimental Facilities DivisionOak Ridge August 15, 2006 SNS/HFIR Software Developments Steve Miller Analysis Software August 15, 2006

2. Experimental Facilities DivisionOak Ridge August 15, 2006 2 Organizational Structure – October 1, 2006

3. Experimental Facilities DivisionOak Ridge August 15, 2006 3 SNS Recent Milestones Achieved SNS Facility CD-4: First beam on Target – April 28 Backscattering Spectrometer first data – May 19 Reflectometers 4A and 4B first data – July 19

4. Experimental Facilities DivisionOak Ridge August 15, 2006 4 SNS Facility CD-4 Measurements BL-7 Data Accelerator/Target Controls

5. Experimental Facilities DivisionOak Ridge August 15, 2006 5 BSS Data Reduction – Diffraction Bank (Mica) 1. TOF to Wavelength 2. Wavelength to d-spacing Instrument Geometry Step 1: pathlength Step 2: angle Courtesy of Michael Reuter

6. Experimental Facilities DivisionOak Ridge August 15, 2006 6 BSS Inelastic Data Courtesy of Ken Herwig and Eugene Mamontov 4-methyl pyridine N-oxide 5 kWatt, 3 hour, ¼ current detector/analyzer, T = 3 K Tunneling Peaks Visible! Fit using DAVE Software

7. Experimental Facilities DivisionOak Ridge August 15, 2006 7 Reflectometer 4B Incident Beam Wavelength Spectrum Courtesy of John Ankner

8. Experimental Facilities DivisionOak Ridge August 15, 2006 8 Sample: Ni 50 Å / Ti 50 Å multilayer SNS source power: 250 W Data collection time: 2.5 h Preparation time needed: 8 years TOF spectrum: integrated over whole detector Position spectrum: integrated over all TOF channels Same sample measured at IPNS: Bragg Peak Neutrons reflected from sample at 1 deg. grazing incidence First measurement on BL4A July 21, 2006 Courtesy of Frank Klose

9. Experimental Facilities DivisionOak Ridge August 15, 2006 9 Magnetic Reflectometer Reflectivity 50 A Ni / 50 A Ti * 20 Multilayer 2 kw – low power

10. Experimental Facilities DivisionOak Ridge August 15, 2006 10 Instrument Commissioning Challenges Low accelerator repetition rates: –Design point 60Hz, current rates between 1 to 10 Hz –Results in long time intervals between pulses on target –Keeping all neutrons with fast (~25 uSec) sample rate gives a large number of TOF channels – 5000 to 10000 typical –Reflectometry histogram files on the order of ~1.5GB each –Currently only able to perform linear histogramming Low beam power –Measurements contend with background –Must acquire for longer periods of time –Must contend with occasional accelerator glitches

11. Experimental Facilities DivisionOak Ridge August 15, 2006 11 Creating, Processing, and Storing Data Event Histogramming Detector to Pixel mapping Instrument Geometry Metadata extraction Create NeXus file  Catalog and Store  Reduce Data All subsystems functional to some degree

12. Experimental Facilities DivisionOak Ridge August 15, 2006 12 Data Reduction Architecture Overview DOM (Data Object Model) SCL (SNS Common Lib.) Three levels of reduction at SNS Level 1: Driver is the overall mechanism that runs the data reduction process. It based on the requirements that are given by the instrument scientists. Level 2: HLR is the representation of functions. It unifies calls to retrieve data and to call low level functions. Levels 1 and 2 python. Level 3: (mostly C++) –DOM provides abstract layer for data manipulation. –SCL is a toolbox of reusable primitive functions necessary for data reduction process. Driver HLR (High Level Reduction)

13. Experimental Facilities DivisionOak Ridge August 15, 2006 13 Visualizing Data via the Portal ISAW Plot metadata NeXus Files NeXus tags MCA Data

14. Experimental Facilities DivisionOak Ridge August 15, 2006 14 Metadata Search via ICAT Optional Search Fields Search String

15. Experimental Facilities DivisionOak Ridge August 15, 2006 15 Networking Diagram

16. Experimental Facilities DivisionOak Ridge August 15, 2006 16 Instrument Commissioning Lessons Learned Detector and Instrument Definitions… –Non-trivial to determine –May need an iterative approach to derive calibration factors Using NeXus… –Adds overhead for examining raw data –Question about how best to associate updated calibration data with existing NeXus files. Note that data reduction can take an optional calibration file along with NeXus data. –Create NeXus files during acquisition or “on-demand”? Event Data –Had to adapt to provide histogramming tools sooner than originally anticipated Detector Mapping –Needed to create and validate detector to pixel mapping files

17. Experimental Facilities DivisionOak Ridge August 15, 2006 17 Problems and Issues Encountered Python memory management issue: –Deleted ints and floats still remain in heap memory, thus memory footprint can grow very large. 6GB not uncommon for us and “larger data” to come. –Causing us to rethink how best to utilize python for memory intensive applications –Memory de-allocation fix in python 2.5 does not address this problem –Does DANSE team have experience with this? Need for tools to explore “pre-NeXus” data –Responded by creating prototype IDL tools –More tools needed, such as for producing rocking curves Need more tools for working with event data

18. Experimental Facilities DivisionOak Ridge August 15, 2006 18 Pre-NeXus Data Examination Tools Instrument Scientists need tools to explore data. Currently producing prototype IDL tools to enable exploring and extracting data. Considering producing tools derived from this which will integrate with the portal. Using IDL VM to distribute applications Courtesy of Jean Bilheux

19. Experimental Facilities DivisionOak Ridge August 15, 2006 19 Instrument Commissioning Pattern Emerging Analysis Software group provides: – the data “plumbing” –Data reduction –Visualization tools –Utilizing existing analysis software tools (DAVE, ReflPak, ISAW, GSAS, etc.) The DANSE team can dramatically help SNS/HFIR: –Identify prototype instrument commissioning software tools –Identify and produce Instrument Day-1 analysis software –Keep an eye to the future developing portal-based advanced analysis software –Urge DANSE Science team leaders to continue to work closely with ORNL instrument teams and the Analysis SW group!

20. Experimental Facilities DivisionOak Ridge August 15, 2006 20 Instrument Schedule

21. Experimental Facilities DivisionOak Ridge August 15, 2006 21 Instrument Commissioning Schedule by year: Ongoing: –Backscattering Spectrometer –Liquids Reflectometer –Magnetic Reflectometer 2007 –ARCS –Powder Diffraction –CNCS 2008 –SANS –SNAP –Sequoia 2009 –Vulcan Engineering Diffraction –Spin Echo –TOPAZ SCD Take Home Message: We should anticipate the desire to bring instruments on-line sooner in order to broaden the scope of the user program. Urge DANSE team members to continue to work closely with corresponding SNS and HFIR personnel (and vice versa). With higher neutron flux in later years, the push will be for “first science”, not just “first data”. Acting now can enable world class software to be available with first neutrons on instruments – our desire and a worthy goal.