1. Autoprocessing updates at the MX beamlines
Jun Aishima
Postdoc – Monash University Advanced Molecular Imaging Centre of Excellence and Australian Synchrotron

2. Feedback to users Processing software Analysis software
Introduction to MX
Feedback to users
Processing software
Analysis software

3. MX processing system outline
Expt. type
Indexing
screening
dataset
Availability
MX1 only
MX2 only
Both
Images collected
1-2 images
100 images
1800+ images
Info obtained
Unit cell parameters
Dataset quality
Dataset
Example output
I23
78, 78, 78, 90, 90, 90

4. Software used to process data at the MX beamlines
Custom pipeline software – Nathan Mudie -
Custom web application software – Nathan Mudie -
xdsme – my upgrades to add handling inverted rotation axis, improve integration speed, etc.- (BSD license following from original)
Pierre Legrand (Synchrotron Soleil) (BSD license)
XDS
Pointless, Aimless (CCP4)
Sadabs and Xprep for CX data

5. What additional features beyond just processing the data?
Merging datasets
Dataset plots
Retriggering (reprocessing)
Results spreadsheet
Whole goal is to help users evaluate their data, get the best out of it, and then keep track of the results

6. The Eiger 16M detector Rastering plots New feature s
Put in rastering tab pic – show diffraction image, move to

7. Rastering plots

8. Eiger is a pixel array detector
CCD
PAD

9. How is the Eiger different from previous detectors?
Data size:
Eiger test crystal dataset, 180 degrees, 0.1 degrees/image – 9.5 GB
ADSC test crystal dataset, 180 degrees, 1.0 degree/image – 800 MB
Collection speed:
Eiger test crystal dataset – 18 sec – continuous collection (3 microsecond readout)
ADSC test crystal dataset – 7.2 minutes – ~1.4 sec readout per 1 second image
Data rate: hundreds times faster!
HDF5 files – master and multiple data files in NXMX format (full metadata
necessary for processing)
Typically 200 images/data file and experiments have 1800 images (9 files)

10. What is the real effect of the Eiger on data produced on the beamlines?
1 TB typical on a single visit

11. XDS also had support for the Eiger
How much did we need to change the processing software to integrate the Eiger?
The good news
When we installed the Eiger 16M on MX2, xdsme already had been upgraded to support Eigers
XDS also had support for the Eiger
In terms of processing, not much code had to be rewritten
Testing infrastructure also helped to rapidly develop and test new code while the beamlines were both still running CCD detectors
=> We get for free all of the features we have developed for the past few years with the new detector!! (merging, retriggering, web application, results spreadsheet)
But the large amount of data -> slow!

12. Data Acquisition System
Current Eiger processing system outline – MX2
Data Acquisition System
Web app
autodataset
autodataset-cpu1,2,3
host:EPU
autodataset
host:EPU02
autodataset
host:EPU03
autodataset
host:ASCI05
autodataset
host:ASCI06
Some processing (integration in XDS) is split between multiple nodes

13. Data Acquisition System
In progress… Australian Synchrotron Computing Infrastructure (ASCI) processing cluster
Data Acquisition System
Web app
autodataset
autodataset
host: node 1
autodataset
host: node 2
autodataset
host: node 3
autodataset
host: node 4
autodataset
host: node 5
autodataset
host: node 6
autodataset
host: node 7
autodataset
host: node 8
autodataset
host: node 9
autodataset
host: node 10
autodataset
host: node 11
autodataset
host: node 12
autodataset
host: node 13
autodataset
host: node 14

14. What’s coming up in the next year?
Collection strategies
Compare with ADSC
Sample services
Collection Strategies
Sample tracking
- these mean better automatic processing, combining datasets that should be combined
- scientists think about the science, not each individual dataset

15. Main dataset processing Retriggering Merging Results spreadsheet
Summary
Eiger
Main dataset processing
Retriggering
Merging
Results spreadsheet
Rastering plots
Plots
Show diffraction image
Move to image location
In progress
ASCI processing cluster
Future
Collection strategies
Sample tracking
LCP injector

16. Thanks to…
The MX beamline – Tom Caradoc-Davies, David Aragao, Daniel Eriksson, Santosh Panjikar, Jason Price, Alan Riboldi-Tunnicliffe, Rachel Williamson
Scientific Computing group – Andreas Moll, John Marcou, Robbie Clarken, Nathan Mudie, Ron Bosworth
Australian Cancer Research Foundation for helping fund the Eiger 16M detector
Users – for being patient, requesting features, providing feedback to our systems
High Data Rate MX meeting – great advice and collaboration opportunities with other beamline scientists and Dectris to help deal with the Eiger detectors