1. The Science Data Processor and Regional Centre Overview Paul Alexander UK Science Director the SKA Organisation Leader the Science Data Processor Consortium

2. SKA: A Leading Big Data Challenge for 2020 decade Antennas Digital Signal Processing (DSP) High Performance Computing Facility (HPC) Transfer antennas to DSP 2020: 5,000 PBytes/day 2030: 100,000 PBytes/day Over 10’s to 1000’s kms To Process in HPC 2020: 50 PBytes/day 2030: 10,000 PBytes/day Over 10’s to 1000’s kms HPC Processing 2020: 300 PFlop 2028: 30 EFlop

3. XXXXXX SKY Image Detect & amplify Digitise & delay Correlate Process Calibrate, grid, FFT Integrate s B B. s 12 Astronomical signal (EM wave) Visibility: V(B)= E 1 E 2 * = I(s) exp( i  B.s/c ) Resolution determined by maximum baseline  max ~ / B max Field of View (FoV) determined by the size of each dish  dish ~ / D Standard interferometer

4. Image formation computationally intensive Images formed by an inverse algorithm similar to that used in MRI Typical image sizes up to: 30k x 30k x 64k voxels

5. Must also solve for calibration errors

6. Challenge Very Dependent on Experiment

7. One SDP Two Telescopes Ingest (GB/s) SKA1_Low500 SKA1_Mid1000 In total need to deploy eventually a system which is close to 0.5 EFlop of processing

8. … and regional centres

9. Scope of the SDP

10. The SDP System

11. Imaging Processing Model Correlator RFI excision and phase rotation

12. Performance Requirements Imaging and calibration determines system sizing Data is Buffered after Ingest Double buffered Buffer size >100 PByte Imaging must account for very large field of view o Algorithms complex and computationally expensive o Will need to evolve algorithms during life of the telescope Data products will be calibrated multi- dimensional images and time-series data Volume of potential data products very large May only be able archive data specific to observation requested Commensal fast-imaging mode for slow transients Processing to be achieved (Pflop) Ingest (GB/s) SKA1_LOW25500 SKA1_Mid901000 Detailed analysis is complex

13. Illustrative Computing Requirements ~100 PetaFLOPS total achieved ~200 PetaByte/s aggregate BW to fast working memory ~50 PetaByte Storage ~1 TeraByte/s sustained write to storage ~10 TeraByte/s sustained read from storage –~ 10000 FLOPS/byte read from storage 13

14. SDP High-Level Architecture

15. How do we get performance and manage data volume? Graph-based approach Hadoop

16. Data Driven Architecture

17. o Smaller FFT size at cost of data duplication

18. SDP Data Flow Approach: Next Generation Data Science Engine?

19. Hardware Platform

20. Wilkes Built November 2013, HPCS/SDP + Dell, NVIDIA and Mellanox UK’s fastest academic cluster 128 Dell T620 servers 256 NVIDIA K20 GPUs 256 Mellanox Connect InfiniBand cards + switch Performance of 240TFlop with a Top500 position of 166 Funded: STFC, SKA, and industrial sponsorship from Rolls Royce and Mitsubishi Heavy Industries. most energy efficient air cooled supercomputer in the world second in Green500 with 3631 MFLOP/w most energy efficient air cooled supercomputer in the world second in Green500 with 3631 MFLOP/w

21. 3-30 EBytes / year of fully processed data for SKA2 Data rates and processing increase by FACTOR ~100 for SKA2

22. END