TDP activities in calibration and processing Athol Kemball for TDP Calibration & Processing Group (CPG) Department of Astronomy, University of Illinois at Urbana-Champaign

Current CPG membership Athol Kemball (Illinois) (Chair) Sanjay Bhatnagar (NRAO) Geoff Bower (UCB) Shami Chatterjee (Cornell) Jim Cordes (Cornell; TDP PI) Shep Doeleman (Haystack/MIT) Joe Lazio (NRL) Colin Lonsdale (Haystack/MIT) Lynn Matthews (Haystack/MIT) Steve Myers (NRAO) Jeroen Stil (Calgary) Greg Taylor (UNM) David Whysong (UCB) Mark Yashar (UIUC) Calgary Cornell NRL UIUC MIT NRAO UCB UN M

Primary CPG goal Contribute to a construction- ready Phase 1 SKA design and proposal. Provide a quantitative cost and feasibility model for calibration and processing elements of the US TDP LNSD SKA design.

Current CPG activities Continued progress on Project Execution Plan (PEP) tasks. Key recent and continuing focus is coordination with needs of DVA-1 project within TDP: –Processing Verification Program (PVP). Liaison with PrepSKA project and pathfinders. Graduate students research involvement and engagement. Outreach at meetings and in community. (L. Matthews)

Feasibility: tracing science requirements SKA science requires unprecedented sensitivity and dynamic range. Need too meet thermal noise limit: (Lazio et al. (2009); Chakraborty & Kemball (2009)) We are considering these requirements on calibration and processing per science use case.

Processing verification strategy Trace calibration and processing requirements to the SKA science cases. Assess practical community experience in high-dynamic range imaging with current interferometers. Parallel/simultaneous strategies: –Analytic or semi-analytic approaches. –Numerical simulations. –Pathfinder tests. (3C286; ATA Whysong & Bower)

PVP: projecting DVA-1 performance Antenna and feed design parameters: –Mount type –Reflector: Size, shape, and manufacturing method –Optics –Feed and illumination –Polarization purity –Net bandwidth –Etc. Overall system performance verification: –Cost per unit achieved sensitivity as a function of: Angular distance from center of main lobe: ρ Polarization: {I,Q,U,V} Frequency: ω –Feasibility: Limiting sensitivity in {I,Q,U,V} (ρ,ω) due to uncorrected systematic errors.

Feasibility: practical community experience in high-DR imaging Community survey of current DR limitations (EVLA, WSRT, GMRT, ATCA) (Chakraborty & Kemball 2009): –Hardware is destiny Sky-mount vs rotating antennas Closure errors –Direction-dependent errors: Pointing Beam pattern errors –RFI –Computational and software limits Longitudinal study on 3C273, (Chakraborty & Kemball, 2010). –DR is tracking thermal sensitivity evolution, but at cost of calibration complexity. (Chakraborty & Kemball 2010) Mean DR evolution over time

Numerical simulations Current focus of numerical simulations: –Calibrating semi-analytic dynamic-range scaling laws derived earlier. –Placing constraints on radiation patterns (e.g. sidelobe levels and stability, pointing) as part of DVA-1 support.

1.4 GHz leakages for antennas (numbered) in real,imaginary space as a function of frequency. Typical values up to 10%. Smooth changes with frequency. Rate of ~3%/10 MHz at 1.4 GHz. Spiral and loop patterns seen in real-imag space. Period of loops similar to that of log-periodic feed (e.g., 3/4 of a turn at 1.4 GHz). Leakages are contiguous between adjacent bands. This suggests leakages originate in the feed. Leakages for two adjacent correlator tunings. The leakages change smoothly between bands. Feasibility: wide-band polarization calibration Pathfinder ATA studies (Bower et al)

Leakages are measured in offset pointings from calibrator. Similar leakages throughout the primary beam. Random contribution increases off-axis and toward higher frequencies. Random component of order 3% within half the HP point at 1.4 GHz. Leakages for ant 11x at center and half power points. Color shows frequency dependence. Axes show leakage scale. Same leakages, but showing diff with center. Feasibility: direction-dependent polarization calibration Pathfinder ATA studies (Bower et al)

Feasibility: primary beam radiation pattern stability (ATA: Harp et al. 2010)

Correlator FOV Shaping: Smart (t, ν) averaging Multiply the sky by a weighting (window) function W convolve the u-v plane by Fourier transform of the window function, effectively tailoring the FOV Can be used to mitigate sidelobes (shape FOV) in high-resolution imaging W Jinc/top hat function Applying single weighting function in (u, v) plane will impose same FOV on all baselines Do this for each visibility, during correlation, pre-calibration

Using both real (MERLIN) and simulated (MAPS) data, the MIT Haystack group has been exploring: - Attenuation of distant sources due to realistic primary beams (from Cortes) and array synthesis beams (based on Bolton & Millenaar) - Expected noise contributions from distant sources before and after convolution - How well various choices of the correlator FOV function reduce the noise contributions from sources outside the field of interest; impact on achievable dynamic range - Requirements for implementation in future correlators

No convolutionGaussianTop Hat 3C343 3C343.1 from Lonsdale et al., in prep.

Costing: emerging trends in exaflop power consumption Exaflop architectures will have extreme parallelism and heterogeneous processing elements. –Energy costs will be a key architectural driver. Future energy costs lie predominantly with: –Memory –Data transfer/movement costs within system –Both costs will likely dominate processor power. –Many technical innovations under consideration Proprietary information and variable technological risk This introduces uncertainty into energy extrapolation. Yashar & Kemball (2009)

Current power projections Considering system power only: –Exascale general-purpose system built out using today’s technology: 150 MW++ –Exascale general-purpose system assuming timely but evolutionary energy efficiency advances: MW+ Very aggressive technological evolution assumed: 30MW+ –Exascale system with limited computational modes (i.e. tuned for modest number of SKA operation types): Low tens of MW (say 30 MW). A hardware-software co-design (e.g. a vendor partnership) – or an “off-the- shelf” configurable supercomputer. –A single-mode exaflop system: < 20 MW, perhaps << 20 MW. (Gara 2008) (Donofrio et al. 2009)

Power utilization effectiveness Cooling (chilling) and circulation sub-systems (+20-35%, but extensive room for improvement). –Optimized flow of coolant or air (at the board and chip level). –Extensive environmental monitoring of system and data center. –Passive environmental cooling. –Minimizing number of voltage down-conversions from HT to chip board voltage. E.g. variable voltage computer boards. –Processor and board power optimization (slowdown) modes. –A strong industry driver: You can’t sell an exaflop system that needs 150 MW of power! Regulatory interventions loom for large data centers. Example: National Petascale Compute Facility: –LEED Gold certification for green buildings –Power utilization effectiveness (PUE) ~ 1.1 to 1.2 –Opening June 2010 (Cray Ecophlex system document)

Liaison with PrepSKA and international SKA project Participation in key PrepSKA and SKA meetings: –PrepSKA WP2 PY#2 meeting (Manchester UK, Nov 09). –SKA –PrepSKA WP 2.6 group participation. TDP is lead institution on: –Calibration and Imaging Techniques for WBSPF (PrepSKA WBS ) –Exascale Computing & Hardware (PrepSKA WBS 2.6.7) TDP and PrepSKA have close synergies in goals during design and development phase Close engagement with PrepSKA in refining and coordinating cal. & processing work in WP2; TDP funding facilitates US engagement.