1. ALMA Pipeline Heuristics - Interferometry - Frederic Boone, LERMA Lindsey Davis, NRAO Heiko Hafok, MPIfR John Lightfoot, UKATC Dirk Muders, MPIfR Christine Wilson, McMaster Friedrich Wyrowski, MPIfR Luis Zapata, MPIfR

2. The Mission After commissioning and early science, deliver pipeline to reduce data automatically Publishable quality? Trustworthy Easy to use / develop SFI, mosaic, single dish

3. Development Capture expertise Using Casapy tools, can use tasks Incremental

4. Overview Python program, recipe driven oSeries of stages oDisplay data ‘views’, detect and flag bad data, search for best calibration method oFlagging improves MS quality, calibration method produces best results possible for that dataset oLastly, produce data products; final calibrations, cleaned cubes Output in HTML

5. Stages - 1 Calculate and display a result view=BandpassCalibration display=ComplexSliceDisplay Calculate bandpass calibration Display as amplitude/phase v channel view=FluxCalibration(BandpassCalibration) display=ComplexSliceDisplay Calculate bandpass calibration Use it in calculation of flux calibrated gains Display as amplitude/phase v time

6. view=CleanImage(BandpassCalibration, FluxCalibration, sourceType=‘GAIN’, ) display=SkyDisplay Calculate bandpass calibration Use it in calculation of flux calibration Apply calibrations Produce a cleaned image Display as an image on the sky

7. Results data ‘views’ BandpassCalibration UnNormalisedBandpassCalibration GainCalibration FluxCalibration GroupSplineFluxCalibration CleanImage MosaicCleanImage

8. Stages - 2 Calculate ‘view’, flag, display view=GainCalibrationSNR(BandpassCalibration) flagger=ImageFlagger(rules=[{‘rule’:’min abs’, ‘limit’:2.0}] display=ImageDisplay Calculate bandpass calibration Use it in calculation of gains, export gain SNR  Apply flagging rules, flag the MS Display as a greyscale image with colour coded flags

9. MS & CalTable data ‘views’ AmplitudeDeviationPerBaseline PhaseDeviationPerBaseline BandpassCalibration FluxCalibration GainCalibrationSNR ClosureError

10. view=ClosureError (CleanImage, BandpassCalibration, FluxCalibration, outputDataAxes=‘ANTENNA’, iterateAxes=‘CORR’, dataCompressOperations=[‘abs’, ‘median’], sourceType=‘GAIN’) flagger=SequenceFlagger(rules=[…]) display=SliceDisplay Calculate clean image CORRECTED_DATA - MODEL_DATA [corr, antenna1, t, antenna2] -> [antenna] Assemble list from data belonging to each antenna then compress by taking abs, median  Look for abnormal antennas, flag the MS Display with flags

11. Displays SliceDisplay / ComplexSliceDisplay matplotlib ImageDisplaymatplotlib SkyDisplaymatplotlib CalPlotDisplayplotCal

12. Flaggers TAQLFlagger autocorrelation PdB Gibbs channels SequenceFlagger outlier in chunk high outlier low outlier max abs min abs too many flags ImageFlagger high outlier low outlier max abs min abs too many flags BandpassEdgeFlagger VLA edge template PdB edge template

13. The Future User Test 6 ? Get some real ALMA data! Polarization, transfer of calibrations between SpW Parallelization