Development Status and Priorities Jeff Kern NRAO

CASA Interferometry Progress Re-architecting of Clean task implementation to tClean – MT-MFS with mosaics, AW-Projections, parallelization – Available for use, many “nice” features still to be added – Used in ALMA imaging pipeline OTF calibration (cal library) – Supported and well tested in plotMS – Support in tasks implemented, needs to be commissioned – Not yet hooked up to tClean MPI Based parallelization – Fully available and documented for calibration (see talk tomorrow). – Easier integration with most scheduling systems 2015 CASA Users Committee Face to Face2

CASA Interferometry Progress (2) MS Transform – Single location for many similar tasks (deprecating old versions) – Fully MMS aware (parallel where available) Robust statistics algorithms in imstat and ia.statistics – Several algorithms Channel based weights, weight handling defined, and tools for weight manipulation. Linear mosaic (lm) tool to mosaic weighted images – Important for VLASS msuvbin – UV plane accumulation – Designed for CHILES project, allows managing very large data sets 2015 CASA Users Committee Face to Face3

CASA Single Dish Progress MS transition (since 2014 Q4) – All highest priority capabilities, i.e., ones used in standard manual and pipeline data processing of ALMA-TP, have already been implemented. (But the standard data processing of ALMA-TP is still based on ASAP) – 21 out of 58 items are implemented in CASA 4.5 release Support ALMA-TP data processing: manually processed data started to be delivered in Aug Bug fixes and minor enhancements in ASAP to process Cycle 1 and 2 observation data Support of New multi-beam receiver, FOREST, on Nobeyama 45-m – Bug fixes in ASAP to support multi-beam data and enhancements for FOREST – Currently, a dataset should be split by beam for proper imaging CASA Users Committee Face to Face4

CASA Pipeline Progress Successfully supported ALMA Cycle 2 operations. ALMA SD Pipeline acceptance ALMA Imaging Pipeline – Continuum Image – Per SPW continuum subtracted cubes VLA Calibration Pipeline in operations Parallelization – Implemented in pipeline, not yet deployed – See talk tomorrow ALMA-QA0 Flagging Implemented – Additional online flags applied for ALMA 2015 CASA Users Committee Face to Face5 Cycle 2 Pipeline Processing Success w/ Additional Flagging Tweaked SuccessManual

CASA Build and Test Progress Support for OSX 10.9 and Daily automated test versions with automated tests on all four platforms – Stables (those that pass tests) are made available to the community. 89 stable versions available during 4.5 development cycle Integration with github::CASACore project – Unified casacore with rest of community Normalization and standardization of build process across OSX and Linux libsakura integration – High performance vector processor library developed at NAOJ C++11 Support in compilers – Allows removal of custom solutions: container classes, smartpointers MPI Integration 2015 CASA Users Committee Face to Face6

CASA Misses (Things we didn’t do). Updates to FFTW (Done, but to late for 4.5 release) Complete switch of calibration to new IO infrastructure layer – Full scratchless operation in calibration (done, but needs validation) Rewrite of visstat – Scope increase (support Single Dish) and new developer. 4.6 tClean: – Transition to new IO infrastructure (scratchless operation) – Improved image weighting (fixes interpolation issues) – Interactive interface improvements (descoped early) – automasking – Parallelization (Done, but needs validation) 2015 CASA Users Committee Face to Face7

CASA Misses (Things we didn’t do). SetJy – Full reuse of imaging infrasturucture StatWt improvements Enhanced RFI mitigation in RFLAG (requirement mismatch) 2015 CASA Users Committee Face to Face8

CASA Interferometric Priorities Reliability and Stability Initiatives – Testing Enabling and requiring tests at the C++ module level – Technical Debt Continued migration to standard IO routines Namespace cleanup, removal of bespoke code – Obsolete Code Removal of unused or unsupported options Consolidation of routines to use a single pattern for accomplishing the same thing CASA Users Committee Face to Face9

CASA Interferometric Priorities Close out of ongoing initiatives – tClean – Scratchless operation in Calibration, testing and perfecting Some “behind the scenes” changes for maintainability – VisStat Speed, parallelization, and support for large data Robust estimation – Scratchless operation extended to tClean No more applycal? New initiatives: – Performance effort: Particularly looking at low level IO access patterns 2015 CASA Users Committee Face to Face10

CASA Pipeline Priorities ALMA: – Implement antenna outlier amplitude vs time flagging heuristic – ALMA imaging acceptance (and improvement) – HPC in production – Sessions (?) Multiple contiguous executions of the same SB Sharing calibration across exectuions VLA – VLASS specializations (OTF Mosaic support) – Spectral calibration support Both – Low signal to noise ratio heuristics – Polarization calibration – Science Ready Data Products 2015 CASA Users Committee Face to Face11

CASA Single Dish Priorities Complete the MS transition project Aim finishing migration of most of the items by the end of 2016 Support ALMA to start using the new single dish module based on MSes The new single dish module is ready for use for ALMA-TP. Need to update manual reduction scripts and pipeline Support ALMA Cycle 4 TP observations Major new capabilities are fast-scanning observations of the Sun and observations of multiple science targets by an SB Support Nobeyama start using CASA as the primary data reduction software Nobeyama decided to adopt CASA for new backends Preliminary schedule: 2016 Q4 - the observatory test use of CASA, 2017 Q4 - observatory start supporting reduction by CASA 2015 CASA Users Committee Face to Face12

CASA Build and Test Priorities Add support for more Oss – OSX (10.9 going away!) – RHEL 7 – Ubuntu support (MeerKAT?) Resources permitting Transition to Git as version control system for all of CASA – Investigating more integrated tool chain Automation of new tests under development by development team – Introduce and track test coverage metrics Addition of tools to enforce Software Quality initiatives Implementation of framework to support user documentation effort 2015 CASA Users Committee Face to Face13

CASA 2014 CUC Report Recommendations Since speed and ease-of-use are the main laments of CASA users, we recommend to emphasize improvements in performance and parallelization, and to improve the overall robustness of basic operation. The CASA HPC framework is now in place and available for general use. Parallel imaging is implemented and tested in pipeline context but not yet undergone full validation. Scratchless operation is available in calibration, plotMS, and MSTransform, but not yet in imaging. Additional attention to setting user expectations may also help ameliorate these issues, such as (crude) time estimators for critical tasks. This has not yet been addressed, it is on the list of remaining imaging re-architecting tasks CASA Users Committee Face to Face14

CASA 2014 CUC Report Recommendations Complete the CLEAN refactor with high priority, as this will bring important capabilities as well as retire some speed and usability issues. The first version of this is now available in CASA 4.5 Improve and curate better CASA documentation; a small number of well- maintained documents is preferable to many that are not maintained and become out of date. See discussion in talk on documentation Provide updates to CASA users about ongoing and planned developments, as well as prioritized fixes, in a regular and easily digestible form. The first CASA newsletter was published in April, the second is in prepartion and will be out this week or next. Improve feedback to users on helpdesk/ticket disposition, especially where challenging problems may take a long time to address, or will not be addressed. User surveys can uncover problematic issues and help to guide future development; we would like to work with the CASA team to develop and deploy more user surveys. As there always will be a longer wish list of CASA improvements than available re- sources can support, we recommend to engage the external community by stabilizing and documenting interfaces and import/export formats to better enable users to con- tribute and to maintain CASA tasks and scripts. Continue to promote CASA for telescopes other than ALMA and the VLA on a best effort basis; major initiatives formulated by external groups should be undertaken only if the additional resources required are provided CASA Users Committee Face to Face15

CASA 2014 CUC Report Recommendations Improve feedback to users on helpdesk/ticket disposition, especially where challenging problems may take a long time to address, or will not be addressed. We have created a web page to address this, some of the CASA management consultant recommendations are focused on improving determinism for development. User surveys can uncover problematic issues and help to guide future development; we would like to work with the CASA team to develop and deploy more user surveys. First user survey was conducted and a response has been prepared (see upcoming newsletter). Further details in discussion on user survey CASA Users Committee Face to Face16

CASA 2014 CUC Report Recommendations As there always will be a longer wish list of CASA improvements than available resources can support, we recommend to engage the external community by stabilizing and documenting interfaces and import/export formats to better enable users to contribute and to maintain CASA tasks and scripts. We have addressed a number of known issues with the uvfits import/export tasks. Documentation was discussed earlier. The transition to an enterprise Git implementation will allow more dynamic interaction with external developers. Continue to promote CASA for telescopes other than ALMA and the VLA on a best effort basis; major initiatives formulated by external groups should be undertaken only if the additional resources required are provided. Under these auspices we have entered into a collaboration to support the ACA reduction with CASA. We are working to incorporate resources from other observatories in the NRAO helpdesk to assist in the support of CASA CASA Users Committee Face to Face17