KIDS requirements Dave Wilman for the KIDS team The KIDS operational plan and the role of AstroWise. List focussed on open questions: discussion required! By no means an exhaustive list…

1. Monitoring Tool Requirements: We would like a tool to: –Define the observations And to monitor and manage: –Progress of observations –Progress of reduction –Delivery to ESO

Current Implementation: The VISTA consortium is developing the Survey Area Definition Tool (SADT) which will interface with P2PP to automatically generate OBs and guide stars. Availability: early P2PP will be upgraded to allow timelink, concatonation and grouping of OBs so that survey strategy can be optimized. VISTA consortium developing a tool to monitor survey progress: The EST will look at this and see if a version of this tool can be made available to ESO and other survey teams. 1. Monitoring Tool

Action Items: Check the existence, capability and availability of the ESO monitoring tool.

2. Strategic Organisation and Distribution of Data Reduction Management Tool (item 1) Uniform plan for data reduction: –Consistent scripts / use of web-interface Distribution of data for actual reduction. Distribution of responsibilities for recipes and testing.

3. Field Layout and meshing with VIKING Observing Strategy and Dithering of KIDS: –see KIDS webpage Meshing with VISTA: –Probable solution: Matching KIDS tiles in Dec, but not in RA.

4. Dealing with Tiling and overlaps Split into 1 square degree chunks (imaging and catalogue)? Provide catalogue only once the full depth is available in all bands for that chunk? Deal separately with the overlap regions and the resulting cross-calibration? How will these divisions work technically? (i.e. can SWARP apply its own boundaries to a set of images, which due to dithering and tiling overlaps will cover a larger area than the chunk?)

5. The size of KIDS There has been discussion with ESO about extending the area of KIDS. Technically, what is the maximum data-rate which we could handle for reduction? –Manpower (reduction, testing, re-reduction) –Computer power (rate of processing)

6. Absolute Photometric Calibration Calibration of the absolute photometric zeropoint: North: Calibrate using SDSS sources: –SDSS accuracy not good enough for our relative photometry (~0.02mags) but OK for the absolute photometry? –Well understood calibration in worse seeing SDSS data –SDSS photometry already exists. South: Calibrate using our own photometric pass: –Easier to calibrate, with reduced systematics –No reliance on external source of data –Possible synergy with ATLAS –May help improve absolute astrometry –Alternatives (e.g. SSS) rely on data which does not yet exist, and with unknown systematics.

7. Quality Control Local / Short-term: –during data processing; –Simple checks of data products: plots and QC parameters (see next slide); Global / Long-term: –Photometric accuracy using multi-band catalogues; –Colour-colour diagrams (and photo-z); –Number counts; –Delegated responsibility by expertise? (e.g. Munich has expertise in fine-tuning photometry using a multi-band catalogue)

8. QC Parameters and Plots QC parameters to be computed from each node when processing data? Code required to make certain tests: build on existing inspect methods (plots) and QC parameters (values) Store QC parameters in database and process to management tool? Require a full list of such tests, frequency and responsible persons.

8. QC Parameters and Plots Example QC Plots and Parameters: –Astrometry: Local: plots of residuals (dx,dy,dra,ddec) vs (x,y,ra,dec) Global: cross-checks (and a long-term check might be to look for systematic and random offsets between bands) Values: rms, Npairs, coefficients and errors on linear fits of residuals?.... –Photometry: Images: –Background level variation in subwindows? –PSF(time, chip, position, colour, mag) –Manual(?) check for fringe-patterns, ghosts, satellite tracks, cosmic rays etc Catalogues: –STD: plots: raw ZP (time, chip, position, colour, mag) –STD: values: raw ZP rms, Nobj etc –SCI: in overlap regions: ZP residuals (time, chip, position, colour, mag) etc etc…..

QC Params: KIDS page 1 TestwhenBy whom PSF: plots: width, PA, ellip as fn of(position, mag, time, chip, colour) Vector plot New Reduced, Regridded, Coadded frames - Becomes less frequent when more stable Creator PSF: pars: width, ellip, PA: mean/median and dispersion, (radial gradients?); Later: quantify comparison with the expectation New Reduced, Regridded, Coadded frames Creator Astrometry: Local: Plots: residuals (dx,dy,dra,ddec) vs (x,y,ra,dec) Vector plot Vs mag New Astrometric Solution - At beginning, and when solution is poor Creator

QC Params: KIDS page 2 TestwhenBy whom Astrometry: Local: pars: Rms, Npairs, rms(dra, ddec, dX, dY), NrejectedPairs, Fit params, subwindow stats (where rejected stars are clustered) New Astrometric Solution Creator Gastrometry: plots using all overlaps – residuals vs position, chip; stars used for solution New GAstrometric Solution - When fit is poor Creator Gastrometry pars: rms, Npairs, rms(dra, ddec), NrejectedPairs, Fit params, subwindow stats (where rejected stars are clustered) Contrast with expectation (trend analysis on solution) New GAstrometric Solution Creator

QC Params: KIDS page 3 TestwhenBy whom Astrometric Solution in different filters: compare positions in multi- band catalogue Multi-band catalogue: NOT aperture matched Local Overlaps between adjacent fields (not dithers): similar to the Gastrometric tests Single-band catalogues PMs / Astrometry team Photometric: plots – STD : raw ZP (chip, position, time, filter, colour, mag, airmass, PSF width) New Photometric Parameters - When required Creator / Local (time/filter dependance) Photometric Pars: - STD : ZP, rms(ZP), Npairs, NRejectedPairs, fit params for above plots New Photometric Parameters Creator / Local (time/filter dependance)

QC Params: KIDS page 4 TestwhenBy whom Photometry of Science Observations – overlaps: ZP consistency (dithers, pointing overlaps): dmag between different exposures – 1 number per overlap computed using high S/N stars Before coaddition (on regidded frames) using SourceLists from regridded frames Local Stellar Colour-Colour plots & photo-z (via spec-z) - implementation? Multi-band catalogues Taskforce (Munich?) Number Counts – includes depth check (also in overlap regions with greater depth) - implementation? Single-band catalogues Taskforce (Paris?) Cross-Survey Comparisons (SDSS, CFHT-LS, SSS…) Multi-band catalogues with matched magnitude types Designated

QC Params: KIDS page 5 TestwhenBy whom Visual Inspection of background: Fringes, flat-fielding, bad pixels, artifacts, actual sky background New ReducedScienceF rames Creator

9. Photometric Accuracy Require accuracy of 0.01mag required for photo-z. (Potential) Astrowise tools to help reach this accuracy: Galctic Extinction Map Ingestion Sextractor Double-Image mode (measure the flux at the same position in all bands, even where in some bands we only get upper limits) Shapelets: The GAAP (PSF-matching) method (Kuijken et al): can we reach the required accuracy with this method? Convolution (brute-force): Johannes has plans to implement within AstroWise, and to test the computational requirements.

10. Ingestion of NIR data How will NIR data be ingested into AstroWise? At what stage of reduction will it be ingested? Pre- regridding? What kind of Weight-Maps will/can come with the data? Is Ingestion of reduced images already tested? UKIDSS / VISTA / 2MASS??

11. Data Storage Deletion of files !!! Is it necessary to consider deletion of: Redundant files? Intermediate files? Our tutorial in Munich, plus Johannes’ WFI data is already filling our diskspace……. Please consider how we can safely delete data within a Project context!!!

12. Visual Inspection of Data PanoraPix: Will it be ready in time for KIDS? What is the status of the cut-out server? It would be nice to view each ‘chunk’ of KIDS data separately: if PanoraPix is not ready, will this be possible?

13. Tracking of artifacts We will want to know automatically which sources in the catalogue are real. How do we treat: Ghosts? Bright-star artifacts?

14. Node Project Managers Who are they? Munich: David Wilman, Hans Vaith, Groningen: Edwin A. Valentijn, Paris: Herve Aussel, Leiden: Merijn Smit, Bonn: Oliver Cordes, Napoli: Mario Radovich Each manager should be able to manage/organise the people at their node. A good start would be to get started with test cases ASAP!

15. KIDS Daily Operations Contact between nodes / people. Frequency / method: A forum / mailing list? Regular telecons? On a longer timescale, regular meetings?

16. Timescale for ESO delivery What we told ESO in the Management Plan RIx reply: “A good multi-band catalogue is essential to qualify the photometric (and astrometric) calibration of our data products, and to our science (in particular to get good photo-z's). We will reevaluate seeing requirements for u, g & i (but not r) bands each semester, with the aim to keep the bands in step. However, particularly at the beginning of the survey, observing practicalities will mean that there is no way that we have identical coverage in all bands. Photometric calibration will require the photometric pass which will precede the main survey pass and establish zero-points across the KIDS area. Also we require re-observation with OmegaCAM of our secondary standard star fields during the first year, as outlined in the OmegaCAM Calibration Plan. Multi-band photometry is also required for vital fine-tuning of photometric accuracy (colour-colour and photo-z checks). For all these reasons, we request a full year (after delivery of the final band for a given tile) for initial catalogue delivery. Once the photometric calibration is better established, we are happy to catch-up to a 6 month timescale. The frequency of delivery can be as ESO prefers. This can be almost as continuous as desired.”

17. WFI test program There is a need for a ‘federated’ test program which ‘simulates’ the reduction of KIDS data (i.e. managing the distribution of reduction and testing amongst ourselves). Pre-requirements: Federation of AstroWise - required Nodes ready with the AstroWise system installed working on the clusters – required Stable and relatively bug-free installation of AstroWise – required Consistent plan for reduction on an appropriate dataset – required All software elements in place (e.g. aperture-matched photometry etc) - preferable

18. VLT followup Is there any action required regarding the ESO VLT followup of KIDS targets?