1. Gaia Data Processing Coryn A.L. Bailer-Jones Max-Planck-Institut für Astronomie, Heidelberg on behalf of the Data Processing and Analysis Consortium (DPAC) ● objectives and principles of the data processing ● main processing steps ● plans for ground-based support observations ● the Data Processing and Analysis Consortium

2. Objectives of the data processing ● Provide a catalogue of calibrated astrometric, photometric and spectroscopic data – includes radial velocities, astrophysical parameters and variability information ● Derive accurate astrometric solutions for all sources – absolute scale for parallaxes – single, extragalactic reference frame for proper motions – reference frame to coincide with ICRS

3. From transit to astrometry focal plane pixel coordinates (t, y) field angles ( ,  ) absolute astrometry (a, d) Source parameters nominal astrometry transform to proper directions Attitude parameters optical projection CCD layout Calibration parameters correct for aberration, light bending etc. Global parameters

4. From astrometry to parallax and PM ● in practice measure relative 1D positions along quasi-great circles – 7000 great circle scans of each field of view over 5 years ● scans at any sky point obtained at a range of position angles ● fixed “Basic Angle” between the two fields of view – build up spherical mesh linking all sources (“closure condition”)

5. From astrometry to parallax and PM ● cross-match different scans of same sources ● map the 1D along scan positions to 2D (astrometric) positions ● typically 80 independent measurements per source ● from positions as a function of time (over 5 years) derive parallax and proper motions

6. Initial Data Treatment ● housekeeping data provides on-board attitude ● ingest data into DB ● determine preliminary source image parameters – fit line-spread function (LSF) → position (“centroiding”), flux – flagging moving sources (e.g. a solar system object) ● estimate source field angles – using initial/current calibration and attitude parameters ● cross-matching of observation – attach observation to existing source in DB; create new source if none ● provides starting point for the AGIS

7. Model for the astrometric solution Write symbolically as O = G + S + A + C + n transit time, across-scan position e.g. PPN  six basic astrometric parameters last is fixed smoothed with cubic splines optics, distortion, chromaticity, charge transfer, etc.

8. Astrometric Global Iterative Solution S G A C AGIS iteration improved image parameters, new source selection, new data outer iteration inner iterations

9. Astrometric Global Iterative Solution ● linearize equations: O = G + S + A + C + n ● solve using least squares – block-iterative methods – solve parts in parallel on multiple CPUs ● solve for A, G and C parameters using the primary sources – Gaia is self-calibrating ● these parameters then used in solution of all other sources, including variable stars, binary systems ● map internal reference frame to ICRS using quasars

10. Main processing steps Initial Data Treatment First Look and Science Alerts raw data main database Astrometric Core Processing (AGIS) final catalogue simulations Special Object Processing Photometric Processing Spectroscopic Processing Variability Analysis Astrophysical Parameters

11. Spectroscopic processing: RP/BP ● extraction ● combination ● wavelength and flux calibration ● used for: – chromaticity correction, astrophysical parameters ● complicating issues – overlapping spectra, variable PSF, variable dispersion, CTI effects

12. Spectroscopic processing: RVS ● primarily for radial velocity, also – rotational velocity – emission lines – individual chemical abundances ● tasks/issues as for RP/BP, but also – Spectroscopic GIS – wavelength calibration more critical

13. Special objects ● Poor fits to 5-parameter astrometric solution (flagged by AGIS) ● Nearby rapidly moving stars – perspective acceleration (time-variable parallax) – need radial velocity to accurately solve for astrometry ● Multiple systems – binary stars, exoplanets – solve for Keplerian orbits ● Solar system objects – most detected as rapidly moving on-board

14. Classification and Variability ● discrete classification – QSO selection for AGIS; identification of new types of objects ● stellar astrophysical parameters (APs) – T eff, log g, [Fe/H], A V, [  /Fe] – include parallax to derive luminosity; evolutionary model to derive age ● variability detection and characterization – eclipsing binaries, pulsating stars, spotted stars, planetary transits,... – timescales, periods, amplitudes ● non-trivial – very wide parameter space; issues of parameter degeneracy etc. – optimal combination of photometry, spectroscopy and astrometry

15. Auxiliary (ground-based) data ● satellite tracking (ephemeris) – need to reach 1mm/s precision (G=17-18, PM of 1 deg/day) – nightly observations with 20mas precision (always at opposition) ● flux calibration of RP/BP spectra – spectrophotometry (G=14-20) ● calibration of AP estimation algorithms – echelle spectroscopy of ~1000 stars over wide AP range to determine accurate, internally-consistent APs ● radial velocity calibrators – RVs of ~1000 bright stars with RV variations below 300 m/s ● Significant ground-based programme with large telescopes required

16. Data Processing and Analysis Consortium ● responsible for all data processing and catalogue publication – from telemetry stream to final catalogue – includes analysis tools and value-added products – simulations, science alerts, early data releases, ground-based data ● formally started 15 June 2006. Operates until 2019 – all partners identified (following “Letters of Intent” exercise) ● organized into 9 “Coordination Units” ● geographically distributed, nationally-funded partners – currently ~250 members in 15 countries ● follows five years of studies in Working Groups

17. DPAC structure location of data centre(s)

18. DPAC mode of operation ● development and operation phases in 6 month cycles – pre-launch: successive improvement of software prototypes and simulations – during mission: new versions of main database (MDB) ● flexibility paramount – modify software to reflect changes in mission/instrument and incorporate lessons learned from real data ● DPAC-wide standards/sharing – data formats, programming language (Java), software libraries, development tools, quality assurance (documents, testing, validation, etc.) – significant effort placed into simulations for development (dedicated CU)

19. DPAC Executive Francois Mignard (Nice)chair Ronald Drimmel (Torino)deputy chair William O'Mullane (ESAC)manager of CU1 (System Architecture) Xavier Luri (Barcelona)manager of CU2 (Data Simulations) Ulrich Bastian (Heidelberg)manager of CU3 (Core Processing) Dimitri Pourbaix (Brussels)manager of CU4 (Object Processing) Floor van Leeuwen (Cambridge)manager of CU5 (Photometric Processing) David Katz (Paris) manager of CU6 (Spectroscopic Processing) Laurent Eyer (Geneva) manager of CU7 (Variability Analysis) Coryn Bailer-Jones (Heidelberg) manager of CU8 (Astrophysical Parameters) Claude Huc (CNES) DPC representative

20. Summary ● Data processing is iterative and self-calibrating – simultaneous solution over ~10 8 stars – objects mixed up in space and time – astrometry, photometry and spectroscopy ● Significant challenges – e.g. instrument calibration following radiation damage – managing a large, dispersed consortium over 13 years – data quantity modest (1 Mb/s for 5 years, ~ 100 TB raw)... –... processing complexity is not! (~10 21 FLOPS. Cf. 10 17 FLOPS from 1 PC in 1 year) ● Data Processing and Analysis Consortium (DPAC)

21. Data products ● position, parallax, proper motions (5 parameters) ● radial velocity for brighter sources (6 th phase space coordinate) ● G-band magnitude ● RP/BP spectrum ● RVS spectrum for brighter sources ● astrophysical parameters for stars, galaxies and QSOs ● (spectro)photometric variability information ● binary system parameters, orbital solutions (inc. exoplanets) ● solar system object (asteroid) orbits and taxonomy ● alerts of transient events (e.g. supernovae)