1. Towards a demonstrator for autonomous object detection on board Gaia From stars to silicon... and back Shan Mignot

2. Outline I. Gaia user requirements II. On-board processing constraints and technologies III. Image analysis methods and implications algorithmic framework IV. Elements on the demonstrator some solutions to some problems V. Synthesis scientific and technical assessment PhD defense – January 10 th 2008 – Shan Mignot 2/49

3. I. Gaia

4. Timeline ESA cornerstone mission (cosmic vision program) proposal:L. Lindegren, M.A.C Perryman, and S. Loiseau (1995) Global Astrometric Interferometer for Astrophysics (GAIA) phase A: mission & system definition (2001-2006) Payload Data Handling working group Payload Data Handling Electronics technical definition activity phase B2: definition (2006-2007) support to Astrium (PDH-S contract) phases C & D: development phases (2007-2011) launch in December 2011 phases E & F: operational phase (2011-2016 +1) final products in 2020 Gaia PhD defense – January 10 th 2008 – Shan Mignot 4/49

5. Scientific return structure & evolution of the Galaxy solar system (satellites, minor planets etc.) large catalogue of objects (stars, quasars, extra-solar planets etc.) fundamental physics (general relativity, gravitational waves etc.) Dataset phase-space map of the Galaxy (astrometry) positions: right ascension, declination & parallax velocities: proper motion & radial velocity astrophysical database multi-band & multi-epoch photometry intermediate resolution spectroscopy very precise very numerous highly homogeneous Gaia Science case PhD defense – January 10 th 2008 – Shan Mignot 5/49

6. HiPParCoS´s legacy Global astrometry scanning principle: spin & precession motions on-the-fly data acquisition cross-matching between transits 2 lines of sight: 2 telescopes (combined focal planes) Gaia PhD defense – January 10 th 2008 – Shan Mignot 6/49

7. Beyond HiPParCoS Magnitude-limited survey no catalogue on-board object detection ~50 TB of data reliability 14 magnitudes flux ratio up to 1/400 000 stellar densities max/mean > 120 Detectors 106 CCDs full read-out: 403 MB/s (total) digital image processing: detect Orbit around L2 (1.5 10 6 km) dynamical & thermal stability attitude & orbit control limited bandwidth to earth & visibility autonomous data management: characterise Gaia PhD defense – January 10 th 2008 – Shan Mignot 7/49

8. Payload I Gaia PhD defense – January 10 th 2008 – Shan Mignot 8/49

9. Payload II Gaia detection confirmation observation propagation detection PhD defense – January 10 th 2008 – Shan Mignot 9/49

10. Time-delay integration (TDI) Gaia 4 phase charge transfer column per column read-out (every 0.9828 ms) pixel packets sent to Video Processing Unit (SpaceWire serial interface) 4.4s PhD defense – January 10 th 2008 – Shan Mignot 10/49

11. Stars ? Gaia G = 2 G = 6 G = 10 G = 20 G = 15 PhD defense – January 10 th 2008 – Shan Mignot 11/49

12. Needs Individual objects galactic physics all densities high background extra-galactic physics unresolved galaxies resolved stars high densities stellar physics star types & colour multiple stars variable stars rare events (supernovae, micro-lensing) reliability solar system (minor planets, satellites) resolved objects moving objects Collectively statistical quality of the catalogue selection function global iterative solution predictability reference frame coherence Gaia PhD defense – January 10 th 2008 – Shan Mignot 12/49

13. double stars density Some cases of interest Gaia prompt particle events extra-galactic complex skies PhD defense – January 10 th 2008 – Shan Mignot 13/49

14. Requirements Gaia Selection function performance 98% detection probability sensitivity to object types increase signal to noise ratio estimate the total noise (sky background) homogeneity calibrate CCD data & correct defects stationary (along scan) uniform (across scan) graceful degradation priority-driven detection signal undetected objects Processing & resources enforce limiting magnitudes avoid false detections filter prompt particle events adapt imaging properties (sampling) PhD defense – January 10 th 2008 – Shan Mignot 14/49

15. II. On-board processing

16. Dependability & reliability part selection & testing operational modes / reconfiguration redundancies Survival & operation mechanical withstand launch temperature range (-55 o C to 125 o C) cycling dissipation vacuum outgassing (contamination) electromagnetic: power supply compatibility electrostatic discharges radiation Electronics in space On-board processing PhD defense – January 10 th 2008 – Shan Mignot 16/49

17. Radiative environment On-board processing Photons Sun & Earth (albedo & black body) thermal stability: sun shield, no eclipse, constant orientation attitude: radiation pressure Solar wind plasma interplanetary orbit protons & α particle (< 1 keV) electrostatic discharges erosion of covering High energy particles solar protons (1 keV to GeV) cosmic rays (1 keV to GeV) secondary particles PhD defense – January 10 th 2008 – Shan Mignot 17/49 number energy

18. Radiation effects On-board processing Processes electron / positron pairs ionisation displacement damage electrostatic discharges Impact single event upset material degradation latch-up PhD defense – January 10 th 2008 – Shan Mignot 18/49

19. Microelectronics On-board processing Radiation tolerance fineness of engraving frequency of operation hardness triple module redundancy error detection and correction Performance space availability delay (adapted from commercial) degraded density & speed commercial parts procurement problem design complexity (TMR & EDAC) PhD defense – January 10 th 2008 – Shan Mignot 19/49

20. Mixed architecture On-board processing PDHE TDA multiplicity of tasks context switches data intensive cache misses & slow memory bus interface with CCDs 50% of TDI mixed architecture recommended : CPU (software) + FPGA / ASIC (dedicated hardware) FPGA / ASIC custom processor library of elementary logical blocks interconnexions development (codesign) bit-level power timing performances efficient for control inefficient for arithmetics PhD defense – January 10 th 2008 – Shan Mignot 20/49

21. III. Image analysis

22. Rationale Image analysis Principle locate the objects of interest characterise them Generic approach locate all objects with the same logic simplify verification / validation avoid problems at the interface save resources (single process) Hardware / software partition detection is compression: exhaustive pixel list abstract description real-time constraints & complexity pixels are regular & simple: hardware objects are random & complex: software PhD defense – January 10 th 2008 – Shan Mignot 22/49

23. Transforms global view concentrate the information: 2 steps in one requires a priori information vs. variability of imaging: object types (extended, colour, brightness) heterogeneities (smearing, aberrations) complex analysis of transform space linear transforms are convolutions systematic arithmetically intensive: adders & multipliers in AC only (data access vs. TDI read-out) Approaches I Image analysis PhD defense – January 10 th 2008 – Shan Mignot 23/49

24. Approaches II Image analysis Local analysis detect & characterise within a neighbourhood of predefined shape limited & systematic data accesses (fixed pattern) limited information in the neighborhood: discriminate:artefacts related to noise & PSF particles identify:saturated stars (degeneracy) extended objects (different patterns)compound objects (multiple stars & density) need for a selective yet flexible generic approach: difficult type I (false positive) & type II (false negative) trade-off SWA working window PhD defense – January 10 th 2008 – Shan Mignot 24/49

25. Approaches III Image analysis Segmentation region-growing method attribute each pixel to background or objects based on total noise estimate discard background ( 90% pixels) identify objects in binary mask compatible with raster order (TDI read-out) characterisation flexible geometry: rich content independent for each domain priority-driven hardware / software partition PhD defense – January 10 th 2008 – Shan Mignot 25/49

26. Functional architecture Image analysis raw data thresholded background map deblended connected components measurements pixel level object level PhD defense – January 10 th 2008 – Shan Mignot 26/49

27. Sampling Image analysis Needs ensure completeness increase signal to noise ratio save resources decrease resolution Method hardware pixel binning (at CCD level): 2x2-pixel samples Consequences data flow reduction: 3.8 MB/s 0.95 MB/s real-time: 2 TDIs per column same hardware/software for the two detection CCDs precision: OK for science, 4x more objects for attitude control PhD defense – January 10 th 2008 – Shan Mignot 27/49

28. Pre-calibration Image analysis Needs selection function (intra & inter CCDs) stationary detection probabilities accurate measurements control false detections graceful degradation in time Functional calibration: pixel response & dark & offset cosmetic defects: black & white pixels Method linear transform (generalises flat-field & dark) fixed-point arithmetics replacement mechanism VIMOS CCD PhD defense – January 10 th 2008 – Shan Mignot 28/49

29. Background I Image analysis Needs estimate the total noise (including the sky background) stationary detection probabilities accurate measurements control false detections Functional latency & resolution trade-off adapted to pixel statistics robust to stellar content systematic calculation Method regional estimates: hyperpixels histograms: 4 ADU bins interpolated mode: precise & robust 2D bilinear interpolation fixed point arithmetics PhD defense – January 10 th 2008 – Shan Mignot 29/49

30. Background II Image analysis hyperpixel mode values interpolation PhD defense – January 10 th 2008 – Shan Mignot 30/49

31. Background III Image analysis PhD defense – January 10 th 2008 – Shan Mignot 31/49

32. Pixel selection Image analysis Needs save resources discard background pixels control false detections robustness to noise filter faint stars Functional signal to noise threshold Method signal: subtract background (bkgd) noise: Poisson noise (pix) & read-out noise (σ RON ) fixed-point arithmetics PhD defense – January 10 th 2008 – Shan Mignot 32/49

33. Simple object model I Image analysis Needs transition from hardware to software priority-driven characterisation Functional form object data units insert in priority-ordered interface with software Method connected-component labelling compatible with raster-order: label / merge / relabel simple descriptors (flux, background flux, number of pixels) PhD defense – January 10 th 2008 – Shan Mignot 33/49

34. Simple object model II Image analysis object 2object 1 Label Merge Relabel Extract PhD defense – January 10 th 2008 – Shan Mignot 34/49

35. Characterisation Image analysis Needs filter unwanted object refine object model measure objects Functional save resources: software-optimised filter prompt particle events: cascade of descriptors enforce limiting magnitudes identify components in compound objects Method adaptive sequence (decision tree) object-wise SNR test, energy density, number of interior pixels etc. watershed-based component segmentation compute flux & barycentre PhD defense – January 10 th 2008 – Shan Mignot 35/49

36. IV. Elements on the demonstrator

37. Software model Demonstrator Uses R&D performance evaluation software part characterisation engine reference model fixed-point arithmetics data accesses PDHE TDA (ANSI C) hardware developments PhD defense – January 10 th 2008 – Shan Mignot 37/49

38. Architecture Demonstrator Interfaces input: CCDs via serial SpaceWire link video reception buffer: PC with IO board no output intermediate data storage 2 SRAMs Simplified FPGA board developments no real-time processor design simplifications no connected component labelling no management of software interface free pins inspectable design output data stored in PC PhD defense – January 10 th 2008 – Shan Mignot 38/49

39. Platform Demonstrator Part Actel: ProASIC3E instead of RTAX-S reprogrammable: flash-based instead of antifuse slower (interconnections) less dense Starter Kit ProASIC3E 600 SRAMs ISSI ISI61LV51216 static asynchronous 16-bit data 19-bit address PC interface handshake 16-bit data PhD defense – January 10 th 2008 – Shan Mignot 39/49

40. Processing Demonstrator Sequential Parallel Pipeline 1 10 1 110 01 00 PhD defense – January 10 th 2008 – Shan Mignot 40/49

41. Design Demonstrator Pipeline pre-calibration background pixel selection each is a pipeline Clocks DCLK: data clock pipeline control (~1 MHz) SCLK: SRAM clock sequential optimisations (~32 MHz) CLK: main clock SRAM interface (125 MHz) Design for test processing core & debug core conditional instantiation: inspectable, piece-wise verification PhD defense – January 10 th 2008 – Shan Mignot 41/49

42. Pixel selection Demonstrator Tasks background interpolation on demand signal to noise threshold output stream of selected pixels Pipeline DCLK mode addresses (4) AL & AC coordinates interpolation coefficients read modes (4) contributions (4)sum signal signal 2 threshold test output PhD defense – January 10 th 2008 – Shan Mignot 42/49

43. V. Synthesis

44. Demonstrator Synthesis VHDL ESA standard (except for testing: verification & validation) Simulation validated pre-synthesis & post-synthesis Synthesis 14722 cells > ProASIC3E 600 ProASIC3E 1500 (100% margin) slow routing target: RTAX-S 1000 (ITAR) PhD defense – January 10 th 2008 – Shan Mignot 44/49

45. Science aims Synthesis Performances realistic scenes to verify processing completeness location magnitude false detections PhD defense – January 10 th 2008 – Shan Mignot 45/49

46. Gaia Synthesis OBDH working group Payload Data Handling Support phase A: PDHE algorithms port to real-time software platform analysis phase B: support to prime algorithmic specification reviews validation ALGOL ACI contribution to highly constrained embedded computing Publications 23 technical notes: detection & other on-board processing 1 conference proceedings PhD defense – January 10 th 2008 – Shan Mignot 46/49

47. Perspectives Synthesis Demonstrator existing place & route validate interfaces verification validation campaign extension second generation complete architecture software interface real-time software engine Collaborations experts in electronics experts in embedded applications experts in space PhD defense – January 10 th 2008 – Shan Mignot 47/49

48. If it looks easy, you do not understand it. M.A.C. Perryman, 2001.

49. Acknowledgements Pyxis F. Chéreau, C. Macabiau, J. Chaussard, F. Arenou based on APM, Sextractor, the watershed transform Simulations F. Chéreau, F. Arenou, C. Babusiaux Demonstrator P. Laporte, F. Rigaud ALGOL ACI PRiSM, LaMI Acknowledgements PhD defense – January 10 th 2008 – Shan Mignot 49/49

50. Image credits EADS Astrium SAS Data Processing and Analysis Consortium (DPAC) Gaia Image and Basic Instrument Simulator (GIBIS) European Space Agency (ESA) National Aeronautics & Space Administration (NASA) European Southern Observatory (ESO) A. Short E. Oseret F. Rigaud internet: N. Giffin, A. Bloom, answers.com, lipcoop.com Image credits PhD defense – January 10 th 2008 – Shan Mignot 50/49