1. SOAR Data Reduction Pipelines
Progress Report
Simón Torres R.
Data Analyst

2. The Team La Serena Software Team
Bruno Quint
César Briceño
Simón Torres
See final slide for other people involved

3. Summary Goodman HTS Pipeline Overview
We have something that works but still requires some development
Other SOAR Data Reduction Pipelines
Just some info since I’m not personally involved

4. Goodman Pipeline: What is it?
Python-based tools to:
Reduce Goodman’s raw data
Imaging
Spectroscopy
Perform Spectroscopic Reduction
Identification
Trace
Extract
Wavelength Calibration
(future plans) Flux Calibration
Full process is split in two sub-processes

5. Our Goals We we’ll need beta testers!
Very well documented software
Most processes transparent to the user
Different levels of debug available
Easy code maintenance
We’ll see that but we keep that in mind as we work
Group development in mind
Keep in mind new developers
We we’ll need beta testers!
Users/Community input is required

6. How is it being made? We use Python2.7 (Check compat. w/P3.5)
General Coding Standards
PEP8 – Style Guide
PEP257 – Docstrings Convention (in-code documentation) – Google Style
GIT + GitHub
PyCharm Development Platform
Open Source
Multiplatform Compatibility
“Modular Design”
In theory could be used as a library
PEP: Python Enhancement Proposal

7. Development Platform Centos 7 64bits No benchmarking yet but… 32GB RAM
i7 Processor
Solid State Drive
6TB Total Space, RAID 6
No benchmarking yet but…
CCD Reduction: 286 Images, 98 Sci/Comp
~ 7:30 Minutes
Spectroscopic Reduction 69 Science Images
~ 5:00 Minutes
Including user input
Only one spectroscopic configuration 400m2

8. Documentation Status In-Code Documentation – Docstrings
Updated version of ccd reduction is incomplete
GitHub Hosted Wiki
Not extensive but precise
User Documentation
Requires more work but working on it

9. PEP257 Example

10. Status and Schedule Works up to Wavelength Calibration Schedule
Not ready for a release
Important modules missing (Flux Calibration)
Other important Features missing
Automatic Wavelength Calibration
User documentation
Cross platform compatible GUI (Qt4/5?)
Schedule
Internal testing and quality controls
June Start Beta Testing (ends August)
First 1.0 Release by the end of year
Complete and stable

11. Conditions to use our pipeline
Be consistent in getting your data. Some initial pre- pipeline data cleaning & book keeping will be necessary. Example: delete focus sequences
Header information must be correct: e.g. no wrong object types
Don’t mix ROIs, Imaging/Spectroscopy data
Basic CCD reduction can handle BLUE/RED Camera as well as Imaging/spectroscopy. But not together.
Use one of the “Supported Obs. Modes”, e.g. flats taken in afternoon, or together with science frames during the night
We seek user input in order to make a better decision

12. Problems Developing a pipeline for an instrument with:
“Infinite possible configurations”
Two different cameras
Headers and Keywords differ from each other
Automatic Wavelength Solution
There are some non-linearities in the data
Maybe a characterization of them might help
Graphical Interface to find w/solution Interactively
Non-linear w/solution FITS documentation
It was easier to linearize the spectrum

13. Different Screen sizes

14. Slit Sizes
(Line detection solved)

15. How to get the code

16. Documentation Availability
This is how we will present the documentation. Will not be up dated unless there is an official release.

17. Snapshots and Data

18. Snapshots and Data
Compared to iraf you don’t need to know…

19. Screen Recording on Youtube

20. Future Plans Add Automatic Wavelength Calibration
Add Flux Calibration Module
Improve Multi-Target Capability
MOS mode
Create a Dedicated GUI
We will provide Data Reduction Computers
Three i7 32GB RAM 6TB RAID 6 Storage Computers
Pipeline will still be publicly available

21. GUI Concept

22. SOAR Pipelines Summary
Instrument
Developer
Language
Status
SIFS
Luciano Fraga
Python
Development
STELES
Eder Martioli
C++/Python
Development*
Goodman
Simón Torres
SOI
PyRAF
Production
SAMI
SAM-FP
Bruno Quint
SPARTAN
Patrick O’Brien
C++/Others
Development**
* STELES Pipeline based on OPERA by Canadian-French-Hawaii Telescope which is done but needs to be adapted to STELES. Eder Martioli from LNA. He was a post-doc at CFHT and it is a member o OPERA developed team.
** THELI
* Based on OPERA by CFHT. Eder is part of OPERA dev. team.
** THELI, Was developed by Mischa Schirmer