1. ASAC Meeting - May 2004 Computing IPT Science Requirements: Status Report Robert Lucas & Debra Shepherd

2. ASAC Meeting - May 20042 Requirements status In project-level document 70.10.00.00-002-I-SPE prepared by SSR Committee –Previously sw-memo11, in formal approval process (minor revisions) –Will need revision based on final Operations Plan Requirements priorities to be updated as recommended by PDR Review Committee: Prioritize the requirements in light of schedule and cost drivers, science priorities, and the importance of achieving early-science operations. This should be done in the context of the science operations plan, not independent of it. However …Operations and Commissioning Plans not finalized

3. ASAC Meeting - May 20043 Coordination with Operations Planning Participation of Computing IPT in Operations Group discussions (R. Lucas / J. Schwarz) Operations Group & Computing IPT coordination meeting in January 2004: provided further needed inputs for Draft G of Operations Plan. –Defer breakpoint development –Quick look system for staff / ARCs, results communicated to users in case of problems –Expert mode available, imply staff collaboration Expect inputs from Operations Group in May 2004 on requirements for operations management tools, e.g., types of tools required, monitor data to display, functionality requirements.

4. ASAC Meeting - May 20044 Prioritization linked to development Longer term prioritization: –ATF critical for software development to have realistic test environment –Repeat ATF tests on 1 & 3 antennas in Chile (1 st level science commissioning tests: 1 ant : R3 Oct 2005, 3 ants : R4 Oct 2006) –Delay offline GUI development, focus on command line processing for early science –Early science: use Offline software for data reduction, expect Pipeline heuristics in development (but not ready for production use).

5. ASAC Meeting - May 20045 Issues to be addressed(1) Clarify SSR requirements, add granularity if needed –now in progress (details to follow ) Identify requirements that drive development costs & uncertainty (needed for global prioritization/critical path analysis). – Japan impact: some re-costing for CDR2, – Operations plan has driven some prioritization issues: e.g. deferral of breakpoint development

6. ASAC Meeting - May 20046 Issues to be addressed (2) Identify Early Science operations modes, in collaboration with Science IPT –needed to prioritize requirements in light of science priorities –Will be completed by CDR2 –Community feedback suggests: start simple  complex, promise only what has been tested (ATF) Identify missing requirements needed for the critical path (e.g. operations, commissioning, integration requirements) –Requirements management plan in place by CDR2 –Need detailed integration & commissioning tests defined –PMCS adopted soon, will provide critical path analysis

7. ASAC Meeting - May 20047 Status of Development Passed PDR in Spring 2003, CDR1 Aug. 2003 Delivered prototype ACS, Control and Correlator sw. to ATF First R1.0 Release Oct.2003 (in time), integrated by Dec. 2003 (in time) Successful first user-end tests: –Offline data reduction –Observing Tool prototype –Control software (ATF) –On-line calibration tested on real ATF data (pointing) –Pipeline –Scheduling (simulation mode) Release R1.1 on 1 Apr.2004

8. ASAC Meeting - May 20048 Next Steps in Development CDR2 incremental design review (2004 July 8-10) –External review –Major topics left over from PDR: Impact of Operations Plan Status of AIPS++ Release R2.0 expected in time for 2004 Oct 1

9. ASAC Meeting - May 20049 R2 release For R2 (2004 Oct 1): focus on single baseline interferometry at the ATF: –Simulation mode or actual hardware use (after system integration) –Control system: command language available –Scheduling: interactive scheduling mode implemented –Correlator: store raw data in VOTable format –Observing Tool: support interactive scheduling, one standard mode, one calibration mode, and ‘expert’ mode –Telescope calibration (phase, amplitude, pointing, atmosphere) –Pipeline: provide initial Quick Look Display –Archive: first real-world ALMA use of NGAST Note: ambitious plan!

10. ASAC Meeting - May 200410 Export Data Format Raw data: use ALMA-specific format (AEDF) –Bulk correlator data flows directly into archive –Meta data, Auxiliary data prepared in quasi real time by the Control system –Include on-line calibration data, used in quasi-real time by Quick Look system, and by Science Pipeline at the end of the execution of a set of SBs (scheduling Blocks) –Exported to users as sets of VOTables or FITS tables from the Archiver –This is the DRP-independent input data interface Document in DAR: (ALMA-70.00.00.00-004-A-SPE, F.Viallefond & R.Lucas): Comments welcome, please!

11. ASAC Meeting - May 200411 Science Requirements Evaluation: User Test Plan Status At least one and often many Use Cases available for all subsystems (http://www.aoc.nrao.edu/~dshepher/alma/usecases)http://www.aoc.nrao.edu/~dshepher/alma/usecases Test Plans available for all subsystems (http://www.aoc.nrao.edu/~dshepher/alma/testplans)http://www.aoc.nrao.edu/~dshepher/alma/testplans Test Reports available at (http://www.aoc.nrao.edu/~dshepher/alma/testreports) The first user tests are completed. A good example is the offline subsystem - consider as an example to show test plan and execution preparation.

12. ASAC Meeting - May 200412 User Test Plan Example Offline Science Software Development & Test Schedule 20032004200520062007 Single Field VLA & PdBI 13mm 1/3mm User Acceptance Tests Small Multi-fld Mosaic Interferometer/TP combo Complex Interferometric Mosaic Sim Data: TP & Mosaic (separate) Sim Data: Large Multi-field Mosaic Sim/test data: Sgl Fld, Mosaic, TP (separate&combo) ALMA Early Science Begins Increased capability VLA & BIMA mosaics

13. ASAC Meeting - May 200413 Test Example: Offline User TST1 http://projectoffice.aips2.nrao.edu Information provided to testers: Goals, scope, AIPS++ installation info, detailed cookbook and dataset summaries, blank questionnaire and Offline requirements spread sheet for testers to fill out. Expected results provided in Testing comparison page, final results also posted. ALMA Offline TST1

14. ASAC Meeting - May 200414 Offline User TST1: Testing Comparison Page ALMA Offline TST1

15. ASAC Meeting - May 200415 ALMA Offline User TST1: Selected Test Results 3mm continuum image 1mm Spectral line cube PdBI line & continuum emission dataset Detailed reduction scripts

16. ASAC Meeting - May 200416 Requirements Tracking Test Cycle 1: Compliance matrix (7 subsystems) Subsystem Expected CompletionGoalsAchieved? Archive1nov03- XMLStore, MonitorStore, BulkStore work with unit tests and in monthly integrations Yes Control1apr04- Optical pointing telescope integration complete - Holography done with ATF software - Radiometry done with ATF software Yes Scheduling11dec03- Schedule simple simulated projects - Develop draft logs/evaluation output Yes Obs Prep15feb04- Stubs to support defined observing programs existYes Tel Cal1mar04- Process Single Dish pointing & focus scans in ALMATI format - Perform simple atmospheric calibration - Archive results in xml format Yes Offline1mar04- Single field, high SNR (VLA) DR/imaging works - Single field, low SNR (PdBI) DR/imaging works Yes Pipeline7may04- Test Proto-pipeline basic function (GRB data) - Extension to similar data (another GRB dataset) - Extension to different data (3 & 1mm spectral line data) Yes

17. ASAC Meeting - May 200417 Pipeline User Test 1 Results GRB 040329 Pipeline image Offline (glish)-Pipeline (ACS/python) image  GRB GG Tau, 3mm continuum Pipeline image rms=0.0025mJy/bm Peak=0.49mJy/bm rms = 0.47mJy/bm Peak = 7.2 mJy/bm rms=7.8e-4mJy/bm Peak=7.5e-3mJy/bm (Less than 1% difference, slight divergence after CLEAN iteration ~800) (http://www.aoc.nrao.edu/~dshepher/alma/testreports/Pipeline.TestReport1.txt) rms = 0.0mJy/bm Identical images Offline (glish)-Pipeline (ACS/python) image

18. ASAC Meeting - May 200418 Requirements Tracking - Test Cycle 2: Compliance matrix (being defined) Subsystem Expected Completion GoalsAchieved? ArchiveTBD- XMLStore works with EDF-like data - Archive browser can be used by ObsPrep, PL, TelCal - Simulated Correlator stream captured by BulkStore fctn ------ ControlTBD- TBD- Correlator1jul04-Verify that test fixture produces intermediate processing products that are usable for external tests. - Verify functionality of GUI for the few modes supported at the ATF ---- ExecutiveTBD- System startup/shutdown works - Lifecycle, Statetree, Schedblock, Dataflow panels work ---- Obs PrepTBD- TBD- Offline15sep04- Mosaic, high SNR DR/imaging works (VLA SiO mosaic) - Mosaic, low SNR DR/imaging works (BIMA galaxy mosaic) - PipelineTBD- focus on testing first heuristics & calibrator source flagging. - Scheduling13aug04- Ensure that R1.1 deliverables work using sim-tool on stand-alone subsystem software - Tel CalTBD- Test interf. solvers, interf pointing and focus, pointing session -

19. ASAC Meeting - May 200419 Master Test Schedule – 1 http://www.aoc.nrao.edu/~dshepher/testplans/Master.TestSchedule.pdf Subsystem testing

20. ASAC Meeting - May 200420 Master Test Schedule - 2 Subsystem testing

21. ASAC Meeting - May 200421 Master Test Schedule - 3

22. ASAC Meeting - May 200422 Master Test Schedule - 4 Integrated testing

23. ASAC Meeting - May 200423 Master Test Schedule - 5

24. ASAC Meeting - May 200424 Master Test Schedule - 6

25. ASAC Meeting - May 200425 Master Test Schedule - 7

26. ASAC Meeting - May 200426 Requirements Tracking: Computing Management Plan: “It is desirable that requirements be of such granularity that it is easy to observe whether a requirement has been fulfilled or not.... It is not desirable that most requirements for a subsystem be in the state of ‘partially implemented’ during the development period.” Science requirements being evaluated to determine if ‘granularity’ is adequate. Options to increase granularity: 1.No change in scope: Generate secondary requirements with adequate granularity to track development progress. 2.Submit change request: Propose additions/corrections along with explanation of impact on project.  No Change in scope: SSR/Computing management review.  Change in Scope: ALMA Change Control board review.

27. ASAC Meeting - May 200427 User tests grade full/partial requirements. Subsystem Scientist will make recommendations to SSR & ALMA Computing IPT on which requirements are deemed ‘Adequate.’ Overall Grade Test Grades

28. ASAC Meeting - May 200428 Offline Subsystem Performance Requirements AIPS++ & Gildas Comparison PdBI GG Tau at 1 & 3mm spectral line, extended to 64 ants via simulation * Phase III AIPS++/Gildas comparison with same data: A2/G = 4 (Mar 2003)

29. ASAC Meeting - May 200429 Offline Subsystem Performance Requirements AIPS++, AIPS & Miriad Comparison VLA grav lens in polarized cm continuum, self-cal, time extended via simulation * AIPS++/AIPS comparison with same data: A2/A ~ 4 (Aug 2003)

30. ASAC Meeting - May 200430 Summary Ongoing discussions with Operations and Commissioning teams to determine non-SSR requirements and priorities. Development prioritization based on project-wide goals (e.g. hardware interface with ATF and early commissioning milestones, expected operations requirements). Still some issues to be worked out once the Operations and Commissioning/Science Verifications plans have been finalized and Early Science Modes have been identified. SSR requirements being clarified with increased granularity to track progress (will be completed by CDR2 in July 2004) Coordinated software test plans reflect development priorities. Testing underway, first tests successful. Software tests will be coordinated with commissioning tests. Science requirements acceptance procedures defined. AIPS++ performance improved significantly since last ASAC meeting.