1. From delivery to acceptance
E. Villard

2. Reference document ALMA Software Delivery Process
COMP E-PLA
Version: E
Status: final

3. Stage 1: Patch Version Candidate Announced
Actor: Pipeline Development Team
The pipeline team has tested the version internally and it meets the agreed requirements. Often the Pipeline Working Group (PLWG) will have already been involved in testing new features.  

4. Stage 2: Validation Testing
Actor: Pipeline Working Group
The pipeline working group performs such tests as necessary to ensure that the new heuristics are performing as expected and can safely be used in production use for current cycle ALMA data.

5. Stage 3: Recommendation for acceptance.
Actor: Subsystem Scientist / Deputy Subsystem Scientist
Recommendation for Acceptance is sent to the head of the data management group (DMG) stating that the updated version is ready for deployment

6. Stage 4: Local deployment and validation at JAO.
Actor: Science Archive Content Managers (SACM)
The deputy of DMG coordinates with the Science Archive Content Managers the deployment of the new version in the test environment at JAO, and the validation.
The validation only involves running the TEST version of the Pipeline release with a few datasets and compare the outputs with previous version runs to check:
That the new pipeline heuristic software has been installed properly at JAO
That the new version does produce the SAME results as before for cases when the new release is supposed to have NOT done anything to the data
For specially important fixes to known bugs in previous releases, just run one/two examples of MOUS (Member Obs Unit Set) that previously showed the problem and check that they are solved by using the new heuristic version.
The whole process is expected to take just 1-2 days of work (usually only one SACM is needed for this) and only involves usually ~5 MOUS. Since the number of ExecBlocks is NOT a problem (except when the fix to a bug specifically deals with multi-ExecBlock issues), MOUS with a SINGLE ExecBlock are OK.
Once the tests are done, the Science Archive Content Managers will send an to the Head of DMG and his/her deputy, with the results of the whole validation.

7. Stage 5: Approval of Recommendation
Actor: Head of Data Management Group
Head of DMG reviews acceptance recommendation and obtains approval from head of DSO for deployment.  Open ticket to APO group requesting the deployment of the newly accepted version.

8. Stage 6: Deployment Actor: Archive / Pipeline Operations Manager (APO)
APO Manager coordinates the deployment of this version at JAO and at ARCs.
Actor: Subsystem Scientist / Deputy Subsystem Scientist
Updates a wiki page on the Science Portal to indicate the date change of the Operations processing version, and the effects of the change. 

9. Recent improvements Use of data from first online TRR (Apr)
Helps detect early on unexpected changes in metadata
Use of E2E test data (Aug) for PL acceptance
Helps test thoroughly the PL on representative data
No major change implemented into PL after E2E test

10. Suggestions for improvement
More involvement of PL development team into deployment
Freeze of cluster hardware and software after PL validation and deployment
Better coordination between clusters around the world