Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Cross Cal Meeting Mar 2015 Automated Pipeline Processing.

Published byJob Cooper Modified over 8 years ago

Similar presentations

Presentation on theme: "1 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Cross Cal Meeting Mar 2015 Automated Pipeline Processing."— Presentation transcript:

1 1 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Cross Cal Meeting Mar 2015 Automated Pipeline Processing

2 2 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Pipeline Automation A detailed description of the CAA pipeline automation system was given in the last Cross-Cal Updates since then:- Inclusion of EFW L2/L3 pipelines Inclusion of WBD conversion pipeline Web based monitoring dashboard Routine processing based on new deliveries Processing of WBD conversion backlog Work in progress:- Inclusion of FGM & EDI ISR2 conversion pipeline Inclusion of PEACE & CIS moments in inertial frame (for both GSE & ISR2) Bug fixing

3 3 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Pipeline Automation New pipelines implemented since last cross cal 2000-11-01 yesterday C1_CP_WBD_WAVEFORM C1_CE_WBD_WAVEFORM_CDF 2000-11-01 yesterday C2_CP_WBD_WAVEFORM C2_CE_WBD_WAVEFORM_CDF 2000-11-01 yesterday C3_CP_WBD_WAVEFORM C3_CE_WBD_WAVEFORM_CDF 2000-11-01 yesterday C4_CP_WBD_WAVEFORM C4_CE_WBD_WAVEFORM_CDF 2000-11-01 yesterday C1_CP_EFW_L2_E3D_INERT C1_CP_AUX_POSGSE_1M C1_CP_FGM_FULL,2014-05-11 C1_CP_EFW_L2_E CL_SP_AUX 2000-11-01 yesterday C2_CP_EFW_L2_E3D_INERT C2_CP_AUX_POSGSE_1M C2_CP_FGM_FULL,2014-05-11 C2_CP_EFW_L2_E CL_SP_AUX 2000-11-01 yesterday C3_CP_EFW_L2_E3D_INERT C3_CP_AUX_POSGSE_1M C3_CP_FGM_FULL,2014-05-11 C3_CP_EFW_L2_E CL_SP_AUX 2000-11-01 yesterday C4_CP_EFW_L2_E3D_INERT C4_CP_AUX_POSGSE_1M C4_CP_FGM_FULL,2014-05-11 C4_CP_EFW_L2_E CL_SP_AUX 2000-11-01 yesterday C1_CP_EFW_L3_E3D_INERT C1_CP_AUX_POSGSE_1M C1_CP_FGM_5VPS,2014-05-11 C1_CP_EFW_L3_E CL_SP_AUX 2000-11-01 yesterday C2_CP_EFW_L3_E3D_INERT C2_CP_AUX_POSGSE_1M C2_CP_FGM_5VPS,2014-05-11 C2_CP_EFW_L3_E CL_SP_AUX 2000-11-01 yesterday C3_CP_EFW_L3_E3D_INERT C3_CP_AUX_POSGSE_1M C3_CP_FGM_5VPS,2014-05-11 C3_CP_EFW_L3_E CL_SP_AUX 2000-11-01 yesterday C4_CP_EFW_L3_E3D_INERT C4_CP_AUX_POSGSE_1M C4_CP_FGM_5VPS,2014-05-11 C4_CP_EFW_L3_E CL_SP_AUX C1_CP_EFW_L2_E3D_INERT 3 4 2 /home/caa_ops/PIPELINE/EFW_L2_L3/bdgp_efw_l23_caa.sh C2_CP_EFW_L2_E3D_INERT 3 4 2 /home/caa_ops/PIPELINE/EFW_L2_L3/bdgp_efw_l23_caa.sh C3_CP_EFW_L2_E3D_INERT 3 4 2 /home/caa_ops/PIPELINE/EFW_L2_L3/bdgp_efw_l23_caa.sh C4_CP_EFW_L2_E3D_INERT 3 4 2 /home/caa_ops/PIPELINE/EFW_L2_L3/bdgp_efw_l23_caa.sh C1_CP_EFW_L3_E3D_INERT 5 4 2 /home/caa_ops/PIPELINE/EFW_L2_L3/bdgp_efw_l23_caa.sh C2_CP_EFW_L3_E3D_INERT 5 4 2 /home/caa_ops/PIPELINE/EFW_L2_L3/bdgp_efw_l23_caa.sh C3_CP_EFW_L3_E3D_INERT 5 4 2 /home/caa_ops/PIPELINE/EFW_L2_L3/bdgp_efw_l23_caa.sh C4_CP_EFW_L3_E3D_INERT 5 4 2 /home/caa_ops/PIPELINE/EFW_L2_L3/bdgp_efw_l23_caa.sh C1_CP_WBD_WAVEFORM 0 8 -2 /home/caa_ops/PIPELINE/WBD_CEF/bdgp_wbd_cdf2cef.sh caa[3467] C2_CP_WBD_WAVEFORM 0 2 -2 /home/caa_ops/PIPELINE/WBD_CEF/bdgp_wbd_cdf2cef.sh caa[3467] C3_CP_WBD_WAVEFORM 0 2 -2 /home/caa_ops/PIPELINE/WBD_CEF/bdgp_wbd_cdf2cef.sh caa[3467] C4_CP_WBD_WAVEFORM 0 2 -2 /home/caa_ops/PIPELINE/WBD_CEF/bdgp_wbd_cdf2cef.sh caa[3467]

4 4 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Pipeline Automation Web Based Monitoring

5 5 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Pipeline Automation Web Based Monitoring

6 6 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Pipeline Automation Web Based Monitoring

7 7 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Pipeline Automation Web Based Monitoring

8 8 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Pipeline Automation

9 9 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Pipeline Automation WBD conversion has been the dominant processing activity More than 7 years backlog to process ~60 cpu cores across 4 servers @ 24x7 for nearly 3 months Finally completed last week of Feb Other tasks are quick by comparison Triggered by availability of new dependent data For example EFW L2/L3 by FGM re-deliveries A log file consolidation process helps to track failed jobs and tracks when they have been resolved C2_CT_AUX_TMMODE__20010101_000000_20060101_000000_V03 2015-03-23T23:30:36Z ERROR: Cleared 2015-03-24T09:07:46Z C3_CT_AUX_TMMODE__20010101_000000_20060101_000000_V03 2015-03-23T23:29:57Z ERROR: Cleared 2015-03-24T09:07:46Z C4_CP_AUX_SPIN_TIME__20150115_000000_20150116_000000_V01 2015-02-09T15:21:36Z ERROR: C3_CP_AUX_SPIN_TIME__20150115_000000_20150116_000000_V01 2015-02-09T15:21:34Z ERROR: C2_CP_AUX_SPIN_AXIS__20141001_000000_20141101_000000_V02 2015-02-09T15:17:38Z ERROR: Cleared 2015-02-10T08:56:16Z C1_CP_AUX_SPIN_AXIS__20141001_000000_20141101_000000_V02 2015-02-09T15:17:38Z ERROR: Cleared 2015-02-10T08:56:15Z C1_CP_AUX_SPIN_AXIS__20140901_000000_20141001_000000_V02 2015-02-09T15:17:38Z ERROR: Cleared 2015-02-10T08:56:11Z

10 10 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 Summary Automated pipeline processing system running Being extended for additional pipeline Some minor bugs being addressed

Download ppt "1 CAA 21 th Cross Calibration Meeting, Leiden, 24-26 th Mar 2015 CAA Cross Cal Meeting Mar 2015 Automated Pipeline Processing."

Similar presentations

Building ontologies using Jenkins. Changing requirements for ontology engineering Original ontology build pipeline – What pipeline? – Life on the bleeding.

Building ontologies using Jenkins. Changing requirements for ontology engineering Original ontology build pipeline – What pipeline? – Life on the bleeding.
Scrum Jarred Payne Ashrith Pillarisetti. Scrum Prepare for Project Plan the Project Plan a Sprint Run a Sprint Track the Sprint.

Scrum Jarred Payne Ashrith Pillarisetti. Scrum Prepare for Project Plan the Project Plan a Sprint Run a Sprint Track the Sprint.
Visual Studio Online. What it Provides Visual Studio Online, based on the capabilities of Team Foundation Server with additional cloud services, is the.

Visual Studio Online. What it Provides Visual Studio Online, based on the capabilities of Team Foundation Server with additional cloud services, is the.
Getting the Most Out of Blue Mountain RAM

Getting the Most Out of Blue Mountain RAM
Integrated Circulation Reporting Prototype Demo 30 th May 2013.

Integrated Circulation Reporting Prototype Demo 30 th May 2013.
FGM report 10 th Cross calibration workshop Elizabeth Lucek, Patrick Brown, Chris Carr, Tim Oddy, André Balogh I mperial College London November 2009.

FGM report 10 th Cross calibration workshop Elizabeth Lucek, Patrick Brown, Chris Carr, Tim Oddy, André Balogh I mperial College London November 2009.
CS 360 Lecture 3.  The software process is a structured set of activities required to develop a software system.  Fundamental Assumption:  Good software.

CS 360 Lecture 3.  The software process is a structured set of activities required to develop a software system.  Fundamental Assumption:  Good software.
November 01, 2005 MTAC 93 Design & Concept Testing Workgroup Status.

November 01, 2005 MTAC 93 Design & Concept Testing Workgroup Status.
ASPERA-3 April. 25, 01 Monthly Status Report ASPERA Monthly Status ReportMonthly 4_25/ Page 1 Monthly Status Report Report due 4/25/01 Schedule statused.

ASPERA-3 April. 25, 01 Monthly Status Report ASPERA Monthly Status ReportMonthly 4_25/ Page 1 Monthly Status Report Report due 4/25/01 Schedule statused.
Microsoft FrontPage 2003 Illustrated Complete Finalizing a Web Site.

Microsoft FrontPage 2003 Illustrated Complete Finalizing a Web Site.
FGM report 9 th Cross calibration workshop Elizabeth Lucek, Patrick Brown, Paul French, Chris Carr, Tim Oddy, André Balogh I mperial College London March.

FGM report 9 th Cross calibration workshop Elizabeth Lucek, Patrick Brown, Paul French, Chris Carr, Tim Oddy, André Balogh I mperial College London March.
11 14th CAA Cross-Calibration meeting, York, 5-7 Oct 2011 STAFF CAA products & Cross-Calibration activities Patrick ROBERT & STAFF Team 5) STAFF-SC CWF.

11 14th CAA Cross-Calibration meeting, York, 5-7 Oct 2011 STAFF CAA products & Cross-Calibration activities Patrick ROBERT & STAFF Team 5) STAFF-SC CWF.
PanDA Monitor Development ATLAS S&C Workshop by V.Fine (BNL)

PanDA Monitor Development ATLAS S&C Workshop by V.Fine (BNL)
Swift HUG 12-13 April 2006 1 Swift data archive Lorella Angelini HEASARC.

Swift HUG April Swift data archive Lorella Angelini HEASARC.
Milestone 5 Review Control Phase Complete Cost Data Integrity Project February 28, 2008.

Milestone 5 Review Control Phase Complete Cost Data Integrity Project February 28, 2008.
Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Data Processing Progress.

Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Data Processing Progress.
ASPERA-3 Mar. 25, 01 Monthly Status Report ASPERA Monthly Status ReportMonthly 3_25/ Page 1 Monthly Status Report Report due 3/25/01 Schedule statused.

ASPERA-3 Mar. 25, 01 Monthly Status Report ASPERA Monthly Status ReportMonthly 3_25/ Page 1 Monthly Status Report Report due 3/25/01 Schedule statused.
TNMP LSE File Delivery Delays RMS Update 4/7/2015.

TNMP LSE File Delivery Delays RMS Update 4/7/2015.
STAFF Report Patrick Robert, Rodrigue Piberne & STAFF team.

STAFF Report Patrick Robert, Rodrigue Piberne & STAFF team.
18 th GIST Meeting 14 th –16 th May 2003 Ground Segment (GGSPS) Report 1 GERB Ground Segment B.C.Stewart RAL.

18 th GIST Meeting 14 th –16 th May 2003 Ground Segment (GGSPS) Report 1 GERB Ground Segment B.C.Stewart RAL.

Similar presentations

Ads by Google