Marine Core Service MY OCEAN Scientific Assessment Review – QuARG chair F. Hernandez Review of the Product Quality and Validation phase of V1 Stream1 Production Centres Context Quality Assurance & Review Group review of the ScVR Validation outcomes, remarks

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Dedicated ressources planned for Cal/Val X.5 Coordination of x.5 activity inside Production Units Provide diagnostics for scientific assessment in coordination with QuARG reviews

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Context of the validation phase n Every PC has dedicated ressources to perform scientific assessment activities for: –Monitoring the V1 system performance during operations –Assessing the quality of V1 Product disseminated to users n The V1 Stream1 subsystem cal/val tools succeeded the VARR: the calibration phase proved already that: –Validation activity is in place in PC/PU –V1 Stream1 system offer an upgraded quality compared to V0 n In parallel, following the Stream2 V1 system agenda: –V1 S2 calibration phase has been performed –Based on similar system to V1, Reanalysis and Reprocessed Analysis have been performed and scientifically validated (strengthening validation synthesis for the operational system) –This has been reviewed with success at the PC Acceptance Stream2 V1.

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Objective of the validation phase n Verify that scientific validation procedures are running operationnaly at PC/PU level to asses the product quality n Verify that these procedures are used, and the product quality is as expected  Ask for a Scientific Validation Report, based on the routine validation performed since the 15th of December 2010

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Cal/Val activities for the second MyO year Prepare ScCP, ScVP… ….. V1S1 devlpt. PC Acceptance review of ScCP, ScVP, ScCR June 2010End September 2010 VAlid. Read. Rev. Start V1 S1 operation 15/12/2010 Test op. validation V1S1 scientific assessment….. V1S1 pre-operated….. V1S1 in operation Operate routinely PQ validation: reg. reporting 29/4/2011 P. Read. Rev. Review V1 S1 ScVR Calibration ….. V1S2 devlpt. / REAV1S2 & REA scientific assessment PC Acceptance review of ScCP, ScVP, ScCR, ScVR-Rea March 2011June 2011Sept VARR Start V1S2 operation PRR V1S2 ScVR ….. V1S2 in operation Validation n Validation requirements: –Compliance of routine validation activity with ScVP –Identify and report about success/defaults of V1 –Pay attention to V1 performance versus user requirements (if any) –Consider dependencies (ie, quality/impact of input data for the period)

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Status of the Calibration and Validation PC Acceptance Stream1 V1, Roma, September 2010 VARR Stream1 V1, Brussels, December 2010 PC Acceptance Stream2 V1, Copenhagen, March 2011 User Forum, Stockholm, April 2011 PRR Stream1 V1, Roma, April 2011 VARR Stream2 V1, June 2011 PRR Stream2 V1, September 2011 Prod. Centres ScCP/ScVPScCRScVR-ReaScVR (PRR) GLOxStream 15 docs 1 doc + 2 docs ARCxStream 11 doc BALxStream 1 NWSxStream 22 docs IBIxStream 2 MEDxStream 11 doc BLSxStream 21 doc SLxStream 11 doc OCxStream 1/21 doc SSTNo (in prep)Stream 1 in 2OSTIA doc1 doc SIWxStream 1 7 docs for sea ice no Wind INSxNO (in prep) 1 doc

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of V1 Product Quality assessment, from reporting n Every Production Centres fulfulling the scientific validation, some delays in delivery, but all project management procedures not already efficient: –GLO MFC: 3 PU, detailed and valuable validation –ARC MFC: detailed and valuable validation –MED MFC: detailed and valuable validation –SL TAC: detailed and valuable validation –OC TAC: detailed and valuable validation –SST TAC: ScVP expected. Detailed and valuable validation –SIW TAC: 7 doc. for sea ice. No doc for wind –> No delivery justified by Quickscat failure in 11/2009. Assessment for other products scheduled in V2 –INS TAC: no ScCR. But RTQC performed regularly. Particular attention also paid to reprocessed dataset

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of the V1 Product Quality assessment: SST TAC SST 13

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Expected outcome from calibration: accuracy levels per product (ex. SST TAC summary) Comparison to match-up database

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review ScCR results: MEDProducts example Location of the MED in situ-satellite matchups SensorN datar2r2 RMSbiasRPD [%] MODIS MERIS OC 12

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of the V1 Product Quality assessment: OC TAC OC 12

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of Product Quality assessment, from reporting: proposing new approach: OC TAC and MED MFC validating Chl-a From MED ScVR MED 9

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of the V1 Product Quality assessment: ARC MFC ARC 5

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of the V1 Product Quality assessment: GLO MFC In this section we display examples of CLASS1 routine consistency check of oceanic large scale signals and mesoscale structures. Several dates were chosen in order to illustrate the major good points and biases that have been diagnosed or monitored since the operational launch of V1. The plots presented here follow the CLASS1 metrics philosophy as they compare the model outputs and observations on a same grid. The MyOcean products themselves are interpolated on standard grids and in GODAE regions and constitute the “official” CLASS1 files for intercomparisons of models. We can see in Figure 1 a very good agreement between both Global and HR-Zoom systems and SLA observations. Nevertheless strong discrepancies are visible in the English Channel, the North Sea and the Baltic Sea. This illustrates one of the current limitations of the system which produces a spicy bias in the North Sea and the Baltic. Spicy biases are also observed in the Celtic Sea GLO 4

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of the V1 Product Quality assessment: GLO MFC GLO 4

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of the V1 Product Quality assessment: GLO MFC Backup GLO 4

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Flavour of V1 Product Quality assessment, GLO PHYS PU WOA09 Synth Comb GLO 4

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Product Quality Assessment: V1 Stream1 Scientific Validation Review n Reporting has been performed, reporting mechanism had some minor deficiencies n The scientific validation procedures, and the corresponding reporting is: –Scientifically valuable –Reliable, and proved to be performed anytime (secured by many types of diagnostics) –Fair: good and bad aspects are addressed and reported –Consistent with plans and project management –Offering a continuity with system and products assessment (refer to calibration phase, to V0, compliant with MERSEA heritage) –Clear, providing useful synthesis for users n The scientific validation of V1 Stream1 is effective, and corresponding to QuARG expectation

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Remarks, considerations, and ongoing work n This huge effort of validation has never been done elsewhere in the ocean community n Scientific validation of reanalysis products is promising: a strong effort is done to provide common validation strategies (Global reanalysis) that might be positively considered by the international community n The validation reporting is not fully organised for V1 and V2, but interest to maintain regular reporting is obvious, and will prepare MYO-2 operational validation

Marine Core Service MyOcean : PRR Stream1 V1 – 29th April 2011 / Roma – QuARG review Link with users n Reporting is established, and could support regular information to users n Request from the User Forum in Stockholm, due to effort in reporting, actions can be commited for: –Estimated accuracy level (september 2011) –QUID (october 2011)