1. Evaluation of CONCEPTS Global and Regional Sea Ice Forecasts
Greg Smith and the CONCEPTS Team
Meteorological Research Division, RPN-E
AOMIP Workshop, Woods Hole, USA
October 23-26, 2012

2. Hierarchy of CONCEPTS Systems
Coupled Gulf of St. Lawrence Atmosphere-Ice- Ocean Forecasting System
Daily coupled 48hr forecasts
Resolution of 15km (atm) and 5km (ice-ocean)
Global Ice-Ocean Prediction System (GIOPS)
Weekly 1/4° resolution global ice-ocean analyses and 10 day forecasts
Regional Ice Prediction System (RIPS)
3DVAR ice analysis system providing 4 analyses/day
North American domain on 5km grid
48hr forecasts using CICE sea ice model

3. Gulf of St. Lawrence Coupled Atm-Ice-Ocean Forecasting System
Coupled – Uncoupled differences
Operational since June 2011
48 forecast daily at 00Z
Coupled system:
Atm: GEM (10km)
Ice: CICE (5km)
Ocean: MoGSL (5km)
Mar. 10, 2012
Sea Ice
2m Temperature
Jul. 10, 2012
10m Winds
2m Temperature
15km, 58 levels

4. CONCEPTS Global Ice-Ocean Prediction System
Mercator Ocean Assimilation System (SAM2-PSY3V2):
Sea surface temperature
CMCSST
Sub-surface T&S
from Argo, CTDs, XBTs, Moorings
Sea level anomaly from satellite altimeters
J1,J2,EN with Rio05 MDT
CMC Ice analysis:
SSM/I, CIS charts, Radarsat
Blended ice-ocean analysis and NEMO 10day forecast produced every Wednesday
since Dec
Model configuration:
ORCA025 (~1/4°), <15km in Arctic
NEMOv3.1, LIM2-EVP

5. Regional Ice Prediction System
Daily 48hr forecasts:
CICEv4.1 forced by CMC RDPS
5km North American grid
3DVar-FGAT Sea Ice Analysis:
Four analyses per day of ice concentration at 5 km resolution
Currently assimilates:
SSM/I, AMSR-E, CIS daily charts, Radarsat image analyses
Work in progress to add:
SSMIS, scatterometer, visible-infrared, SAR and ice thickness satellite-based observations
SSM/I
AMSR-E
CIS Chart
Radarsat

6. Evaluation against ice analyses
Compare forecasts to analyses as function of lead time(t)
Apply two criteria:
Remove points of low confidence
Mask out regions where “Days Since Last Obs” > 0.5
Focus on ice edge
Only include points where: | Conc(t) – Conc(0) | > 0.1
Compare forecast to persistence of initial analysis
equivalent to a forecast of “no change”
Perform evaluation for GIOPS and RIPS for :
North American domain (RIPS analysis as “truth”)
NH domain (CMC analysis as “truth”)

7. Evaluation against RIPS: lead 24h
North American domain
RIPS beats persistence throughout 2010
GIOPS beats CMC-pers throughout year
Large differences in spring melt season
RMS
Bias

8. Evaluation against RIPS: lead 48h
North American domain
RIPS beats persistence throughout 2010
GIOPS beats CMC-pers throughout year
Large differences in spring melt season
RMS
Bias

9. Evaluation against RIPS: lead 120h
North American domain
GIOPS shows much reduced RMS throughout year
GIOPS bias also smaller than CMC persistence
Esp. during fall
Large differences in spring melt season
RMS
Bias

10. Evaluation against RIPS: lead 240h
North American domain
GIOPS shows much reduced RMS throughout year
GIOPS bias also smaller than CMC persistence
Esp. during fall
Large differences in spring melt season
RMS
Bias

11. RMS errors vs CMC : Lead 120hr
Weekly forecasts from 2010
GIOPS shows smaller errors than persistence over most of the NH
Largest errors in :
E. Greenland Current
Barents Sea
Labrador Sea
GIOPS
Persistence

12. Evaluation against IMS analyses
Interactive Multisensor Snow and Ice Mapping System (NOAA-NIC)
Daily Northern Hemisphere ice analyses on 4km grid (ice/water)
Assimilates : AVHRR, GOES, SSM/I
Evaluation Methodology:
Interpolate model forecasts to IMS grid
Calculate contingency table values using 0.4 ice concentration cutoff
Bin results on 1° lat-lon grid
Compare to persistence
IMS Ice
IMS No ice
Forecast Ice
Hit ice
False Alarm
Forecast No ice
Miss
Hit water

13. Accumulated statistics for 5 day lead Jan-Dec, 2011 (52 forecasts)
Ice?
YIMS
NIMS
YFcst
Hit ice
False Alarm
NFcst
Miss
Hit water
Persistence of ice analysis
Model forecasts
Misses
Here we compare of 5 day model forecast with the persistence of CMC ice analyses
Note that we use CMC analysis as initial condition, so difference between CMC and model is due to evolution of ice fields in model
Images show accumulated statistics of Wednesday forecasts from Jan-Oct, 2011 (43 forecasts)
Colour scale shows number of occurences. Values not particularly important, but warm colours show where errors occur most often. Note also that the colour scales are the same for persistence and model forecasts. Values near zero are shown in white.
Misses: Model able to forecast ice edge (formation and/or advection) with some skill. In particular: Lab Sea, GIN seas, Barents, Bering.
False Alarms: Model shows increased false alarms in many or the places where misses were reduced. Likely due to biases in atmospheric forcing as well as over-formation (weak SST constraint in SAM2 near ice edge and SST not affected by ice assimilation) and excessive offshore advection of ice (due perhaps to lack of land-fasting? Or excessive wind stress? TBD)
False Alarms

14. Accumulated statistics for 5 day lead Jan-Dec, 2011 (52 forecasts)
Ice?
YIMS
NIMS
YFcst
Hit ice
False Alarm
NFcst
Miss
Hit water
ΔPCI ( Fcst – Pers)
Proportion Correct Ice
PCI= Hits ice / (Hits + Misses)
Skillful forecasts along ice edge
Some errors near coast
Proportion Correct Water
PCW= Hits water / (Hits water + False Alarms)
Small errors along ice edge
Correct lead formation near coast
ΔPCW ( Fcst – Pers)
PCI: Proportion Correct Ice = Correct Ice / ( Correct Ice + Misses)
Varies from 0-1 (1=perfect)
PCW: Proportion Correct Water = Correct No ice / ( Correct no ice + False Alarms )
Red: Forecast better, Blue: Forecast worse
PCI: Can see that model forecasts are better for most ice edges, with errors mostly along coastlines (error from landfast ice or wind stress?)
PCW: Again, errors along ice edge (over formation or over advection?)

15. 10 day forecast for 2011-02-05 Persistence of ice analysis
YIMS
NIMS
YFcst
Hit ice
False Alarm
NFcst
Miss
Hit water
10 day forecast for
Persistence of ice analysis
Model Forecast
Misses
Colours show number of Misses (top) and False Alarms (bottom) for forecast valid Feb 5, 2011.
Red contours show rough position of ice edge (ie “Hit Ice”)
Persistance misses large formation event along ice edge in Lab Sea and Baffin Bay right down to Newfoundland
Forecast captures well the formation event
Forecast overestimates formation in Baffin Bay and near Hudson Strait (and E. Greenland)
Interesting detail: Along Lab Coast forecast shows misses and false alarms. Misses mostly right at coast, whereas false alarms are slightly offshore: this points to either excessive wind stress or lack of landfasting as a possible issue.

16. 10 day forecast for 2011-02-05 Persistence of ice analysis
YIMS
NIMS
YFcst
Hit ice
False Alarm
NFcst
Miss
Hit water
10 day forecast for
Persistence of ice analysis
Model Forecast
Misses
Colours show number of Misses (top) and False Alarms (bottom) for forecast valid Feb 5, 2011.
Red contours show rough position of ice edge (ie “Hit Ice”)
Persistance misses large formation event along ice edge in Lab Sea and Baffin Bay right down to Newfoundland
Forecast captures well the formation event
Forecast overestimates formation in Baffin Bay and near Hudson Strait (and E. Greenland)
Interesting detail: Along Lab Coast forecast shows misses and false alarms. Misses mostly right at coast, whereas false alarms are slightly offshore: this points to either excessive wind stress or lack of landfasting as a possible issue.
False Alarms

17. Accumulated statistics for 5 day lead Jan-Dec, 2011 (52 forecasts)
Ice?
YIMS
NIMS
YFcst
Hit ice
False Alarm
NFcst
Miss
Hit water
ΔPCI ( Fcst – Pers)
Proportion Correct Ice
PCI= Hits ice / (Hits + Misses)
Skillful forecasts along ice edge
Some errors near coast
Proportion Correct Water
PCW= Hits water / (Hits water + False Alarms)
Small errors along ice edge
Correct lead formation near coast
ΔPCW ( Fcst – Pers)
PCI: Proportion Correct Ice = Correct Ice / ( Correct Ice + Misses)
Varies from 0-1 (1=perfect)
PCW: Proportion Correct Water = Correct No ice / ( Correct no ice + False Alarms )
Red: Forecast better, Blue: Forecast worse
PCI: Can see that model forecasts are better for most ice edges, with errors mostly along coastlines (error from landfast ice or wind stress?)
PCW: Again, errors along ice edge (over formation or over advection?)

18. Summary Two models, two analyses, two evaluation methods :
Results highlight similarities and differences between systems
Cause of ice edge errors :
Sensitivity to P*
Wave-ice interactions
Landfast ice
Atm-ice and ice-ocean drag
Need for atmosphere coupling
Model is slow to melt in spring and slow to form ice in fall
Could a short-range forecasting intercomparison be a useful addition to AOMIP?
E.g. Philips (2004), Rodwell (2007)

19. Thank you! Main contributors & collaborators Francois Roy, CMDN
Matteusz Reszka, CMDA
Jean-Marc Belanger, RPN-E
Frederic Dupont, CMDN
Gilles Garric, Mercator-Ocean
Jean-Francois Lemieux, RPN-E
Christiane Beaudoin, RPN-E
Fraser Davidson, DFO
Youyu Lu, DFO
Hal Ritchie, RPN-E
Mark Buehner, ARMA
Alain Caya, ARMA
Tom Carrieres, CIS
Zhongjie He, RPN-E
Simon Higginson, DFO …
Model error for 2010 – 120hr lead

20. Extras…

21. Summary
Gulf of St. Lawrence coupled atmosphere-ice-ocean forecasting system has demonstrated potential benefits of coupling to polar prediction
Through CONCEPTS EC-DFO-DND collaboration, global and regional coupled atmosphere-ice-ocean forecasting systems under development
Over the next 4 years, EC will be developing an integrated marine prediction system (atmosphere, snow, ice, ocean, waves) to provide high-resolution analyses and forecasts of marine conditions over these new METAREAS.

22. Plans for Global Ice-Ocean Prediction System
Continue routine analysis production and evaluate analyses and forecasts
With bias correction to SAT and SW/LW radiation
Possible transfer of system to operations in 2012
Further Development:
Combine SAM2 with 3DVAR ice analysis system
Improve feedback between ice and ocean analyses (e.g. SST in MIZ)
Daily analyses
Coupled experiments

23. EC’s involvement in METAREAs
Development of an integrated marine Arctic prediction system in support of METAREA monitoring and warnings.
Produce short-term marine forecasts using a regional high-resolution coupled multi-component modelling and data assimilation system
Atm, sea ice, ocean, snow, wave
Improved Arctic monitoring

24. Summary and Plans for Regional Ice Prediction
Demonstrated skill of ice-only forecasting model
Verification remains a challenge…
Strong sensitivity to:
Analyzed SST in MIZ
Lack of fast ice
Ongoing efforts:
Coupling with ocean
NEMO, 3-8km configuration
Coupling with atmosphere
GEM, 10km
GEM RDPS
10km

25. Case study: Oct. 7, 2010 (33h lead)
Ice formation in Beaufort Sea
Forecast with dynamics only shows some improvement over persistence
but misses formation
Forecast with full model captures about half of ice formation
Accurate initial SST near ice edge critical
33h Forecast
Fcst(dyn)-Radarsat
Fcst(full)-Radarsat
Pers-Radarsat

26. METAREA Ice Forecasting System
C. Beaudoin, J.-F. Lemieux
Results show significant forecast skill in ice concentration
Equivalent contribution from currents and wind forcing on Labrador Coast
Evaluating separate contribution from dynamics and thermodynamics
Planned operational implementation for 2012
Development of coupled atmosphere-ice-ocean configuration underway
Verification against Radarsat (24hr lead)
Lab Coast – May10
Persistence Forecast (no currents) Forecast (wt currents)
East Arctic – Jan10
Persistence Forecast (dyn only) Forecast (wt thermo)
Pers Fcst-dyn only Fcst wt thermo