1. © Crown copyright Met Office Scientific background and content of new gridded products Bob Lunnon, Aviation Outcomes Manager, Met Office WAFS Workshop on Gridded SIGWX forecasts, Paris, 14-15/9/2009

2. © Crown copyright Met Office Summary of presentation Reminder of content of new products Background on Numerical Weather Prediction models used in Washington and Exeter Scientific Background of new products CAT In-cloud turbulence Icing Cbs Background to verification Use of products (illustrated by CAT) Issues Provision of maxima Provision of objects Use of common interim grid Work still to be done

3. © Crown copyright Met Office Content of new products (all T+6 to T+36 at 3 hour increments) Range of pressures Increment of pressure Range of Flight Levels Icing (mean/max) 800-300hPa100hPa060-300 In cloud turbulence (mean/max) 700-300hPa100hPa100-300 CAT (mean/max) 400-150hPa50hPa240-450 Cb Horizontal Extent Pressure of base and top coded separately N/AFLs of base and top derived from pressures

4. © Crown copyright Met Office Comparison of NWP prediction schemes used at two WAFCs CharacteristicWashingtonExeter FormulationHydrostaticNon-hydrostatic Data AssimilationSpectral Statistical Interpolation 4-dimensional variational assimilation Horizontal Coordinate System SpectralGrid point Horizontal resolution 35km for physics40 km Vertical coordinateHybrid sigma/pressure Hybrid height coordinate Vertical levels6450 Characteristics affecting hazard products VariousOften different to Washington (to be discussed later)

5. © Crown copyright Met Office Causes of CAT Vertical wind shear – low Richardson Number Mountain waves Convection All three causes contribute roughly equally to CAT occurrences, globally In principle the products from both WAFCs should reflect all causes

6. © Crown copyright Met Office Example of representation of recent global CAT forecast

7. © Crown copyright Met Office Prediction of shear-induced CAT Prediction of shear induced CAT has longer history than either mountain wave induced CAT or Convectively Induced CAT Both centres are currently using Ellrod algorithm TI1 (Ellrod and Knapp, 1992) This predictor is the scalar product of deformation and vertical wind shear There is interest in more recent algorithms, e.g. Ellrod (2008?), Knox McCann and Williams (2008), use of ensembles. Met Office will be pursuing this in 2010-11 However more sophisticated algorithms can amplify errors in non-linear, differentiated quantities so it is likely that Ellrod will out-perform other algorithms when verified globally

8. © Crown copyright Met Office Prediction of mountain wave CAT WAFC London has been generating automated forecasts of mountain wave CAT since the late 1990s They are included in the current GRIB product There is an issue with scaling of mountain wave CAT – it is currently too high – we know how to rescale it – we will do this before GRIB2 is introduced UK algorithm is based on gravity wave drag scheme (Gregory, Shutts and Mitchell, 1998) We understand that GFS model uses Kim and Arakawa (1995) mountain wave scheme, and we believe that it is possible to use this scheme to predict mountain wave CAT Comparisons between the two centres gravity wave drag scheme are under way

9. © Crown copyright Met Office Prediction of Convectively induced CAT WAFC London is conducting preliminary experiments on the prediction of convectively induced turbulence

10. © Crown copyright Met Office Prediction of in-cloud turbulence Both centres use vertical gradient of equivalent potential temperature (a measure of how unstable a cloud is) No verification has been done so far on this product

11. © Crown copyright Met Office Target Weather Object "Cb" Cb top volumes: convective turbulence, lightning Cb bottom volumes: hail, icing, lightning, heavy rain, wind shear, turbulence FLYSAFE Thunderstorm (Cb) hazard object

12. © Crown copyright Met Office Example of representation of recent global Cb forecast

13. © Crown copyright Met Office Prediction of Cbs by Numerical Weather Prediction Models – convective parameterisation schemes I If a NWP model is run with no convective parameterisation scheme, convective instability can occur at individual grid boxes giving rise to grid-box storms which are unrealistic and can cause the model to fail due to computational instability Convective parameterisation schemes redistribute energy (in various forms) and moisture in the vertical so that Model is stable Vertical distribution of these variables approximately reflects how they might be distributed in the real atmosphere after one or more Cb have occurred.

14. © Crown copyright Met Office Prediction of Cbs by Numerical Weather Prediction Models – convective parameterisation schemes II Convective parameterisation schemes diagnose many atmospheric variables – indeed some diagnose turbulent kinetic energy (TKE), which might appear to be an appropriate predictor of severity of hazard However, TKE is an internal diagnostic and it is unlikely that two schemes will generate consistent diagnoses At the February 2007 WAFS coordination meeting in Boulder significant differences between Cb products were noted In contrast, convective rainfall rate is used (a) to predict expected rainfall, and (b) to diagnose latent heating, so there is likely to be consistency between different models After the February 2007 meeting, it was agreed to base prediction of Cb severity on convective rainfall rate

15. © Crown copyright Met Office Prediction of Cbs by Numerical Weather Prediction Models – convective parameterisation schemes III Convective parameterisation schemes also diagnose base and top of convection Hong Kong Observatory highlighted difference in cloud top diagnosed by the two WAFCs in July 2008 The UK convective parameterisation scheme was adjusted in late 2008 to correct this difference More work needed in this area

16. © Crown copyright Met Office Icing – information available from models for prediction Temperature Cloud top temperature Cloud cover (layer cloud) Cloud cover (convective cloud) Vertical velocity Liquid water content (non-zero if temperature less than zero for UK only)

17. © Crown copyright Met Office Example of representation of recent icing forecast – FL100

18. © Crown copyright Met Office Example of representation of recent icing forecast – FL240

19. © Crown copyright Met Office Different approaches to diagnosis of icing potential in grid box UK Cloud fraction in temperature range 0 to -20 has shown considerable skill Predictor is a cloud and temperature mask combined with relative humidity US Generate interest maps in temperature, cloud cover, cloud top temperature and vertical velocity Interest map says whether, given value of individual parameter, icing is likely (can be negative) Icing potential is a non-linear function of the interest maps

20. © Crown copyright Met Office Different approaches to deriving mean and maximum icing UK Calculate icing potential for box with 100hPa vertical depth and 40km*40km horizontal extent Derive mean and maximum for all such boxes within dissemination grid box (140km*140km) US Calculate icing potential for box with ~25hPa vertical depth and 35km*35km horizontal extent Derive mean and maximum for all such boxes within dissemination grid box (140km*140km*100hPa) It has been shown that difference between mean and max for two WAFCs is at least partly due to this difference in processing

21. © Crown copyright Met Office Strategy for verification of GRIB hazard products UK verifies forecasts of CAT and Cb from both World Area Forecast Centres (both manual and automated products) CAT verified using GADS data (from BA – not available in US) Cb verified using data from UK Sferics system (not available in US) US verifies forecasts of icing from both WAFCs (both manual and automated products) Icing verified largely using CloudSat data (not readily available in UK) Use Relative Operating Characteristic curves, together with statistics used to generate them

22. © Crown copyright Met Office Use of verification data to address initial concerns from users Objective verification will enable us to ensure that the two WAFCs adopt consistent intensity measures (this has not yet been done) Objective verification measures will be made available routinely which will enable users to make best use of the gridded forecast Objective verification of in-cloud turbulence (yet to be performed) will enable us to address high latitude bias Objective verification of icing will enable us to address high latitude bias Objective verification of CAT has enabled us to develop a consistent scaling of shear induced turbulence and mountain wave induced turbulence (not yet implemented) Objective verification of Cb has led us to remove smearing function (last week – the only change that has been possible for WAFC London in the last 6 months)

23. © Crown copyright Met Office Background to verification of products Imagine for now that a user of a GRIB forecast is using a single threshold of, for example, CAT probability so that the world is divided into no- go areas where this threshold is exceeded and go areas where it isnt. We describe this kind of forecast as categorical Imagine further that the data we are using to verify the forecast are also yes/no (no shades of grey) e.g. a report of moderate or severe turbulence, a report from a lightning detection system Then a simple contingency table can be constructed

24. © Crown copyright Met Office Contingency tables a = number of hits b = number of false alarms c = number of misses d = number of correct rejections a+c = number of events b+d = number of non-events Hit rate = a/(a+c) False alarm rate = b/(b+d)

25. © Crown copyright Met Office Contingency tables (actually frequency tables) for different user thresholds CAT threshold of 3% CAT threshold of 6% For the WAFC GRIB forecasts thresholds are used on each forecast to determine if it is a yes or no forecast. A new contingency table is created for each threshold used and for each resulting table the hit rate and false alarm rate are calculated.

26. © Crown copyright Met Office Distribution of events and non events The distribution of events and non-events can be plotted –one point for each threshold. A good forecast should be able to discriminate between the two distributions.

27. © Crown copyright Met Office Relative operating characteristic (ROC) Plot hit rate against false alarm rate by varying the threshold on the forecasts – one point for each threshold. Diagonal represents line of no skill Points nearest the top left corner have greatest skill Area under ROC curve is a measure of skill (A=1 perfect forecast, A=0.5 no skill) For rare events points cluster towards lower left corner – hard to determine ROC area properly. The curve from a continuous forecast (eg GRIB) and single point from a categorical forecast (eg BUFR SIGWX) can be compared.

28. © Crown copyright Met Office Example of perfect forecast ROC area measures the ability of the forecast to discriminate between events and non-events. Perfect forecast – area under ROC curve = 1

29. © Crown copyright Met Office Example of no skill forecast Area under ROC curve = 0.5

30. © Crown copyright Met Office Example of good forecast Area under ROC curve = 0.8

31. © Crown copyright Met Office ROC curve – choosing thresholds ROC curve is shown for various thresholds t1 to t5 Also shown are hit rate and false alarm rate for SIGWX chart (indicated by S) The use of threshold t2 will result in same false alarm rate as SIGWX chart (better hit rate) The use of threshold t4 will result in same hit rate as SIGWX chart (better false alarm rate)

32. © Crown copyright Met Office Other verification statistics ROC area gives a single useful measure of skill Other scores should also be considered together with ROC area Contingency tables enable many other scores to be calculated Heidke Skill Score, True Skill Statistic, Equitable threat score We will provide statistics which tell the end user what are the characteristics of the atmosphere as a function of the forecast value of hazard expectation For example, we can provide statistics which indicate, for an aircraft flying for a protracted period of time in areas where the forecast CAT is between two thresholds, how many CAT encounters will occur

33. © Crown copyright Met Office Use of products (illustrated by CAT) A longer version of this part of the presentation was given at the WAFS Science meeting in Washington in April Since then Lido/Lufthansa have done some work in collaboration with the UK Met Office on costs of avoiding areas of forecast CAT General comments about how a flight planning company could use the gridded forecasts still pertain

34. © Crown copyright Met Office Flying from A to B through area of forecast high frequency of CAT A B Black in centre of plot indicates area of high frequency of CAT Dark grey in plot indicates area of moderate frequency of CAT Light grey in plot indicates area of low frequency of CAT This graphic could be either a map or a cross-section (later we will assume it is a map)

35. © Crown copyright Met Office Consider direct route from A to B Graph at bottom of plot indicates expected frequency of CAT as aircraft flies directly from A to B A B

36. © Crown copyright Met Office Consider route chosen to maximise CAT avoidance Graph at bottom of plot indicates expected frequency of CAT as aircraft flies from A to B along indicated route

37. © Crown copyright Met Office Compare routes Information in graphs should be generated by flight planning companies and fed to airlines/pilots. WAFCs can provide statistics which enable a user to work out how much CAT they will encounter

38. © Crown copyright Met Office Three issues Provision of maxima in GRIB hazard products Provision of objects representing hazardous areas Use of common interim grid by WAFCs

39. © Crown copyright Met Office Issue #1 – Provision of maxima It will be recalled that for CAT, in-cloud turbulence and icing we generate mean and maximum values for each dissemination grid box The inclusion of two values reflects the fact that, in general, model grid boxes are much smaller than dissemination grid boxes and we can give an indication of variability within the dissemination grid box The inclusion of two values, does not, as far as I know, reflect a strongly expressed requirement from the end users Do we really need both mean and maximum? I recommend retaining both in the data (because removing them and then reinstating them later will prove administratively irksome) but focussing efforts (by both suppliers and users) on the mean fields

40. © Crown copyright Met Office Issue # 2- Provision of objects For continuous fields (e.g. wind, temperature) a regular grid of data is the best way to present the information For discontinuous fields such as CAT, icing, Cb, where there tend to be small, relatively isolated hazardous volumes, representation of the hazardous volumes as objects is appealing In the European project FLYSAFE, we generated objects representing CAT, icing, Cb, and uplinked the information to a test aircraft for display to the flight crew (we also displayed the objects in a flight simulator) We have sent CAT objects to Lufthansa/LIDO for them to calculate a trajectory avoiding the hazardous area Are objects the way to go? They would be generated in addition to gridded data but they could be instead of the high at-a-glance value charts and/or the single field charts.

41. © Crown copyright Met Office Issue # 3 – use of common interim grid Models have finer resolution grids than the GRIB dissemination grid The two WAFCs employ different processing to go from the model grid to the dissemination grid It has been shown that this is at least partly responsible for some of the differences between the products It has been suggested that there should be a common interim grid employed by both WAFCs as an interim step between the model grids (which will differ between the two centres) and the dissemination grid The interim common grid would help reduce the tendency for changes in products when model resolution is changed The use of a common interim grid would probably lead to greater consistency between the two centres products, but possibly at the expense of the absolute accuracy of the products How important is consistency relative to absolute accuracy?

42. © Crown copyright Met Office Work planned for FY 10-11 Following slides give proposed and approved work for FY 10-11 Cb CAT

43. © Crown copyright Met Office Cb Proposal – I To develop a prototype scheme to correct Cb forecasts for spatial variability and temporal duration. The scheme should be based upon climatology of observed Cb events. A prototype is necessary to assess the most suitable observations to build a global climatology; to determine the most appropriate atmospheric state parameters to include in the modification scheme; to determine the schemes properties and how it should modify the forecast; to afford evaluation of the scheme and its effects on the forecast, validated against existing observations; to facilitate requirements capture for a routine production environment to determine the benefits from the correct use of the product

44. © Crown copyright Met Office Cb Proposal - II First stage development: A climatology scheme would need at least five years of observation data so that monthly and diurnal behaviour could be captured. Initially to consider only spatial variability thus summer seasons (Jun Jul Aug) since convective events that occur tend to be short and intense. Use satellite based observations to build the climatology, e.g., Tropical Rainfall Monitoring Mission, Meteosat Use corresponding forecast periods to capture model climatology – one period to be used for validation of the correction scheme. Initially consider a simple bias correction scheme. Document requirements for a production environment. Report back to model developers on which atmospheric state parameters require correction. Assess outcomes to decide whether to develop further the methodology.

45. © Crown copyright Met Office Cb Proposal - III Second stage development – dependent on assessment of first stage: Develop further the bias correction scheme for the remaining seasons. Investigate the benefits of a correction scheme based on a diurnal climatology. Develop an alternative or augment the correction scheme, e.g., investigate using a Kalman Filter to afford dynamic adjustment of correction scheme based on latest climatology. Develop a process to build and maintain a global climatology to cover a rolling ten year period. Document requirements for a production environment. (All development should occur within an architecture for a production environment, including system lifecycle management.) Provide a facility for impact analysis on Cb forecasts for changes in the NWP model. (Submit papers describing the correction to journals – a means for an independent review of the developments.)

46. © Crown copyright Met Office CAT proposal - I Ellrod Divergence-modified Turbulence Index (DTI) Developed by Gary Ellrod and John Knox in 2008. Simplified divergence trend term to modify the original deformation and vertical wind shear Ellrod indicator. Significant improvement in trials over US DTI = DVSI + DVT Where DVSI = Def x VWS and DVT = C[(du/dx + dv/dy)h2 – (du/dx + dv/dy)h1] h1 and h2 represent different forecast times

47. © Crown copyright Met Office CAT proposal - II Lighthill-Ford theory Different approach to turbulence forecasting Uses Lighthill-Ford theory of Spontaneous imbalance Developed by McCann, Knox and Williams in 2007 Trials over the US look very promising

48. © Crown copyright Met Office CAT proposal - III Convective turbulence Current WAFC turbulence forecasts warn of shear or mountain-wave induced turbulence only. In a 3-month global study approx 35% of turbulence incidents had convective activity as a major factor. In 65% of incidents convection is likely to play some part. Published studies quote figures between 55% and 60%. Objective verification of WAFC forecasts show relatively poor performance in the tropics where convection is a major factor.

49. © Crown copyright Met Office CAT proposal - IV Mountain wave turbulence Review choice of mountain wave physical tuning parameters. Obtain full benefit from vertical resolution upgrades. Objective verification against aircraft observations. Use results to advise WAFC Washington on developing an equivalent scheme for their model.

50. © Crown copyright Met Office Questions & answers

51. © Crown copyright Met Office Costs of CAT encounters and CAT avoidance If fly round light grey area, cost of CAT encounters is low, cost of CAT avoidance is high If fly round black area, cost of CAT encounters is high, cost of CAT avoidance is low It should be possible to identify CAT avoidance strategy which minimises total cost

52. © Crown copyright Met Office CAT avoidance Note that a very limited number of flights were used to generate this figure

53. © Crown copyright Met Office Information needed to generate cost-benefit graph Information on extra distance flown for each avoidance strategy (from flight planning company) Information on cost of extra distance flown (from airlines) Information on frequency of actual CAT encounters for each avoidance strategy (from WAFC)(depends on accuracy of forecasts) Information on cost of CAT encounters (from airlines)

54. © Crown copyright Met Office Figures used to generate cost-benefit graph Study undertaken in 1990s Used Met Office Optimum route package to calculate extra distance flown, and extra time to fly additional distance Used then price of fuel to derive a cost Used verification statistics to derive frequency of CAT encounters Used figures from Tom Fahey for cost of CAT encounter Avoiding areas of high (>6%) CAT probability was financially better than no CAT avoidance Our forecasts have improved since then!

55. © Crown copyright Met Office Use of hazard data in airline flight planning systems For any route under consideration calculate time to fly, conventional cost, fuel, as at present For any route, calculate average forecast CAT frequency Using data from WAFC, calculate expected frequency of CAT encounters for all prospective routes Derive cost of CAT encounters for all prospective routes

56. © Crown copyright Met Office Optimum routes westbound for 10/12/2008, from 5/12/2008

57. © Crown copyright Met Office FLYSAFE – Flight Simulator demonstration Following two slides show typical output on Navigation Display with output from WIMS in addition to information about other aircraft In the simulation, the on-board computers generated modified flight plan which avoided the hazardous areas identified by the WIMS All the pilot had to do was accept the modified flight plan and the aircraft would fly round the hazardous areas

58. © Crown copyright Met Office Navigation Display/Vertical Situation Display with WXR and WIMS data, ADS-B traffic

59. © Crown copyright Met Office As previous slide with Strategic Data Consolidation weather conflict and no-go arc

60. © Crown copyright Met Office Forecast icing and measured Liquid Water Content

61. © Crown copyright Met Office Use forecast of CAT as an example for other hazards We have more experience in forecasting CAT than other hazards, and in verifying our forecasts We have estimates of the cost of a CAT encounter We have done calculations on the total cost of a flight, including cost of CAT encounters Approach can be extended to other hazards (Cb, icing) if costs are known

62. © Crown copyright Met Office Possible strategies for avoiding areas where forecast CAT is above some threshold A B Can in principle choose routes which tangentially touch areas where CAT frequency is above some threshold (with Met Office optimum route system can do this)

63. © Crown copyright Met Office Most flight planning systems consider a network of fixed routes A B Here we show only routes which realistically might be chosen when flying from A to B (Network of routes may be different scale to CAT areas)