1. Forecast Sensitivity to Observations in ACCESS
Chris Tingwell, Sergei Soldatenko, Jin Lee and Peter Steinle Bureau of Meteorology Research and Development
Acknowledgements: Boris Kelly-Gerreyn, Robert De Maid & Agnes Lane Bureau of Meteorology Observations & Infrastructure
The Met Office
WMO
6th Workshop on the Impact of Various Observing Systems on NWP
Shanghai, China, May, 2016

2. Outline Overview of ACCESS
A quick history of Observation impact studies in ACCESS
Adjoint-based Forecast Sensitivity to Observations in ACCESS
Some preliminary results
Ongoing work, future plans

3. ACCESS Australian Community Climate and Earth System Simulator
Collaboration between the Bureau of Meteorology, the CSIRO, Australian universities, the Met Office and other international partners.
The NWP subset of ACCESS provides the Bureau with its operational NWP capability.
ACCESS NWP is built around the Met Office Unified Model and Observations Processing System (OPS) and 4D-VAR assimilation.

4. ACCESS: APS1  APS2 70 vertical levels Grid size (km) G 40 GE - R 12TC C 4

5. ACCESS: APS1  APS2 70 vertical levels Grid size (km) G 40 25 GE - 6012 TC C 4
1.5
GE: 24 members

6. ACCESS: APS1  APS2 APS1 Surface: synops, ships, buoys
Sondes, wind profilers
Aircraft: AIREPS, AMDARS
Satellite observations (1)
Wind: Scatterometer surface winds (ASCAT), AMVs from GEOS & POES
GNSS-RO: bending angle observations
Satellite observations (1I): IR and MW radiances
Platform
Instrument
NOAA-16
NOAA-17
NOAA-18
NOAA-19
MetOp-A
EOS: Aqua
AMSU-A/B + HIRS
AMSU-B + HIRS
AMSU-A/B + HIRS
AMSU-A/B HIRS
IASI (138 channels)
AIRS (48 channels) (old instrument)

7. ACCESS: APS1  APS2 APS2 Surface: synops, ships, buoys
Sondes, extra wind profilers
Aircraft: AIREPS, AMDARS
Satellite observations (1)
Wind: Scatterometer surface winds (ASCAT), AMVs from GEOS & POES
GNSS-RO: bending angle observations
Satellite observations (1I): IR and MW radiances reduced thinning
Platform
Instrument
NOAA-18
NOAA-19
MetOp-A
MetOp-B
EOS: Aqua
Suomi-NPP
MTSAT-2 / Himawari-8
AMSU-A/B
AMSU-A/B + HIRS
IASI (138 channels)
AIRS (139 channels)
CrIS (134 channels)
ATMS
Clear Sky Radiances / AMVs

8. ACCESS: APS1  APS2 Current operational system is APS1(APS2)
APS = "Australian Parallel Suite” as per Met Office PS "Parallel Suite"
Currently upgrading APS1 to APS2
APS0
PS17
2010
APS1
PS24
2012
APS2
PS32
2016
APS3
PS37
2018
APS0 PS17
APS1 PS24
APS2 PS32
APS3 PS37

9. Global Observation Coverage: APS2 ACCESS-G

10. Summary history of observation impact studies in ACCESS
Most observation impact studies to date have been OSEs
Data denial experiments to test the impact of upper-air network modifications on regional ACCESS-R system (at behest of Bureau observation network planners).
Impact of satellite observations: new instruments, locally processed AMVs, and demonstrations of the impact that all satellite observations have on Southern Hemisphere forecast skill.
(Some of these reported at Sedona.)
Done rarely: expensive and time consuming.

11. Adjoint-based FSO in ACCESS
The use of Met Office VAR in ACCESS makes the Met Office FSO system available to us:
Can run inexpensively - in real-time - downstream of the operational ACCESS-G suite
Provides continuous information about the impact of each assimilated observation on the 24 h forecast error
Has attracted a lot of interest within the Bureau from observation network planners
Data assimilators see it as a potentially valuable tool to assess "health" of the ACCESS assimilation system and guide DA R&D

12. Impact of analysis on forecast error: ∆𝑒 𝑔,𝑓 =𝑒 𝑥 𝑔 , 𝑥 𝑡 −𝑒 𝑥 𝑓 , 𝑥 𝑡
𝑥 𝑡 is a "true" state, estimated
by the analysis at the time of
the forecast
𝑥 𝑓 is the forecast from analysis 𝑥 𝑎
𝑥 𝑔 is the forecast from first-guess
at the time of the analysis
Forecast error norm: 𝑒= 𝑥 𝑓 − 𝑥 𝑡 𝑇 𝐶 𝑥 𝑓 − 𝑥 𝑡
Weighting matrix C :
To produce total moist energy norm
calculated from the surface up to 150 hPa e
𝑥 𝑔
𝑥 𝑏
𝑥 𝑓
𝑒 𝑔
𝑒 𝑓
𝑥 𝑡
𝑥 𝑎
𝑡 0 - 6ℎ𝑟𝑠
𝑡 0
𝑡=24 ℎ𝑟𝑠

13. Data Assimilation 4D-Var
Global NWP system 24 h forecast impact
Moist energy norm up to 150 hPa: global domain
Calculations performed at VAR resolution (N216) on simplified forecast states
Over several weeks, the reduction of forecast error due to observing systems, instruments, groups of observations etc. can be aggregated and compared.
Observations
Analysis
Data Assimilation 4D-Var
Forecast error
T + 24h
model
Forecast T + 24h
model
Forecast T + 30h
Forecast error
T + 30h
Change in forecast error due to each observation
Adjoint VAR
Change in forecast error due to analysis
Adjoint PF model
Change in forecast error

14. First work: APS1
APS1 ACCESS-G
A full APS1 ACCESS-G NWP suite with FSO built-in was set up in 2012 and ran for a month
Interest generated led to a collaboration between Bureau Observations & Infrastructure and Earth Systems Modelling to set and run a real-time APS2 FSO system and develop tools to generate results of interest to stakeholders

15. APS2
APS1 ACCESS-G
APS2 ACCESS-G (early results)
In 2015 the Met Office (APS2-equivalent) FSO Rose suite was adapted to run downstream of the main APS2 ACCESS-G suite

16. APS2: data flow Output post processing, analysis and visualisation
ACCESS-G operational NWP suite
FSO suite
Output post processing, analysis and visualisation
(JSON, python)
Ngamai Bureau of Meteorology Oracle (Melbourne)
(Operational machine)
Raijin
National Computational Infrastructure (NCI) Fujitsu (Canberra)
(R&D – not operational – machine)

17. Global impact per OBS type
IASI (IR)
SONDES
AMSU-A (MW)
CrIS (IR)
Aircraft
Synop
AIRS (IR)
ATMS (MW) BUOYS
NOAA AMVs
ASCAT
MSG AMVs
AMSU-B
PILOT (SONDE)
WIND PROF.
ESA AMVs
JMA AMVs
GNSS-RO
SHIP
HIRS
MTSAT2 IR
00/12Z
only

18. Global impact/observation per OBS type
IASI (IR)
SONDES
AMSU-A (MW)
CrIS (IR)
Aircraft
Synop
AIRS (IR)
ATMS (MW) BUOYS
NOAA AMVs
ASCAT
MSG AMVs
AMSU-B
PILOT (SONDE)
WIND PROF.
ESA AMVs
JMA AMVs
GNSS-RO
SHIP
HIRS
MTSAT2 IR

19. Global impact: Southern Hemisphere observations
IASI (IR)
SONDES
AMSU-A (MW)
CrIS (IR)
Aircraft
Synop
AIRS (IR)
ATMS (MW) BUOYS
NOAA AMVs
ASCAT
MSG AMVs
AMSU-B
PILOT (SONDE)
WIND PROF.
ESA AMVs
JMA AMVs
GNSS-RO
SHIP
HIRS
MTSAT2 IR

20. Global impact: Northern Hemisphere observations
IASI (IR)
SONDES
AMSU-A (MW)
CrIS (IR)
Aircraft
Synop
AIRS (IR)
ATMS (MW) BUOYS
NOAA AMVs
ASCAT
MSG AMVs
AMSU-B
PILOT (SONDE)
WIND PROF.
ESA AMVs
JMA AMVs
GNSS-RO
SHIP
HIRS
MTSAT2 IR

21. Satellite impact: GPS-RO (APS1)

22. Satellite impact: GPS-RO (APS2)
2015 Oct-Dec

23. AUS region radiosonde impacts: previous work
Assessment of impact of radiosonde observations and AMDARs in Regional and Australian ACCESS NWP systems
Observing System Experiments (OSEs) conducted March – June when additional 12UTC radiosonde flights flew at 24 mainland stations (“experiment of opportunity” – timing not ideal).
OSE:
Control: Rerun of operational forecasts using all available observations.
Without 12UTC Radiosondes: Observations from extra 12UTC radiosondes excluded.
Without Australian radiosonde network: Observations from entire Australia radiosonde network excluded.
Without AMDAR network: Observations from the Australian AMDAR network excluded
(Reported at Sedona)

24. AUS region radiosonde impacts

25. AUS region radiosonde impacts
Macquarie Island

26. AUS region radiosonde impact: ranking
Daily Impact of Sonde Observations 𝟏× 𝟏𝟎 𝟑 J/kg
Rank
Station ID
Station name
Impact 1
89611
Casey (Antarctic) 2
89571
Davis (Antarctic) 3
94998
Macquarie Island 4
94120
Darwin 5
94461
Giles Met Office 6
94975
Hobart 7
94299
Willis Island 8
94203
Broom Airport 9
96996
Cocos Island 10
94659
Woomera Aero 11
94326
Alice Spring 12
94995
Lord Howe Island 13
94150
Gove Aero 14
94302
Learmonth Aero 15
94170
Weipa Aero 16
94610
Perth Aero 17
94510
Charleville Aero 18
94672
Adelaide 19
94637
Kalgoorlie-Boulder 20
94638
Esperance 21
94802
Albany Aero 22
94312
Port Hedland Aero 23
94430
Meekatharra Aero 24
94866
Melbourne 25
94527
Moree Aero 26
94294
Townsville 27
94711
Cobar Mo 28
94776
Williamstown 29
94821
Mount Gambier Aero 30
94374
Rockhampton Aero 31
94653
Ceduna 32
94910
Wagga Wagga 33
94767
Sydney

27. AUS region radiosonde average impact

28. AUS region radiosondes: station statistics
Higher ranking stations

29. AUS region radiosondes: station statistics
Lower ranking stations

30. AUS region radiosondes: quartiles

31. TODO (1) Similar analyses for:
(1) Surface network (2) aircraft observations
(3) Wind profilers
(4) Marine networks

32. TODO (2)
AUS station ranking statistics (how many times does station X appear in the top quarter ? Second quarter ? …
Stakeholder engagement:
What do other people want ?
How and where would they like to see it ?
Events/periods of interest ?
How best to use these results to guide network planning ?
Stakeholder education: not the only way to assess the value of observations (e.g. the anchoring role of unbiased observations).

33. TODO (3) Use an AUS-region energy norm in the global suite
Retrospective ACCESS-G runs for periods of particular interest
Regional NWP: ACCESS-R FSO ?
Satellite channel impact analyses
Relative impact of locally received and processed satellite observations (important source of low latency data and input-stream robustness).
Other error metrics ?

34. Summary
We have commenced routine running of an FSO suite in conjunction with the Bureau's operational global ACCESS NWP suite
Development of analysis and visualisation tools is in progress
Bureau stakeholder engagement will inform the way in which the FSO output is analysed and presented