1. Reinaldo Garcia, PhD A proposal for testing two-dimensional models to use in the National Flood Insurance Program

2. Presentation Outline Motivation Status quo Why we need improvements What are others doing Suggested components for new model review guidelines

3. Model performance and predictive capability Depends on many factors, including: Model mathematical formulation Numerical method Appropriate use of boundary conditions Adequate configuration of grid or mesh Choice of roughness coefficients and time steps Data quality Expertise of the modeler to perform the simulation. To ensure adequately testing the predictive capabilities of a 2D flood model, testing guidelines should be based on standardized benchmarks and specifications

4. Status Quo must be reviewed, tested and accepted by a government agency FEMA requires that any hydrologic or hydraulic computer model used in support of flood insurance studies must be reviewed, tested and accepted by a government agency responsible for the implementation of programs for flood control and/or regulation of floodplains National Flood Insurance Program (NFIP) regulations. Subparagraph 65.6. relies on other agencies scientifically correct and technically sound. FEMA relies on other agencies to review and test a model and determine whether it is scientifically correct and technically sound. FEMA List: Meeting the Minimum Requirement FEMA List: Numerical Models Meeting the Minimum Requirement of NFIP Current Nationally Accepted Hydraulic Models

6. Why we need improvements FEMA does not establish a standard testing procedure FEMA does not establish a standard testing procedure to determine if a model is scientifically or computationally correct No specifications or metrics No specifications or metrics to verify that the model can provide adequate or accurate information to support NFIP studies. Inconsistent testing methods Inconsistent testing methods depending on the agency or modeler doing the evaluation. no budget or time test a model Federal agencies often have no budget or time test a model that is not on the FEMA list. government agencies also develop their own models… Some government agencies also develop their own models…

7. Status Quo Is not open Is not fully reliable Hampers innovation Does not serve well FEMA’s mission

8. What are others doing?

9. Components of sound model review guidelines Model verification Model validation Standardized benchmarks MetricsOpen

10. Model vs. Real World Calibration Adjusting numerical or physical parameters in the program to improve comparison with data Verification Determining that the program accurately represent the conceptual model (program solves the equations right) Qualification Fidelity of the equations to reality Validation Determining the degree to which the model is an accurate representation of the real world.

11. Verification

12. Example of model verification Comparison with Lynch & Gray 2-D analytical solutions

13. Example of model verification

14. Validation “No one believes model results except the one who performed the calculation,...everyone believes observed or measured data except the one who did the measurements” P. J. Roache

15. Validation

17. UK Benchmarks Provide a dataset against which 2D hydraulic modeling packages can be evaluated Provide evidence to ensure that such models used for flood and coastal risk management are capable of adequately predicting the variables upon which flood risk management decisions are based.

18. UK Benchmarks Flooding of a disconnected water body Assess basic capability to simulate flooding of disconnected water bodies on floodplains or coastal areas and wetting and drying of floodplains Filling of flood plain depressions Tests capability to predict inundation extent and final flood depth flow low momentum flow over complex topographies. Also tests flooding of disconnected water bodies and wetting and drying of floodplains. Predicts inundation extent, with an emphasis on final distribution of water rather than peak levels. Momentum conservation over a small (0.25m high) obstruction Tests capability to simulate flow at relatively low depths over an obstruction with an adverse slope. Speed of flood propagation over an extended flood plain Tests simulation of speed of propagation of flood wave and the prediction of velocities at the leading edge of the advancing flood. Valley floodingTests simulation of major flood inundation at the valley scale. Dam break Tests simulation of shocks and wake zones close to a failing dam. This could also be applied to other examples like rapid opening of sluice gates, failures of embankments, etc. River to floodplain linking Evaluates capability to simulate flood volume transfer between rivers and floodplains using 1D and 2D model linking. Rainfall and sewer surchargeTests capability to simulate shallow flows in urban areas with inputs from rainfall and sewer surcharge.

19. Test relevance for specific applications ApplicationTest Large scale flood risk mapping1 & 2 Catchment Flood Management Plan1, 2 & 7 Flood Risk Assessment and detailed flood mapping 1, 2, 3 & 7 Strategic Flood Risk Assessment1, 2, 3, 4, 7 & 8 Flood hazard mapping1, 2, 3, 4, 7 & 8 Contingency planning for real time flood management 1, 2, 3, 4, 5, 7 & 8 Reservoir flooding mapping1, 2, 3, 4, 5 & 6

20. Benchmark Examples Flooding a disconnected water body

21. UK Benchmark Examples Assess model’s ability to conserve momentum over an obstruction in the topography. Important when simulating flooding in urbanized floodplains.

23. UK Benchmarks verification Should be complemented with verification

24. Americas Society of Civil Engineers ASCE: Verification and Validation of 3D Free-Surface Flow Models, 2008 Three main components: – Analytical solutions – Lab/flume experiments – Field scale real-world applications Provides a data set for 3D models only

25. Suggestions for Model Review Guidelines Verification Verification should precede validation Validation Validation based on set of benchmarks error tolerances Acceptable error tolerances (metrics) Benchmarks intended use: Benchmarks based on intended use: not all models are necessarily adequate for all projects not all models are necessarily adequate for all projects public and freely available Standardized test and benchmark data sets should be public and freely available Implementing these model review guidelines would better serve FEMA’s mission.

26. Thank you / Questions rey@hydronia.com