Presentation is loading. Please wait.

Presentation is loading. Please wait.

L. Mentaschi, G. Besio, F. Cassola, A. Mazzino Dipartimento di Ingegneria Civile, Chimica ed Ambientale (DICCA), Università degli studi di Genova.

Similar presentations


Presentation on theme: "L. Mentaschi, G. Besio, F. Cassola, A. Mazzino Dipartimento di Ingegneria Civile, Chimica ed Ambientale (DICCA), Università degli studi di Genova."— Presentation transcript:

1 L. Mentaschi, G. Besio, F. Cassola, A. Mazzino Dipartimento di Ingegneria Civile, Chimica ed Ambientale (DICCA), Università degli studi di Genova

2 Validation of wave model in the Mediterranean sea Model Wavewatch III Ardhuin et al. (2008) source terms. Tolman and Chalikov (1996) source terms, not tuned to the Mediterranean sea conditions. Normalized Bias (NBI) Statistical indicators used for validation Normalized Root Mean Square Error (NRMSE) Correlation coeff. (ρ) Mentaschi et al Why NRMSE is not completely reliable

3 Statistics on 17 storms and 23 buoys ACC350T & C NBI2.1%-11.2% ρ NRMSE (-2.3%) February 1990 storm Mentaschi et al Why NRMSE is not completely reliable

4 In what sense NRMSE is not completely reliable? A numerical example ρ=0.614 for both of the simulations NBI = -12% for the red simulation One would say the best simulation is the blue one. But … ρ=0.614 for both of the simulations NBI = -12% for the red simulation One would say the best simulation is the blue one. But … NRMSE(blue) = NRMSE(red) = (~ -7.2%) NRMSE(blue) = NRMSE(red) = (~ -7.2%) Mentaschi et al Why NRMSE is not completely reliable

5 Geometrical decomposition of NRMSE Bias component Scatter component (sometimes called Scatter Index) where Are SI and BC independent? Mentaschi et al Why NRMSE is not completely reliable

6 Example of set of simulations with constant ratio : In general and are not independent. where is the unbiased simulation. Lets assume this relation: Mentaschi et al Why NRMSE is not completely reliable Holds for amplifications Amplification factor:

7 Relation SI – NBI SI grows linearly in NBI around NBI=0 Mentaschi et al Why NRMSE is not completely reliable

8 Fits well real world data Significant wave height Mean period Mentaschi et al Why NRMSE is not completely reliable

9 The simulation with the minimum value of NRMSE underestimates the average value Mentaschi et al Why NRMSE is not completely reliable

10 How to overcome this problem? Hanna and Heinold (1985) indicator: Demonstration that with ρ constant HH is minimum when bias is null Mentaschi et al Why NRMSE is not completely reliable

11 HH has a minimum for null bias Mentaschi et al Why NRMSE is not completely reliable

12 Wavewatch III validation on the Mediterranean sea. ACC350T & C NBI ρ NRMSE (-2.3%) HH (+4.8%) HH has a minimum for null bias Mentaschi et al Why NRMSE is not completely reliable

13 Future development NRMSE tends to assume smaller values for simulations that underestimate observations, and this is mostly evident when relation holds. HH indicator overcomes this problem introducing a different normalization of the root mean square error. Conclusions Usage of HH indicator to validate Wavewatch III model on the Mediterranean sea. Mentaschi et al Why NRMSE is not completely reliable

14 Thank you! Mentaschi et al Why NRMSE is not completely reliable L. Mentaschi, G. Besio, F. Cassola, A. Mazzino Dipartimento di Ingegneria Civile, Chimica ed Ambientale (DICCA), Università degli studi di Genova


Download ppt "L. Mentaschi, G. Besio, F. Cassola, A. Mazzino Dipartimento di Ingegneria Civile, Chimica ed Ambientale (DICCA), Università degli studi di Genova."

Similar presentations


Ads by Google