Presentation is loading. Please wait.

Presentation is loading. Please wait.

Applied Hydrology RSLAB-NTU Lab for Remote Sensing Hydrology and Spatial Modeling 1 Frequency Analysis Professor Ke-Sheng Cheng Dept. of Bioenvironmental.

Similar presentations


Presentation on theme: "Applied Hydrology RSLAB-NTU Lab for Remote Sensing Hydrology and Spatial Modeling 1 Frequency Analysis Professor Ke-Sheng Cheng Dept. of Bioenvironmental."— Presentation transcript:

1 Applied Hydrology RSLAB-NTU Lab for Remote Sensing Hydrology and Spatial Modeling 1 Frequency Analysis Professor Ke-Sheng Cheng Dept. of Bioenvironmental Systems Engineering National Taiwan University

2 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 2 General interpretation of hydrological frequency analysis Hydrological frequency analysis is the work of determining the magnitude of hydrological variables that corresponds to a given probability of exceedance. Frequency analysis can be conducted for many hydrological variables including floods, rainfalls, and droughts. The work can be better perceived by treating the interested variable as a random variable.

3 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 3 Let X represent the hydrological (random) variable under investigation. A value x c associating to some event is chosen such that if X assumes a value exceeding x c the event is said to occur. Every time when a random experiment (or a trial) is conducted the event may or may not occur. We are interested in the number of Bernoulli trials in which the first success occur. This can be described by the geometric distribution.

4 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 4 Geometric distribution Geometric distribution represents the probability of obtaining the first success in x independent and identical Bernoulli trials.

5 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 5

6 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 6 Average number of trials to achieve the first success. Recurrence interval vs return period

7 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 7

8 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 8 The frequency factor equation

9 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 9

10 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 10 It is apparent that calculation of involves determining the type of distribution for X and estimation of its mean and standard deviation. The former can be done by GOF test and the latter is accomplished by parametric point estimation. 1.Collecting required data. 2.Determining appropriate distribution. 3.Estimating the mean and standard deviation. 4.Calculating x T using the general eq.

11 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 11 Data series used for frequency analysis Complete duration series A complete duration series consists of all the observed data. Partial duration series A partial duration series is a series of data which are selected so that their magnitude is greater than a predefined base value. If the base value is selected so that the number of values in the series is equal to the number of years of the record, the series is called anannual exceedance series.

12 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 12 Extreme value series An extreme value series is a data series that includes the largest or smallest values occurring in each of the equally-long time intervals of the record. If the time interval is taken as one year and the largest values are used, then we have anannual maximum series. Data independency Why is it important?

13 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 13 Techniques for goodness-of-fit test A good reference for detailed discussion about GOF test is: Goodness-of-fit Techniques. Edited by R.B. DAgostino and M.A. Stephens, Probability plotting Chi-square test Kolmogorov-Smirnov Test Moment-ratios diagram method L-moments based GOF tests

14 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 14 Rainfall frequency analysis Consider event total rainfall at a location. What is a storm event? Parameters related to partition of storm events Minimum inter-event-time A threshold value for rainfall depth

15 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 15

16 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 16 Total depths of storm events Total rainfall depth of a storm event varies with its storm duration. [A bivariate distribution for (D, tr).] For a given storm duration tr, the total depth D(tr) is considered as a random variable and its magnitudes corresponding to specific exceedance probabilities are estimated. [Conditional distribution] In general,

17 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 17 Probabilistic Interpretation of the Design Storm Depth

18 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 18 Random Sample For Estimation of Design Storm Depth The design storm depth of a specified duration with return period T is the value of D(tr) with the probability of exceedance equals /T. Estimation of the design storm depth requires collecting a random sample of size n, i.e., {x 1, x 2, …, x n }. A random sample is a collection of independently observed and identically distributed (IID) data.

19 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 19 Annual Maximum Series Data in an annual maximum series are considered IID and therefore form a random sample. For a given design duration tr, we continuously move a window of size tr along the time axis and select the maximum total values within the window in each year. Determination of the annual maximum rainfall is NOT based on the real storm duration; instead, a design duration which is artificially picked is used for this purpose.

20 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 20 Fitting A Probability Distribution to Annual Maximum Series How do we fit a probability distribution to a random sample? What type of distribution should be adopted? What are the parameter values for the distribution? How good is our fit?

21 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 21 Chi-square GOF test

22 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 22

23 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 23

24 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 24

25 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 25

26 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 26

27 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 27

28 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 28 Kolmogorov-Smirnov GOF test The chi-square test compares the empirical histogram against the theoretical histogram. In contrast, the K-S test compares the empirical cumulative distribution function (ECDF) against the theoretical CDF.

29 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 29

30 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 30

31 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 31

32 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 32 In order to measure the difference between F n (X) and F(X), ECDF statistics based on the vertical distances between F n (X) and F(X) have been proposed.

33 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 33

34 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 34

35 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 35

36 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 36

37 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 37

38 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 38 Hypothesis test using D n

39 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 39 Values of for the Kolmogorov- Smirnov test

40 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 40 GOF test using L-moment-ratios diagram (LMRD) Concept of identifying appropriate distributions using moment-ratio diagrams (MRD). Product-moment-ratio diagram (PMRD) L-moment-ratio diagram (LMRD) Two-parameter distributions Normal, Gumbel (EV-1), etc. Three-parameter distributions Log-normal, Pearson type III, GEV, etc.

41 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 41 Moment ratios are unique properties of probability distributions and sample moment ratios of ordinary skewness and kurtosis have been used for selection of probability distribution. The L-moments uniquely define the distribution if the mean of the distribution exists, and the L-skewness and L-kurtosis are much less biased than the ordinary skewness and kurtosis.

42 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 42 A two-parameter distribution with a location and a scale parameter plots as a single point on the LMRD, whereas a three-parameter distribution with location, scale and shape parameters plots as a curve on the LMRD, and distributions with more than one shape parameter generally are associated with regions on the diagram. However, theoretical points or curves of various probability distributions on the LMRD cannot accommodate for uncertainties induced by parameter estimation using random samples.

43 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 43 Ordinary (or product) moment- ratios diagram (PMRD)

44 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 44 The ordinary (or product) moment ratios diagram

45 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 45

46 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 46 Sample estimates of product moment ratios

47 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 47

48 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 48 (D'Agostino and Stephens, 1986) 90 95

49 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 49 Even though joint distribution of the ordinary sample skewness and sample kurtosis is asymptotically normal, such asymptotic property is a poor approximation in small and moderately samples, particularly when the underlying distribution is even moderately skew.

50 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 50

51 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 51 Scattering of sample moment ratios of the normal distribution (100,000 random samples)

52 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 52 L-moments and the L-moment ratios diagram

53 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 53

54 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 54

55 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 55

56 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 56 L-moment-ratio diagram of various distributions

57 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 57 Sample estimates of L-moment ratios (probability weighted moment estimators)

58 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 58

59 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 59 Sample estimates of L-moment ratios (plotting-position estimators)

60 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 60 Hosking and Wallis (1997) indicated that is not an unbiased estimator of, but its bias tends to zero in large samples. and are respectively referred to as the probability-weighted-moment estimator and the plotting-position estimator of the L- moment ratio.

61 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 61 Establishing acceptance region for L-moment ratios The standard normal and standard Gumbel distributions (zero mean and unit standard deviation) are used to exemplify the approach for construction of acceptance regions for L-moment ratio diagram. L-moment-ratios (, ) of the normal and Gumbel distributions are respectively (0, ) and (0.1699, ).

62 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 62 Stochastic simulation of the normal and Gumbel distributions For either of the standard normal and standard Gumbel distribution, a total of 100,000 random samples were generated with respect to the specified sample size20, 30, 40, 50, 60, 75, 100, 150, 250, 500, and 1,000. For each of the 100,000 samples, sample L- skewness and L-kurtosis were calculated using the probability-weighted-moment estimator and the plotting-position estimator.

63 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 63 Scattering of sample L-moment ratios Normal distribution (100,000 random samples) Normal distribution !

64 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 64 (100,000 random samples) Normal distribution ?

65 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 65 (100,000 random samples) Non-normal distribution ! 95% acceptance region 99% acceptance region

66 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 66 Scattering of sample L-moment ratios Gumbel distribution (100,000 random samples)

67 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 67 (100,000 random samples)

68 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 68 (100,000 random samples)

69 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 69 For both distribution types, the joint distribution of sample L-skewness and L- kurtosis seem to resemble a bivariate normal distribution for a larger sample size (n = 100). However, for sample size n = 20, the joint distribution of sample L-skewness and L- kurtosis seems to differ from the bivariate normal. Particularly for Gumbel distribution, sample L-moments of both estimators are positively skewed.

70 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 70 For smaller sample sizes (n = 20 and 50), the distribution cloud of sample L- moment-ratios estimated by the plotting- position method appears to have its center located away from (, ), an indication of biased estimation. However, for sample size n = 100, the bias is almost unnoticeable, suggesting that the bias in L-moment-ratio estimation using the plotting-position estimator is negligible for larger sample sizes.

71 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 71 In contrast, the distribution cloud of the sample L-moment-ratios estimated by the probability-weighted-moment method appears to have its center almost coincide with (, ).

72 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 72 Bias of sample L-skewness and L- kurtosis - Normal distribution

73 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 73

74 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 74 Bias of sample L-skewness and L- kurtosis - Gumbel distribution

75 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 75

76 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 76 Mardia test for bivariate normality of sample L-skewness and L-kurtosis

77 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 77

78 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 78

79 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 79

80 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 80 Mardia test for bivariate normality of sample L-skewness and L-kurtosis

81 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 81 Mardia test for bivariate normality of sample L-skewness and L-kurtosis

82 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 82 It appears that the assumption of bivariate normal distribution for sample L-skewness and L-kurtosis of both distributions is valid for moderate to large sample sizes. However, for random samples of normal distribution with sample size, the bivariate normal assumption may not be adequate. Similarly, the bivariate normal assumption for sample L-skewness and L- kurtosis of the Gumbel distribution may not be adequate for sample size.

83 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 83 Establishing acceptance regions for LMRD-based GOF tests For moderate to large sample sizes, the sample L- skewness and L-kurtosis of both the normal and Gumbel distributions have asymptotic bivariate normal distributions. Using this property, the acceptance region of a GOF test based on sample L-skewness and L-kurtosis can be determined by the equiprobable density contour of the bivariate normal distribution with its encompassing area equivalent to.

84 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 84 The probability density function of a multivariate normal distribution is generally expressed by The probability density function depends on the random vector X only through the quadratic form which has a chi-square distribution with p degrees of freedom.

85 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 85 Therefore, probability density contours of a multivariate normal distribution can be expressed by for any constant. For a bivariate normal distribution (p=2) the above equation represents an equiprobable ellipse, and a set of equiprobable ellipses can be constructed by assigning to c for various values of.

86 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 86 Consequently, the acceptance region of a GOF test based on the sample L- skewness and L-kurtosis is expressed by where is the upper quantile of the distribution at significance level.

87 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 87 For bivariate normal random vector, the density contour of can also be expressed as However, the expected values and covariance matrix of sample L-skewness and L-kurtosis are unknown and can only be estimated from random samples generated by stochastic simulation.

88 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 88 Thus, in construction of the equiprobable ellipses, population parameters must be respectively replaced by their sample estimates. The Hotellings T 2 statistic

89 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 89 The Hotellings T 2 is distributed as a multiple of an F-distribution, i.e., For large N, Therefore, the distribution of the Hotellings T 2 can be well approximated by the chi-square distribution with degree of freedom 2.

90 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 90 Thus, if the sample L-moments of a random sample of size n falls outside of the corresponding ellipse, i.e. the null hypothesis that the random sample is originated from a normal or Gumbel distribution is rejected.

91 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 91 Scattering of sample L-moment ratios Normal distribution (100,000 random samples)

92 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 92 (100,000 random samples) Normal distribution ?

93 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 93 Variation of 95% acceptance regions with respect to sample size n (100,000 random samples) Non-normal distribution ! 95% acceptance region n=100 n=50 n=20 What if n=36?

94 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 94 Empirical relationships between parameters of acceptance regions and sample size Since the 95% acceptance regions of the proposed GOF tests are dependent on the sample size n, it is therefore worthy to investigate the feasibility of establishing empirical relationships between the 95% acceptance region and the sample size. Such empirical relationships can be established using the following regression model

95 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 95 Empirical relationships between the sample size and parameters of the bivariate distribution of sample L- skewness and L-kurtosis

96 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 96 Empirical relationships between the sample size and parameters of the bivariate distribution of sample L- skewness and L-kurtosis

97 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 97 Example Suppose that a random sample of size n = 44 is available, and the plotting-position sample L-skewness and L-kurtosis are calculated as (, ) = (0.214, 0.116). We want to test whether the sample is originated from the Gumbel distribution.

98 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 98 From the regression models for plotting- position estimators, we find to be respectively , , , , and The Hotellings T 2 is then calculated as The value of T 2 is much smaller than the threshold value

99 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 99 The null hypothesis that the random sample is originated from the Gumbel distribution is not rejected.

100 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU % acceptance regions of L-moments-based GOF test for the normal distribution Acceptance ellipses correspond to various sample sizes (n = 20, 30, 40, 50, 60, 75, 100, 150, 250, 500, and 1,000).

101 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 101 Acceptance ellipses correspond to various sample sizes (n = 20, 30, 40, 50, 60, 75, 100, 150, 250, 500, and 1,000).

102 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU % acceptance regions of L-moments-based GOF test for the Gumbel distribution Acceptance ellipses correspond to various sample sizes (n = 20, 30, 40, 50, 60, 75, 100, 150, 250, 500, and 1,000).

103 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 103 Acceptance ellipses correspond to various sample sizes (n = 20, 30, 40, 50, 60, 75, 100, 150, 250, 500, and 1,000).

104 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 104 Validity check of the LMRD acceptance regions The sample-size-dependent confidence intervals established using empirical relationships described in the last section are further checked for their validity. This is done by stochastically generating 10,000 random samples for both the standard normal and Gumbel distributions, with sample size20, 30, 40, 50, 60, 75, 100, 150, 250, 500, and 1,000.

105 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 105

106 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 106 For validity of the sample-size-dependent 95% acceptance regions, the rejection rate should be very close to the level of significance ( 0.05) or the acceptance rate be very close to 0.95.

107 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 107 Acceptance rate of the validity check for sample-size-dependent 95% acceptance regions of sample L-skewness and L-kurtosis pairs. Based on 10,000 random samples for any given sample size n.

108 Lab for Remote Sensing Hydrology and Spatial Modeling RSLAB-NTU 108 End of this session.


Download ppt "Applied Hydrology RSLAB-NTU Lab for Remote Sensing Hydrology and Spatial Modeling 1 Frequency Analysis Professor Ke-Sheng Cheng Dept. of Bioenvironmental."

Similar presentations


Ads by Google