STOCHASTIC HYDROLOGY Random Processes

STOCHASTIC HYDROLOGY Random Processes
Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University

Introduction A random process is a process (i.e., variation in time or one dimensional space) whose behavior is not completely predictable and can be characterized by statistical laws. Examples of random processes Daily stream flow Hourly rainfall of storm events Stock index 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Random Variable A random variable is a mapping function which assigns outcomes of a random experiment to real numbers. Occurrence of the outcome follows certain probability distribution. Therefore, a random variable is completely characterized by its probability density function (PDF). 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The term “stochastic processes” appears mostly in statistical textbooks; however, the term “random processes” are frequently used in books of many engineering applications. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Characterizations of a Stochastic Processes
First-order densities of a random process A stochastic process is defined to be completely or totally characterized if the joint densities for the random variables are known for all times and all n. In general, a complete characterization is practically impossible, except in rare cases. As a result, it is desirable to define and work with various partial characterizations. Depending on the objectives of applications, a partial characterization often suffices to ensure the desired outputs. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

For a specific t, X(t) is a random variable with distribution .
The function is defined as the first-order distribution of the random variable X(t). Its derivative with respect to x is the first-order density of X(t). 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

If the first-order densities defined for all time t, i. e
If the first-order densities defined for all time t, i.e. f(x,t), are all the same, then f(x,t) does not depend on t and we call the resulting density the first-order density of the random process ; otherwise, we have a family of first-order densities. The first-order densities (or distributions) are only a partial characterization of the random process as they do not contain information that specifies the joint densities of the random variables defined at two or more different times. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Mean and variance of a random process
The first-order density of a random process, f(x,t), gives the probability density of the random variables X(t) defined for all time t. The mean of a random process, mX(t), is thus a function of time specified by 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

For the case where the mean of X(t) does not depend on t, we have
The variance of a random process, also a function of time, is defined by 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Second-order densities of a random process
For any pair of two random variables X(t1) and X(t2), we define the second-order densities of a random process as or Nth-order densities of a random process The nth order density functions for at times are given by or 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Autocorrelation and autocovariance functions of random processes
Given two random variables X(t1) and X(t2), a measure of linear relationship between them is specified by E[X(t1)X(t2)]. For a random process, t1 and t2 go through all possible values, and therefore, E[X(t1)X(t2)] can change and is a function of t1 and t2. The autocorrelation function of a random process is thus defined by 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Stationarity of random processes
Strict-sense stationarity seldom holds for random processes, except for some Gaussian processes. Therefore, weaker forms of stationarity are needed. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Equality of random processes
Note that “x(t, i) = y(t, i) for every i” is not the same as “x(t, i) = y(t, i) with probability 1”. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Mean square equality 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Stochastic Convergence
A random sequence or a discrete-time random process is a sequence of random variables {X1(), X2(), …, Xn(),…} = {Xn()},   . For a specific , {Xn()} is a sequence of numbers that might or might not converge. The notion of convergence of a random sequence can be given several interpretations. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Sure convergence (convergence everywhere)
The sequence of random variables {Xn()} converges surely to the random variable X() if the sequence of functions Xn() converges to X() as n   for all   , i.e., Xn()  X() as n   for all   . 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Almost-sure convergence (Convergence with probability 1)
Limit of the sequence. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Mean-square convergence

Convergence in probability
Limit of the probability 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Convergence in probability at no.

n Beyond this point, n is considered large. It maybe possible that a random process converges in probability to a constant (or a random variable X), however none realization converges to the constant (or X). 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Convergence in distribution

Remarks Convergence with probability one applies to individual realizations of the random process. Convergence in probability does not. The weak law of large numbers is an example of convergence in probability. The strong law of large numbers is an example of convergence with probability 1. The central limit theorem is an example of convergence in distribution. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Weak Law of Large Numbers (WLLN)

Strong Law of Large Numbers (SLLN)

The Central Limit Theorem

Venn diagram of relation of types of convergence
Note that even sure convergence may not imply mean square convergence. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Example 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Ergodic Theorem 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The Mean-Square Ergodic Theorem

The above theorem shows that one can expect a sample average to converge to a constant in mean square sense if and only if the average of the means converges and if the memory dies out asymptotically, that is , if the covariance decreases as the lag increases. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Mean-Ergodic Processes

Strong or Individual Ergodic Theorem

Examples of Stochastic Processes
iid random process A discrete time random process {X(t), t = 1, 2, …} is said to be independent and identically distributed (iid) if any finite number, say k, of random variables X(t1), X(t2), …, X(tk) are mutually independent and have a common cumulative distribution function FX() . 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The joint cdf for X(t1), X(t2), …, X(tk) is given by
It also yields where p(x) represents the common probability mass function. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Random walk process 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Let 0 denote the probability mass function of X0
Let 0 denote the probability mass function of X0. The joint probability of X0, X1,  Xn is 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

is known as the first order Markov property.
The property is known as the first order Markov property. A special case of random walk: the Brownian motion. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Gaussian process A random process {X(t)} is said to be a Gaussian random process if all finite collections of the random process, X1=X(t1), X2=X(t2), …, Xk=X(tk), are jointly Gaussian random variables for all k, and all choices of t1, t2, …, tk. Joint pdf of jointly Gaussian random variables X1, X2, …, Xk: 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Time series – AR random process

The Brownian motion (one-dimensional, also known as random walk)
Consider a particle randomly moves on a real line. Suppose that at small time intervals  the particle jumps a small distance  randomly and equally likely to the left or to the right. Let be the position of the particle on the real line at time t. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Assume the initial position of the particle is at the origin, i.e.
Position of the particle at time t can be expressed as where are independent random variables, each having probability 1/2 of equating 1 and 1. ( represents the largest integer not exceeding ) 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

X t 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Distribution of X(t) Let the step length equal , then
For fixed t, if  is small then the distribution of is approximately normal with mean 0 and variance t, i.e., 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Graphical illustration of Distribution of X(t)
Time, t PDF of X(t) X(t) 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

If t and h are fixed and  is sufficiently small then

Distribution of the displacement
The random variable is normally distributed with mean 0 and variance h, i.e. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Variance of is dependent on t, while variance of is not. If , then ,
are independent random variables. 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Covariance and Correlation functions of

How can we assess the model performance?
Can a stochastic model be validated using a set of observed time series? How can we assess the model performance? 1/3/2019 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

STOCHASTIC HYDROLOGY Random Processes

Similar presentations

Presentation on theme: "STOCHASTIC HYDROLOGY Random Processes"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

STOCHASTIC HYDROLOGY Random Processes

Similar presentations

Presentation on theme: "STOCHASTIC HYDROLOGY Random Processes"— Presentation transcript:

Similar presentations

About project

Feedback