ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations 2.1 Stationarity and sampling - In principle, the more a scientific measurement of any kind is useful, the more it is reproducible. In time series measurement, reproducibility is closely connected to two different notions of stationarity. - During measurement period, the form of stationarity requires that all system dynamics parameters have to be fixed and constant.  In fact, constant external parameters may not induce a stationary process. example) The process under observation is probabilistic one.  Because probabilities may not depend on time. - In some cases, we can handle a simple change of parameter once the change is noticed.  Periodic modulation of a parameter can be interpreted as a dynamical variable rather than a parameter. This dose not necessarily destroy stationarity.

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations - Second concept of stationarity is based on the available data itself.  How long and how precise the time series must be, and how frequently measurement should be taken. Example 2.1) How the temperature changes on the roof of one’s house? - What happens in the course of a day?  take a measurement once every hour throughout one week - Seasonal temperature fluctuations  take a measurement at a fixed time once every day throughout the year. - temperature fluctuations from year to year  take a couple of measurements every year - complete discription of phenomenon  climb up to the roof once an hour every day of the year for a couple of years

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations - More generally, a signal is stationary if all joint probabilities of finding the systme at some time in one state and at some later time in another state are independent of time within the observation period, i,e. when calculated from the data. - If the observed signal is quite regular almost all of the time, but contains one very irregular burst every so often, then the time series has to be considered to be non-stationary for our purpose. - Non-stationary signals are very common when observing natural or cultural phenomenon. - Almost all the methods and results on time series analysis assume both conditions : - the parameters of the system remain constant, - the phenomenon is sufficiently sampled.

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations 2.2 Testing for stationary - How non-stationarity can be detected for a given data set? - Stationary requirement differs depending on the application. - First requirement : the time series should cover a stretch of time which is much longer than the longest characteristic time scale that is relevant for the evolution of the system. That is, a time series can be considered stationary only on much larger time scale. example) concentration of sugar in the blood of a human - Strong violations of the basic requirement can be checked simply by measuring dynamical properties for several segments of the data set. - Statistically most stable quantities are the mean and the variance.

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations Example 2.2) Running variance of voice data - Compute the sample mean and variance for 50 consecutive segments of the data set, each containing 2048 data points. The standard error of the estimated mean is given by, neglecting the fact that the samples in the time series are not independent. - The observed fluctuations of the running mean of the 50 segments were within these errors. However, the running variance showed variability beyond the statistical fluctuations. (see Fig2.1 in textbook)

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations - In experimental chaotic systems, parameter drift dose not result in visible drift in the mean or the distribution of values. Linear correlations and the spectrum may also be unaffected. - Only the nonlinear dynamical relations and transition probabilities change. - Whether the data set is a sufficient sample for a particular application, such as the estimate of a characteristic quantity, may be tested by observing the convergence of that quantity when larger and larger fractions of the available data are used for its computation. - Correlation dimention : a quantity which suffers from non-stationarity. - While a simple drift of the calibration usually increase the dimension, most other types of the non-stationary and insufficient sampling yield spuriously low dimension estimates.

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations 2.3 Linear correlations and the power spectrum - Linear statistical inference is an extremely well-developed field. But, nonlinear inference cannot be studied to rigorously prove their correctness and convergence. - The solutions of stable linear deterministic equations of motions with constant coefficients are linear superpositions of exponentially damped harmonic oscillations. Hence, linear systems always need irregular inputs to produce irregular signals. - The most simple systems which produces non-periodic signals is a linear stochastic process. - A measurement of the state at time n of a stochastic process can be regarded as drawn from an underlying probability distribution for observing different values or sequences of values. - A stochastic process is called stationary if these probabilities are constant over time:

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations - The mean of the probability distribution p(s), defined as can be estimated by the mean of the finite time series where denotes the average over time and N is the total number of measurements in the time series. The variance of the probability distribution will be estimated by the variance of the time series:

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations - For these two quantities the time ordering of the measurements is irrelevant and thus they cannot give any information about the time evolution of a system. Such information can be obtained from the autocorrelations of a signal. - The autocorrelation at lag is given by - The estimation of autocorrelations from a time series is straightforward as long as the lag is small compared to the total length of the time series. - A particular case is given if the measurements are drawn from a Gaussian distribution, And, the joint distribution of multiple measurements are also Gaussian.

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations - If a signal is periodic in time, then the autocorrelation function is periodic in the lag. - Stocastic process have decaying autocorrelations but the rate of decay depends on the properties of the process. Autocorrelations of signals from deterministic chaotic systems typically also decay exponentially with increasing lag. - Autocorrelations are not characteristic enough to distinguish random from deterministic chaotic signals. - The Fourier transform establishes a one-to-one correspondence between to the signal at certain times(time domain) and how certain frequencies contribute to the signal, and how the phases of the oscillations are related to the phase of other oscillations (frequency domain).

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations - The power spectrum of a process is defined to be the squared modules of the continuous Fourier transform, It is the square of the amplitude, by which the frequency contributes to the signal. - Wiener-Khinchin theorem : the Fourier transform of autocorrelation function equals the power spectrum. - Parseval’s theorem : the total power increases linearly with the length of the time series and can be computed both in the time and frequency domain, via - The power spectrum is particularly useful for studying the oscillations of a system. There will be sharper or broader peaks at the dominant frequencies and at their harmonics.

ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations - Spectrogram : - in frequency domain and in time average, we lose all time information - Spectral analysis can be done on consecutive segments of a longer time series to track temporal changes. This is called a spectrogram. - Example 2.4) Spectrogram of voice data - Stationary and the low-frequency component in the power spectrum - The power spectrum reflects the contribution of all possible periods and should contain relevant information. - If there is too much power in the low frequencies, the time series must considered non-stationary since the corresponding Fourier modes have only very few oscillations during the observation time.