Lecture series: Data analysis Lectures: Each Tuesday at 16:00 (First lecture: May 21, last lecture: June 25) Thomas Kreuz, ISC, CNR

Lecture 1: Example (Epilepsy & spike train synchrony), Data acquisition, Dynamical systems Lecture 2: Linear measures, Introduction to non-linear dynamics Lecture 3: Non-linear measures Lecture 4: Measures of continuous synchronization Lecture 5: Measures of discrete synchronization (spike trains) Lecture 6: Measure comparison & Application to epileptic seizure prediction Schedule

Example: Epileptic seizure prediction Data acquisition Introduction to dynamical systems First lecture

Non-linear model systems Linear measures Introduction to non-linear dynamics Non-linear measures - Introduction to phase space reconstruction - Lyapunov exponent Second lecture

Non-linear measures - Dimension [ Excursion: Fractals ] - Entropies - Relationships among non-linear measures Third lecture

Characterizition of a dynamic in phase space Predictability (Information / Entropy) Density Self-similarityLinearity / Non-linearity Determinism / Stochasticity (Dimension) Stability (sensitivity to initial conditions)

Dimension (classical) Number of degrees of freedom necessary to characterize a geometric object Euclidean geometry: Integer dimensions Object Dimension Point0 Line1 Square (Area)2 Cube (Volume)3 N-cuben Time series analysis: Number of equations necessary to model a physical system

Hausdorff-dimension

Box-counting

Richardson: Counter-intuitive notion that a coastline's measured length changes with the length of the measuring stick used. Fractal dimension of a coastline: How does the number of measuring sticks required to measure the coastline change with the scale of the stick?

Example: Koch-curve Some properties: - Infinite length - Continuous everywhere - Differentiable nowhere - Fractal dimension D=log4/log3≈ 1.26

Strange attractors are fractals Logistic map Hénon map 2,01

Self-similarity of the logistic attractor

Generalized dimensions

Generalized entropies

Lyapunov-exponent

Summary

Motivation Measures of synchronization for continuous data Linear measures: Cross correlation, coherence Mutual information Phase synchronization (Hilbert transform) Non-linear interdependences Measure comparison on model systems Measures of directionality Granger causality Transfer entropy Today’s lecture

Motivation

Motivation: Bivariate time series analysis Three different scenarios: Repeated measurement from one system (different times)  Stationarity, Reliability Simultaneous measurement from one system (same time)  Coupling, Correlation, Synchronization, Directionality Simultaneous measurement from two systems (same time)  Coupling, Correlation, Synchronization, Directionality

Synchronization [Huygens: Horologium Oscillatorium. 1673]

Synchronization [Pecora & Carroll. Synchronization in chaotic systems. Phys Rev Lett 1990]

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] In-phase synchronization

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] Anti-phase synchronization

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] Synchronization with phase shift

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] No synchronization

Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] In-phase synchronizationAnti-phase synchronization No synchronizationSynchronization with phase shift

Measures of synchronization SynchronizationDirectionality Cross correlation / Coherence Mutual Information Index of phase synchronization - based on Hilbert transform - based on Wavelet transform Non-linear interdependence Event synchronization Delay asymmetry Transfer entropy Granger causality

Linear correlation

Static linear correlation: Pearson’s r -1 - completely anti-correlated r = 0 - uncorrelated (linearly!) 1 - completely correlated Two sets of data points:

Examples: Pearson’s r Undefined [An example of the correlation of x and y for various distributions of (x,y) pairs; Denis Boigelot 2011]

Cross correlation Maximum cross correlation:

Coherence Linear correlation in the frequency domain Cross spectrum: Coherence = Normalized power in the cross spectrum Welch’s method: average over estimated periodograms of subintervals of equal length Complex number  Phase

Mutual information

Shannon entropy ~ ‘Uncertainty’ Binary probabilities:In general:

Mutual Information Marginal Shannon entropy: Joint Shannon entropy: Mutual Information: Estimation based on k-nearest neighbor distances: [Kraskov, Stögbauer, Grassberger: Estimating Mutual Information. Phys Rev E 2004] Kullback-Leibler entropy compares to probability distributions Mutual Information = KL-Entropy with respect to independence

Mutual Information Properties: Non-negativity: Symmetry: Minimum: Independent time series Maximum: for identical systems Venn diagram (Set theory)

Cross correlation & Mutual Information C max I C max I C max I

Phase synchronization

Definition of a phase - Rice phase - Hilbert phase - Wavelet phase Index of phase synchronization - Index based on circular variance - [Index based on Shannon entropy] - [Index based on conditional entropy] Phase synchronization [Tass et al. PRL 1998]

Linear interpolation between ‘marker events’ - threshold crossings (mostly zero, sometimes after demeaning) - discrete events (begin of a new cycle) Problem: Can be very sensitive to noise Rice phase

Hilbert phase [Rosenblum et al., Phys. Rev. Lett. 1996] Analytic signal: ‘Artificial’ imaginary part: Instantaneous Hilbert phase: - Cauchy principal value

Wavelet phase Basis functions with finite support Example: complex Morlet wavelet Wavelet = Hilbert + filter [ Quian Quiroga, Kraskov, Kreuz, Grassberger. Phys. Rev. E 2002 ] Wavelet phase:

Index of phase synchronization: Circular variance (CV)

Non-linear interdependence

Taken’s embedding theorem [F. Takens. Detecting strange attractors in turbulence. Springer, Berlin, 1980]

Non-linear interdependences Nonlinear interdependence SNonlinear interdependence H Synchronization Directionality [Arnhold, Lehnertz, Grassberger, Elger. Physica D 1999]

Non-linear interdependence

Event synchronization

Event synchronization Event times: Synchronicity: Event synchronization:Delay asymmetry: [Quian Quiroga, Kreuz, Grassberger. Phys Rev E 2002] Window: with Avoids double-counting

Event synchronization [Quian Quiroga, Kreuz, Grassberger. Phys Rev E 2002] Q q

Measure comparison on model systems

Measure comparison on model systems [Kreuz, Mormann, Andrzejak, Kraskov, Lehnertz, Grassberger. Phys D 2007]

Model systems & Coupling schemes

Hénon map Introduced by Michel Hénon as a simplified model of the Poincaré section of the Lorenz model One of the most studied examples of dynamical systems that exhibit chaotic behavior [M. Hénon. A two-dimensional mapping with a strange attractor. Commun. Math. Phys., 50:69, 1976]

Hénon map

Coupled Hénon maps Driver: Responder: Identical systems: Coupling strength:

Coupled Hénon maps

Coupled Hénon systems

Rössler system designed in 1976, for purely theoretical reasons later found to be useful in modeling equilibrium in chemical reactions [ O. E. Rössler. An equation for continuous chaos. Phys. Lett. A, 57:397, 1976 ]

Rössler system

Coupled Rössler systems Driver: Responder: Parameter mismatch: Coupling strength:

Coupled Rössler systems

Lorenz system Developed in 1963 as a simplified mathematical model for atmospheric convection Arise in simplified models for lasers, dynamos, electric circuits, and chemical reactions [ E. N. Lorenz. Deterministic non-periodic flow. J. Atmos. Sci., 20:130, 1963 ]

Lorenz system

Coupled Lorenz systems Driver: Responder: Small parameter mismatch in second component Coupling strength:

Coupled Lorenz systems

Noise-free case

Criterion I: Degree of monotonicity = 1 - strictly monotonic increase M(s) = 0 - flat line (or equal decrease and increase) = -1 - strictly monotonic decrease

Degree of monotonicity: Examples Sequences: 100 values  5050 pairs Left: Monotonicity Right: # positive # negative

Comparison: No Noise

Summary: No-Noise-Comparison Results for Rössler are more consistent than for the other systems Mutual Information slightly better than cross correlation (Non-linearity matters) Wavelet phase synchronization not appropiate for broadband systems (inherent filtering looses information)

Robustness against noise

Criterion II: Robustness against noise

Example: White noise

Hénon system: White noise

Rössler system : White noise

Lorenz system: White noise

Hénon system

Comparison: White noise

Summary: White noise For systems opposite order as in the noise-free case (Lorenz more robust then Hénon and than Rössler)  the more monotonous a system has been without noise, the less noise is necessary to destroy this monotonicity Highest robustness is obtained for cross correlation followed by mutual information.

Iso-spectral noise: Example

Iso-spectral noise: Fourier spectrum complex Autocorrelation Fourier spectrum Time domain Frequency domain x (t) Amplitude Physical phenomenon Time series

Generation of iso-spectral noise Phase-randomized surrogates: Take Fourier transform of original signal Randomize phases Take inverse Fourier transform  Iso-spectral surrogate (By construction identical Power spectrum, just different phases) Add to original signal with given NSR

Lorenz system: Iso-spectral noise

Comparison: Iso-spectral noise

Summary: Iso-spectral noise Again results for Rössler are more consistent than for the other systems Sometimes M never crosses critical threshold (monotonicity of the noise-free case is not destroyed by iso-spectral noise). Sometimes synchronization increases for more noise: (spurious) synchronization between contaminating noise-signals, only for narrow-band systems

Correlation among measures

Correlation among measures

Summary: Correlation All correlation values rather high (Minimum: ~0.65) Highest correlations for cross correlation and Hilbert phase synchronization Event synchronization and Hilbert phase synchronization appear least correlated Overall correlation between two phase synchronization methods low (but only due to different frequency sensitivity in the Hénon system)

Overall summary: Comparison of measures Capability to distinguish different coupling strengths Obvious and objective criterion exists only in some special cases (e.g., wavelet phase is not very suitable for a system with a broadband spectrum). Robustness against noise varies (Important criterion for noisy data)  Pragmatic solution: Choose measure which most reliably yields valuable information (e.g., information useful for diagnostic purposes) in test applications

Measures of directionality

Measures of directionality

Granger causality [Granger: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, (1969)]

Granger causality Univariate model: Bivariate model: – Model parameters; – Prediction errors; [Granger: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, (1969)] Fit via linear regression

Transfer Entropy: Conditional entropy Venn diagram (Set theory) Conditional entropy: Mutual Information:

Transfer entropy : Conditional entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

Transfer entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

Transfer entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

Motivation Measures of synchronization for continuous data Linear measures: Cross correlation, coherence Mutual information Phase synchronization (Hilbert transform) Non-linear interdependences Measure comparison on model systems Measures of directionality Granger causality Transfer entropy Today’s lecture

Measures of synchronization for discrete data (e.g. spike trains) Victor-Purpura distance Van Rossum distance Schreiber correlation measure ISI-distance SPIKE-distance Measure comparison Next lecture