1. Lecture series: Data analysis Lectures: Each Tuesday at 16:00 (First lecture: May 21, last lecture: June 25) Thomas Kreuz, ISC, CNR thomas.kreuz@cnr.it http://www.fi.isc.cnr.it/users/thomas.kreuz/

2. Lecture 1: Example (Epilepsy & spike train synchrony), Data acquisition, Dynamical systems Lecture 2: Linear measures, Introduction to non-linear dynamics Lecture 3: Non-linear measures Lecture 4: Measures of continuous synchronization Lecture 5: Measures of discrete synchronization (spike trains) Lecture 6: Measure comparison & Application to epileptic seizure prediction Schedule

3. Example: Epileptic seizure prediction Data acquisition Introduction to dynamical systems First lecture

4. Non-linear model systems Linear measures Introduction to non-linear dynamics Non-linear measures - Introduction to phase space reconstruction - Lyapunov exponent Second lecture

5. Non-linear measures - Dimension [ Excursion: Fractals ] - Entropies - Relationships among non-linear measures Third lecture

6. Characterizition of a dynamic in phase space Predictability (Information / Entropy) Density Self-similarityLinearity / Non-linearity Determinism / Stochasticity (Dimension) Stability (sensitivity to initial conditions)

7. Dimension (classical) Number of degrees of freedom necessary to characterize a geometric object Euclidean geometry: Integer dimensions Object Dimension Point0 Line1 Square (Area)2 Cube (Volume)3 N-cuben Time series analysis: Number of equations necessary to model a physical system

8. Hausdorff-dimension

9. Box-counting

10. Richardson: Counter-intuitive notion that a coastline's measured length changes with the length of the measuring stick used. Fractal dimension of a coastline: How does the number of measuring sticks required to measure the coastline change with the scale of the stick?

11. Example: Koch-curve Some properties: - Infinite length - Continuous everywhere - Differentiable nowhere - Fractal dimension D=log4/log3≈ 1.26

12. Strange attractors are fractals Logistic map Hénon map 2,01

13. Self-similarity of the logistic attractor

14. Generalized dimensions

15. Generalized entropies

16. Lyapunov-exponent

17. Summary

18. Motivation Measures of synchronization for continuous data Linear measures: Cross correlation, coherence Mutual information Phase synchronization (Hilbert transform) Non-linear interdependences Measure comparison on model systems Measures of directionality Granger causality Transfer entropy Today’s lecture

19. Motivation

20. Motivation: Bivariate time series analysis Three different scenarios: Repeated measurement from one system (different times)  Stationarity, Reliability Simultaneous measurement from one system (same time)  Coupling, Correlation, Synchronization, Directionality Simultaneous measurement from two systems (same time)  Coupling, Correlation, Synchronization, Directionality

21. Synchronization [Huygens: Horologium Oscillatorium. 1673]

22. Synchronization [Pecora & Carroll. Synchronization in chaotic systems. Phys Rev Lett 1990]

23. Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] In-phase synchronization

24. Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] Anti-phase synchronization

25. Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] Synchronization with phase shift

26. Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] No synchronization

27. Synchronization [Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)] In-phase synchronizationAnti-phase synchronization No synchronizationSynchronization with phase shift

28. Measures of synchronization SynchronizationDirectionality Cross correlation / Coherence Mutual Information Index of phase synchronization - based on Hilbert transform - based on Wavelet transform Non-linear interdependence Event synchronization Delay asymmetry Transfer entropy Granger causality

29. Linear correlation

30. Static linear correlation: Pearson’s r -1 - completely anti-correlated r = 0 - uncorrelated (linearly!) 1 - completely correlated Two sets of data points:

31. Examples: Pearson’s r Undefined [An example of the correlation of x and y for various distributions of (x,y) pairs; Denis Boigelot 2011]

32. Cross correlation Maximum cross correlation:

33. Coherence Linear correlation in the frequency domain Cross spectrum: Coherence = Normalized power in the cross spectrum Welch’s method: average over estimated periodograms of subintervals of equal length Complex number  Phase

34. Mutual information

35. Shannon entropy ~ ‘Uncertainty’ Binary probabilities:In general:

36. Mutual Information Marginal Shannon entropy: Joint Shannon entropy: Mutual Information: Estimation based on k-nearest neighbor distances: [Kraskov, Stögbauer, Grassberger: Estimating Mutual Information. Phys Rev E 2004] Kullback-Leibler entropy compares to probability distributions Mutual Information = KL-Entropy with respect to independence

37. Mutual Information Properties: Non-negativity: Symmetry: Minimum: Independent time series Maximum: for identical systems Venn diagram (Set theory)

38. Cross correlation & Mutual Information 1.0 0.5 0.0 C max I 1.0 0.5 0.0 C max I 1.0 0.5 0.0 C max I

39. Phase synchronization

40. Definition of a phase - Rice phase - Hilbert phase - Wavelet phase Index of phase synchronization - Index based on circular variance - [Index based on Shannon entropy] - [Index based on conditional entropy] Phase synchronization [Tass et al. PRL 1998]

41. Linear interpolation between ‘marker events’ - threshold crossings (mostly zero, sometimes after demeaning) - discrete events (begin of a new cycle) Problem: Can be very sensitive to noise Rice phase

42. Hilbert phase [Rosenblum et al., Phys. Rev. Lett. 1996] Analytic signal: ‘Artificial’ imaginary part: Instantaneous Hilbert phase: - Cauchy principal value

43. Wavelet phase Basis functions with finite support Example: complex Morlet wavelet Wavelet = Hilbert + filter [ Quian Quiroga, Kraskov, Kreuz, Grassberger. Phys. Rev. E 2002 ] Wavelet phase:

44. Index of phase synchronization: Circular variance (CV)

45. Non-linear interdependence

46. Taken’s embedding theorem [F. Takens. Detecting strange attractors in turbulence. Springer, Berlin, 1980]

47. Non-linear interdependences Nonlinear interdependence SNonlinear interdependence H Synchronization Directionality [Arnhold, Lehnertz, Grassberger, Elger. Physica D 1999]

48. Non-linear interdependence

58. Event synchronization

59. Event synchronization Event times: Synchronicity: Event synchronization:Delay asymmetry: [Quian Quiroga, Kreuz, Grassberger. Phys Rev E 2002] Window: with Avoids double-counting

60. Event synchronization [Quian Quiroga, Kreuz, Grassberger. Phys Rev E 2002] Q q

61. Measure comparison on model systems

62. Measure comparison on model systems [Kreuz, Mormann, Andrzejak, Kraskov, Lehnertz, Grassberger. Phys D 2007]

63. Model systems & Coupling schemes

64. Hénon map Introduced by Michel Hénon as a simplified model of the Poincaré section of the Lorenz model One of the most studied examples of dynamical systems that exhibit chaotic behavior [M. Hénon. A two-dimensional mapping with a strange attractor. Commun. Math. Phys., 50:69, 1976]

65. Hénon map

66. Coupled Hénon maps Driver: Responder: Identical systems: Coupling strength:

67. Coupled Hénon maps

68. Coupled Hénon systems

69. Rössler system designed in 1976, for purely theoretical reasons later found to be useful in modeling equilibrium in chemical reactions [ O. E. Rössler. An equation for continuous chaos. Phys. Lett. A, 57:397, 1976 ]

70. Rössler system

71. Coupled Rössler systems Driver: Responder: Parameter mismatch: Coupling strength:

72. Coupled Rössler systems

74. Lorenz system Developed in 1963 as a simplified mathematical model for atmospheric convection Arise in simplified models for lasers, dynamos, electric circuits, and chemical reactions [ E. N. Lorenz. Deterministic non-periodic flow. J. Atmos. Sci., 20:130, 1963 ]

75. Lorenz system

76. Coupled Lorenz systems Driver: Responder: Small parameter mismatch in second component Coupling strength:

77. Coupled Lorenz systems

79. Noise-free case

80. Criterion I: Degree of monotonicity = 1 - strictly monotonic increase M(s) = 0 - flat line (or equal decrease and increase) = -1 - strictly monotonic decrease

81. Degree of monotonicity: Examples Sequences: 100 values  5050 pairs Left: Monotonicity Right: # positive # negative

82. Comparison: No Noise

83. Summary: No-Noise-Comparison Results for Rössler are more consistent than for the other systems Mutual Information slightly better than cross correlation (Non-linearity matters) Wavelet phase synchronization not appropiate for broadband systems (inherent filtering looses information)

84. Robustness against noise

85. Criterion II: Robustness against noise

86. Example: White noise

87. Hénon system: White noise

88. Rössler system : White noise

89. Lorenz system: White noise

90. Hénon system

91. Comparison: White noise

92. Summary: White noise For systems opposite order as in the noise-free case (Lorenz more robust then Hénon and than Rössler)  the more monotonous a system has been without noise, the less noise is necessary to destroy this monotonicity Highest robustness is obtained for cross correlation followed by mutual information.

93. Iso-spectral noise: Example

94. Iso-spectral noise: Fourier spectrum complex Autocorrelation Fourier spectrum Time domain Frequency domain x (t) Amplitude Physical phenomenon Time series

95. Generation of iso-spectral noise Phase-randomized surrogates: Take Fourier transform of original signal Randomize phases Take inverse Fourier transform  Iso-spectral surrogate (By construction identical Power spectrum, just different phases) Add to original signal with given NSR

96. Lorenz system: Iso-spectral noise

97. Comparison: Iso-spectral noise

98. Summary: Iso-spectral noise Again results for Rössler are more consistent than for the other systems Sometimes M never crosses critical threshold (monotonicity of the noise-free case is not destroyed by iso-spectral noise). Sometimes synchronization increases for more noise: (spurious) synchronization between contaminating noise-signals, only for narrow-band systems

99. Correlation among measures

100. Correlation among measures

104. Summary: Correlation All correlation values rather high (Minimum: ~0.65) Highest correlations for cross correlation and Hilbert phase synchronization Event synchronization and Hilbert phase synchronization appear least correlated Overall correlation between two phase synchronization methods low (but only due to different frequency sensitivity in the Hénon system)

105. Overall summary: Comparison of measures Capability to distinguish different coupling strengths Obvious and objective criterion exists only in some special cases (e.g., wavelet phase is not very suitable for a system with a broadband spectrum). Robustness against noise varies (Important criterion for noisy data)  Pragmatic solution: Choose measure which most reliably yields valuable information (e.g., information useful for diagnostic purposes) in test applications

106. Measures of directionality

107. Measures of directionality

108. Granger causality [Granger: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424-438 (1969)]

109. Granger causality Univariate model: Bivariate model: – Model parameters; – Prediction errors; [Granger: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424-438 (1969)] Fit via linear regression

110. Transfer Entropy: Conditional entropy Venn diagram (Set theory) Conditional entropy: Mutual Information:

111. Transfer entropy : Conditional entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

112. Transfer entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

113. Transfer entropy [T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]

114. Motivation Measures of synchronization for continuous data Linear measures: Cross correlation, coherence Mutual information Phase synchronization (Hilbert transform) Non-linear interdependences Measure comparison on model systems Measures of directionality Granger causality Transfer entropy Today’s lecture

115. Measures of synchronization for discrete data (e.g. spike trains) Victor-Purpura distance Van Rossum distance Schreiber correlation measure ISI-distance SPIKE-distance Measure comparison Next lecture