1. A.A. Soloviev, Sh.R. Bogoutdinov, S.M. Agayan, A.D. Gvishiani (GC RAS, Russia), A. Chulliat (IPGP, France) Automated recognition of spikes on 1-minute and 1-second magnetograms

2. Construction scheme of DMA 2Geophysical Center RASUglich, 28 January 2011

3. Interpreter’s Logic. Illustration Global level - searching for the uplifts on rectification Local level - rectification of the record Record 3Geophysical Center RASUglich, 28 January 2011

4. Illustration of rectification Record Rectification “Energy” Rectification “Length” 4Geophysical Center RASUglich, 28 January 2011

5. Instrumental failure examples on 1-min magnetograms Spikes with an amplitude of less than 30 nT are not considered * Spikes with an amplitude of less than 30 nT are not considered * 5Geophysical Center RASUglich, 28 January 2011

6. 6 Easter Island Observatory Example Big two-side spike is induced by an airplane’s touchdown. Small one-side spikes are induced by moving of big trucks Geophysical Center RASUglich, 28 January 2011

7. SP algorithm 7Geophysical Center RAS Global level Local level Rectification “Oscillation” Elementary dynamics Spikes Chain of dynamics Informal logic underlying the algorithm: “Spike is a chain of connected elementary dynamics (vertically significant and horizontally insignificant disturbances) on a record not leading to its baseline shift” Uglich, 28 January 2011

8. SP algorithm. Local level  Recognition of  -fragments containing singular dynamics (1/2)

9. SP algorithm. Local level 9Geophysical Center RAS Examples of elementary dynamics ,, where, Recognition of  -fragments containing singular dynamics (2/2) 2  -fragment Uglich, 28 January 2011

10. SP algorithm. Global level 10Geophysical Center RAS Wings Wings: Wings are marked with green Condition for spike (1/2): Uglich, 28 January 2011

11. SP algorithm. Global level 11Geophysical Center RAS  Condition for spike (2/2): spike no spike Uglich, 28 January 2011

12. SP algorithm. Learning 12 In total 25 records for 1/1/2007-31/12/2007, 525 600 points each 275 spikes recognized by INTERMAGNET experts Learning materials Geophysical Center RASUglich, 28 January 2011

13. SP algorithm. Learning Probability of an error of the first kind (target miss): Probability of an error of the second kind (false alarm): Overall criterion of recognition quality: Indicator of recognition quality on a record fragment 13Geophysical Center RASUglich, 28 January 2011

14. SP algorithm. Learning Brute-force search of free parameter values: 4 600 sets of values sets – result of routine processing of magnetograms by experts In criterion = 0.9, which expresses higher degree of expert opinion importance ( criterion) comparing to SP algorithm ( criterion) Optimal parameters of the algorithm: Probability of target miss = 0.0, probability of false alarm = 0.055 The quality of the recognition = 0.005 Selection of the optimal values of free parameters 14Geophysical Center RAS 0,9 Uglich, 28 January 2011

15. SP algorithm. Learning 15Geophysical Center RASUglich, 28 January 2011

16. SP algorithm. Exam 16 In total 17 records for 1/1/2008-31/12/2008, 525 600 points each 102 spikes recognized by INTERMAGNET experts Exam materials and results Geophysical Center RASUglich, 28 January 2011

17. SP algorithm. Exam 17, 110 events recognized on exam materials by algorithm Target miss = 1 spike and so = 1/102 = 0.01 False alarm = 9 events and so = 9/110 = 0.09 The quality of the recognition = 0.018 Geophysical Center RAS Result of exam of SP with optimal values of free parameters: 0,9 [Sh.R. Bogoutdinov et al. Recognition of disturbances with specified morphology in time series. Part 1: Spikes on magnetograms of the worldwide INTERMAGNET network // Izvestiya, Physics of the Solid Earth, 2010, Vol.46, No.11, pp. 1004–1016] [Ш.Р.Богоутдинов и др. Распознавание возмущений с заданной морфологией на временных рядах. I. Выбросы на магнитограммах всемирной сети ИНТЕРМАГНЕТ // Физика Земли. 2010. №11. С. 99-112] Uglich, 28 January 2011

18. SP algorithm. Examples 18Geophysical Center RAS Three cases of spike recognition (bottom – preliminary data, top – definitive data) Uglich, 28 January 2011

19. Each record should be examined by an expert very attentively SP algorithm. Examples 19Geophysical Center RASUglich, 28 January 2011

20. Each record should be examined by an expert very attentively SP algorithm. Examples 20Geophysical Center RASUglich, 28 January 2011

21. SPm: application to 1-sec data (1/5) Raw one-second data XYZF acquired at the Easter Island magnetic observatory in July 2009 (86 400 points of registration for 1-day 1-channel record) Testing dataset (X component) cleaned manually after a detailed inspection of daily magnetograms: – –1-20 July 2009 for learning the algorithm – –21-31 July 2009 for testing the algorithm Blind test data (X component) recorded on 1-31 August 2009 with no a priori expert opinion 21Geophysical Center RASUglich, 28 January 2011

22. Natural spikes, which represent geomagnetic pulsations (X component, 01/07/2009) SPm: application to 1-sec data (2/5) Artificial spikes, which have to be removed by experts manually (X component, 05/07/2009) 22Geophysical Center RASUglich, 28 January 2011

23. SPm: application to 1-sec data (3/5) Geophysical Center RAS23Uglich, 28 January 2011

24. Examples of spike recognition by the algorithm SPm O O SPm: application to 1-sec data (4/5) 24Geophysical Center RASUglich, 28 January 2011

25. Learning phase, 1-20 July 2009 (1119 spikes): – –Missed targets: 4.7% – –False alarms: 8.7% Testing phase, 21-31 July 2009 (853 spikes): – –Missed targets: 5.9% – –False alarms: 6.0% Blind test, 1-31 August 2009 (2210 spikes): – –Missed targets: 4.85% – –False alarms: 0.6% SPm: application to 1-sec data (5/5), recognition statistics 25Geophysical Center RASUglich, 28 January 2011

26. Step 2. Calculating measures of jumpiness using fuzzy bounds, Potential jump (red), wings (green), fuzzy bounds (black) Step 3. Test for absolute values Free parameters: Baseline jump. JM algorithm Definition. “Jump is an anomaly on a record leading to its baseline shift” Step 1. Recognition of all anomalies by FCARS algorithm 26Geophysical Center RASUglich, 28 January 2011

27. INTERMAGNET 1-min data Comparison of jump recognition results based on preliminary data records processing with definitive data records 27Geophysical Center RASUglich, 28 January 2011

28. GOES 2 Hz geomag data 28Geophysical Center RASUglich, 28 January 2011