A.A. Soloviev, Sh.R. Bogoutdinov, S.M. Agayan, A.D. Gvishiani (GC RAS, Russia), A. Chulliat (IPGP, France) Automated recognition of spikes on 1-minute and 1-second magnetograms

Construction scheme of DMA 2Geophysical Center RASUglich, 28 January 2011

Interpreter’s Logic. Illustration Global level - searching for the uplifts on rectification Local level - rectification of the record Record 3Geophysical Center RASUglich, 28 January 2011

Illustration of rectification Record Rectification “Energy” Rectification “Length” 4Geophysical Center RASUglich, 28 January 2011

Instrumental failure examples on 1-min magnetograms Spikes with an amplitude of less than 30 nT are not considered * Spikes with an amplitude of less than 30 nT are not considered * 5Geophysical Center RASUglich, 28 January 2011

6 Easter Island Observatory Example Big two-side spike is induced by an airplane’s touchdown. Small one-side spikes are induced by moving of big trucks Geophysical Center RASUglich, 28 January 2011

SP algorithm 7Geophysical Center RAS Global level Local level Rectification “Oscillation” Elementary dynamics Spikes Chain of dynamics Informal logic underlying the algorithm: “Spike is a chain of connected elementary dynamics (vertically significant and horizontally insignificant disturbances) on a record not leading to its baseline shift” Uglich, 28 January 2011

SP algorithm. Local level  Recognition of  -fragments containing singular dynamics (1/2)

SP algorithm. Local level 9Geophysical Center RAS Examples of elementary dynamics ,, where, Recognition of  -fragments containing singular dynamics (2/2) 2  -fragment Uglich, 28 January 2011

SP algorithm. Global level 10Geophysical Center RAS Wings Wings: Wings are marked with green Condition for spike (1/2): Uglich, 28 January 2011

SP algorithm. Global level 11Geophysical Center RAS  Condition for spike (2/2): spike no spike Uglich, 28 January 2011

SP algorithm. Learning 12 In total 25 records for 1/1/ /12/2007, points each 275 spikes recognized by INTERMAGNET experts Learning materials Geophysical Center RASUglich, 28 January 2011

SP algorithm. Learning Probability of an error of the first kind (target miss): Probability of an error of the second kind (false alarm): Overall criterion of recognition quality: Indicator of recognition quality on a record fragment 13Geophysical Center RASUglich, 28 January 2011

SP algorithm. Learning Brute-force search of free parameter values: sets of values sets – result of routine processing of magnetograms by experts In criterion = 0.9, which expresses higher degree of expert opinion importance ( criterion) comparing to SP algorithm ( criterion) Optimal parameters of the algorithm: Probability of target miss = 0.0, probability of false alarm = The quality of the recognition = Selection of the optimal values of free parameters 14Geophysical Center RAS 0,9 Uglich, 28 January 2011

SP algorithm. Learning 15Geophysical Center RASUglich, 28 January 2011

SP algorithm. Exam 16 In total 17 records for 1/1/ /12/2008, points each 102 spikes recognized by INTERMAGNET experts Exam materials and results Geophysical Center RASUglich, 28 January 2011

SP algorithm. Exam 17, 110 events recognized on exam materials by algorithm Target miss = 1 spike and so = 1/102 = 0.01 False alarm = 9 events and so = 9/110 = 0.09 The quality of the recognition = Geophysical Center RAS Result of exam of SP with optimal values of free parameters: 0,9 [Sh.R. Bogoutdinov et al. Recognition of disturbances with specified morphology in time series. Part 1: Spikes on magnetograms of the worldwide INTERMAGNET network // Izvestiya, Physics of the Solid Earth, 2010, Vol.46, No.11, pp. 1004–1016] [Ш.Р.Богоутдинов и др. Распознавание возмущений с заданной морфологией на временных рядах. I. Выбросы на магнитограммах всемирной сети ИНТЕРМАГНЕТ // Физика Земли №11. С ] Uglich, 28 January 2011

SP algorithm. Examples 18Geophysical Center RAS Three cases of spike recognition (bottom – preliminary data, top – definitive data) Uglich, 28 January 2011

Each record should be examined by an expert very attentively SP algorithm. Examples 19Geophysical Center RASUglich, 28 January 2011

Each record should be examined by an expert very attentively SP algorithm. Examples 20Geophysical Center RASUglich, 28 January 2011

SPm: application to 1-sec data (1/5) Raw one-second data XYZF acquired at the Easter Island magnetic observatory in July 2009 ( points of registration for 1-day 1-channel record) Testing dataset (X component) cleaned manually after a detailed inspection of daily magnetograms: – –1-20 July 2009 for learning the algorithm – –21-31 July 2009 for testing the algorithm Blind test data (X component) recorded on 1-31 August 2009 with no a priori expert opinion 21Geophysical Center RASUglich, 28 January 2011

Natural spikes, which represent geomagnetic pulsations (X component, 01/07/2009) SPm: application to 1-sec data (2/5) Artificial spikes, which have to be removed by experts manually (X component, 05/07/2009) 22Geophysical Center RASUglich, 28 January 2011

SPm: application to 1-sec data (3/5) Geophysical Center RAS23Uglich, 28 January 2011

Examples of spike recognition by the algorithm SPm O O SPm: application to 1-sec data (4/5) 24Geophysical Center RASUglich, 28 January 2011

Learning phase, 1-20 July 2009 (1119 spikes): – –Missed targets: 4.7% – –False alarms: 8.7% Testing phase, July 2009 (853 spikes): – –Missed targets: 5.9% – –False alarms: 6.0% Blind test, 1-31 August 2009 (2210 spikes): – –Missed targets: 4.85% – –False alarms: 0.6% SPm: application to 1-sec data (5/5), recognition statistics 25Geophysical Center RASUglich, 28 January 2011

Step 2. Calculating measures of jumpiness using fuzzy bounds, Potential jump (red), wings (green), fuzzy bounds (black) Step 3. Test for absolute values Free parameters: Baseline jump. JM algorithm Definition. “Jump is an anomaly on a record leading to its baseline shift” Step 1. Recognition of all anomalies by FCARS algorithm 26Geophysical Center RASUglich, 28 January 2011

INTERMAGNET 1-min data Comparison of jump recognition results based on preliminary data records processing with definitive data records 27Geophysical Center RASUglich, 28 January 2011

GOES 2 Hz geomag data 28Geophysical Center RASUglich, 28 January 2011