July 23, BSA, a Fast and Accurate Spike Train Encoding Scheme Benjamin Schrauwen

July 23, Overview Introduction Reading the spiking language Speaking the spiking language Results Conclusions and further work

July 23, Introduction NN as engineering tools Operate on analog quantities Analog input, analog output tt

July 23, Introduction When using NN as engineering tools Operate on analog quantities Analog input, analog output Spiking neural networks: More powerful than classic NN Interesting to implement in hardware Problem: internally work with spike trains ≠ analog t t t t ED

July 23, Introduction When using NN as engineering tools Operate on analog quantities Analog input, analog output Spiking neural networks: More powerful than classic NN Interesting to implement in hardware Problem: internally work with spike trains ≠ analog Use spiking NN as engineering tool: Communication problem must be solved Convert spike trains into analog function: decoding Convert analog function into spike trains: encoding E D

July 23, Overview Introduction Reading the spiking language Speaking the spiking language Results Conclusions and further work

July 23, Reading the spiking language Has been studied thoroughly by neuroscientists Classic methods: Average rate: Analog value = sum of spikes in fixed time window of spike train Exact spike time, ea. Phase coding: Analog value = time from periodic event to first spike Stimulus estimation or filter coding: Analog output = filtered version of spike train Filter = low-pass

July 23, Speaking the spiking language How does it work: Filter decoding Filter f input output t t

July 23, Overview Introduction Reading the spiking language Speaking the spiking language Results Conclusions and further work

July 23, Speaking the spiking language Classic decoding: almost trivial encoding Filter decoding: not so simple Hough et al. introduced spike train encoding algorithm HSA (Hough’s spike algorithm) Tries to reverse the filtering decoding Based on a heuristic Deterministic Filter used for decoding is also used for encoding We use slightly different algorithm than the one used in original paper (allows threshold optimization)

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA Heuristic used by HSA: min(x,f)-f if heuristic > threshold then fire negative !

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA Bigger than threshold ≈

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA FIRE ! Subtract f from Input signal

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA Bigger than threshold

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA FIRE ! Subtract f from Input signal

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA Remaining error signalSpike train

July 23, Speaking the spiking language HSA problems: Better performance ? What must the threshold be ? What's the meaning of the heuristic ? Filters with negative taps not supported BSA: Basic algorithm same as HSA, but other heuristic Calculate two error functions Error 1 = mean error if we don’t fire a spike now Error 2 = mean error if we fire a spike now Mean absolute error (MSE possible but performs worse) Subtract, compare to threshold and according to this fire Time (samples) Value

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using BSA Heuristic used by BSA: |x|-|x-f| + -

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using BSA Heuristic used by BSA: |x|-|x-f| + -

July 23, Speaking the spiking language How does it work: output input t t Filter encoding using HSA + Heuristic used by BSA: |x|-|x-f| FIRE ! Bigger than threshold ≈

July 23, Speaking the spiking language What about the threshold: Actual threshold largely determines performance Original HSA publication: threshold = 0 SNR in dB threshold BSA Average over different test functions

July 23, Overview Introduction Reading the spiking language Speaking the spiking language Results Conclusions and further work

July 23, Results The filter used in the results has 24 taps 10 bit quantized coefficients Was used in original HSA publications All results: Input encoded into spikes Spikes decoded into output Difference between input and output Paper Unipolar and bipolar case Here only bipolar samples Impulse response amplitude

July 23, Results BSAHSA Normalized input frequency SNR (dB) Optimal threshold Fixed threshold Optimal threshold Original publication Normalized input frequency 5dB 8dB Input amplitude = 0.25

July 23, Results HSA and BSA are non-linear systems Not only characterized by SNR for different frequencies Also give SNR for different amplitudes

July 23, Results BSAHSA Input amplitude (dB) SNR (dB) Optimal threshold Fixed threshold Optimal threshold Original publication 5dB 10dB Dynamic range Overloading Input frequency =

July 23, Overview Introduction Reading the spiking language Speaking the spiking language Results Conclusions and further work

July 23, Conclusions and further work Threshold optimization: 5 dB increase for original HSA BSA: 15 dB increase over published HSA Frequency and amplitude characteristics are smoother Still very limited calculation overhead Delay introduced equal to filter length

July 23, Conclusions and further work Analyze why these algorithms work Even better algorithm possible ? Optimal ? How does the filter change the characteristics of the algorithms ? How does performance compare to other coding schemes ? Is there any biological relevance ?