Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign

How Electric Fish Work

Distribution of Electric Fish Fish tank upstairs black ghost knifefish elephant- nose fish

Electric Organ Discharge (EOD) - Spatial

EOD - Temporal

Electric Organ Discharge (EOD)

Principle of active electrolocation

mechano MacIver, from Carr et al., 1982 Electroreceptors ~15,000 tuberous electroreceptor organs 1 nerve fiber per electroreceptor organ up to 1000 spikes/s per nerve fiber

Individual Sensors (Electroreceptors)  V IN nerve spikes OUT

Neural coding in electrosensory afferent fibers

Probability coding (P-type) afferent spike trains P head =  P head  = P head = 0.333

Principle of active electrolocation

Electrosensory Image Formation

Prey-capture video analysis

Prey capture behavior

Fish Body Model

Motion capture software

MOVIE: prey capture behavior

Electrosensory Image Reconstruction

Voltage perturbation at skin  : Estimating Daphnia signal strength electrical contrast prey volume fish E-field at prey distance from prey to receptor THIS FORMULA CAN BE USED TO COMPUTE THE SIGNAL AT EVERY POINT ON THE BODY SURFACE

MOVIE: Electrosensory Images

System Capabilities Electric fish can analyze electrosensory images to extract information on target direction (bearing) distance size shape composition (impedance)

Distance Discrimination

Shape Discrimination

Shape Generalization

Shape “completion”

Impedance Discrimination

How Do They Do It? Electric fish analyze dynamic 2D electrosensory images on the body surface to determine target direction, distance, size, shape and composition (impedance) Fish might perform an inverse mapping from 2D sensor data to obtain a dense 3D neural representation of world conductivity sensor data  3D conductivity  action Alternatively, fish might use sensor data to directly estimate target parameters sensor data  target parameters  action

Parameter estimation (bearing)

Parameter Estimation (cont.)

Dynamic Movement Strategies Fish are constantly in motion not a single, static ‘snapshot’ dynamic, spatiotemporal data stream With respect to target objects in the environment, fish body movements simultaneously influence the relative positioning of the sensor array the electric organ effector organs (e.g. mouth)

MOVIE: Electrosensory Images

Active motor strategies: Dorsal roll toward prey

Probing Motor Acts chin probing back-and-forth (va et vient ) lateral probing tangential probing stationary probing

Fish exploring a 4 cm cube

CNS Signal Processing Strategies Multi-scale filtering spatial and temporal Adaptive background subtraction tail-bend suppression Attentional ‘spotlight’ mechanisms local gain control

Multiple Maps

Multi-scale Filtering INPUT (from skin receptors) Centromedial map High spatial acuity Low temporal acuity Centrolateral map Inter spatial acuity Inter temporal acuity Lateral map Low spatial acuity High temporal acuity temporal integration both spatial integration HINDBRAIN PROCESSING PERIPHERAL SENSORS

Adaptive Background Subtraction

Attentional ‘spotlight’ mechanism

Summary Fish can evaluate direction, distance, size, shape and composition of target objects How? model-based parameter estimation based on 2D image analysis, not full 3D reconstruction presumably some sort of (adaptive) (extended) (unscented) Kalman-like algorithm extensive pre-filtering (virtual sensors?)  self-calibrating, adaptive noise suppression, multi- scale spatial and temporal signal averaging dynamic control of source and array position

Acknowledgements Colleagues Curtis Bell (OHSU) Len Maler (Univ. Ottawa) Gerhard von der Emde (Univ. Bonn) Nelson Lab Members Ling Chen, Rüdiger Krahe, Malcolm MacIver Funding Agencies NIMH, NSF