2. Electrosensory data acquisition and signal processing strategies in electric fish Mark E. Nelson Beckman Institute Univ. of Illinois, Urbana-Champaign

3. How Electric Fish Work

4. Distribution of Electric Fish Fish tank upstairs black ghost knifefish elephant- nose fish

5. Electric Organ Discharge (EOD) - Spatial

6. EOD - Temporal

7. Electric Organ Discharge (EOD)

8. Principle of active electrolocation

9. mechano MacIver, from Carr et al., 1982 Electroreceptors ~15,000 tuberous electroreceptor organs 1 nerve fiber per electroreceptor organ up to 1000 spikes/s per nerve fiber

10. Individual Sensors (Electroreceptors)  V IN nerve spikes OUT

11. Neural coding in electrosensory afferent fibers

12. Probability coding (P-type) afferent spike trains 00010101100101010011001010000101001010 P head = 0.333  P head  = 0.337 P head = 0.333

13. Principle of active electrolocation

14. Electrosensory Image Formation

17. Prey-capture video analysis

18. Prey capture behavior

19. Fish Body Model

20. Motion capture software

21. MOVIE: prey capture behavior

22. Electrosensory Image Reconstruction

23. Voltage perturbation at skin  : Estimating Daphnia signal strength electrical contrast prey volume fish E-field at prey distance from prey to receptor THIS FORMULA CAN BE USED TO COMPUTE THE SIGNAL AT EVERY POINT ON THE BODY SURFACE

24. MOVIE: Electrosensory Images

25. System Capabilities Electric fish can analyze electrosensory images to extract information on target direction (bearing) distance size shape composition (impedance)

26. Distance Discrimination

28. Shape Discrimination

30. Shape Generalization

31. Shape “completion”

32. Impedance Discrimination

33. How Do They Do It? Electric fish analyze dynamic 2D electrosensory images on the body surface to determine target direction, distance, size, shape and composition (impedance) Fish might perform an inverse mapping from 2D sensor data to obtain a dense 3D neural representation of world conductivity sensor data  3D conductivity  action Alternatively, fish might use sensor data to directly estimate target parameters sensor data  target parameters  action

34. Parameter estimation (bearing)

35. Parameter Estimation (cont.)

36. Dynamic Movement Strategies Fish are constantly in motion not a single, static ‘snapshot’ dynamic, spatiotemporal data stream With respect to target objects in the environment, fish body movements simultaneously influence the relative positioning of the sensor array the electric organ effector organs (e.g. mouth)

37. MOVIE: Electrosensory Images

38. Active motor strategies: Dorsal roll toward prey

39. Probing Motor Acts chin probing back-and-forth (va et vient ) lateral probing tangential probing stationary probing

40. Fish exploring a 4 cm cube

41. CNS Signal Processing Strategies Multi-scale filtering spatial and temporal Adaptive background subtraction tail-bend suppression Attentional ‘spotlight’ mechanisms local gain control

42. Multiple Maps

43. Multi-scale Filtering INPUT (from skin receptors) Centromedial map High spatial acuity Low temporal acuity Centrolateral map Inter spatial acuity Inter temporal acuity Lateral map Low spatial acuity High temporal acuity temporal integration both spatial integration HINDBRAIN PROCESSING PERIPHERAL SENSORS

44. Adaptive Background Subtraction

46. Attentional ‘spotlight’ mechanism

47. Summary Fish can evaluate direction, distance, size, shape and composition of target objects How? model-based parameter estimation based on 2D image analysis, not full 3D reconstruction presumably some sort of (adaptive) (extended) (unscented) Kalman-like algorithm extensive pre-filtering (virtual sensors?)  self-calibrating, adaptive noise suppression, multi- scale spatial and temporal signal averaging dynamic control of source and array position

48. Acknowledgements Colleagues Curtis Bell (OHSU) Len Maler (Univ. Ottawa) Gerhard von der Emde (Univ. Bonn) Nelson Lab Members Ling Chen, Rüdiger Krahe, Malcolm MacIver Funding Agencies NIMH, NSF