1. Thomas J. Deerinck Digital Image Gallery Purkinje Neurons and Glia of Rat Cerebellum. Specimen: Double fluorescent labeled thin section Technique: Two-Photon Microscopy

2. Adaptive Filter Model PC implements linear-in-weights adaptive filter Model of Marr-Albus type: mossy fibre input analysed by granule cell layer re-synthesised by Purkinje cells output given by weighted sum

3. Learning Rule Weights are adapted during complex spikes generated by input on climbing fibre learning rule models both LTP and LTD at synapse identical to LMS rule of adaptive control theory learning stops when CF and PF inputs are uncorrelated - “ decorrelation-control”

4. Motor Error Problem: 3D-VOR e horizontal e vertical e torsional uncalibrated VOR produces retinal slip sensory error motor error ?

5. Forward Architecture

6. Open loop - full connectivity Schematic VOR: Cerebellar output supplies corrections to 6 motor commands to muscles –cells controlling e.g. superior oblique muscle (SO) must know their contribution to error –so-called ‘motor error’ –it is a combination of different components of sensory error –use of motor error requires a complex ‘reference’ structure

7. Motor Error Problem Open loop forms basis of previous computational accounts –learning requires unavailable motor error signal –hypothesised neural ‘reference’ structure recovers motor error from sensory signals –climbing fibres carry motor error signal but –reference structure is an approximate plant inverse –has similar complexity to plant compensation problem it solves & gets worse for non-linear problems –uncertain experimental evidence for motor error signal on climbing fibres

8. For mathematical details see: Porrill, Dean & Stone, 2004 Recurrent Architecture

9. In recurrent loop cerebellar output corrects control inut: –sensor error is a suitable teaching signal –no need for reference structures to compute motor error –decorrelates sensor error from motor command –greatly simplifies modular, task-based, control Dean, Porrill & Stone, Proc Roy Soc B, 2002 Recurrent Architecture: VOR

10. Analysis of Recurrent Loop sum-square synaptic weight error is a Lyapounov function (decreases during learning): gives stability proof near solution learning rule is gradient descent on mean square output error (generalisation of output error learning property of all-pole adaptive filters; e.g. Sastry, Adaptive Control, 1989)

11. Connections to Anatomy Cerebellum is often embedded in a closed loop circuit (Eccles called this the ‘dynamic loop’). Recent investigations have confirmed that these loops are ubiquitous –“multiple closed loop circuits represent a fundamental architectural feature of cerebrocerebellar interactions” and highly specific –“regions of the cerebellar cortex that receive input from M1 are the same as those that project to M1” a challenge: –“A common closed-loop architecture describes the organisation of cerebrocerebellar interactions... A challenge for future studies is to determine the computations that are supported by this architecture” Kelly RM & Strick PL, Journal of Neuroscience (2003)

12. Effect of Delay There is up to 100ms delay in calculating retinal slip –well known that this restricts useful frequency range for error feedback –delayed feedback causes instability neuroscience ‘folk theorem’ says we can sidestep this –just use retinal slip as a teaching signal for an adaptive controller not so well known that the resulting learning rule can be unstable

13. No slip delay: plant compensation 0246810 -2 0 1 2 3 VOR response time (s) angular velocity (deg/s) retinal slip (pre) retinal slip (post) 012345 0 0.5 1 1.5 Step response time (s) angular velocity (deg/s) eye velocity (pre) eye velocity (post) 0246810 -2 0 1 2 3 VOR response time (s) angular velocity (deg/s) retinal slip (pre) retinal slip (post) 012345 0 0.5 1 1.5 Step response time (s) angular velocity (deg/s) eye velocity (pre) eye velocity (post) 0246810 -2 0 1 2 3 VOR response time (s) angular velocity (deg/s) retinal slip (pre) retinal slip (post) 012345 0 0.5 1 1.5 Step response time (s) angular velocity (deg/s) eye velocity (pre) eye velocity (post) 0246810 -2 0 1 2 3 VOR response time (s) angular velocity (deg/s) retinal slip (pre) retinal slip (post) 012345 0 0.5 1 1.5 Step response time (s) angular velocity (deg/s) eye velocity (pre) eye velocity (post) 0246810 -2 0 1 2 3 VOR response time (s) angular velocity (deg/s) retinal slip (pre) retinal slip (post) 012345 0 0.5 1 1.5 Step response time (s) angular velocity (deg/s) eye velocity (pre) eye velocity (post) 0246810 -2 0 1 2 3 VOR response time (s) angular velocity (deg/s) retinal slip (pre) retinal slip (post) 012345 0 0.5 1 1.5 Step response time (s) angular velocity (deg/s) eye velocity (pre) eye velocity (post) 0246810 -2 0 1 2 3 VOR response time (s) angular velocity (deg/s) retinal slip (pre) retinal slip (post) 012345 0 0.5 1 1.5 Step response time (s) angular velocity (deg/s) eye velocity (pre) eye velocity (post) 10 10 0 1 -10 -5 0 5 Bode plot (gain of VOR) Frequency (Hz) Gain (dB) pre-training post-training 0246810 -2 0 1 2 3 VOR response time (s) angular velocity (deg/s) retinal slip (pre) retinal slip (post) Good plant compensation with a single site of plasticity - parallel-fibre/Purkinje cell but no retinal slip delay

14. Slip Delay 100 ms: Unstable Learning Learning becomes unstable, because the sign of the correlation with the delayed teaching signal reverses when the frequency reaches 2.5 Hz

15. Multiple Sites of Plasticity Experimental data on VOR adaptation suggests two sites of plasticity –in the cerebellum –but also in brainstem nuclei to which the cerebellum projects If cerebellum is so powerful why is a second site needed? -Cerebellum cannot learn high frequency gain because of retinal slip delay -can output of cerebellum act as surrogate training signal for structures to which it projects? this hypothesis suggests novel learning rules for the brainstem neurons involved in VOR

16. B g x(t) z(t) cerebellar cortex head velocity + - Proposed Brainstem Plasticity Rule The correlation between cerebellar output z(t) and the head velocity signal x(t) in the range 1.5 to 2.5 Hz is used to adjust the intrinsic gain g of the brainstem  g = -  b BP BP = bandpassed above 2 Hz

17. Brainstem Plasticity Improves Learning cerebellar cortex learns average gain just below cut-off learning rule then transfers cerebellar gain to brainstem –then gain is approx correct at higher frequencies –can use ‘eligibility trace’ to improve on this

18. Electrophysiology Use in vitro recording techniques from neurons in MVN slices to investigate plastic changes induced in FTN neurons produced by correlated changes in –inhibitory cerebellar and –excitatory vestibular inputs. Experimental programme has three stages 1.identification of relevant neurons 2.use of conjunctive stimulation to characterise learning rules 3.investigation of the role of PC inputs by mimicking the action of their GABA neurotransmitters. Electrode patched onto MVN neuron. –Potential across membrane can be measured –Input current can be manipulated to mimic synaptic input

19. Non-conjunctive stimulation Intrinsic excitability of the MVN cells can be regulated by patterns of inhibitory inputs applied to the cells. A intrinsic excitability of MVN cells is increased following intermittent inhibitory hyperpolarising current injections. The increase in excitability is long-lasting, and can be mimicked by drugs that block specific potassium channels (data not shown) Drugs which open potassium channels reduce the intrinsic excitability of MVN neurons (data not shown) Regulation of potassium channel function is one important way in which the firing rate gain of the MVN neurons may be regulated B In contrast to the effects of hyperpolarising current inputs, depolarising current pulses applied in the same patterns do not affect the intrinsic excitability of MVN neurons (Fig.2B).

20. Robotics Algorithms are currently being transferred to neuro-controller implemented in FPGA hardware –particularly suitable for implementing real-time distributed algorithms

21. Mechatronic System for Algorithm Evaluation

22. Flexible Hardware/Software Solution Gimbal mounted 3dof camera provides near human performance custom built 3 degree of freedom head movement simulator (not shown) allows us to test algorithm performance three MEMS gyroscopes emulate the vestibular system visual feedback processing performed on neuro-controller Top: gimbal mounted camera with driver boards for roll/pitch/yaw control Right: BenNUEY stack, on top are two Virtex 11 2V8000 FPGA’s with their heat sinks Left: 7x7mm 1axis gyroscope

23. Next Steps Continue theoretical studies of algorithm performance extend modelling to other components of VOR circuitry conjunctive stimulation experiments hardware and software performance evaluation algorithm performance evaluation