Estimating mutual information Kenneth D. Harris 25/3/2015.

Slides:

Advertisements

Similar presentations

What is the neural code? Puchalla et al., What is the neural code? Encoding: how does a stimulus cause the pattern of responses? what are the responses.

Advertisements

EE 4780 Huffman Coding Example. Bahadir K. Gunturk2 Huffman Coding Example Suppose X is a source producing symbols; the symbols comes from the alphabet.

Beauty in the Eye of the Beholder The Relativity of Visual Experience Andrew Duggins Westmead Hospital, University of Sydney

Biological Modeling of Neural Networks: Week 9 – Coding and Decoding Wulfram Gerstner EPFL, Lausanne, Switzerland 9.1 What is a good neuron model? - Models.

Neural Computation Chapter 3. Neural Computation Outline Comparison of behavioral and neural response on a discrimination task –Bayes rule –ROC curves.

Spike Train Statistics Sabri IPM. Review of spike train  Extracting information from spike trains  Noisy environment:  in vitro  in vivo  measurement.

Spike Trains Kenneth D. Harris 3/2/2015. You have recorded one neuron How do you analyse the data? Different types of experiment: Controlled presentation.

Spike Sorting Goal: Extract neural spike trains from MEA electrode data Method 1: Convolution of template spikes Method 2: Sort by spikes features.

Artificial Spiking Neural Networks

Information Theory in Neuroscience Noise, probability and information theory MSc Neuroscience Prof. Jan Schnupp Noise, probability.

Shin Ishii Nara Institute of Science and Technology

The University of Manchester Introducción al análisis del código neuronal con métodos de la teoría de la información Dr Marcelo A Montemurro

How well can we learn what the stimulus is by looking at the neural responses? We will discuss two approaches: devise and evaluate explicit algorithms.

For stimulus s, have estimated s est Bias: Cramer-Rao bound: Mean square error: Variance: Fisher information How good is our estimate? (ML is unbiased:

Electrophysiology of Visual Attention. Moran and Desimone (1985) “Classical” RF prediction: there should be no difference in responses in these two conditions.

Spectral analysis Kenneth D. Harris 18/2/15. Continuous processes A continuous process defines a probability distribution over the space of possible signals.

For a random variable X with distribution p(x), entropy is given by H[X] = -  x p(x) log 2 p(x) “Information” = mutual information: how much knowing the.

Herding: The Nonlinear Dynamics of Learning Max Welling SCIVI LAB - UCIrvine.

CS 8751 ML & KDDEvaluating Hypotheses1 Sample error, true error Confidence intervals for observed hypothesis error Estimators Binomial distribution, Normal.

Today: Entropy Information Theory. Claude Shannon Ph.D

An exactly solvable maximum entropy model Peter Latham Gatsby Computational Neuroscience Unit UCL CNS July 20, 2006.

Spike Train decoding Summary Decoding of stimulus from response –Two choice case Discrimination ROC curves –Population decoding MAP and ML estimators.

Information Theory Kenneth D. Harris 18/3/2015. Information theory is… 1.Information theory is a branch of applied mathematics, electrical engineering,

Circular statistics Maximum likelihood Local likelihood Kenneth D. Harris 4/3/2015.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 7: Coding and Representation 1 Computational Architectures in.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

From T. McMillen & P. Holmes, J. Math. Psych. 50: 30-57, MURI Center for Human and Robot Decision Dynamics, Sept 13, Phil Holmes, Jonathan.

Modeling fMRI data generated by overlapping cognitive processes with unknown onsets using Hidden Process Models Rebecca A. Hutchinson (1) Tom M. Mitchell.

-Gaurav Mishra -Pulkit Agrawal. How do neurons work Stimuli  Neurons respond (Excite/Inhibit) ‘Electrical Signals’ called Spikes Spikes encode information.

1 Advanced Smoothing, Evaluation of Language Models.

Using Error-Correcting Codes For Text Classification Rayid Ghani Center for Automated Learning & Discovery, Carnegie Mellon University.

Tahereh Toosi IPM. Recap 2 [Churchland and Abbott, 2012]

沈致远. Test error(generalization error): the expected prediction error over an independent test sample Training error: the average loss over the training.

1 A(n) (extremely) brief/crude introduction to minimum description length principle jdu

Learning Structure in Bayes Nets (Typically also learn CPTs here) Given the set of random variables (features), the space of all possible networks.

Efficient Minimal Perfect Hash Language Models David Guthrie, Mark Hepple, Wei Liu University of Sheffield.

Statistical NLP: Lecture 8 Statistical Inference: n-gram Models over Sparse Data (Ch 6)

Regents Strategies: ELIMINATION Ruling out answers that we know are incorrect.

Projects: 1.Predictive coding in balanced spiking networks (Erwan Ledoux). 2.Using Canonical Correlation Analysis (CCA) to analyse neural data (David Schulz).

Synergy, redundancy, and independence in population codes, revisited. or Are correlations important? Peter Latham* and Sheila Nirenberg † *Gatsby Computational.

Business Skills Decision Making & Problem solving Negotiation Planning Evaluating Performance Business strategy Motivating talent Developing & communicating.

Web page: Textbook. Abbott and Dayan. Homework and grades Office Hours.

Original Goal: Info in multi-unit signals Measure effect of summing 2 neurons’ responses BB CC CC DD.

BCS547 Neural Decoding.

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

Lecture 4: Lossless Compression(1) Hongli Luo Fall 2011.

Psych 480: Fundamentals of Perception and Sensation

School of Computer Science 1 Information Extraction with HMM Structures Learned by Stochastic Optimization Dayne Freitag and Andrew McCallum Presented.

EFFECTS OF SPIKE SORTING ERROR ON INFORMATION CONTENT IN MULTI-NEURON RECORDINGS Deborah S. Won, David Y. Chong, and Patrick D. Wolf Department of Biomedical.

Cluster analysis and spike sorting

Bahareh Sarrafzadeh 6111 Fall 2009

Intro to NLP - J. Eisner1 A MAXENT viewpoint.

6. Population Codes Presented by Rhee, Je-Keun © 2008, SNU Biointelligence Lab,

CHARACTERIZATION OF NONLINEAR NEURON RESPONSES AMSC 664 Final Presentation Matt Whiteway Dr. Daniel A. Butts Neuroscience.

Confirmatory analysis for multiple spike trains Kenneth D. Harris 29/7/15.

July 23, BSA, a Fast and Accurate Spike Train Encoding Scheme Benjamin Schrauwen.

Signal Detection Theory.

Eizaburo Doi, CNS meeting at CNBC/CMU, 2005/09/21 Redundancy in the Population Code of the Retina Puchalla, Schneidman, Harris, and Berry (2005)

3 More Strategies:  Plug in Choices  Use Your Own Numbers  Multi-Variable Replacing.

Information Processing by Neuronal Populations Chapter 6: Single-neuron and ensemble contributions to decoding simultaneously recoded spike trains Information.

1 5. Representations and the neural code Lecture Notes on Brain and Computation Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science.

Neural Codes. Neuronal codes Spiking models: Hodgkin Huxley Model (brief repetition) Reduction of the HH-Model to two dimensions (general) FitzHugh-Nagumo.

The Neural Code Baktash Babadi SCS, IPM Fall 2004.

“In The Name of God” Presented by: Yoosef Najian An Introduction to Information Theory.

Decoding How well can we learn what the stimulus is by looking

Artificial Intelligence

What does the synapse tell the axon?

LECTURE 23: INFORMATION THEORY REVIEW

Neural Coding: Sparse but On Time

Cell Assemblies of the Superficial Cortex

Presentation transcript:

Estimating mutual information Kenneth D. Harris 25/3/2015

Entropy

Mutual information

“Plug in” measure

No information

Bias correction methods Not always perfect Only use them if you truly understand how they work! Panzeri et al, J Neurophys 2007

Cross-validation Mutual information measures how many bits I save telling you about the spike train, if we both know the stimulus Or how many bits I save telling you the stimulus, if we both know the spike train We agree a code based on the training set How many bits do we save on the test set? (might be negative)

Strategy Codeword length when we don’t know stimulus Codeword length when we do know stimulus

This underestimates information Can show expected bias is negative of plug-in bias

Two choices: Predict stimulus from spike train(s) Predicted spike train(s) from stimulus

Predicting spike counts Likelihood ratio

Unit of measurement “Information theory is probability theory with logs taken to base 2” Bits / stimulus Bits / second (Bits/stimulus divided stimulus length) Bits / spike (Bits/second divided mean firing rate) High bits/second => dense code High bits/spike => sparse code.

Bits per stimulus and bits per spike 1 bit if spike 1 bit if no spike 1 bit/stimulus.5 spikes/stimulus 2 bits/spike

Measuring sparseness with bits/spike Sakata and Harris, Neuron 2009

Continuous time Itskov et al, Neural computation 2008

Likelihood ratio

Predicting firing rate from place Harris et al, Nature 2003

Comparing different predictions Harris et al, Nature 2003