Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 1 Computational Architectures in Biological Vision, USC Fall 2004 Lecture 1. Overview and Introduction Reading Assignments: Textbook: “Foundations of Vision,” Brian A. Wandell, Sinauer, Read Introduction and browse through book

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 2 Organization Lectures: Tuesdays, 5-7:50pm, GFS-107 Textbook: “Foundations of Vision,” by Brian A. Wandell, Sinauer Associates, Inc. ISBN Office hour: Mon 2-4pm in HNB-30A Homepage: under “classes.” Grading: 3 units. Based on project: - initial project proposal: 20%- 9/21/04 - midterm written progress report: 20%- 10/19/04 - final project writeup: 20%- 11/30/04 - project deliverable: 30%- 11/30/04 - final presentation: 10%- 11/30/04

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 3 Class Web Site Soon, you will find there: - Lecture notes: user “ilab” and password “2cool” - Reading assignments - Grades - General announcements - Project topics

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 4 Project ideas 1 – Auditory cueing to visually salient locations This potentially addresses the problem of building the next generation of machines to help the visually impaired. We will exploit the finding (Duncan et al., Nature, 1999) that visual and auditory attention may recruit two processing streams that can largely operate in parallel. Thus, we will supplement a person’s poor vision by using auditory cueing towards visually salient locations. Tasks:- learn about attention/saliency model and how to run it - learn about auditory source localization in space - implement a program that will generate various beeps and/or spoken words and give the listener the impression that they came from a specific location in space - interface the saliency computation to the new auditory code - test and demo

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 5 Project ideas 2 – Saliency-based video compression We can use attention/saliency to prioritize regions in video streams that should be encoded with best quality. See Itti, IEEE Trans Image Proc, In that paper, the prioritization simply used a variable amount of blur. A better way would be to go deeper into the workings of an MPEG4 codec. Tasks:- learn about how MPEG4 works - find some good open-source codec - implement an ROI facility in it - get the ROIs from the saliency program (easy) - test and demo

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 6 Project ideas 3 – Theoretical study: computational role of neurogenesis It has been recently shown that new neurons are created in the hippocampus (a brain region concerned with memory), more so when animals are exposed to rich environments. Interestingly, the more new cells are created, the faster the death of other cells seem to occur. Our goal is to understand whether this makes sense computationally. Tasks:- learn about Hopfield associative memories - what is the cost of adding a new memory? - what are the consequences of killing one neuron? - what would be the cost of adding a new memory if we could also add new neurons to achieve that? - deliverable is a paper/presentation answering these questions

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 7 Project ideas 4 – Theoretical study: role of feedback in object recognition When we have a hard time recognizing an object, we try harder. What exactly does this entail? Here we propose to study that as an optimization problem. That is, trying harder will alter the object recognition network so as to maximally disambiguate between the top two recognized candidates. Tasks:- learn about multilayer backprop networks - devise a cost function that differs from the standard backprop rule: while in backprop we want to minimize the overall difference between teaching signal and network output, in this project we want to maximize the difference between top and second output strengths - design a small backprop network for object recognition - show that the proposed feedback rule works in disambiguating recognition

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 8 Project ideas 5 – Landmark-based navigation Humans are very good at rapidly identifying which objects in their visual environment could serve as landmark points to describe a route. But what makes a good landmark? It should be rather unique, easy to find among clutter, rather permanent, maybe fairly viewpoint- invariant, etc. Let’s try to formalize those characteristics into a visual analysis system. Tasks:- assume you can detect salient points in an image - come up with various image analysis schemes to evaluate uniqueness, permanency, etc - write test code and conduct a demo

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 9 Project ideas 6 – Constraint-free eye/head tracker With a single webcam it should be possible to track eye/head movements of persons sitting in front of a computer screen, and to move the mouse accordingly. Tasks:- see what has been done in this research field (in particular at the von der Malsburg group) - estimate accuracy, given camera resolution, distance to observer, size of field of view, etc - if it turns out to be hopeless (too poor), consider a 2-camera system where a fixed wide-field camera finds the head and one eye, and a pan/tilt high-zoom narrow-field camera then tracks the pupil - demonstrate a working system

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 10 Project ideas 7 – statistical center-surround LGN neurons are thought to detect the difference between light levels in a central region of their receptive field and in a broader concentric surround region. What if things actually were more complicated and instead of a simple difference we computed a statistical test for how different the distributions of light intensity values are in the two regions? Tasks:- decide on a test - implement these new LGN neurons, as well as the old ones - find critical image stimuli where the old neurons would see nothing while the new ones would see a difference - expand to multiple scales - demo and writeup