Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 1 Computational Architectures in Biological Vision,

Slides:

Advertisements

Similar presentations

Visual Saliency: the signal from V1 to capture attention Li Zhaoping Head, Laboratory of natural intelligence Department of Psychology University College.

Advertisements

A Neural Model for Detecting and Labeling Motion Patterns in Image Sequences Marc Pomplun 1 Julio Martinez-Trujillo 2 Yueju Liu 2 Evgueni Simine 2 John.

Sparse Coding in Sparse Winner networks Janusz A. Starzyk 1, Yinyin Liu 1, David Vogel 2 1 School of Electrical Engineering & Computer Science Ohio University,

Human (ERP and imaging) and monkey (cell recording) data together 1. Modality specific extrastriate cortex is modulated by attention (V4, IT, MT). 2. V1.

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 1 Laurent Itti: CS564 - Brain Theory and Artificial Intelligence Lecture.

Universal Design for Learning October, What about reading? What part of the brain do we read with?

Attention I Attention Wolfe et al Ch 7. Dana said that most vision is agenda-driven. He introduced the slide where the people attended to the many weird.

Covert Attention Mariel Velez What is attention? Attention is the ability to select objects of interest from the surrounding environment Involuntary.

Attention Wolfe et al Ch 7, Werner & Chalupa Ch 75, 78.

Attention, Awareness, and the Computational Theory of Surprise Research Qualifying Exam August 30 th, 2006.

Read this article for Friday [1]Chelazzi L, Miller EK, Duncan J, Desimone R. A neural basis for visual search in inferior temporal cortex. Nature 1993;

September 7, 2010Neural Networks Lecture 1: Motivation & History 1 Welcome to CS 672 – Neural Networks Fall 2010 Instructor: Marc Pomplun Instructor: Marc.

Exam 1 week from today in class assortment of question types including written answers.

Michigan State University1 Visual Attention and Recognition Through Neuromorphic Modeling of “Where” and “What” Pathways Zhengping Ji Embodied Intelligence.

How does the visual system represent visual information? How does the visual system represent features of scenes? Vision is analytical - the system breaks.

CS564 – Lecture 21 Visual Attention Laurent Itti USC.

Organizational Notes no study guide no review session not sufficient to just read book and glance at lecture material midterm/final is considered hard.

Neural Networks Basic concepts ArchitectureOperation.

Pattern Recognition using Hebbian Learning and Floating-Gates Certain pattern recognition problems have been shown to be easily solved by Artificial neural.

Visual Attention More information in visual field than we can process at a given moment Solutions Shifts of Visual Attention related to eye movements Some.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 4: Introduction to Vision 1 Computational Architectures in Biological.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 6: Low-level features 1 Computational Architectures in Biological.

December 1, 2009Introduction to Cognitive Science Lecture 22: Neural Models of Mental Processes 1 Some YouTube movies: The Neocognitron Part I:

CS 664, Session 19 1 General architecture. CS 664, Session 19 2 Minimal Subscene Working definition: The smallest set of objects, actors and actions in.

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 1 CS 664, USC Spring 2002 Lecture 7. Visual Attention (top-down)

What is Cognitive Science? … is the interdisciplinary study of mind and intelligence, embracing philosophy, psychology, artificial intelligence, neuroscience,

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 1 Computational Architectures in Biological.

Reading. Reading Research Processes involved in reading –Orthography (the spelling of words) –Phonology (the sound of words) –Word meaning –Syntax –Higher-level.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 2: Neuroscience Basics 1 Computational Architectures in Biological.

Michigan State University 1 “Saliency-Based Visual Attention” “Computational Modeling of Visual Attention”, Itti, Koch, (Nature Reviews – Neuroscience.

Overview 1.The Structure of the Visual Cortex 2.Using Selective Tuning to Model Visual Attention 3.The Motion Hierarchy Model 4.Simulation Results 5.Conclusions.

Michael Arbib & Laurent Itti: CS664 – USC, spring Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:

What is Cognitive Science? … is the interdisciplinary study of mind and intelligence, embracing philosophy, psychology, artificial intelligence, neuroscience,

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 2: Neuroscience Basics 1 Computational Architectures in Biological.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 4: Introduction to Vision 1 Computational Architectures in Biological.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 5: Introduction to Vision 2 1 Computational Architectures in.

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 1 CS 664, USC Spring 2002 Lecture 5. Visual Attention (bottom-up)

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 13: Scene Perception 1 Computational Architectures in Biological Vision,

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 13: Scene Perception 1 Computational Architectures in Biological.

Computer Science Department, Duke UniversityPhD Defense TalkMay 4, 2005 Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis Nikos P.

Studying Visual Attention with the Visual Search Paradigm Marc Pomplun Department of Computer Science University of Massachusetts at Boston

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis Laurent Itti, Christof Koch, and Ernst Niebur IEEE PAMI, 1998.

Manipulating Attention in Computer Games Matthias Bernhard, Le Zhang, Michael Wimmer Institute of Computer Graphics and Algorithms Vienna University of.

Active Vision Key points: Acting to obtain information Eye movements Depth from motion parallax Extracting motion information from a spatio-temporal pattern.

Psych 216: Movement Attention. What is attention? Covert and overt selection appear to recruit the same areas of the brain.

2 2  Background  Vision in Human Brain  Efficient Coding Theory  Motivation  Natural Pictures  Methodology  Statistical Characteristics  Models.

Visual Attention Derek Hoiem March 14, 2007 Misc Reading Group.

黃文中 Introduction The Model Results Conclusion 2.

Assessment of Computational Visual Attention Models on Medical Images Varun Jampani 1, Ujjwal 1, Jayanthi Sivaswamy 1 and Vivek Vaidya 2 1 CVIT, IIIT Hyderabad,

Department of Psychology & The Human Computer Interaction Program Vision Sciences Society’s Annual Meeting, Sarasota, FL May 13, 2007 Jeremiah D. Still,

Street Smarts: Visual Attention on the Go Alexander Patrikalakis May 13, XXX.

Computer Science Readings: Reinforcement Learning Presentation by: Arif OZGELEN.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 12: Visual Attention 1 Computational Architectures in Biological.

Neural Models of Visual Attention John K. Tsotsos Center for Vision Research York University, Toronto, Canada Marc Pomplun Department of Computer Science.

Interneuron diversity and the cortical circuit for attention

Spatio-temporal saliency model to predict eye movements in video free viewing Gipsa-lab, Grenoble Département Images et Signal CNRS, UMR 5216 S. Marat,

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern.

Ahissar, Hochstein (1997) Nature Task difficulty and specificity of perceptual learning 1 st third 2nd third Final session Task difficulty Stimulus-to-mask.

Network Management Lecture 13. MACHINE LEARNING TECHNIQUES 2 Dr. Atiq Ahmed Université de Balouchistan.

March 31, 2016Introduction to Artificial Intelligence Lecture 16: Neural Network Paradigms I 1 … let us move on to… Artificial Neural Networks.

(A review by D.J. Kravitz et. al)

Attention to Orientation Results in an Inhibitory Surround in Orientation Space Acknowledgements Funding for this project was provided to MT through a.

A Neurodynamical Cortical Model of Visual Attention and Invariant Object Recognition Gustavo Deco Edmund T. Rolls Vision Research, 2004.

Implementation of a Visual Attention Model

Computer Vision Lecture 2: Vision, Attention, and Eye Movements

Artificial Neural Networks

The Network Approach: Mind as a Web

Reading Assignments: Lecture 16. Saccades 2 The NSL Book

Presentation transcript:

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 1 Computational Architectures in Biological Vision, USC Lecture 12. Visual Attention Reading Assignments: None

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 2

3

4 Several Forms of Attention Attention and eye movements: - overt attention (with eye movements) - covert attention (without eye movements) Bottom-up and top-down control: - bottom-up control based on image features very fast (up to 20 shifts/s) involuntary / automatic - top-down control may target inconspicuous locations in visual scene slower (5 shifts/s or fewer; like eye movements) volitional Control and modulation: - direct attention towards specific visual locations - attention modulates early visual processing at attended location

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 5 What is attention then? Attention is often described as an information processing bottleneck. Controls access to higher levels of processing, short-term memory and consciousness. Hence, the strategy nature has developed to cope with information overload is to break down the problem of analyzing a visual scene: from a massively parallel approach to a rapid sequence of circumscribed recognitions.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 6

7

8

9

10

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 11

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 12

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 13

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 14

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 15

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 16 First Computational Model Koch & Ullman, Hum. Neurobiol., 1895 Introduce concept of a single topo- graphic saliency map. Most salient location selected by a winner-take-all network.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 17

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 18 Shifter Circuits Anderson & van Essen, PNAS, 1987 Information dynamically routed through cortical hierarchy. Yields rotation- and scale-independent representation.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 19 Shifter Circuits (cont.) Olshausen et al., J Neurosci, 1993 Implemented shifter circuits and demonstrated proof of concept. Control neurons in the pulvinar send the (attention-based) control signals that will determine the “passing” region of the circuit, through a modulation of intracortical connection weights. Perform recognition using associative memory at top level.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 20 only attended item reaches output layer

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 21 Selective Tuning Model Tsotsos et al., Artificial Intelligence, attention modulates neurons to earliest levels; wherever there is a many-to-one mapping many-to-one mapping - signal interference controlled by surround inhibition throughout processing network throughout processing network -task knowledge biases computations throughout processing network - attentional control is local, distributed and internal - competition is based on WTA (different form than previous models) (different form than previous models) - pyramid representation with reciprocal convergence and divergence neuron ‘sees’ this receptive field subject ‘attends’ to single item

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 22 The basic idea (BBS 1990)

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 23 Selective Tuning Model processing pyramid inhibited pathways pass pathways unit of interest at top input Caputo & Guerra 1998 Bahcall & Kowler 1999 Vanduffel, Tootell, Orban 2000 Smith et al Kastner, De Weerd, Desimone, Ungerleider, 1998

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 24

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 25 Guided Search Wolfe, Psychonomic Bull. & Rev., 1994 How can we combine information from several modalities? Use top-down (task-dependent) weighting.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 26

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 27

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 28

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 29

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 30

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 31

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 32

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 33

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 34

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 35

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 36

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 37

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 38

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 39

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 40

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 41

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 42

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 43 Image Compression

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 44 Evaluation of Advertising

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 45

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 46

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 47 Brefczynski & DeYoe, Nature Neuroscience 1999

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 48

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 49 Treue & Martinez-Trujillo, Nature 1999

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 50

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 51 Attentional Modulation in Humans Gandhi et al, 1999

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 52

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 53

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 54

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 55

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 56 Attentional Modulation Hernandez et al. Picture naming by bi-lingual persons. Increased attention/concentration due to increased difficulty when non-native tongue? No, same patterns of activation. But increased activation when Switching between languages, Probably reflecting increased Attentional load. Broca: speech generation Supramarginal: articulation & phonology processing Cingulate: emotion, memory, vigilance, attention?

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 57

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 58

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 59