A saliency map model explains the effects of random variations along irrelevant dimensions in texture segmentation and visual search Li Zhaoping, University College London, UK, V1’s output as a saliency map is viewed under the idealization of the top-down feedback to V1 being disabled, e.g., shortly after visual exposure or under anesthesia. Background: V1 produces a saliency map Signaling saliency regardless of features: Contrary to common beliefs, this does not mean that the cells reporting salience must be un-tuned to specific features. In other words, here “regardless of” means the following — in this saliency map, the meaning of firing rates for saliency is universal, and, given an input scene, the same firing rate from two V1 (output) neurons selective to different features mean the same salience value of the two corresponding inputs even if, say, one of the cells is color selective, responding to a static red bar, and the other cell is tuned to motion, responding to a moving black dot. Usually, an image item, say, a red short bar, evokes responses from many cells with different optimal features and overlapping tuning curves or receptive fields. The actual input features have to be decoded in a complex and feature specific manner from the population responses. However, locating the most responsive cell to a scene locates the most salient item whether or not features can be decoded beforehand or simultaneously from the same cell population. It is economical not to use subsequent cell layers (whether they are feature tuned or not) for a saliency map; the small receptive fields in V1 also mean that this saliency map can have a higher resolution. For more details, see “A saliency map in primary visual cortex” in Trends in Cognitive Sciences, Vol. 6, No.1 January 2002, p V1 model Input to model Model output Highlighting important image locations. These locations evoke stronger responses because they have fewer iso-orientation neighbors that suppress them and/or more co-linear neighbors that facilitate them. The V1 model is based on V1 physiology and anatomy (e.g., horizontal connections linking cells tuned to similar orientations), tested to be consistent with physiological data on contextual influences (e.g., iso-orientation suppression, Knierim and van Essen (1992) co- linear facilitation, Kapadia et al 1995). V1 processing Z = (S-S)/σ, z score, measuring saliencies of items Saliency of an item is assumed to increase with its evoked V1 response. We assume that efficiency of a visual search task increases with the salience of the target (or its most salient part, e.g., the horizontal bar in the target cross above). The high z score, z = 7, (of the horizontal bar), a measure of the cross’ salience, enables the cross to pop out, since its evoked V1 response (to the horizontal bar) is much higher than the average population response of the whole image. The cross has a unique feature, the horizontal bar, which evokes the highest response since it experiences no iso-orientation suppression while all distractors do. Hence, intra-cortical interaction is a neural basis for why feature searches are often efficient. Histogram of all responses S regardless of features Original input V1 response S S=0.2, z=1.0 S=0.4, z=7 S=0.12,z=-1.3 S=0.22, z=1.7 Stimulus Model V1 outputs Dominant response at each location All bars have the same input contrast. Each excites, with the same input strength, one color tuned cell (not orientation selective) and one orientation tuned cell (not color selective). Thickness of a black bar indicates the response magnitude from an corresponding orientation tuned cell, size of a colored dot indicates the response level from a color tuned cell. Size of a dot indicates the response level of the dominant cell at that location, color of the dot indicates color selectivity of the cell. Saliency of an item (cross) is determined by the most responsive or dominant cell (horizontal tuned cell) to the item Popout by orientnation Texture segmentation by orientation Observation by Snowden (J. Exp. Psychol. 1998) that random variations in color make texture segmentation by orientation difficult, but do not affect pop out by orientation. Difficult taskEasy task We assume that the easy of the tasks are determined by the saliencies of the pop-out target or the texture border. The saliency of a bar is determined by the response magnitude of the most responsive cell to that bar, whether it is color or orientation selective. In a single extended homogeneous texture of bars of the same orientation and color, all responding cells, whether color or orientation selective, have weak responses since they either experience iso-orientation suppression or iso-color suppression from neighboring cells of the same modality. Stimulus bars are such that the color tuned and orientation tuned cells have comparable responses. A single popout target (by orientation) in a uniform texture background induces a strongest activities (compared to other cells) in an orientation selective cell which experiences no iso-orientation suppression. This pops out the target. A texture border by orientation contrast between two homogenous textures also induces higher responses in the corresponding orientation tuned cells, which experience less iso-orientation suppression since they have fewer iso-orientation neighbors. This causes the border to popout. Random color variations in a texture make color tuned cells more responsive (than the case without color variations), since each color cell is having fewer iso-color neighbors to suppress it. The saliency of the texture bars are determined by the color tuned cells whose responses are stronger than the orientation tuned cells. A popout task is not affected by random color variations since the color tuned cells (responding to the background) do not have strong enough activities to match the orientation tuned cell responding to the target. This is because the orientation tuned cell experiences no iso-orientation suppression, while the color tuned cells still experience, albeit weaker, iso-color suppressions. Texture segmentation by orientation is strongly affected by random color variations since the orientation tuned cells responding to the texture border have comparable activity levels as the color tuned cells responding to background. Both cell groups experience comparable iso- feature suppression, since a texture bar at the border share the same orientation with half of its contextual neighbors while a texture bar anywhere also shares the same color with roughly half its contextual neighbors. This orientation tuned cell responds most since it experiences no iso-feature suppression that affects all other responding cells. Random color variations increases the activities of the color tuned cells which experience weaker iso-color suppression compared to situations of uniform color. Orientation tuned cells at the border experiences less iso- orietation suppression than other orientation tuned cells. Histogram of responses from orientation tuned cells. Histogram of responses from color tuned cells. Histogram of responses from the dominant cell at each location. homogeneous response to homogeneous color Inhomogeneous response to Inhomogeneous color homogeneous response to homogeneous color Inhomogeneous response to Inhomogeneous color Increase in mean and variance of the responses from color tuned cells. Popout of the border is prevented because responses to color swamp the responses to border. Increased responses to color is insignificant compared with the response to the popout target in an orientation tuned cell. Some bars induce stronger responses from orientation tuned cells, others induce stronger response from color tuned cells, depending on whether a bar has fewer iso-orientation or iso-color neighbors and how strongly active the iso-feature neighboring cells are. The saliency of a bar is determined by the stronger response, whether it is from the color or orientation tuned cell. Prediction: texture segmentation under random color variation can be made easier with thinner (and longer) texture bars, which induce weaker responses in color tuned cells, which are less sensitive to finer scales. These results are simulated in a V1 model which is constructed by augmenting my original V1 model, which has only orientation selective cells, with cells tuned to color but not tuned to orientation. Each color tuned cell suppresses its contextual neighbors tuned to the same color --- iso-color suppression. All cells also experience general, feature non-specific, suppression from their contextual neighbors (activity normalization). For simplicity, cells tuned to both color and orientation are omitted in the simulation. By the same argument, random variations in other feature dimensions should have similar effects.