Effects of V1 surround modulation tuning on visual saliency and the tilt illusion

Keemink SW, Boucsein C, van Rossum MC. Effects of V1 surround modulation tuning on visual saliency and the tilt illusion. J Neurophysiol 120: 942–952, 2018. First published May 30, 2018; doi:10.1152/jn.00864.2017.—Neurons in the primary visual cortex respond to oriented stimuli placed in the center of their receptive field, yet their response is modulated by stimuli outside the receptive field (the surround). Classically, this surround modulation is assumed to be strongest if the orientation of the surround stimulus aligns with the neuron’s preferred orientation, irrespective of the actual center stimulus. This neuron-dependent surround modulation has been used to explain a wide range of psychophysical phenomena, such as biased tilt perception and saliency of stimuli with contrasting orientation. However, several neurophysiological studies have shown that for most neurons surround modulation is instead center dependent: it is strongest if the surround orientation aligns with the center stimulus. As the impact of such center-dependent modulation on the population level is unknown, we examine this using computational models. We find that with neuron-dependent modulation the biases in orientation coding, commonly used to explain the tilt illusion, are larger than psychophysically reported, but disappear with center-dependent modulation. Therefore we suggest that a mixture of the two modulation types is necessary to quantitatively explain the psychophysically observed biases. Next, we find that under center-dependent modulation average population responses are more sensitive to orientation differences between stimuli, which in theory could improve saliency detection. However, this effect depends on the specific saliency model. Overall, our results thus show that center-dependent modulation reduces coding bias, while possibly increasing the sensitivity to salient features.


INTRODUCTION
Neurons in the primary visual cortex (V1) of mammals respond to stimuli in the center of their receptive field (RF).
Although stimuli that are in the surround outside the RF do not cause a response by themselves, they can strongly modulate the response, as has been shown by stimulation with centersurround grating pairs (Fig. 1A) (Blakemore and Tobin 1972;Freeman et al. 2001;Fries et al. 1977;Gilbert and Wiesel 1990;Girman et al. 1999;Jones et al. 2001;Maffei and Fiorentini 1976;Nelson and Frost 1978;Seriès et al. 2003;Shushruth et al. 2012;Sillito and Jones 1996).Such surround modulation is thought to underlie many perceptual phenomena, such as contrast perception of center-surround gratings (Shushruth et al. 2013), the saliency of a differently oriented bar among a background of identically oriented bars (Li 1999;Petrov and McKee 2006;Sillito et al. 1995) and contour integration (Keemink and van Rossum 2016;Li 1998).Furthermore, surround modulation has been proposed to underlie the tilt illusion in which the center orientation is misjudged in the presence of a surround grating (Clifford et al. 2000;Keemink and van Rossum 2016;Qiu et al. 2013;Schwartz et al. 2009).Crucially, the vast majority of studies investigating the functional consequences of surround modulation assume that modulation depends on the angular difference between the orientation of the surround stimulus and the preferred orientation of the neuron.We term this type of modulation "neuron-dependent modulation" (Fig. 1B).
However, a number of experimental studies has found that when both the center and surround orientations are varied, only a minority of neurons is modulated this way.For the majority of V1 neurons the modulation depends instead on the difference between the orientation of the surround stimulus and the center stimulus (Cavanaugh et al. 2002a;Shushruth et al. 2012;Sillito et al. 1995).This has previously been succinctly expressed as "The surround maximally suppresses responses to what the center sees, not to what the center prefers" (Cavanaugh et al. 2002a).We term this form of modulation "center-dependent modulation" (Fig. 1C).Center-dependent modulation also emerges naturally from Bayesian models of perception (Lochmann and Deneve 2011;Lochmann et al. 2012) and neural models trained by image statistics (Coen-Cagli et al. 2012), but despite being a well-known phenomenon, the functional consequences of center-dependent modulation are largely unknown.
In this study we compare neuron-dependent to center-dependent modulation using a phenomenological model of V1 in which the surround modulation tuning can be set to either Address for reprint requests and other correspondence: S. W. Keemink, Institute for Adaptive and Neural Computation, School of Informatics, Univ. of Edinburgh, Edinburgh EH8 9AB, UK (e-mail: swkeemink@scimail.eu).variant, without affecting other properties of the model, so that any functional difference can be solely attributed to the difference in modulation tuning.First, we consider the response to center-surround stimuli and reproduce the well-known result that neuron-dependent modulation leads to a bias in the decoding of the center orientation.This bias has been interpreted as a possible neurophysiological correlate of the tilt illusion.In contrast, we find that center-dependent modulation yields an unbiased representation of the center orientation (i.e., no tilt illusion).A mixed population with both modulation types quantitatively explains the psychophysically observed tilt illusion magnitude.
Second, Shushruth et al. (2012) predicted that center-dependent modulation might lead to increased relative responses to salient features (e.g., a single deviant bar among a field of homogeneous bars).We test this by examining the effect of surround modulation on the coding and saliency of fields of bars.As the exact mechanism for saliency detection is unknown, we consider two extremes: the saliency of a bar either depends on its mean population response relative to that of other bars, or on its maximum population response relative to that of other bars.With the former saliency computation, center-dependent modulation leads to a higher saliency signal than neuron-dependent modulation.With the latter, neuron-dependent modulation leads to slightly higher saliency.Thus center-dependent modulation could indeed potentially lead to a stronger saliency signal than neurondependent modulation, but this depends on the exact saliency computation.

Encoding Model
Center surround model.To examine the functional differences between neuron-dependent and center-dependent surround modulation, we compare two phenomenological models consisting of N ϭ 32 neurons with preferred orientations equally spaced in the interval [0, ].Presented with just a center grating, a neuron's firing rate as a function of the orientation of the center stimulus, c , is modeled by a von Mises function (von Mises 1918) where i is the neuron's preferred orientation, A c is the peak firing rate (set to 20 Hz) which is reached when the center equals the preferred orientation ( i ϭ c ), and k c determines the tuning width.
Presented with both a center and surround stimulus the response f() is modeled as where ref is the reference orientation (see below), s is the surround orientation, and h() models multiplicative modulation (Cavanaugh et al. 2002b).As surround modulation is typically suppressive at medium and high contrast (Shushruth et al. 2012), we model it as where A s (0 Յ A s Յ 1) determines the modulation strength, and k s sets the surround modulation tuning width.The strongest suppression is 1 -A s .We fitted the parameters to the average normalized tuning and modulation curves in figure 3D of Cavanaugh et al. (2002a) in the region ϭ -⁄2...⁄2, which yielded A s ϭ 0.5, k c ϭ 0.6, k s ϭ 0.5.Note that the center and surround tuning widths are quite similar in those data; however, this is not required for our findings.
Crucially, ref in the modulation function h() can be set to either 1) the neuron's preferred orientation ( ref ϭ i ), reflecting neuron-dependent modulation; or 2) the center orientation ( ref ϭ c ), reflecting center-dependent modulation.
To test if the results hold more generally, in addition to this multiplicative modulation we also examined a model with subtractive modulation (i.e., f ϭ gh), as has been observed in a minor fraction of neurons (Cavanaugh et al. 2002b).This gave qualitatively similar results (not shown).
In some cases noise was introduced by modeling the observed neural spike count as a Poisson processes with a rate given by the tuning functions f(), such that r i ϭ Poisson (f i T).The observation time T was set to 0.5 s unless indicated otherwise.Qualitatively, the results extend to Gaussian additive and multiplicative noise models (not shown).
Using an explicit model as above allows for both qualitative and quantitative analysis.It is phenomenological, and includes neither biophysical mechanisms nor any dynamics, making it easy and intuitive to analyze.However, in the visual cortex surround modulation is mediated through other neurons, thus involving recurrent interactions.Indeed, Shushruth et al. (2012) demonstrated that center-dependent Fig. 1.Two types of V1 surround modulation.A: center-surround stimuli consisting of a center grating covering the neuron's classical receptive field and a larger grating covering the surround.B: responses of an example cell with neuron-dependent modulation.Modulation is strongest when the surround matches the neuron's preferred orientation (stars indicate the strongest suppression for each center orientation).The height of the boxes is proportional to the neural response."Opt" corresponds to the neuron's preferred center orientation, "Orth" to the orthogonal orientation, and "Sub" and "Wk" correspond to 2 intermediate orientations.C: same as B, but for the more common case of a neuron with center-dependent modulation.Here the stars shift with the center orientation, indicating that suppression is strongest when the center and surround stimulus orientations are aligned.Data in B and C are courtesy of Shushruth and Angelucci (Shushruth et al. 2012).
modulation can be achieved through strong local recurrent connections.However, in such a model it is not possible to clearly dissociate the modulation strength from its degree of center dependency.Using the phenomenological model allows us to study the effect of centerdependent modulation in isolation, without needing to resort to lower level recurrent models.
Encoding configurations of multiple bars.The above model describes the response of a population of neurons with identical receptive field locations to a center-surround grating pair.To encode stimuli consisting of arbitrary fields of bars, we extend the model so that a neuron's surround modulation is a product composed of the modulation from the surrounding bars.The modulation is assumed distancedependent so that far away bars have little influence.The response of neuron i at 2D location x with orientation i , surrounded by K bars at locations y k is where c is a length scaling factor set to one, the product is over all other bars, and k is the orientation of bar k.As before, ref is either the preferred orientation of neuron i at location x, or the presented bar orientation at location x.

Decoding Models
Population vector decoding.The population vector is given by the sum of the neurons' preferred orientation vectors weighted by their firing rate (Georgopoulos et al. 1986;Schwartz et al. 2009), where r i is the firing rate, and u i ϭ (sin2 i ,cos2 i ) is the unit vector pointing in neuron i's preferred orientation (multiplied by 2 to ensure circularity).The estimated center orientation ˆc follows from the angle of the population vector ˆc ϭ 1 2
In the absence of surround stimulation, the estimated center orientation is unbiased.Symmetry and circularity arguments yield that any orientation tuning curve g() that is symmetric around its preferred orientation, i.e., a function of | i Ϫ c | only, yields a bias-free estimator.This can be shown explicitly by using that for dense coding with many neurons, Eq. 2 can be written as where f(x) is an arbitrary function symmetric around 0 with periodicity in 2, and u ϭ (sin2,cos2) T is the unit vector with angle .
We make the substitution x ϭ -c such that v ˆc ϭ ͐ 0 f͑2x͒u xϩ c dx, where due to the circularity of f(2x) the integral limits are unaltered.Considering the first vector element of v ˆc , where v c ϭ (sin2 c ,cos2 c ) T is the vector representation of the center orientation.Hence, unsurprisingly, it is unbiased.
Population vector decoding of center-surround stimuli.Next, we derive the bias in the presence of a surround grating.For neurondependent surround modulation the estimated center orientation vector can be written as ) is a scalar that does not affect the amount of bias, I 1 is the Bessel function of the first kind, and biases the decoded orientation, where v s ϭ (sin2 s ,cos2 s ) T is the unit vector associated to the surround.The bias is absent only when the surround is parallel or orthogonal to the center.To our knowledge no such exact expression for the population vector for von Mises tuning curves was published before.
In contrast, for center-dependent surround modulation, the estimated center orientation vector becomes which is always unbiased.
Maximum likelihood decoding.With maximum likelihood decoding the likelihood of finding a particular population response is maximized over all possible stimuli to find the most likely stimulus where L indicates the log likelihood and r is the population response.

Under Poisson noise the log likelihood L͑rԽ
for dense tuning curves, and the stimulus dependent part of the log likelihood L͑rԽ c ͒ ϭ T ͚iϭ1 N r i log g͑ i , c ͒.In the limit of low noise, so that there are no secondary maxima (Xie 2002), the estimated center orientation ˆc can be found by setting the derivative of the likelihood with respect to c to zero, resulting in ˆc ϭ arctan which is the angle of the population vector, Eq. 2. Hence at low noise the naive ML decoder and the population vector decoder give fully identical results.
To decode the full center-surround stimulus we use an ML decoder which decodes both the center and surround orientations.Under this condition ͚iϭ1 N f͑ i , c , s ͒ is no longer approximately constant, and the stimulus dependent part of the log likelihood becomes To find the estimate of both center and surround orientation, ͑ ˆc, ˆs͒, the likelihood needs to be maximized with respect to both c and s .As to our knowledge there is no closed expression for its solution, we maximized the log likelihood numerically, starting from different initial conditions for the estimated stimulus to avoid local maxima.We generated several noisy realizations of the population response r, and used gradient descent to find the stimulus pair which maximized the likelihood.

Computer Simulations
All data analysis and models were implemented in Python 2.7.13, using the Numpy 1.13.1 and SciPy 0.19.1 toolboxes.The figures were plotted using the HoloViews toolbox (Stevens et al. 2015).

RESULTS
To examine the functional differences between neurondependent and center-dependent surround modulation, we compare two phenomenological models of V1, with parameters constrained by experimental data (METHODS).The response of a neuron is modeled as the combination of the tuning curve for the orientation at the center, g(), and a multiplicative modulation term h().Apart from the surround orientation s , the modulation term depends either on the center orientation c , or on the preferred orientation of the neuron i .This leads to two subtly different tuning curve variants: These two variants can be seen as two extreme types of modulation; intermediate forms have also been observed (Shushruth et al. 2012).
The single neuron responses of the two models are illustrated in Fig. 2. For neuron-dependent modulation the suppression depends on the surround orientation and the preferred orientation, and thus is strongest at the preferred orientation (Fig. 2A).In contrast, for the center-dependent model, the modulation is strongest whenever center and surround orientations are aligned (Fig. 2B).

Tilt Illusion and Center Orientation Decoding
To examine the effect of the modulation type on coding biases, we decode the center orientation from the population response.Whereas many population coding studies concern the decoding accuracy, i.e., the trial-to-trial variation and its relation to the neural noise model (e.g., Shamir 2014), here we are interested in the biases in decoding, that is, the systematic mis-estimation of the stimulus that remains after averaging over many trials (Cortes et al. 2012;Keemink et al. In press;Seriès et al. 2009).
A well-known perceptual bias is the tilt illusion (e.g., Clifford 2014 ;Westheimer 1990).In the tilt illusion the perceived orientation of a center grating is influenced by the presence of a surround grating.For small and intermediate angles the tilt illusion is repulsive, and for larger angles the illusion becomes weakly attractive.A shift in the population response under neuron-dependent modulation has been hypothesized to underlie the tilt illusion (Clifford et al. 2000;Qiu et al. 2013;Schwartz et al. 2009).
First, we decode the center orientation from the population response using the population vector decoder (see METHODS; Georgopoulos et al. 1986;Schwartz et al. 2009).We present a center grating and a range of surround orientations and decode the center orientation.The biases for both models are plotted against surround orientation (Fig. 3A).As expected and in line with previous models, we find that the neuron-dependent model has a strong repulsive bias.However, with centerdependent modulation this bias completely disappears.
These results can be understood from how the two modulation types affect the population response (Fig. 4).Neurondependent modulation depends on the preferred orientation of each neuron, thus modulating each neuron differently and shifting the population response away from the surround orientation, resulting in a repulsive illusion (Fig. 4, solid curves in the middle and bottom rows).Center-dependent modulation instead depends on the center stimulus, irrespective of the preferred orientation, thus modulating each neuron equally.This reduces, but does not shift, the population response, resulting in zero bias (Fig. 4, dashed curves in the middle and bottom rows).The absence of bias extends to all models where surround modulation is felt equally across neurons, whether the surround modulation is multiplicative, subtractive, or some combination of both, that is, for all models of the form where k() is a function describing subtractive modulation.
The bias for neuron-dependent modulation can be calculated exactly.The population vector v ˆc ndep is a function of both the center stimulus (represented by unit vector v c ) and the surround stimulus (METHODS, Eq. 3), where ␤ is a positive number.The shift vector it points at v c ). Due to the minus sign in Eq. 7, the vector v ˆc fix is repelled from v shift .The amount of repulsion, given by ␤ and v shift , depends on the modulation strength A s , as well as the tuning widths, as given by k c and k s .The bias increases with decreasing k c (broader tuning curves) and increasing k s (sharper surround modulation).
The above results use the population vector, which does not take the effect of the surround modulation into account.To get a better idea of what the population can in principle encode we use the maximum likelihood (ML) decoder, which finds the stimulus that most likely caused the observed response (Kay 1993;Pilarski and Pokora 2015;Xie 2002).When the ML decoder is constructed such that it is naive to the effects of surround modulation, it performs identically to the population vector (see METHODS).Next, we implement an ML decoder that takes the full encoding model into account and infers both the center and surround orientation from the population response.
The ML decoder is applied to responses drawn from a Poisson process (see METHODS, Eq. 4).The estimate ͑ ˆc, ˆs͒ is found by maximizing the likelihood with respect to both angles, and the corresponding center biases is given by b c ϭ ͗ ˆc͘ Ϫ c .For neuron-dependent modulation the ML decoder bias is still present, but reduced compared with the population vector decoder (Fig. 3B).The average bias across surround orientations is shown as a function of stimulus duration in Fig. 3C, assuming perfect temporal integration by the decoder.In the limit of very long measurement time (i.e., zero noise) the likelihood landscape becomes very steep and is maximal when ͑ ˆc, ˆs͒, ϭ ( c , s ), i.e., the estimate equals the true value and hence the bias is zero for both neuron-and center-dependent modulation.In contrast, the bias of the population vector decoder is independent of stimulus duration (gray curves).For center-dependent modulation the estimates are again bias-free, independent of decoding model (Fig. 3, B and C).
Decoding from a mixed population.While a majority of neurons seems to exhibit center-dependent modulation, some neurons exhibit neuron-dependent modulation, and others show intermediate tuning (Shushruth et al. 2012).We measure the bias in such a mixed population by varying the percentage of center-dependent vs. neuron-dependent neurons.We model 3,200 neurons, such that there are 100 neurons per preferred orientation.When we decode the center using the population vector, the amount of bias strength is simply proportional to the percentage of neurons with neuron-dependent modulation (Fig. 3D).The ML decoder qualitatively shows the same dependence on neurons with neuron-dependent modulation, with a lower overall bias (not shown).In the DISCUSSION we use these results to compare psychophysical to neural data.

Saliency Based on Orientation Contrast
Surround modulation is thought to underlie the saliency of stimuli that have an orientation different from their surround (Li 1999;Petrov and McKee 2006;Sillito et al. 1995).While saliency computation likely includes feedback from higher areas, it has been proposed that part of the saliency is computed from the V1 responses in a feed-forward manner, so-called bottom-up saliency (Li 1999(Li , 2002)).In this model, salient features have a higher response than less salient features.Compared with neuron-dependent modulation, center-dependent modulation has previously been hypothesized to increase the response to salient features relative to less salient features (Shushruth et al. 2012).
We examined how surround modulation type affects visual saliency, using stimuli consisting of multiple bars.A population of orientation-selective neurons is associated to each bar; the neurons' surround consists of the other bars, which individually modulate its response in a distance-dependent manner (METHODS).We present the neuron-dependent and center-dependent modulation variants of this model with various bar configurations (Fig. 5, left column).
As the exact nature of saliency computation is unknown, we define two measures of saliency.First, we assume that the saliency of a given bar (the target) is given by the maximum response across all orientations at the target location, relative to the average maximum response to all bars in the image, where r t and r x are the population responses at the target and x locations respectively, max(r) is the maximum across a local population response, and ͗͘ denotes an average across all locations.Thus if the maximum response to a bar is higher than most other bars in an image, its saliency will be larger than 1.Second, we consider a saliency measure which compares the mean response of a target-bar population to the mean response of all bars in the image (Li 1999), Both averaging and max-like pooling have been proposed as canonical computations for the visual cortex.The two types of saliency computation can be seen as extremes of the more generic computation where the saliency is a nonlinear sum of firing rates in a population s͑p͒ϰ͓ ͚i r i p ͔ 1⁄p , where the exponent P ϭ 1 for average-based salience, and p ¡ϱ for max-based saliency.
Using either saliency signal, under either surround modulation type, features of interest have increasing saliency as the orientation differences increase.This is illustrated for a single contrasting bar or central set of bars (Fig. 5, A and B), for a simple contour (Fig. 5C), and a boundary region between two differing groups (Fig. 5D).These results are consistent with the experimentally observed increase in neural responses and the perceived contrast of a deviant central grating (Cannon and Fullenkamp 1991;Shushruth et al. 2013), as well as with previous saliency models using neuron-dependent modulation (e.g., Keemink and van Rossum 2016;Li 1999).The increase in saliency is due to target populations being less strongly suppressed than background populations (which have many surrounding stimuli of the same orientation).
With max-based saliency, neuron-dependent modulation will lead to slightly higher saliency for intermediate orientation differences and the same saliency for perpendicular orientation differences (Fig. 5, middle column).The saliencies are very similar because either modulation type leads to similar changes in the maximum firing rates when the orientation difference is increased (see Fig. 5, insets of middle column).
Mean-based saliency leads to the opposite scenario, with center-dependent modulation leading to higher saliency across all orientation differences (Fig. 5, right column).This is due to center-dependent modulation leading to stronger mean response changes than neuron-dependent modulation (see Fig. 5, insets of right column).Similarly to the bias results, this occurs because center-dependent modulation is the same across a population.In the case of parallel bars, with center-dependent modulation, all neurons responding to one bar will be suppressed as strongly as the most suppressed neuron under neuron-dependent modulation (Fig. 4, left column), and as weakly as the least suppressed neuron under neuron-dependent modulation for perpendicular bars (Fig. 4, right column).
While here we used experimentally constrained parameters, by assuming that bars in the feature have negligible influence on the responses to the rest of the image (as is the case for small features such as a single bar), we show that these results hold more generally (see APPENDIX): s mean is always larger for center-dependent modulation at perpendicular orientations, whereas under some broad assumptions s max is always lower for intermediate orientations.

DISCUSSION
We compared two types of surround modulation in V1: 1) neuron-dependent modulation, which is strongest when the Fig. 5. Saliency for center-dependent and neuron-dependent modulation and for 2 saliency measures.A: a single deviant bar among a homogeneous background.Left: presented scene.Middle: the saliency of the central deviant bar as a function of its orientation difference with the background, calculated from the maximum responses.The inset shows how the maximum firing rate to the central bar changes under the 2 modulation types.Right: same as the middle panel, but with the saliency calculated from the mean responses.The inset shows the mean firing rates to the central bar.B: a group of deviant bars among a homogeneous background.The saliency and firing rates were averaged over the 9 deviant bars.C: a set of diagonal bars among a homogeneous background.The saliency and firing rates were averaged over the bars on the diagonal.D: 2 groups of bars with differing orientations, creating a boundary region.The saliency and firing rates were averaged over the bars at the boundary region.Periodic boundary conditions were imposed in all panels to prevent edge effects.
surround orientation equals the neuron's preferred orientation; and 2) center-dependent modulation, which is strongest when the center and surround stimuli are aligned.Surround modulation is often used to explain observed perceptual biases and orientation saliency, assuming neuron-dependent modulation.In this paper we asked how center-dependent modulation, the biologically more prevalent modulation, impacts these findings.
We first examined orientation coding biases by decoding the center orientation from the population response.Under neurondependent surround modulation, due to each neuron being modulated differently, a population vector decoder has a strong repulsive tilt bias, irrespective of the observation time or number of neurons in the population, as is well known from existing tilt illusion models (Blakemore et al. 1970;Clifford et al. 2000;Schwartz et al. 2009).For a maximum likelihood (ML) decoder that takes the effect of surround modulation into account, a similar bias emerges.However, in this case the bias depends on the neuronal noise level and disappears in the zero noise limit.In contrast, for a center-dependent modulation, where the modulation is identical across the population, bias is completely absent for either decoder.
In the tilt-after effect, where the orientation of a full field grating is misjudged when presented after a differently oriented adapter grating (Schwartz et al. 2007), similar dependences of the bias on modulation type have previously been described.Seriès et al. (2009) showed that an ML decoder with no knowledge of the adapter is biased if the adaptation depends on the preferred orientation, but that if the adaptation instead depends on the test orientation (the equivalent of centerdependent modulation), the bias disappears [see figure 6C of Seriès et al. (2009)].Jin et al. (2005) investigated the effect of specific tuning curve changes on the tilt-after-effect.They found that tuning curve shifts away from the adapter cause an attractive bias, and tuning curve magnitude changes cause a repulsive bias.For neuron-dependent surround modulation, the tuning curves are modulated, but not shifted, resulting in the repulsive bias.For center-dependent modulation both tuning curve shifts and magnitude changes occur, with the net effect of these two tuning curve changes resulting in zero bias.
What do our results mean for the origin and magnitude of the tilt illusion?Using a model based on the majority of V1 neurons, i.e., with center-dependent modulation, the illusion disappears.Using our neuron-dependent model and with parameters fitted to monkey V1 data, a population vector decoder leads to a repulsive bias of maximally 12°, compared with ~3°p sychophysically (Clifford 2014).We therefore propose that the tilt illusion stems from a mix of center-and neurondependent modulated neurons (Fig. 3D).A maximum repulsive bias of 3°would correspond to a population with ~75% center-dependent and 25% neuron-dependent modulation.Although extracting a quantitative match between monkey neural tuning curve properties and human psychophysics is full of pitfalls, and depends on the decoder used, this ratio is reasonable.Similar arguments would hold for a population that is not strictly binary in its modulation type, but has some distribution of modulation ranging from fully center-to neuron-dependent.Both types of modulation (as well as intermediate cases) have been observed electrophysiologically and although a specific ratio is hard to infer from current literature, center-dependent modulation is indeed more prevalent (Cavanaugh et al. 2002a;Shushruth et al. 2012;Sillito et al. 1995).
The bias of an ML decoder decoding only from neurondependent neurons was smaller than that of the population vector, providing an alternative explanation for the magnitude of the psychophysically observed bias.In contrast to the population vector (and modulation-naive ML decoders), the bias in the ML decoder depends on the noise and disappears at lower noise levels (Fig. 3C).While several studies have reported effects of the presentation time on the bias magnitude, the results are conflicting.Several studies indeed reported a decrease in bias magnitude with presentation time (Calvert and Harris 1985;Wenderoth and Johnstone 1988b;Wenderoth and van der Zwan, 1989), but some studies found an increase in tilt illusion with presentation time (O'Toole, 1979) or an increase for shorter time scales and a decrease for longer time scales (Calvert and Harris 1988).The illusion was present in all these studies, even when subjects were free to rotate a test grating until it matched the perceived vertical (Wenderoth and Johnstone 1988a), thus arguing against a ML decoder with perfect temporal integration and rendering this explanation of reduced tilt illusion less parsimonious.
The full-tilt illusion has both a repulsive (for most orientation differences) and a weakly attractive effect (for larger orientation differences).However, our current model does not exhibit an attractive bias.While the origin of the attractive illusion is debated, in our model it could be achieved by adding an excitatory term similar to the current h term.Excitatory modulation from a surround stimulus does exist with low stimulation (a low contrast center grating with a thin surround annulus), but is also mainly center dependent (Shushruth et al. 2012).Yet in preliminary work we were not able to construct modulation curves that would lead to an attractive tilt illusion and also reasonably fit the Cavanaugh data; a bias only appeared for neuron-dependent modulation (whether inhibitory or excitatory).
Center-dependent modulation has previously been hypothesized to increase the relative response to more salient features (Shushruth et al. 2012).We examined the saliency of a variety of stimuli consisting of bar configurations containing differing features among a homogeneous background.Either surround modulation type results in a bottom-up saliency signal as the response to the target is higher than the background response.For saliency based on the maximum response there was only a small difference between the two surround modulation types, whereas for saliency based on the mean population response, the larger response differences between salient and non-salient locations leads to a stronger saliency signal for center-dependent modulation.These results confirm that center-dependent modulation might lead to stronger saliency signals, but show that this depends strongly on the exact saliency computation.
Recent work has argued that center-dependent modulation emerges from a normative perspective.Lochmann andDeneve (2011) andLochmann et al. (2012) built a spiking network model with connections that are loosely derived from a Bayesian probability model.Similarly, Coen-Cagli et al. (2012) based the connectivity in a neural model on the image statistics at several surround locations.In both models center-dependent modulation seems to emerge automatically, from which one could argue that that center dependence follows from these theories.One reason their models are center-dependent could be that they implicitly assumed an unbiased encoding of the stimulus, as they are defined as probabilistic representations of an unbiased scene; the only way to fit parameters under such a decoder is for the surround modulations to be center dependent.We predict that in both these models the tilt illusion is absent.
In summary, seemingly minor changes in the modulation of neuronal tuning curves can have important functional consequences.Recurrent effects are currently not explicitly included in the model, and it would be of interest if the model responses to more complex inputs (e.g., Fig. 5D) correctly predict electrophysiological or psychophysical data.It would be of interest to know whether our arguments translate to other domains and levels of sensory processing, where most experimental studies have examined contextual modulation while presenting the preferred stimulus in the RF.

Maximum-Based Saliency
Here we show that under some assumptions center-dependent modulation leads to a lower saliency signal for saliency based on the maximum response.In other words, the solid line in the inset of Fig. 5A, middle column, lies above the dashed line.
We denote the response of a neuron with neuron-dependent modulation by f ndep (, s ) ϭ g(, 0)h(, s ), and a center-dependent neuron by f cdep (, s ) ϭ g(, 0)h(0, s ), where we treat the preferred orientation as a continuous variable (i.e., in the limit of infinite neurons) and assumed c ϭ 0. In other words, s equals the center-surround orientation difference.We as- sume negligible influence from the target on the overall responses to the image, such that background bars only experience parallel surrounds, and target bars experience only the background as a surround.The saliencies of the target are then First we note that across the population, f cdep (, s ) is maximal when ϭ 0, i.e., when the center stimulus aligns with the preferred orientation, independent of the surround orientation.This maximum is g(0,0)h(0, s ).Meanwhile, in the neuron-dependent model the activity of the neuron with preferred orientation c is f ndep (0, s ) ϭ g(0, 0)h(0, s ) ϭ max f cdep (, s ).
In other words, the neuron with the highest activity in the centerdependent model always has the same activity as the corresponding neuron in the neuron-dependent model, as can be observed from the intersection of solid and dashed curves at ϭ 0 in the inset of the middle column of Fig. 4.
However, in the neuron-dependent model the corresponding neuron is not the neuron with the highest activity.To see this, consider the derivative of the activity with respect to the preferred orientation This is only zero if s ϭ 0 ϩ n, showing that f ndep (0,0) is generally not an extremum.There must therefore be a preferred orientation where f ndep (, s ) Ͼ f cdep (0, s ).Under the reasonable assumption that in the background condition the maximum response f ndep (, s ) occurs when ϭ 0, center-dependent modulation thus leads to a lower maximum-based saliency signal for intermediate orientations.

Mean-Based Saliency
Here we demonstrate that saliency based on the mean responses always gives higher saliency with center-dependent modulation at perpendicular orientations, than with neuron-dependent modulation, as in Fig. 5A.We assume negligible influence from the target on the overall responses to the image, such that background bars only experience parallel surrounds, and target bars experience only the background as a surround.
For dense coding across many neurons the mean response to a single bar with orientation c ϭ 0 can be written as an integral  Although we found no proof that s ndep Ͻ s cdep for all orientations s , we can compare the nonsalient to the most salient condition.In the nonsalient colinear condition, i.e., s ϭ 0, the saliency is the same for both modulation types: In other words, the relative change in firing rate from isooriented to perpendicular orientations in Fig. 4A is larger for center-dependent modulation than for neuron-dependent modulation.

Fig. 2 .
Fig.2.Modeled single neuron responses to center-surround gratings, for neuron-dependent and center-dependent surround modulation.A: the neural response for different center orientations (0°, 30°, and 60°) against the surround orientation, for neuron-dependent surround modulation.The center orientations are indicated by the dashed lines and their respective colors.The curves do not shift horizontally, since modulation does not depend on the center orientation.B: same as A, but for center-dependent modulation.As modulation is now strongest when center and surround are aligned, the curves shift with the center orientation.

Fig. 3 .Fig. 4 .
Fig.3.Decoding biases in the estimate of the center stimulus orientation with neuron-and center-dependent modulation.A: center orientation bias of the population vector decoder (derived and measured curves overlap exactly in absence of noise).B: the bias for the maximum likelihood (ML) decoder, which takes the influence from the surround into account.The bias was averaged across presentation angles over 5,000 realizations and 0.5-s measurement time.C: the measurement time dependence of the absolute bias averaged over all surround orientation.The bias for each measurement time was averaged over 500 trials.Neurondependent modulation always leads to a bias, which decreases for longer measurement times for the ML decoder.Center-dependent modulation never results in a bias (black and gray dashed curves overlap).D: the bias in the population vector decoder pooling from a mixed population with both types of surround modulation.
For a background bar (which is surrounded by equally oriented bars, and thus c ϭ s ) the mean response is mean(r b ) ϭ 1 ͐ 0 g(, 0)h( ref , s )d.Substituting c ϭ 0 and i for ref we find mean(r b ) ϭ 2A c exp(Ϫk c )I 0 (k c )h(0, 0) center dependent and mean(r b) ϭ 2A c exp(Ϫk c )͓I 0 (k c ) Ϫ A s exp(Ϫk s )I 0 ( Խ (k c ϩ k s )v c Խ)͔ neuron dependent.The mean response to the target bar (which has a different orientation from the surrounding bars) meanwhile ismean(r t ) ϭ 2A c exp(Ϫk c )I 0 (k c )͓1 Ϫ m(0, s )͔ center dependent and mean(r t ) ϭ 2A c exp(Ϫk c )͓I 0 (k c ) Ϫ A s exp(Ϫk s )I 0 ( Խ k c v c ϩ k s v s Խ)͔ neuron dependent.The saliency of a deviant bar follows ass cdep ( s ) ϭ hs ) ϭ 1 Ϫ A s exp(Ϫk s )I 0 ( Խ k c v c ϩ k s v s Խ) ⁄ I 0 (k c )1 Ϫ A s exp(Ϫk s )I 0 (k c ϩ k s ) ⁄ I 0 (k c ) neuron dependent.

s
ndep (0) ϭ s cdep (0) ϭ 1In the most salient condition of orthogonality, i.e., s ϭ ⁄2, the saliences are given by s exp(Ϫks )I 0 (k c Ϫ k s ) ⁄ I 0 (k c ) 1 Ϫ A s exp(Ϫk s )I 0 (k c ϩ k s ) ⁄ I 0 (k c ) neuron dependent.Assuming that k c ,k s Ͼ 0 and k s Ͼ 0, it follows that1 Ϫ A s exp(Ϫ2k s ) Ͼ 1 Ϫ A s exp(Ϫk s ) I 0 (k c Ϫ k s ) I 0 (k c ) and 1 Ϫ A s Ͻ 1 Ϫ A s exp(Ϫk s ) I 0 (k c ϩ k s ) I 0 (k c ) ,and thus