INTRODUCTION

Temporal onsets are important features of visual stimuli. Like sharp contours in the spatial domain, abrupt temporal changes carry important information in the temporal domain. For example, a stimulus flashed on a static background is visually salient and “attention grabbing,” making it easily detected amid other stimuli that were already present (Yantis and Jonides 1984, 1996).

Although flashing an isolated stimulus onto an unchanging background is common in neurophysiology experiments, it is not common for objects to suddenly appear on a static background in natural vision. Instead when we shift our gaze around in the natural environment, saccades bring not only new local stimuli onto neurons’ receptive fields (RFs) but also change the “background” in the surrounding area simultaneously. Little is known about how visual information is processed when a stimulus is introduced simultaneously with a background change (i.e., the more normal situation). A background change may stimulate areas both inside and outside a V1 neuron’s classical receptive field depending on how much of the RF is covered by the “local” stimulus. V1 neurons are subject to potent contextual influences from beyond the RF (e.g., Allman et al. 1985; Bishop et al. 1973; DeAngelis et al. 1994; Gilbert and Wiesel 1990; Knierim and Van Essen 1992; Li and Li 1994; MacEvoy et al. 1998; Maffei and Fiorentini 1976; Sillito et al. 1995; also see Albright and Stoner 2002 for a review), and it is likely that a background change can exert contextual influences on V1 neurons’ responses to a local stimulus inside the RF.

To understand neural representations in more natural visual situations, it is important to determine the extent to which responses are influenced by the temporal onset of a background change. The goal of this study is to characterize V1 neurons’ responses when visual stimuli conventionally used to study V1 receptive fields are introduced into RFs with a background change. In natural situations, saccadic eye movements generally lead to changes of the mean background luminance and the background pattern (see discussion). To study neuronal responses in a controllable manner, we used the following reduced paradigms that do not involve saccades. A Gabor patch or a bar of light was introduced into a neuron’s RF together with either a background luminance change (increment or decrement) or a background pattern change with no change in mean luminance. We refer to these stimulation paradigms as “changing-background conditions.” For comparison, the same visual stimuli were shown in the RF without a background change (the “static-background conditions”).

We find that in the changing-background conditions, V1 neurons exhibit a temporal response pattern that is qualitatively different from the static-background condition. When a visual stimulus is presented with a simultaneous background change, the representation of form information in V1 is delayed. In the following paper, psychophysical experiments show that aspects of human detection and discrimination are correctly predicted from the neural responses recorded in macaque V1. Some of these results have been published previously in abstract form (Huang and Paradiso 2000a,b).

METHODS

Surgical preparation

Experiments were performed on two female macaque monkeys (Macaca mulatta), weighing 5.0 ∼ 6.3 kg. Before recording, under isoflurane anesthesia, each monkey was surgically implanted with a head post and a recording chamber overlaying the operculum of area V1. In one monkey, a scleral search coil for monitoring eye position was implanted (Judge et al. 1980; Robinson 1963). All procedures were approved by Brown University’s institutional animal care and use committee and were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals.

Experimental protocol

During training and recording, animals performed a simple fixation task in which they foveated a spot on a visual display. The eye position of the monkey was required to remain within a square fixation window, 0.7 ∼ 1.0° on a side, centered on the fixation spot. We observed the precision of fixation to be far finer than the size of this fixation window. Fixation locations and stability were found to be similar in all the paradigms described below. After the animal held fixation for ∼300 ms, a visual stimulus was presented on the screen for 550–600 ms, except in one experiment a brief presentation of 67 ms was used. The monkey was required to maintain fixation during the period of stimulus presentation and an additional 500 ms after stimulus offset. The experimental trial was aborted if eye position left the fixation window. The animal received juice rewards for successfully maintaining fixation throughout the trial. The monkeys were free to move their eyes around during the inter-trial interval, which was ∼1.5 s. In initial experiments, eye position was monitored at 1 kHz with a search coil (spatial resolution: 0.1°). In later experiments, eye position was measured at 60 Hz using an infrared video system (ISCAN) with spatial resolution of 0.2–0.3°.

Electrophysiological recording and data collection

The behavioral paradigm and data acquisition were controlled by REX software (Hays et al. 1982). Extracellular potentials from isolated neurons or clusters of neurons were recorded using insulated tungsten microelectrodes (Frederick Haer). Action potentials of single neurons or a small cluster of neurons were discriminated with a template-based software system (Alpha-Omega).

RF properties were determined initially with a computer-generated bar of light, whose parameters were varied manually. The receptive field was defined as the area on the screen within which a flashed or moving bar of light elicited a response from the neuron (i.e., the minimum response field). The RFs of the V1 neurons were located in the lower contralateral visual field with eccentricities of 3–6°. The sizes of the RFs were usually within the range of 0.6–1.2°.

Visual stimuli

Visual stimuli were presented on a 27-in video monitor, which subtended 33 × 25° at the viewing distance of 93 cm. The monitor was driven by a Number Nine graphics board with 640 × 480-pixel resolution at a refresh rate of 60 Hz. A bar of light was presented in the RF either with or without a simultaneous background change. The length of the stimulus bar was comparable to the size of the RF. The width of the bar was 0.15°. Orientation preference was assessed by presenting the bar in 30° steps spanning 180°. Sensitivity to contrast was studied by varying Michelson contrast from 0.3 to 0.9.

In natural visual situations, when a saccade shifts the retinal image, both the mean luminance and the pattern in a receptive field may change. In our experiments, we examined the consequences of luminance and pattern changes independently. Changes in the stimulus background, which sometimes accompanied the presentation of a bar or Gabor stimulus, were of three types. The background change was either an increment (from 0.2 to 8.6 cd/m²) or a decrement (from 19.2 to 8.6 cd/m²) in the luminance of a uniform background or a texture pattern change that preserved mean luminance (Fig. 1). The texture pattern background was composed of random dots with two gray levels of 0.2 and 36.4 cd/m² (pattern mean luminance of 18.3 cd/m²). To minimize the variation in the mean luminance of the pattern background at different spatial scales, each random dot was defined by a single pixel. At stimulus offset, the background reverted to the background present before stimulus onset. The static and changing-background conditions were blocked and interleaved. Because the background stimuli were either uniform in luminance or random noise patterns, any small fixation errors would not introduce salient stimuli into the RF.

In some experiments, a Gabor patch, rather than a bar, was presented in the receptive field along with a background luminance increment from 0.2 to 18.6 cd/m². The SD of the Gabor patch’s Gaussian profile was 1.1°, and the patch diameter was 3.3° such that the Gabor patch blended into the background gray level. The Gabor patch was set at the preferred orientation of the neuron under study and the spatial frequency was 2 cycles/°. Four peak contrast values, 0.3, 0.5, 0.7 and 0.9, were used.

Data analysis

Poststimulus time histograms (PSTHs) with a bin width of 10 ms were constructed and smoothed using a Savitzky-Golay (SG) filter with a second degree underlying polynomial and a window size of 50 ms. The SG filter (also known as a digital smoothing polynomial filter or the least-squares smoothing filter) is a least squares moving average that preserves features better than simple adjacent bin averaging. The raw PSTH for each neuron was normalized to the maximum firing rate of the neuron to facilitate averaging across the population studied. To allow for the comparison of responses across stimulus paradigms, responses in all paradigms were normalized to the same across-paradigm response maximum.

To summarize orientation selectivity for the population, data were averaged across neurons for the bar orientation that gave the highest response (the “optimal orientation”), the lowest response (the “least-optimal orientation”), and the response with a stimulus 30° away from the least-optimal orientation in the direction of optimal (the “intermediate orientation”).

A sliding 60-ms analysis window was sometimes used to compare the firing rates of neurons in response to the static and the changing-background conditions. To evaluate whether the responses to the changing and the static-background conditions were significantly different, a paired t-test was used on the population (P < 0.05 to be significant).

To calculate the response latency for each neuron, a PSTH with a bin-width of 5 ms was generated and then smoothed using a SG filter with a window size of 25 ms. Because we never encountered neurons response latencies <20 ms, we used the time window of −150 ∼ 20 ms relative to stimulus onset to determine the background activity. To avoid reporting a very small fluctuation in the firing rate as significantly higher than very low background activity, we empirically set the background firing rate to 0.5 spikes/s when the actual mean background firing rate was lower than 0.5 spikes/s. The response latency was measured with a method adapted from Maunsell and Gibson (1992). Latency was taken to be the middle time point of the first of three successive bins whose firing rates all exceeded a criterion. The criterion was a firing rate with a probability of P = 0.01 based on a Poisson distribution of the background activity. Overall response latencies were usually 30 ∼ 90 ms after stimulus onset.

We found some neurons showed a delayed response after a distinct early transient response. The early response usually had a latency <90 ms using the method described above, and the delayed response usually occurred after 90 ms. To determine the latency of the delayed response, the following algorithm was used: 1) the bin with the maximum firing rate within the time window of 30 ∼ 90 ms was identified (the “early maximum”). 2) Between the time of the early maximum and 200 ms, “effective local minima” were identified as inflection points in the response that were significantly below the early maximum (P < 0.05, based on a Poisson distribution). 3) Starting at each effective local minimum until 230 ms after stimulus onset, we identified groups of three successive bins the firing rates of which were all significantly higher than the effective local minimum (P < 0.05). 4) The latency of the delayed response was derived from the earliest group of three bins that were significantly above their associated local minimum. The latency was defined as the middle of the first bin of the earliest three-bin group.

In some cells, the latency of the overall response was >90 ms, and no distinct delayed response was found with the algorithm in the preceding text. In these cases, the long latency was taken to be the delayed response latency. There were other situations in which the above procedures yielded a shorter latency for the delayed response than the overall response. This was quite uncommon but occasionally occurred because the significance level for determining the overall response latency was more stringent than for determining the delayed response latency. When this happened, the overall response latency was taken to be the latency determined with the delayed-response criterion.

It is important to note that the algorithms we used to quantify latency agreed well with those assigned by human visual inspection.

RESULTS

We recorded from a total of 272 V1 neurons or small clusters of neurons in two awake macaque monkeys. Of these, 168 were single units and 104 were multiunits. Single- and multiunits displayed similar response properties and were combined in the following analyses. We find that the temporal response pattern and the timing of response selectivity are influenced by background changes in addition to the “local” stimulus. We explored the response characteristics with three types of background changes: luminance increments, luminance decrements, and texture pattern changes with a fixed mean luminance.

Background luminance change (increment)

In comparison to the situation with a static background, V1 neurons show a qualitatively different temporal response pattern when the visual stimulus is introduced simultaneously with an increment of the background luminance. In the following analysis, note that the responses in the static- and changing-background conditions are to the identical bar and background; the only difference between conditions is what background precedes the test stimulus.

Figure 2 shows responses of three exemplary neurons to a bar of light in the static and the changing-background conditions. In the static-background condition, the response of one neuron (Fig. 2A) consists of a single peak in the firing rate (the red PSTH). The same neuron responded to the identical stimulus bar in the changing-background condition with an initial transient response, followed by a reduction of firing rate and then a delayed response peak (the black PSTH). The response pattern change is visible on a trial-by-trial basis in the raster plots.

A second neuron responded to a stimulus bar on the static background with a strong transient response, followed by a lower sustained response (Fig. 2B). In the changing-background condition, the transient response was delayed and suppressed, while the later sustained response-component was similar to the static condition. In a third pattern, the overall response is delayed in the changing-background condition, but there is no suppression of the peak firing rate relative to the static condition (Fig. 2C).

RESPONSES TO DIFFERENT ORIENTATIONS.

A bar with a fixed contrast of 0.35 was used to characterize V1 responses to different stimulus orientations. A total of 105 units were tested with the orientation paradigm in both the static and the changing-background conditions.

Figure 3A shows the averaged population responses to different orientations. The black and red traces show responses to the optimal orientation in the changing and static-background conditions, respectively. The green trace is the response to the least-optimal orientation, this was often, but not always, orthogonal to the optimal. The cyan trace is the response to the orientation 30° away from the least-optimal orientation in the direction of optimal (the “intermediate” orientation). The orange curve is the response to the background-change-alone (BCA) condition (i.e., no bar) and the dark blue curve shows the null response with no stimulus.

In the static-background condition, orientation selectivity appears in the very beginning of the neural response (Fig. 3A, top). This finding is consistent with previous studies (Celebrini et al. 1993; Vogels and Orban 1991). In contrast, when an oriented bar is introduced onto the RF simultaneously with a background luminance change, the initial response peak is relatively insensitive to orientation (Fig. 3A, middle). Orientation information is delayed and is represented in the amplitude of the later response. Note, however, that the response latency is similar in the static and changing-background conditions. It is also worth noting that the population response to the BCA condition is quite transient and similar to the initial response to a bar of light. The bottom panel in Fig. 3A shows the evolution of orientation selectivity. The magenta and gray curves show the difference in response rate with the optimal and least-optimal orientations for the static and changing-background conditions, respectively. In the changing-background condition, it takes significantly longer for this measure of orientation selectivity to reach a value comparable to the static-background condition.

To test the significance of apparent differences in the temporal development of orientation selectivity, we compared average firing rates in different time epochs (Fig. 3B). Comparisons were made in 60-ms analysis windows moved in 20-ms increments. At the time epoch between 10 and 70 ms after stimulus onset, the firing rates to the optimal, the intermediate, the least-optimal orientations and the null condition are all significantly different from each other in the static-background condition (Fig. 3B, top). Therefore the response distinguishes between different orientations and also between a stimulus and no stimulus. In the changing-background condition (solid histogram bars), the firing rates are not significantly different between any pair of the optimal, the intermediate, the least-optimal orientations and the BCA condition. Thus at this early time epoch, the population response is orientation selective in the static-background condition, whereas in the changing-background condition, the neurons can neither distinguish the orientation of the stimulus nor the bar stimulus from an increment of the background luminance (with no bar). In Fig. 3B, top, the response in the BCA condition is significantly higher than that in the null condition. This means that the background luminance change can be detected based on the early response.

In later time epochs, orientation selectivity is retained in the static-background condition (Fig. 3B, middle and bottom). In the changing-background condition, orientation selectivity evolves with time. At the time epoch between 30 and 90 ms, the population response to the optimal orientation is significantly higher than the responses to the other orientations and the BCA condition. The response to the least-optimal orientation is significantly greater than the response to the BCA condition. However, the response to the medium orientation is not significantly greater than that to the least-optimal orientation (Fig. 3B, middle). The time epoch between 90 and 150 ms is the first in which the responses between any pair of the orientations and the BCA condition are significantly different (Fig. 3B, bottom); consistent with Fig. 3A, bottom, orientation selectivity is almost fully developed.

TWO CATEGORIES OF RESPONSES IN THE CHANGING-BACKGROUND CONDITION.

In the changing-background condition, the temporal response of some neurons was biphasic, having an initial transient and a delayed component. The temporal response of another group of neurons was simply delayed relative to the response in the static-background condition. We sought to classify neurons into these two categories with an objective criterion. We noticed that when a neuron showed a biphasic response with a bar of light in the changing-background condition, it usually responded to the BCA condition with an early transient response. On the other hand, when a neuron showed only a delayed response with a bar of light in the changing-background condition, it usually did not respond to the BCA condition or it showed a much delayed response with small magnitude. Therefore we chose to use the occurrence and latency of the response to the BCA condition as a basis for classifying the response of the population.

We grouped the units with response latencies in the BCA condition that were <90 ms into subpopulation I. There were 51 of 105 units (49%) in this group. Figure 4A shows the averaged responses of neurons in subpopulation I to different orientations and backgrounds (static background, top; changing background, middle). Consistent with the observation made in the preceding text regarding the correlation between the biphasic response to the oriented stimuli introduced with a background increment and the transient response to the BCA condition, the averaged response of subpopulation I showed an early and a delayed response component (Fig. 4A, middle). In the changing-background condition, the average initial responses to different orientations and the BCA condition are almost identical. Orientation selectivity appears only in the delayed response when the background changes. In contrast, the same group of neurons shows orientation selectivity in the very early response with a static background (Fig. 4A, top). The difference between static and changing-background conditions, in the development of orientation selectivity, is seen clearly in Fig. 4A, bottom. The response difference to optimal and least-optimal orientations appears much earlier in the static-background condition; orientation selectivity is delayed with the changing background.

The mean firing rates of subpopulation I in different time epochs are shown in Fig. 4B. In the static-background condition, the firing rates are significantly different within any condition pair in all three time epochs. In the changing-background condition, the firing rates at the earlier time epochs (10–70, 30–90 ms) are not significantly different between any pair of responses to the optimal, the intermediate, the least-optimal orientations and the BCA condition (Fig. 4B, top and middle). The 90- to 150-ms epoch is the earliest in which the firing rates are significantly different between any pair of orientations and the BCA condition (Fig. 4B, bottom). As for the total population, in the changing-background condition, subpopulation I shows a delayed development of both orientation selectivity and a response that distinguishes a background change alone from the introduction of a bar on a changing background.

We grouped the other 54 of the 105 units (51%) into subpopulation II. Neurons in this subpopulation responded to the BCA condition with latencies >90 ms or did not show a significant excitatory response to the BCA condition (failed the latency measure). A trend of this subpopulation is that with the background luminance change, the response to a bar of light is simply delayed compared with that in the static-background condition. In other words, the responses often looked similar to those of neurons in subpopulation I except there was little or no early response transient.

Figure 5 shows two neurons illustrating two rather different response types found in subpopulation II. The neuron in Fig. 5A had a much longer response latency (>90 ms) with a changing background than with a static background and orientation selectivity appeared much later in the changing-background condition. Roughly half of the neurons in subpopulation II were of this type. The other half were more similar to the example in Fig. 5B. This neuron had a changing-background response delayed relative to the static condition, but the latency was relatively short (<90 ms). Orientation selectivity was delayed relative to the static-background condition but to a much lesser extent than in Fig. 5A. Such cells were categorized into subpopulation II because they did not have a significant response to the BCA condition.

As defined in the preceding text, neurons in subpopulation II either had latencies >90 ms in the BCA condition or no BCA response (and recall that the BCA response appears to correlate with the response being mono- or biphasic in the changing-background condition). Aside from the BCA response, neurons in subpopulation II also had late responses to optimal stimuli. Figure 6A compares response latency to an optimal bar stimulus in the static and changing-background conditions for all cells in sub-population II. Two important observations come from the distribution. First, responses to optimal stimuli are clearly delayed in the changing-background condition. Second, there is not a strong correlation between latency in the static and changing-background conditions. It is not the case that neurons with delayed responses in the changing-background condition are simply “slow cells.” Instead the longer latencies in the changing-background condition are specific to that condition.

Figure 6B histograms the latencies of responses to the optimal orientation in the static and the changing-background conditions for neurons in subpopulation II. The mean response latency in the changing-background condition is 87 ms, significantly longer than the mean of 63 ms in the static-background condition (paired t-test, P = 2.6 × 10⁻⁵).

Figure 6C shows the evolution of orientation selectivity for neurons in subpopulation II. Similar to neurons in subpopulation I, responses in the static-background condition reflect selectivity at earlier time epochs than in the changing-background condition. We also observed that in a time epoch of 50–100 ms, the units in subpopulation II that displayed only a delayed response to the optimal orientation (response latencies >70 ms), showed a response to the BCA condition significantly weaker than their baseline activity (paired t-test, P = 0.02). That is to say, an increment in the background luminance caused early inhibition in this subset of neurons. This inhibition might be related to the delay of the responses to the oriented bars in the changing-background condition. In comparison, for the group of units in subpopulation II that showed a short response latency to the optimal orientation in the changing-background condition, there was not significant early inhibition.

RESPONSE AFTER STIMULUS OFFSET.

Orientation selectivity was also evident in the responses after stimulus offset in both the static and the changing-background situations. However, the duration of the response after stimulus offset in the changing-background condition was shorter (∼100 ms difference at half height) than in the static-background condition. This was true of neurons in both subpopulations I and II. The averaged responses after stimulus offset of the whole population (n = 105) are shown in Fig. 7 (note that this figure is aligned on stimulus offset, not onset). Some neurons showed a response peak after the offset of an isolated stimulus bar (the static-background condition), but they did not show such a peak when the stimulus bar and the background were turned off at the same time. Other neurons showed the response peak to the stimulus offset in both the static and changing-background conditions. As the summary in Fig. 7 illustrates, on average, there was a larger transient off response in the static-background condition.

RESPONSES TO DIFFERENT CONTRASTS.

By introducing a Gabor patch into RFs of V1 neurons along with a background luminance change, we determined the effect of stimulus contrast in the changing-background condition. Because the size of the Gabor patch was two to approximately three times larger than that of the RF, the background luminance change took place only outside the RF.

Figure 8 shows two representative examples of responses to a Gabor patch with different contrasts. In the static-background condition, response latency is slightly reduced as stimulus contrast increases (Fig. 8, top). This is consistent with previous findings (Gawne et al. 1996; Maunsell and Gibson 1992; Reich et al. 2001). In the changing-background condition, some neurons showed a biphasic temporal response pattern with an initial response and a delayed response (Fig. 8A). Unlike what was seen in the static-background condition, the latency of the initial response was relatively unaffected by the stimulus contrast. Instead, the latency of the delayed response was inversely correlated with contrast. Other neurons showed only the delayed response component in the changing-background condition and no initial response. For those neurons, contrast was also reflected in the latency of the delayed response (Fig. 8B). When stimulus contrast was varied across the same range (0.3–0.9), the latency shift of the delayed response in the changing-background condition was generally larger than the latency shift of the early response in the static-background condition.

The technique we used to measure latency (see methods) successfully found initial and delayed response latencies for ∼85% of the neurons studied. In the changing-background condition, 84 units that responded to at least one contrast level with a latency <90 ms were incorporated into the analysis. Figure 8C shows the cumulative distributions of the initial and the delayed response latencies to a Gabor patch with four different peak contrasts. Stimulus contrast has little effect on the initial response latency, but the delayed response latency decreases as stimulus contrast increases. With the linearly spaced contrasts used, the change in the delayed response latency is compressed at high contrast levels.

Similar contrast effects on the temporal response pattern of the V1 neurons were found when using a stimulus bar instead of a Gabor patch: the response elicited by a bar with a simultaneous background luminance change was either biphasic or delayed, and the delayed response latency, not the early one, was inversely correlated with stimulus bar contrast. Because the width of the stimulus bar was always smaller than the size of the RF, with a stimulus bar, the background luminance change happened both inside and outside the RF. The similarity between bar and Gabor effects is potentially important as the Gabor surround was exclusively outside the RF.

RESPONSE TO A BRIEFLY PRESENTED STIMULUS.

In the experiments described in the preceding text, the stimulus duration was 550–600 ms, meaning that the initial and delayed responses we observed took place while the stimulus was on the visual display. However, when a stimulus bar was presented for a much briefer period of time (67 ms) with a simultaneous background luminance change, the response patterns were similar to the long duration situation. With the brief stimulus, V1 neurons showed a response that lasted up to a few hundred milliseconds after stimulus offset. The higher the stimulus contrast, the more persistent was the response. Two single neuron examples are shown in Fig. 9. To the briefly presented stimulus, some neurons showed an initial response and a delayed response (Fig. 9A), whereas other neurons showed only the delayed response (Fig. 9B). As with the long-duration stimuli, the initial response was relatively unaffected by contrast but the delayed response latency was anti-correlated with contrast. The similarity between response patterns with long- and short-duration stimuli suggests that the pattern is stereotypical and based largely on stimulus onset.

Background luminance change (decrement)

To test the generality of the findings described in the preceding text, we recorded the responses of some neurons when the background luminance change was a decrement rather than an increment. In doing this, we found delays in the responses and orientation selectivity similar to the situation with a luminance increment.

SINGLE-CELL EXAMPLES.

Figure 10 shows responses of two representative neurons to a bar of light in the static and the decrement background luminance conditions. Compared with the response in the static-background condition, one neuron showed a suppressed and delayed response in the background luminance decrement condition (Fig. 10A) and the other neuron showed only a delayed response (Fig. 10B).

POPULATION RESPONSE.

We tested the orientation selectivity of 17 units to a bar of light with a fixed contrast of 0.57 in the luminance decrement condition. The averaged PSTHs in response to the different orientations and backgrounds are shown in Fig. 10C. In the static-background condition, orientation selectivity appears early in the response. With the luminance decrement, the entire response is delayed and consequently orientation selectivity is delayed (note the timing relative to the vertical line in Fig. 10C). The mean response latency to the optimal orientation is 70 ms in the changing-background condition, significantly longer than the mean of 57 ms in the static-background condition (t-test, P = 0.01). Figure 10C, bottom, shows that orientation selectivity develops later in the changing-background condition.

As with a luminance increment, the off response in the decrement background luminance change condition also had a shorter time course than it did in the static-background condition (data not shown).

Background pattern change

So far, we have shown that the representation of form information in V1 is delayed when visual stimuli are introduced onto RFs with a background luminance change (either increment or decrement). Because in the natural environment, saccadic eye movements cause both background luminance and pattern changes, we set out to characterize V1 neurons’ responses when a stimulus bar is introduced onto RFs together with a background pattern change with fixed mean luminance.

BASIC RESPONSE PATTERNS (SINGLE-CELL EXAMPLES).

Figure 11 shows responses of four representative neurons to a bar of light in the static- and changing-pattern background conditions.

Suppression of the early response is common in the condition with the background pattern change. Figure 11A illustrates a neuron giving a strong transient response to a stimulus bar introduced onto a static pattern background. In the changing-pattern background condition, the late portion of the response was similar, but the transient was absent. This can be interpreted as either a suppression of the transient or a delay and suppression of the entire response. Analogous to the temporal response patterns observed with a background luminance change, some neurons displayed a biphasic response pattern (Fig. 11C) and some neurons showed only a delayed response (Fig. 11B) in the changing-pattern background condition. Additionally, some neurons showed a small statistically significant delay but otherwise little difference in response patterns in the static- and the changing-pattern background conditions (Fig. 11D). As the examples in Fig. 11 illustrate, there were sometimes marked differences in the response to a bar presented on static and changing pattern backgrounds. This is particularly surprising given that the change in the finely textured background pattern was perceptually quite subtle.

RESPONSES TO DIFFERENT ORIENTATIONS.

An oriented bar with a fixed contrast of 0.35 was used to characterize V1 neurons’ temporal responses to stimulus orientation. A total of 58 units were tested using the orientation paradigm in both the static- and changing-pattern background conditions.

Figure 12 shows averaged responses to a stimulus bar with different orientations in the static- and changing-pattern background conditions. In the static-background condition, the average response quickly climbed to a peak followed by a fall off and plateau (Fig. 12A). The changing-background response did not have the rapid peak, instead climbing more slowly to a response that appeared comparable to the plateau of the static response. Figure 12A, bottom, confirms that orientation selectivity develops more slowly in the changing-background condition. Figure 12B shows quantitatively that in an early time epoch (10–70 ms), there is greater orientation selectivity in the static-background condition. Selectivity appears at a later epoch (30–90 ms) in the changing-background condition. Across the population, the response latency to the optimal stimulus orientation in the changing-pattern background condition is significantly longer (paired t-test, P = 4.5 × 10⁻⁴) than in the static-background condition (Fig. 12C). The mean response latency in the changing-pattern background condition is 78 ms, compared to 63 ms in the static-pattern background condition.

Similar to the response difference observed after the stimulus onset, there was a comparable difference in the static and changing-background responses at stimulus offset: the transient peak in the offset response to the optimal orientation in the static-pattern background condition was absent when the stimulus bar was turned off simultaneously with a background pattern change (Fig. 13). Similar to the result with a background luminance change, the time course of the response after stimulus offset was shorter in the changing-pattern background condition compared with the static-pattern background condition.

RESPONSES TO DIFFERENT CONTRASTS.

A stimulus bar at five different contrasts (0.35, 0.40, 0.50, 0.60, 0.70) relative to the mean of the background pattern was used to characterize V1 neurons’ responses to stimulus contrast. A total of 33 units were tested with the contrast paradigm in both the static- and the changing-pattern background conditions.

In the changing-pattern background condition, the effects of stimulus contrast are similar to those elicited in the condition with a background luminance change. Figure 14A shows a single neuron’s response to a stimulus bar at different contrasts. The response to a stimulus bar shown against a static background has an abrupt onset; as the stimulus contrast increases, the response latency changes little (Fig. 14A, top). By contrast, when the stimulus is introduced with a background pattern change, the same neuron’s response is delayed; as the stimulus contrast increases, the latency tends to decrease and the latency shift is larger relative to the static-background condition.

Figure 14B shows the mean response latencies of the 33 units tested at different stimulus contrasts. At the stimulus contrasts of 0.35 and 0.40, the average response latencies in the changing-pattern background condition are significantly longer than those in the static-pattern background condition. Some individual neurons also showed significantly longer latencies at higher contrasts in the changing-background condition. However, the average response latencies at higher stimulus contrasts in the changing-pattern background condition, while longer, are not significantly longer, than those in the static-background condition. In the population average, as contrast was increased in the changing-pattern background condition, latency changed from 79 to 62 ms. The average effect of contrast in the static-background condition was much less—latency changed from 64 to 60 ms.

DISCUSSION

In natural behavioral situations, saccadic eye movements not only introduce new stimuli into RFs, they also cause changes in the background. In ongoing experiments, we are directly testing the consequences of saccades. However, implementing a “natural” saccade paradigm introduces multiple challenges. In the present study, we used a more easily controlled fixation paradigm but added background changes of the sort that would result from saccades. We have observed that when we introduce a stimulus along with a background change, V1 neurons show a qualitatively different temporal response pattern than they show to a stimulus introduced on a static background. This finding may have important implications for neural representations in natural visual situations.

Is the reduced paradigm realistic?

To allow tighter control of stimulus parameters, instead of using a complex natural image background, we chose to use a reduced paradigm with either a background luminance change or a background pattern change with fixed mean luminance. Can the reduced paradigm capture essential aspects of visual scene changes in the natural environment? We believe it can because both pattern and luminance changes are commonplace with saccades.

Because the natural environment is complex rather than uniform, saccadic eye movements cause frequent changes in the background pattern contained in and beyond RFs. In some of our experiments, we used a random-dot pattern as the background, something we viewed as a conservative approximation to a natural background. To human observers, the change in the fixed-mean-luminance background texture pattern is perceptually subtle. The fact that significant effects were observed even in this situation suggests that response changes in natural complex environments would be significant.

To demonstrate that saccadic eye movements in the natural environment may also lead to background luminance changes, we simulated saccades within a natural image and calculated the probability of encountering a change of luminance within an artificial window meant to represent an RF and its immediate surround (Fig. 15). The natural image was arbitrarily chosen and calibrated to match the mean luminance of the backgrounds used in the experiments. Of 5,000 simulated saccades, 57% led to an absolute luminance change that was >18.4 cd/m² (the larger luminance change used in our experiments); the percentage is higher (79%) for the smaller luminance change of 8.5 cd/m² used in the experiments. Most of these saccades involved a background pattern change in addition to the background luminance change.

This simulation must be viewed as a qualitative examination rather than a quantitatively accurate simulation of the natural effects of saccades on monkey receptive field stimulation. To accomplish the latter in a truly natural way would require recording fixations in a freely moving animal in the wild, something that to the best of our knowledge has not been done. Despite our inability to accurately mimic the natural situation, our simple analysis (and introspection) suggests that background luminance changes, along with pattern changes, are a frequent byproduct of saccades.

Temporal representation of visual information may be different in natural situations

Many neurons in visual cortex respond with an abrupt onset transient when a new stimulus is presented. Numerous studies have shown that the initial response onset carries significant information about the stimulus. This has been reported in areas V1 (Celebrini et al. 1993; Gawne et al. 1996; Heller et al. 1995; Müller et al. 2001; Reich et al. 2001; Vogels and Orban 1991; Zohary et al. 1990), V2 (Hegdé and Van Essen 2004) and MT (Osborne et al. 2004; Palanca and DeAngelis 2003). Consistent with these studies, we found that when a visual stimulus was presented on a static background, form information was reflected in the initial onset response of V1 neurons—the early response was orientation selective and contrast was anti-correlated with the initial response latency.

When a stimulus was introduced with a background change, the representation of form information in V1 was delayed with two different response patterns. For the reasons described in the previous section, it is our working hypothesis that the background changes create response patterns that are closer to natural responses. Compared with the response to a stimulus introduced on a static background, many V1 neurons showed a delayed response when there was a simultaneous background change. Orientation and contrast were represented as soon as the neuron responded in the changing-background condition, but this was significantly later than in the static-background situation. Other neurons showed a biphasic response pattern. In these neurons, the initial response was relatively unselective for the stimulus features and only in a delayed response period were differential responses to different orientations and contrasts observed. It appeared that the initial response signaled only that the scene had changed.

When there is a background change, the biphasic temporal response shown in many V1 neurons may be useful for extracting information about stimulus contrast. We observed that contrast was inversely correlated with the latency of the delayed phase of the response, so contrast could be inferred if latency could be measured. To obtain a measure of response latency, a straightforward approach is to compare the response onset time with the stimulus onset time, but the latter is not necessarily known to the visual system. Because the initial response latency was relatively unchanged as stimulus contrast varies in the changing-background condition (Fig. 8C), it might be used as a reference time or “sync” signal. In other words, stimulus contrast could be computed from the interval between the initial response and the delayed response.

One might wonder whether the different response patterns we observed in static and changing-background conditions could result from a difference in temporal contrast for the stimulus bar or Gabor. For example, with a background luminance increment, the temporal contrast at the location of the stimulus bar was higher than in the static-background condition. This difference in temporal contrast is unlikely to account for the results for two reasons. First, instead of observing a greater response and shorter response latency as one would expect with a higher temporal contrast, we found that the responses in the changing-background condition were actually suppressed and delayed. Second, with the texture background, the mean luminance of the texture patterns in the static and changing-background conditions were identical, thus there is no temporal contrast difference at the bar location between the two conditions. The fact that we found similar suppression and delay of V1 responses in the background pattern change condition suggests that these effects cannot be explained by the temporal contrast difference of the stimulus bar.

Our results suggest a change in perspective about when and how information is represented in neural responses. When a local stimulus is introduced simultaneously with a change in background luminance or pattern, the early V1 response is suppressed and relatively unselective for the attributes of the stimulus. Information about stimulus orientation and contrast is instead represented in a delayed response. This delayed representation may be more representative of natural visual situations in which saccades routinely change the luminance and pattern within and outside receptive fields.

Visual responses in natural situations may involve more extensive neural circuitry

In our experiments, the significance of background changes in shaping the response to a local stimulus implies that input from beyond the RF may play an important role in natural visual responses. Thus it may be critical that feedforward input is supplemented by additional input via lateral and/or feedback circuits. There is extensive connectivity within V1 and between V1 and other cortical areas, and in some situations, the scale of perceptual interactions appears consistent with the scale of these connections (Angelucci et al. 2002). Intracellular recordings in V1 of anesthetized cats show that electrical stimulation of the cortical afferents activates a cortical circuit that produces a stereotypical response pattern consisting of a transient depolarization followed by hyperpolarization (Douglas and Martin 1991). While electrical stimulation is hardly a natural input, the results obtained, combined with our data, suggest that broad activation of visual cortex either electrically or visually may produce a response (particularly an inhibitory component) that is not seen with focused visual stimulation of a receptive field.

There is ample evidence that stimuli placed outside the RFs of V1 neurons influence responses to stimuli in the RF (Allman et al. 1985; Bair et al. 2003; Blakemore and Tobin 1972; DeAngelis et al. 1994; Fries et al. 1977; Kapadia et al. 1999; Knierim and Van Essen 1992; Li et al. 2000; Nelson and Frost 1978; Nothdurft et al. 1999), even if the surround stimulus is an area of uniform luminance (MacEvoy et al. 1998). The dominant interaction in most studies is suppression from the surround. Also, previous physiological studies of visual masking in V1 (Bridgeman 1980; Lamme et al. 2002; Macknik and Livingstone 1998) have found that introducing a visual mask suppresses V1 responses.

Our paradigm differs from these previous studies. In our experiments, we did not compare conditions with different visual stimuli surrounding the RF, situations in which previous studies would predict different amounts of surround suppression. Instead, the only difference between the static and the changing-background paradigms was the background displayed before the stimulus onset. Nonetheless, the background change appears to have a dramatic effect on V1 responses.

Key role for response suppression by the background

While the data we collected cannot determine the mechanism underlying different response patterns, viewing our results in the context of earlier studies suggests explanations of our data that are more or less probable.

One possible explanation for the different temporal response patterns we observed with static and changing backgrounds is that they result from distinct afferent populations. For example, in the cat, there are lagged and nonlagged neurons in the LGN with different temporal response patterns (Hartveit and Heggelund 1992; Mastronarde 1987; Saul and Humphrey 1990). In the monkey LGN, parvocellular and magnocellular neurons have distinct temporal responses and they project to overlapping cortical populations (Callaway 1998; Nealey and Maunsell 1994). We believe it is unlikely that the response patterns of particular subsets of the V1 input can account for the different V1 response patterns in static and changing-background conditions. This is because for the same neurons, we observed significantly different response patterns when visual stimuli were introduced differently.

It is also unlikely that the delay and suppression of the V1 response that we observed are due to retinal adaptation (Drum 1984; Cleland and Enroth-Cugell 1970; Grabowski and Pak 1975; Purpura et al. 1990) because the increment and the decrement background luminance change and the fixed mean luminance background pattern change all gave rise to a similar delay and suppression of the V1 response. Also inconsistent with adaptation is our finding of suppression of both on and off responses despite opposite luminance changes at these transitions. If retinal adaptation played a key role in establishing the different temporal response patterns, we would expect to see opposite effects on the V1 response in the background increment and decrement conditions and opposite effects at stimulus onset and offset. We would also expect little or no response difference with a background change when there is no change in mean luminance.

A clue to a possible explanation of the results is the observation that the differences between static and changing-background responses are generally consistent with an appropriately timed increase in inhibition in the changing-background case. Indeed, there was a strong correlation between the response of a neuron in the BCA condition (background condition alone) and the response pattern in the changing-background condition (biphasic or delayed). When there was an early transient response in the BCA condition, the response to a bar plus background change was typically biphasic. For these neurons, it appeared that the background produced transient delayed inhibition so that there was an initial response to the bar plus background followed by response suppression and then rebound excitation. On the other hand, when there was little or no response in the BCA condition, the bar plus background response usually followed the delayed-only pattern. In this case, the suppression caused by the background is likely to be early and strong enough to counterbalance the excitation elicited by the local stimulus and the background change.

There are a number of previous studies that suggest that stimulation of RF surrounds sculpts the temporal response of V1 neurons. For example, Zipser et al. (1996) reported that a stimulus presented in the RF produced a more biphasic response if it was part of a figure rather than background. Also related to our results, it appeared that the figure/ground information was only present at a relatively delayed time relative to stimulus onset (but see Rossi et al. 2001). Kapadia et al. (1999) found that embedding a high-contrast stimulus in a surround of randomly oriented bars caused a suppression of the early transient response and a shift of spikes from the early to the late part of the response. By using temporal sequences of gratings, Bair et al. (2003) found that iso-orientation surround suppression usually arrives later than the RF excitation, but for some V1 neurons, suppression arrives sooner.

Response suppression also appears to be present at stimulus offset. When a visual stimulus was switched off simultaneously with a background luminance or pattern change, the transient off response in V1 was reduced and the time course of the off response was shortened relative to the situation when the stimulus was turned off on a static background. The reduction of the off response is consistent with the suppressive effects found with visual masking in the LGN (Schiller 1968) and V1 (Macknik and Livingstone 1998). A decrease of excitation to the stimulus offset and inhibition elicited by the background change may both contribute to the off response reduction when there was a background change.

In the changing-background condition, the V1 response to stimulus offset returned to baseline faster than in the static-background condition. This is potentially advantageous as it may improve temporal resolution in the midst of rapid stimulus changes resulting from the saccadic eye movements encountered in natural visual situations.

The relative strength of the local stimulus and the background change might be important in determining when the signal specific to stimulus features emerges in V1. With a background change, we found that the delay of the V1 response was reduced as the contrast of the local stimulus increased. To represent the feature information of a local stimulus, the visual system needs to overcome the “interference” produced by the background change. Two mechanisms could potentially contribute to this.

First, V1 neurons may integrate inputs from LGN neurons that respond to either the local stimulus or the background. If the LGN input related to the local stimulus is more sustained than that generated by a background change, the signal specific to the local stimulus will dominate the later V1 response. The faster the LGN responses to the local stimulus and the background diverge, the earlier the local stimulus specific response may emerge in V1. Second, differences in recurrent activity associated with responses to the RF and background stimuli could “amplify” the stimulus specific signal. Recurrent excitation is thought to play an important role in shaping basic response properties of cortical neurons such as orientation and direction selectivity (Adorjan et al. 1999; Ben-Yishai et al. 1995; Douglas et al. 1995; Somers et al. 1995; Sompolinsky et al. 1991; Suarez et al. 1995). Recurrent input might also imbue the selective neural response with contextual information that would not be present if the response was entirely determined by feedforward input. Our finding of persistent activity after a brief stimulus presentation is consistent with the idea of recurrent excitation in V1.

A new stimulus suddenly appearing in a receptive field, for example from a saccade, produces high “temporal contrast.” In many laboratory paradigms and in our static-background condition, high temporal contrast is restricted to a small patch of the visual field. In our changing-background condition, and presumably in natural vision, new stimuli appear throughout the visual field, producing widely distributed temporal contrast. Our data show that the distribution of temporal contrast across the visual field has significant effects on neuronal responses in V1. The psychophysical results in the following paper suggest that changes in the brain’s temporal response also have significant consequences for human visual perception.

GRANTS

This research was supported by the National Eye Institute and the Keck Foundation.

FOOTNOTES

• The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.