Fig. 1 STDP and conditioned responses. (A) The fractional strengthening or weakening of a synapse per spike pair as a function of the time difference between the pre- and postsynaptic action potentials, also known as the STDP window function. (B) The learning (more ...)

The ability of animals to detect temporal correlations and use them to make predictions is a basic result of classical condition. More recent experiments have found neural correlates of prediction. Dopaminergic neurons initially respond to rewards such as food or juice, but over repeated pairings, they lose these direct reward responses and instead begin responding to stimuli that predict reward or even to stimuli that predict stimuli that predict reward (3). In the hippocampus, the receptive fields of place cells shift and expand after repeated traversals of a path, making them respond to positions that precede their initial place field during the run (4, 5) and leading to a predictive map of space (6). To connect such predictive activity to synaptic plasticity, we consider the fate of a single neuron within the complex circuitry involved in actual conditioning behavior. The key issue is whether a form of synaptic plasticity that depends on spike timing at the millisecond timescale can induce modifications in the response of this model neuron that depend on temporal sequences occurring over vastly longer timescales.

It is well known that correlations between pre- and postsynaptic firing can lead to synaptic potentiation or depression through spike-timing-dependent forms of long-term synaptic plasticity (6–11). Our purpose is to explore whether this mechanism and the firing patterns it requires are a plausible way of bridging the gap in time scales between synaptic plasticity and behavior.

The amount of synaptic potentiation or depression due to STDP is typically measured by pairing a number of pre- and postsynaptic action potentials with a specified time interval between them (2, 12–17). This procedure results in an STDP “window function” such as that shown in Fig. 1A, which indicates the amount of synaptic modification caused by pairing pre- and postsynaptic spikes as a function of the time interval between them. If firing rates are low enough so that only a single spike pair typically appears within the STDP window at any given time, the effects of individual spike pairs appear to sum fairly linearly, although this breaks down at higher firing rates (15, 16). Within this linear regime, the total impact of a sequence of spike pairings can be computed by calculating the integral of the STDP window function multiplied by the cross-correlation function of the pre- and postsynaptic spike trains (6–11). This calculation is equivalent to determining the total amount of synaptic modification over a trial by multiplying the amount of synaptic change in Fig. 1A for a particular interspike interval by the probability of a pre-/postsynaptic spike pair occurring with that interval, and then integrating it over all possible intervals. Experimental work supports the validity of this method for calculating the total amount of potentiation or depression produced by STDP (14, 17).

For STDP to produce net potentiation over time, pre- and postsynaptic spike sequences must be correlated in such a way that it is more likely for a pre-then-post temporal ordering to occur than a post-then-pre sequence. Such correlations require an appropriate relationship between time-dependent pre- and postsynaptic firing rates. An obvious requirement, shared by any Hebbian mechanism of plasticity, is that pre- and postsynaptic firing must overlap in time, at least to within the interval covered by the STDP window function. When two stimuli are separated by seconds, as is the case between the conditioned stimulus (CS) and the unconditioned stimulus (US) in the classical conditioning paradigm, we consider that this is not an easy requirement to satisfy. Our approach to this problem is to assume that both the US and CS generate persistent responses that decay over time but nevertheless outlast these stimuli long enough to span the interval between them. Modeling the mechanism that sustains these responses and makes them decay slowly is beyond the scope of our work, but we discuss possible mechanisms in Discussion.

We begin our analysis by looking at cross-correlations between spike sequences generated by neurons excited by stimuli that then exhibit sustained but slowly decaying activity. Fig. 2 Inset shows the pattern of pre- and postsynaptic firing we consider. We assume that the postsynaptic neuron and the presynaptic afferents are driven by separate stimuli. The stimulus that excites the presynaptic afferents corresponds to the CS in a classical conditioning paradigm, whereas the stimulus that excites the postsynaptic neuron is the US. Both stimuli drive their targets to fire at a constant rate of 45 Hz for 1 s. The interval between the US and the CS is defined as T, so that the CS precedes the US if T < 0 and follows it if T > 0. After stimulation, we assume that the firing rates are sustained but decay slowly back to zero with an exponential time constant of 2 s (Fig. 2 Inset).

Fig. 2 Cross-correlations resulting from decaying firing patterns. Inset shows the firing rates of two neurons (one presynaptic and one postsynaptic), and the two panels indicate the resulting cross-correlations, normalized to the value for synchronous spikes. (more ...)

Fig. 2 shows cross-correlations between pre- and postsynaptic spike trains with rates as shown in Fig. 2 Inset that decay exponentially after a sustained plateau, with the decay constants for the pre- and postsynaptic firing rates given by τ_pre and τ_post. Slowly decaying firing rates create asymmetries in these cross-correlations over the time range relevant for STDP, even when the stimuli that excite the neurons are separated by many seconds. Furthermore, the ordering of the initiation of firing is preserved in the ordering of spikes, as revealed by the different slopes in Fig. 2. When presynaptic firing is initiated before postsynaptic firing (T < 0), presynaptic spikes tend to lead postsynaptic spikes more often than the other way around, even on a millisecond timescale (Fig. 2 Left). The situation is reversed if postsynaptic firing is initiated before presynaptic activity (T > 0; Fig. 2 Right). This effect is due to the slow decay of the firing rate, not simply to the order of firing. Because the correlations in Fig. 2 are normalized and the firing rate decays are exponential, the same plots apply independent of the magnitude of T, but the normalization factor is proportional to exp(−|T|/τ_pre) when T < 0 and exp(−T/τ_post) when T > 0. Thus, the effect gets very small if T is large relative to τ_pre and τ_post. In general, these asymmetries are small, as in Fig. 2, but they are sufficient to evoke significant levels of synaptic plasticity.

To examine the effect of these cross-correlations on STDP, we drove a single integrate-and-fire model postsynaptic neuron with 1,000 presynaptic afferents. The excitatory synapses were subject to STDP given by the window function in Fig. 1A. Presynaptic afferents fired in response to stimulus 1 as in Fig. 2 Inset (i.e., for 1 s at 45 Hz followed by an exponential decay to zero with a time constant τ_pre = 2 s). In this case, the US and the firing rate decay following it were represented by a current injected into the postsynaptic neuron that was constant for 1 s, producing firing at 45 Hz, and then decayed exponentially to zero with a time constant τ_post = 2 s. This current thus contains contributions arising directly from the US and from whatever mechanism sustains the response.

For Fig. 3, we activated the CS 5 s before the US (as in the Fig. 2 Inset). Fig. 3A shows the situation at the beginning of our simulations. All synapses began at zero strength (histogram in Fig. 3A Right), so the postsynaptic neuron only responded to the US (membrane potential trace in Fig. 3A Left). However, after 20 trials, the synapses were potentiated by STDP due to the asymmetry in pre–post spike correlations shown in Fig. 2. As seen in Fig. 3B, the broadened distribution of synaptic strengths leads to a depolarization of the postsynaptic potential in response to the CS acting through the strengthened synapses. After 40 trials (Fig. 3C), enough synapses have been strengthened to generate robust responses to both stimuli. Due to its response to the CS, the postsynaptic neuron has “learned” the association between the two stimuli.

Fig. 3 Development of a conditioned response. A CS drives the afferents to a postsynaptic neuron at 45 Hz from time 1s to 2s, followed by an exponential decay of afferent rates with a decay time constant of 2 s. The postsynaptic neuron is driven by a US, represented (more ...)

Fig. 4 shows the effect of interstimulus timing and ordering on the association learning. When the interstimulus interval is shortened to 4 s, an even stronger response to the CS is generated after 40 trials (Fig. 4A). When the interstimulus interval is lengthened to 6 s, there is insufficient potentiation after 40 trials to drive spikes, although the postsynaptic neuron is depolarized by the CS (Fig. 4B). Finally, if the order of the stimuli is reversed, so that the US drives the postsynaptic neuron before the CS excites its afferents, no strengthening of the synapses occurs, and no postsynaptic response to the CS is generated (Fig. 4C). The reverse ordering produces weakening rather than strengthening of the synapses, but this is not evident in this simulation because the synapses started out at zero strength and could not be weakened.

Fig. 4 Effect of timing and order on the conditioned response. The format and procedures are identical to those in Fig. 3 except for the timing and ordering between the CS and the US. All results are after 40 trials. (A) When the CS precedes the US by 4 s (rather (more ...)

The sustained but slowly decaying component of the firing of the postsynaptic neuron is not important for the effects shown in Figs. 3 and 4. Virtually identical effects can be produced by having the postsynaptic neuron fire only for the 1-s interval while the US is being presented. We include the postsynaptic firing decay, however, because it plays an important role in accounting for the results of Fig. 1B, as discussed below (see Fig. 1C). The period of constant firing we have assumed is not necessary, but increases the asymmetry in the cross-correlation and consequently increases the amount of synaptic change per trial. Finally, we have assumed that the effects of separate spike pairs due to STDP sum linearly. Nonlinear effects have been observed when both pre- and postsynaptic neurons fire simultaneously >40 Hz (15, 16), which does not happen in our examples. Although it is not clear how to describe the nonlinear summation, we have tried to model it with saturating long-term potentiation (LTP) (16), and the effects shown in Figs. 3 and 4 were still found. In addition, we can reproduce these results with significantly lower rates as well. We can also introduce jitter into the postsynaptic response and preserve the effects. The essential correlations can even be induced in a Poisson model, as in Fig. 2.

To further clarify the phenomenon being studied, we computed the amount of synaptic modification analytically as a function of the interstimulus interval T, the pre- and postsynaptic firing rate decay constants, τ_pre and τ_post, and the parameters of the STDP window function (Fig. 1A). Details of this calculation are presented in Methods. To make these calculations tractable, we assume that the synaptic strengths are not close to their lower or upper limits, and we do not include the effects of the afferents on postsynaptic firing. Under these assumptions, when the stimulus driving the presynaptic neuron (the CS) precedes that driving the postsynaptic neuron (the US) (T < 0), synapses strengthen by an amount:

In our model, asymmetric cross-correlations allow a fast correlation-based mechanism of synaptic plasticity to react to associations that involve timescales relevant to behavior. In the case we considered, sustained activity that decays away slowly allows associations to be made between events across timescales that are far greater than the width of the STDP window. In our simulations, we do not explicitly model the mechanism for sustaining activity or for making it decay. Reverberations in a network with high gain could generate such a pattern of activity if slightly detuned from the conditions required to sustain constant firing (18, 19). Slow cellular or synaptic mechanisms could also contribute (20–22). Persistent activity lasting several seconds or more in response to a transient input has been observed in single neurons (23), in slice culture (24), in the locust antennal lobe (25), and in the prefrontal cortex of monkeys (26). Thus, it is not unlikely that an odor or shock will generate a long-lasting response.

There are a number of other ways to produce the asymmetric cross-correlations needed in the model. Up and down states (27, 28) might generate asymmetries if they occurred in an appropriate temporal sequence. More complicated cellular or network mechanisms that, for example, change the latency or order of spikes could generate correlations that favor potentiation or depression depending on the history of stimulation. Network mechanisms could also cause changes in the spontaneous background activity, altering correlations, as a form of off-line learning (29). Whatever the source, long time correlations that could support the type of learning we have presented have been observed in spike trains (30, 31), spike counts (32), and local field potentials (33) from many different areas.

Our model requires a relative fine balance between LTP and long-term depression (LTD) for the synaptic weights to respond to the asymmetric rather than the symmetric component of the unsubtracted cross-correlation. We found that reliable formation of temporally sensitive associative plasticity in the example of Fig. 3 requires the ratio of the area under the LTD part of the STDP window to the area under the LTP part (A₋τ₋/A₊τ₊) to be between 1.00 and 1.01. Such tuning is a universal feature in such cases when a small asymmetry (the asymmetry of the cross-correlation function) must be detected on top of a larger background. If tight control is not maintained on the balance between synaptic potentiation and depression due to STDP, uncorrelated background firing will have a dominate effect, and synapses will be pushed to the upper or lower limits of strength independent of the temporal correlations of the pre- and postsynaptic inputs. Mechanisms for dynamically regulating this balance might control it automatically without requiring fine tuning of parameters (34).

The model we studied requires tens of trials for the association to be made, but this number can be decreased simply by increasing the amplitude of the STDP. We have used a form of STDP in which hard boundaries are imposed at the limits of the range of allowed synaptic conductances, but other models with soft bounds have been proposed and studied (35–38). The upper boundary played no role in our simulations because it was never reached, but the lower boundary is needed to prevent synaptic conductances from becoming negative. It is not essential that this boundary is hard, but it is critical that the pre–postsynaptic correlations, not the boundary constraints, are the dominant factor in determining the fate of the synapses through STDP.

We considered only one synaptic relay, but the same mechanism active during the presentation of reliably repeated sequences of events to more complicated networks should lead to the formations of synfire chains (39, 40). The position of a neuron in such a chain will then correspond to the position of the event in the driving sequence. In this case, the mechanism that we propose would link the timescale of behavioral events to the much faster neuronal firing sequences of the synfire chain, resulting in a temporally compressed representation (41–43).

Correlations. The correlation shown in Fig. 2 is:

Simulations. The model we use for all of the simulations is similar to that described in ref. 10. The postsynaptic neuron shown in Figs. 3 and 4 is described by an integrate-and-fire model with membrane potential V and driving current I satisfying:

_a

_a

During the simulations for Figs. 3 and 4 (with obvious modifications), the postsynaptic neuron was driven by a current:

STDP is modeled as ref. 10 by introducing the functions M(t) and P_a(t) for a = 1, 2, …, N satisfying:

_a

_a

_a

_a

_a

_a

_a

_a

_a

Analytic Calculation. For pre- and postsynaptic firing rates r_pre(t) and r_post(t), the amount of LTP over one trial is given by:

We take the presynaptic rate, r_pre, to be exponentially decaying with time constant τ_pre from an initial value R_pre and zero for t < 0. We take the postsynaptic rate to be zero for t < T and, for t ≥ T, r_post(t) = R_post exp(−(t − T)/τ_post). The analytic result reported in the text comes from integrating the equations for LTP and LTD given above with these rates. To simplify the resulting equations, we have taken the area under the STDP curve to be zero (A₋τ₋ = A₊τ₊), and we have removed terms that are negligibly small when, as is the case here, τ_pre, τ_post, and |T| are all much larger than τ₊ and τ₋. The condition A₋τ₋ = A₊τ₊ is not required for the results we report; the requirement is actually τ₊τ₋(A₊ + A₋) > τ_pre(A₋τ₋ − A₊τ₊), which requires A₋τ₋ − A₊τ₊ to be small but not necessarily zero.

A useful form of the above results (8, 9, 11) can be obtained if the pre- and postsynaptic firing rates vary slowly on the scale of τ₊ and τ₋. Then, by Taylor expanding r_pre and r_post, we find that the total plasticity Δ = LTP − LTD is given by:

We thank S. Fusi and S. Nelson for valuable discussions and J. Lisman for encouragement. This research was supported by National Institutes of Health (NIH) Grant MH58754, the Swartz Foundation, and an NIH Director's Pioneer Award, part of the NIH Roadmap for Medical Research, through Grant 5-DP1-OD114-02.