Abstract

Neural Activity in OFC and OFC-Dependent Behavior Reflect Outcome Expectancies

The ability to maintain information so that it can be manipulated, integrated with other information, and then used to guide behavior has been described as working or representational memory, and it depends on prefrontal cortex (Goldman-Rakic, 1987). Within prefrontal cortex, the orbitofrontal cortex (OFC), by its connections with limbic areas, is positioned to allow associative information regarding outcomes access to representational memory. Studies suggest that a neural correlate of expected outcome value is present in OFC. For example, imaging studies show changes in blood flow in OFC during anticipation of outcomes and when the value of the outcome is modified (Gottfried et al., 2003; Nobre et al., 1999; O'Doherty et al., 2002). Furthermore, neural activity in OFC preceding predicted rewards or punishments reflects the values of these outcomes (Roesch and Olson, 2004; Schoenbaum et al., 1998, 2003b; Tremblay and Schultz, 1999).

These responses are acquired as animals learn what outcome to expect and can be observed in the absence of signaling cues. These features are illustrated in Figure 1, which shows an OFC neuron recorded in a rat learning to discriminate between two odors on the basis of the outcomes they predict. The rat must learn that one odor predicts availability of sucrose solution in a nearby fluid well, while the other odor predicts availability of quinine solution. Early in learning, this neuron responds when the sucrose reward is presented and also prior to sucrose presentation, during a delay after the response has been made. During this delay, activity in this neuron effectively signals the impending outcome, even though there are no external cues available to indicate which will be delivered. We have found that about 20% of the neurons in OFC develop such activity, firing in anticipation of either sucrose or quinine (Schoenbaum et al., 1998, 2003b). The activity in this population reflects the value of the expected outcome of the response, maintained in representational memory. In addition, many of these neurons, including the example in Figure 1, also come to be activated by the cues that predict their preferred outcomes after learning, thereby signaling the expected outcome before a response is made. Together, these neural signals may bias behavior to reflect expected outcomes and facilitate learning when expectations for outcomes are violated.

Open in a separate window

Figure 1

Neural Correlates of Outcome Expectancies in OFC

Single-unit activity is shown in raster format and perievent time histogram, synchronized to odor onset (100 ms bins; dotted line is 99% CI; top panels, Odor 1+; bottom panels, Odor 2−; odor sampling, gray shading). After odor sampling, the rat had to respond at the fluid well. On odor 1 and 2 trials, the response produced sucrose and quinine, respectively. A delay (pre-sucrose, green; pre-quinine, red) interposed between a response at the well and fluid delivery serves to isolate neural activity signaling the expected outcome.

The notion that OFC facilitates learning and guides behavior by generating outcome expectancies is consistent with the effects of OFC damage on behavior. These effects are evident when the appropriate response requires associations to be considered in the context of other cues to evaluate competing responses. This is often the case in learning tasks in which the values of the predicted outcomes are ambiguous. For example, humans with OFC damage are unable to guide behavior in the Iowa gambling task (Bechara et al., 1999). In this task, subjects choose cards that have high rewards and occasional large penalties or low rewards but no penalties. Normal subjects initially choose from high-reward decks but later bias their choices toward the low-reward decks as the penalties become apparent. OFC-damaged patients also initially select more often from the high-reward decks; however, they fail to modify their responding to reflect the penalties, as if they are unable to appreciate the penalties. Integrating information about the probabilistic penalties would be facilitated by an ability to maintain information about the value of the expected outcome in representational memory so that violations of this expectation could be recognized.

The importance of signaling outcome expectations, and the critical role of OFC in this process, is also reflected in emotional reactions to outcomes. These reactions were disrupted in a study of OFC-damaged patients making choices between two stimuli that predicted punishment or reward (Camille et al., 2004). After each trial, the subjects rated their affect from sad to happy. The affective ratings of controls varied with the amount of reward they received, but their rating also changed when they were made aware of the reward or punishment associated with the stimulus they did not select. For example, a small reward made them happier when they knew that they had avoided a large penalty. Normal subjects also performed better when they received such feedback. Patients with OFC damage showed normal emotional responses to rewards and punishments; however, knowledge about the unselected outcome had no effect on either their affective ratings or performance. The failure of OFC-lesioned patients to modulate emotional reactions based on this comparison may reflect an inability to generate outcome expectancies required to compare the relative value of the selected and unselected outcomes.

The critical role of OFC in signaling outcome expectancies is also evident in reinforcer devaluation tasks. These tasks assess the control of behavior by an internal representation of the value of an expected outcome. In a Pavlovian version of this procedure, rats are trained to associate a light cue with food, and then the food is devalued by pairing it with illness. When the light cue is presented again in an extinction session, rats for which the food has been devalued respond less to the cue than nondevalued controls This decrease in responding is in addition to the normal decrease caused by extinction. Thus, it is thought to reflect the use of an internal representation of the new value of the food, in concert with the original light-food association. These two pieces of information are combined without reexposure to the food outcome, which presumably requires representational memory.

OFC-lesioned rats fail to change their behavior in this setting after devaluation (Gallagher et al., 1999). These rats condition and devalue normally; however, they show no effect of devaluation on conditioned responding. They continue to respond to the light to obtain the food, even though they will not eat it if it is presented. This effect is observed whether lesions are made before or after learning, indicating that OFC is not solely involved in acquiring the cue-outcome association (Pickens et al., 2003). Rather, OFC is critical to control conditioned responding according to internal representations of the new value of the expected outcome. Similar results have been reported in monkeys (Izquierdo et al., 2004).

The idea that OFC plays a critical role in behavior by virtue of its importance in signaling outcome expectancies is at least partially consistent with other theories of OFC function. An old idea regarding OFC function is that OFC is critical for inhibiting existing responses. By our view, OFC is critical for this because the outcome expectancies generated by OFC allow the animal to contrast predicted and actual outcomes to facilitate learning. It has also been proposed that OFC biases behavior by allowing cues to activate representations of the outcome's emotional valence (Bechara et al., 1999). Although this account has come under scrutiny of late (Maia and McClelland, 2004), our proposal would contradict it insofar as we do not believe OFC is critical for associative learning (Bechara et al., 1999). Most recently, it has been proposed that OFC normalizes the value of competing outcomes (Montague and Berns, 2002), so that one can compare, say, apples and oranges. We would differ with this account only in the emphasis it places on the encoding of value. Clearly, value is an important component of the information represented in OFC, but it is also a component of encoding elsewhere. Roberts and colleagues have suggested that OFC allows the comparison of values of selecting goals (Arana et al., 2003). We would suggest that the ability to compare relative values of different goals arises from the role OFC plays in generating expectancies.

Differentiating Orbitofrontal and Amygdalar Contributions

Much of the evidence for the role of OFC in signaling outcome expectancies can also be found in studies of amygdalar function. Damage to basolateral amygdala (ABL) causes OFC-like deficits in the devaluation task and in discounting tasks (Bechara et al., 1999; Hatfield et al., 1996; Malkova et al., 1997; Winstanley et al., 2004). In addition, damage to either area alters the ability of a cue to serve as a conditioned reinforcer (Parkinson et al., 2001; Pears et al., 2003). These findings indicate that there is substantial overlap in the function of ABL and OFC.

Other evidence suggests a dissociation of these functions such that ABL is critical for acquiring cue-outcome associations, while OFC is involved in allowing that information to be utilized to generate expectancies to guide behavior. For example, in a recent imaging study, BOLD signal in amygdala correlates with the incentive value of menu items, whereas signal in OFC correlated with the value only when it was the basis of a decision (Arana et al., 2003). Similarly, in single-unit studies, cue-selective firing in ABL emerges earlier during learning and tracks the associated outcome with greater fidelity than comparable activity in OFC. By contrast, encoding in OFC is more strongly related to the use of that information to guide accurate responding (Schoenbaum et al., 1998, 1999).

Recently, we have tested this hypothesis directly. OFC neurons were recorded from ABL-lesioned rats as they learned and reversed odor discriminations (Schoenbaum et al., 2003b). ABL lesions caused a loss of associative information in OFC, particularly during cue sampling (“ABL lesion”; Figure 2). ABL lesions reduced the proportion of cue-selective neurons in OFC, and those present were less associative. Furthermore, ABL lesions abolished the activation of outcome-expectant neurons during cue sampling. At the same time, firing after responding in anticipation of the outcomes was largely unaffected by ABL lesions. These results are consistent with a model in which ABL is required to input and store associative information concerning the value of outcomes in OFC.

Open in a separate window

Figure 2

OFC-ABL Interactions in Establishing Neural Representations

ABL lesions have no effect on outcome-expectant activity in OFC but abolish the activation of these cells during cue sampling. By contrast, OFC lesions disrupt outcome-expectant activity in ABL. C, cue; D, delay before outcome; O, outcome.

By contrast, OFC lesions disrupted outcome-expectant firing in ABL (Saddoris et al., 2005) (“OFC lesion”; Figure 2). The loss of outcome-expectant firing suggests that this signal is generated in OFC. ABL neurons also became cue-selective much more slowly without input from OFC, particularly after the cue-outcome associations were reversed. Slower associative encoding in ABL as a result of OFC lesions is consistent with the idea that learning in other structures should be facilitated by the outcome expectancies generated in OFC.

As illustrated in Figure 2, OFC and ABL normally interact in intact rats, such that outcome-expectant encoding from OFC influences processing in ABL to facilitate rapid learning, while associative encoding in ABL influences processing in OFC to allow the storage of these associations so that they may be used to guide behavior. This model provides a potential explanation for the different deficits that have been reported after ABL and OFC lesions. For example, OFC but not ABL lesions slow reversal learning (Schoenbaum et al., 2003a). This would be expected if output from OFC were facilitating reversal of associative encoding in ABL (Saddoris et al., 2005). This model is also consistent with results from the Iowa gambling task (Bechara et al., 1999). Patients with ABL damage, like OFC-damaged subjects, fail to bias responding away from large-reward decks and do not generate autonomic signals when making choices from these decks. However, patients with ABL damage are also unable to generate autonomic signals during conditioning in which a cue is paired with a loud noise, whereas OFC-lesioned subjects retain this ability. These data show that ABL damage affects associative learning, whereas OFC damage does not.

The model presented in Figure 2 also predicts different effects of ABL and OFC damage on behavior, depending on whether that damage is sustained before or after learning. For example, ABL lesions made before but not after learning impair the ability of a cue to support second-order conditioning or conditioned reinforcement (Cousens and Otto, 2003; Malkova et al., 1997; Setlow et al., 2002). Similarly, ABL lesions made before but not after learning in the devaluation task impair the normal reduction in conditioned responding. By contrast, OFC lesions made after learning continue to affect behavior in these settings (Cousens and Otto, 2003; Pears et al., 2003; Pickens et al., 2003).

A similar dissociation has also been reported in a delay-discounting task. In this task, rats choose between small, immediate rewards or larger, delayed rewards. Rats normally respond for the large reward, but if the delay becomes longer, rats change their behavior and instead choose the small, immediate reward. OFC-lesioned rats exhibit abnormal breakpoints for choosing the small rather than the large reward. Rats lesioned before any learning exhibit “impulsive” responding, defined as responding for the small reward even when the large reward is available at relatively short delays. Rats lesioned after learning exhibit “perseverative” responding, defined as responding for the large reward at longer delays than controls (Mobini et al., 2002; Winstanley et al., 2004).

This difference may reflect the dual roles OFC plays in this paradigm. One role is to allow associative information from ABL access to representational memory. This role is important in performance, because it allows the comparison of the expected values of immediate and delayed rewards. Thus, rats lesioned after initial training know which responses produce the large and small rewards but are unable to maintain this information in representational memory. Therefore, they are unable to discount the value of the large reward, which results in “perseverative” responding.

However, OFC also helps facilitate associative learning in ABL, particularly during initial training. As a result, rats lesioned before any training fail to encode the actual associations normally. This deficit renders meaningless any loss of the delayed discounting function in OFC. The resultant behavior would depend on the associability of the response-outcome contingencies. We would speculate that “impulsive” responding was observed because the large reward was often delayed in training, resulting in a weaker association with the response.

By this account many behaviors that reflect the straightforward use of easily learned associations might be affected by ABL but not OFC lesions. A review of the literature shows this is in fact true, for tasks as diverse as fear conditioning and CS-potentiated feeding. These tasks typically do not engage representational memory systems to manipulate the associative information encoded by downstream limbic areas. As such, we would agree with Dr Goldman-Rakic that OFC, like other prefrontal regions, makes a critical contribution to behavior “when internalized or inner models of reality are used to govern behavior” (Goldman-Rakic, 1987).

Acknowledgments

Selected Readings