Shinsuke Suzuki*, Kazuhisa Niki, Syoken Fujisaki and Eizo Akiyama, “Neural basis of conditional cooperation”, Social Cognitive and Affective Neuroscience, Vol. 6, No. 3, pp. 338-347, 2011.
Shinsuke Suzuki* and Hiromichi Kimura, "Oscillatory dynamics in the coevolution of cooperation and mobility", Journal of Theoretical Biology, Vol. 287, No. 1, pp. 42-47, 2011.
Daniel W. Bryden, Emily E. Johnson, Steven C. Tobia, Vadim Kashtelyan, and Matthew R. Roesch
J. Neurosci. 2011;31 18266-18274
Learning theory suggests that animals attend to pertinent environmental cues when reward contingencies unexpectedly change so that learning can occur. We have previously shown that activity in basolateral nucleus of amygdala (ABL) responds to unexpected changes in reward value, consistent with unsigned prediction error signals theorized by Pearce and Hall. However, changes in activity were present only at the time of unexpected reward delivery, not during the time when the animal needed to attend to conditioned stimuli that would come to predict the reward. This suggested that a different brain area must be signaling the need for attention necessary for learning. One likely candidate to fulfill this role is the anterior cingulate cortex (ACC). To test this hypothesis, we recorded from single neurons in ACC as rats performed the same behavioral task that we have used to dissociate signed from unsigned prediction errors in dopamine and ABL neurons. In this task, rats chose between two fluid wells that produced varying magnitudes of and delays to reward. Consistent with previous work, we found that ACC detected errors of commission and reward prediction errors. We also found that activity during cue sampling encoded reward size, but not expected delay to reward. Finally, activity in ACC was elevated during trials in which attention was increased following unexpected upshifts and downshifts in value. We conclude that ACC not only signals errors in reward prediction, as previously reported, but also signals the need for enhanced neural resources during learning on trials subsequent to those errors.
Encoding of Both Positive and Negative Reward Prediction Errors by Neurons of the Primate Lateral Prefrontal Cortex and Caudate Nucleus
Wael F. Asaad and Emad N. Eskandar
The Journal of Neuroscience, 7 December 2011, 31(49): 17772-17787; doi: 10.1523/JNEUROSCI.3793-11.2011
Learning can be motivated by unanticipated success or unexpected failure. The former encourages us to repeat an action or activity, whereas the latter leads us to find an alternative strategy. Understanding the neural representation of these unexpected events is therefore critical to elucidate learning-related circuits. We examined the activity of neurons in the lateral prefrontal cortex (PFC) and caudate nucleus of monkeys as they performed a trial-and-error learning task. Unexpected outcomes were widely represented in both structures, and neurons driven by unexpectedly negative outcomes were as frequent as those activated by unexpectedly positive outcomes. Moreover, both positive and negative reward prediction errors (RPEs) were represented primarily by increases in firing rate, unlike the manner in which dopamine neurons have been observed to reflect these values. Interestingly, positive RPEs tended to appear with shorter latency than negative RPEs, perhaps reflecting the mechanism of their generation. Last, in the PFC but not the caudate, trial-by-trial variations in outcome-related activity were linked to the animals' subsequent behavioral decisions. More broadly, the robustness of RPE signaling by these neurons suggests that actor-critic models of reinforcement learning in which the PFC and particularly the caudate are considered primarily to be “actors” rather than “critics,” should be reconsidered to include a prominent evaluative role for these structures.
F. Pestilli, M. Carrasco, D.J. Heeger, and J.L. Gardner
Neuron, Volume 72, Issue 5, 832-846, 8 December 2011
The computational processes by which attention improves behavioral performance were characterized by measuring visual cortical activity with functional magnetic resonance imaging as humans performed a contrast-discrimination task with focal and distributed attention. Focal attention yielded robust improvements in behavioral performance accompanied by increases in cortical responses. Quantitative analysis revealed that if performance were limited only by the sensitivity of the measured sensory signals, the improvements in behavioral performance would have corresponded to an unrealistically large reduction in response variability. Instead, behavioral performance was well characterized by a pooling and selection process for which the largest sensory responses, those most strongly modulated by attention, dominated the perceptual decision. This characterization predicts that high-contrast distracters that evoke large responses should negatively impact behavioral performance. We tested and confirmed this prediction. We conclude that attention enhanced behavioral performance predominantly by enabling efficient selection of the behaviorally relevant sensory signals.
Annals of the New York Academy of Sciences
December 2011, Volume 1239, Pages 1–163
Orbitofrontal Cortex (OFC)の特集。
Balkanizing the primate orbitofrontal cortex: distinct subregions for comparing and contrasting values (pages 1–13)
Peter H. Rudebeck and Elisabeth A. Murray
Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world (pages 14–24)
Mark E. Walton, Timothy E.J. Behrens, MaryAnn P. Noonan and Matthew F.S. Rushworth
The orbitofrontal cortex and response selection (pages 25–32)
James J. Young and Matthew L. Shapiro
Contrasting reward signals in the orbitofrontal cortex and anterior cingulate cortex (pages 33–42)
Jonathan D. Wallis and Steven W. Kennerley
The orbitofrontal cortex, predicted value, and choice (pages 43–50)
Bernard W. Balleine, Beatrice K. Leung and Sean B. Ostlund
Orbitofrontal contributions to value-based decision making: evidence from humans with frontal lobe damage (pages 51–58)
Lesley K. Fellows
Representations of appetitive and aversive information in the primate orbitofrontal cortex (pages 59–70)
Sara E. Morrison and C. Daniel Salzman
Behavioral outcomes of late-onset or early-onset orbital frontal cortex (areas 11/13) lesions in rhesus monkeys (pages 71–86)
Jocelyne Bachevalier, Christopher J. Machado and Andy Kazama
Does the orbitofrontal cortex signal value? (pages 87–99)
Geoffrey Schoenbaum, Yuji Takahashi, Tzu-Lan Liu and Michael A. McDannald
The prefrontal cortex and hybrid learning during iterative competitive games (pages 100–108)
Hiroshi Abe, Hyojung Seo and Daeyeol Lee
Neuronal signals for reward risk in frontal cortex (pages 109–117)
Wolfram Schultz, Martin O’Neill, Philippe N. Tobler and Shunsuke Kobayashi
Contributions of the ventromedial prefrontal cortex to goal-directed action selection (pages 118–129)
John P. O’Doherty
The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives (pages 130–137)
Camillo Padoa-Schioppa and Xinying Cai
The value of identity: olfactory notes on orbitofrontal cortex function (pages 138–148)
Jay A. Gottfried and Christina Zelano
Population coding and neural rhythmicity in the orbitofrontal cortex (pages 149–161)
Cyriel M.A. Pennartz, Marijn van Wingerden and Martin Vinck
Victor de Lafuente and Ranulfo Romo
PNAS December 6, 2011 vol. 108 no. 49 19767-19771
Midbrain dopamine (DA) neurons respond to sensory stimuli associated with future rewards. When reward is delivered probabilistically, DA neurons reflect this uncertainty by increasing their firing rates in a period between the sensory cue and reward delivery time. Probability of reward, however, has been externally conveyed by visual cues, and it is not known whether DA neurons would signal uncertainty arising internally. Here we show that DA neurons code the uncertainty associated with a perceptual judgment about the presence or absence of a vibrotactile stimulus. We observed that uncertainty modulates the activity elicited by a go cue instructing monkey subjects to communicate their decisions. That is, the same go cue generates different DA responses depending on the uncertainty level of a judgment made a few seconds before the go instruction. Easily detected suprathreshold stimuli elicit small DA responses, indicating that future reward will not be a surprising event. In contrast, the absence of a sensory stimulus generates large DA responses associated with uncertainty: was the stimulus truly absent, or did a low-amplitude vibration go undetected? In addition, the responses of DA neurons to the stimulus itself increase with vibration amplitude, but only when monkeys correctly detect its presence. This finding suggests that DA activity is not related to actual intensity but rather to perceived intensity. Therefore, in addition to their well-known role in reward prediction, DA neurons code subjective sensory experience and uncertainty arising internally from perceptual decisions.
Jamil Zaki and Jason P. Mitchell
PNAS December 6, 2011 vol. 108 no. 49 19761-19766
Standard economic and evolutionary models assume that humans are fundamentally selfish. On this view, any acts of prosociality—such as cooperation, giving, and other forms of altruism—result from covert attempts to avoid social injunctions against selfishness. However, even in the absence of social pressure, individuals routinely forego personal gain to share resources with others. Such anomalous giving cannot be accounted for by standard models of social behavior. Recent observations have suggested that, instead, prosocial behavior may reflect an intrinsic value placed on social ideals such as equity and charity. Here, we show that, consistent with this alternative account, making equitable interpersonal decisions engaged neural structures involved in computing subjective value, even when doing so required foregoing material resources. By contrast, making inequitable decisions produced activity in the anterior insula, a region linked to the experience of subjective disutility. Moreover, inequity-related insula response predicted individuals’ unwillingness to make inequitable choices. Together, these data suggest that prosocial behavior is not simply a response to external pressure, but instead represents an intrinsic, and intrinsically social, class of reward.