Dynamic Coding for Cognitive Control in Prefrontal Cortex

Mark G. Stokes, Makoto Kusunoki, Natasha Sigala, Hamed Nili, David Gaffan, John Duncan
Neuron, Volume 78, Issue 2, 364-375, 04 April 2013

Cognitive flexibility is fundamental to adaptive intelligent behavior. Prefrontal cortex has long been associated with flexible cognitive function, but the neurophysiological principles that enable prefrontal cells to adapt their response properties according to context-dependent rules remain poorly understood. Here, we use time-resolved population-level neural pattern analyses to explore how context is encoded and maintained in primate prefrontal cortex and used in flexible decision making. We show that an instruction cue triggers a rapid series of state transitions before settling into a stable low-activity state. The postcue state is differentially tuned according to the current task-relevant rule. During decision making, the response to a choice stimulus is characterized by an initial stimulus-specific population response but evolves to different final decision-related states depending on the current rule. These results demonstrate how neural tuning profiles in prefrontal cortex adapt to accommodate changes in behavioral context. Highly flexible tuning could be mediated via short-term synaptic plasticity.


The dorsomedial striatum encodes net expected return, critical for energizing performance vigor

Alice Y Wang, Keiji Miura & Naoshige Uchida
Nature Neuroscience 16, 639–647 (2013)

背内側線条体はNet Value、つまり「二つの選択肢の報酬量の和」をコードしている。

Decision making requires an actor to not only steer behavior toward specific goals but also determine the optimal vigor of performance. Current research and models have largely focused on the former problem of how actions are directed while overlooking the latter problem of how they are energized. Here we designed a self-paced decision-making paradigm, which showed that rats' performance vigor globally fluctuates with the net value of their options, suggesting that they maintain long-term estimates of the value of their current state. Lesions of the dorsomedial striatum (DMS) and, to a lesser degree, in the ventral striatum impaired such state-dependent modulation of vigor, rendering vigor to depend more exclusively on the outcomes of immediately preceding trials. The lesions, however, spared choice biases. Neuronal recordings showed that the DMS is enriched in net value–coding neurons. In sum, the DMS encodes one's net expected return, which drives the general motivation to perform.


Dopamine restores reward prediction errors in old age

Rumana Chowdhury, Marc Guitart-Masip, Christian Lambert, Peter Dayan, Quentin Huys, Emrah Düzel & Raymond J Dolan
Nature Neuroscience 16, 648–653 (2013)

Senescence affects the ability to utilize information about the likelihood of rewards for optimal decision-making. Using functional magnetic resonance imaging in humans, we found that healthy older adults had an abnormal signature of expected value, resulting in an incomplete reward prediction error (RPE) signal in the nucleus accumbens, a brain region that receives rich input projections from substantia nigra/ventral tegmental area (SN/VTA) dopaminergic neurons. Structural connectivity between SN/VTA and striatum, measured by diffusion tensor imaging, was tightly coupled to inter-individual differences in the expression of this expected reward value signal. The dopamine precursor levodopa (L-DOPA) increased the task-based learning rate and task performance in some older adults to the level of young adults. This drug effect was linked to restoration of a canonical neural RPE. Our results identify a neurochemical signature underlying abnormal reward processing in older adults and indicate that this can be modulated by L-DOPA.


Distinct extended amygdala circuits for divergent motivational states

Joshua H. Jennings, Dennis R. Sparta, Alice M. Stamatakis, Randall L. Ung, Kristen E. Pleil, Thomas L. Kash & Garret D. Stuber
Nature 496, 224–228 (11 April 2013)

The co-morbidity of anxiety and dysfunctional reward processing in illnesses such as addiction1 and depression2 suggests that common neural circuitry contributes to these disparate neuropsychiatric symptoms. The extended amygdala, including the bed nucleus of the stria terminalis (BNST), modulates fear and anxiety3, 4, but also projects to the ventral tegmental area (VTA)5, 6, a region implicated in reward and aversion7, 8, 9, 10, 11, 12, 13, thus providing a candidate neural substrate for integrating diverse emotional states. However, the precise functional connectivity between distinct BNST projection neurons and their postsynaptic targets in the VTA, as well as the role of this circuit in controlling motivational states, have not been described. Here we record and manipulate the activity of genetically and neurochemically identified VTA-projecting BNST neurons in freely behaving mice. Collectively, aversive stimuli exposure produced heterogeneous firing patterns in VTA-projecting BNST neurons. By contrast, in vivo optically identified glutamatergic projection neurons displayed a net enhancement of activity to aversive stimuli, whereas the firing rate of identified GABAergic (γ-aminobutyric acid-containing) projection neurons was suppressed. Channelrhodopsin-2-assisted circuit mapping revealed that both BNST glutamatergic and GABAergic projections preferentially innervate postsynaptic non-dopaminergic VTA neurons, thus providing a mechanistic framework for in vivo circuit perturbations. In vivo photostimulation of BNST glutamatergic projections resulted in aversive and anxiogenic behavioural phenotypes. Conversely, activation of BNST GABAergic projections produced rewarding and anxiolytic phenotypes, which were also recapitulated by direct inhibition of VTA GABAergic neurons. These data demonstrate that functionally opposing BNST to VTA circuits regulate rewarding and aversive motivational states, and may serve as a crucial circuit node for bidirectionally normalizing maladaptive behaviours.


Rats and Humans Can Optimally Accumulate Evidence for Decision-Making

Bingni W. Brunton, Matthew M. Botvinick, Carlos D. Brody
Science 5 April 2013: Vol. 340 no. 6128 pp. 95-98

The gradual and noisy accumulation of evidence is a fundamental component of decision-making, with noise playing a key role as the source of variability and errors. However, the origins of this noise have never been determined. We developed decision-making tasks in which sensory evidence is delivered in randomly timed pulses, and analyzed the resulting data with models that use the richly detailed information of each trial’s pulse timing to distinguish between different decision-making mechanisms. This analysis allowed measurement of the magnitude of noise in the accumulator’s memory, separately from noise associated with incoming sensory evidence. In our tasks, the accumulator’s memory was noiseless, for both rats and humans. In contrast, the addition of new sensory evidence was the primary source of variability. We suggest our task and modeling approach as a powerful method for revealing internal properties of decision-making processes.


Functional Disconnection of the Orbitofrontal Cortex and Basolateral Amygdala Impairs Acquisition of a Rat Gambling Task and Disrupts Animals' Ability to Alter Decision-Making Behavior after Reinforcer Devaluation

Fiona D. Zeeb and Catharine A. Winstanley
J. Neurosci. 2013;33 6434-6443

An inability to adjust choice preferences in response to changes in reward value may underlie key symptoms of many psychiatric disorders, including chemical and behavioral addictions. We developed the rat gambling task (rGT) to investigate the neurobiology underlying complex decision-making processes. As in the Iowa Gambling task, the optimal strategy is to avoid choosing larger, riskier rewards and to instead favor options associated with smaller rewards but less loss and, ultimately, greater long-term gain. Given the demonstrated importance of the orbitofrontal cortex (OFC) and basolateral amygdala (BLA) in acquisition of the rGT and Iowa Gambling task, we used a contralateral disconnection lesion procedure to assess whether functional connectivity between these regions is necessary for optimal decision-making. Disrupting the OFC-BLA pathway retarded acquisition of the rGT. Devaluing the reinforcer by inducing sensory-specific satiety altered decision-making in control groups. In contrast, disconnected rats did not update their choice preference following reward devaluation, either when the devalued reward was still delivered or when animals needed to rely on stored representations of reward value (i.e., during extinction). However, all rats exhibited decreased premature responding and slower response latencies after satiety manipulations. Hence, disconnecting the OFC and BLA did not affect general behavioral changes caused by reduced motivation, but instead prevented alterations in the value of a specific reward from contributing appropriately to cost-benefit decision-making. These results highlight the role of the OFC-BLA pathway in the decision-making process and suggest that communication between these areas is vital for the appropriate assessment of reward value to influence choice.


Hierarchical Organization of Parietofrontal Circuits during Goal-Directed Action

Lennart Verhagen, H. Chris Dijkerman, W. Pieter Medendorp, and Ivan Toni
J. Neurosci. 2013;33 6492-6503

Two parietofrontal networks share the control of goal-directed movements: a dorsomedial circuit that includes the superior parieto-occipital sulcus (sPOS) and a dorsolateral circuit comprising the anterior intraparietal sulcus (aIPS). These circuits are thought to independently control either reach and grip components (a functional dissociation), or planning and execution phases of grasping movements (a temporal dissociation). However, recent evidence of functional and temporal overlap between these circuits has undermined those models.

Here, we test an alternative model that subsumes previous accounts: the dorsolateral and dorsomedial circuits operate at different hierarchical levels, resulting in functional and temporal dependencies between their computations. We asked human participants to grasp a visually presented object, manipulating movement complexity by varying object slant. We used concurrent single-pulse transcranial magnetic stimulation and electroencephalography (TMS-EEG) to probe and record neurophysiological activity in the two circuits. Changes in alpha-band oscillations (8–12 Hz) characterized the effects of task manipulations and TMS interferences over aIPS and sPOS.

Increasing the complexity of the grasping movement was accompanied by alpha-suppression over dorsomedial parietofrontal regions, including sPOS, during both planning and execution stages. TMS interference over either aIPS or sPOS disrupted this index of dorsomedial computations; early when aIPS was perturbed, later when sPOS was perturbed, indicating that the dorsomedial circuit is temporally dependent on aIPS. TMS over sPOS enhanced alpha-suppression in inferior parietal cortex, indicating that the dorsolateral circuit can compensate for a transient sPOS perturbation. These findings suggest that both circuits specify the same grasping parameters, with dorsomedial computations depending on dorsolateral contributions.


Normalization is a general neural mechanism for context-dependent decision making

Kenway Louie, Mel W. Khaw, and Paul W. Glimcher
PNAS April 9, 2013 vol. 110 no. 15 6139-6144

Understanding the neural code is critical to linking brain and behavior. In sensory systems, divisive normalization seems to be a canonical neural computation, observed in areas ranging from retina to cortex and mediating processes including contrast adaptation, surround suppression, visual attention, and multisensory integration. Recent electrophysiological studies have extended these insights beyond the sensory domain, demonstrating an analogous algorithm for the value signals that guide decision making, but the effects of normalization on choice behavior are unknown. Here, we show that choice models using normalization generate significant (and classically irrational) choice phenomena driven by either the value or number of alternative options. In value-guided choice experiments, both monkey and human choosers show novel context-dependent behavior consistent with normalization. These findings suggest that the neural mechanism of value coding critically influences stochastic choice behavior and provide a generalizable quantitative framework for examining context effects in decision making.


Effort and Valuation in the Brain: The Effects of Anticipation and Execution

Irma T. Kurniawan, Marc Guitart-Masip, Peter Dayan, and Raymond J. Dolan
J. Neurosci. 2013;33 6160-6169

Neural representations of the effort deployed in performing actions, and the valence of the outcomes they yield, form the foundation of action choice. To discover whether brain areas represent effort and outcome valence together or if they represent one but not the other, we examined these variables in an explicitly orthogonal way. We did this by asking human subjects to exert one of two levels of effort to improve their chances of either winning or avoiding the loss of money. Subjects responded faster both when exerting greater effort and when exerting effort in anticipation of winning money. Using fMRI, we inspected BOLD responses during anticipation (before any action was executed) and when the outcome was delivered. In this way, we indexed BOLD signals associated with an anticipated need to exert effort and its affective consequences, as well as the effect of executed effort on the representation of outcomes. Anterior cingulate cortex and dorsal striatum (dorsal putamen) signaled the anticipation of effort independently of the prospect of winning or losing. Activity in ventral striatum (ventral putamen) was greater for better-than-expected outcomes compared with worse-than-expected outcomes, an effect attenuated in the context of having exerted greater effort. Our findings provide evidence that neural representations of anticipated actions are sensitive to the expected demands, but not to the expected value of their consequence, whereas representations of outcome value are discounted by exertion, commensurate with an integration of cost and benefit so as to approximate net value.










The Limits of Deliberation in a Perceptual Decision Task

Zariwala HA, Kepecs A, Uchida N, Hirokawa J, Mainen ZF.
Neuron. 2013 Mar 27. pii: S0896-6273(13)00168-2

While it is commonly assumed that decisions taken slowly result in superior outcomes, is it possible that optimal decision making does not always require sacrificing speed? For odor categorization decisions, it was previously shown that rats use <300 ms regardless of difficulty, but these findings could be interpreted as a tradeoff of accuracy for speed. Here, by systematically manipulating the task contingencies, we demonstrate that this is the maximum time over which sampling time can improve accuracy. Furthermore, we show that decision accuracy increases at no temporal cost when rats can better anticipate either the identity of stimuli or the required timing of responses. These experiments suggest that uncertainty in odor category decisions arises from noise sources that fluctuate slowly from trial-to-trial rather than rapidly within trials and that category decisions in other species and modalities might likewise be optimally served by rapid choices.

A Role for the Medial Temporal Lobe in Feedback-Driven Learning: Evidence from Amnesia

Karin Foerde, Elizabeth Race, Mieke Verfaellie, and Daphna Shohamy
The Journal of Neuroscience, 27 March 2013, 33(13): 5698-5704

The ability to learn from feedback is a key component of adaptive behavior. This type of learning is traditionally thought to depend on neural substrates in the striatum and not on the medial temporal lobe (MTL). Here we show that in humans the MTL becomes necessary for feedback-based learning when feedback is delayed. Specifically, amnesic patients with MTL damage were impaired at probabilistic learning of cue–outcome associations when response-contingent feedback was delayed by a few seconds, but not when feedback was immediate. By contrast, patients with striatal dysfunction due to Parkinson's disease demonstrated the opposite pattern: impaired learning when trial-by-trial feedback was immediate but not when feedback was delayed, indicating that the striatum is necessary for learning only when feedback is immediate. Together, these results reveal that multiple complementary learning processes support what appears to be identical behavior in healthy individuals and point to an important role for the MTL in feedback-driven learning.

Uncertainty Increases Pain: Evidence for a Novel Mechanism of Pain Modulation Involving the Periaqueductal Gray

Wako Yoshida, Ben Seymour, Martin Koltzenburg, and Raymond J. Dolan
The Journal of Neuroscience, 27 March 2013, 33(13): 5638-5646

Predictions about sensory input exert a dominant effect on what we perceive, and this is particularly true for the experience of pain. However, it remains unclear what component of prediction, from an information-theoretic perspective, controls this effect. We used a vicarious pain observation paradigm to study how the underlying statistics of predictive information modulate experience. Subjects observed judgments that a group of people made to a painful thermal stimulus, before receiving the same stimulus themselves. We show that the mean observed rating exerted a strong assimilative effect on subjective pain. In addition, we show that observed uncertainty had a specific and potent hyperalgesic effect. Using computational functional magnetic resonance imaging, we found that this effect correlated with activity in the periaqueductal gray. Our results provide evidence for a novel form of cognitive hyperalgesia relating to perceptual uncertainty, induced here by vicarious observation, with control mediated by the brainstem pain modulatory system.

Decoding moral judgments from neural representations of intentions

Jorie Koster-Hale, Rebecca Saxe, James Dungan, and Liane L. Young
PNAS April 2, 2013 vol. 110 no. 14 5648-5653

Intentional harms are typically judged to be morally worse than accidental harms. Distinguishing between intentional harms and accidents depends on the capacity for mental state reasoning (i.e., reasoning about beliefs and intentions), which is supported by a group of brain regions including the right temporo-parietal junction (RTPJ). Prior research has found that interfering with activity in RTPJ can impair mental state reasoning for moral judgment and that high-functioning individuals with autism spectrum disorders make moral judgments based less on intent information than neurotypical participants. Three experiments, using multivoxel pattern analysis, find that (i) in neurotypical adults, the RTPJ shows reliable and distinct spatial patterns of responses across voxels for intentional vs. accidental harms, and (ii) individual differences in this neural pattern predict differences in participants’ moral judgments. These effects are specific to RTPJ. By contrast, (iii) this distinction was absent in adults with autism spectrum disorders. We conclude that multivoxel pattern analysis can detect features of mental state representations (e.g., intent), and that the corresponding neural patterns are behaviorally and clinically relevant.