2014年6月17日火曜日

Role of the Primate Ventral Tegmental Area in Reinforcement and Motivation

John T. Arsenault, Samy Rima, Heiko Stemmann, Wim Vanduffel

サルで微小電気刺激+fMRI。中脳腹側被蓋野(VTA)を刺激すると人為的に強化学習を促進できる。また、その際、線条体や眼窩前頭野など報酬系のfMRI信号が増加する。「VTAの活動」は「強化学習」や「報酬関連脳部位の活動」の原因になっている。

Monkey electrophysiology [1, 2] suggests that the activity of the ventral tegmental area (VTA) helps regulate reinforcement learning and motivated behavior, in part by broadcasting prediction error signals throughout the reward system. However, electrophysiological studies do not allow causal inferences regarding the activity of VTA neurons with respect to these processes because they require artificial manipulation of neuronal firing. Rodent studies fulfilled this requirement by demonstrating that electrical and optogenetic VTA stimulation can induce learning and modulate downstream structures [3–7]. Still, the primate dopamine system has diverged significantly from that of rodents, exhibiting greatly expanded and uniquely distributed cortical and subcortical innervation patterns [8]. Here, we bridge the gap between rodent perturbation studies and monkey electrophysiology using chronic electrical microstimulation of macaque VTA (VTA-EM). VTA-EM was found to reinforce cue selection in an operant task and to motivate future cue selection using a Pavlovian paradigm. Moreover, by combining VTA-EM with concurrent fMRI, we demonstrated that VTA-EM increased fMRI activity throughout most of the dopaminergic reward system. These results establish a causative role for primate VTA in regulating stimulus-specific reinforcement and motivation as well as in modulating activity throughout the reward system.

2014年6月16日月曜日

Decision Making: The Neuroethological Turn

John M. Pearson, Karli K. Watson, Michael L. Platt
Neuron Volume 82, Issue 5, p950–965, 4 June 2014

報酬に基づく意思決定についての総説論文。

Neuroeconomics applies models from economics and psychology to inform neurobiological studies of choice. This approach has revealed neural signatures of concepts like value, risk, and ambiguity, which are known to influence decision making. Such observations have led theorists to hypothesize a single, unified decision process that mediates choice behavior via a common neural currency for outcomes like food, money, or social praise. In parallel, recent neuroethological studies of decision making have focused on natural behaviors like foraging, mate choice, and social interactions. These decisions strongly impact evolutionary fitness and thus are likely to have played a key role in shaping the neural circuits that mediate decision making. This approach has revealed a suite of computational motifs that appear to be shared across a wide variety of organisms. We argue that the existence of deep homologies in the neural circuits mediating choice may have profound implications for understanding human decision making in health and disease.

2014年6月5日木曜日

Value Signals in the Prefrontal Cortex Predict Individual Preferences across Reward Categories

Jörg Gross, Eva Woelbert, Jan Zimmermann, Sanae Okamoto-Barth, Arno
Riedl, and Rainer Goebel
The Journal of Neuroscience, 28 May 2014, 34(22):7580-7586;

「スナック菓子を食べる」vs.「スポーツをする」のような「種類の異なる選択肢間の比較」を行うためには、共通の尺度(効用)が脳内に必要になってくる。
もし共通の尺度があるなら、「種々のスナック菓子の価値に対する脳活動から種々のスポーツをする価値を予測する」ことができるはずだ。
→ 実際、fMRI MVPAを用いて、前頭前野の脳活動でその予測ができた。

Humans can choose between fundamentally different options, such as watching a movie or going out for dinner. According to the utility concept, put forward by utilitarian philosophers and widely used in economics, this may be accomplished by mapping the value of different options onto a common scale, independent of specific option characteristics (Fehr and Rangel, 2011; Levy and Glimcher, 2012). If this is the case, value-related activity patterns in the brain should allow predictions of individual preferences across fundamentally different reward categories. We analyze fMRI data of the prefrontal cortex while subjects imagine the pleasure they would derive from items belonging to two distinct reward categories: engaging activities (like going out for drinks, daydreaming, or doing sports) and snack foods. Support vector machines trained on brain patterns related to one category reliably predict individual preferences of the other category and vice versa. Further, we predict preferences across participants. These findings demonstrate that prefrontal cortex value signals follow a common scale representation of value that is even comparable across individuals and could, in principle, be used to predict choice.

2014年6月4日水曜日

Novelty Enhances Visual Salience Independently of Reward in the Parietal Lobe

Nicholas C. Foley, David C. Jangraw, Christopher Peck, and Jacqueline Gottlieb
The Journal of Neuroscience, 4 June 2014, 34(23):7947-7957;

刺激の「価値(報酬予測)」と「新規性」はどちらもその刺激に対する注意を増加させることが知られている。
価値と新規性は別々に注意に影響を与えているのか?それとも、お互いに関連するのか?
サルの視線とLIPニューロンの活動を調べたところ、前者っぽい。

Novelty modulates sensory and reward processes, but it remains unknown how these effects interact, i.e., how the visual effects of novelty are related to its motivational effects. A widespread hypothesis, based on findings that novelty activates reward-related structures, is that all the effects of novelty are explained in terms of reward. According to this idea, a novel stimulus is by default assigned high reward value and hence high salience, but this salience rapidly decreases if the stimulus signals a negative outcome. Here we show that, contrary to this idea, novelty affects visual salience in the monkey lateral intraparietal area (LIP) in ways that are independent of expected reward. Monkeys viewed peripheral visual cues that were novel or familiar (received few or many exposures) and predicted whether the trial will have a positive or a negative outcome—i.e., end in a reward or a lack of reward. We used a saccade-based assay to detect whether the cues automatically attracted or repelled attention from their visual field location. We show that salience—measured in saccades and LIP responses—was enhanced by both novelty and positive reward associations, but these factors were dissociable and habituated on different timescales. The monkeys rapidly recognized that a novel stimulus signaled a negative outcome (and withheld anticipatory licking within the first few presentations), but the salience of that stimulus remained high for multiple subsequent presentations. Therefore, novelty can provide an intrinsic bonus for attention that extends beyond the first presentation and is independent of physical rewards.

Cognitive Control Functions of Anterior Cingulate Cortex in Macaque Monkeys Performing a Wisconsin Card Sorting Test Analog

Masaru Kuwabara, Farshad A. Mansouri, Mark J. Buckley, and Keiji Tanaka
The Journal of Neuroscience, 28 May 2014, 34(22):7531-7547;

サルの前帯状溝(ACCs)は「実験課題のルールが変わったことによる間違い」と「自分自身のミスによる間違い」の両方をコードしている。

Monkeys were trained to select one of three targets by matching in color or matching in shape to a sample. Because the matching rule frequently changed and there were no cues for the currently relevant rule, monkeys had to maintain the relevant rule in working memory to select the correct target. We found that monkeys' error commission was not limited to the period after the rule change and occasionally occurred even after several consecutive correct trials, indicating that the task was cognitively demanding. In trials immediately after such error trials, monkeys' speed of selecting targets was slower. Additionally, in trials following consecutive correct trials, the monkeys' target selections for erroneous responses were slower than those for correct responses. We further found evidence for the involvement of the cortex in the anterior cingulate sulcus (ACCs) in these error-related behavioral modulations. First, ACCs cell activity differed between after-error and after-correct trials. In another group of ACCs cells, the activity differed depending on whether the monkeys were making a correct or erroneous decision in target selection. Second, bilateral ACCs lesions significantly abolished the response slowing both in after-error trials and in error trials. The error likelihood in after-error trials could be inferred by the error feedback in the previous trial, whereas the likelihood of erroneous responses after consecutive correct trials could be monitored only internally. These results suggest that ACCs represent both context-dependent and internally detected error likelihoods and promote modes of response selections in situations that involve these two types of error likelihood.