2013年10月31日木曜日

Human development of the ability to learn from bad news

Christina Moutsiana, Neil Garrett, Richard C. Clarke, R. Beau Lotto, Sarah-Jayne Blakemore, and Tali Sharot
PNAS October 8, 2013 vol. 110 no. 41 16396-16401

「良いニュースからの学習」と「悪いニュースからの学習」の方法が発達によってどのように変わるか?
前者は9歳から26歳にかけて変化しないが、後者は年齢に伴って変化する。

Humans show a natural tendency to discount bad news while incorporating good news into beliefs (the “good news–bad news effect”), an effect that may help explain seemingly irrational risk taking. Understanding how this bias develops with age is important because adolescents are prone to engage in risky behavior; thus, educating them about danger is crucial. We reveal a striking valence-dependent asymmetry in how belief updating develops with age. In the ages tested (9–26 y), younger age was associated with inaccurate updating of beliefs in response to undesirable information regarding vulnerability. In contrast, the ability to update beliefs accurately in response to desirable information remained relatively stable with age. This asymmetry was mediated by adequate computational use of positive but not negative estimation errors to alter beliefs. The results are important for understanding how belief formation develops and might help explain why adolescents do not respond adequately to warnings.

2013年10月30日水曜日

Disruption of Dorsolateral Prefrontal Cortex Decreases Model-Based in Favor of Model-free Control in Humans

Peter Smittenaar, Thomas H.B. FitzGerald, Vincenzo Romei, Nicholas D. Wright, and Raymond J. Dolan
Neuron, 24 October 2013

「単純で計算負荷は軽いが柔軟性に欠けるモデル・フリー強化学習」と「柔軟に行動を変えられるが計算負荷の高いモデル・ベースド強化学習」、ヒトは両方を組み合わせて使っていることが知られている。
右側の背外側前頭前皮質の活動を磁気刺激で抑えると、ヒトはモデル・フリー強化学習をより使うようになる。

Human choice behavior often reflects a competition between inflexible computationally efficient control on the one hand and a slower more flexible system of control on the other. This distinction is well captured by model-free and model-based reinforcement learning algorithms. Here, studying human subjects, we show it is possible to shift the balance of control between these systems by disruption of right dorsolateral prefrontal cortex, such that participants manifest a dominance of the less optimal model-free control. In contrast, disruption of left dorsolateral prefrontal cortex impaired model-based performance only in those participants with low working memory capacity.

2013年10月29日火曜日

Neural Underpinnings of the Identifiable Victim Effect: Affect Shifts Preferences for Giving

Alexander Genevsky, Daniel Vastfjall, Paul Slovic, and Brian Knutson
J. Neurosci. 2013;33 17188-17196 Open Access

ヒトfMRI。
「匿名の他人」よりも「顔/名前などが分かっている他人」を助けたくなるという「identifiable victim effect」の神経基盤を研究。
「他人の顔写真を見ることができるケース」と「影絵のみのケース」を比較。
行動データ:顔写真が見られるケースの方が、他者に対する寄付額が大きくなる。
fMRIデータ:顔写真が見られるケースでは情動関連や顔認知関連の様々な脳部位が活動するが、その中で「腹側線条体の側坐核(nucleus accumbens)」だけが寄付額を予測出来る。

The “identifiable victim effect” refers to peoples' tendency to preferentially give to identified versus anonymous victims of misfortune, and has been proposed to partly depend on affect. By soliciting charitable donations from human subjects during behavioral and neural (i.e., functional magnetic resonance imaging) experiments, we sought to determine whether and how affect might promote the identifiable victim effect. Behaviorally, subjects gave more to orphans depicted by photographs versus silhouettes, and their shift in preferences was mediated by photograph-induced feelings of positive arousal, but not negative arousal. Neurally, while photographs versus silhouettes elicited activity in widespread circuits associated with facial and affective processing, only nucleus accumbens activity predicted and could statistically account for increased donations. Together, these findings suggest that presenting evaluable identifiable information can recruit positive arousal, which then promotes giving. We propose that affect elicited by identifiable stimuli can compel people to give more to strangers, even despite costs to the self.

2013年10月28日月曜日

Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning

Sandra Iglesias, Christoph Mathys, Kay H. Brodersen, Lars Kasper, Marco Piccirelli, Hanneke E.M. den Ouden, Klaas E. Stephan
Neuron, Volume 80, Issue 2, 519-530, 16 October 2013

ヒトfMRI。
「手掛かり刺激と結果の連合強度」や「その強度の不安定さ(ボラティリティ)」など、「環境についての様々なレベルの情報」についての学習がヒトの脳内でどのように行われているのか?
低次な予測誤差(刺激と結果の連合の学習に利用される)は中脳(midbrain、ドーパミン系)で処理され、高次な予測誤差(連合の不安定さの学習に使われる)は前脳基底部(basal forebrain、アセチルコリン系)で処理されている。

In Bayesian brain theories, hierarchically related prediction errors (PEs) play a central role for predicting sensory inputs and inferring their underlying causes, e.g., the probabilistic structure of the environment and its volatility. Notably, PEs at different hierarchical levels may be encoded by different neuromodulatory transmitters. Here, we tested this possibility in computational fMRI studies of audio-visual learning. Using a hierarchical Bayesian model, we found that low-level PEs about visual stimulus outcome were reflected by widespread activity in visual and supramodal areas but also in the midbrain. In contrast, high-level PEs about stimulus probabilities were encoded by the basal forebrain. These findings were replicated in two groups of healthy volunteers. While our fMRI measures do not reveal the exact neuron types activated in midbrain and basal forebrain, they suggest a dichotomy between neuromodulatory systems, linking dopamine to low-level PEs about stimulus outcome and acetylcholine to more abstract PEs about stimulus probabilities.

2013年10月27日日曜日

Goals and Habits in the Brain

Ray J. Dolan, Peter Dayan
Neuron, Volume 80, Issue 2, 312-325, 16 October 2013

「Goal-directed behavior」と「Habitual behavior」の違いについての総説論文。
過去の膨大な研究を世代ごとに分けて整理し、その歴史を概観している。
なお、二つの行動様式(goal-directed vs. habitual)はそれぞれ、「model-based vs. model-free」、「prospective vs. retrospective」と呼ばれることもある。

An enduring and richly elaborated dichotomy in cognitive neuroscience is that of reflective versus reflexive decision making and choice. Other literatures refer to the two ends of what is likely to be a spectrum with terms such as goal-directed versus habitual, model-based versus model-free or prospective versus retrospective. One of the most rigorous traditions of experimental work in the field started with studies in rodents and graduated via human versions and enrichments of those experiments to a current state in which new paradigms are probing and challenging the very heart of the distinction. We review four generations of work in this tradition and provide pointers to the forefront of the field’s fifth generation.

2013年10月24日木曜日

Extended practice of a motor skill is associated with reduced metabolic activity in M1.

Picard, N., Matsuzaka, Y., & Strick, P. L. (2013).
Nature neuroscience, 16(9), 1340–7.

運動(スキル)学習を行っても、一次運動野(M1)の神経活動(ニューロンの発火頻度)は変化しない。
しかし、それを支える代謝量は減少する。
つまり、スキル練習/学習によって、一次運動野の神経活動の効率が良くなる(少ない代謝で同じ活動を維持できるようになる)。

How does long-term training and the development of motor skills modify the activity of the primary motor cortex (M1)? To address this issue, we trained monkeys for ~1–6 years to perform visually guided and internally generated sequences of reaching movements. Then, we used [14C]2-deoxyglucose (2DG) uptake and single-neuron recording to measure metabolic and neuron activity in M1. After extended practice, we observed a profound reduction of metabolic activity in M1 for the performance of internally generated compared to visually guided tasks. In contrast, measures of neuron firing displayed little difference during the two tasks. These findings suggest that the development of skill through extended practice results in a reduction in the synaptic activity required to produce internally generated, but not visually guided, sequences of movements. Thus, practice leading to skilled performance results in more efficient generation of neuronal activity in M1.

2013年10月23日水曜日

Creating a False Memory in the Hippocampus

Steve Ramirez, Xu Liu, Pei-Ann Lin, Junghyup Suh, Michele Pignatelli, Roger L. Redondo, Tomás J. Ryan, and Susumu Tonegawa
Science 26 July 2013: 387-391.

光遺伝学(オプトジェネティクス)を用いて海馬の神経細胞を活動させることで偽の記憶を形成することができる。
手順は以下の通り:
(1)マウスを部屋Aに置き、そこで活動する海馬の神経細胞群を特定する。
(2)部屋Bでマウスに電気ショックを与える。同時に光刺激を行い、上記(1)で特定した「部屋Aで反応した神経細胞」を発火させる。
(3)再び部屋Aにマウスを置き、「電気ショックを怖がる(動かず、固まってしまう)」ことを確認(部屋Cでは怖がらない)。
つまり、「電気ショックと共に、部屋Aに反応する細胞を光刺激で発火させる」ことにより、「部屋で電気ショックを受けた」という偽の記憶を作り出すことができた(実際には部屋Aでは電気ショックを受けていないのに)。

Memories can be unreliable. We created a false memory in mice by optogenetically manipulating memory engram–bearing cells in the hippocampus. Dentate gyrus (DG) or CA1 neurons activated by exposure to a particular context were labeled with channelrhodopsin-2. These neurons were later optically reactivated during fear conditioning in a different context. The DG experimental group showed increased freezing in the original context, in which a foot shock was never delivered. The recall of this false memory was context-specific, activated similar downstream regions engaged during natural fear memory recall, and was also capable of driving an active fear response. Our data demonstrate that it is possible to generate an internally represented and behaviorally expressed fear memory via artificial means.

2013年10月22日火曜日

Cortical Signals for Rewarded Actions and Strategic Exploration

Christopher H. Donahue, Hyojung Seo, and Daeyeol Lee
Neuron, 05 September 2013

環境が安定している時は「過去に学習した行動」を取り続ければ良い。
一方、環境が不安定な時は、「探索行動」を頻繁に行う必要がある。
探索行動が必要な状況では(必要ない状況と比べ)、Supplementary eye field(補足眼球野)の活動が「過去に成功した行動/選択」の情報を反映する。
また、その活動は「探索行動が必要な状況で、探索行動を行う頻度」と相関する。

In stable environments, decision makers can exploit their previously learned strategies for optimal outcomes, while exploration might lead to better options in unstable environments. Here, to investigate the cortical contributions to exploratory behavior, we analyzed single-neuron activity recorded from four different cortical areas of monkeys performing a matching-pennies task and a visual search task, which encouraged and discouraged exploration, respectively. We found that neurons in multiple regions in the frontal and parietal cortex tended to encode signals related to previously rewarded actions more reliably than unrewarded actions. In addition, signals for rewarded choices in the supplementary eye field were attenuated during the visual search task and were correlated with the tendency to switch choices during the matching-pennies task. These results suggest that the supplementary eye field might play a unique role in encouraging animals to explore alternative decision-making strategies.

2013年10月21日月曜日

The Functional and Structural Neural Basis of Individual Differences in Loss Aversion

Nicola Canessa, Chiara Crespi, Matteo Motterlini, Gabriel Baud-Bovy, Gabriele Chierchia, Giuseppe Pantaleo, Marco Tettamanti, and Stefano F. Cappa
J. Neurosci. 2013;33 14307-14317
http://www.jneurosci.org/cgi/content/abstract/33/36/14307?etoc

ヒトfMRI。
プロスペクト理論における損失回避傾向の神経基盤。
「扁桃体が損失に特異的に反応し、損失回避的になる」という説と「損失/報酬の区別をせずに中脳辺縁系が価値に反応し、損失回避的になる」という説があった。
本研究で分かったことは:
・扁桃体は損失に反応する。
・一方、中脳辺縁系は損失回避傾向の個人差を説明出来る。
つまり、上記の説は両方正しそう。

Decision making under risk entails the anticipation of prospective outcomes, typically leading to the greater sensitivity to losses than gains known as loss aversion. Previous studies on the neural bases of choice-outcome anticipation and loss aversion provided inconsistent results, showing either bidirectional mesolimbic responses of activation for gains and deactivation for losses, or a specific amygdala involvement in processing losses. Here we focused on loss aversion with the aim to address interindividual differences in the neural bases of choice-outcome anticipation. Fifty-six healthy human participants accepted or rejected 104 mixed gambles offering equal (50%) chances of gaining or losing different amounts of money while their brain activity was measured with functional magnetic resonance imaging (fMRI). We report both bidirectional and gain/loss-specific responses while evaluating risky gambles, with amygdala and posterior insula specifically tracking the magnitude of potential losses. At the individual level, loss aversion was reflected both in limbic fMRI responses and in gray matter volume in a structural amygdala–thalamus–striatum network, in which the volume of the “output” centromedial amygdala nuclei mediating avoidance behavior was negatively correlated with monetary performance. We conclude that outcome anticipation and ensuing loss aversion involve multiple neural systems, showing functional and structural individual variability directly related to the actual financial outcomes of choices. By supporting the simultaneous involvement of both appetitive and aversive processing in economic decision making, these results contribute to the interpretation of existing inconsistencies on the neural bases of anticipating choice outcomes.

2013年10月20日日曜日

Prior Expectations Bias Sensory Representations in Visual Cortex

Peter Kok, Gijs Joost Brouwer, Marcel A.J. van Gerven, and Floris P. de Lange
J. Neurosci. 2013;33 16275-16284

ヒトfMRI。
ヒトや動物が物事を「知覚」する時、「実際の感覚情報(例:視覚刺激)」と「事前の知識」を統合していることが知られている。
その統合は、脳内の知覚プロセスの初期(例:視覚野)で行われているのか?
それとも、もっと高次な処理を司る脳部位で行われているのか?
→視覚野の活動が「事前知識」にモジュレートされていた。つまり、感覚情報と事前知識の統合は知覚プロセスの初期段階で行われている。

Perception is strongly influenced by expectations. Accordingly, perception has sometimes been cast as a process of inference, whereby sensory inputs are combined with prior knowledge. However, despite a wealth of behavioral literature supporting an account of perception as probabilistic inference, the neural mechanisms underlying this process remain largely unknown. One important question is whether top-down expectation biases stimulus representations in early sensory cortex, i.e., whether the integration of prior knowledge and bottom-up inputs is already observable at the earliest levels of sensory processing. Alternatively, early sensory processing may be unaffected by top-down expectations, and integration of prior knowledge and bottom-up input may take place in downstream association areas that are proposed to be involved in perceptual decision-making. Here, we implicitly manipulated human subjects' prior expectations about visual motion stimuli, and probed the effects on both perception and sensory representations in visual cortex. To this end, we measured neural activity noninvasively using functional magnetic resonance imaging, and applied a forward modeling approach to reconstruct the motion direction of the perceived stimuli from the signal in visual cortex. Our results show that top-down expectations bias representations in visual cortex, demonstrating that the integration of prior information and sensory input is reflected at the earliest stages of sensory processing.

2013年10月17日木曜日

Neural Evidence for Individual and Cultural Variability in the Social Comparison Effect

Pyungwon Kang, Yongsil Lee, Incheol Choi, and Hackjin Kim
J. Neurosci. 2013;33 16200-16208

fMRIを用いた文化差の研究。
アメリカ人と韓国人の比較。アメリカは「Independentな文化」の例、韓国は「Interdependentな文化」の例らしい。
Interdependentな文化で育った韓国人の方が(Independentな文化で育ったアメリカ人に比べ)、報酬系の脳部位(腹側線条体、腹内側前頭前皮質)が「自分と他者の報酬の違い」に敏感に反応した。

Although several studies have investigated the neural mechanism of social comparison, it remains unclear whether and how cultural membership, particularly independent versus interdependent cultures, may differentially shape the neural processes underlying social comparison. In the present functional magnetic resonance imaging (fMRI) study, we examined the behaviors and neural response patterns of Korean (i.e., interdependent culture) and American (i.e., independent culture) participants while performing a financial gambling task simultaneously and independently with a partner. Upon seeing the partner's income, greater modulation of the activity in the ventral striatum (VS) and the ventromedial prefrontal cortex (vmPFC) by relative gain was observed in Korean than American participants, suggesting greater sensitivity of Koreans toward social comparison. The strength of functional connectivity between the VS and the vmPFC predicted individual variability in the degree to which participants' decisions were affected by relative incomes. Additional model-based fMRI analysis further confirmed the primary role of the vmPFC in biasing decisions based on relative incomes. In summary, the present study provides the first neural evidence for decision biases due to social comparison and their individual and cultural variations.

2013年10月16日水曜日

Dissociable Neural Mechanisms for Goal-Directed Versus Incidental Memory Reactivation

Brice A. Kuhl, Marcia K. Johnson, and Marvin M. Chun
J. Neurosci. 2013;33 16099-16109

ヒトfMRI。
「実験課題の遂行に必要(Goal-Directed)な情報を記憶から呼び出す」のと「課題遂行に必要ない偶発的(Incidental)な情報を呼び出す」のはどう違うのか?
腹内側頭葉(ventral/medial temporal lobe: VMTL)は両者に関連する。つまり、記憶の呼び出し一般を司る。
一方、前頭頭頂皮質は前者のみに関連。つまり、何らかの目的に応じて記憶を呼び出すことに関与している。

Remembering a past event involves reactivation of distributed patterns of neural activity that represent the features of that event—a process that depends on associative mechanisms supported by medial temporal lobe structures. Although efficient use of memory requires prioritizing those features of a memory that are relevant to current behavioral goals (target features) over features that may be goal-irrelevant (incidental features), there remains ambiguity concerning how this is achieved. We tested the hypothesis that although medial temporal lobe structures may support reactivation of both target and incidental event features, frontoparietal cortex preferentially reactivates those features that match current goals. Here, human participants were cued to remember either the category (face/scene) to which a picture belonged (category trials) or the location (left/right) in which a picture appeared (location trials). Multivoxel pattern analysis of fMRI data were used to measure reactivation of category information as a function of its behavioral relevance (target vs incidental reactivation). In ventral/medial temporal lobe (VMTL) structures, incidental reactivation was as robust as target reactivation. In contrast, frontoparietal cortex exhibited stronger target than incidental reactivation; that is, goal-modulated reactivation. Reactivation was also associated with later memory. Frontoparietal biases toward target reactivation predicted subsequent memory for target features, whereas incidental reactivation in VMTL predicted subsequent memory for nontested features. These findings reveal a striking dissociation between goal-modulated reactivation in frontoparietal cortex and incidental reactivation in VMTL.

2013年10月15日火曜日

Mice infer probabilistic models for timing

Yi Li and Joshua Tate Dudman
PNAS October 15, 2013 vol. 110 no. 42 17154-17159

マウスの行動実験。
マウスは「タイミング(レバーを押してから報酬が来るまでの時間)」について、その平均だけではなく分散/標準偏差も学習できる。
「最適な選択をするためには、報酬が来るタイミングの平均だけではなく分散も同時に学習しないといけない」実験課題を組んだのがポイント。

Animals learn both whether and when a reward will occur. Neural models of timing posit that animals learn the mean time until reward perturbed by a fixed relative uncertainty. Nonetheless, animals can learn to perform actions for reward even in highly variable natural environments. Optimal inference in the presence of variable information requires probabilistic models, yet it is unclear whether animals can infer such models for reward timing. Here, we develop a behavioral paradigm in which optimal performance required knowledge of the distribution from which reward delays were chosen. We found that mice were able to accurately adjust their behavior to the SD of the reward delay distribution. Importantly, mice were able to flexibly adjust the amount of prior information used for inference according to the moment-by-moment demands of the task. The ability to infer probabilistic models for timing may allow mice to adapt to complex and dynamic natural environments.

2013年10月13日日曜日

Imaging Social Motivation: Distinct Brain Mechanisms Drive Effort Production during Collaboration versus Competition

Raphael Le Bouc and Mathias Pessiglione
J. Neurosci. 2013;33 15894-15902

ヒトfMRI。
【実験課題】
基本的には、「ハンドグリップを握る強さに応じて報酬が得られる」という課題。
三条件あり、それぞれ「自分の握りの強さ(個人条件)」、「自分と他者の握りの強さの平均(恊働条件)」、「自分と他者の握りの強さの差(競争条件)」に応じて報酬が貰える。
【結果】
行動:被験者は恊働条件で、個人条件に比べて、強い力でグリップを握った。一方、競争条件では個人条件との差は見られなかった。
fMRI:「恊働条件で(個人条件に比べて)より強い力を出した被験者」は「TPJ(側頭頭頂接合部)が恊働条件で強く賦活して」おり、さらに「TPJの灰白質の体積が大きい」ことが分かった。
【結論】
「自分の報酬/コストの最大化を目指すなら、恊働条件では力を緩めるべきである」にも関わらず、ヒトは他者と恊働するときに、単独での場合と比べて、大きな力を出す。
そして、その傾向はTPJの活動によってもたらされる。

Collaborative and competitive interactions have been investigated extensively so as to understand how the brain makes choices in the context of strategic games, yet such interactions are known to influence a more basic dimension of behavior: the energy invested in the task. The cognitive mechanisms that motivate effort production in social situations remain poorly understood, and their neural counterparts have not been explored so far. A dominant idea is that the motivation provided by the social context is reducible to the personal utility of effort production, which decreases in collaboration and increases in competition. Using functional magnetic resonance imaging, we scanned human participants while they produced a physical effort in a collaborative or competitive context. We found that motivation was indeed primarily driven by personal utility, which was reflected in brain regions devoted to reward processing (the ventral basal ganglia). However, subjects who departed from utility maximization, working more in collaborative situations, showed greater functional activation and anatomical volume in a brain region implicated previously in social cognition (the temporoparietal junction). Therefore, this region might mediate a purely pro-social motivation to produce greater effort in the context of collaboration. More generally, our findings suggest that the individual propensity to invest energy in collaborative work might have an identifiable counterpart in the brain functional architecture.

2013年10月10日木曜日

A Shared Representation of the Space Near Oneself and Others in the Human Premotor Cortex

Claudio Brozzoli, Giovanni Gentile, Loretxu Bergouignan, H. Henrik Ehrsson
Current Biology, Volume 23, Issue 18, 1764-1768, 05 September 2013

ヒトfMRI。
腹側運動前野(ventral premotor)は「自分の手の近くにある物体」と「他者の手の近くにある物体」の両方に反応する。

Interactions between people require shared high-level cognitive representations of action goals, intentions [1], and mental states [2], but do people also share their representation of space? The human ventral premotor (PMv) and parietal cortices contain neuronal populations coding for the execution and observation of actions [1,3,4,5], analogous to the mirror neurons identified in monkeys [1,5]. This neuronal system is tuned to the location of the acting person relative to the observer and the target of the action [4,5]. Therefore, it can be theorized that the observer’s brain constructs a low-level, body-centered representation of the space around others similar to one’s own peripersonal space representation [6,7,8,9,10,11]. Single-cell recordings have reported that parietal visuotactile neurons discharge for objects near specific parts of a monkey’s own body and near the corresponding body parts of another individual [9]. In humans, no neuroimaging study has investigated this issue. Here, we identified neuronal populations in the human PMv that encode the space near both one’s own hand and another person’s hand. The shared peripersonal space representation could support social interactions by coding sensory events, actions, and cognitive processes in a common spatial reference frame.

2013年10月9日水曜日

Simultaneous modeling of visual saliency and value computation improves predictions of economic choice

R. Blythe Towal, Milica Mormann, and Christof Koch
PNAS October 1, 2013 vol. 110 no. 40 E3858-E3867

ヒト行動実験。
人の意思決定は「選択肢の価値」のだけではなく「視覚的顕著性(どの程度目立つか)」に影響される(なお、影響は前者の方が強いが後者の影響も有意である)。
両者を考慮した計算論モデル(drift–diffusion model)が被験者の行動を最も良く予測できる。

Many decisions we make require visually identifying and evaluating numerous alternatives quickly. These usually vary in reward, or value, and in low-level visual properties, such as saliency. Both saliency and value influence the final decision. In particular, saliency affects fixation locations and durations, which are predictive of choices. However, it is unknown how saliency propagates to the final decision. Moreover, the relative influence of saliency and value is unclear. Here we address these questions with an integrated model that combines a perceptual decision process about where and when to look with an economic decision process about what to choose. The perceptual decision process is modeled as a drift–diffusion model (DDM) process for each alternative. Using psychophysical data from a multiple-alternative, forced-choice task, in which subjects have to pick one food item from a crowded display via eye movements, we test four models where each DDM process is driven by (i) saliency or (ii) value alone or (iii) an additive or (iv) a multiplicative combination of both. We find that models including both saliency and value weighted in a one-third to two-thirds ratio (saliency-to-value) significantly outperform models based on either quantity alone. These eye fixation patterns modulate an economic decision process, also described as a DDM process driven by value. Our combined model quantitatively explains fixation patterns and choices with similar or better accuracy than previous models, suggesting that visual saliency has a smaller, but significant, influence than value and that saliency affects choices indirectly through perceptual decisions that modulate economic decisions.

2013年10月8日火曜日

Risk Prediction Error Coding in Orbitofrontal Neurons.

Martin O'Neill and Wolfram Schultz
J. Neurosci. 2013;33 15810-15814

サル電気生理。
眼窩前島皮質のニューロンは「リスク(報酬の分散)に関する予測誤差」をコードしている。
リスク予測誤差の研究は「被験者が本当にリスクを学習している」ことを示す行動データが貧弱なのが痛い…

Risk is a ubiquitous feature of life. It plays an important role in economic decisions by affecting subjective reward value. Informed decisions require accurate risk information for each choice option. However, risk is often not constant but changes dynamically in the environment. Therefore, risk information should be updated to the current risk level. Potential mechanisms involve error-driven updating, whereby differences between current and predicted risk levels (risk prediction errors) are used to obtain currently accurate risk predictions. As a major reward structure, the orbitofrontal cortex is involved in coding key reward parameters such as reward value and risk. In this study, monkeys viewed different visual stimuli indicating specific levels of risk that deviated from the overall risk predicted by a common earlier stimulus. A group of orbitofrontal neurons displayed a risk signal that tracked the discrepancy between current and predicted risk. Such neuronal signals may be involved in the updating of risk information.

2013年10月7日月曜日

Trial-Type Dependent Frames of Reference for Value Comparison

Laurence T. Hunt, Mark W. Woolrich, Matthew F. S. Rushworth, Timothy E. J. Behrens
PLoS Comput Biol 9(9): e1003225.

ヒトMEG。
意思決定を行う際には「複数の選択肢の価値の比較」が不可欠である。
その比較は「行動のレベル(例:右/左)」で行われているのか、もしくは「財のレベル(例:りんご/みかん)」で行われているのか?
→ 状況によって違う。選択肢が左右に同時に呈示される場合は「財のレベル」で価値の比較が行われるが、左右に順に呈示される場合は「行動のレベル」で行われる。

A central question in cognitive neuroscience regards the means by which options are compared and decisions are resolved during value-guided choice. It is clear that several component processes are needed; these include identifying options, a value-based comparison, and implementation of actions to execute the decision. What is less clear is the temporal precedence and functional organisation of these component processes in the brain. Competing models of decision making have proposed that value comparison may occur in the space of alternative actions, or in the space of abstract goods. We hypothesized that the signals observed might in fact depend upon the framing of the decision. We recorded magnetoencephalographic data from humans performing value-guided choices in which two closely related trial types were interleaved. In the first trial type, each option was revealed separately, potentially causing subjects to estimate each action's value as it was revealed and perform comparison in action-space. In the second trial type, both options were presented simultaneously, potentially leading to comparison in abstract goods-space prior to commitment to a specific action. Distinct activity patterns (in distinct brain regions) on the two trial types demonstrated that the observed frame of reference used for decision making indeed differed, despite the information presented being formally identical, between the two trial types. This provides a potential reconciliation of conflicting accounts of value-guided choice.

2013年10月6日日曜日

Online evaluation of novel choices by simultaneous representation of multiple memories

Helen C Barron, Raymond J Dolan & Timothy E J Behrens
Published online: 08 September 2013 | doi:10.1038/nn.3515

ヒトfMRI。
脳は「新しい(経験したことのない)選択肢の価値」をどう評価しているのか?
fMRI adaptation(「同じ種類の刺激を続けて呈示すると、その刺激を処理する脳部位のfMRI信号が低下する」現象)を用いて検証。
→ 海馬(Hippocampus)、前頭前野内側部(mPFC)から「(過去に経験済みの)選択肢の価値」を読み出し、それらを組み合わせて「新しい選択肢の価値」を計算している。

Prior experience is critical for decision-making. It enables explicit representation of potential outcomes and provides training to valuation mechanisms. However, we can also make choices in the absence of prior experience by merely imagining the consequences of a new experience. Using functional magnetic resonance imaging repetition suppression in humans, we examined how neuronal representations of novel rewards can be constructed and evaluated. A likely novel experience was constructed by invoking multiple independent memories in hippocampus and medial prefrontal cortex. This construction persisted for only a short time period, during which new associations were observed between the memories for component items. Together, these findings suggest that, in the absence of direct experience, coactivation of multiple relevant memories can provide a training signal to the valuation system that allows the consequences of new experiences to be imagined and acted on.

2013年10月3日木曜日

The Neural Basis of Following Advice

Biele G, Rieskamp J, Krugel LK, Heekeren HR
PLoS Biol 2011 9(6): e1001089. doi:10.1371/journal.pbio.1001089

報酬予測/強化学習(道具的条件づけ)に「他者からのアドバイス(学習を行う前に、一度だけ与えられる)」はどのように影響するか?
(1)被験者は「他者のアドバイス」に従う傾向がある。つまり、他者が奨めた選択肢を選びやすい。
(2)「価値(報酬予測)の初期値」に影響を与えるだけでなく、学習の過程全般にわたって影響を与える。
(3)具体的には、「強化子=実際の報酬+アドバイス・ボーナス」という形で学習に効いてくる。
(4)アドバイス・ボーナスは、報酬を処理する脳部位「尾状核(Caudate)」でコードされている。

Learning by following explicit advice is fundamental for human cultural evolution, yet the neurobiology of adaptive social learning is largely unknown. Here, we used simulations to analyze the adaptive value of social learning mechanisms, computational modeling of behavioral data to describe cognitive mechanisms involved in social learning, and model-based functional magnetic resonance imaging (fMRI) to identify the neurobiological basis of following advice. One-time advice received before learning had a sustained influence on people’s learning processes. This was best explained by social learning mechanisms implementing a more positive evaluation of the outcomes from recommended options. Computer simulations showed that this ‘‘outcome-bonus’’ accumulates more rewards than an alternative mechanism implementing higher initial reward expectation for recommended options. fMRI results revealed a neural outcome-bonus signal in the septal area and the left caudate. This neural signal coded rewards in the absence of advice, and crucially, it signaled greater positive rewards for positive and negative feedback after recommended rather than after non-recommended choices. Hence, our results indicate that following advice is intrinsically rewarding. A positive correlation between the model’s outcome-bonus parameter and amygdala activity after positive feedback directly relates the computational model to brain activity. These results advance the understanding of social learning by providing a neurobiological account for adaptive learning from advice.

2013年10月2日水曜日

In the Mind of the Market: Theory of Mind Biases Value Computation during Financial Bubbles

Benedetto De Martino, John P. O'Doherty, Debajyoti Ray, Peter Bossaerts, Colin Camerer
Neuron, Volume 79, Issue 6, 1222-1231, 18 September 2013

バブル状態の市場における意思決定(売り買い)の心理的/脳神経的メカニズム。

The ability to infer intentions of other agents, called theory of mind (ToM), confers strong advantages for individuals in social situations. Here, we show that ToM can also be maladaptive when people interact with complex modern institutions like financial markets. We tested participants who were investing in an experimental bubble market, a situation in which the price of an asset is much higher than its underlying fundamental value. We describe a mechanism by which social signals computed in the dorsomedial prefrontal cortex affect value computations in ventromedial prefrontal cortex, thereby increasing an individual’s propensity to ‘ride’ financial bubbles and lose money. These regions compute a financial metric that signals variations in order flow intensity, prompting inference about other traders’ intentions. Our results suggest that incorporating inferences about the intentions of others when making value judgments in a complex financial market could lead to the formation of market bubbles.

2013年10月1日火曜日

Thirst-dependent risk preferences in monkeys identify a primitive form of wealth

Hiroshi Yamada, Agnieszka Tymula, Kenway Louie, and Paul W. Glimcher
PNAS September 9, 2013

「サルは(ヒトと違って)リスク愛好的だ」と信じられていたが、実験課題について十分に訓練したら、サルもヒトと同様にリスク回避的だった。
http://www.pnas.org/content/early/2013/09/04/1308718110

サルのリスク下の意思決定(報酬は水)。
十分なトレーニングを積むことで、(ヒトと同様に)合理的な選択を行う事ができ、ややリスク回避的。
また、喉が乾いているときは、よりリスク回避的になる。

Experimental economic techniques have been widely used to evaluate human risk attitudes, but how these measured attitudes relate to overall individual wealth levels is unclear. Previous noneconomic work has addressed this uncertainty in animals by asking the following: (i) Do our close evolutionary relatives share both our risk attitudes and our degree of economic rationality? And (ii) how does the amount of food or water one holds (a nonpecuniary form of “wealth”) alter risk attitudes in these choosers? Unfortunately, existing noneconomic studies have provided conflicting insights from an economic point of view. We therefore used standard techniques from human experimental economics to measure monkey risk attitudes for water rewards as a function of blood osmolality (an objective measure of how much water the subjects possess). Early in training, monkeys behaved randomly, consistently violating first-order stochastic dominance and monotonicity. After training, they behaved like human choosers—technically consistent in their choices and weakly risk averse (i.e., risk averse or risk neutral on average)—suggesting that well-trained monkeys can serve as a model for human choice behavior. As with attitudes about money in humans, these risk attitudes were strongly wealth dependent; as the animals became “poorer,” risk aversion increased, a finding incompatible with some models of wealth and risk in human decision making.