2012年1月12日木曜日

Neural Prediction Errors Reveal a Risk-Sensitive Reinforcement-Learning Process in the Human Brain


Yael Niv, Jeffrey A. Edlund, Peter Dayan, and John P. O'Doherty
J. Neurosci. 2012;32 551-562
http://www.jneurosci.org/cgi/content/abstract/32/2/551?etoc

試行錯誤で価値を学習する場合でもリスク回避傾向が見られ、その傾向は(非線形な効用関数ではなく)「報酬予測誤差が負の時の学習率を大きくする」ことで実現されている。さらに、これは線条体の活動パターンとも一致する。 http://bit.ly/yTvRLP

Humans and animals are exquisitely, though idiosyncratically, sensitive to risk or variance in the outcomes of their actions. Economic, psychological, and neural aspects of this are well studied when information about risk is provided explicitly. However, we must normally learn about outcomes from experience, through trial and error. Traditional models of such reinforcement learning focus on learning about the mean reward value of cues and ignore higher order moments such as variance. We used fMRI to test whether the neural correlates of human reinforcement learning are sensitive to experienced risk. Our analysis focused on anatomically delineated regions of a priori interest in the nucleus accumbens, where blood oxygenation level-dependent (BOLD) signals have been suggested as correlating with quantities derived from reinforcement learning. We first provide unbiased evidence that the raw BOLD signal in these regions corresponds closely to a reward prediction error. We then derive from this signal the learned values of cues that predict rewards of equal mean but different variance and show that these values are indeed modulated by experienced risk. Moreover, a close neurometric–psychometric coupling exists between the fluctuations of the experience-based evaluations of risky options that we measured neurally and the fluctuations in behavioral risk aversion. This suggests that risk sensitivity is integral to human learning, illuminating economic models of choice, neuroscientific models of affective learning, and the workings of the underlying neural mechanisms.

0 件のコメント:

コメントを投稿