j. psy. sci., 29(2): 164-170 (2011)

7
臜臕膣膖膜臊腙臋臓臇臊膆腝膵膍膆膘 膲膒膐臌腂膲膒腠腚腊腂膾 膡 ῎῎腂膬 臒 膋 膶῎῎῎ 膊膹臀膃臑腼膃膑膈膯腂臍臛膃膭膴膉膀 腂腳腴腷腲腷臀膃῎῎ 腫腵腰腪腶腮腩膗腼臀膃῎῎῎ Neural mechanisms of goal-directed behavior and intrinsic motivation Kenji MATSUMOTO, Madoka MATSUMOTO, Kou MURAYAMA, and Keise IZUMA Tamagawa University Brain Science Institute, Japan Society for the Promotion of Science , University of Munich῎῎, California Institute of Technology῎῎῎ The prefrontal cortices play important roles in cognitive control of behavior. The medial prefrontal cortex guides actions on the basis of the representation of action-outcome contingency, because many neurons represent action-outcome contingency when represented information is examined in a visually cued go/no-go task with asymmetrical rewards with reversals. Also, the medial prefrontal cortex evaluates the correctness of actions on the basis of classifying the outcome of actions, because both success- and failure-responsive neurons were observed in an action-learning task. On the other hand, the lateral prefrontal cortex regulates the relationship between actions and rewards by comparing the values of intrinsic and extrinsic rewards, because the lateral prefrontal cortex shows the activity corresponding to the undermining e#ect of extrinsic rewards on intrinsic motivation in a task that could be voluntarily engaged in. Both the medial and lateral prefrontal cortices may interact to link between goal-directed behaviors and intrinsic motivation through the cortico-basal ganglia loops. Key words : prefrontal cortex, striatum, goal-based action selection, performance monitoring, in- trinsic motivation ῎ῌ῏῍ 膻臉臠腝臅腘腡腒腝膻臗腜腹臄腑腦膻臉膻臝 (Figure 1) 腞腁 膜臊腝臏臃臇膷膔腜膫臡腛臞膄腨腽腓腐腗腆腦腙 膚腈腤腧腗腆腦 (Stuss & Benson, 1986; Damasio, 1994; Fuster, 1997; Roberts, Robbins, Weiskrantz, 1998)腀 腺臙腁 膺膳膿 (Figure 2) 腞腁 臟膼腣腥腡臢腆膏腽 腋膇腌腓腙腌腜臓腾腐腗臘膪臟膽膕膟腨腬腃腭腐腗腆腦 臅臑腭腃腯腳腷膠臚腊腤腝臎臣腨膨腍腦腎腙腜腣腕腗腁 膢膎腢膜臊腝腻臂腝膃膩腙臖膓腜膫臡腛臞膄腨腽腓腐腗 腆腦腙膚腈腤腧腗腆腦 (Aosaki, Graybiel, & Kimura, 1994; Schultz, 1998; Sutton & Barto, 1998; Doya, 2002; O’Doherty, Dayan, Schultz, Deichmann, Fris- ton, & Dolan, 2004)腀 腎腧腤膻臉膻臝腙膺膳膿腙腞腁 膻 臉膻臝腄膺膳膿腄膺膳膿腸膁腝臀臑膅臆膂腝膰膂腄膥膱 腄膻臉膻臝腙腆腇臔膧膅臆膂腶腃腱腨膌膸腐腗腉腥 (Alexander, DeLong, & Strick, 1986)腁 腎腝腶腃腱腜 腣腕腗腁 膜臊腝臏臃臇膷膔腙膢膎腢腻臂腝膃膩腙腞臈膝 腏腧腗腆腦腙膚腈腤腧腦腀 臛膙腘腞腁 腎腝臔膧膅臆膂 腶腃腱腋腚腝腣腇腛膆臐腨臁腕腗腆腦腊腜腖腆腗腁 膤腓 腔腝膑膈腊腤膦膞腏腧腦腎腙腨膮腟腦腀 Tamagawa University Brain Science Institute, 611, Tamagawa-gakuen, Machida, Tokyo Copyright 2011. The Japanese Psychonomic Society. All rights reserved. ῑῐΐῒ ῑῐΐῒ The Japanese Journal of Psychonomic Science 2011, Vol. 29, No. 2, 164170

Upload: others

Post on 24-Apr-2022

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: J. Psy. Sci., 29(2): 164-170 (2011)

���������� ����

� � � ������������ � ����� � � ����

!"#$%&$'()��*+$,-./��01232#$��456789:;&#$���

Neural mechanisms of goal-directed behavior

and intrinsic motivation

Kenji MATSUMOTO, Madoka MATSUMOTO,

Kou MURAYAMA, and Keise IZUMA

Tamagawa University Brain Science Institute�,Japan Society for the Promotion of Science�, University of Munich��,

California Institute of Technology���

The prefrontal cortices play important roles in cognitive control of behavior. The medialprefrontal cortex guides actions on the basis of the representation of action-outcome contingency,because many neurons represent action-outcome contingency when represented information isexamined in a visually cued go/no-go task with asymmetrical rewards with reversals. Also, themedial prefrontal cortex evaluates the correctness of actions on the basis of classifying theoutcome of actions, because both success- and failure-responsive neurons were observed in anaction-learning task. On the other hand, the lateral prefrontal cortex regulates the relationshipbetween actions and rewards by comparing the values of intrinsic and extrinsic rewards, becausethe lateral prefrontal cortex shows the activity corresponding to the undermining e#ect ofextrinsic rewards on intrinsic motivation in a task that could be voluntarily engaged in. Both themedial and lateral prefrontal cortices may interact to link between goal-directed behaviors andintrinsic motivation through the cortico-basal ganglia loops.

Key words : prefrontal cortex, striatum, goal-based action selection, performance monitoring, in-trinsic motivation

� � � �

<=> ?@AB <CDEFGH<=<I (Figure

1)JK �� LM�NODPQRSTUVWXYZH�[\]^YZH (Stuss & Benson, 1986; Damasio,

1994; Fuster, 1997; Roberts, Robbins, Weiskrantz,

1998)_ `aK bcd (Figure 2)JK efghAiZjVklmW�mDnXYopeqrsUtuvXYZH?%vuw02xy�] z{U|}H~�Dg�YK����� �� $����DPQRSTUVWXY

ZH�[\]^YZH (Aosaki, Graybiel, & Kimura,

1994; Schultz, 1998; Sutton & Barto, 1998; Doya,

2002; O’Doherty, Dayan, Schultz, Deichmann, Fris-

ton, & Dolan, 2004)_ ~^]<=<I�bcd�JK <=<I�bcd�bcd�� #%��� ������<=<I�Z�������8u�U��XY�h(Alexander, DeLong, & Strick, 1986)K ~ 8u�Dg�YK �� LM�NO������ $��J���^YZH�[\]^H_ +�@JK ~ ������8u�k� g�R��U��YZH�D�ZYK �W  '(�]¡¢�^H~�U£¤H_

� Tamagawa University Brain Science Institute,6�1�1, Tamagawa-gakuen, Machida, Tokyo

Copyright 2011. The Japanese Psychonomic Society. All rights reserved.

������The Japanese Journal of Psychonomic Science

2011, Vol. 29, No. 2, 164�170

Page 2: J. Psy. Sci., 29(2): 164-170 (2011)

���������

�������� � ����������������������� !"� 2001#$ � ����������% 1 &'���()��$ *�% 1 &'������������ +,# �-./� *����0*/1����2�345�� % 2���)��$6/�*�% 2�������������-./� 6����2�345�% 3���)��778�191�� � ��5�:;�&'���)���<� =�

� >**?@�A<���B��$ CDE� FDGD@&'��86��HI�������8�������JKL��MCN�O��*8�9P�� � �������*8QRS8@�$ /�QP�� ������T'� ��8OU8�VW�����XY��� Z;������JKL?[�$

Matsumoto, Suzuki, & Tanaka (2003) � ��8OU�VW�����\]^_`ab���5������Hc��P�$ 2 C�de� 2 C���� 2 C�OU������fghi (Figure 2a) �jk�l�/�$ Dm 2 Z�node de 1� de 2# �pq�5r�Qstuvwxyz{|}~��F�� jknode������� GO�}�w��M� D� NG�����'������# ��P�$ �/M��/������8�6�����������OU ��[E�����D���/����# Q����$ ��P�������jkQ��8�8��pq�N���@5P�$ ������5�@�����K��� ���@2 ���de����OU��� ¡¢Q£D�� ¤��?pq���� ¡¢�¥15� 2 Z�1q5�¦t§b�¨©��node�ª��9EB����$ ����?��K�Q«E¬¡���� de����OU���­®�¯���?� jk6��°�de8��8���­®���±²?fg/@³�©@�@5P

Figure 1. Location of the prefrontal cortex andthe striatum.

Figure 2. (a) Behavioral task to examine goal-based action selection. Dashed box surrounds the visualcues that provide the earliest opportunity to select action in a trial. (b) A medial prefrontal neuronthat represent a specific action-outcome combination (NG/�) in the responses to the visual cues.

!"´!"´µ¶´.·¸ ������8]¹��º�»¼º½ 165

Page 3: J. Psy. Sci., 29(2): 164-170 (2011)

��������� ���������������� ��� !"#$�%&��'()� *+�,-.�/01� 2�,341567� 28�9:�;<�=5>?@'� 2?@�A7�$'BC?DEF�GHI=� !" (Figure 2b)-JKLEMN�� (�<� !"5� BC�$'BC?@'?DEF�GH�OP=� ��-NO� � �����5� �$'?@QRSTU�VWP�� XYZ[\6�$]^�_`6ab�@��OP='cd<�=��M�� ]^���$-XY'��?@�e>�fgFh(I'5i<6P� ��-NO� �$�XYZ[\

�jk�O� 9lXY�mn�opI=�q�5� gFh(��?@-� 2�$XY'��?@6M� 2r16PM�stI=('-_`1u=�

�������

�$?@-XY��vI=MwrMTU�EPOx�yz{|}�~<M�I=�qm��Matsu-

moto, Matsumoto, Abe, & Tanaka (2007)5�N���/����]^��/����*+�,��=���������NO�I���r6�$����(Figure 3a)�������� �>� ��BC��'�� ��� ���� ¡O �¢¡=('��NO�:1���£'�O¤r���¥d�� ¦�� §{

Figure 3. (a) Behavioral task to examine evaluation of outcome of action selection. (b) Averagedresponses of positive feedback-preferring neurons (top) and negative feedback-preferring ones in themedial prefrontal cortex (bottom).

V¨©ª�«¬ ­ 29® ­ 2¯166

Page 4: J. Psy. Sci., 29(2): 164-170 (2011)

���������� ��� ������������������������������������������ �! "#$%&����'�� ����(�� �! ��'��)*����+��� ��������������! ,�� -���./�� 10� ,�12��3! ������! �&45��6����� 7 ,�12�� ��!��&������ �89:�;�<2�=>&$?�@AB �� �� 30C&� 4

0���$�������@B �� ��! DE�������FGHI������ J/$��� ������������&� ?����������B�K���L�$&AM�������� �N!"O�P�#5$ $Q%RS&�T'�N!��� ���U(��� ����U(��� � �����VW�%RS&�)X YT'Z �[��*\�� ������ �! ���U(��� �[])X@A%RS& (Figure 3b, top) ����U(��� �[])X@A%RS& (Figure 3b, bot-

tom)� 2+,�(AB �^���� B5��%RS&! ������������5���� 10���89:�;�<2P_ `ab - C�cd�.�5� �[])X��� a��������^������89:�;�<2P_ `ab/�C�cd��.�5C��� �! ��&������0�$���U(��$e B5��%RS&!f ��)X�4 -�X1�- �! -�L'�����`&$��cd�0X��2g�3��� B�B ! ������ �+`���B5��%RS&! ��/����U(�4#A�cd�5h�$&AB �3i�$%b ������ ! jB��L'��678�9a�&e���k��:l�e;<C=m�n��$&A o#�5A�

�����

�6>p?L'�8@@A�/�! L'�0@A9:�;�<2q-5�AkBC��rs�(A� BC �$@t�D&`]�! E\FquvFC��GH?Iw��J?�xy@AQK?BCq 3J?�xy@AzLC��MK?BC{|k� ��� -k��}?CBC�C&NUe (AK��ObP~B QR���]$ K��%�A78QR��S?CBC�C�$%b �6Tp?L'�Gj��U�5$&AB eV&� � #� W�e�����4#A X�}?CBC�C]$eQS?�ObP~� B5!����

�]B ��A78���S?CBC C�$ Y��.L'�Z#$&A��(A� ��C��'F �&[3�Q�\��V]! Bk���S?CBC�Z#�5$&A ]�^_�- & (Deci & Ryan, 1985;

Carver & Scheier, 1998)� -B� Murayama, Matsu-

moto, Izuma, & Matsumoto (2010) YCamerer (2010)

e�`�B Z ! ����S?CBCaBbH�c���*�I�*\A�/� ,� fMRI (functional

magnetic resonance imaging) �d&$L��� ,�! 28e�f,� Yg 10e h 18eZ � K

��8i�X�$zLBC�j#A���� YBC�Z K��8i !kxy�D&��C]zLBC�j#����� Y0`�Z �]��� K�! ��<���<�� ����G��@B ���$� A{�5l�]�m/AK� Y��<���<�K�� 4.95�5.05lC� �8�� -5�}C� �no� ��Z �(A (Figure 4a, top)� Qp q\K� �$��<���<��Q'?�maA��+$������G��@K� Y��<���<�K�� �8�� e ��8�� eC�Z �d&� (Figure 4a, bottom)� B� 2`�K��30rL4`FGHICst� a5A¡<¢£G� 2

¡<¢£GL�$j& -�M��cT'� fMRI���$¤��� ��<���<�K��! 8�@Au� ¥v��53�5A�¦§�¨©�5��;<CB � w 1¡<¢£G�"3! �3�ª��$4#�� BC��! ��<���<�K��8�u� «200���¬��.�5A "3� ­�w1¡<¢£G���®¯x3� yz{b���¬��4#�� Qp 0`��! ��¬��`&$�"3!Q|4#4 w 1¡<¢£G���®¯x3� }[�8i !kxy���¬��4#� YC% ��¬��°!FGHI���{BC��[ ±<����Z� `ab ~p���e���������BC�4#�5$&A� BC��!8i�X��BC�4#�5� &kB �CA�w 2¡<¢£G�! w 1¡<¢£G ²Cb ������e ��¬�!Q|C&B ��#$ ��K��L�$e���� a� w 2¡<¢£G����! -5³5�f,�!�U�� 3]3 Q[���$e�& -�3 ´����$(A�µ¦G� ��<���<�K� ��<���<�K��Q������G�����¶B �� A�k��$%&�� !f,��B� 3]3� B5��K��QS?�·0��{�� ¦G¸¹���N!�$%b Yf,�!B�B ���C&Z -5��S?BC�=>]'º

�»��»��¼���� �6TpL' �S?'º�%Rº½ 167

Page 5: J. Psy. Sci., 29(2): 164-170 (2011)

�� ������� �� ��� (Deci & Ryan,

1985)� ����� ����������� ���!�"�#��$%&'(�)*��+,�-./01234��56�!� 78�9:�#�$

%;<=,�� #�$%&�>�?��@�� ���AB,������&CDE�FGHIJB,��&KLMH),� �M;NOPQRSTOUVW�X (Deci, 1971; Deci, Koestner, & Ryan, 1999; Murayama

Figure 4. (a) Behavioral task to examine intrinsic motivation. (b) Anterior striatum activation bysuccess vs. failure in the stop-watch task.

YZ23456 [ 29\ [ 2]168

Page 6: J. Psy. Sci., 29(2): 164-170 (2011)

& Kuhbandner, in press)� ������ �� �������������� ����� � !��"#�$!�� %&'� 3�(�)*+(�� ,-./01.234�5����6�7��8�9:;�� ���<� �$!���=>?� @ 1A.BC�D�@ 2A.BC�D� E� !��"#F 9GH#I��<JKH#IL� �$!���M�&� @ 1A.BC�N �OH#P;�Q R,S J��<H#34�TUVWXY� �� ��������Z[,-./01.234�\]^L�<_�����$<H#�`a�9b����<�cL<de'�L��f<H#� �$! � !<� gh�i��jI#,-./01.234� k\]lH�<m� kn�l H�<m<�"#�8�oIf��'�� (Fig-

ure 4b)� ��gh�i��fJ!(��pH#qL<�$!�� !cU��8�o:;�����<�@ 1A.BC�N �$!�� k\]l �r��stu�$�YvwI#jU ��tu�$��^Lf�oIfJxmy�H���<de'�L� z� @ 2

A.BC��� �!<����tu�$��esI<I{|}�[h~�P;���::�'� @ 1A.BC�<��� gh�i��jI# � !��$!cU��8�oIf��'��� ���<� �$!�� ,-./01.234� k\]l 5����s�$<H#�`a�9bH#IL� � !�� _�c{s9b��'�s:;��<J��H#IL<de'�L����� ,-./01.234< ��34�PL01.2,-./34���H# ��:U���K��#I� (Figure 4b)� ����:U���^L�fJ�"#qL< i�i�r��� ��%�gh��f<�[�� ���f�q'��� ^s�& @1A.BC���� !��"#�$!�f���oI� @ 2A.BC����$!�� !cU�i�i�r���f��J9G�����PL� ���<� %&'�34:<I{����� ,-./01.234�j?L k\]l 5�����$<H#�`a���J�?���i�i�r������� ���gh�i��j?L,-./01.234� k\]l <I{���$�`a���J�eL<I{�������  ¡.¢£�� �¤�¥¦§¨ /�jI#=�H#IL©��JKH#IL�

� � �

i�i���� ��Jª\^L�«���¬­

®¬­H�����ª\� I�:�!` i�i�r���r���$<���$�%&'��J¯">?L:��°�¯�H#IL<de'�L� DM±:U�s} I����¤�¥¦§¨ /J²[#¯�H#IL<de'�L�

� �

Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986).Parallel organization of functionally segregatedcircuits linking basal ganglia and cortex. Annual

Review of Neuroscience, 9, 357�381.

Aosaki, T., Graybiel, A. M., & Kimura, M. (1994).E#ect of the nigrostriatal dopamine system onacquired neural responses in the striatum of be-having monkeys. Science, 265, 412�415.

Camerer, C. F. (2010) Removing financial incentivesdemotivates the brain. Proceedings of the National

Academy of Sciences of the USA, 107, 20849�20850.

Carver, C. S., & Scheier, M. (1998). On the self-

regulation of behavior. Cambridge: Cambridge Uni-versity Press.

Damasio, A. R. (1994). Descartes’ error: emotion, rea-

son, and the human brain. New York: G. P. Putnam.Deci, E. L. (1971). E#ects of externally mediated re-

wards on intrinsic motivation. Journal of Personal-

ity and Social Psychology, 18, 105�115.Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-

analytic review of experiments examining thee#ects of extrinsic rewards on intrinsic motiva-tion. Psychological Bulletin, 125, 627�668; discus-sion 692�700.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation

and self-determination in human behavior. NewYork: Plenum.

Doya, K. (2002). Metalearning and neuromodulation.Neural Networks, 15, 495�506.

Fuster, J. M. (1997). The prefrontal cortex: anatomy,

physiology, and neuropsychology of the frontal lobe.Philadelphia: Lippincott-Raven.

Matsumoto, K., Suzuki, W., & Tanaka, K. (2003). Neu-ronal correlates of goal-based motor selection inthe prefrontal cortex. Science, 301, 229�232.

Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K.(2007). Medial prefrontal cell activity signalingprediction errors of action values. Nature Neurosci-

ence, 10, 647�656.Murayama, K., & Kuhbandner, C. (in press). Money

enhances memory consolidation�But only for bor-ing materials. Cognition, 119, 120�124.

Murayama, K., Matsumoto, M., Izuma, K., & Matsu-moto, K. (2010). Neural basis of the undermininge#ect of monetary reward on intrinsic motivation.Proceedings of the National Academy of Sciences of

the USA, 107, 20911�20916.

³´µ³´µ¶·µ�#¸ ��¹º�<��=�»¼=½ 169

Page 7: J. Psy. Sci., 29(2): 164-170 (2011)

O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R.,Friston, K., & Dolan, R. J. (2004). Dissociable rolesof ventral and dorsal striatum in instrumental con-ditioning. Science, 304, 452�454.

Roberts, A. C., Robbins, T. W., & Weiskrantz, L., (Eds.)(1998). The prefrontal cortex: executive and cognitive

functions. Oxford: Oxford University Press.Schultz, W. (1998). Predictive reward signal of dopa-

mine neurons. Journal of neurophysiology, 80, 1�27.Stuss, D. T., & Benson, D. F. (1986). The frontal lobes.

New York: Raven Press,.Sutton, R. S., & Barto, A. G. (1998). Reinforcement

learning: an introduction. Cambridge, MA: MITPress.��������� (2001). �� ��������� 39, 76�83.

������� � 29� � 2�170