Metalearning and Neuromodulators

Asahi S., Okamoto, Y., Okada, G., Morinobu, S., Yamawaki, S., , Doya K. (2002). Relationship between brain activation during GO/NOGO task and impulsiveness: A fMRI study. 32nd Annual Meeting, Society for Neuroscience, Orlando, USA.

Doya K. (2003). A computational theory of neuromodulation. International Symposium "New Horizons in Molecular Sciences and Systems: An Integrated Approach," 50.

Doya K. (2003). Enjoy now or strive for future: Neural mechanisms of reward prediction at different time scales. Summer Program 2003 Progarm and Abstracts, 36.

Doya K. (2002). Metalearning and neuromodulation. Neural Networks, 15, 495-506. [pdf]

Doya K. (2002). Computational Models of Neuromodulation. Neural Networks 2002 Special Issue on Computational Models of Neuromodulation.

Doya K. (2001). Metalearning and neuromodulation. CREST Workshop on Metalearning and Neuromodulation, Seika, Kyoto, 6.

Doya K. (2000). Metalearning, neuromodulation, and emotion. Hatano G, Okada N, Tanabe H, Affective Minds, Elsevier Science, 101-104. [pdf]

Doya K. (2000). A possible role of serotonin in regulating the time scale of reward prediction. Serotonin Conference.

Doya K. (2000). Possible roles of neuromodulators in the regulation of learning processes. 30th Annual Meeting, Society for Neuroscience.

Doya K. (1999). Metalearning, neuromodulation and emotion. 13th Toyota Conference on Affective Minds, Mikkabi, Japan, 46-47. [1999E06tc.pdf]

Doya K., Okada G., Ueda K., Okamoto Y., Yamawaki S. (2001). Prediction of short- and long-term reward: A functional MRI study with a Markov decision problem. 31st Annual Meeting, Society for Neuroscience, San Diego, USA.

Ito, M., Doya, K., Shirao, T., Sekino, Y. (2004). Fos imaging reveals that the supramammillary nucleus enhances hippocampal activity of rats placed in a novel open field. Society for Neuroscience 34th Annual Meeting, 96.

Okada G., Okamoto, Y., Ueda, K., Morinobu, S., Yamawaki, S., Doya, K. (2002). Selection between small, immediate rewards and large, delayed rewards in prediction of future reward: A functional magnetic resonance imaging study. 8th International Conference on Functional Mapping of the Human Brain, Sendai, Miyagi.

Okada G., Okamoto Y., Ueda K., Yamashita H., Kagaya A., Morinobu S., Yamawaki S., Doya K. (2001). Localization of brain activity in prediction of future reward using fMRI and MEG. 31st Annual Meeting, Society for Neuroscience, San Diego, USA.

Schweighofer, N., Doya, K., Kuroda, S. (2004). Cerebellar aminergic neuromodulation: towards a functional understanding. Brain research reviews, 44, 103-116. [pdf]

Schweighofer,N., Tanaka, S., Asahi, S., Okamoto, Y., Doya, K., Yamawaki, S. (2004). An fMRI study of the delay discounting of reward after tryptophan depletion and loading. 1: decision-making. Society for Neuroscience 34th Annual Meeting, 97.

Schweighofer N., Doya K. (2003). Meta-learning of reinforcement learning. Neural Networks, 16, 5-9.

Schweighofer N., Doya K. (2002). A biologically plausible computational model of meta-learning in reinforcement learning. 32nd Annual Meeting, Society for Neuroscience, Orlando, USA.

Tanaka, S., Doya, K., Okada, G., Ueda, K., Okamoto, Y., Yamawaki, S. (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neuroscience, 7(8), 887-893. [doi:10.1038/nn1279] [pdf] [pdf-s]

Tanaka, S., Schweighofer,N., Asahi, S., Okamoto, Y., Yamawaki, S., Doya, K. (2004). An fMRI study of the delay discounting of reward after tryptophan depletion and loading. 2: reward expectation. Society for Neuroscience 34th Annual Meeting, 98.

Tanaka S., Doya K., Okada G., Ueda K., Okamoto Y., Yamawaki S. (2004). Different cortico-basal ganglia loops specialize in reward prediction on different time scales. Advances in Neural Information Processing Systems16, 701-708, MIT Press.[pdf]

Tanaka S., Doya K., Okada G., Ueda K., Okamoto Y., Yamawaki S. (2003). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Society for Neuroscience 33rd Annual Meeting, 58.

Ueda Y., Samejima K., Doya K., Kimura M. (2002). Reward value-dependent striate neuron activity of monkey performing trial-and-error behavioral decision task. 32nd Annual Meeting, Society for Neuroscience, Orlando, USA.

Back to the top

Specialization and Integration of Brain Areas

Doya K. (2001). Robotic neuroscience: A synthetic approach to the brain. Neuroscience Research, Supplement, 24, S16.

Doya K., Kimura H., Kawato M. (2001). Neural mechanisms of learning and control. IEEE Control Systems Magazine, 21, 42-54.

Doya K., Kimura H., Miyamura A. (2001). Motor control: Neural models and system theory. International Journal of Applied Mathematics and Computer Science, 11, 101-128. [pdf]

Kimura H., Doya K. (2000). Motor control: Neural models and system theory. 14th International Symposium on Mathematical Theory and Networks and Systems, 232.

Back to the top

Reinforcement Learning

Doya K. (2000). Reinforcement learning in continuous time and space. Neural Computation, 12, 219-245. [pdf]

Doya K. (1997). Efficient nonlinear control with actor-tutor architecture. Mozer MC, Jordan MI, Petsche T, Advances in Neural Information Processing Systems 9, MIT Press, 1012-1018.

Doya K. (1996). Temporal difference learning in continuous time and space. Touretzky DS, Mozer MC, Hasselmo ME, Advances in Neural Information Processing Systems 8, MIT Press, 1073-1079.

Doya K. (1996). Reinforcement learning in animals and robots. International Workshop on Brainware, 69-71.

Koike Y., Doya K. (1999). Multi state estimation reinforcement learning for driving model. IEEE International Conference on System, Man and Cybernetics, Tokyo, V, 504-509.

Morimoto, J., Doya, K. (2005). Robust reinforcement learning. Neural Computation, 17, 335-359.

Morimoto J., Doya K. (2001). Robust reinforcement learning. Advances in Neural Information Processing Systems 13, MIT Press, 1061-1067. [2001E01nips.pdf]

Morimoto J., Doya K. (2000). Robust reinforcement learning. Technical Report of IEICE, NC2000-49, 59-66.

Nakahara H., Doya K. (1998). Near saddle-node bifurcation behavior as dynamics in working memory for goal-directed behavior. Neural Computation, 10, 113-132.

松原崇充, 森本淳, 中西淳, 佐藤雅昭, 銅谷賢治. 方策こう配法を用いた動的行動則の獲得：2足歩行運動への適用. 電子情報通信学会論文誌, Vol.J88-D-II, (1), 53-65.

銅谷賢治 (2004). 学習する身体ーロボットの脳をつくるー.　けいはんな社会的知能発生学研究会編，知能の謎, 111-131, ブルーバックス，講談社．

Back to the top

Bayesian Inference

Samejima K., Doya K., Ueda K., Kimura M. (2004). Estimating internal variables and parameters of a learning agent by a particle filter. Advances in Neural Information Processing Systems16, 1335-1342, MIT Press. [pdf]

Sato, M., Yoshioka, T., Kajiwara, S., Toyama, K., Goda, N., Doya, K., Kawato, M. (2004). Hierarchical bayesian estimation for MEG inverse problem. NeuroImage, 23, 806-826.

Back to the top

Cyber Rodent

Capi, G., Doya, K. (2005). Evolution of neural architecture fitting environmental dynamics. Adaptive Behavior, 13, 53-66. [pdf]

Capi G., Doya K. (2003). Evolving recurrent neural controllers for sequential tasks - A parallel implementation. Congress on Evolutionary Computation, 1, 514-519.

Capi G., Uchibe E., Doya K. (2003). Selection of neural architecture and the environment complexity. Dynamic Systems Approach for Embodiment and Sociality From Ecological Psychology to Robotics, 6, 311-317. Advanced Knowledge International.

Capi G., Uchibe E., Doya K. (2002). Selection of neural architecture and the environment complexity. The 3rd International Symposium on Human and Artificial Intelligence Systems: Dynamic Systems Approach for Embodiment and Sociality, Fukui, Japan, 231-237. [pdf]

Doya K., Uchibe E. (2005). The Cyber Rodent project: Exploration of adaptive mechanisms for self-preservation and self-reproduction. Adaptive Behavior.

Doya K. (2003). Cyber rodents: Self-preserving, self-reproducing robotic colony. ATR Up to Date, Summer 2003, 12-13.

Elfwing S., Uchibe E., Doya K. (2003). An Evolutionary Approach to Automatic Construction of the Structure in Hierarchical Reinforcement Learning. Genetic and Evolutionary Computation - GECCO 2003 Proceedings, Part 1, Chicago, IL, Springer, GECCO 2003, LNCS 2723, 507-509. [pdf]

Eriksson A., Capi G., Doya K. (2003). Evolution of meta-parameters in reinforcement learning algorithm. IEEE/RSJ IROS. [pdf]

Back to the top

Learning to Stand Up

Morimoto J., Doya K. (2001). Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning. Robotics and Autonomous Systems, 36, 37-51.

Morimoto J., Doya K. (2000). Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning. 17th International Conference on Machine Learning, 1, 623-630.

Morimoto J., Doya K. (1999). Hierarchical reinforcement learning for motion learning: learning "stand-up" trajectories. Advanced Robotics, 13, 267-268.

Morimoto J., Doya K. (1998). Reinforcement learning of dynamic motor sequence: Learning to stand up. Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 3, 1721-1726.

Morimoto J., Doya K. (1998). Hierarchical reinforcement learning of low-dimensional subgoals and high-dimensional trajectories. The 5th International Conference on Neural Information Processing, 2, 850-853.

Back to the top

Multiple Models

Bapi R.S., Doya K. (2001). Multiple forward model architecture for sequence processing. Sun R, Giles L, Sequence Learning: Paradigms, Algorithms, and Applications, Springer Verlag, 309-320.

Doya K., Katagiri K., Wolpert D.M., Kawato M. (2000). Recognition and imitation of movement paterns by a multiple predictor-controller architecture. Technical Report of IEICE, TL2000-11, 33-40.

Doya K., Samejima K., Katagiri K., Kawato M. (2002). Multiple model-based reinforcement learning. Neural Computation, 14, 1347-1369. [pdf]

Doya K., Samejima K., Katagiri K., Kawato M. (2001). Task decomposition and imitation by MOSAIC architecture. HFSP Arundel Meeting / Wolpert Group, Arundel, Canada

Doya K., Samejima K., Katagiri K., Kawato M. (2000). Multiple model-based reinforcement learning. Japan Science and Technology Corporation. [pdf]

Samejima K., Doya K., Kawato M. (2003). Inter-module credit assignment in modular reinforcement learning. Neural Networks, 16, 985- 994.

Samejima K., Doya K., Kawato M. (2002). Inter-module credit assignment in modular reinforcement learning. Neural Networks.

Uchibe, E., Doya, K. (2004). Competitive-cooperative-concurrent reinforcement learning with importance sampling. The Eighth International Conference on the Simulation of Adaptive Behavior, 287-296.

内部英治, 銅谷賢治 (2004). 複数報酬のもとでの階層強化学習. 日本ロボット学会誌.

杉本徳和, 鮫島和行, 銅谷賢治, 川人光男 (2004). 複数の状態予測と報酬予測モデルによる強化学習と行動目標の推定. 電子情報通信学会論文誌, J87-D-�U(2), 683-694.

Back to the top

Communication

Doya K., Sugimoto N., Wolpert D.M., Kawato M. (2003). Selecting Optimal Behaviors Based on Contexts. International Symposium on Emergent Mechanisms of Communication, Awaji, 19-23. [pdf]

Nagayuki Y., Ishii S., Doya K. (2000). Multi-agent reinforcement learning: an approach based on the other agent's internal model. Fourth International Conference on Multi-Agent Systems, 215-221.

Nagayuki Y., Ishii S., Ito M., Shimohara K., Doya K. (2000). A multi-agent reinforcement learning method with the estimation of the other agent's actions. Fifth International Symposium on Artifical Life and Robotics, 1, 255-259. [2000E10alr00.pdf]

Wolpert D.M., Doya K., Kawato M. (2003). A unifying computational framework for motor control and social interaction. Philosophical Transactions of the Royal Society, 358. [pdf]

Back to the top

Cerebellum

Kuroda S., Yamamoto K., Miyamoto H., Doya K., Kawato M. (2001). Statistical characteristics of climbing fiber spikes necessary for efficient cerebellar learning. Biological Cybernetics, 84, 183-192. [pdf]

Kuroda S., Yamamoto K., Miyamoto H., Doya K., Kawato M. (2000). Statistical characteristics of climbing fiber spikes necessary for efficient cerebellar learning. Kawato Dynamic Brain Project.

Schweighofer, N., Doya, K., Fukai, H. , Chiron, Jean V., Furukawa, T., Kawato, M. (2004). Chaos may enhance information transmission in the inferior olive. Proceedings of the National Academy of Sciences, USA, 101(13), 4655-4660. [pdf]

Schweighofer N., Doya K., Kawato M. (1999). Electrophysiological properties of inferior olive neurons: A compartmental model. Journal of Neurophysiology, 82, 804-817.

Schweighofer N., Doya K., Kawato M. (1998). A model of the electrophysiological properties of the inferior olive neurons. 28th Annual Meeting, Society for Neuroscience, 24, 667.

Schweighofer N., Doya K., Lay F. (2001). Unsupervised learning of granule cell sparse codes enhances cerebellar adaptive control. Neuroscience, 103, 35-50. [pdf]

Back to the top

Sequence Learning and Basal Ganglia

Bapi R.S., Doya K. (1999). MFM: Multiple forward model architecture for sequence processing. IJCAI'99 Workshop on Sequence Learning, Stockholm, Sweden.

Bapi R.S., Doya K. (1998). A sequence learning architecture based on cortico-basal ganglionic loops and reinforcement learning. The 5th International Conference on Neural Information Processing, 1, 260-263.

Bapi R.S., Doya K. (1998). Evidence for effector independent and dependent components in motor sequence learning. 28th Annual Meeting, Society for Neuroscience, 24, 167.

Bapi R.S., Doya K., Harner A.M. (2000). Evidence for effector independent and dependent representations and their differential time course of acquisition during motor sequence learning. Experimental Brain Research, 132, 149-62.

Bapi R.S., Doya K., Harner A.M. (1999). Visual and motor representations for sequence learning. Japan Science and Technology Corporation.

Bissmarck, F., Nakahara, H., Doya, K., Hikosaka, O. (2005). Responding to modalities with different latencies. Advances in Neural Information Processing Systems, MIT Press.

Bissmarck F., Nakahara H., Doya K., Hikosaka O. (2003). Parallel network mechanisms for motor sequence acquisition in real time. IEICE Technical Report, 102, 113-118.

Doya K. (2001). Specialization of cerebellum, basal ganglia, and cerebral cortex in supervised, reinforcement, and unsupervised learning. Stockholm Workshop on Computational Neuroscience, Stockholm, 13.

Doya K. (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Current Opinion in Neurobiology, 10, 732-739.[pdf]

Doya K. (1999). What are the computations of the cerebellum, the basal ganglia, and the cerebral cortex. Neural Networks, 12, 961-974. [pdf]

Doya K. (1999). Multiple representation and algorithms for sequence learning. 2nd International Conference on Cognitive Science, Tokyo, 17-19. [pdf]

Doya K. (1998). Integration of cortical, cerebellar and basal ganglionic modules specialized in unsupervised, supervised and reinforcement learning. International Basal Ganglia Society 6th Triennial Meeting, 27.

Doya K. (1997). How basal ganglia, cerebellum and cerebral motor areas work together in sequential control tasks. Neural Control of Movement, 7th Annual Meeting Abstracts, 28.

Doya K. (1996). An integrated model of basal ganglia and cerebellum in sequential control tasks. 26th Annual Meeting, Society for Neuroscience, 22, 2029.

Haruno, M., Kuroda, T., Doya, K. , Toyama, K. , Kimura, M. , Samejima, K. , Imamizu, H. , Kawato, M. (2004). A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. Journal of Neuroscience, 24(7), 1660-1665.

Haruno M., Kuroda T., Doya K., Toyama K., Kimura M., Samejima K., Imamizu H., Kawato M. (2001). fMRI study of human brain activity during reinforcement learning. 31st Annual Meeting, Society for Neuroscience, San Diego, USA.

Hikosaka O., Nakahara H., Rand M.K., Sakai K., Lu X., Nakamura K., Miyachi S., Doya K. (1999). Parallel neural networks for learning sequential procedures. Trends in Neurosciences, 22, 464-471. [pdf]

Miyapuram K.P., Bapi R.S., Samejima K., Doya K. (2001). fMRI investigation of the learning of visuo-motor sequences. 31st Annual Meeting, Society for Neuroscience, San Diego, USA.

Nakahara H., Doya K., Hikosaka O. (2001). Parallel cortico-basal ganglia mechanisms for acquisition and execution of visuo-motor sequences - A computational approach. Journal of Cognitive Neuroscience, 13, 626-647. [pdf]

Nakahara H., Doya K., Hikosaka O. (1998). Benefit of multiple representations in parallel cortico-basal ganglia mechanisms for acquisition and execution of visuo-motor sequences. International Basal Ganglia Society 6th Triennial Meeting, 29.

Nakahara H., Doya K., Hikosaka O. (1998). Benefit of multiple representaitons for motor sequence control in the basal ganglia loops. RIKEN Brain Science Institute. [pdf]

Nakahara H., Doya K., Hikosaka O., Nagano S. (1998). Reinforcement learning with multiple representations in the basal ganglia loops for sequential motor control. International Joint Conference on Neural Networks, 1553-1558.

Nakahara H., Doya K., Hikosaka O., Nagano S. (1997). Multiple representations in the basal ganglia loops for acquisition and execution of sequential motor control. 27th Annual Meeting, Society for Neuroscience, 23, 778.

Nakahara H., Doya K., Hikosaka O., Nagano S. (1997). Multiple representations in the basal ganglia loops for sequential decision making. Technical Report of IEICE, NC97-24.

Samejima K., Ueda Y., Doya K., Kimura M. (2003). Activity of striate projection neurons encodes action-selective reward expectations. Society for Neuroscience 33rd Annual Meeting, 78.

Samejima K., Ueda Y., Kimura M., Doya K., Schweighofer N. (2000). Information coding of the striatal neurons during seqential movement. 30th Annual Meeting, Society for Neuroscience.

Ueda Y., Samejima K., Doya K., Kimura M. (2003). Reward Value Dependent Striate Neuron Activity of Monkey Performing Trial and Error Behavioral Decision Task. Neuroscience Research, Vol. 46 Suppl. 1 S1-S220, S50.

Back to the top

Cortical Dynamics

Matsumoto N., Okada M., Doya K., Sugase Y., Yamane S., Kawano K. (2001). Dynamics of the face-responsive neurons in the temporal cortex. Neuroscience Research, Supplement, 24, S73.

Okada M., Toya K., Kimoto T., Doya K. (1999). Retrieval dynamics of associative memory model can explain temporal dynamics of face-responsive neurons in the IT cortex. 29th Annual Meeting, Society for Neuroscience, Miami Beach, Florida, USA.

Tabata H., Shibata T., Taguchi S., Doya K., Kawato M. (2001). A simulation study on smooth pursuit and ocular following responses based on an MST neural-field model. Society for Neuroscience, San Diego, USA.

Back to the top

Bird Song

Doya K., Sejnowski T.J. (1999). A computational model of avian song learning. Gazzaniga MS, The New Cognitive Neurosciences, MIT Press, 469-482.

Doya K., Sejnowski T.J. (1998). A computational model of birdsong learning by auditory experience and auditory feedback. Brugge J, Poon P, Central Auditory Processing and Neural Modeling, 77-88.

Doya K., Sejnowski T.J. (1995). A novel reinforcement model of birdsong vocalization learning. Tesauro G, Touretzky DS, Leen TK, Advances in Neural Information Processing Systems 7, MIT Press, 101-108.

Doya K., Sejnowski T.J. (1995). A computational model of birdsong vocalization learning. Fourth IBRO World Congress of Neuroscience Abstracts, 502.

Doya K., Sejnowski T.J. (1995). A model of birdsong vocalization learning. Burrows M, Matheson T, Newland PL, Schuppe H, Nervous Systems and Behavior, 76.

Doya K., Sejnowski T.J. (1994). A computational model of song learning in the anterior forebrain pathway of the birdsong control system. 24th Annual Meeting, Society for Neuroscience, 20, 166.

Back to the top

Neural Oscillators

Doya K., Boyle M.E.T., Beauchamp M., Selverston A.I. (1993). Computational modeling of the musculoskeletal system of the lobster gastric mill. 23rd Annual Meeting, Society for Neuroscience, 19, 1602.

Doya K., Boyle M.E.T., Selverston A.I. (1993). Mapping between neural and physical activities of the lobster gastric mill. Giles CL, Hanson SJ, Cowan JD, Advances in Neural Information Processing Systems 5, Morgan Kaufmann, 913-920.

Doya K., Selverston A.I. (1994). Dimension reduction of biological neuron models by artificial neural networks. Neural Computation, 6, 696-717.

Doya K., Selverston A.I. (1993). A learning algorithm for Hodgkin-Huxley type neuron models. Proceedings of 1993 International Joint Conference on Neural Networks, 1108-1111.

Doya K., Selverston A.I., Rowat P.F. (1994). A Hodgkin-Huxley type neuron model that learns slow non-spike oscillation. Cowan JD, Tesauro G, Alspector J, Advances in Neural Information Processing Systems 6, Morgan Kaufmann, 566-573.

Schweighofer N., Doya K., Kawato M. (1998). A model of the electrophysiological properties of the inferior olive neurons. The 5th International Conference on Neural Information Processing, 3, 1525-1528.

Back to the top

Recurrent Networks

Doya K. (2002). Recurrent neural networks: Supervised Learning. Arbib M, The Handbook of Brain Theory and Neural Networks, Second Edition. [pdf]

Doya K. (1995). Recurrent networks: Supervised learning. Arbib M, The Handbook of Brain Theory and Neural Networks, (796-800)

Doya K. (1992). Bifurcations in the learning of recurrent neural networks. Proceedings of 1992 IEEE International Symposium on Circuits and Systems, 2777-2780.

Doya K. (1991). A study of learning algorithms for continuous-time recurrent neural networks. Department of Mathematical Engineering and Information Physics, University of Tokyo.

Doya K. (1990). Learning temporal patterns in recurrent neural networks. Proceedings of 1990 IEEE System, Man and Cybernetics Conference, 170-172.

Doya K., Yoshizawa S. (1992). Adaptive synchronization of neural and physical oscillators. Moody JE, Hanson SJ, Lippmann RP, Advances in Neural Information Processing Systems 4, 109-116.

Doya K., Yoshizawa S. (1991). Neural network model of temporal pattern memory. Systems and Computers in Japan, 22, 61-69.

Doya K., Yoshizawa S. (1991). Geometric analysis of the dynamics of autocorrelation associative memory. International Conference on Artificial Neural Networks, 1, 261-266.

Doya K., Yoshizawa S. (1990). Memorizing hierarchical temporal patterns in analog neuron networks. Proceedings of 1990 International Joint Conference on Neural Networks, San Diego, III:299-304.

Doya K., Yoshizawa S. (1989). Adaptive neural oscillator using continuous-time back-propagation learning. Neural Networks, 2, 375-386.

Doya K., Yoshizawa S. (1989). Memorizing oscillatory patterns in the analog neuron network. Proceedings of 1989 International Joint Conference on Neural Networks, I:27-32.

Nakahara H., Doya K. (1996). Dynamics of attention as near saddle-node bifurcation behavior. Touretzky DS, Mozer MC, Hasselmo ME, Advances in Neural Information Processing Systems 8, MIT Press, 38-44.

Back to the top