==== Accepted for presentation at ICONIP'98 ======
====  Kitakiyushu,Japan, Oct 21--26, 1998   ======

A SEQUENCE LEARNING ARCHITECTURE BASED ON CORTICO-BASAL 
GANGLIONIC LOOPS AND REINFORCEMENT LEARNING

Raju S. Bapi and Kenji Doya}
{rajubapi, doya}@erato.atr.co.jp

Computational Neurobiology Group, 
Kawato Dynamic Brain Project, ERATO, JST
2-2 Hikaridai, Seika, Soraku, Kyoto 619-0288, Japan

Abstract 
A sequence processing neural network architecture is proposed based on
the known cortico-basal ganglionic structure.  Learning in the network
is implemented using biologically plausible reinforcement learning
paradigm.  The network is designed in order to understand the
functional roles of various brain areas involved in the performance of
sequential arm movements.  Findings from Tanji & Shima's experiments
on monkeys are used as specific test case. We propose that the matrix
and patch compartments of the basal ganglia implement an Actor and a
Critic, respectively and learn a sequence based on the motor efference
copy of previous movement.  Further, in order to flexibly shift among
various sequences, supplementary motor area structures learn the
sequence simultaneously and act via a biasing mechanism to enable the
basal ganglia-actor recall the correct sequence.  Simulation results
are presented to illustrate how these proposals can be implemented.

Keywords: 
Sequences, Reinforcement Learning, Cortico-Basal Ganglionic Loops