Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats

Learning to form appropriate, task-relevant working memory representations is a complex process central to cognition. Gating models frame working memory as a collection of past observations and use reinforcement learning (RL) to solve the problem of when to update these observations. Investigation o...

Full description

Bibliographic Details
Main Authors:	Lloyd, Kevin, Becker, Nadine, Jones, Matthew W., Bogacz, Rafal
Format:	Online
Language:	English
Published:	Frontiers Media S.A. 2012
Online Access:	https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3483721/

id	pubmed-3483721
recordtype	oai_dc
spelling	pubmed-34837212012-10-31 Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats Lloyd, Kevin Becker, Nadine Jones, Matthew W. Bogacz, Rafal Neuroscience Learning to form appropriate, task-relevant working memory representations is a complex process central to cognition. Gating models frame working memory as a collection of past observations and use reinforcement learning (RL) to solve the problem of when to update these observations. Investigation of how gating models relate to brain and behavior remains, however, at an early stage. The current study sought to explore the ability of simple RL gating models to replicate rule learning behavior in rats. Rats were trained in a maze-based spatial learning task that required animals to make trial-by-trial choices contingent upon their previous experience. Using an abstract version of this task, we tested the ability of two gating algorithms, one based on the Actor-Critic and the other on the State-Action-Reward-State-Action (SARSA) algorithm, to generate behavior consistent with the rats'. Both models produced rule-acquisition behavior consistent with the experimental data, though only the SARSA gating model mirrored faster learning following rule reversal. We also found that both gating models learned multiple strategies in solving the initial task, a property which highlights the multi-agent nature of such models and which is of importance in considering the neural basis of individual differences in behavior. Frontiers Media S.A. 2012-10-30 /pmc/articles/PMC3483721/ /pubmed/23115551 http://dx.doi.org/10.3389/fncom.2012.00087 Text en Copyright © 2012 Lloyd, Becker, Jones and Bogacz. http://www.frontiersin.org/licenseagreement This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.
repository_type	Open Access Journal
institution_category	Foreign Institution
institution	US National Center for Biotechnology Information
building	NCBI PubMed
collection	Online Access
language	English
format	Online
author	Lloyd, Kevin Becker, Nadine Jones, Matthew W. Bogacz, Rafal
spellingShingle	Lloyd, Kevin Becker, Nadine Jones, Matthew W. Bogacz, Rafal Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats
author_facet	Lloyd, Kevin Becker, Nadine Jones, Matthew W. Bogacz, Rafal
author_sort	Lloyd, Kevin
title	Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats
title_short	Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats
title_full	Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats
title_fullStr	Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats
title_full_unstemmed	Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats
title_sort	learning to use working memory: a reinforcement learning gating model of rule acquisition in rats
description	Learning to form appropriate, task-relevant working memory representations is a complex process central to cognition. Gating models frame working memory as a collection of past observations and use reinforcement learning (RL) to solve the problem of when to update these observations. Investigation of how gating models relate to brain and behavior remains, however, at an early stage. The current study sought to explore the ability of simple RL gating models to replicate rule learning behavior in rats. Rats were trained in a maze-based spatial learning task that required animals to make trial-by-trial choices contingent upon their previous experience. Using an abstract version of this task, we tested the ability of two gating algorithms, one based on the Actor-Critic and the other on the State-Action-Reward-State-Action (SARSA) algorithm, to generate behavior consistent with the rats'. Both models produced rule-acquisition behavior consistent with the experimental data, though only the SARSA gating model mirrored faster learning following rule reversal. We also found that both gating models learned multiple strategies in solving the initial task, a property which highlights the multi-agent nature of such models and which is of importance in considering the neural basis of individual differences in behavior.
publisher	Frontiers Media S.A.
publishDate	2012
url	https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3483721/
_version_	1611919770310934528

Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats

Similar Items