Kernel Temporal Differences for Neural Decoding

We study the feasibility and capability of the kernel temporal difference (KTD)(λ) algorithm for neural decoding. KTD(λ) is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representatio...

Full description

Bibliographic Details
Main Authors:	Bae, Jihye, Sanchez Giraldo, Luis G., Pohlmeyer, Eric A., Francis, Joseph T., Sanchez, Justin C., Príncipe, José C.
Format:	Online
Language:	English
Published:	Hindawi Publishing Corporation 2015
Online Access:	https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4381863/

id	pubmed-4381863
recordtype	oai_dc
spelling	pubmed-43818632015-04-12 Kernel Temporal Differences for Neural Decoding Bae, Jihye Sanchez Giraldo, Luis G. Pohlmeyer, Eric A. Francis, Joseph T. Sanchez, Justin C. Príncipe, José C. Research Article We study the feasibility and capability of the kernel temporal difference (KTD)(λ) algorithm for neural decoding. KTD(λ) is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representations with the temporal difference approach to learning. One of our key observations is that by using strictly positive definite kernels, algorithm's convergence can be guaranteed for policy evaluation. The algorithm's nonlinear functional approximation capabilities are shown in both simulations of policy evaluation and neural decoding problems (policy improvement). KTD can handle high-dimensional neural states containing spatial-temporal information at a reasonable computational complexity allowing real-time applications. When the algorithm seeks a proper mapping between a monkey's neural states and desired positions of a computer cursor or a robot arm, in both open-loop and closed-loop experiments, it can effectively learn the neural state to action mapping. Finally, a visualization of the coadaptation process between the decoder and the subject shows the algorithm's capabilities in reinforcement learning brain machine interfaces. Hindawi Publishing Corporation 2015 2015-03-17 /pmc/articles/PMC4381863/ /pubmed/25866504 http://dx.doi.org/10.1155/2015/481375 Text en Copyright © 2015 Jihye Bae et al. https://creativecommons.org/licenses/by/3.0/ This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
repository_type	Open Access Journal
institution_category	Foreign Institution
institution	US National Center for Biotechnology Information
building	NCBI PubMed
collection	Online Access
language	English
format	Online
author	Bae, Jihye Sanchez Giraldo, Luis G. Pohlmeyer, Eric A. Francis, Joseph T. Sanchez, Justin C. Príncipe, José C.
spellingShingle	Bae, Jihye Sanchez Giraldo, Luis G. Pohlmeyer, Eric A. Francis, Joseph T. Sanchez, Justin C. Príncipe, José C. Kernel Temporal Differences for Neural Decoding
author_facet	Bae, Jihye Sanchez Giraldo, Luis G. Pohlmeyer, Eric A. Francis, Joseph T. Sanchez, Justin C. Príncipe, José C.
author_sort	Bae, Jihye
title	Kernel Temporal Differences for Neural Decoding
title_short	Kernel Temporal Differences for Neural Decoding
title_full	Kernel Temporal Differences for Neural Decoding
title_fullStr	Kernel Temporal Differences for Neural Decoding
title_full_unstemmed	Kernel Temporal Differences for Neural Decoding
title_sort	kernel temporal differences for neural decoding
description	We study the feasibility and capability of the kernel temporal difference (KTD)(λ) algorithm for neural decoding. KTD(λ) is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representations with the temporal difference approach to learning. One of our key observations is that by using strictly positive definite kernels, algorithm's convergence can be guaranteed for policy evaluation. The algorithm's nonlinear functional approximation capabilities are shown in both simulations of policy evaluation and neural decoding problems (policy improvement). KTD can handle high-dimensional neural states containing spatial-temporal information at a reasonable computational complexity allowing real-time applications. When the algorithm seeks a proper mapping between a monkey's neural states and desired positions of a computer cursor or a robot arm, in both open-loop and closed-loop experiments, it can effectively learn the neural state to action mapping. Finally, a visualization of the coadaptation process between the decoder and the subject shows the algorithm's capabilities in reinforcement learning brain machine interfaces.
publisher	Hindawi Publishing Corporation
publishDate	2015
url	https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4381863/
_version_	1613206376572518400

Kernel Temporal Differences for Neural Decoding

Similar Items