A smart cache content update policy based on deep reinforcement learning
This paper proposes a DRL-based cache content update policy in the cache-enabled network to improve the cache hit ratio and reduce the average latency. In contrast to the existing policies, a more practical cache scenario is considered in this work, in which the content requests vary by both time an...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Hindawi Limited
2020
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/64083/ |
| _version_ | 1848800087985094656 |
|---|---|
| author | Li, Lincan Kwong, Chiew Foong Liu, Qianyu Wang, Jing |
| author_facet | Li, Lincan Kwong, Chiew Foong Liu, Qianyu Wang, Jing |
| author_sort | Li, Lincan |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | This paper proposes a DRL-based cache content update policy in the cache-enabled network to improve the cache hit ratio and reduce the average latency. In contrast to the existing policies, a more practical cache scenario is considered in this work, in which the content requests vary by both time and location. Considering the constraint of the limited cache capacity, the dynamic content update problem is modeled as a Markov decision process (MDP). Besides that, the deep Q-learning network (DQN) algorithm is utilised to solve the MDP problem. Specifically, the neural network is optimised to approximate the Q value where the training data are chosen from the experience replay memory. The DQN agent derives the optimal policy for the cache decision. Compared with the existing policies, the simulation results show that our proposed policy is 56%-64% improved in terms of the cache hit ratio and 56%-59% decreased in terms of the average latency. |
| first_indexed | 2025-11-14T20:46:00Z |
| format | Article |
| id | nottingham-64083 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:46:00Z |
| publishDate | 2020 |
| publisher | Hindawi Limited |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-640832020-12-18T08:45:11Z https://eprints.nottingham.ac.uk/64083/ A smart cache content update policy based on deep reinforcement learning Li, Lincan Kwong, Chiew Foong Liu, Qianyu Wang, Jing This paper proposes a DRL-based cache content update policy in the cache-enabled network to improve the cache hit ratio and reduce the average latency. In contrast to the existing policies, a more practical cache scenario is considered in this work, in which the content requests vary by both time and location. Considering the constraint of the limited cache capacity, the dynamic content update problem is modeled as a Markov decision process (MDP). Besides that, the deep Q-learning network (DQN) algorithm is utilised to solve the MDP problem. Specifically, the neural network is optimised to approximate the Q value where the training data are chosen from the experience replay memory. The DQN agent derives the optimal policy for the cache decision. Compared with the existing policies, the simulation results show that our proposed policy is 56%-64% improved in terms of the cache hit ratio and 56%-59% decreased in terms of the average latency. Hindawi Limited 2020-11-09 Article PeerReviewed application/pdf en cc_by https://eprints.nottingham.ac.uk/64083/1/A%20Smart%20Cache%20Content%20Update%20Policy%20Based%20on%20Deep%20Reinforcement%20Learning.pdf Li, Lincan, Kwong, Chiew Foong, Liu, Qianyu and Wang, Jing (2020) A smart cache content update policy based on deep reinforcement learning. Wireless Communications and Mobile Computing, 2020 . pp. 1-11. ISSN 1530-8669 cache; deep reinforcement learning (DRL); deep Q-learning network (DQN); content update; cache hit ratio; average latency; spatiotemporally varying http://dx.doi.org/10.1155/2020/8836592 doi:10.1155/2020/8836592 doi:10.1155/2020/8836592 |
| spellingShingle | cache; deep reinforcement learning (DRL); deep Q-learning network (DQN); content update; cache hit ratio; average latency; spatiotemporally varying Li, Lincan Kwong, Chiew Foong Liu, Qianyu Wang, Jing A smart cache content update policy based on deep reinforcement learning |
| title | A smart cache content update policy based on deep reinforcement learning |
| title_full | A smart cache content update policy based on deep reinforcement learning |
| title_fullStr | A smart cache content update policy based on deep reinforcement learning |
| title_full_unstemmed | A smart cache content update policy based on deep reinforcement learning |
| title_short | A smart cache content update policy based on deep reinforcement learning |
| title_sort | smart cache content update policy based on deep reinforcement learning |
| topic | cache; deep reinforcement learning (DRL); deep Q-learning network (DQN); content update; cache hit ratio; average latency; spatiotemporally varying |
| url | https://eprints.nottingham.ac.uk/64083/ https://eprints.nottingham.ac.uk/64083/ https://eprints.nottingham.ac.uk/64083/ |