Parallel circuit - a modular neural network architecture
One of the obstacles that hinder the development of Artificial Neural Networks (ANNs) is the heavy computational cost of the training process. In an attempt to address this problem, I proposed a lightweight model named Parallel Circuits (PCs), with an emphasis on modularity. One of the key inspirati...
| Main Author: | |
|---|---|
| Format: | Thesis (University of Nottingham only) |
| Language: | English |
| Published: |
2019
|
| Online Access: | https://eprints.nottingham.ac.uk/56941/ |
| _version_ | 1848799408896868352 |
|---|---|
| author | Phan, Kien Tuong |
| author_facet | Phan, Kien Tuong |
| author_sort | Phan, Kien Tuong |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | One of the obstacles that hinder the development of Artificial Neural Networks (ANNs) is the heavy computational cost of the training process. In an attempt to address this problem, I proposed a lightweight model named Parallel Circuits (PCs), with an emphasis on modularity. One of the key inspirations for the proposed model is the human retina, which consists of various cell types that only respond to particular visual stimuli. Similarly, conventional ANNs with high redundancy are decomposed into semi-independent modules, which is deemed to provide more efficient learning, both in terms of speed and generalizability.
Owing to the benefits of having fewer connections, the PC models were empirically shown to be considerably faster, especially when implemented in larger models. I also pursued the ability of automatic problem decomposition, and discovered that diversifying the learning process in each circuit strongly benefits the generalization of the proposed model. PC was shown to be advantageous in term of sparsity, which is highly correlated to modularity.
DropCircuit, a regularizer that targets circuits, was introduced to further enhance their specialities. Together with PCs, DropCircuit outperformed models with dense connectivity in several experiments. The circuit-level DropCircuit also exhibited better performance compared to conventional DropOut in conjunction with both PC and non-PC configurations, demonstrating the benefits of modularity. The modularity was further enhanced by imposing a set of biologically inspired constraints. Circuits are modelled as either excitatory or inhibitory types with contrastive properties. Modified PC networks were shown to discover sparse and part-based representations, showing further improvement in generalization. |
| first_indexed | 2025-11-14T20:35:12Z |
| format | Thesis (University of Nottingham only) |
| id | nottingham-56941 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:35:12Z |
| publishDate | 2019 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-569412025-02-28T14:34:18Z https://eprints.nottingham.ac.uk/56941/ Parallel circuit - a modular neural network architecture Phan, Kien Tuong One of the obstacles that hinder the development of Artificial Neural Networks (ANNs) is the heavy computational cost of the training process. In an attempt to address this problem, I proposed a lightweight model named Parallel Circuits (PCs), with an emphasis on modularity. One of the key inspirations for the proposed model is the human retina, which consists of various cell types that only respond to particular visual stimuli. Similarly, conventional ANNs with high redundancy are decomposed into semi-independent modules, which is deemed to provide more efficient learning, both in terms of speed and generalizability. Owing to the benefits of having fewer connections, the PC models were empirically shown to be considerably faster, especially when implemented in larger models. I also pursued the ability of automatic problem decomposition, and discovered that diversifying the learning process in each circuit strongly benefits the generalization of the proposed model. PC was shown to be advantageous in term of sparsity, which is highly correlated to modularity. DropCircuit, a regularizer that targets circuits, was introduced to further enhance their specialities. Together with PCs, DropCircuit outperformed models with dense connectivity in several experiments. The circuit-level DropCircuit also exhibited better performance compared to conventional DropOut in conjunction with both PC and non-PC configurations, demonstrating the benefits of modularity. The modularity was further enhanced by imposing a set of biologically inspired constraints. Circuits are modelled as either excitatory or inhibitory types with contrastive properties. Modified PC networks were shown to discover sparse and part-based representations, showing further improvement in generalization. 2019-07-29 Thesis (University of Nottingham only) NonPeerReviewed application/pdf en arr https://eprints.nottingham.ac.uk/56941/1/TuongPK_Thesis_Revised.pdf Phan, Kien Tuong (2019) Parallel circuit - a modular neural network architecture. PhD thesis, University of Nottingham. |
| spellingShingle | Phan, Kien Tuong Parallel circuit - a modular neural network architecture |
| title | Parallel circuit - a modular neural network architecture |
| title_full | Parallel circuit - a modular neural network architecture |
| title_fullStr | Parallel circuit - a modular neural network architecture |
| title_full_unstemmed | Parallel circuit - a modular neural network architecture |
| title_short | Parallel circuit - a modular neural network architecture |
| title_sort | parallel circuit - a modular neural network architecture |
| url | https://eprints.nottingham.ac.uk/56941/ |