A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system
The harvesting job of cutting and collecting fruit bunches in oil palm plantations remains the most labour-intensive job in the oil palm processing cycle. The introduction of an autonomous vehicle to assist workers in the harvesting job promises better productivity. Such a driverless vehicle require...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Penerbit Universiti Kebangsaan Malaysia
2021
|
| Online Access: | http://journalarticle.ukm.my/18965/ http://journalarticle.ukm.my/18965/1/34.pdf |
| _version_ | 1848814713245270016 |
|---|---|
| author | Chua Zheng Siong, Mohd Faisal Ibrahim, Aqilah Baseri Huddin, Mohd Hairi Mohd Zaman, Fazida Hanim Hashim, |
| author_facet | Chua Zheng Siong, Mohd Faisal Ibrahim, Aqilah Baseri Huddin, Mohd Hairi Mohd Zaman, Fazida Hanim Hashim, |
| author_sort | Chua Zheng Siong, |
| building | UKM Institutional Repository |
| collection | Online Access |
| description | The harvesting job of cutting and collecting fruit bunches in oil palm plantations remains the most labour-intensive job in the oil palm processing cycle. The introduction of an autonomous vehicle to assist workers in the harvesting job promises better productivity. Such a driverless vehicle requires a software module known as simultaneous localisation and mapping (SLAM) to guide the vehicle to navigate autonomously. This work proposes a visual SLAM system with a distinctive capability of detecting and localising oil palm loose fresh fruit bunches (FFB) on the ground using intelligent image processing. This vehicle is equipped with a depth camera capable of capturing RGB images and depth images concurrently. Two VGG16-based convolutional neural network (CNN) models are trained using the acquired RGB and depth images dataset of loose FFBs on the ground. The output from the combinatorial FFB detection model is then fed into a visual SLAM system called RTAB-Map. By combining the FFB detection model and the visual SLAM system, the vehicle can plan for autonomous navigation safely, perform bunch pick-up tasks, and avoid collision with fruit bunches on the ground. The experiment results show that the proposed CNN model can detect and localise loose FFBs with significant accuracy in various lighting conditions. |
| first_indexed | 2025-11-15T00:38:27Z |
| format | Article |
| id | oai:generic.eprints.org:18965 |
| institution | Universiti Kebangasaan Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T00:38:27Z |
| publishDate | 2021 |
| publisher | Penerbit Universiti Kebangsaan Malaysia |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | oai:generic.eprints.org:189652022-07-13T07:55:35Z http://journalarticle.ukm.my/18965/ A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system Chua Zheng Siong, Mohd Faisal Ibrahim, Aqilah Baseri Huddin, Mohd Hairi Mohd Zaman, Fazida Hanim Hashim, The harvesting job of cutting and collecting fruit bunches in oil palm plantations remains the most labour-intensive job in the oil palm processing cycle. The introduction of an autonomous vehicle to assist workers in the harvesting job promises better productivity. Such a driverless vehicle requires a software module known as simultaneous localisation and mapping (SLAM) to guide the vehicle to navigate autonomously. This work proposes a visual SLAM system with a distinctive capability of detecting and localising oil palm loose fresh fruit bunches (FFB) on the ground using intelligent image processing. This vehicle is equipped with a depth camera capable of capturing RGB images and depth images concurrently. Two VGG16-based convolutional neural network (CNN) models are trained using the acquired RGB and depth images dataset of loose FFBs on the ground. The output from the combinatorial FFB detection model is then fed into a visual SLAM system called RTAB-Map. By combining the FFB detection model and the visual SLAM system, the vehicle can plan for autonomous navigation safely, perform bunch pick-up tasks, and avoid collision with fruit bunches on the ground. The experiment results show that the proposed CNN model can detect and localise loose FFBs with significant accuracy in various lighting conditions. Penerbit Universiti Kebangsaan Malaysia 2021 Article PeerReviewed application/pdf en http://journalarticle.ukm.my/18965/1/34.pdf Chua Zheng Siong, and Mohd Faisal Ibrahim, and Aqilah Baseri Huddin, and Mohd Hairi Mohd Zaman, and Fazida Hanim Hashim, (2021) A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system. Jurnal Kejuruteraan, 33 (4). pp. 1113-1121. ISSN 0128-0198 https://www.ukm.my/jkukm/volume-334-2021/ |
| spellingShingle | Chua Zheng Siong, Mohd Faisal Ibrahim, Aqilah Baseri Huddin, Mohd Hairi Mohd Zaman, Fazida Hanim Hashim, A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system |
| title | A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system |
| title_full | A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system |
| title_fullStr | A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system |
| title_full_unstemmed | A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system |
| title_short | A combinatorial RGB and depth images CNN-based model for oil palm fruit bunch detection and heatmap localisation for a visual SLAM system |
| title_sort | combinatorial rgb and depth images cnn-based model for oil palm fruit bunch detection and heatmap localisation for a visual slam system |
| url | http://journalarticle.ukm.my/18965/ http://journalarticle.ukm.my/18965/ http://journalarticle.ukm.my/18965/1/34.pdf |