Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation
In response to the current issues of poor real-time performance, high computational costs, and excessive memory usage of object detection algorithms based on deep convolutional neural networks in embedded devices, a method for improving deep convolutional neural networks based on model compression a...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Published: |
Multidisciplinary Digital Publishing Institute
2024
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/106292/ |
| _version_ | 1848864737751728128 |
|---|---|
| author | Chen, Qipeng Xiong, Qiaoqiao Huang, Haisong Tang, Saihong Liu, Zhenghong |
| author_facet | Chen, Qipeng Xiong, Qiaoqiao Huang, Haisong Tang, Saihong Liu, Zhenghong |
| author_sort | Chen, Qipeng |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | In response to the current issues of poor real-time performance, high computational costs, and excessive memory usage of object detection algorithms based on deep convolutional neural networks in embedded devices, a method for improving deep convolutional neural networks based on model compression and knowledge distillation is proposed. Firstly, data augmentation is employed in the preprocessing stage to increase the diversity of training samples, thereby improving the model’s robustness and generalization capability. The K-means++ clustering algorithm generates candidate bounding boxes, adapting to defects of different sizes and selecting finer features earlier. Secondly, the cross stage partial (CSP) Darknet53 network and spatial pyramid pooling (SPP) module extract features from the input raw images, enhancing the accuracy of defect location detection and recognition in YOLO. Finally, the concept of model compression is integrated, utilizing scaling factors in the batch normalization (BN) layer, and introducing sparse factors to perform sparse training on the network. Channel pruning and layer pruning are applied to the sparse model, and post-processing methods using knowledge distillation are used to effectively reduce the model size and forward inference time while maintaining model accuracy. The improved model size decreases from 244 M to 4.19 M, the detection speed increases from 32.8 f/s to 68 f/s, and mAP reaches 97.41. Experimental results demonstrate that this method is conducive to deploying network models on embedded devices with limited GPU computing and storage resources. It can be applied in distributed service architectures for edge computing, providing new technological references for deploying deep learning models in the industrial sector. |
| first_indexed | 2025-11-15T13:53:34Z |
| format | Article |
| id | upm-106292 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-15T13:53:34Z |
| publishDate | 2024 |
| publisher | Multidisciplinary Digital Publishing Institute |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-1062922024-05-14T13:11:36Z http://psasir.upm.edu.my/id/eprint/106292/ Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation Chen, Qipeng Xiong, Qiaoqiao Huang, Haisong Tang, Saihong Liu, Zhenghong In response to the current issues of poor real-time performance, high computational costs, and excessive memory usage of object detection algorithms based on deep convolutional neural networks in embedded devices, a method for improving deep convolutional neural networks based on model compression and knowledge distillation is proposed. Firstly, data augmentation is employed in the preprocessing stage to increase the diversity of training samples, thereby improving the model’s robustness and generalization capability. The K-means++ clustering algorithm generates candidate bounding boxes, adapting to defects of different sizes and selecting finer features earlier. Secondly, the cross stage partial (CSP) Darknet53 network and spatial pyramid pooling (SPP) module extract features from the input raw images, enhancing the accuracy of defect location detection and recognition in YOLO. Finally, the concept of model compression is integrated, utilizing scaling factors in the batch normalization (BN) layer, and introducing sparse factors to perform sparse training on the network. Channel pruning and layer pruning are applied to the sparse model, and post-processing methods using knowledge distillation are used to effectively reduce the model size and forward inference time while maintaining model accuracy. The improved model size decreases from 244 M to 4.19 M, the detection speed increases from 32.8 f/s to 68 f/s, and mAP reaches 97.41. Experimental results demonstrate that this method is conducive to deploying network models on embedded devices with limited GPU computing and storage resources. It can be applied in distributed service architectures for edge computing, providing new technological references for deploying deep learning models in the industrial sector. Multidisciplinary Digital Publishing Institute 2024 Article PeerReviewed Chen, Qipeng and Xiong, Qiaoqiao and Huang, Haisong and Tang, Saihong and Liu, Zhenghong (2024) Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation. Electronics (Switzerland), 13 (2). pp. 1-27. ISSN 2079-9292 https://www.mdpi.com/2079-9292/13/2/253 10.3390/electronics13020253 |
| spellingShingle | Chen, Qipeng Xiong, Qiaoqiao Huang, Haisong Tang, Saihong Liu, Zhenghong Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation |
| title | Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation |
| title_full | Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation |
| title_fullStr | Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation |
| title_full_unstemmed | Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation |
| title_short | Research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation |
| title_sort | research on the construction of an efficient and lightweight online detection method for tiny surface defects through model compression and knowledge distillation |
| url | http://psasir.upm.edu.my/id/eprint/106292/ http://psasir.upm.edu.my/id/eprint/106292/ http://psasir.upm.edu.my/id/eprint/106292/ |