A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory
Computer vision and pattern recognition approaches have been applied to determine unsafe behaviors on construction sites. Such approaches have been reliant on the computation of artificially complex image features that utilize a cumbersome parameter re-adjustment process. The creation of image featu...
| Main Authors: | , , , , , |
|---|---|
| Format: | Journal Article |
| Published: |
Elsevier BV
2018
|
| Online Access: | http://hdl.handle.net/20.500.11937/59296 |
| _version_ | 1848760440589385728 |
|---|---|
| author | Ding, L. Fang, W. Luo, H. Love, Peter Zhong, B. Ouyang, X. |
| author_facet | Ding, L. Fang, W. Luo, H. Love, Peter Zhong, B. Ouyang, X. |
| author_sort | Ding, L. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | Computer vision and pattern recognition approaches have been applied to determine unsafe behaviors on construction sites. Such approaches have been reliant on the computation of artificially complex image features that utilize a cumbersome parameter re-adjustment process. The creation of image features that can recognize unsafe actions, however, poses a significant research challenge on construction sites. This due to the prevailing complexity of spatio-temporal features, lighting, and the array of viewpoints that are required to identify an unsafe action. Considering these challenges, a new hybrid deep learning model that integrates a convolution neural network (CNN) and long short-term memory (LSTM) that automatically recognizes workers' unsafe actions is developed. The proposed hybrid deep learning model is used to: (1) identify unsafe actions; (2) collect motion data and site videos; (3) extract the visual features from videos using a CNN model; and (4) sequence the learning features that are enabled by the use of LSTM models. An experiment is used to test the model's ability to detect unsafe actions. The results reveal that the developed hybrid model (CNN + LSTM) is able to accurately detect safe/unsafe actions conducted by workers on-site. The model's accuracy exceeds the current state-of-the-art descriptor-based methods for detecting points of interest on images. |
| first_indexed | 2025-11-14T10:15:49Z |
| format | Journal Article |
| id | curtin-20.500.11937-59296 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T10:15:49Z |
| publishDate | 2018 |
| publisher | Elsevier BV |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-592962018-03-28T01:50:51Z A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory Ding, L. Fang, W. Luo, H. Love, Peter Zhong, B. Ouyang, X. Computer vision and pattern recognition approaches have been applied to determine unsafe behaviors on construction sites. Such approaches have been reliant on the computation of artificially complex image features that utilize a cumbersome parameter re-adjustment process. The creation of image features that can recognize unsafe actions, however, poses a significant research challenge on construction sites. This due to the prevailing complexity of spatio-temporal features, lighting, and the array of viewpoints that are required to identify an unsafe action. Considering these challenges, a new hybrid deep learning model that integrates a convolution neural network (CNN) and long short-term memory (LSTM) that automatically recognizes workers' unsafe actions is developed. The proposed hybrid deep learning model is used to: (1) identify unsafe actions; (2) collect motion data and site videos; (3) extract the visual features from videos using a CNN model; and (4) sequence the learning features that are enabled by the use of LSTM models. An experiment is used to test the model's ability to detect unsafe actions. The results reveal that the developed hybrid model (CNN + LSTM) is able to accurately detect safe/unsafe actions conducted by workers on-site. The model's accuracy exceeds the current state-of-the-art descriptor-based methods for detecting points of interest on images. 2018 Journal Article http://hdl.handle.net/20.500.11937/59296 10.1016/j.autcon.2017.11.002 Elsevier BV restricted |
| spellingShingle | Ding, L. Fang, W. Luo, H. Love, Peter Zhong, B. Ouyang, X. A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory |
| title | A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory |
| title_full | A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory |
| title_fullStr | A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory |
| title_full_unstemmed | A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory |
| title_short | A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory |
| title_sort | deep hybrid learning model to detect unsafe behavior: integrating convolution neural networks and long short-term memory |
| url | http://hdl.handle.net/20.500.11937/59296 |