Learning Clip Representations for Skeleton-Based 3D Action Recognition
This paper presents a new representation of skeleton sequences for 3D action recognition. Existing methods based on hand-crafted features or recurrent neural networks cannot adequately capture the complex spatial structures and the long-term temporal dynamics of the skeleton sequences, which are ver...
| Main Authors: | , , , , |
|---|---|
| Format: | Journal Article |
| Published: |
IEEE
2018
|
| Online Access: | http://hdl.handle.net/20.500.11937/70054 |
| _version_ | 1848762202901708800 |
|---|---|
| author | Ke, Q. Bennamoun, M. An, Senjian Sohel, F. Boussaid, F. |
| author_facet | Ke, Q. Bennamoun, M. An, Senjian Sohel, F. Boussaid, F. |
| author_sort | Ke, Q. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | This paper presents a new representation of skeleton sequences for 3D action recognition. Existing methods based on hand-crafted features or recurrent neural networks cannot adequately capture the complex spatial structures and the long-term temporal dynamics of the skeleton sequences, which are very important to recognize the actions. In this paper, we propose to transform each channel of the 3D coordinates of a skeleton sequence into a clip. Each frame of the generated clip represents the temporal information of the entire skeleton sequence and one particular spatial relationship between the skeleton joints. The entire clip incorporates multiple frames with different spatial relationships, which provide useful spatial structural information of the human skeleton. We also propose a multitask convolutional neural network (MTCNN) to learn the generated clips for action recognition. The proposed MTCNN processes all the frames of the generated clips in parallel to explore the spatial and temporal information of the skeleton sequences. The proposed method has been extensively tested on six challenging benchmark datasets. Experimental results consistently demonstrate the superiority of the proposed clip representation and the feature learning method for 3D action recognition compared to the existing techniques. |
| first_indexed | 2025-11-14T10:43:49Z |
| format | Journal Article |
| id | curtin-20.500.11937-70054 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T10:43:49Z |
| publishDate | 2018 |
| publisher | IEEE |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-700542019-01-24T02:53:13Z Learning Clip Representations for Skeleton-Based 3D Action Recognition Ke, Q. Bennamoun, M. An, Senjian Sohel, F. Boussaid, F. This paper presents a new representation of skeleton sequences for 3D action recognition. Existing methods based on hand-crafted features or recurrent neural networks cannot adequately capture the complex spatial structures and the long-term temporal dynamics of the skeleton sequences, which are very important to recognize the actions. In this paper, we propose to transform each channel of the 3D coordinates of a skeleton sequence into a clip. Each frame of the generated clip represents the temporal information of the entire skeleton sequence and one particular spatial relationship between the skeleton joints. The entire clip incorporates multiple frames with different spatial relationships, which provide useful spatial structural information of the human skeleton. We also propose a multitask convolutional neural network (MTCNN) to learn the generated clips for action recognition. The proposed MTCNN processes all the frames of the generated clips in parallel to explore the spatial and temporal information of the skeleton sequences. The proposed method has been extensively tested on six challenging benchmark datasets. Experimental results consistently demonstrate the superiority of the proposed clip representation and the feature learning method for 3D action recognition compared to the existing techniques. 2018 Journal Article http://hdl.handle.net/20.500.11937/70054 10.1109/TIP.2018.2812099 IEEE restricted |
| spellingShingle | Ke, Q. Bennamoun, M. An, Senjian Sohel, F. Boussaid, F. Learning Clip Representations for Skeleton-Based 3D Action Recognition |
| title | Learning Clip Representations for Skeleton-Based 3D Action Recognition |
| title_full | Learning Clip Representations for Skeleton-Based 3D Action Recognition |
| title_fullStr | Learning Clip Representations for Skeleton-Based 3D Action Recognition |
| title_full_unstemmed | Learning Clip Representations for Skeleton-Based 3D Action Recognition |
| title_short | Learning Clip Representations for Skeleton-Based 3D Action Recognition |
| title_sort | learning clip representations for skeleton-based 3d action recognition |
| url | http://hdl.handle.net/20.500.11937/70054 |