Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction
Predicting an interaction before it is fully executed is very important in applications, such as human-robot interaction and video surveillance. In a two-human interaction scenario, there are often contextual dependency structures between the global interaction context of the two humans and the loca...
| Main Authors: | , , , , |
|---|---|
| Format: | Journal Article |
| Published: |
IEEE
2018
|
| Online Access: | http://hdl.handle.net/20.500.11937/69792 |
| _version_ | 1848762135906091008 |
|---|---|
| author | Ke, Q. Bennamoun, M. An, Senjian Sohel, F. Boussaid, F. |
| author_facet | Ke, Q. Bennamoun, M. An, Senjian Sohel, F. Boussaid, F. |
| author_sort | Ke, Q. |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | Predicting an interaction before it is fully executed is very important in applications, such as human-robot interaction and video surveillance. In a two-human interaction scenario, there are often contextual dependency structures between the global interaction context of the two humans and the local context of the different body parts of each human. In this paper, we propose to learn the structure of the interaction contexts and combine it with the spatial and temporal information of a video sequence to better predict the interaction class. The structural models, including the spatial and the temporal models, are learned with long short term memory (LSTM) networks to capture the dependency of the global and local contexts of each RGB frame and each optical flow image, respectively. LSTM networks are also capable of detecting the key information from global and local interaction contexts. Moreover, to effectively combine the structural models with the spatial and temporal models for interaction prediction, a ranking score fusion method is introduced to automatically compute the optimal weight of each model for score fusion. Experimental results on the BIT-Interaction Dataset and the UT-Interaction Dataset clearly demonstrate the benefits of the proposed method. |
| first_indexed | 2025-11-14T10:42:46Z |
| format | Journal Article |
| id | curtin-20.500.11937-69792 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T10:42:46Z |
| publishDate | 2018 |
| publisher | IEEE |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-697922019-01-24T02:51:17Z Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction Ke, Q. Bennamoun, M. An, Senjian Sohel, F. Boussaid, F. Predicting an interaction before it is fully executed is very important in applications, such as human-robot interaction and video surveillance. In a two-human interaction scenario, there are often contextual dependency structures between the global interaction context of the two humans and the local context of the different body parts of each human. In this paper, we propose to learn the structure of the interaction contexts and combine it with the spatial and temporal information of a video sequence to better predict the interaction class. The structural models, including the spatial and the temporal models, are learned with long short term memory (LSTM) networks to capture the dependency of the global and local contexts of each RGB frame and each optical flow image, respectively. LSTM networks are also capable of detecting the key information from global and local interaction contexts. Moreover, to effectively combine the structural models with the spatial and temporal models for interaction prediction, a ranking score fusion method is introduced to automatically compute the optimal weight of each model for score fusion. Experimental results on the BIT-Interaction Dataset and the UT-Interaction Dataset clearly demonstrate the benefits of the proposed method. 2018 Journal Article http://hdl.handle.net/20.500.11937/69792 10.1109/TMM.2017.2778559 IEEE restricted |
| spellingShingle | Ke, Q. Bennamoun, M. An, Senjian Sohel, F. Boussaid, F. Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction |
| title | Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction |
| title_full | Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction |
| title_fullStr | Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction |
| title_full_unstemmed | Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction |
| title_short | Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction |
| title_sort | leveraging structural context models and ranking score fusion for human interaction prediction |
| url | http://hdl.handle.net/20.500.11937/69792 |