Deep learning the dynamic appearance and shape of facial action units
Spontaneous facial expression recognition under uncontrolled conditions is a hard task. It depends on multiple factors including shape, appearance and dynamics of the facial features, all of which are adversely affected by environmental noise and low intensity signals typical of such conditions. In...
| Main Authors: | , |
|---|---|
| Format: | Conference or Workshop Item |
| Published: |
2016
|
| Online Access: | https://eprints.nottingham.ac.uk/31301/ |
| _version_ | 1848794171839610880 |
|---|---|
| author | Jaiswal, Shashank Valstar, Michel F. |
| author_facet | Jaiswal, Shashank Valstar, Michel F. |
| author_sort | Jaiswal, Shashank |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Spontaneous facial expression recognition under uncontrolled conditions is a hard task. It depends on multiple factors including shape, appearance and dynamics of the facial features, all of which are adversely affected by environmental noise and low intensity signals typical of such conditions. In this work, we present a novel approach to Facial Action Unit detection using a combination of Convolutional and Bi-directional Long Short-Term Memory Neural Networks (CNN-BLSTM), which jointly learns shape, appearance and dynamics in a deep learning manner. In addition, we introduce a novel way to encode shape features using binary image masks computed from the locations of facial landmarks. We show that the combination of dynamic CNN features and Bi-directional Long Short-Term Memory excels at modelling the temporal information. We thoroughly evaluate the contributions of each component in our system and show that it achieves state-of-the-art performance on the FERA-2015 Challenge dataset. |
| first_indexed | 2025-11-14T19:11:57Z |
| format | Conference or Workshop Item |
| id | nottingham-31301 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| last_indexed | 2025-11-14T19:11:57Z |
| publishDate | 2016 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-313012020-05-04T20:05:18Z https://eprints.nottingham.ac.uk/31301/ Deep learning the dynamic appearance and shape of facial action units Jaiswal, Shashank Valstar, Michel F. Spontaneous facial expression recognition under uncontrolled conditions is a hard task. It depends on multiple factors including shape, appearance and dynamics of the facial features, all of which are adversely affected by environmental noise and low intensity signals typical of such conditions. In this work, we present a novel approach to Facial Action Unit detection using a combination of Convolutional and Bi-directional Long Short-Term Memory Neural Networks (CNN-BLSTM), which jointly learns shape, appearance and dynamics in a deep learning manner. In addition, we introduce a novel way to encode shape features using binary image masks computed from the locations of facial landmarks. We show that the combination of dynamic CNN features and Bi-directional Long Short-Term Memory excels at modelling the temporal information. We thoroughly evaluate the contributions of each component in our system and show that it achieves state-of-the-art performance on the FERA-2015 Challenge dataset. 2016 Conference or Workshop Item PeerReviewed Jaiswal, Shashank and Valstar, Michel F. (2016) Deep learning the dynamic appearance and shape of facial action units. In: Winter Conference on Applications of Computer Vision (WACV), 7-9 March 2016, Lake Placid, USA. (In Press) |
| spellingShingle | Jaiswal, Shashank Valstar, Michel F. Deep learning the dynamic appearance and shape of facial action units |
| title | Deep learning the dynamic appearance and shape of facial action units |
| title_full | Deep learning the dynamic appearance and shape of facial action units |
| title_fullStr | Deep learning the dynamic appearance and shape of facial action units |
| title_full_unstemmed | Deep learning the dynamic appearance and shape of facial action units |
| title_short | Deep learning the dynamic appearance and shape of facial action units |
| title_sort | deep learning the dynamic appearance and shape of facial action units |
| url | https://eprints.nottingham.ac.uk/31301/ |