Exploring deception using automatically detected facial action units
Lie detection has always gripped mankind. Today, applications range from individual employee screening to mass terror scenarios. Yet, signs of deception are still not well understood and there is general agreement that humans are bad at detecting them. We suffer from bias and subjectivity as well as...
| Main Author: | |
|---|---|
| Format: | Thesis (University of Nottingham only) |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/63751/ |
| _version_ | 1848800053165031424 |
|---|---|
| author | Vinkemeier, Doratha |
| author_facet | Vinkemeier, Doratha |
| author_sort | Vinkemeier, Doratha |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Lie detection has always gripped mankind. Today, applications range from individual employee screening to mass terror scenarios. Yet, signs of deception are still not well understood and there is general agreement that humans are bad at detecting them. We suffer from bias and subjectivity as well as a lack of stamina and observational acuity. For this reason, hope is now being placed on automatic methods such as action unit (AU) detectors, which detect facial muscle movements that can reveal affective states.
Automatic AU detectors are still in their developmental stage; they are proving, however, to be useful for both detecting and learning about deception. This thesis used
CNN-BLSTM and OpenFace AU detectors and decision trees in two different deception scenarios. In one of them, the game of poker, deception is integral and desirable. Videos obtained from the University of Southern California showed pairs of players who communicated over a network and behaved spontaneously in a laboratory setting. I ascertained that players who were folding, as opposed to calling or raising, displayed significantly more AU12 and AU5, action units associated with smiling and other emotions, whereby CNN-BLSTM and OpenFace showed only limited overlap.
The study of deceit is hindered also by the lack of relevant datasets that simultaneously have a ground truth. For that reason, the second part of my thesis was dedicated to building and researching such a dataset - the dice rolling experiment - where participants roll a virtual die and decide themselves whether or not to lie to increase their earnings. This dataset consists of over 1.7 million frames of good quality video along with concurrent mouse tracking information and timestamps of events covering 373 different subjects. It has a defined ground truth and also investigates the effects of cold water stress on deceptive behaviour. This experiment revealed that males lied more than females and that stress reduced lying. Low detection levels and distinct patterns of false positive facial AUs lead me to use head pose estimators, which showed that under stress, deceptive participants moved their heads significantly more than honest ones.
In summary, this study automatically detected scenario-specific clues of deception, explored the limitations of current AU detectors, and generated a large, novel data set uniquely suitable for studying deception and its automatic detection. |
| first_indexed | 2025-11-14T20:45:26Z |
| format | Thesis (University of Nottingham only) |
| id | nottingham-63751 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:45:26Z |
| publishDate | 2020 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-637512025-02-28T15:06:52Z https://eprints.nottingham.ac.uk/63751/ Exploring deception using automatically detected facial action units Vinkemeier, Doratha Lie detection has always gripped mankind. Today, applications range from individual employee screening to mass terror scenarios. Yet, signs of deception are still not well understood and there is general agreement that humans are bad at detecting them. We suffer from bias and subjectivity as well as a lack of stamina and observational acuity. For this reason, hope is now being placed on automatic methods such as action unit (AU) detectors, which detect facial muscle movements that can reveal affective states. Automatic AU detectors are still in their developmental stage; they are proving, however, to be useful for both detecting and learning about deception. This thesis used CNN-BLSTM and OpenFace AU detectors and decision trees in two different deception scenarios. In one of them, the game of poker, deception is integral and desirable. Videos obtained from the University of Southern California showed pairs of players who communicated over a network and behaved spontaneously in a laboratory setting. I ascertained that players who were folding, as opposed to calling or raising, displayed significantly more AU12 and AU5, action units associated with smiling and other emotions, whereby CNN-BLSTM and OpenFace showed only limited overlap. The study of deceit is hindered also by the lack of relevant datasets that simultaneously have a ground truth. For that reason, the second part of my thesis was dedicated to building and researching such a dataset - the dice rolling experiment - where participants roll a virtual die and decide themselves whether or not to lie to increase their earnings. This dataset consists of over 1.7 million frames of good quality video along with concurrent mouse tracking information and timestamps of events covering 373 different subjects. It has a defined ground truth and also investigates the effects of cold water stress on deceptive behaviour. This experiment revealed that males lied more than females and that stress reduced lying. Low detection levels and distinct patterns of false positive facial AUs lead me to use head pose estimators, which showed that under stress, deceptive participants moved their heads significantly more than honest ones. In summary, this study automatically detected scenario-specific clues of deception, explored the limitations of current AU detectors, and generated a large, novel data set uniquely suitable for studying deception and its automatic detection. 2020-12-11 Thesis (University of Nottingham only) NonPeerReviewed application/pdf en arr https://eprints.nottingham.ac.uk/63751/1/Thesis_Doratha_Vinkemeier_Corrected_Version.pdf Vinkemeier, Doratha (2020) Exploring deception using automatically detected facial action units. PhD thesis, University of Nottingham. Computer vision Affective computing Action units Emotion recognition Deception Machine learning Behavioral economics Poker Stress. |
| spellingShingle | Computer vision Affective computing Action units Emotion recognition Deception Machine learning Behavioral economics Poker Stress. Vinkemeier, Doratha Exploring deception using automatically detected facial action units |
| title | Exploring deception using automatically detected facial action units |
| title_full | Exploring deception using automatically detected facial action units |
| title_fullStr | Exploring deception using automatically detected facial action units |
| title_full_unstemmed | Exploring deception using automatically detected facial action units |
| title_short | Exploring deception using automatically detected facial action units |
| title_sort | exploring deception using automatically detected facial action units |
| topic | Computer vision Affective computing Action units Emotion recognition Deception Machine learning Behavioral economics Poker Stress. |
| url | https://eprints.nottingham.ac.uk/63751/ |