Progressive kernel extreme learning machine for food image analysis via optimal features / Ghalib Ahmed Tahir
Food recognition systems recently garnered much research attention in the relevant field due to their ability to obtain objective measurements for dietary intake. The goal is to improve food diaries by addressing challenges faced by existing methodologies. In addition to the classical challenge o...
| Main Author: | |
|---|---|
| Format: | Thesis |
| Published: |
2022
|
| Subjects: | |
| Online Access: | http://studentsrepo.um.edu.my/14558/ http://studentsrepo.um.edu.my/14558/2/Ghalib_Ahmed.pdf http://studentsrepo.um.edu.my/14558/1/Ghalib.pdf |
| Summary: | Food recognition systems recently garnered much research attention in the relevant field due
to their ability to obtain objective measurements for dietary intake. The goal is to improve
food diaries by addressing challenges faced by existing methodologies. In addition to the
classical challenge of the absence of rigid food structure and intra-class variations, food
diaries employing deep networks trained with pristine samples are susceptible to quality
variations during image acquisition and transmission. Similarly, most deep learning models
and other hybrid frameworks using visual features from the convolutional neural network
(CNN) do not progressively learn new food categories and their ingredients. Finally, many
existing frameworks integrated with dietary assessment apps are non-comprehensive, as
they can not recognize food ingredients or filter non-food images from the users. This
thesis tackled these challenges, aiming to provide food image analysis frameworks that use
computational resources on edge devices (offline accessibility) and cloud servers (online
accessibility). A framework with offline accessibility performs food image analysis on
edge devices by employing efficient neural networks and a novel online data augmentation
strategy random iterative mixup (RIMixUp). RIMixUp generates synthetic images during
fine-tuning to train ensembles models, resilient to various quality distortions in test images.
Then to increase the trust of the involved parties, this thesis proposed a user-centered
explainable artificial intelligence (AI) framework by inferencing and rationalizing the
results according to needs and user profile. The framework with online accessibility
extracts and selects the optimal subset of quality resilient features from CNNs and
subsequently incorporates the parallel type of classification. The first progressive classifier recognizes food categories, and its multilabel extension detects food ingredients. Following
this idea, after extracting quality resilient features from category CNN and ingredient
CNN model by fine-tuning it on synthetic images generated using the novel online data
augmentation method random iterative mixup, the feature selection strategy uses SHAP
scores from gradient explainer to select the reliable features. Then novel progressive
kernel extreme learning machine (PKELM) is exploited to tackle domain variations due
to quality distortions, intra-class variations, etc., by remodeling the network structure
based on activity value with the nodes. PKELM extension for multilabel classification
detects ingredients by employing bipolar step function to process test output and then
selecting the column labels of the resulting matrix with value one. Moreover, during online
learning, PKELM is equipped with a mechanism to label unlabeled instances and detect
noisy samples. Experimental results showed superior performance of the frameworks
on an integrated set of measures over other methodologies on publically available food
datasets and a newly introduced dataset of Malaysian foods.
|
|---|