New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification
This study investigates the use discriminative training methods of minimum classification error (MCE) to estimate the parameter of hidden Markov model (HMM). The conventional training of HMM is based on the maximum likelihood estimation (MLE) which aims to model the true probabilistic distribution o...
| Main Authors: | , , |
|---|---|
| Format: | Monograph |
| Language: | English |
| Published: |
Faculty of Biomedical Engineering and Health Science
2009
|
| Subjects: | |
| Online Access: | http://eprints.utm.my/9734/ http://eprints.utm.my/9734/1/78208.pdf |
| _version_ | 1848891925404319744 |
|---|---|
| author | Balakrishnan, Malarvili Ting, Chee Ming Shaikh Salleh, Sheikh Hussain |
| author_facet | Balakrishnan, Malarvili Ting, Chee Ming Shaikh Salleh, Sheikh Hussain |
| author_sort | Balakrishnan, Malarvili |
| building | UTeM Institutional Repository |
| collection | Online Access |
| description | This study investigates the use discriminative training methods of minimum classification error (MCE) to estimate the parameter of hidden Markov model (HMM). The conventional training of HMM is based on the maximum likelihood estimation (MLE) which aims to model the true probabilistic distribution of the data in term of maximizing the likelihood. This requires sufficient training data and correct choice of probabilistic models, which in reality hardly achievable. The insufficient training data and incorrect modeling assumption of HMM often yield an incorrect and unreliable model. Instead of learning the true distribution, the MCE based training targeted to minimizing the probability of error is used to obtain optimal Bayes classification. The central idea of MCE based training is to define a continuous, differentiable loss function to approximate the actual performance error rate. Gradient based optimization methods can be used to minimize this loss. In this study the first order online generalized probabilistic descent is used as optimization methods. The continuous density HMM is used as the classifier structure in the MCE framework. The MCE based training is evaluated on speaker-independent Malay isolated digit recognition. The MCE training achieves the classification accuracy of 96.4% compared to 96.1% of using MLE with small improvement rate of 0.31%. The small vocabulary is unable to reflect the performance comparison of the two methods, the MLE training given sufficient training data is sufficient to provide optimal classification accuracy. Future work will extend the evaluation on difficult classification task such as phoneme classification, to better access the discriminative ability of the both methods. |
| first_indexed | 2025-11-15T21:05:43Z |
| format | Monograph |
| id | utm-9734 |
| institution | Universiti Teknologi Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T21:05:43Z |
| publishDate | 2009 |
| publisher | Faculty of Biomedical Engineering and Health Science |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | utm-97342017-08-15T03:29:39Z http://eprints.utm.my/9734/ New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification Balakrishnan, Malarvili Ting, Chee Ming Shaikh Salleh, Sheikh Hussain TA Engineering (General). Civil engineering (General) This study investigates the use discriminative training methods of minimum classification error (MCE) to estimate the parameter of hidden Markov model (HMM). The conventional training of HMM is based on the maximum likelihood estimation (MLE) which aims to model the true probabilistic distribution of the data in term of maximizing the likelihood. This requires sufficient training data and correct choice of probabilistic models, which in reality hardly achievable. The insufficient training data and incorrect modeling assumption of HMM often yield an incorrect and unreliable model. Instead of learning the true distribution, the MCE based training targeted to minimizing the probability of error is used to obtain optimal Bayes classification. The central idea of MCE based training is to define a continuous, differentiable loss function to approximate the actual performance error rate. Gradient based optimization methods can be used to minimize this loss. In this study the first order online generalized probabilistic descent is used as optimization methods. The continuous density HMM is used as the classifier structure in the MCE framework. The MCE based training is evaluated on speaker-independent Malay isolated digit recognition. The MCE training achieves the classification accuracy of 96.4% compared to 96.1% of using MLE with small improvement rate of 0.31%. The small vocabulary is unable to reflect the performance comparison of the two methods, the MLE training given sufficient training data is sufficient to provide optimal classification accuracy. Future work will extend the evaluation on difficult classification task such as phoneme classification, to better access the discriminative ability of the both methods. Faculty of Biomedical Engineering and Health Science 2009-08-31 Monograph NonPeerReviewed application/pdf en http://eprints.utm.my/9734/1/78208.pdf Balakrishnan, Malarvili and Ting, Chee Ming and Shaikh Salleh, Sheikh Hussain (2009) New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification. Project Report. Faculty of Biomedical Engineering and Health Science, Skudai, Johor. (Unpublished) |
| spellingShingle | TA Engineering (General). Civil engineering (General) Balakrishnan, Malarvili Ting, Chee Ming Shaikh Salleh, Sheikh Hussain New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification |
| title | New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification |
| title_full | New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification |
| title_fullStr | New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification |
| title_full_unstemmed | New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification |
| title_short | New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification |
| title_sort | new learning algorithm based on hidden markov model (hmm) as stochastic modelling for pattern calssification |
| topic | TA Engineering (General). Civil engineering (General) |
| url | http://eprints.utm.my/9734/ http://eprints.utm.my/9734/1/78208.pdf |