A regularized attribute weighting framework for naive bayes
The Bayesian classification framework has been widely used in many fields, but the covariance matrix is usually difficult to estimate reliably. To alleviate the problem, many naive Bayes (NB) approaches with good performance have been developed. However, the assumption of conditional independence be...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers Inc.
2020
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/64311/ |
| _version_ | 1848800114864291840 |
|---|---|
| author | Wang, Shihe Ren, Jianfeng Bai, Ruibin |
| author_facet | Wang, Shihe Ren, Jianfeng Bai, Ruibin |
| author_sort | Wang, Shihe |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | The Bayesian classification framework has been widely used in many fields, but the covariance matrix is usually difficult to estimate reliably. To alleviate the problem, many naive Bayes (NB) approaches with good performance have been developed. However, the assumption of conditional independence between attributes in NB rarely holds in reality. Various attribute-weighting schemes have been developed to address this problem. Among them, class-specific attribute weighted naive Bayes (CAWNB) has recently achieved good performance by using classification feedback to optimize the attribute weights of each class. However, the derived model may be over-fitted to the training dataset, especially when the dataset is insufficient to train a model with good generalization performance. This paper proposes a regularization technique to improve the generalization capability of CAWNB, which could well balance the trade-off between discrimination power and generalization capability. More specifically, by introducing the regularization term, the proposed method, namely regularized naive Bayes (RNB), could well capture the data characteristics when the dataset is large, and exhibit good generalization performance when the dataset is small. RNB is compared with the state-of-the-art naive Bayes methods. Experiments on 33 machine-learning benchmark datasets demonstrate that RNB outperforms the compared methods significantly. |
| first_indexed | 2025-11-14T20:46:25Z |
| format | Article |
| id | nottingham-64311 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:46:25Z |
| publishDate | 2020 |
| publisher | Institute of Electrical and Electronics Engineers Inc. |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-643112021-01-15T00:58:05Z https://eprints.nottingham.ac.uk/64311/ A regularized attribute weighting framework for naive bayes Wang, Shihe Ren, Jianfeng Bai, Ruibin The Bayesian classification framework has been widely used in many fields, but the covariance matrix is usually difficult to estimate reliably. To alleviate the problem, many naive Bayes (NB) approaches with good performance have been developed. However, the assumption of conditional independence between attributes in NB rarely holds in reality. Various attribute-weighting schemes have been developed to address this problem. Among them, class-specific attribute weighted naive Bayes (CAWNB) has recently achieved good performance by using classification feedback to optimize the attribute weights of each class. However, the derived model may be over-fitted to the training dataset, especially when the dataset is insufficient to train a model with good generalization performance. This paper proposes a regularization technique to improve the generalization capability of CAWNB, which could well balance the trade-off between discrimination power and generalization capability. More specifically, by introducing the regularization term, the proposed method, namely regularized naive Bayes (RNB), could well capture the data characteristics when the dataset is large, and exhibit good generalization performance when the dataset is small. RNB is compared with the state-of-the-art naive Bayes methods. Experiments on 33 machine-learning benchmark datasets demonstrate that RNB outperforms the compared methods significantly. Institute of Electrical and Electronics Engineers Inc. 2020-12-15 Article PeerReviewed application/pdf en cc_by https://eprints.nottingham.ac.uk/64311/1/Wang-2020-A-regularized-attribute-weighting-f.pdf Wang, Shihe, Ren, Jianfeng and Bai, Ruibin (2020) A regularized attribute weighting framework for naive bayes. IEEE Access . p. 1. ISSN 2169-3536 Attribute weighting；classification；naive Bayes； regularization http://dx.doi.org/10.1109/ACCESS.2020.3044946 doi:10.1109/ACCESS.2020.3044946 doi:10.1109/ACCESS.2020.3044946 |
| spellingShingle | Attribute weighting；classification；naive Bayes； regularization Wang, Shihe Ren, Jianfeng Bai, Ruibin A regularized attribute weighting framework for naive bayes |
| title | A regularized attribute weighting framework for naive bayes |
| title_full | A regularized attribute weighting framework for naive bayes |
| title_fullStr | A regularized attribute weighting framework for naive bayes |
| title_full_unstemmed | A regularized attribute weighting framework for naive bayes |
| title_short | A regularized attribute weighting framework for naive bayes |
| title_sort | regularized attribute weighting framework for naive bayes |
| topic | Attribute weighting；classification；naive Bayes； regularization |
| url | https://eprints.nottingham.ac.uk/64311/ https://eprints.nottingham.ac.uk/64311/ https://eprints.nottingham.ac.uk/64311/ |