Towards principled learner selection
Long learner evaluation times are no longer exceptional and often there is insufficient time to exhaustively test all candidate options. When deciding which learners to use, practitioners must rely on ad hoc testing and luck to identify the most accurate one. Given the importance of classification i...
| Main Author: | |
|---|---|
| Format: | Thesis (University of Nottingham only) |
| Language: | English |
| Published: |
2013
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/13299/ |
| _version_ | 1848791700301938688 |
|---|---|
| author | Rattray, Andrew |
| author_facet | Rattray, Andrew |
| author_sort | Rattray, Andrew |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Long learner evaluation times are no longer exceptional and often there is insufficient time to exhaustively test all candidate options. When deciding which learners to use, practitioners must rely on ad hoc testing and luck to identify the most accurate one. Given the importance of classification in decision making, this is unsatisfactory. Progress towards a principled approach requires accurate predictions of learner accuracy and evaluation time and this study examines the potential of traditional meta-learning approaches, with their emphasis on indirect explanatory variables, to deliver the required solutions.
Here, 57 different indirect dataset characteristics, including those related to geometrical complexity, are used as explanatory variables, alongside sample-estimates, in building regression models of accuracy and time.
The evidence presented firmly suggests that these indirect variables lack both the required predictive power and the time efficiency required for the development of practically useful models, and points instead towards basing the prediction of learner accuracy solely on sample-based models. The attempt at modelling learner evaluation time reveals some of the difficulties that this tough challenge presents. |
| first_indexed | 2025-11-14T18:32:40Z |
| format | Thesis (University of Nottingham only) |
| id | nottingham-13299 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T18:32:40Z |
| publishDate | 2013 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-132992025-02-28T11:24:21Z https://eprints.nottingham.ac.uk/13299/ Towards principled learner selection Rattray, Andrew Long learner evaluation times are no longer exceptional and often there is insufficient time to exhaustively test all candidate options. When deciding which learners to use, practitioners must rely on ad hoc testing and luck to identify the most accurate one. Given the importance of classification in decision making, this is unsatisfactory. Progress towards a principled approach requires accurate predictions of learner accuracy and evaluation time and this study examines the potential of traditional meta-learning approaches, with their emphasis on indirect explanatory variables, to deliver the required solutions. Here, 57 different indirect dataset characteristics, including those related to geometrical complexity, are used as explanatory variables, alongside sample-estimates, in building regression models of accuracy and time. The evidence presented firmly suggests that these indirect variables lack both the required predictive power and the time efficiency required for the development of practically useful models, and points instead towards basing the prediction of learner accuracy solely on sample-based models. The attempt at modelling learner evaluation time reveals some of the difficulties that this tough challenge presents. 2013-07-15 Thesis (University of Nottingham only) NonPeerReviewed application/pdf en arr https://eprints.nottingham.ac.uk/13299/1/MRES_ARATTRAY_4145685_FINAL_CORRECTED.pdf Rattray, Andrew (2013) Towards principled learner selection. MRes thesis, University of Nottingham. machine learning algorithms |
| spellingShingle | machine learning algorithms Rattray, Andrew Towards principled learner selection |
| title | Towards principled learner selection |
| title_full | Towards principled learner selection |
| title_fullStr | Towards principled learner selection |
| title_full_unstemmed | Towards principled learner selection |
| title_short | Towards principled learner selection |
| title_sort | towards principled learner selection |
| topic | machine learning algorithms |
| url | https://eprints.nottingham.ac.uk/13299/ |