Improving visual-to-auditory cross-modality information conversions
Sensory substitution devices have been widely used as an assistive tool, mainly for the purpose of rehabilitation for people with disabilities. With the development of electronics and computing devices, the application of visual-to-auditory sensory substitution (VASS) is becoming widespread in senso...
| Main Author: | |
|---|---|
| Format: | Thesis (University of Nottingham only) |
| Language: | English |
| Published: |
2019
|
| Subjects: | |
| Online Access: | https://eprints.nottingham.ac.uk/55721/ |
| _version_ | 1848799203512287232 |
|---|---|
| author | Tan, Shern Shiou |
| author_facet | Tan, Shern Shiou |
| author_sort | Tan, Shern Shiou |
| building | Nottingham Research Data Repository |
| collection | Online Access |
| description | Sensory substitution devices have been widely used as an assistive tool, mainly for the purpose of rehabilitation for people with disabilities. With the development of electronics and computing devices, the application of visual-to-auditory sensory substitution (VASS) is becoming widespread in sensory substitution devices for the visually impaired. These devices convert visual information from images into an auditory form, known as a soundscape, allowing listeners to visualize their surrounding by interpreting the audio representation they hear. Despite its potential benefits, the technology has not been gaining acceptance among the public because of its weaknesses, such as the interpretability of the soundscapes and the quality of the user experience. The aims of this study were to improve cross-modality conversions in areas that include interpretability, information preservation, and the generation of soundscapes that afford a better listening experience. The use of image processing methods for the purpose of visual feature extraction is demonstrated in order to help the user to better interpret the soundscape they hear. By combining audio synthesis with the sounds of musical instruments and mapping colours to these sounds, systems that generate soundscapes that not only contain more information than that produced by traditional devices but also a_ord a more pleasant listening experience are created. Finally, a new evaluation and optimization methods are proposed to allow better visual-to-auditory feature mapping and foster a more up-to-date means of developing such devices. According to the experimental results and user feedback, the performance of VASS systems created using proposed techniques, in general, improves compared to the traditional systems in terms of ease of usage and user utility. It is encouraging that in the future improved devices can be developed following the direction proposed in this research coupled with more up-to-date techniques, such as machine learning. |
| first_indexed | 2025-11-14T20:31:56Z |
| format | Thesis (University of Nottingham only) |
| id | nottingham-55721 |
| institution | University of Nottingham Malaysia Campus |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-14T20:31:56Z |
| publishDate | 2019 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | nottingham-557212025-02-28T12:09:14Z https://eprints.nottingham.ac.uk/55721/ Improving visual-to-auditory cross-modality information conversions Tan, Shern Shiou Sensory substitution devices have been widely used as an assistive tool, mainly for the purpose of rehabilitation for people with disabilities. With the development of electronics and computing devices, the application of visual-to-auditory sensory substitution (VASS) is becoming widespread in sensory substitution devices for the visually impaired. These devices convert visual information from images into an auditory form, known as a soundscape, allowing listeners to visualize their surrounding by interpreting the audio representation they hear. Despite its potential benefits, the technology has not been gaining acceptance among the public because of its weaknesses, such as the interpretability of the soundscapes and the quality of the user experience. The aims of this study were to improve cross-modality conversions in areas that include interpretability, information preservation, and the generation of soundscapes that afford a better listening experience. The use of image processing methods for the purpose of visual feature extraction is demonstrated in order to help the user to better interpret the soundscape they hear. By combining audio synthesis with the sounds of musical instruments and mapping colours to these sounds, systems that generate soundscapes that not only contain more information than that produced by traditional devices but also a_ord a more pleasant listening experience are created. Finally, a new evaluation and optimization methods are proposed to allow better visual-to-auditory feature mapping and foster a more up-to-date means of developing such devices. According to the experimental results and user feedback, the performance of VASS systems created using proposed techniques, in general, improves compared to the traditional systems in terms of ease of usage and user utility. It is encouraging that in the future improved devices can be developed following the direction proposed in this research coupled with more up-to-date techniques, such as machine learning. 2019-02-23 Thesis (University of Nottingham only) NonPeerReviewed application/pdf en arr https://eprints.nottingham.ac.uk/55721/1/thesis_shernshiou_final.pdf Tan, Shern Shiou (2019) Improving visual-to-auditory cross-modality information conversions. PhD thesis, University of Nottingham. image processing computer vision sensory substitution sonification information theory experimental pyschology vision imaging |
| spellingShingle | image processing computer vision sensory substitution sonification information theory experimental pyschology vision imaging Tan, Shern Shiou Improving visual-to-auditory cross-modality information conversions |
| title | Improving visual-to-auditory cross-modality information conversions |
| title_full | Improving visual-to-auditory cross-modality information conversions |
| title_fullStr | Improving visual-to-auditory cross-modality information conversions |
| title_full_unstemmed | Improving visual-to-auditory cross-modality information conversions |
| title_short | Improving visual-to-auditory cross-modality information conversions |
| title_sort | improving visual-to-auditory cross-modality information conversions |
| topic | image processing computer vision sensory substitution sonification information theory experimental pyschology vision imaging |
| url | https://eprints.nottingham.ac.uk/55721/ |