Visual and semantic context modeling for scene-centric image annotation
Automatic image annotation enables efficient indexing and retrieval of the images in the large-scale image collections, where manual image labeling is an expensive and labor intensive task. This paper proposes a novel approach to automatically annotate images by coherent semantic concepts learned fr...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer New York LLC
2015
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/46866/ http://psasir.upm.edu.my/id/eprint/46866/1/Visual%20and%20semantic%20context%20modeling%20for%20scene-centric%20image%20annotation.pdf |
| _version_ | 1848850752618889216 |
|---|---|
| author | Zand, Mohsen Doraisamy, Shyamala Abdul Halin, Alfian Mustaffa, Mas Rina |
| author_facet | Zand, Mohsen Doraisamy, Shyamala Abdul Halin, Alfian Mustaffa, Mas Rina |
| author_sort | Zand, Mohsen |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | Automatic image annotation enables efficient indexing and retrieval of the images in the large-scale image collections, where manual image labeling is an expensive and labor intensive task. This paper proposes a novel approach to automatically annotate images by coherent semantic concepts learned from image contents. It exploits sub-visual distributions from each visually complex semantic class, disambiguates visual descriptors in a visual context space, and assigns image annotations by modeling image semantic context. The sub-visual distributions are discovered through a clustering algorithm, and probabilistically associated with semantic classes using mixture models. The clustering algorithm can handle the inner-category visual diversity of the semantic concepts with the curse of dimensionality of the image descriptors. Hence, mixture models that formulate the sub-visual distributions assign relevant semantic classes to local descriptors. To capture non-ambiguous and visual-consistent local descriptors, the visual context is learned by a probabilistic Latent Semantic Analysis (pLSA) model that ties up images and their visual contents. In order to maximize the annotation consistency for each image, another context model characterizes the contextual relationships between semantic concepts using a concept graph. Therefore, image labels are finally specialized for each image in a scene-centric view, where images are considered as unified entities. In this way, highly consistent annotations are probabilistically assigned to images, which are closely correlated with the visual contents and true semantics of the images. Experimental validation on several datasets shows that this method outperforms state-of-the-art annotation algorithms, while effectively captures consistent labels for each image. |
| first_indexed | 2025-11-15T10:11:17Z |
| format | Article |
| id | upm-46866 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T10:11:17Z |
| publishDate | 2015 |
| publisher | Springer New York LLC |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-468662018-01-30T07:53:58Z http://psasir.upm.edu.my/id/eprint/46866/ Visual and semantic context modeling for scene-centric image annotation Zand, Mohsen Doraisamy, Shyamala Abdul Halin, Alfian Mustaffa, Mas Rina Automatic image annotation enables efficient indexing and retrieval of the images in the large-scale image collections, where manual image labeling is an expensive and labor intensive task. This paper proposes a novel approach to automatically annotate images by coherent semantic concepts learned from image contents. It exploits sub-visual distributions from each visually complex semantic class, disambiguates visual descriptors in a visual context space, and assigns image annotations by modeling image semantic context. The sub-visual distributions are discovered through a clustering algorithm, and probabilistically associated with semantic classes using mixture models. The clustering algorithm can handle the inner-category visual diversity of the semantic concepts with the curse of dimensionality of the image descriptors. Hence, mixture models that formulate the sub-visual distributions assign relevant semantic classes to local descriptors. To capture non-ambiguous and visual-consistent local descriptors, the visual context is learned by a probabilistic Latent Semantic Analysis (pLSA) model that ties up images and their visual contents. In order to maximize the annotation consistency for each image, another context model characterizes the contextual relationships between semantic concepts using a concept graph. Therefore, image labels are finally specialized for each image in a scene-centric view, where images are considered as unified entities. In this way, highly consistent annotations are probabilistically assigned to images, which are closely correlated with the visual contents and true semantics of the images. Experimental validation on several datasets shows that this method outperforms state-of-the-art annotation algorithms, while effectively captures consistent labels for each image. Springer New York LLC 2015 Article PeerReviewed application/pdf en http://psasir.upm.edu.my/id/eprint/46866/1/Visual%20and%20semantic%20context%20modeling%20for%20scene-centric%20image%20annotation.pdf Zand, Mohsen and Doraisamy, Shyamala and Abdul Halin, Alfian and Mustaffa, Mas Rina (2015) Visual and semantic context modeling for scene-centric image annotation. Multimedia Tools and Applications, 76 (6). pp. 8547-8571. ISSN 1380-7501; ESSN: 1573-7721 http://www.springer.com/computer/information+systems+and+applications/journal/11042 10.1007/s11042-016-3500-5 |
| spellingShingle | Zand, Mohsen Doraisamy, Shyamala Abdul Halin, Alfian Mustaffa, Mas Rina Visual and semantic context modeling for scene-centric image annotation |
| title | Visual and semantic context modeling for scene-centric image annotation |
| title_full | Visual and semantic context modeling for scene-centric image annotation |
| title_fullStr | Visual and semantic context modeling for scene-centric image annotation |
| title_full_unstemmed | Visual and semantic context modeling for scene-centric image annotation |
| title_short | Visual and semantic context modeling for scene-centric image annotation |
| title_sort | visual and semantic context modeling for scene-centric image annotation |
| url | http://psasir.upm.edu.my/id/eprint/46866/ http://psasir.upm.edu.my/id/eprint/46866/ http://psasir.upm.edu.my/id/eprint/46866/ http://psasir.upm.edu.my/id/eprint/46866/1/Visual%20and%20semantic%20context%20modeling%20for%20scene-centric%20image%20annotation.pdf |