A survey in semantic web technologies-inspired focused crawlers

Crawlers are software which can traverse the internet and retrieve webpages by hyperlinks. In theface of the inundant spam websites, traditional web crawlers cannot function well to solve this problem.Semantic focused crawlers utilize semantic web technologies to analyze the semantics of hyperlinksa...

Full description

Bibliographic Details
Main Authors: Dong, Hai, Hussain, Farookh Khadeer, Chang, Elizabeth
Other Authors: Shoniregun, C.A.
Format: Conference Paper
Published: Institute of Electrical and Electronics Engineers (IEEE) 2008
Online Access:http://hdl.handle.net/20.500.11937/7518
Description
Summary:Crawlers are software which can traverse the internet and retrieve webpages by hyperlinks. In theface of the inundant spam websites, traditional web crawlers cannot function well to solve this problem.Semantic focused crawlers utilize semantic web technologies to analyze the semantics of hyperlinksand web documents. This paper briefly reviews the recent studies on one category of semantic focusedcrawlers ? ontology-based focused crawlers, which are a series of crawlers that utilize ontologies to linkthe fetched web documents with the ontological concepts (topics). The purpose of this is to organizeand categorize web documents, or filtering irrelevant webpages with regards to the topics. A briefcomparison are made among these crawlers, from six perspectives - domain, working environment,special functions, technologies utilized, evaluation metrics and evaluation results. The conclusion withrespect to this comparison is made in the final section.