Web Crawler for eBook Library

Abstract

A crawler is a program that retrieves and stores pages from the Web, commonly for a Web search engine. A crawler often has to download hundreds of millions of pages in a short period of time and has to constantly monitor and refresh the downloaded pages. A focused crawler is a Web crawler aiming to search and retrieve Web pages from the World Wide Web, which are related to a domain-specific topic. Rather than downloading all accessible Web pages, a focused crawler analyzes the frontier of the crawled region to visit only the portion of the Web that contains relevant Web pages, and at the same time, try to skip irrelevant regions. A web crawler for ebook library is a web crawler which replies to ebook related queries. The crawler for ebook Library must crawl through the ebook specific Web pages in the World Wide Web (WWW). For a crawler it is not an easy task to download the ebook specific Web pages. Focus Crawling Mechanism can play a vital role in this context. In our approach we crawl through the Web and store eBooks in a fileSystem, which are related to a eBook domain using Focus Crawling Mechanism.