What is a web craweler?
hey can you please explain me
what is web crawler??????
what is web crawler??????
0
-
Replies
Howdy guest!
Dear guest, you must be logged-in to participate on CrazyEngineers. We would love to have you as a
member of our community. Consider creating an
account or login.
Replies
-
Administrator • Sep 22, 2009
In simple words, it's a computer program that search engines use to index all the data from various websites.goyal420hey can you please explain me
what is web crawler??????Are you sure? This action cannot be undone. -
Member • Sep 22, 2009
a computer program that browses the World Wide Web in a methodical, automated manner
A Web crawler is one type of bot, or software agent. In general, it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies.Are you sure? This action cannot be undone.