What is a web craweler?

hey can you please explain me
what is web crawler??????

Replies

  • Kaustubh Katdare
    Kaustubh Katdare
    goyal420
    hey can you please explain me
    what is web crawler??????
    In simple words, it's a computer program that search engines use to index all the data from various websites.
  • Mahesh Dahale
    Mahesh Dahale
    a computer program that browses the World Wide Web in a methodical, automated manner
    A Web crawler is one type of bot, or software agent. In general, it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies.​

You are reading an archived discussion.

Related Posts

Life going in which direction friends,,,,, Why fullerene is different from other current topical antioxidants? Please suggest
The Planetary Gearset *When you take apart and look inside an automatic transmission, you find a huge assortment of parts in a fairly small space. Among other things, you see:...
please tell me where i can get java project or some one send a java project through e-mail on Admin Notice: Please do not provide your email ID. All...
Hello, Post server maintenance, we ran into troubles with forum notifications. To those who are subscribed to forum notifications [new response to your post, private message etc.] kindly confirm that...
The threads that had posted when i was login at previous time had been deleted and aren't atleast visible.. Previously Pallavi also complained about the same thing.??