About Google Web Crawler
hi guys.......
google downloads the whole of the web thru crawlers.... if i want to download a website to my pc wat software do u think i can use........
do reply.......๐
google downloads the whole of the web thru crawlers.... if i want to download a website to my pc wat software do u think i can use........
do reply.......๐
Replies
-
zia.sepsisplease explain your query clearly. so that people out here can help you.
-
Kaustubh Katdare
I think I got what the question is.bharath muthukuhi guys.......
google downloads the whole of the web thru crawlers.... if i want to download a website to my pc wat software do u think i can use........
do reply.......๐
First of all, Google does not download the whole web. It simply lists all the information on it's servers through its complicated Page Rank algorithm.
What Google does is this :
Google sends it web crawler (web spider) to crawl the website. While crawling, the crawler picks up relevant keywords and information and sends them to Google's servers. Google's servers then analyze the keywords and decide the rank of the page from which they were picked up. As people search for the keywords on Google's search engine, Google lists them on its search results pages according to their ranks.
If you want to do the same; you will have to write your own web crawler. If you know PERL, you may do that. But indexing the whole of web will be too big for your computer ๐ . Google owns server frams that are twice as big as football ground! ๐ -
MaRo- Crawlers don't download the web, they just make indexes for them..like index of a book the index doesn't contain the whole book
- You want to view websites while no connection here you are HTTrack Website Copier - Free Software Offline Browser (GNU GPL)
- You want to make a crawler go learn how to ask first. -
ms_csVery Interesting information about Web Crawler...
You are reading an archived discussion.
Related Posts
Want to do internship abroad..
I want to do internship abroad. Please suggest me how can I get that opportunity.
I have already done some project work on various topics...
Can someone tell me what is the difference between a pen drive and a flash drive? Also is USB drive the same thing?
What exactly is code rendering? Can anyone explain in detail? Thanks
I got this mail in my inbox. If someone can answer -
hiii
i am biswaranjan panda doing b-tech in N I S T berhampur
i want to do summer...
A two-storey building on a highly reactive site is founded on a grid of piles. The soil profile shows a 4m deep reactive clay horizon overlying a deep profile of...