CrazyEngineers
CrazyEngineers

  • About Google Web Crawler

    bharath muthuku

    Member

    Updated: Oct 26, 2024
    Views: 1.1K
    hi guys.......
    google downloads the whole of the web thru crawlers.... if i want to download a website to my pc wat software do u think i can use........
    do reply.......😀
    0
    Replies
Howdy guest!
Dear guest, you must be logged-in to participate on CrazyEngineers. We would love to have you as a member of our community. Consider creating an account or login.
Replies

  • zia.sepsis

    Member • Feb 22, 2009

    please explain your query clearly. so that people out here can help you.
    Are you sure? This action cannot be undone.
    Cancel

  • Kaustubh Katdare

    Administrator • Feb 22, 2009

    bharath muthuku
    hi guys.......
    google downloads the whole of the web thru crawlers.... if i want to download a website to my pc wat software do u think i can use........
    do reply.......😀
    I think I got what the question is.

    First of all, Google does not download the whole web. It simply lists all the information on it's servers through its complicated Page Rank algorithm.

    What Google does is this :

    Google sends it web crawler (web spider) to crawl the website. While crawling, the crawler picks up relevant keywords and information and sends them to Google's servers. Google's servers then analyze the keywords and decide the rank of the page from which they were picked up. As people search for the keywords on Google's search engine, Google lists them on its search results pages according to their ranks.

    If you want to do the same; you will have to write your own web crawler. If you know PERL, you may do that. But indexing the whole of web will be too big for your computer 😉 . Google owns server frams that are twice as big as football ground! 😁
    Are you sure? This action cannot be undone.
    Cancel

  • MaRo

    Member • Feb 22, 2009

    - Crawlers don't download the web, they just make indexes for them..like index of a book the index doesn't contain the whole book

    - You want to view websites while no connection here you are <a href="https://www.httrack.com/" target="_blank" rel="nofollow noopener noreferrer">HTTrack Website Copier - Free Software Offline Browser (GNU GPL)</a>

    - You want to make a crawler go learn how to ask first.
    Are you sure? This action cannot be undone.
    Cancel

  • ms_cs

    Member • Feb 23, 2009

    Very Interesting information about Web Crawler...
    Are you sure? This action cannot be undone.
    Cancel
Home Channels Search Login Register