Google and all other major search engines collect the information they display in their search results by using software programs called 'spiders' or 'bots'. Google's main program is called Googlebot. A spider is a program wherein the search engines send out to read the Metas or the HTML code of the site submitted for indexing. This software trolls the net, 24/7, looking for new websites as well as changes to websites already in the search engine's database.
When the software evaluates your site, it is called 'crawling'. After your site has been 'crawled', the bot takes the new data back to the search engine, and reports what has changed. These changes are incorporated into the search engine's live database after being analyzed by a complex set of mathematical computations or simply called the web page algorithm. Have a look at your websites URL on Google. After the description, you will see the words "cache storage". You click on that and you get the version of your Website. Google keeps a copy of every web site it has visited in it's database, as a representation of the information it has found on that site. The search engine spider of Google (Googlebot) is the best friend whom a webmaster can have. In order to be indexed in Google quickly (and to other search engines), is all that you must receive a couple of other websites that are already in Google to make a connection to your site. Or the other way out is to submit your site URL to search engines directly by using their online submission form. The analogy to keep the search indexing robots spiders visiting your site again and again lies in providing interesting, fresh and updated content. The information on spiders & robots here is all about the functioning of search engine robots and how these search engine crawlers help in indexing the site.
| 
