Soon, this effort was overwhelmed. From the original military logic that encouraged its inception, the Internet enhanced (almost to delirium) the idea of playing without center fixed, without borders, to keep for one of its components succumb. As perhaps no other technology has come in human history, the Internet was developed at a ferocious speed and disconcerting chaos + Maps Very soon, the site became an ocean. How do I know? How virtual guide through the maze? Arose, then the web search engines. Huge databases that tell the browser where to find information of interest, the search engines Devine las of Access doors. Without them, who were drawn to the work of indexing other sites, perhaps virtual space would have been impassable. The search engines rely on a specific program type, called a web crawler (also, web spider or web robot).
A web crawler runs on a virtual space and permanent insomniac. (At this point, while someone reads these lines, thousands of crawlers are active.) Unlike a traditional database, where people draw near specific information relevant to a particular case, the expansion of Web search engines is blind automatically. The reason is clear: to develop a comprehensive index of the web exceed, exceed, and probably will continue to exceed the possibilities of a single person or even a crowd of people. The crawlers are traveling, they encounter a website, stop, sniff it, explore it, drop all the available information and then return to their barracks.