Introduction

A software program that automatically crawls the web to discover and index content for Google Search fetches web pages, analyzes their content, and follows links to find new pages.