Google Expands

By Deane Barker on February 18, 2004 – Google eyes a gaggle of sites: They recently doubled their claim of pages crawled, from 3.3 billion to over 6 billion.

The Mountain View, Calif., firm’s “tens of thousands” of computers crawl the Web often to find pages to add. To grow the index, Google couldn’t add more PC power, for fear of crashing certain Web sites; instead, it formed a secondary index. […]

While 6 billion pages sounds “really impressive,” says Chicago-area Web site designer Shari Thurow, “a considerable part of the database is ‘search spam.’ “