Distributed Search Indexer

By Deane Barker on April 3, 2003

Grub’s Distributed Web Crawling Project: LookSmart is using a distributed computing program (a la SETI @Home) to update its index. I don’t know if I would participate, since this is a Microsoft-backed, for-profit venture. I’ll donate spare CPU cycles to non-profit causes, but Bill’s software sucks up too much of my CPU as it is.

“Grub uses the power of distributed computing to build the best search on the Web. It automatically crawls the Web in the background, borrowing your computer’s spare clock cycles, so you won’t even notice it’s there. “