Gadgetopia

Nutch: I’m not certain whether this is a service they plan to release, or if this is software you install yourself to roll your own search engine.

“Nutch provides a transparent alternative to commercial web search engines. Only open source search results can be fully trusted to be without bias. (Or at least their bias is public.) All existing major search engines have proprietary ranking formulas, and will not explain why a given page ranks as it does. Additionally, some search engines determine which sites to index based on payments, rather than on the merits of the sites themselves. Nutch, on the other hand, has nothing to hide and no motive to bias its results or its crawler in any way other than to try to give each user the best results possible.”

I still think that if you release algorithms, people will take advantage of that fact and spam your engine. They mention in this site that there are “ten or twenty knobs that we can twiddle to adjust the ranking formula.” Perhaps that’s the trick? You just don’t tell people which knobs you’ve turned where.

Business 2.0 has an article about Nutch. (It’s just two pages, so if you don’t dilly-dally, you can beat the two-minute rule.) Nutch has some horsepower behind it:

“Mitch Kapor, who helped found Lotus Development and the Electronic Frontier Foundation and is founder and president of the Open Source Applications Foundation, certainly agrees. He’s thrown his weight behind the project by joining Nutch’s nonprofit board, as has Tim O’Reilly, the CEO of O’Reilly & Associates.”