Wikia Acquires User-Powered Web Crawler Grub

Next Story

Vayama: Plan Your International Travel

GrubCrawlerWikia, the for-profit venture by Wikipedia founder Jimmy Wales, announced today that it has bought Grub, the user-powered distributed Web Crawler, from Looksmart. Any further details were undisclosed, but Wales did reveal that Grub’s technology will be used as a backend to Search Wikia, their work-in-progress “transparent and open platform for search.”

Ever since the dotbomb, the Grub project has continued to exist although it has never managed to gain any major traction. The idea behind Grub is to have volunteers download the Grub client and run it as a screensaver or invisible process, as it crawls–sorry, “grubs”–through the web. The most popular crawlers get featured on the homepage, and that’s about it. See the major problem? No body cares–except those who do.

The interesting thing about the project is that with the decline of Looksmart (its parent), the project was shutdown for a while, until being restarted again with the help of “Wikia, community members, contibuters, and Open Source developers.” Wikia will now use the technology to power its own mysterious patent-pending, paradigm-shifting, user-generated (in Guy Kawasaki’s terms) search engine.

So far, Wikia has only revealed a few things about the search engine on its own wiki page. It will focus on four main components–transparency, community, quality, and privacy–and will most probably have components of both human-powered and machine-powered search.

SearchEngineLand has a good writeup with the gritty details on the acquisition. The official press release is here.


Cisco 15454-AIC Controlled Cooling Fan1YearWarranty 2+Available picture
Cisco 15454-AIC Controlled Cooling Fan1YearWarranty 2+Available
Cisco 2600 2610 2621 2611 3600 Series Rack Mount Ears Brackets 700-01170-02  picture
Cisco 2600 2610 2621 2611 3600 Series Rack Mount Ears Brackets 700-01170-02
Cisco 15540-MDXC-08B0 Mux/Demux Module1YearWarranty 2+Available picture
Cisco 15540-MDXC-08B0 Mux/Demux Module1YearWarranty 2+Available
NEW Cisco Systems XFP-10G-ELPBK-LW 68-3744-01 GBIC Transceiver picture
NEW Cisco Systems XFP-10G-ELPBK-LW 68-3744-01 GBIC Transceiver
Cisco Valet 300Mbps 802.11n b g Wireless Router With Setup Key SATISFACT.GUAR. picture
Cisco Valet 300Mbps 802.11n b g Wireless Router With Setup Key SATISFACT.GUAR.